Method and system for identifying surface-interior collaborative damage of mixed tower structure

Abstract

The invention discloses a mixed tower structure surface-interior collaborative damage identification method and system. The method comprises the following steps: deploying a distributed optical fiber sensing network to collect internal strain and temperature data, and deploying a machine vision system to collect a surface image; respectively preprocessing the two types of data; extracting an internal damage feature vector through a one-dimensional convolutional neural network, and extracting a surface damage feature vector through a two-dimensional convolutional neural network; a six-dimensional collaborative fusion damage feature vector is obtained through a multi-modal feature fusion module; identifying the damage type, position and severity based on the vector, and finally visually outputting a result and triggering early warning, warning and emergency three-level alarm according to a threshold value. The system correspondingly comprises a data acquisition module, a preprocessing module, a feature extraction module, a multi-modal fusion module, a damage identification and positioning module and a result visualization alarm module. According to the invention, deep collaboration of the optical fiber and the machine vision technology is realized, comprehensiveness, accuracy and early warning capability of damage identification of the mixed tower structure are improved, and safety management of the whole life cycle of the mixed tower can be supported.

Description

Technical Field

[0001] This invention relates to the field of structural health monitoring technology, and in particular to a method and system for identifying surface-internal collaborative damage in hybrid tower structures. Background Technology

[0002] Currently, large-capacity onshore wind turbine towers typically employ a steel-concrete hybrid tower design. While hybrid towers combine the advantages of both steel and concrete, their complex material composition and harsh operating environment make timely and accurate structural damage identification technology crucial. Distributed fiber optic sensing and machine vision technologies are representative methods of non-destructive testing. Distributed fiber optic sensing excels at monitoring internal deformation and material degradation within hybrid tower structures, while machine vision technology acquires high-resolution images non-contactly, easily identifying macroscopic surface damage such as cracks, spalling, and corrosion.

[0003] However, each individual technology has its limitations: When relying solely on fiber optic sensing, the data is mostly indirect physical quantity changes, making it difficult to accurately locate the position and morphology of surface damage, and it is not sensitive to minute defects.

[0004] When relying solely on machine vision, it is easily affected by environmental interference such as lighting and occlusion, making it impossible to detect internal damage or stress state, resulting in blind spots in detection.

[0005] More importantly, damage in hybrid towers is often interconnected and co-evolves between the surface and internal components. However, existing methods lack effective multi-source data fusion mechanisms, resulting in isolated internal and external damage information that is difficult to assess collaboratively. This fragmented detection approach is prone to missing crucial information, hindering the reliability and efficiency of full lifecycle safety monitoring for hybrid towers. Summary of the Invention

[0006] To address the technical problems existing in current methods for identifying damage in hybrid tower structures, such as the fragmentation of surface and internal damage identification, limitations of single-modal identification, difficulties in assessing damage across material interfaces, and insufficient deep fusion of multimodal data, the main objective of this invention is to provide a method for identifying surface-internal collaborative damage in hybrid tower structures.

[0007] Another objective of this invention is to propose a surface-internal collaborative damage identification system for hybrid tower structures.

[0008] The third objective of this invention is to provide a computer device.

[0009] To achieve the above objectives, a first aspect of the present invention proposes a method for identifying surface-internal cooperative damage in a hybrid tower structure, comprising: A distributed fiber optic sensor network and a machine vision system are deployed in the hybrid tower structure. The fiber optic sensor network collects internal strain and temperature data in real time, and the machine vision system collects surface visual image data periodically. The fiber optic data is subjected to noise reduction, baseline correction, and temperature compensation to form preprocessed fiber optic data; the visual image data is subjected to distortion correction, illumination normalization, and image registration to form preprocessed visual image data, and multi-source data synchronization is achieved through timestamp alignment. The preprocessed fiber optic data is input into a one-dimensional convolutional neural network to extract internal damage feature vectors, and the preprocessed visual image data is input into a two-dimensional convolutional neural network to extract surface damage feature vectors. The internal damage feature vector and the surface damage feature vector are mapped to the three-dimensional structural model coordinate system through the collaborative association unit, and the feature extraction process is guided by the attention mechanism. A six-dimensional collaborative fusion damage feature vector is generated through the multimodal fusion network. The damage type, location, and severity are determined based on the risk index in the six-dimensional collaborative fusion damage feature vector. The damage spatial coordinates and development trend are output in combination with the three-dimensional model, and a warning, alert, or emergency level three alarm is triggered based on a preset threshold.

[0010] Optionally, the deployment of a distributed fiber optic sensor network and machine vision system in the hybrid tower structure, wherein the fiber optic sensor network collects internal strain and temperature data in real time, and the machine vision system periodically collects surface visual image data, further includes: A first type of distributed optical fiber sensor is laid along a preset path inside or on the surface of the concrete part of the hybrid tower structure. The first type of distributed optical fiber sensor adopts a fiber Bragg grating array. A second type of distributed optical fiber sensor is laid along a preset path on the surface of the steel structure part of the hybrid tower structure. The second type of distributed optical fiber sensor adopts distributed optical fiber sensing technology based on Rayleigh scattering, and the optical fiber is a standard single-mode optical fiber. The first or second type of distributed optical fiber sensor is reinforced in the interface area where different materials connect the hybrid tower structure. A first type of machine vision sensor, which is a high-resolution visible light camera, is installed at a fixed location around the hybrid tower structure or on a flyable drone. A second type of machine vision sensor, which is an infrared thermal imaging camera, is installed at a fixed location around the hybrid tower structure or on a flyable drone. The visible light images of the surface of the hybrid tower structure are periodically captured by the first type of machine vision sensor at a frequency of at least once per hour, wherein the UAV is collecting images while flying at a fixed altitude and speed along a preset route. The second type of machine vision sensor periodically captures infrared thermal images of the surface of the hybrid tower structure at a frequency of at least once a day to assist in detecting temperature anomalies beneath the surface.

[0011] Optionally, the step of performing noise reduction, baseline correction, and temperature compensation processing on the fiber optic data to form preprocessed fiber optic data; performing distortion correction, illumination normalization, and image registration processing on the visual image data to form preprocessed visual image data; and achieving multi-source data synchronization through timestamp alignment further includes: The collected strain and temperature data were subjected to wavelet threshold denoising, with the Daubechies series wavelet selected for the strain of the concrete part and the Coiflets wavelet used for the steel structure part. Baseline drift correction is performed on the denoised data using moving average filtering or polynomial fitting. Temperature compensation is performed on the strain data based on the temperature sensitivity of the fiber optic sensor itself or by using data from an independent temperature sensor. Radial and tangential distortion corrections are performed on the visible light image and infrared thermal image; The image is subjected to illumination normalization processing using adaptive histogram equalization or gamma correction methods. An image registration algorithm based on feature point matching is used to register multiple images into a unified three-dimensional structural model coordinate system; The target monitoring area of ​​the hybrid tower structure is cropped from the registered image.

[0012] Optionally, the step of inputting the preprocessed fiber optic data into a one-dimensional convolutional neural network to extract internal damage feature vectors, and inputting the preprocessed visual image data into a two-dimensional convolutional neural network to extract surface damage feature vectors, further includes: The preprocessed fiber optic data is input into a one-dimensional convolutional neural network containing at least three convolutional layers, a batch normalization layer, and an activation function layer. The abnormal features include local strain, strain gradient anomalies, temperature gradient anomalies, and changes in signal frequency domain energy distribution, forming the internal damage feature vector; The preprocessed visual image data is input into a two-dimensional convolutional neural network employing a semantic segmentation model and an object detection model architecture. The semantic segmentation model is either U-Net or DeepLabV3. The target detection model is either the YOLO series or the Faster R-CNN model; The two-dimensional convolutional neural network outputs information on the type, location, size, and severity of surface damage to the hybrid tower structure. The damage types include cracks, peeling, corrosion, or coating failure, forming the surface damage feature vector.

[0013] Optionally, the step of mapping the internal damage feature vector and the surface damage feature vector to the three-dimensional structural model coordinate system through the collaborative association unit, and guiding the feature extraction process based on the attention mechanism, and generating a six-dimensional collaborative fusion damage feature vector through a multimodal fusion network further includes: Based on the geographic coordinates of the strain concentration area or temperature anomaly area indicated in the internal damage feature vector, potential internal damage areas are marked on the three-dimensional model of the hybrid tower structure. The potential internal damage area is projected onto a two-dimensional plane corresponding to the preprocessed visual image data to determine the key areas of focus for visual feature extraction. Prioritize areas associated with fiber optic anomalies; The internal damage feature vector and the surface damage feature vector are concatenated along the feature dimension to form an initial fused feature vector; The initial fused feature vector is input into a fusion network based on the Transformer architecture or a gated recurrent unit (GRU). The fusion network includes multi-layer self-attention mechanisms and cross-modal attention mechanisms to learn deep correlations and complementary information between the two modalities. The fusion network outputs a six-dimensional collaborative fusion damage feature vector, where the six dimensions represent the risk indices of cracks, spalling, corrosion, internal voids, material fatigue, and overall structural instability, respectively, with the risk indices ranging from zero to one.

[0014] Optionally, the step of determining the damage type, location, and severity based on the risk index in the six-dimensional collaborative fusion damage feature vector, outputting the damage spatial coordinates and development trend in conjunction with the three-dimensional model, and triggering a warning, alert, or emergency level three alarm based on a preset threshold further includes: If any risk index in the collaborative fusion damage feature vector exceeds a preset threshold, it is determined that there is a corresponding type of damage. The damage types include concrete cracks, steel corrosion, internal voids, delamination, or material fatigue. If fiber optic data indicates internal stress concentration and visual data detects microcracks in the corresponding surface area, the system determines that there is potential damage inside the structure that is developing towards the surface, and provides its three-dimensional spatial coordinates and development trend. If visual data detects severe surface cracks and fiber optic data shows abnormal strain in the surrounding area, the system confirms the severity of the damage and its impact on the overall load-bearing capacity of the structure, and provides precise three-dimensional spatial positioning and geometric dimensions.

[0015] Optional, also includes: A structural health report is generated, which includes historical damage data, damage development trend prediction, and maintenance recommendations. The development trend prediction is based on time series analysis of the six-dimensional collaborative fusion damage feature vector, and the maintenance recommendations are matched with a preset maintenance strategy library according to the damage type and severity.

[0016] To achieve the above objectives, a second aspect of the present invention provides a surface-internal collaborative damage identification system for hybrid tower structures, comprising: The sensor deployment module is used to deploy a distributed fiber optic sensor network and a machine vision system in the hybrid tower structure. The fiber optic sensor network collects internal strain and temperature data in real time, and the machine vision system collects surface visual image data periodically. The data preprocessing and synchronization module is used to perform noise reduction, baseline correction and temperature compensation on the fiber optic data to form preprocessed fiber optic data; to perform distortion correction, illumination normalization and image registration on the visual image data to form preprocessed visual image data; and to achieve multi-source data synchronization through timestamp alignment. The feature extraction module is used to input the preprocessed optical fiber data into a one-dimensional convolutional neural network to extract internal damage feature vectors, and to input the preprocessed visual image data into a two-dimensional convolutional neural network to extract surface damage feature vectors. The collaborative association and multimodal fusion module is used to map the internal damage feature vector and the surface damage feature vector to the three-dimensional structural model coordinate system through the collaborative association unit, and guide the feature extraction process based on the attention mechanism, and generate a six-dimensional collaborative fusion damage feature vector through the multimodal fusion network. The risk assessment and alarm module is used to determine the damage type, location and severity based on the risk index in the six-dimensional collaborative fusion damage feature vector, output the damage spatial coordinates and development trend in combination with the three-dimensional model, and trigger a warning, alert or emergency three-level alarm based on a preset threshold.

[0017] To achieve the above objectives, a third aspect of this application provides a computer device, including a processor and a memory; wherein the processor runs a program corresponding to the executable program code by reading executable program code stored in the memory, for implementing the method described in the first aspect embodiment.

[0018] The embodiments of the present invention have the following beneficial effects: First, this invention achieves deep integration and collaborative operation of distributed fiber optic sensing and machine vision technologies. Distributed fiber optic sensors comprehensively monitor strain and temperature field changes within and around the hybrid tower structure, effectively compensating for the blind spots of machine vision in internal damage detection. Simultaneously, the machine vision system provides high-resolution surface images, accurately identifying and quantifying macroscopic surface damage, compensating for the insufficient spatial resolution of fiber optic sensors in identifying localized, minute surface damage. This collaborative mechanism ensures comprehensive coverage and identification of surface and internal damage in the hybrid tower structure, overcoming the limitations of single-sensor modes.

[0019] Second, this invention introduces a deep learning-driven multimodal feature extraction and collaborative fusion model. High-dimensional features are extracted from fiber optic time-series data and visual image data using one-dimensional and two-dimensional convolutional neural networks, respectively, effectively capturing damage-sensitive features of each modality. The collaborative association unit achieves precise mapping and guidance between fiber optic anomaly regions and visually focused regions of interest, while the damage state fusion and discrimination unit performs deep interaction and complementary verification of heterogeneous feature vectors through a fusion network based on Transformer or gated recurrent units. This deep fusion mechanism significantly improves the accuracy, robustness, and early warning capability for concealed damage in damage identification, greatly reducing false alarms and false negatives.

[0020] Third, this invention specifically optimizes the weak points at the interface regions connecting different materials in hybrid tower structures. The deployment of fiber optic sensors is strengthened in the material interface regions, and a fusion model is designed to collaboratively analyze the response characteristics of different materials. When fiber optic data indicates internal stress concentration while visual data detects microcracks in the corresponding surface region, the system can determine that there is potential damage within the structure that is progressing towards the surface. This is particularly suitable for detecting complex damage behaviors such as delamination and cracking across material interfaces, providing a more accurate and comprehensive structural health assessment.

[0021] Fourth, this invention provides precise damage identification, localization, and quantification capabilities. By fusion discriminant network outputting a multidimensional risk index, the system can identify various damage types such as concrete cracks, steel corrosion, internal voids, delamination, and material fatigue, and provide precise coordinates and quantification parameters of the damage in three-dimensional space (such as crack width and spalling area), providing a specific and reliable basis for subsequent maintenance and repair.

[0022] Fifth, this invention achieves automation and intelligence in damage monitoring. Through real-time data acquisition, automatic preprocessing, intelligent feature extraction, deep fusion discrimination, and result visualization with a multi-level alarm mechanism, it significantly reduces the cost and risk of manual inspections and improves monitoring efficiency. Furthermore, the system can generate structural health reports, including damage development trend predictions, providing strong technical support for the full lifecycle management of hybrid tower structures. Attached Figure Description

[0023] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 A flowchart illustrating a method for identifying surface-internal collaborative damage in a hybrid tower structure, as provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the generation of damage defects in the mixed tower in this invention; Figure 3 This is a diagram of the multi-sensor monitoring network for the mixed tower in this invention; Figure 4 This is a structural diagram of a hybrid tower structure surface-internal collaborative damage identification system provided in an embodiment of the present invention. Detailed Implementation

[0024] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0025] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0026] The following describes, with reference to the accompanying drawings, a method and system for identifying surface-internal collaborative damage in a hybrid tower structure according to an embodiment of the present invention.

[0027] Example 1 This embodiment provides a method for collaborative surface-internal damage identification in hybrid tower structures. This method aims to overcome the limitations of single-sensor modalities in detecting complex damage in hybrid tower structures. For example, distributed fiber optic sensing technology is sensitive to internal strain and temperature changes, but struggles to directly reflect the microscopic morphology of surface damage; while machine vision technology accurately identifies macroscopic defects such as surface cracks and corrosion, it cannot detect internal structural defects. By deeply integrating the advantages of both modalities, this method achieves collaborative perception, accurate identification, and precise location of surface and internal damage in hybrid tower structures, improving the comprehensiveness and accuracy of structural health monitoring. This method is particularly suitable for long-term health monitoring and early damage warning of complex hybrid tower structures such as wind turbine towers, bridge towers, and high-rise building support structures.

[0028] The core technical idea of ​​this method is as follows: First, a high-precision distributed fiber optic sensing system continuously collects strain and temperature data inside the hybrid tower structure, while a high-resolution machine vision system periodically captures surface image data, forming a multi-source heterogeneous time-series sensing data stream. Second, customized data preprocessing and feature extraction are performed according to the characteristics of the two modalities, transforming the raw data into high-dimensional, physically meaningful feature vectors. Furthermore, a cross-modal feature collaborative association mechanism is constructed. By establishing a spatial geometric mapping relationship between the fiber optic sensing area and the machine vision observation area, as well as a temporal behavioral synchronization relationship, deep coupling of the two types of information is achieved at the feature level. Finally, the collaboratively associated multimodal fused features are input into a deep learning damage recognition model, outputting the damage type, location, and severity, and generating a visual report and real-time alarm information. This method overcomes the problems of poor timeliness, limited coverage, and high false alarm / false negative rates of traditional detection methods, providing an efficient, intelligent, and comprehensive solution for the full life-cycle health management of hybrid tower structures.

[0029] like Figure 1 As shown, the method includes the following steps: Step S1: Deploy a distributed fiber optic sensor network and a machine vision system in the hybrid tower structure. The fiber optic sensor network collects internal strain and temperature data in real time, and the machine vision system periodically collects surface visual image data.

[0030] First, in one embodiment of the present invention, a sensor arrangement optimization algorithm based on the strain transfer efficiency factor is proposed to achieve optimal sensor layout:

[0031] in: J is the overall performance index of the sensor network (objective function value), which is dimensionless; The weighting coefficient for the i-th sensor is determined based on the importance of key regions in the structural finite element analysis. is the strain sensitivity coefficient of the i-th sensor to a specific damage; The strain value measured by sensor i; For the j-th type of damage parameter; The strain response correlation coefficient between sensor i and sensor j; cov represents the covariance, and σ represents the standard deviation. n is the total number of candidate sensors; The subscripts i and j are the sensor indices, i, j = 1, 2, ..., n.

[0032] Optimized layout parameters for distributed fiber optic sensors:

[0033] In one embodiment of the present invention, a first type of distributed optical fiber sensor is laid inside or on the surface of the concrete part of the mixed tower structure along a preset path. The first type of distributed optical fiber sensor adopts a fiber Bragg grating array, and the center wavelength of each fiber Bragg grating in the array is 1550 nanometers and the bandwidth is 0.2 nanometers. A second type of distributed optical fiber sensor is laid along a preset path on the surface of the steel structure of the hybrid tower structure. The second type of distributed optical fiber sensor adopts distributed optical fiber sensing technology based on Rayleigh scattering. The optical fiber is a standard single-mode optical fiber with a spatial resolution of 0.5 meters. In the interface area between different materials in the hybrid tower structure, the first type of distributed optical fiber sensor or the second type of distributed optical fiber sensor is reinforced and laid. A first-class machine vision sensor is installed at a fixed position around the hybrid tower structure or on a flyable drone. The first-class machine vision sensor is a high-resolution visible light camera with a resolution of 4,000 by 3,000 pixels and a focal length range of 10 to 100 millimeters. A second type of machine vision sensor is installed at a fixed location around the hybrid tower structure or on a flyable drone. The second type of machine vision sensor is an infrared thermal imaging camera with a resolution of 640 x 512 pixels and a thermal sensitivity of less than 50 millikrvin. The first and second type of distributed optical fiber sensors are interrogated in real time using a distributed optical fiber demodulator at a sampling frequency of 100 Hz to obtain the strain and temperature values ​​of each sensing point or sensing segment. Visible light images of the surface of the hybrid tower structure are periodically captured by a first-class machine vision sensor at a frequency of at least once per hour, wherein the drone collects the images while flying at a constant altitude and speed along a preset flight path. Infrared thermal images of the surface of the mixed tower structure are periodically captured by a second type of machine vision sensor at a frequency of at least once a day to help detect temperature anomalies beneath the surface.

[0034] Step S2: Denoising, baseline correction, and temperature compensation are performed on the fiber optic data to form preprocessed fiber optic data; distortion correction, illumination normalization, and image registration are performed on the visual image data to form preprocessed visual image data, and multi-source data synchronization is achieved through timestamp alignment.

[0035] In this embodiment of the invention, S2 is achieved through the following steps: Wavelet threshold denoising was performed on the collected strain and temperature data to filter out high-frequency random noise; Daubechies wavelet series was selected for the strain of concrete, and Coiflets wavelet was used for the steel structure; for mixed tower monitoring signals with a sampling frequency of 100Hz, decomposition to the 3rd or 4th layer can usually effectively separate the noise. The baseline drift is corrected by using moving average filtering or polynomial fitting to eliminate long-term drift in the denoised data; Based on the temperature sensitivity of the fiber optic sensor itself or by using data from an independent temperature sensor, temperature compensation is performed on the strain data to eliminate the influence of temperature changes on strain measurement. The preprocessed fiber optic data and visual image data are timestamped to achieve data synchronization.

[0036] Visual data preprocessing of surface visual image data specifically includes: Radial and tangential distortion corrections were performed on the acquired visible light and infrared thermal images to eliminate optical distortion. Adaptive histogram equalization or gamma correction methods are used to normalize the illumination of the image to reduce the impact of uneven illumination. Image registration algorithms based on feature point matching, such as scale-invariant feature transformation or accelerated robust feature algorithms, are used to register multiple images into a unified three-dimensional structural model coordinate system to eliminate viewpoint differences and construct a complete structural surface image. The target monitoring area of ​​the hybrid tower structure is cropped from the registered image.

[0037] Step S3: Input the preprocessed optical fiber data into a one-dimensional convolutional neural network to extract internal damage feature vectors, and input the preprocessed visual image data into a two-dimensional convolutional neural network to extract surface damage feature vectors.

[0038] First, the preprocessed fiber optic data is input into a one-dimensional convolutional neural network (CNN), which contains at least three convolutional layers, a batch normalization layer, and an activation function layer, to extract time-domain and frequency-domain anomalous features of strain and temperature from the time-series data. These anomalous features include abrupt changes in local strain, strain gradient anomalies, temperature gradient anomalies, and changes in signal frequency-domain energy distribution, forming an internal damage feature vector. Then, the preprocessed visual image data is input into a two-dimensional convolutional neural network. The two-dimensional convolutional neural network adopts a semantic segmentation model and an object detection model architecture. The semantic segmentation model is U-Net or DeepLabV3+, and the object detection model is YOLO series or Faster R-CNN. Finally, the two-dimensional convolutional neural network outputs information on the type, location, size, and severity of surface damage in the hybrid tower structure. Damage types include cracks, spalling, corrosion, or coating failure, forming a surface damage feature vector.

[0039] Step S4: The internal damage feature vector and the surface damage feature vector are mapped to the three-dimensional structural model coordinate system through the collaborative association unit, and the feature extraction process is guided by the attention mechanism. A six-dimensional collaborative fusion damage feature vector is generated through the multimodal fusion network.

[0040] In one embodiment of the present invention, the mapping and association of two features are realized through a collaborative association unit, specifically including: marking potential internal damage areas on a three-dimensional model of the hybrid tower structure according to the geographical coordinate information of strain concentration areas or temperature anomaly areas indicated in the internal damage feature vector; projecting the potential internal damage areas onto a two-dimensional plane corresponding to the preprocessed visual image data to determine the key areas of focus for visual feature extraction; in the visual damage feature extraction unit, the key areas of focus are used as input to the attention mechanism to guide the visual feature extraction process to prioritize areas related to fiber optic anomalies.

[0041] In one embodiment of the present invention, the multimodal feature fusion module performs deep fusion through the damage state fusion discrimination unit, specifically including: concatenating the internal damage feature vector and the surface damage feature vector in the feature dimension to form an initial fused feature vector; inputting the initial fused feature vector into a fusion network based on the Transformer architecture or a gated recurrent unit (GRU), wherein the fusion network includes a multi-layer self-attention mechanism and a cross-modal attention mechanism to learn the deep correlation and complementary information between the two modalities.

[0042] It should be noted that the output dimension of the fusion network is a six-dimensional collaborative fusion damage feature vector. The six dimensions represent the risk index of cracks, spalling, corrosion, internal voids, material fatigue and overall structural instability, respectively. The risk index ranges from zero to one.

[0043] Step S5: Determine the damage type, location, and severity based on the risk index in the six-dimensional collaborative fusion damage feature vector, output the damage spatial coordinates and development trend in combination with the three-dimensional model, and trigger a warning, alert, or emergency level three alarm based on a preset threshold.

[0044] In one embodiment of the present invention, based on the risk index in the collaborative fusion damage feature vector, if any risk index exceeds a preset threshold (e.g., 0.7), it is determined that there is a corresponding type of damage; wherein, the damage type includes concrete cracks, steel corrosion, internal voids, delamination, or material fatigue.

[0045] In one embodiment of the present invention, if fiber optic data indicates internal stress concentration and visual data detects microcracks in the corresponding surface area, the system determines that there is potential damage inside the structure that is developing toward the surface, and provides its three-dimensional spatial coordinates and development trend. If visual data detects severe surface cracks and fiber optic data shows abnormal strain in the surrounding area, the system confirms the severity of the damage and its impact on the overall load-bearing capacity of the structure, and provides precise three-dimensional spatial positioning and geometric dimensions.

[0046] It should be noted that the location of the damage is represented by three-dimensional coordinates, and the severity of the damage is represented by damage level (e.g., minor, moderate, severe) and quantitative indicators (e.g., crack width, percentage of spalling area, and abnormal strain amplitude).

[0047] Step S6: Generate a structural health report. The report includes historical damage data, damage development trend prediction, and maintenance recommendations. The development trend prediction is based on time series analysis of the six-dimensional collaborative fusion damage feature vector. The maintenance recommendations are matched with a preset maintenance strategy library according to the damage type and severity.

[0048] First, the identified damage types, locations, and severity information are visualized on a 3D model of the hybrid tower structure to visually display the damage distribution. Next, a structural health report is generated, which includes historical damage data, damage development trend predictions, and maintenance recommendations. Finally, based on preset damage severity thresholds and risk index thresholds, a multi-level alarm mechanism is triggered, including SMS notifications, email notifications, and audible and visual alarms. The alarm levels are divided into three levels: early warning, alert, and emergency.

[0049] Furthermore, the schematic diagram of the generation of damage defects in the mixed tower and the multi-sensor monitoring network diagram of the mixed tower in the embodiments of the present invention are respectively as follows: Figure 2 and Figure 3 As shown in the figure: 1-mixing tower segment, 2-machine vision system, 3-surface damage and cracks of the mixing tower, 4-armored optical cable, 5-fiber grating sensor.

[0050] Example 2 This invention relates to a method for identifying surface-internal collaborative damage in a hybrid tower structure, the method comprising the following steps: S101: Acquire multi-source state perception data of the hybrid tower structure.

[0051] This step is fundamental to the entire damage identification method, aiming to synchronously and accurately acquire raw sensor data reflecting the internal and surface conditions of the hybrid tower structure. Due to the heterogeneity of materials and the complexity of stress in hybrid tower structures, a single sensing technology cannot provide comprehensive health information. Distributed fiber optic sensing technology, with its advantages of long distance, continuity, and resistance to electromagnetic interference, can accurately capture subtle strain and temperature changes within the structure. These changes are often early indicators of damage such as internal debonding, delamination, and concrete cracking. Machine vision technology, with its non-contact and high-resolution characteristics, can efficiently identify visible defects such as cracks, spalling, and corrosion on the structural surface. Therefore, this step, through the coordinated deployment of these two sensing systems, aims to comprehensively cover the internal and external monitoring needs of hybrid tower structures.

[0052] In this embodiment of the invention, S101 further includes the following step: S1011: Acquire distributed fiber optic sensing data.

[0053] Specifically, this sub-step utilizes a distributed fiber optic sensor array deployed inside the hybrid tower structure or embedded in the structural materials to collect strain and temperature data at various points on the structure in real time. The deployment method of the distributed fiber optic sensors is meticulously designed according to the specific form of the hybrid tower structure and the key monitoring areas. For example, they are laid inside the concrete tower cylinder along the direction of the steel mesh or prestressing tendons to monitor the strain distribution and crack initiation of the concrete; or they are attached to the surface of steel structure connections or key stress areas to monitor local stress concentration and fatigue damage.

[0054] Specifically, the parameter acquisition method, verification mechanism, and typical value range in this step are explained as follows: Parameter acquisition method: A Brillouin optical time-domain reflectometer (BDR) system was used as the data acquisition device. This system achieves distributed measurement of strain and temperature along the fiber by injecting laser pulses into the fiber and analyzing the frequency shift and intensity changes of the Brillouin scattered light reflected back along the fiber. The Brillouin frequency shift is linearly related to strain and temperature. The system first acquires raw, unprocessed Brillouin scattering spectrum data.

[0055] Verification Mechanism: To ensure data accuracy and reliability, the distributed fiber optic sensors must be factory calibrated before data acquisition, and zero-point drift correction and system calibration must be performed periodically in the field. The calibration process includes measuring the Brillouin frequency shift under known temperature and strain conditions to establish calibration coefficients. During actual data acquisition, by setting up redundant sensing segments or adopting a cross-validation strategy, consistency verification is performed on data from adjacent measurement points to promptly identify and mark abnormal data points.

[0056] Typical value range: Under normal operating conditions of the hybrid tower structure, strain data typically varies in the micro-strain range, such as -200 microstrains to +200 microstrains; under extreme loads or damage, local strain may reach hundreds or even thousands of microstrains. Temperature data varies with environmental and operating conditions, for example, from -30 degrees Celsius to 60 degrees Celsius. The original Brillouin spectrum data contains information on the light intensity distribution within a specific frequency range.

[0057] In one embodiment of the present invention, S1011 further includes the following step: Sub-step S10111: Fiber optic sensor array initialization and health check. Before data acquisition, the system automatically performs connectivity and integrity checks on all distributed fiber optic sensing links to ensure that the optical signal transmission is not excessively attenuated or interrupted. By sending test optical pulses and analyzing return loss, it determines whether the fiber is broken or poorly connected.

[0058] Sub-step S10112: Brillouin Optical Time Domain Reflectometer (OTDR) parameter configuration. Based on monitoring requirements, configure parameters such as laser pulse width, repetition frequency, and spatial resolution of the Brillouin OTD. For example, to improve damage localization accuracy, set the spatial resolution to a value less than one meter.

[0059] Sub-step S10113: Distributed strain and temperature data stream acquisition. Strain and temperature distribution data along the optical fiber are continuously acquired at a preset sampling frequency, for example, once every minute. The data is stored in time-series format, with each data point containing a location identifier, strain value, and temperature value.

[0060] In this embodiment of the invention, the raw fiber optic sensing data is stored in the form of a three-dimensional tensor, with the dimensions being: time, spatial location, and measured parameters such as strain and temperature. Each timestamp corresponds to a strain and temperature profile along the fiber.

[0061] Furthermore, in terms of interaction and anomaly handling, the data acquisition module and the Brillouin optical time domain reflectometer are connected via an optoelectronic interface and a data communication protocol. If a fiber optic link break is detected or the data acquisition equipment malfunctions, the system will immediately trigger a low-level alarm, record the fault event, and simultaneously attempt to switch to a backup sensor link or recording mode.

[0062] S1012: Acquire machine vision image data.

[0063] This sub-step utilizes a high-resolution machine vision camera to periodically acquire images of the surface of the mixed-structure. The camera is typically mounted on a fixed bracket, and the shooting angle and focal length are adjusted via a remote-controlled gimbal, or it can be mounted on an inspection robot or drone to achieve flexible coverage of a large area of ​​the structural surface.

[0064] Specifically, the parameter acquisition method, verification mechanism, and typical value range in this step are explained as follows: Parameter acquisition method: Visible light industrial cameras or thermal infrared cameras are used, selected according to monitoring requirements. The image acquisition device is connected to the image processing unit via an image acquisition card. The raw data is high-resolution digital image files, such as one to five megapixels, or higher.

[0065] Verification mechanism: The camera is periodically calibrated geometrically and radiometrically to correct image distortion and ensure that image brightness matches actual lighting conditions. During acquisition, image sharpness and contrast are monitored in real time using an image quality assessment algorithm; if the image quality is substandard, a retake is triggered.

[0066] Typical value range: Image resolution is typically between 1,920 x 1,080 pixels and 4,096 x 2,160 pixels. Shooting frequency is set based on the rate of structural change and storage conditions, for example, capturing images of key areas every hour.

[0067] In one embodiment of the present invention, S1012 further includes the following step: Sub-step S10121: Deployment and calibration of the machine vision system. Install fixed or mobile cameras on the key monitoring areas of the mixed tower structure surface and complete the calibration of the camera's internal and external parameters. The calibration process uses a standard calibration board, and calculates parameters such as camera focal length, principal point coordinates, and distortion coefficient by taking images of the calibration board from different angles.

[0068] Sub-step S10122: Image acquisition triggering and shutter control. The image acquisition system precisely controls the camera's shutter and flash to capture images based on a preset schedule or external trigger signal. For example, it automatically turns on the fill light when there is insufficient light.

[0069] Sub-step S10123: High-resolution image sequence acquisition. The acquired image data is stored in a specific format, such as JPEG or PNG, and is accompanied by metadata such as timestamps, camera positions, and shooting angles, forming a continuous image sequence.

[0070] In this embodiment of the invention, the original image data is stored in the form of a two-dimensional array or a three-dimensional tensor, representing the pixel intensity values ​​of the image. Metadata is stored in the form of structured text and associated with the image file.

[0071] Furthermore, in interaction and anomaly handling, the machine vision module and image processing unit communicate via high-speed data transmission interfaces such as Gigabit Ethernet or USB 3.0. If the image acquisition device experiences anomalies such as going offline, failing to focus, or insufficient storage space, the system will log the event, issue an alarm, and automatically attempt to restart the device or adjust the acquisition strategy.

[0072] S102: Perform multimodal data preprocessing and feature extraction.

[0073] This step follows the raw multi-source data acquired in S101. Through a series of refined processing steps, the raw data, which may contain noisy and redundant information, is transformed into structured, highly expressive feature vectors. These feature vectors are the foundation for subsequent damage identification and localization. Different modalities of data have different noise characteristics and information encoding methods, therefore, targeted preprocessing and feature extraction are required.

[0074] In this embodiment of the invention, step S102 further includes the following steps: S1021: Distributed optical fiber data preprocessing.

[0075] This sub-step aims to eliminate noise, drift, and outliers in the raw fiber optic sensing data and to standardize the data to make it more suitable for subsequent feature extraction and model training.

[0076] Specifically, this step is achieved through the following process: Noise filtering: Median filtering, Wiener filtering, or wavelet thresholding are used to remove random noise and environmental interference from the Brillouin scattering signal. Median filtering effectively suppresses impulse noise by replacing the median value within the data window. Wiener filtering minimizes the mean square error based on the statistical properties of signal and noise.

[0077] Baseline correction: For strain baseline shifts caused by temperature drift or systematic errors, polynomial fitting or moving average methods are used for correction to ensure that strain values ​​accurately reflect structural deformation.

[0078] Data synchronization: Data from different fiber optic sensing channels are aligned based on a unified timestamp to ensure synchronization of data from multiple points. Linear interpolation or cubic spline interpolation methods are used to unify the data to a preset sampling frequency.

[0079] Outlier detection and handling: Statistical methods, such as the three-standard-deviation criterion or the local outlier factor algorithm, are used to identify and label outliers in the data. Outliers are then removed or corrected through interpolation of neighboring points.

[0080] Parameter range: The filter window size is typically set to three to seven data points. The baseline correction polynomial order is set to first to third order. Data synchronization accuracy is required to be at the millisecond level. The outlier detection threshold can be adjusted based on practical engineering experience.

[0081] In one embodiment of the present invention, step S1021 further includes: Sub-step S10211: Integrating timestamps and spatial location data. The timestamps, physical location coordinates along the fiber optic cable, strain, and temperature values ​​of the original fiber optic data are strictly bound together to form an observation record with spatiotemporal attributes.

[0082] Sub-step S10212: Data smoothing and denoising. A Gaussian smoothing filter is applied to smooth the original strain and temperature curves to eliminate high-frequency random noise.

[0083] Sub-step S10213: Baseline drift compensation. To address strain baseline drift occurring during long-term fiber optic monitoring, a moving average method is used to estimate and subtract the baseline, ensuring that the strain data reflects the true relative change.

[0084] Sub-step S10214: Data standardization and normalization. The preprocessed strain and temperature data are standardized to zero mean and unit variance or normalized to minimum and maximum values ​​to ensure they fall within a specific numerical range, thereby eliminating the influence of dimensions and facilitating subsequent model processing.

[0085] In this embodiment of the invention, the preprocessed fiber optic data forms a structured time series table, with each row representing the data of a sensing point at a specific time, including time, location, strain, and temperature.

[0086] Regarding the interaction and exception handling in this step: If a large number of abnormal values ​​appear continuously at a certain sensing point, it may indicate that the sensor at that point is faulty. The system will record the fault information and adjust the subsequent feature extraction strategy, such as ignoring the input at that point.

[0087] S1022: Distributed optical fiber feature extraction.

[0088] This sub-step aims to extract impairment-sensitive and discriminative features from the preprocessed fiber optic time-series data.

[0089] In this embodiment of the invention, the features are explained as follows: Time-domain characteristics: These include statistical indicators such as the mean, standard deviation, maximum, minimum, kurtosis, variance, and energy of the strain or temperature series, reflecting the distribution characteristics of the data on the time axis.

[0090] Frequency domain characteristics: By converting the time-domain signal to the frequency domain using Fast Fourier Transform, features such as dominant frequency, spectral energy, bandwidth, and power spectral density can be extracted to reveal the periodicity or harmonic components of the signal. For example, changes in the structural resonant frequency are an indicator of damage.

[0091] Time-frequency domain features: By employing wavelet transform or empirical mode decomposition methods, the time-frequency features of the signal at different scales can be extracted, which can capture the local transient changes and non-stationary characteristics of the signal. For example, wavelet energy and wavelet entropy.

[0092] Modal parameter characteristics: Combining the structural dynamics model, modal parameters such as the natural frequencies, mode shapes, and damping ratios of the structure are extracted from the strain time series data. Changes in these parameters directly reflect the structural stiffness and mass damage.

[0093] Parameter range: The time window length for time-domain feature calculations is typically from tens of seconds to several minutes. The frequency range for frequency-domain analysis is set according to the inherent frequency range of the structure. Wavelet basis functions are selected, such as Daubechies wavelet or Morlet wavelet.

[0094] In one embodiment of the present invention, S1022 further includes: Sub-step S10221: Sliding window data segmentation. Continuous distributed fiber optic time-series data is segmented using a fixed-size sliding window to form a series of local time-series segments for feature computation.

[0095] Sub-step S10222: Calculation of multi-dimensional time-domain features. Calculate the statistical characteristics of strain and temperature, such as mean, variance, slope, and kurtosis, for each time segment.

[0096] Sub-step S10223: Frequency domain feature extraction. Perform a Fast Fourier Transform on each time segment to extract its frequency domain features such as dominant frequency, bandwidth, and energy spectral density.

[0097] Sub-step S10224: Construct the fiber feature vector. Concatenate all extracted time-domain, frequency-domain features, and optional time-frequency-domain features into a unified fiber feature vector.

[0098] In this embodiment of the invention, the extracted fiber features are stored in the form of one-dimensional vectors, with each vector representing fiber data features within a spatiotemporal window.

[0099] For interaction and exception handling, the feature extraction module starts after receiving the preprocessed data. If numerical overflow or non-numeric results occur during feature calculation, the system will record the error and mark the data in that window, excluding it from subsequent model training or inference.

[0100] S1023: Machine vision image data preprocessing.

[0101] This sub-step aims to perform geometric correction, illumination normalization, and region segmentation on the raw image data, providing high-quality input for subsequent feature extraction.

[0102] In this embodiment of the invention, this step is specifically implemented through the following process: Image distortion correction: Using camera calibration parameters, geometric correction is performed on the acquired image to eliminate radial and tangential distortions caused by the camera lens, ensuring the authenticity of the geometric shape in the image.

[0103] Illumination normalization: By using histogram equalization, gamma correction, or local contrast enhancement algorithms, the effects of uneven illumination on image quality are eliminated, image details are enhanced, and the robustness of feature extraction is improved.

[0104] Region of Interest (ROI) segmentation: Using image segmentation algorithms such as thresholding, edge detection, and deep learning-based semantic segmentation, the main part of the complex tower structure is extracted from the complex background, reducing the interference of irrelevant information.

[0105] Image alignment: For images in a continuous time series, feature point matching or image registration techniques are used to align images at different times to facilitate comparison of changes.

[0106] Parameter range: Distortion correction parameters are obtained from camera calibration. Illumination normalization parameters can be dynamically adjusted according to ambient lighting conditions. The threshold or model parameters of the segmentation algorithm are trained based on the characteristics of the target structure.

[0107] In one embodiment of the present invention, S1023 further includes: Sub-step S10231: Image quality assessment and filtering. Automatically detects image sharpness and noise level. Filters out blurry or severely underexposed images, or triggers a reshoot.

[0108] Sub-step S10232: Extraction of the main structural region. Using an image segmentation model, such as a convolutional neural network based on the U-Net structure, the main structural region in the image is automatically identified and segmented, eliminating background interference.

[0109] Sub-step S10233: Illumination and Color Correction. Brightness, contrast, and color difference corrections are performed on the segmented structural regions to eliminate the influence of ambient lighting changes on image features.

[0110] In this embodiment of the invention, the preprocessed image data is still a two-dimensional pixel matrix, but the image content is focused on the main body of the mixed tower structure, and the lighting conditions are uniform.

[0111] For interaction and exception handling: The image preprocessing module receives the raw image file. If image segmentation fails or image correction is ineffective, the system will log the error and may prompt for manual review.

[0112] S1024: Machine vision feature extraction.

[0113] This sub-step aims to extract high-level visual features that can characterize surface damage from the preprocessed image.

[0114] Specifically, the features in this step are explained as follows: Traditional image features include edge features such as Canny edges, texture features such as the Gray-Level Co-occurrence Matrix (GLCM), and shape features such as Hu invariant moments. These features are sensitive to changes in shape and texture, such as cracks and peeling on the surface.

[0115] Deep learning features: Pre-trained convolutional neural networks such as ResNet and VGGNet are used as feature extractors to extract high-level semantic features of images. These features can capture more abstract and robust damage patterns. For example, feature maps before global average pooling layers can be used as feature vectors.

[0116] Semantic segmentation features: For specific types of damage, such as cracks and rusted areas, the semantic segmentation model can directly output the pixel-level mask of the damage area. This mask itself can serve as a refined feature to characterize the shape, size, and location of the damage.

[0117] Object detection features: Using object detection models such as YOLO and Faster R-CNN, damage instances in the image are detected, and the bounding box coordinates, confidence score, and category of each damage instance are extracted.

[0118] In this embodiment of the invention, the network depth, kernel size, activation function, and other parameters of the deep learning model are optimized through training.

[0119] In one embodiment of the present invention, sub-step S1024 further includes: Sub-step S10241: Semantic feature extraction of the damaged area. Using the trained semantic segmentation network, the preprocessed image is classified at the pixel level to distinguish normal areas from damaged areas such as cracks, corrosion, and peeling, thus obtaining the damage mask.

[0120] Sub-step S10242: Calculation of damage morphology and geometric features. Based on the damage mask, calculate the geometric features of the damage region, such as area, perimeter, aspect ratio, and skeleton length.

[0121] Sub-step S10243: Deep visual feature encoding. The image is forward-propagated through a pre-trained convolutional neural network, and the feature vectors of the penultimate layer or a specified layer are extracted as the deep visual features of the image.

[0122] Sub-step S10244: Construct a visual feature vector. Concatenate all extracted damage geometric features and depth visual features into a unified visual feature vector.

[0123] In this embodiment: the extracted visual features are stored in the form of one-dimensional vectors, and each vector represents the visual features of an image.

[0124] For interaction and exception handling: The visual feature extraction module receives the preprocessed image. If the feature extractor fails to work due to model loading failure or incorrect input image format, the system will log the error and skip feature extraction for the current image.

[0125] S103: Achieve collaborative association and fusion of multimodal features.

[0126] This step is the key innovation of this method, aiming to establish a deep correlation between distributed fiber optic features and machine vision features, overcoming the limitations of single-modal information and achieving collaborative analysis of internal and surface damage. Simple feature stitching and fusion cannot capture the complex physical and geometric relationships between the two modalities, easily leading to information redundancy or loss of crucial information. This step introduces a collaborative correlation mechanism to ensure that, before feature fusion, the information from the two modalities has already established an effective semantic and spatial bridge.

[0127] In this embodiment of the invention, S103 further includes: S1031: Construct a multimodal feature space.

[0128] This sub-step aims to map distributed fiber optic feature vectors and machine vision feature vectors from their respective original feature spaces to a unified, low-dimensional shared embedding space. This eliminates dimensional differences and redundant information between heterogeneous data, enabling features from different modalities to be compared and fused at the same semantic level.

[0129] Specifically, this step is achieved through the following process: Unified feature dimensions: Through independent linear transformation layers or nonlinear mapping networks such as multilayer perceptrons, fiber optic feature vectors and visual feature vectors are projected onto the same dimension. For example, fiber optic feature vectors of length 128 and visual feature vectors of length 256 are mapped to a shared feature space of length 128.

[0130] Shared embedding space: Ensures that the mapped feature vectors are comparable in numerical range and distribution. Standardization or normalization techniques can be used to further process the mapped features.

[0131] In this embodiment of the invention, S1031 further includes: Sub-step S10311: Dimensionality reduction and alignment of fiber feature vectors. A separate fully connected layer is used to reduce the high-dimensional fiber feature vectors to a preset shared feature dimension, such as 128 dimensions.

[0132] Sub-step S10312: Visual feature vector dimensionality reduction and alignment. A separate fully connected layer is used to reduce the high-dimensional visual feature vector to the same shared feature dimension as the fiber optic feature vector.

[0133] In this embodiment, both the fiber optic feature vector and the visual feature vector are one-dimensional floating-point vectors of the same length.

[0134] Regarding interaction and exception handling: If numerical anomalies occur during the mapping process, such as gradient explosion, the system will use optimization strategies such as gradient pruning to handle them.

[0135] S1032: Perform feature collaborative association.

[0136] This sub-step is the core of achieving deep coupling between the two modalities of information. Its goal is to establish the spatial geometric mapping relationship between the fiber optic sensing area and the machine vision observation area, as well as their behavioral synchronization relationship in the time series, so as to achieve mutual verification and supplementation of information.

[0137] In this embodiment of the invention, spatial geometric mapping is implemented in the following manner: Coordinate system transformation: Establish a unified three-dimensional coordinate system for the hybrid tower structure. Accurately map the physical deployment path of the distributed fiber optic sensors onto this three-dimensional coordinate system. Simultaneously, by using camera intrinsic and extrinsic parameters and shooting posture, backproject the pixel coordinates in the machine vision image onto this three-dimensional coordinate system to obtain the three-dimensional point coordinates of the structural surface corresponding to each pixel in the image.

[0138] Region Correspondence: Defines the correspondence between local areas sensed by fiber optic sensors (e.g., areas spaced one meter apart) and local surface areas observed by machine vision. For example, the internal area of ​​a concrete tower monitored by a segment of fiber optic cable corresponds to a specific rectangular area in the surface image. When the fiber optic cable detects an abnormal strain in this internal area, the system will focus on the visual features of its corresponding surface area.

[0139] Correlation Matrix Construction: Construct a sparse correlation matrix to represent the spatial proximity or physical coupling relationship between fiber optic sensing points and image pixels or image regions. Matrix elements can be determined based on geometric distances, structural connectivity, etc.

[0140] Time series alignment and synchronization: Ensure that the fiber optic timing characteristics and the visual image sequence are accurately synchronized in time. If there is a time offset, the Dynamic Time Warping (DTW) algorithm or cross-correlation analysis is used for alignment.

[0141] Attention Mechanism: A cross-modal attention mechanism is introduced. When the fiber feature vector indicates the presence of potential internal damage, attention weights are used to increase the contribution of the corresponding surface region's visual features in the fusion process. Conversely, when visual features identify surface cracks, the attention mechanism also strengthens the fiber features of the corresponding internal region to identify associated internal damage.

[0142] In one embodiment of the present invention, S1032 further includes: Sub-step S10321: Establish a unified three-dimensional structural model. Based on the three-dimensional design drawings of the hybrid tower structure, construct its accurate three-dimensional digital model and define a unified global coordinate system.

[0143] Sub-step S10322: Geometric mapping between fiber optic sensing points and visual observation areas. The actual laying path of the distributed optical fiber, the precise three-dimensional coordinates of each sensing point, and the pose and field of view of the machine vision camera are accurately projected onto a unified three-dimensional structural model to establish a geometric correspondence between the internal sensing area and the external observation area.

[0144] Sub-step S10323: Construct a multimodal association weight matrix. Based on the geometric mapping relationship, calculate the spatial distance or regional overlap between each fiber feature vector and each visual feature vector, and convert it into a normalized association weight to form a matrix reflecting the spatial association strength.

[0145] Sub-step S10324: Dynamic alignment of time series. To address the potential differences in acquisition time between fiber optic data and visual data, a dynamic time warping algorithm is used to align the time series of the two types of data, ensuring synchronization in the time dimension.

[0146] In this embodiment: the association weight matrix is ​​a two-dimensional floating-point matrix.

[0147] Regarding interaction and exception handling: Geometric mapping failure or inaccurate camera pose data may lead to mapping errors. The system will record the error and prompt for recalibration or manual intervention.

[0148] S1033: Deep fusion of execution features.

[0149] This sub-step, based on collaborative correlation, uses a deep learning model to deeply fuse fiber feature vectors and visual feature vectors, generating a multimodal fusion feature vector that comprehensively reflects the health status of the hybrid tower structure.

[0150] In this embodiment of the invention, the fused network structure achieves the following functions: Stitching and fusion: The fiber feature vector and the visual feature vector are simply stitched together in terms of dimension to form a longer fused vector, which is then input into the fully connected layer for nonlinear transformation.

[0151] Weighted fusion: Based on the correlation weights calculated in S1032, the fiber optic features and visual features are weighted and summed, or the contributions of the two modes are dynamically adjusted through a gating mechanism.

[0152] Attention-based fusion: Employing the multi-head self-attention or cross-attention mechanism in the Transformer architecture, features from different modalities learn from and pay attention to each other during the fusion process, automatically discovering important feature combinations. For example, crack information in visual features can be used to focus on the corresponding strain abrupt change region in optical fiber features.

[0153] In this embodiment of the invention, the fusion strategy includes: Early fusion: Fusion is performed directly after the feature extraction stage. The advantage is that it can capture more original feature interactions.

[0154] Intermediate fusion: After feature extraction, some high-level semantic features are processed before fusion. The advantage is that each modality feature already has a certain abstraction ability.

[0155] Fusion parameter learning: The weights and biases of the fusion network are learned in an end-to-end manner with the goal of minimizing the loss function of the damage identification task.

[0156] The fused multimodal feature vector can be represented as:

[0157] in, This represents the fused feature vector. MLP represents the nonlinear mapping performed by the multilayer perceptron, and Concat represents the concatenation operation of the feature vectors. and These are the characteristics of optical fibers. and visual features The linear transformation weight matrix is ​​used to adjust the dimension and importance of features.

[0158] In this embodiment of the invention, S1033 includes: Sub-step S10331: Modal Feature Interaction Learning. The fiber feature vector and visual feature vector are input into a multimodal interaction module. This module contains a multi-layered cross-attention mechanism, allowing fiber features to focus on relevant information in visual features and vice versa.

[0159] Sub-step S10332: Constructing the fusion vector. The fiber and visual feature vectors after interactive learning are concatenated to form an initial fusion feature vector.

[0160] Sub-step S10333: Deep nonlinear fusion transformation. The initial fused feature vector is input into a deep neural network containing multiple fully connected layers and activation functions, and a nonlinear transformation is performed to generate the final battery state-charging behavior interactive encoding vector.

[0161] In this embodiment: the fused feature vector is a one-dimensional floating-point vector, with a dimension of, for example, 256 or 512.

[0162] Regarding interaction and anomaly handling: If the loss function has difficulty converging or overfitting occurs during the training of the fusion network, the network structure or training hyperparameters should be adjusted.

[0163] S104: Perform damage identification and localization.

[0164] This step is the core output of this method. It utilizes the fused feature vector generated by S103 and, through a trained deep learning model, classifies the type of damage, precisely locates it, and assesses its severity in the hybrid tower structure. This is crucial for transforming perceived data into actionable insights.

[0165] In this embodiment of the invention, S104 includes: S1041: Damage classification and identification.

[0166] This sub-step uses a classification model to identify the types of damage present in the hybrid tower structure based on the fused feature vectors.

[0167] In this embodiment of the invention, the details of this step are as follows: Classification model: Employs a deep classification network, such as a multilayer perceptron, convolutional neural network, or a Transformer-based classifier. This network is trained on a large amount of labeled damage data, learning the mapping from fused features to damage categories.

[0168] Damage types: Identifiable damage types include, but are not limited to: surface cracks, surface spalling, surface corrosion, internal concrete cracking, internal steel bar buckling, internal debonding, internal voids, etc.

[0169] Classification output: The classification model outputs the probability distribution for each damage category.

[0170] In this embodiment of the invention, S1041 further includes: sub-step S10411: inputting fused features to the classifier. The fused feature vector generated in S1033 is used as input and passed to a pre-trained deep classification network.

[0171] Sub-step S10412: Damage category probability prediction. The classification network calculates the probability values ​​of various types of damage in the hybrid tower structure based on the fusion features. For example, it outputs the probability of damage types such as cracks, debonding, and corrosion.

[0172] Sub-step S10413: Determine the primary damage type. Select the category with the highest probability as the final damage identification result.

[0173] In this embodiment, the classification result is a damage category name and a confidence score.

[0174] S1042: Precise damage localization.

[0175] The goal of this sub-step is to precisely pinpoint the location of the damage on the hybrid tower structure. By combining the linear positioning capabilities of distributed optical fibers with the surface positioning accuracy of machine vision, higher-precision collaborative positioning can be achieved.

[0176] In this embodiment of the invention, the details of this step are as follows: Fiber optic positioning capability: Distributed fiber optic sensing can provide approximate areas of damage along the fiber optic cable's path. For example, when a segment of fiber experiences a sudden change in strain, the structural area covered by that segment can be located.

[0177] Visual positioning accuracy: Machine vision, through target detection or semantic segmentation models, can directly outline damaged areas in images and provide pixel-level coordinates. By back-projecting the image onto a 3D model, the pixel coordinates can be converted into precise 3D coordinates of the structural surface.

[0178] Cooperative localization: When the optical fiber indicates signs of internal damage, the system focuses on analyzing the visual image of the corresponding surface area to look for further evidence of surface damage. Conversely, when vision detects surface damage, it also verifies whether there are related internal defects through optical fiber data. By combining and complementing these two types of information, the localization range can be narrowed down, and the localization accuracy can be improved. For example, if the optical fiber locates region X, and vision identifies a crack in the image of this region, then the damage is determined to be located at the Y-coordinate of the surface in that region.

[0179] In one embodiment of the invention, the set of positioning points of the fiber optic sensor is considered. = { , , ..., } and the set of damaged areas identified by machine vision = { , , ..., The goal of co-localization is to find an optimal location of damage. , and A subset of and A subset of the data has the greatest consistency.

[0180]

[0181] Where S represents all possible spatial locations of the hybrid tower structure. It is a subset of fiber optic positioning points associated with L. It is a subset of visually impaired regions associated with L, and Similarity is a function that measures location correlation.

[0182] In this embodiment of the invention, S1042 includes: Sub-step S10421: Coarse localization of the damage area based on fiber optic data. Based on the location of fiber strain or temperature abrupt changes, and combined with the three-dimensional coordinates of the fiber laying path, determine the internal structural regions where damage may exist.

[0183] Sub-step S10422: Precise localization of the damaged surface based on visual data. Utilizing the results of machine vision object detection or semantic segmentation, combined with camera pose and a 3D structural model, the damaged area in the image is precisely mapped to the 3D coordinates of the structural surface.

[0184] Sub-step S10423: Multimodal positioning information fusion and optimization. The spatial intersection of the coarsely located fiber optic region and the precisely located visual region is performed, and combined with structural topology, the precise three-dimensional positioning coordinates of the damage are further optimized.

[0185] In this embodiment: the location result is the precise coordinates or region boundary box of the damage in the three-dimensional model of the hybrid tower structure.

[0186] S1043: Assessment of the severity of injury.

[0187] This sub-step quantifies the severity of damage based on the identified damage type and location, combined with the geometric characteristics of the damage, such as crack length, width, and depth, as well as the structural mechanics model.

[0188] In this embodiment of the invention, the details of this step are as follows: Feature quantization: Extract the length and width of the crack from the image, and analyze the amplitude and duration of strain abrupt changes from the fiber optic data.

[0189] Risk assessment model: Establish a damage risk assessment model that comprehensively considers damage type, size, location, material properties, and load conditions, and outputs a numerical damage severity level or risk index. For example, based on fracture mechanics or damage mechanics theory, calculate the stress intensity factor of the crack.

[0190] Grading: The severity of the injury is classified into different levels, such as minor, moderate, and severe, and corresponding treatment recommendations are given.

[0191] In this embodiment of the invention, S1043 includes: sub-step S10431: damage feature quantification. Quantitative attributes of the damage are extracted, such as the millimeter-level length and width of the crack, the micro-strain magnitude of the internal strain abrupt change, and the geometric dimensions of the damaged region.

[0192] Sub-step S10432: Structural damage mechanical response analysis. Based on the finite element model of the hybrid tower structure, mechanical simulations are performed on the identified and located damages to analyze their impact on the structural bearing capacity, stiffness, and remaining life.

[0193] Sub-step S10433: Damage severity classification. Based on the quantitative characteristics of the damage and the results of mechanical response analysis, the damage is classified into three or more levels: minor, moderate, and severe.

[0194] In this embodiment: the assessment result is the severity level of the injury and the corresponding numerical risk index.

[0195] S105: Generate visual reports and real-time alarm information.

[0196] This step is the final presentation of the damage identification results, designed to convey damage information to users or managers in an intuitive and efficient manner, and to trigger corresponding alarm mechanisms based on preset thresholds.

[0197] S1051: Generate damage distribution map and 3D model annotation.

[0198] This sub-step integrates the identified and located damage information into a 3D model of the hybrid tower structure, generating an intuitive visualization report.

[0199] In this embodiment of the invention, the details of this step are as follows: 3D model rendering: Render the 3D digital model of the hybrid tower structure to provide a realistic visual background.

[0200] Damage overlay annotation: The type, location, and severity of damage are overlaid onto the 3D model using color coding, labels, arrows, etc. For example, red represents severe damage, yellow represents moderate damage, and green represents minor damage. Cracks are indicated by lines, and internal cavities are represented by transparent blocks.

[0201] Multi-view display: Provides views from different angles, cross-sectional views, and enlarged views of specific areas so that users can have a comprehensive understanding of the damage.

[0202] Dynamic display: For time-series damage data, damage evolution animations can be generated to show the initiation and expansion process of damage.

[0203] In this embodiment of the invention, S1051 includes: Sub-step S10511: Damage data format conversion and integration. This involves converting the damage identification, localization, and assessment results from the model output format into the 3D annotated data format required by the visualization system.

[0204] Sub-step S10512: Damage 3D Model Rendering. On the 3D model of the hybrid tower structure, the identified damage types and severity are marked with different colors and icons. For example, cracks are drawn as red lines on the surface model, and internal debonding is displayed as translucent blocks within the internal structure.

[0205] Sub-step S10513: Generate damage information report. Automatically generate a detailed text report containing damage type, location, size, severity, development trend, and recommended treatment measures.

[0206] In this embodiment: the visualization report is stored in the form of an interactive 3D model or a static image file, accompanied by a detailed text report.

[0207] S1052: Issue a real-time alarm.

[0208] This sub-step sends alarm information to relevant management personnel in a timely manner based on the severity of the damage and the preset alarm threshold, so that emergency response measures can be taken.

[0209] In this embodiment of the invention, the details of this step are as follows: Alarm threshold setting: Set corresponding alarm thresholds for different types of damage and different severity levels. For example, a moderate alarm is triggered when the crack width exceeds 0.5 mm or the internal strain mutation exceeds 500 microstrains.

[0210] Alarm levels: Alarm information is divided into different levels such as emergency alarm, important alarm, and general alarm, corresponding to different response priorities.

[0211] Alarm channels: Alarm information is sent through multiple channels, including audible and visual alarms, SMS notifications, emails, and mobile application push notifications.

[0212] Alarm Log: All alarm events and their response times will be recorded in the log for subsequent event tracing and system performance evaluation.

[0213] In this embodiment of the invention, S1052 includes: Sub-step S10521: Alarm threshold detection. Real-time comparison of damage identification results, such as damage severity level or risk index, with preset alarm thresholds.

[0214] Sub-step S10522: Generate an alarm message. Once damage is detected exceeding the alarm threshold, the system immediately generates an alarm message containing detailed damage information.

[0215] Sub-step S10523: Send alarm via multiple channels. Send the generated alarm message to the preset recipient via SMS service, email service, or application programming interface, and simultaneously issue an audible and visual alarm on the local monitoring interface.

[0216] In this embodiment, the alarm information is stored in a structured text format, including alarm time, damage type, location, severity, alarm level, and handling suggestions.

[0217] Regarding interaction and exception handling: If an alarm fails to be sent, such as due to a failure in the SMS service, it will trigger an internal error log entry and attempt to send the alarm through another available channel.

[0218] Example 3 This invention also provides a surface-internal collaborative damage identification system for hybrid tower structures, such as... Figure 4 As shown, the system includes: The sensor deployment module 100 is used to deploy a distributed fiber optic sensor network and a machine vision system in the hybrid tower structure. The fiber optic sensor network collects internal strain and temperature data in real time, and the machine vision system collects surface visual image data periodically. The data preprocessing and synchronization module 200 is used to perform noise reduction, baseline correction and temperature compensation processing on the optical fiber data to form preprocessed optical fiber data; to perform distortion correction, illumination normalization and image registration processing on the visual image data to form preprocessed visual image data; and to achieve multi-source data synchronization through timestamp alignment. The feature extraction module 300 is used to input the preprocessed optical fiber data into a one-dimensional convolutional neural network to extract internal damage feature vectors, and to input the preprocessed visual image data into a two-dimensional convolutional neural network to extract surface damage feature vectors. The collaborative association and multimodal fusion module 400 is used to map the internal damage feature vector and the surface damage feature vector to the three-dimensional structural model coordinate system through the collaborative association unit, and guide the feature extraction process based on the attention mechanism, and generate a six-dimensional collaborative fusion damage feature vector through the multimodal fusion network. The risk assessment and alarm module 500 is used to determine the damage type, location and severity based on the risk index in the six-dimensional collaborative fusion damage feature vector, output the damage spatial coordinates and development trend in combination with the three-dimensional model, and trigger a warning, alert or emergency level three alarm based on a preset threshold.

[0219] The method and system of this embodiment deeply integrate the internal sensing capabilities of distributed fiber optic sensing with the external fine-grained recognition capabilities of machine vision to construct a full-link intelligent monitoring framework from data acquisition to damage identification, location, and early warning. This framework effectively improves the comprehensiveness, accuracy, and intelligence level of health monitoring for hybrid tower structures, providing solid technical support for ensuring structural safety and extending service life.

[0220] Example 4 To implement the methods of the above embodiments, the present invention also provides a computer device, which includes a memory and a processor; wherein the processor runs a program corresponding to the executable program code by reading executable program code stored in the memory, so as to implement the various steps of the methods described above.

[0221] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0222] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0223] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A method for identifying surface-internal collaborative damage in a hybrid tower structure, characterized in that, include: A distributed fiber optic sensor network and a machine vision system are deployed in the hybrid tower structure. The fiber optic sensor network collects internal strain and temperature data in real time, and the machine vision system collects surface visual image data periodically. The fiber optic data is subjected to noise reduction, baseline correction, and temperature compensation to form preprocessed fiber optic data; the visual image data is subjected to distortion correction, illumination normalization, and image registration to form preprocessed visual image data, and multi-source data synchronization is achieved through timestamp alignment. The preprocessed fiber optic data is input into a one-dimensional convolutional neural network to extract internal damage feature vectors, and the preprocessed visual image data is input into a two-dimensional convolutional neural network to extract surface damage feature vectors. The internal damage feature vector and the surface damage feature vector are mapped to the three-dimensional structural model coordinate system through the collaborative association unit, and the feature extraction process is guided by the attention mechanism. A six-dimensional collaborative fusion damage feature vector is generated through the multimodal fusion network. The damage type, location, and severity are determined based on the risk index in the six-dimensional collaborative fusion damage feature vector. The damage spatial coordinates and development trend are output in combination with the three-dimensional model, and a warning, alert, or emergency level three alarm is triggered based on a preset threshold.

2. The method as described in claim 1, characterized in that, The deployment of a distributed fiber optic sensor network and machine vision system in the hybrid tower structure, including real-time acquisition of internal strain and temperature data via the fiber optic sensor network and periodic acquisition of surface visual image data via the machine vision system, further includes: A first type of distributed optical fiber sensor is laid along a preset path inside or on the surface of the concrete part of the hybrid tower structure. The first type of distributed optical fiber sensor adopts a fiber Bragg grating array. A second type of distributed optical fiber sensor is laid along a preset path on the surface of the steel structure part of the hybrid tower structure. The second type of distributed optical fiber sensor adopts distributed optical fiber sensing technology based on Rayleigh scattering, and the optical fiber is a standard single-mode optical fiber. The first or second type of distributed optical fiber sensor is reinforced in the interface area where different materials connect the hybrid tower structure. A first type of machine vision sensor, which is a high-resolution visible light camera, is installed at a fixed location around the hybrid tower structure or on a flyable drone. A second type of machine vision sensor, which is an infrared thermal imaging camera, is installed at a fixed location around the hybrid tower structure or on a flyable drone. The visible light images of the surface of the hybrid tower structure are periodically captured by the first type of machine vision sensor at a frequency of at least once per hour, wherein the UAV is collecting images while flying at a fixed altitude and speed along a preset route. The second type of machine vision sensor periodically captures infrared thermal images of the surface of the hybrid tower structure at a frequency of at least once a day to assist in detecting temperature anomalies beneath the surface.

3. The method as described in claim 1, characterized in that, The process of performing noise reduction, baseline correction, and temperature compensation on the fiber optic data to form preprocessed fiber optic data; performing distortion correction, illumination normalization, and image registration on the visual image data to form preprocessed visual image data; and achieving multi-source data synchronization through timestamp alignment also includes: The collected strain and temperature data were subjected to wavelet threshold denoising, with the Daubechies series wavelet selected for the strain of the concrete part and the Coiflets wavelet used for the steel structure part. Baseline drift correction is performed on the denoised data using moving average filtering or polynomial fitting. Temperature compensation is performed on the strain data based on the temperature sensitivity of the fiber optic sensor itself or by using data from an independent temperature sensor. Radial and tangential distortion corrections are performed on the visible light image and infrared thermal image; The image is subjected to illumination normalization processing using adaptive histogram equalization or gamma correction methods. An image registration algorithm based on feature point matching is used to register multiple images into a unified three-dimensional structural model coordinate system; The target monitoring area of ​​the hybrid tower structure is cropped from the registered image.

4. The method as described in claim 1, characterized in that, The step of inputting the preprocessed fiber optic data into a one-dimensional convolutional neural network to extract internal damage feature vectors, and inputting the preprocessed visual image data into a two-dimensional convolutional neural network to extract surface damage feature vectors, further includes: The preprocessed fiber optic data is input into a one-dimensional convolutional neural network containing at least three convolutional layers, a batch normalization layer, and an activation function layer. The abnormal features include local strain, strain gradient anomalies, temperature gradient anomalies, and changes in signal frequency domain energy distribution, forming the internal damage feature vector; The preprocessed visual image data is input into a two-dimensional convolutional neural network employing a semantic segmentation model and an object detection model architecture. The semantic segmentation model is either U-Net or DeepLabV3. The target detection model is either the YOLO series or the Faster R-CNN model; The two-dimensional convolutional neural network outputs information on the type, location, size, and severity of surface damage to the hybrid tower structure. The damage types include cracks, peeling, corrosion, or coating failure, forming the surface damage feature vector.

5. The method as described in claim 1, characterized in that, The process of mapping the internal damage feature vector and the surface damage feature vector to the three-dimensional structural model coordinate system through a collaborative association unit, and guiding the feature extraction process based on an attention mechanism, and generating a six-dimensional collaborative fusion damage feature vector through a multimodal fusion network, further includes: Based on the geographic coordinates of the strain concentration area or temperature anomaly area indicated in the internal damage feature vector, potential internal damage areas are marked on the three-dimensional model of the hybrid tower structure. The potential internal damage area is projected onto a two-dimensional plane corresponding to the preprocessed visual image data to determine the key areas of focus for visual feature extraction. Prioritize areas associated with fiber optic anomalies; The internal damage feature vector and the surface damage feature vector are concatenated along the feature dimension to form an initial fused feature vector; The initial fused feature vector is input into a fusion network based on the Transformer architecture or a gated recurrent unit (GRU). The fusion network includes multi-layer self-attention mechanisms and cross-modal attention mechanisms to learn deep correlations and complementary information between the two modalities. The fusion network outputs a six-dimensional collaborative fusion damage feature vector, where the six dimensions represent the risk indices of cracks, spalling, corrosion, internal voids, material fatigue, and overall structural instability, respectively, with the risk indices ranging from zero to one.

6. The method as described in claim 1, characterized in that, The process of determining the damage type, location, and severity based on the risk index in the six-dimensional collaborative fusion damage feature vector, outputting the damage spatial coordinates and development trend in conjunction with the three-dimensional model, and triggering early warning, alert, or emergency level three alarm based on a preset threshold also includes: If any risk index in the collaborative fusion damage feature vector exceeds a preset threshold, it is determined that there is a corresponding type of damage. The damage types include concrete cracks, steel corrosion, internal voids, delamination, or material fatigue. If fiber optic data indicates internal stress concentration and visual data detects microcracks in the corresponding surface area, the system determines that there is potential damage inside the structure that is developing towards the surface, and provides its three-dimensional spatial coordinates and development trend. If visual data detects severe surface cracks and fiber optic data shows abnormal strain in the surrounding area, the system confirms the severity of the damage and its impact on the overall load-bearing capacity of the structure, and provides precise three-dimensional spatial positioning and geometric dimensions.

7. The method as described in claim 1, characterized in that, Also includes: A structural health report is generated, which includes historical damage data, damage development trend prediction, and maintenance recommendations. The development trend prediction is based on time series analysis of the six-dimensional collaborative fusion damage feature vector, and the maintenance recommendations are matched with a preset maintenance strategy library according to the damage type and severity.

8. A surface-internal collaborative damage identification system for hybrid tower structures, characterized in that, include: The sensor deployment module is used to deploy a distributed fiber optic sensor network and a machine vision system in the hybrid tower structure. The fiber optic sensor network collects internal strain and temperature data in real time, and the machine vision system collects surface visual image data periodically. The data preprocessing and synchronization module is used to perform noise reduction, baseline correction and temperature compensation on the fiber optic data to form preprocessed fiber optic data; to perform distortion correction, illumination normalization and image registration on the visual image data to form preprocessed visual image data; and to achieve multi-source data synchronization through timestamp alignment. The feature extraction module is used to input the preprocessed optical fiber data into a one-dimensional convolutional neural network to extract internal damage feature vectors, and to input the preprocessed visual image data into a two-dimensional convolutional neural network to extract surface damage feature vectors. The collaborative association and multimodal fusion module is used to map the internal damage feature vector and the surface damage feature vector to the three-dimensional structural model coordinate system through the collaborative association unit, and guide the feature extraction process based on the attention mechanism, and generate a six-dimensional collaborative fusion damage feature vector through the multimodal fusion network. The risk assessment and alarm module is used to determine the damage type, location and severity based on the risk index in the six-dimensional collaborative fusion damage feature vector, output the damage spatial coordinates and development trend in combination with the three-dimensional model, and trigger a warning, alert or emergency three-level alarm based on a preset threshold.

Images