Intelligent monitoring method for multi-modal data fusion of water transportation infrastructure

Abstract

The invention relates to the technical field of water transportation infrastructure health monitoring, in particular to an intelligent monitoring method for water transportation infrastructure multi-modal data fusion, which comprises the following steps of: acquiring multi-modal monitoring data of a wharf structure through a distributed sensor array, and performing space-time alignment processing on the data by adopting technologies such as GNSS time synchronization and ICP space registration to obtain a multi-modal monitoring result; and constructing a multi-dimensional fusion feature matrix based on the data after space-time alignment, and dynamically adjusting a health assessment weight in combination with environmental factors such as tidal phase and ship tonnage to generate an environment adaptive health score. And a graph neural network anomaly propagation model is adopted to calculate the spatial-temporal correlation degree, the structural anomaly is accurately identified, a graded early warning signal is generated according to a standard exceeding area, and graded response measures such as sensor rechecking, underwater inspection and structural reinforcement are triggered. According to the invention, the intelligent, automatic and precise level of water transportation infrastructure health monitoring is improved, and the system is suitable for long-term safety monitoring in complex environments such as ports and wharfs.

Description

Technical Field

[0001] The present invention relates to the technical field of health monitoring of water transportation infrastructure, and particularly to an intelligent monitoring method for multi-modal data fusion of water transportation infrastructure. Background Art

[0002] Water transportation infrastructure (such as port terminals, bridge piers, etc.) operates for a long time in complex environments such as the ocean and rivers. Affected by various factors such as tides, water flow scouring, ship berthing impacts, and wind waves, it is prone to structural fatigue, corrosion, scouring and silting changes, and local damage. These structural changes may lead to a decrease in the bearing capacity of the terminal, uneven settlement of the foundation, and even safety accidents. Therefore, accurate monitoring, intelligent assessment, and early warning of the health status of water transportation infrastructure are of great significance for ensuring port operation safety, extending the service life of infrastructure, and optimizing maintenance decisions.

[0003] The existing technologies have the following main problems in the monitoring of water transportation infrastructure: Traditional monitoring methods usually rely on a single type of sensor, such as only based on fiber Bragg grating strain or sonar point cloud, which cannot comprehensively reflect the structural health status and lack the ability of multi-dimensional data fusion; Due to the differences in time sampling intervals and spatial coordinate systems of data from different types of sensors, traditional methods are difficult to effectively align the data, resulting in isolated information and affecting the analysis accuracy; Most of the existing monitoring methods rely on static assessment models and do not fully consider dynamic environmental factors such as tides, hydrological conditions, and ship berthing impacts, resulting in lagging assessment results and inability to adjust the prediction accuracy in real time. Summary of the Invention

[0004] Based on the above purposes, the present invention provides an intelligent monitoring method for multi-modal data fusion of water transportation infrastructure, integrating multi-source sensor data, achieving high-precision spatio-temporal alignment, constructing an environment-adaptive assessment model, and combining an intelligent anomaly detection algorithm to achieve accurate health monitoring and early warning of water transportation infrastructure and improve the intelligent level of safety management.

[0005] An intelligent monitoring method for multi-modal data fusion of water transportation infrastructure includes the following steps: S1, real-time collecting multi-modal monitoring data of the dock structure, including fiber Bragg grating strain sequences, underwater three-dimensional sonar point clouds, tidal water level time series, and ship berthing impact waveforms; S2, performing spatio-temporal alignment processing on the multi-modal monitoring data to generate a standardized data set with a unified time stamp and spatial coordinate system; S3, constructing a multi-dimensional fusion feature matrix based on the standardized data set, where the feature matrix includes a structural deformation gradient field, an underwater scouring and silting volume change rate, and a load impact energy spectral density; S4. Dynamically adjust the weight coefficients of the feature matrix according to the real-time tidal phase and ship tonnage parameters to generate health assessment indicators adaptable to the environment. S5. Calculate the spatio-temporal correlation degree of the health assessment indicators through the abnormal propagation model, and generate a hierarchical early warning signal when the correlation degree exceeds the early warning threshold.

[0006] Optionally, the S1 includes: S11. Generate a fiber Bragg grating strain sequence: Deploy a fiber Bragg grating sensor array along the axis of the wharf pile foundation at a spacing of 2 meters, and continuously collect the original strain vibration waveform through the fiber Bragg grating sensor array at a sampling frequency of 200 Hz. Perform wavelet transform noise reduction processing on the original strain vibration waveform to eliminate electromagnetic interference noise and generate noise-reduced fiber strain data. Perform spatio-temporal interpolation calculation on the noise-reduced fiber strain data to compensate for the sensor blind area data and generate a fiber Bragg grating strain sequence with continuous spatio-temporal characteristics. S12. Detect the coordinates of the abnormal amplitude area: Perform amplitude threshold analysis on the fiber Bragg grating strain sequence, set the dynamic threshold range to ±500 με, and extract the abnormal strain data segments in the fiber Bragg grating strain sequence that exceed the dynamic threshold range. According to the spatial position mapping relationship of the fiber Bragg grating sensor array, convert the sensor node coordinates corresponding to the abnormal strain data segment into the three-dimensional space coordinates of the wharf structure to generate the coordinates of the abnormal amplitude area. S13. Generate an underwater three-dimensional sonar point cloud: Based on the coordinates of the abnormal amplitude area, control an unmanned survey ship to carry a multi-beam sonar to scan the target area, set the sonar working frequency to 500 kHz and the beam opening angle to 120°, and obtain the original reflection signal. Perform incident angle compensation on the original reflection signal to eliminate seabed terrain distortion and generate a sonar signal after angle correction; perform seabed reverberation suppression processing on the sonar signal after angle correction, and use an adaptive filtering algorithm to separate the effective reflection waves to generate a sonar reflection intensity matrix. Perform three-dimensional coordinate transformation on the sonar reflection intensity matrix to map the reflection intensity to the Cartesian coordinate system. Use the iterative closest point algorithm to splice the coordinate data of continuous scan frames to eliminate the pose deviation in the scan overlapping area. Perform point cloud downsampling processing on the spliced three-dimensional coordinate data, retain the key feature points with millimeter-level accuracy, and generate an underwater three-dimensional sonar point cloud. S14. Generate a tidal water level time series: Uniformly arrange 4 groups of radar water level gauges at the wharf front and synchronously collect the original tide level elevation values at a period of 1 minute. Perform differential calculation on the original tide level elevation value and the astronomical tide table to eliminate the sensor zero drift error and generate a calibrated tide level elevation value; Align the time stamps of the calibrated tide level elevation value through the GNSS time synchronization module to generate a time-synchronized tidal water level time series; S15. Generate the ship berthing impact waveform: Install a triaxial acceleration sensor group at the root of the mooring bollard and the center axis of the fender, and synchronously obtain the tidal water level time series as the environmental phase reference; When it is detected that the acceleration amplitude exceeds the set threshold of 5g, trigger a high-speed acquisition mode with a sampling interval of 0.5 ms to capture the original impact vibration waveform at the moment of ship contact; Perform phase synchronization processing on the original impact vibration waveform, and combine the real-time water level phase of the tidal water level time series to generate a ship berthing impact waveform; S16. Construct a multimodal monitoring data packet: Package the fiber Bragg grating strain sequence, underwater three-dimensional sonar point cloud, tidal water level time series, and ship berthing impact waveform according to the original acquisition time sequence into a multimodal monitoring data packet, and the multimodal monitoring data packet retains the original time stamps and spatial coordinate information of each sensor.

[0007] Optionally, the S2 includes: S21. Time stamp synchronization calibration: Extract the original time stamps of each sensor from the multimodal monitoring data packet, calibrate through the GNSS second pulse signal to generate a unified reference time axis, and perform time interpolation resampling on the data stream to output a time-synchronized multimodal data stream; S22. Spatial coordinate system unification: Based on the dock BIM model, perform spatial registration on the output time-synchronized multimodal data stream, including sonar point cloud rigid registration, fiber optic coordinate mapping, and tidal elevation conversion, and output spatio-temporally aligned multimodal data; S23. Spatio-temporal correlation analysis: Map the spatio-temporally aligned data output by S22 to a spatio-temporal correlation matrix, analyze the spatial correlation between fiber optic strain and sonar deformation, and mark the data conflict areas.

[0008] Optionally, the S2 further includes: S24. Conflict area arbitration: Perform multimodal arbitration on the data conflict areas, and generate an arbitrated spatio-temporal correlation matrix based on the synchrony of the ship impact waveform energy spectrum characteristics and the fiber optic strain time series; S25. Generate a standardized data set: Normalize the data of the arbitrated spatio-temporal correlation matrix to generate a standardized data set with a unified time stamp and spatial coordinate system.

[0009] Optionally, the S3 includes: S31. Construct the structural deformation gradient field: Extract the fiber Bragg grating strain sequence and underwater three-dimensional sonar point cloud data from the standardized dataset to generate the structural deformation gradient field; S32. Calculate the underwater scouring and silting volume change rate: Generate the underwater scouring and silting volume change rate based on the underwater three-dimensional sonar point cloud in the standardized dataset.

[0010] Optionally, S3 further includes: S33. Extract the load impact energy spectral density: Generate the load impact energy spectral density based on the ship berthing impact waveform in the standardized dataset; S34. Construct a multi-dimensional fusion feature matrix: Fuse the structural deformation gradient field, underwater scouring and silting volume change rate, and load impact energy spectral density to generate a multi-dimensional fusion feature matrix.

[0011] Optionally, S4 includes: S41. Extract the real-time tidal phase parameter: Analyze the tidal water level time series from the standardized dataset to generate the real-time tidal phase parameter, ensuring that the tidal state can dynamically affect the health assessment calculation; S42. Obtain the ship tonnage parameter: Real-time obtain the tonnage, length, and width parameters of the berthing ship from the port ship scheduling system to generate a ship tonnage feature vector, ensuring that the health assessment calculation can consider the ship impact effect; S43. Determine the basic feature weight coefficient: Based on the multi-dimensional fusion feature matrix, set the initial weights of the structural deformation gradient field, underwater scouring and silting volume change rate, and load impact energy spectral density to ensure reasonable allocation of the basic weights for the health assessment calculation.

[0012] Optionally, S4 further includes: S44. Dynamic weight adjustment of tidal phase: Dynamically correct the initial weight coefficient according to the real-time tidal phase parameter to generate a tidal correction weight, ensuring that the health assessment calculation adapts to the tidal environment change; S45. Dynamic weight adjustment of ship tonnage: Perform a secondary adjustment on the tidal correction weight based on the ship tonnage feature vector to generate a final dynamic weight coefficient, ensuring that the health assessment calculation adapts to the ship berthing impact effect; S46. Generate the health assessment index: Calculate the environment-adaptive health assessment index based on the final dynamic weight coefficient and the multi-dimensional fusion feature matrix to ensure accurate and quantifiable health status assessment.

[0013] Optionally, S5 includes: S51. Construct an anomaly propagation model: Construct a graph neural network anomaly propagation model based on the health assessment index and define the mechanical transfer relationship of the model nodes and edges; S52. Calculate the spatio-temporal correlation degree: Input the real-time health assessment indicators into the abnormal propagation model, calculate the abnormal spatio-temporal correlation degree, and generate a spatio-temporal correlation degree matrix; S53. Determine the warning threshold: Compare the spatio-temporal correlation degree matrix with the preset warning threshold to identify the list of over-standard areas; S54. Generate a hierarchical warning signal: Generate a hierarchical warning signal according to the list of over-standard areas and trigger the corresponding response mechanism; S55. Execute the hierarchical response instruction: Start the corresponding emergency response process according to the level of the warning signal.

[0014] Advantages of the present invention: In the present invention, by collecting multi-modal monitoring data of the wharf structure, including fiber Bragg grating strain sequences, underwater three-dimensional sonar point clouds, tidal water level time series, and ship berthing impact waveforms, the full-field perception and high-precision monitoring of the wharf structure state are realized. Through technologies such as GNSS second pulse synchronization, ICP spatial alignment, and spatio-temporal correlation degree analysis, the time consistency and spatial accuracy of the data are ensured, and methods such as wavelet noise reduction and multi-modal data arbitration are used to eliminate interference and improve the reliability of the monitoring data. In addition, by integrating multi-dimensional features such as the structural deformation gradient field, the underwater scouring and silting volume change rate, and the load impact energy spectral density, the integrity and high precision of the monitoring results are ensured, providing reliable data support for the health state of the wharf structure.

[0015] In the present invention, by combining dynamic factors such as the real-time tidal phase and ship tonnage, the health assessment weight is accurately adjusted to improve the accuracy and adaptability of the assessment. In terms of the influence of tides, the astronomical tide table and the real-time water level change rate are used to dynamically adjust the weights of underwater scouring and silting and structural deformation to adapt to the structural stress changes in different tidal stages; in terms of the influence of ship berthing, the ship tonnage and size information are extracted by parsing AIS data, and the assessment parameters are adjusted in combination with the tonnage impact factor, making the health assessment indicators more in line with the actual working conditions. Finally, the health score is calculated through a multi-dimensional fusion feature matrix and mapped to the 0-100 interval, providing an intuitive and quantitative intelligent assessment for the health state of the wharf structure.

[0016] In the present invention, based on the graph neural network abnormal propagation model, the intelligent detection of wharf structure damage is realized, and the corresponding emergency response is triggered through a hierarchical warning mechanism, improving the initiative and accuracy of safety management. In terms of abnormal detection, a spatio-temporal abnormal propagation model is constructed using the health assessment indicator matrix, the abnormal correlation degree of each partition is calculated, and the abnormal area is accurately identified; in terms of the warning mechanism, three-level warning thresholds are set, and alarms of corresponding levels are triggered for abnormal areas of different severity levels, improving the intelligent level of wharf structure safety management. Description of the Drawings

[0017] To more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only those of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1 Schematic diagram of the method flow of the embodiment of the present invention; Figure 2 Schematic diagram of the S4 process of the embodiment of the present invention. Detailed implementation manners

[0019] The present invention will be described in detail below in conjunction with the drawings and specific embodiments. At the same time, it should be noted here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments. For some well-known technologies, those skilled in the art can also adopt other alternative methods for implementation; moreover, the drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0020] It should be noted that in the specification, when referring to "an embodiment", "embodiment", "exemplary embodiment", "some embodiments", etc., it indicates that the described embodiment may include specific features, structures or characteristics, but not necessarily every embodiment includes this specific feature, structure or characteristic. In addition, when combining an embodiment to describe a specific feature, structure or characteristic, implementing such a feature, structure or characteristic in combination with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the relevant art.

[0021] Generally, terms can be understood at least in part from their use in the context. For example, at least in part depending on the context, the term "one or more" used herein can be used to describe any feature, structure or characteristic in a singular sense, or can be used to describe a combination of features, structures or characteristics in a plural sense. In addition, the term "based on" can be understood as not necessarily intended to convey a set of exclusive factors, but instead, at least in part depending on the context, allowing the existence of other factors that are not necessarily explicitly described.

[0022] As Figure 1 - Figure 2 shown, the intelligent monitoring method for multimodal data fusion of water transportation infrastructure includes the following steps: S1, Real-time collect multimodal monitoring data of the wharf structure, including fiber Bragg grating strain sequences, underwater three-dimensional sonar point clouds, tidal water level time series, and ship berthing impact waveforms; S2, Perform spatio-temporal alignment processing on the multimodal monitoring data to generate a standardized data set with a unified timestamp and spatial coordinate system; S3. Construct a multi-dimensional fusion feature matrix based on a standardized data set. The feature matrix includes a structural deformation gradient field, an underwater scouring and silting volume change rate, and a load impact energy spectral density; S4. Dynamically adjust the weight coefficients of the feature matrix according to the real-time tidal phase and ship tonnage parameters to generate health assessment indicators adaptable to the environment; S5. Calculate the spatio-temporal correlation degree of the health assessment indicators through an anomaly propagation model, and generate a hierarchical early warning signal when the correlation degree exceeds the early warning threshold.

[0023] S1 includes: S11. Generate a fiber Bragg grating strain sequence: Deploy a fiber Bragg grating sensor array along the axis of the wharf pile foundation at an interval of 2 meters, and continuously collect the original strain vibration waveform through the fiber Bragg grating sensor array at a sampling frequency of 200 Hz; Perform wavelet transform noise reduction processing on the original strain vibration waveform to eliminate electromagnetic interference noise and generate noise-reduced fiber strain data; Perform spatio-temporal interpolation calculation on the noise-reduced fiber strain data to compensate for the data in the sensor blind area and generate a fiber Bragg grating strain sequence with continuous spatio-temporal characteristics; S12. Detect the coordinates of the abnormal amplitude area: Perform amplitude threshold analysis on the fiber Bragg grating strain sequence, set the dynamic threshold range to ±500 με, and extract the abnormal strain data segments in the fiber Bragg grating strain sequence that exceed the dynamic threshold range; According to the spatial position mapping relationship of the fiber Bragg grating sensor array, convert the coordinates of the sensor nodes corresponding to the abnormal strain data segments into the three-dimensional space coordinates of the wharf structure to generate the coordinates of the abnormal amplitude area; S13. Generate an underwater three-dimensional sonar point cloud: Based on the coordinates of the abnormal amplitude area, control an unmanned survey ship equipped with a multi-beam sonar to scan the target area, set the sonar working frequency to 500 kHz and the beam opening angle to 120°, and obtain the original reflection signal; Perform incident angle compensation on the original reflection signal to eliminate seabed terrain distortion and generate a sonar signal after angle correction; Perform seabed reverberation suppression processing on the sonar signal after angle correction, and use an adaptive filtering algorithm to separate the effective reflection waves to generate a sonar reflection intensity matrix; Perform three-dimensional coordinate transformation on the sonar reflection intensity matrix to map the reflection intensity to the Cartesian coordinate system; Use the iterative closest point algorithm to splice the coordinate data of consecutive scan frames to eliminate the pose deviation in the scan overlapping area; Perform point cloud downsampling processing on the spliced three-dimensional coordinate data, retain the key feature points with millimeter-level accuracy, and generate an underwater three-dimensional sonar point cloud; S14. Generate a tidal water level time series: Uniformly deploy 4 groups of radar water level gauges at the wharf front, and synchronously collect the original tide level elevation values at a period of 1 minute; Perform differential calculation on the original tide level elevation value and the astronomical tide table to eliminate the zero drift error of the sensor and generate the calibrated tide level elevation value; Align the time stamps of the calibrated tide level elevation values through the GNSS time synchronization module to generate a time-synchronized tidal water level time series; S15, Generate the ship berthing impact waveform: Install a triaxial acceleration sensor group at the root of the mooring bollard at the dock and the central axis of the fender, and synchronously obtain the tidal water level time series as the environmental phase reference; When it is detected that the acceleration amplitude exceeds the set threshold of 5g, trigger the high-speed acquisition mode with a sampling interval of 0.5 ms to capture the original impact vibration waveform at the moment of ship contact; Perform phase synchronization processing on the original impact vibration waveform, and combine the real-time water level phase of the tidal water level time series to generate the ship berthing impact waveform; S16, Construct a multi-modal monitoring data packet: Package the fiber Bragg grating strain sequence, underwater three-dimensional sonar point cloud, tidal water level time series, and ship berthing impact waveform according to the original acquisition time sequence into a multi-modal monitoring data packet, and the multi-modal monitoring data packet retains the original time stamps and spatial coordinate information of each sensor.

[0024] S2 includes: S21, Time stamp synchronization calibration: Extract the original time stamps of each sensor from the multi-modal monitoring data packet, calibrate and generate a unified reference time axis through the GNSS second pulse signal, and perform time interpolation resampling on the data stream to output a time-synchronized multi-modal data stream. The specific steps are as follows: Extract the original time stamps: Separate the original time stamps of the fiber Bragg grating sensor, sonar sensor, radar water level gauge, and triaxial acceleration sensor from the multi-modal monitoring data packet; Clock difference compensation calculation: Generate a reference time axis through the GNSS second pulse signal, and calculate the clock difference offset between the time stamps of each sensor and the reference time; Time stamp correction: Use linear interpolation to compensate the time stamps of the sensors with clock differences to eliminate the clock drift error; Resampling alignment: Resample the data streams with inconsistent time resolutions (such as the 1-minute interval of the tidal water level time series) to a 10-ms interval using the B-spline interpolation algorithm; Generate a synchronized data stream: Output a multi-modal data stream with unified time stamps (GNSS reference) and consistent time resolutions (10 ms), including the fiber Bragg grating strain sequence, underwater three-dimensional sonar point cloud, tidal water level time series, and ship berthing impact waveform; S22, Spatial Coordinate System 1: Based on the dock BIM model, perform spatial registration on the output time-synchronized multimodal data stream, including rigid registration of sonar point clouds, fiber optic coordinate mapping, and tidal elevation conversion, and output spatio-temporally aligned multimodal data. The specific steps are as follows: Load the BIM model benchmark: Call the structural coordinate system of the dock BIM model, and set the GNSS reference station coordinates as the global origin (X = 0, Y = 0, Z = 0); Sonar point cloud registration: Use the Iterative Closest Point (ICP) algorithm to perform rigid registration of the underwater 3D sonar point cloud with the BIM model, and calculate the rotation matrix and translation vector from the point cloud to the structural coordinate system; Fiber optic coordinate mapping: According to the fiber optic grating sensor deployment topology map, convert the physical coordinates (longitude, latitude, elevation) of each sensor node into local coordinates (X, Y, Z) in the BIM model coordinate system; Tidal elevation conversion: Analyze the ellipsoidal height data of the tidal water level time series, and based on the geodetic height parameters of the GNSS reference station, convert it into orthometric height data of the 1985 National Elevation Datum through the height anomaly model; Output spatially aligned data: Generate multimodal data that is exactly the same as the BIM model coordinate system, including sonar point clouds (millimeter-level coordinates), fiber optic strain sequences (mapped to structural coordinate nodes), and tidal water levels (orthometric height reference); S23, Spatio-temporal Correlation Analysis: Map the spatio-temporally aligned data output by S22 to the spatio-temporal correlation matrix, analyze the spatial correlation between fiber optic strain and sonar deformation, and mark the data conflict areas. The specific steps are as follows: (1) Construct the spatio-temporal matrix framework: Use 10 ms as the time unit and the BIM model grid (1 m × 1 m × 0.1 m) as the spatial unit to construct a three-dimensional spatio-temporal correlation matrix, and the matrix dimension is (timestamp, X coordinate, Y coordinate, Z coordinate, data feature); (2) Data mapping and filling: Map the fiber optic grating strain sequence to the corresponding grid according to the sensor node coordinates, and calculate the strain gradient field Map the sonar point cloud deformation data (millimeter displacement) to the grid deformation field; (3) Correlation calculation: For the data points with both fiber optic strain gradient and sonar deformation amount in the same grid, calculate the Pearson correlation coefficient : , where is the strain gradient value, is the deformation amount; (4) Conflict marking rule: If the correlation coefficient , it is determined as a data conflict area, and mark the grid coordinates and time window.

[0025] S2 also includes: S24, Conflict Region Arbitration: Perform multimodal arbitration on the data conflict region, and generate an arbitrated spatio-temporal correlation matrix based on the synchronization of the ship impact waveform energy spectrum characteristics and the optical fiber strain time series. The specific steps are as follows: Extract the conflict spatio-temporal window: Extract the marked conflict grid coordinates and their corresponding time windows (±50 ms) from the spatio-temporal correlation matrix; Impact waveform energy analysis: Extract the ship berthing impact waveform within the conflict time window and calculate the peak moment of its energy spectral density ; Optical fiber strain mutation detection: Detect the moment of optical fiber strain gradient mutation within the same time window (Mutation threshold: ±200 μɛ / ms); Time deviation arbitration: If | , determine that the optical fiber data is valid and retain the original strain value; If | , call the sonar point cloud deformation amount If | , then correct the optical fiber strain value based on the sonar data; Generate arbitration result: Output the spatio-temporal correlation matrix after eliminating conflicts, and replace the conflict grid data with the arbitration value; S25, Generate a standardized data set: Normalize the data of the arbitrated spatio-temporal correlation matrix to generate a standardized data set with a unified timestamp and spatial coordinate system. The specific steps are as follows: (1) Unify physical quantity units: Optical fiber strain value: Convert to microstrain unit (με), retain two decimal places; Sonar deformation amount: Convert to millimeter displacement amount (mm), accuracy 0.1 mm; Tidal water level: Unify to the 1985 National Elevation Datum (meter), accuracy 0.001 m; Impact waveform energy spectrum: Convert to decibel value (dB) according to the 10 ( / ) formula, = 1 μJ; (2) Dimensionless normalization: Use the Z-score method to calculate the mean and standard deviation for each feature dimension, and normalize according to ; (3) Data set encapsulation: Encapsulate the data according to the spatio-temporal correlation matrix structure to generate a standardized data set.

[0026] S3 includes: S31. Construct the structural deformation gradient field: Extract the fiber Bragg grating strain sequence and underwater three-dimensional sonar point cloud data from the standardized dataset to generate the structural deformation gradient field. The specific steps are as follows: Extract fiber strain data: Parse the fiber Bragg grating strain sequence from the standardized dataset to obtain the strain values of each sensor node in the BIM model coordinate system; Extract sonar deformation amount: Parse the millimeter-level deformation amount data of the underwater three-dimensional sonar point cloud to obtain the deformation displacement amount at the same spatial position; Gradient field calculation: Use the spatial difference algorithm for the fiber strain data to calculate the strain gradient between adjacent sensor nodes; Combine the sonar deformation amount data, and perform weighted fusion (weight ratio 6:4) on the strain gradient and deformation amount within the same spatial grid to generate the structural deformation gradient field matrix; Output the structural deformation gradient field: Save the matrix to the feature database. The matrix dimension is (timestamp, X coordinate, Y coordinate, Z coordinate, strain gradient value); S32. Calculate the underwater erosion and deposition volume change rate: Based on the underwater three-dimensional sonar point cloud in the standardized dataset, generate the underwater erosion and deposition volume change rate. The specific steps are as follows: Extract sequential sonar point cloud: Extract the underwater three-dimensional sonar point cloud of consecutive time windows (≥3 tidal cycles) from the standardized dataset; Construct a digital elevation model: Perform Delaunay triangulation on the sonar point cloud at each time point to generate an underwater terrain digital elevation model (DEM) with millimeter-level accuracy; Volume change calculation: Calculate the volume difference between adjacent DEMs in chronological order. The formula is: , where is the elevation of the th grid point at the th moment, , is the grid resolution (0.1m × 0.1m); Change rate quantification: Divide the volume difference by the time interval (unit: hour) to generate the underwater erosion and deposition volume change rate matrix; Output the erosion and deposition volume change rate: Save the matrix to the feature database. The matrix dimension is (time window, X region, Y region, volume change rate).

[0027] S3 also includes: S33. Extract the load impact energy spectral density: Based on the ship berthing impact waveform in the standardized dataset, generate the load impact energy spectral density. The specific steps are as follows: Extract the impact waveform: Extract the decibel value (dB) data of the ship berthing impact waveform from the standardized dataset; Windowed Fourier Transform: Apply a Hanning window function to the impact waveform, perform a Fast Fourier Transform (FFT), and calculate the energy spectral density: ; where is the sampling point of the impact waveform, is the number of sampling points; Characteristic frequency band extraction: Identify the maximum energy density value (dB / Hz) in the 1 - 100 Hz frequency band of the energy spectrum as the load impact energy spectral density; Output energy spectral density: Save the matrix to the feature database, and the matrix dimension is (impact event number, peak energy density, characteristic frequency); S34, Construct a multi - dimensional fusion feature matrix: Integrate the structural deformation gradient field, the underwater scouring and silting volume change rate, and the load impact energy spectral density to generate a multi - dimensional fusion feature matrix. The specific steps are as follows: Time - axis alignment: Perform time interpolation alignment on the structural deformation gradient field matrix (at 10 - ms intervals) and the scouring and silting volume change rate matrix (hour - level) to generate a gradient - scouring and silting joint matrix with a unified time reference; Spatial grid matching: Map the load impact energy spectral density to the BIM model grid of the dock berthing area to establish an energy density - spatial position association table; Feature - level fusion: Use the tensor splicing method to fuse the gradient field, scouring and silting rate, and energy spectral density data along the feature dimension to generate a three - dimensional fusion tensor (time × space × feature); Normalization processing: Perform maximum - minimum normalization on each feature dimension of the tensor to compress the numerical range to the [0, 1] interval; Output feature matrix: Generate a multi - dimensional fusion feature matrix containing the structural deformation gradient field, the underwater scouring and silting volume change rate, and the load impact energy spectral density, and transmit it to S4.

[0028] S4 includes: S41, Extract real - time tidal phase parameters: Analyze the tidal water level time series from the standardized dataset to generate real - time tidal phase parameters, ensuring that the tidal state can dynamically affect the health assessment calculation. The specific steps are as follows: (1) Obtain tidal time - series data: Extract the tidal water level time series from the standardized dataset, and the data format is (timestamp, water level elevation, 1985 National Elevation Datum); (2) Phase state division: Divide the tidal cycle into three states: high - tide phase, low - tide phase, and transition phase according to the astronomical tide table and the real - time water level change rate; (3) Phase parameter quantization, including: High - tide phase: The water level is higher than the mean sea level + 1.0 m and the change rate ≤ 5 cm / min; Low tide phase: water level is lower than the mean sea level by -1.0 m and the change rate ≥ -5 cm / min; Transition phase: water level is between ±1.0 m or the change rate exceeds the above threshold; (4) Output tidal phase parameters: generate a real-time tidal phase parameter matrix including phase state, water level elevation, and change rate; S42. Obtain ship tonnage parameters: obtain the tonnage, length, and width parameters of the berthing ship from the port ship scheduling system in real time, and generate a ship tonnage feature vector to ensure that the health assessment calculation can consider the impact of ship impact. The specific steps are as follows: (1) AIS data parsing: obtain the registered tonnage (DWT), length, and width data of the berthing ship through the Automatic Identification System (AIS) of ships; (2) Tonnage grade classification: classify ships into four grades according to DWT, including: Small (DWT < 10,000 tons): weight base 0.6; Medium (10,000 tons ≤ DWT < 50,000 tons): weight base 0.8; Large (50,000 tons ≤ DWT < 100,000 tons): weight base 1.0; Ultra-large (DWT ≥ 100,000 tons): weight base 1.2; (3) Feature vector encapsulation: generate a ship tonnage feature vector including tonnage grade, length (m), width (m), and weight base value; S43. Determine the basic feature weight coefficient: based on the multi-dimensional fusion feature matrix, set the initial weights of the structural deformation gradient field, underwater scouring and silting volume change rate, and load impact energy spectral density to ensure reasonable allocation of the basic weights for the health assessment calculation. The specific steps are as follows: (1) Feature importance assessment: use historical monitoring data to train a random forest model and calculate the Gini importance coefficient of each feature for structural health assessment; (2) Initial weight allocation, including: Structural deformation gradient field: Gini coefficient normalized weight 0.45; Underwater scouring and silting volume change rate: Gini coefficient normalized weight 0.35; Load impact energy spectral density: Gini coefficient normalized weight 0.20; (3) Generate the basic weight coefficient: output the initial weight vector , expressed as: , where is the initial weight of the structural deformation gradient field, is the initial weight of the underwater scouring and silting volume change rate, is the initial weight of the load impact energy spectral density.

[0029] S4 also includes: S44, dynamic weight adjustment of tidal phase: Dynamically correct the initial weight coefficient according to real-time tidal phase parameters to generate a tidal correction weight, ensuring that the health assessment calculation adapts to the changes in the tidal environment. The specific steps are as follows: (1) Phase weight correction rules, including: High tide phase: Increase the weight of the underwater scouring and silting volume change rate (+30%), and decrease the weight of the load impact energy (-15%); Low tide phase: Increase the weight of the structural deformation gradient field (+20%), and decrease the scouring and silting weight (-10%); Transition phase: Maintain the basic weight; (2) Weight adjustment calculation, including: High tide phase correction: , ; Low tide phase correction: , ; (3) Output the tidal correction weight: Generate the adjusted weight vector , expressed as: , where is the weight of the structural deformation gradient field after tidal phase correction, is the weight of the underwater scouring and silting volume change rate after tidal phase correction, is the weight of the load impact energy spectral density after tidal phase correction; S45, dynamic weight adjustment of ship tonnage: Based on the ship tonnage feature vector, perform a secondary adjustment on the tidal correction weight to generate the final dynamic weight coefficient, ensuring that the health assessment calculation adapts to the impact of ship berthing. The specific steps are as follows: (1) Tonnage impact factor calculation: According to the ship tonnage weight reference value , calculate the load impulse energy weight adjustment coefficient; , where is the weight of the load impact energy spectral density after ship tonnage influence correction; (2) Weight normalization: Normalize the adjusted weight vector to ensure that the sum is 1, expressed as: ; (3) Output the final dynamic weight coefficient: Generate the final dynamic weight coefficient , expressed as: ; S46. Generate health assessment indicators: Based on the final dynamic weight coefficients and the multi-dimensional fusion feature matrix, calculate the environment-adaptive health assessment indicators to ensure accurate and quantifiable health status assessment. The specific steps are as follows: (1) Feature weighted fusion: Perform weighted summation on the multi-dimensional fusion feature matrix according to the weight vector, expressed as: ; where, is the health assessment indicator, , , are the structural deformation gradient field, the underwater scouring and silting volume change rate, and the load impact energy spectrum density eigenvalue respectively; (2) Index normalization: Map the health score to the 0 - 100 interval, where 100 represents the structure in good condition and 0 represents serious damage; (3) Output health assessment indicators: Generate an environment-adaptive health assessment indicator matrix containing the timestamp, spatial location, and health score.

[0030] S5 includes: S51. Construct an abnormal propagation model: Based on the health assessment indicators, construct a graph neural network abnormal propagation model and define the mechanical transfer relationship between the model nodes and edges. The specific steps are as follows: (1) Node definition: Divide the wharf structure into N spatial partitions according to the BIM model, and each partition is used as a node of the graph neural network. The node features include the health assessment indicators of this partition, the attenuation rate of material elastic modulus, and the crack propagation trend coefficient; (2) Edge weight calculation: Calculate the edge weights between adjacent partitions according to the mechanical transfer path of the wharf structure: , where, is the elastic modulus of partition , is the connection surface area between partition and , is the spatial distance between partition and ; (3) Model training: Use historical abnormal data to train the graph neural network, optimize the weight parameters of the propagation path, and generate an abnormal propagation model; S52. Calculate the spatio-temporal correlation degree: Input the real-time health assessment indicators into the abnormal propagation model, calculate the abnormal spatio-temporal correlation degree, and generate a spatio-temporal correlation degree matrix. The specific steps are as follows: Data preprocessing: Extract the health score data of the current time window (±10 minutes) from the health assessment indicator matrix and map it to the nodes of the graph neural network according to the spatial partitions; Abnormal propagation deduction: Calculate the propagation intensity of abnormal energy between adjacent nodes through a graph neural network, and generate a spatio-temporal propagation heat map; Correlation quantification: Calculate the correlation coefficient between the abnormal energy of each node in the heat map and the initial abnormal source. When the correlation coefficient ≥ 0.65, it is determined as a strongly correlated area, and a spatio-temporal correlation matrix is generated; S53, Early warning threshold determination: Compare the spatio-temporal correlation matrix with the preset early warning threshold to identify the list of exceeded standard areas. The specific steps are as follows: Threshold classification setting: Level 1 early warning: Spatio-temporal correlation ≥ 0.8 and the covered area ≤ 3 partitions; Level 2 early warning: Spatio-temporal correlation ≥ 0.8 and the covered area is 4 - 6 partitions; Level 3 early warning: Spatio-temporal correlation ≥ 0.8 and the covered area ≥ 7 partitions; Area marking: Conduct spatial clustering analysis on the exceeded standard areas, and merge adjacent associated partitions to form early warning blocks; Generate a list of exceeded standard areas: Output an exceeded standard area matrix containing the early warning level, spatial coordinates, and correlation values; S54, Generate hierarchical early warning signals: Generate hierarchical early warning signals according to the list of exceeded standard areas, and trigger the corresponding response mechanism. The specific steps are as follows: Response rules: Level 1 early warning: Trigger a local sensor calibration instruction; Level 2 early warning: Trigger a drone inspection and review instruction; Level 3 early warning: Push a structure reinforcement instruction to the operation and maintenance system; Early warning signal encapsulation: Encapsulate the early warning level, spatial coordinates, and timestamp into a standardized early warning signal data packet; Real-time push: Transmit the early warning signal to the port monitoring center and mobile terminals through a 5G communication module; S55, Execute hierarchical response instructions: Start the corresponding emergency response process according to the early warning signal level. The specific steps are as follows: Level 1 response: Call the multi-modal monitoring data packet of S16 to perform online calibration and data re-acquisition on the sensors in the early warning area; Level 2 response: Start the underwater three-dimensional sonar point cloud review scan of S13, and perform multi-source data verification in combination with the standardized data set; Level 3 response: Retrieve the BIM model structure parameter library to generate a structure reinforcement instruction set including reinforcement plans, material lists, and construction drawings.

[0031] The present invention encompasses any alternatives, modifications, equivalent methods, and solutions made to the essence and scope of the present invention. For the public to have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, and those skilled in the art can fully understand the present invention without the description of these details. Additionally, to avoid unnecessary confusion to the essence of the present invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0032] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. An intelligent monitoring method for multimodal data fusion of water transport infrastructure, characterized in that: The following steps are involved: S1, real-time acquisition of multi-modal monitoring data of the wharf structure, including fiber Bragg grating strain series, underwater 3D sonar point cloud, tidal water level time series, and ship berthing impact waveform; S2, performing spatiotemporal alignment processing on the multimodal monitoring data to generate a standardized data set with a unified timestamp and spatial coordinate system; S3, constructing a multi-dimensional fusion feature matrix based on the standardized data set, wherein the feature matrix includes the structural deformation gradient field, the underwater scouring and silting volume change rate, and the load impact energy spectrum density; S4, dynamically adjusts the characteristic matrix weight coefficient according to the real-time tidal phase and ship tonnage parameters to generate environmentally adaptive health assessment indicators; S5, calculates the spatiotemporal correlation of health assessment indicators through the anomaly propagation model, and generates a graded warning signal when the correlation exceeds the warning threshold.

2. The intelligent monitoring method for multimodal data fusion of water transport infrastructure according to claim 1 is characterized in that: The S1 includes: S11, generating a fiber Bragg grating strain sequence: deploying a fiber Bragg grating sensor array along the axis of the wharf pile foundation, and continuously collecting original strain vibration waveforms through the fiber Bragg grating sensor array; Performing wavelet transform noise reduction processing on the original strain vibration waveform to generate noise-reduced optical fiber strain data; Performing spatiotemporal interpolation calculation on the noise-reduced optical fiber strain data, compensating for sensor blind area data, and generating an optical fiber Bragg grating strain sequence; S12, detecting coordinates of abnormal amplitude regions: performing amplitude threshold analysis on the fiber Bragg grating strain sequence, setting a dynamic threshold range of ±500 με, and extracting abnormal strain data segments exceeding the dynamic threshold range in the fiber Bragg grating strain sequence; According to the spatial position mapping relationship of the fiber grating sensor array, the sensor node coordinates corresponding to the abnormal strain data segment are converted into the three-dimensional spatial coordinates of the wharf structure to generate the abnormal amplitude region coordinates; S13, generating an underwater three-dimensional sonar point cloud: based on the coordinates of the abnormal amplitude area, controlling the unmanned survey ship to carry a multi-beam sonar to scan the target area to obtain an original reflection signal; The original reflection signal is compensated for the incident angle to generate an angle-corrected sonar signal; the angle-corrected sonar signal is subjected to seabed reverberation suppression processing, and an adaptive filtering algorithm is used to separate effective reflection waves to generate a sonar reflection intensity matrix; Performing a three-dimensional coordinate transformation on the sonar reflection intensity matrix to map the reflection intensity to a Cartesian coordinate system; The iterative closest point algorithm is used to stitch the coordinate data of the continuous scanning frames; Perform point cloud downsampling processing on the spliced ​​3D coordinate data to generate underwater 3D sonar point cloud; S14, generating a tidal water level time series: deploying 4 sets of radar water level gauges at the front of the wharf to synchronously collect the original tidal elevation values; The original tidal elevation value is differentially calculated with the astronomical tide table to generate a calibrated tidal elevation value; The calibrated tide elevation values ​​are timestamped by a GNSS time synchronization module to generate a time-synchronized tide level time series; S15, generating a ship berthing impact waveform: installing a triaxial acceleration sensor group at the root of the pier bollard and the central axis of the fender, and synchronously acquiring the tidal water level time series as an environmental phase reference; When the acceleration amplitude is detected to exceed the set threshold of 5g, the high-speed acquisition mode with a sampling interval of 0.5ms is triggered to capture the original impact vibration waveform of the ship at the moment of contact; Performing phase synchronization processing on the original impact vibration waveform, combining the real-time water level phase of the tidal water level time series, to generate a ship berthing impact waveform; S16, constructing a multimodal monitoring data packet: encapsulating the fiber grating strain sequence, underwater three-dimensional sonar point cloud, tidal water level time series, and ship berthing impact waveform into a multimodal monitoring data packet according to the original acquisition timing, and the multimodal monitoring data packet retains the original timestamp and spatial coordinate information of each sensor.

3. The intelligent monitoring method for multimodal data fusion of water transport infrastructure according to claim 2 is characterized in that: The S2 includes: S21, timestamp synchronization calibration: extract the original timestamp of each sensor from the multimodal monitoring data packet, generate a unified reference time axis through GNSS second pulse signal calibration, and perform time interpolation resampling on the data stream to output a time-synchronized multimodal data stream; S22, spatial coordinate system 1: Based on the wharf BIM model, the output time-synchronized multimodal data stream is spatially registered, including sonar point cloud rigid registration, fiber coordinate mapping and tidal elevation conversion, and the multimodal data aligned in time and space are output; S23, spatiotemporal correlation analysis: Map the spatiotemporal alignment data output by S22 to the spatiotemporal correlation matrix, analyze the spatial correlation between optical fiber strain and sonar deformation, and mark data conflict areas.

4. The intelligent monitoring method for multimodal data fusion of water transport infrastructure according to claim 3 is characterized in that: The S2 further includes: S24, conflict area arbitration: multi-modal arbitration is performed on the data conflict area, and the time-space correlation matrix after arbitration is generated based on the synchronization of the energy spectrum characteristics of the ship impact waveform and the optical fiber strain time series; S25, generating a standardized data set: performing data normalization on the arbitrated spatiotemporal correlation matrix to generate a standardized data set with a unified timestamp and spatial coordinate system.

5. The intelligent monitoring method for multimodal data fusion of water transport infrastructure according to claim 4 is characterized in that: The S3 includes: S31, constructing structural deformation gradient field: extracting fiber Bragg grating strain sequence and underwater 3D sonar point cloud data from standardized data sets to generate structural deformation gradient field; S32, calculating underwater scouring and silting volume change rate: generating underwater scouring and silting volume change rate based on the underwater three-dimensional sonar point cloud in the standardized data set.

6. The intelligent monitoring method for multimodal data fusion of water transport infrastructure according to claim 5 is characterized in that: The S3 further includes: S33, extracting load impact energy spectrum density: generating load impact energy spectrum density based on the ship berthing impact waveform in the standardized data set; S34, constructing a multi-dimensional fusion feature matrix: integrating the structural deformation gradient field, underwater scouring and silting volume change rate, and load impact energy spectrum density to generate a multi-dimensional fusion feature matrix.

7. The intelligent monitoring method for multimodal data fusion of water transport infrastructure according to claim 6 is characterized in that: The S4 includes: S41, extracting real-time tidal phase parameters: parsing the tidal level time series from the standardized data set to generate real-time tidal phase parameters; S42, obtaining ship tonnage parameters: obtaining the tonnage, length and width parameters of the berthed ship from the port ship dispatching system in real time, and generating a ship tonnage feature vector; S43, determine the basic feature weight coefficient: based on the multi-dimensional fusion feature matrix, set the initial weights of the structural deformation gradient field, the underwater scouring and silting volume change rate, and the load impact energy spectrum density.

8. The intelligent monitoring method for multimodal data fusion of water transport infrastructure according to claim 7 is characterized in that: The S4 further comprises: S44, tidal phase dynamic weight adjustment: dynamically correct the initial weight coefficient according to the real-time tidal phase parameter to generate a tidal correction weight; S45, ship tonnage dynamic weight adjustment: based on the ship tonnage feature vector, the tidal correction weight is adjusted twice to generate the final dynamic weight coefficient; S46, generate health assessment indicators: calculate the environment-adaptive health assessment indicators based on the final dynamic weight coefficient and the multi-dimensional fusion feature matrix.

9. The intelligent monitoring method for multimodal data fusion of water transport infrastructure according to claim 8 is characterized in that: The S5 includes: S51, build an abnormal propagation model: build a graph neural network abnormal propagation model based on health assessment indicators, and define the mechanical transmission relationship between model nodes and edges; S52, calculating the spatiotemporal correlation: inputting the real-time health assessment index into the abnormal propagation model, calculating the abnormal spatiotemporal correlation, and generating a spatiotemporal correlation matrix; S53, early warning threshold determination: compare the spatiotemporal correlation matrix with the preset early warning threshold to identify the list of areas exceeding the standard; S54, generating a graded warning signal: generating a graded warning signal according to the list of areas exceeding the standard, and triggering a corresponding response mechanism; S55, execute the graded response instruction: start the corresponding emergency handling process according to the warning signal level.

Images