A wharf structure monitoring digital twin multi-source heterogeneous data processing and alignment synchronization method and system

Abstract

The present application relates to the technical field of digital twinning and intelligent port, in particular to a wharf structure monitoring digital twinning multi-source heterogeneous data processing and alignment synchronization method and system. Firstly, a data access layer based on a double protocol architecture is constructed, a three-level hierarchical topic routing mechanism is adopted, efficient classification and convergence of structured monitoring data and video stream and reliable transmission are realized; secondly, a two-stage data cleaning mechanism is proposed, abnormal data is accurately removed by using the physical association characteristics of sensors; finally, by constructing a monitoring data dynamic sliding window, based on the piecewise cubic hermite interpolation algorithm, the window slope and the interpolation polynomial are calculated by using the central difference weighted average, the monitoring data sequences of different sampling frequencies are accurately mapped to the standard time axis, and the time sequence consistent fusion data set is generated. The present application effectively improves the integrity, accuracy and time sequence consistency of the wharf structure monitoring data, and provides high-quality data support for real-time safety evaluation of the digital twinning system.

Description

Technical Field

[0001] This invention relates to the field of digital twin and smart port technology, specifically to a method and system for processing and aligning heterogeneous data from multiple sources for monitoring terminal structures using digital twins. Background Technology

[0002] The digital twin for wharf structure monitoring relies on technologies such as sensors, the Internet of Things (IoT), and Building Information Modeling (BIM) to map the service status of the wharf structure into a digital space in real time, enabling visualization of structural status, safety early warning, and health assessment. Its stable operation depends on the high-quality supply and accurate time-series alignment of multi-source heterogeneous monitoring data. Wharf monitoring data includes structured sensor data such as strain and vibration, as well as unstructured data such as video images. It is characterized by dispersed sources, a wide range of acquisition frequencies, and inconsistent transmission protocols, posing significant technical challenges to efficient data processing and high-fidelity synchronization.

[0003] Existing technologies have significant shortcomings in processing multi-source heterogeneous data. On the one hand, data access lacks a standardized mechanism and a hierarchical routing strategy adapted to the dock environment. Data formats vary greatly among different manufacturers and types of equipment, and structured data is coupled with unstructured video streams during transmission, resulting in low data aggregation efficiency and difficulties in parsing and adaptation. On the other hand, the harsh environment of docks easily causes abnormal data jumps and drifts. Traditional cleaning methods rely solely on general statistical algorithms and do not integrate structural physical correlations and mechanical coupling characteristics, making it difficult to distinguish between environmental noise, sensor malfunctions, and actual structural responses. This leads to a high false positive rate in anomaly detection and insufficient data reliability.

[0004] In the timing alignment and synchronization stage, the dynamic analysis of wharf structures has stringent requirements for the consistency of timing of multi-source data, but existing solutions generally suffer from insufficient adaptability. Conventional linear interpolation methods are prone to producing spurious extrema and timing distortion when processing cross-frequency and non-stationary data; Dynamic Time Warping (DTW) algorithms have high computational overhead and are difficult to meet the real-time requirements of digital twins. At the same time, existing methods only use the device's local timestamp for clock calibration, failing to address the timing misalignment problem caused by asynchronous sampling. The interpolation results are prone to exceeding the physically interpretable range, affecting the accuracy of dynamic analysis such as modal parameter identification and load inversion.

[0005] In existing publicly available technologies, such as Chinese patent application CN1121771365A, a method for processing engineering structure monitoring data is disclosed. Although it achieves multi-source data access and preliminary cleaning, it uses a general protocol for transmission and a single statistical threshold for cleaning. It does not design a hierarchical routing and structural correlation verification mechanism specifically for the dock scenario, and time-series alignment still relies on traditional interpolation, which cannot avoid distortion problems. In summary, existing technologies cannot provide standardized, high-quality, and high-fidelity time-series consistent data support for digital twins of dock structure monitoring. Summary of the Invention

[0006] The purpose of this invention is to propose a method and system for processing, aligning and synchronizing multi-source heterogeneous data in digital twins for monitoring wharf structures. By standardizing the access and efficient routing of multi-source heterogeneous data, and combining anomaly detection and cleaning mechanisms based on structural physical characteristics, high-fidelity time-series alignment of multi-sampling frequency data is achieved, thereby meeting the requirements of wharf structure dynamic analysis for time-series consistency and high-precision data.

[0007] According to a first aspect of the present disclosure, a method for processing and aligning heterogeneous multi-source data of a digital twin for monitoring a wharf structure is provided, comprising the following steps: The system adopts a dual-protocol architecture of "MQTT+RTSP" for unified access and aggregation of multi-source heterogeneous data. It uses a three-level hierarchical MQTT topic routing mechanism of "region-structure-sensor type" to classify structured sensor data and standardizes RTSP access for unstructured video streams. At the same time, it configures differentiated QoS levels and disconnection retransmission mechanisms according to the importance of data to generate standardized data streams to be processed. The standardized data stream is subjected to a two-stage cleaning process, which includes: obtaining the mean and standard deviation of single sensor data using a sliding time window, marking suspected anomalies according to the 3σ criterion; dividing sensors with physical location associations into association groups, constructing a trend association model based on the majority consensus principle, and verifying the changing trend of the suspected anomalies to distinguish between real structural anomalies and sensor fault data. The cleaned data is time-series aligned and synchronized to determine the standard time axis and alignment point of the digital twin system. A dynamic sliding window for monitoring data is constructed, and the window slope is obtained by using the central difference weighted average based on the piecewise cubic Hermit interpolation PCHIP algorithm. Monitoring data sequences with different sampling frequencies are mapped to the standard time axis to generate a time-series consistent fusion dataset.

[0008] In one embodiment, a three-tiered MQTT topic routing mechanism based on "region-structure-sensor type" is used to classify structured sensor data and to standardize RTSP access for unstructured video streams, specifically including: Obtain device metadata and generate a globally unique identifier; among them, the unique identifier of the sensor follows the naming rule of "sensor type_structure ID_serial number", and the unique identifier of the camera follows the naming rule of "camera_structure ID_serial number". The hierarchical MQTT topic format is used to build ,in Corresponding area code, The corresponding unique identifier of the structure Corresponding sensor type; To construct a standardized RTSP access address format By locating the camera deployment position through the address hierarchy, the region and structure identifiers of the video stream address are kept consistent with the MQTT topic.

[0009] In one embodiment, configuring differentiated QoS levels and retransmission mechanisms based on data importance specifically includes the following steps: For sensor data transmitted via the MQTT protocol, differentiated QoS levels are configured based on the data value: critical monitoring data involving structural safety are configured as QoS level 2 to ensure data transmission without duplication or loss; ordinary environmental data such as temperature and humidity are configured as QoS level 1 to balance transmission reliability and efficiency. The sensor data is encapsulated using a unified field standard. The encapsulation format includes at least the following: a millisecond-level Unix timestamp collect_ts, a unique sensor ID sensor_id following the rule of "type_structure_serial number", and a monitoring value data_value matching the corresponding physical unit. For real-time video streams transmitted via the RTSP protocol, an "RTP+RTCP" transmission system is constructed, in which RTP is responsible for real-time transmission of video frames and RTCP is responsible for transmission quality feedback; a packet loss retransmission threshold and a dynamic bitrate adjustment range are configured. When the packet loss rate exceeds the set threshold, retransmission is triggered, and the video bitrate is dynamically adjusted within the range of 2Mbps to 8Mbps according to the network bandwidth. Establish a transmission exception handling mechanism: when a connection is detected to be broken, start an MQTT persistent session and use the exponential backoff optimization algorithm to reconnect; After a successful reconnection, the continuity of the collect_ts timestamps is checked to determine if there is any data loss. If the time difference between adjacent data is greater than 1.5 times the sampling period, it is determined to be a packet loss, and a data traceability request carrying the missing data sensor_id and timestamp range is sent to the Broker. The Broker sends a retransmission command to the sensor based on the locally retained message logs, and the sensor retransmits the data according to the specified timestamp range and the original QoS level.

[0010] In one embodiment, sensors associated with physical locations are divided into association groups, and a trend association model based on the majority consensus principle is constructed, specifically including: Based on the physical topology and functional coupling relationship of the wharf structure, various preset sensor association group types are defined; The pile foundation monitoring group includes strain sensors deployed on the pile body, tilt sensors deployed on the pile top, and a vertical level deployed near the pile cap, used to verify the consistency between the tilt angle change when the pile foundation tilts and the settlement trend detected by the level. The beam monitoring group includes strain sensors deployed in the middle of the span and tilt sensors deployed in the same row of frames, used to verify the consistency between the increase in strain and the change in tilt angle in the same direction when the beam bends. The panel monitoring group includes strain sensors deployed in the middle of the panel span and vertical level instruments deployed at the panel supports, used to verify the consistency between the changes in panel bending strain and the settlement trend at the supports; The environmental correlation group includes temperature sensors, humidity sensors, and strain sensors in the same area, used for data verification based on the strain thermal effect caused by temperature changes.

[0011] In one embodiment, verifying the changing trend of the suspected anomaly points specifically involves: Divide the sliding time window W into multiple continuous sub-intervals, and then, for the associated groups... Each sensor inside The average slope of each sub-interval is obtained, and the trend of change is quantified into a symbolic sequence based on the sign of the slope. Positive values ​​indicate growth, negative values ​​indicate decrease, and values ​​close to zero indicate stability. Number of subintervals Indicates the average slope; For each sub-interval The symbolic slope distribution of all sensors within the correlation group in this sub-interval is statistically analyzed, and the mode is taken as the mainstream direction of this sub-interval. ,in Symbolic function; integrate the main directions of all subintervals to obtain the association group. Mainstream trend sequence ; Locating suspected abnormal data points The position of the sub-interval Extract its corresponding sensor Trend direction in this sub-interval ; Direction of the trend The mainstream direction of the corresponding sub-interval in the mainstream trend sequence Compare: If and If they are consistent, then the data point is determined to be a structural anomaly reflecting the true structural response; if... and If there is a discrepancy, the data point is determined to be sensor fault data.

[0012] In one embodiment, the dynamic sliding window for monitoring data is constructed as follows: Based on the accuracy requirements of structural dynamic analysis and the sensor sampling frequency, a data dynamic update frequency is set, and the data update time interval Δt is obtained; based on the current standard clock. Get the next data alignment standard timestamp ; For each monitoring data sequence, a dynamic sliding window containing four data points is constructed. ,in The timestamps of the monitoring data are arranged in ascending order of time. The corresponding monitoring data value; When constructing the sliding window, constrain the next data to align with the standard timestamp. The second data point timestamp within the window With the third data point timestamp Between, that is, satisfying and followed The monitoring data is updated dynamically within the push window.

[0013] In one embodiment, based on the piecewise cubic Hermitian interpolation PCHIP algorithm, the window slope is obtained by using the central difference weighted average, and the monitoring data sequences with different sampling frequencies are mapped to the standard time axis to generate a time-consistent fused dataset. The specific steps include: Dynamic sliding window for each monitoring data sequence [ Get the corresponding slope window [ ,in and Sliding windows and The slope of the monitoring data at the corresponding time point is calculated using the central difference weighted average method; The next data alignment standard timestamp is obtained using the cubic Hermit interpolation algorithm. Corresponding alignment data value ,as follows: in, For normalization parameters, , , , The basis functions are: The obtained alignment data values As this monitoring data sequence on the standard time axis The mapping values ​​at each time point are used to generate a time-consistent fused dataset by summing the mapping values ​​of each sequence.

[0014] According to a second aspect of the present disclosure, a digital twin multi-source heterogeneous data processing and alignment synchronization system for wharf structure monitoring is provided, comprising: The data access module adopts a dual-protocol architecture of "MQTT+RTSP" for unified access and aggregation of multi-source heterogeneous data. It uses a three-level hierarchical MQTT topic routing mechanism of "region-structure-sensor type" to classify structured sensor data and standardizes RTSP access for unstructured video streams. At the same time, it configures differentiated QoS levels and disconnection retransmission mechanisms according to the importance of data to generate standardized data streams to be processed. The data cleaning module performs two-stage cleaning processing on the standardized data stream, specifically including: obtaining the mean and standard deviation of single sensor data using a sliding time window, marking suspected anomalies according to the 3σ criterion; dividing sensors with physical location associations into association groups, constructing a trend association model based on the majority consensus principle, and verifying the changing trend of the suspected anomalies to distinguish between real structural anomalies and sensor fault data. The data alignment and synchronization module performs time-series alignment and synchronization on the cleaned data, determines the standard time axis and alignment point of the digital twin system, constructs a dynamic sliding window for monitoring data, and uses the PCHIP algorithm of piecewise cubic Hermit interpolation to obtain the window slope by using the central difference weighted average, mapping monitoring data sequences with different sampling frequencies to the standard time axis, and generating a time-consistent fused dataset.

[0015] According to a third aspect of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and running on the memory, wherein the processor executes the program to implement the aforementioned method for processing and aligning heterogeneous data from a digital twin for monitoring a wharf structure.

[0016] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the aforementioned method for processing and aligning heterogeneous multi-source data of a digital twin for monitoring a wharf structure.

[0017] The advantages of the above technical solutions adopted in this invention compared with the prior art are as follows: First, at the data access level, this invention adopts a dual-protocol architecture of "MQTT+RTSP" and a three-level hierarchical topic routing mechanism, using unique device identifiers to achieve the classification and efficient aggregation of structured data and video streams. This not only unifies the data encapsulation format but also significantly improves the reliability and integrity of data transmission through differentiated QoS levels, persistent sessions, and reconnection mechanisms, providing a stable, standardized, and traceable data foundation for subsequent analysis.

[0018] Secondly, at the data cleaning level, the "sliding window 3σ initial screening + correlation group trend verification" two-stage mechanism proposed in this invention effectively overcomes the high misjudgment rate of the single threshold method. By introducing trend consistency analysis of correlation sensor groups, it is possible to distinguish between real structural anomalies and sensor malfunctions, significantly improving the accuracy of anomaly identification and realizing intelligent early warning and cleaning of abnormal data, thus ensuring data quality.

[0019] Finally, at the temporal alignment level, based on the cubic Hermitian interpolation PCHIP algorithm, combined with dynamic sliding window and central difference weighted average slope calculation, high-precision mapping of monitoring data with different sampling frequencies on the standard time axis was achieved. This effectively eliminated temporal misalignment of multi-source data, generated a temporally consistent fused dataset, and provided high-quality data support for real-time structural analysis and security assessment of digital twin systems. Attached Figure Description

[0020] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an undue limitation of this application.

[0021] Figure 1 Flowchart for processing multi-source heterogeneous data from digital twins for wharf structure monitoring; Figure 2 Schematic diagram of a digital twin multi-source heterogeneous data processing and alignment synchronization system for wharf structure monitoring. Detailed Implementation

[0022] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0023] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0024] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0025] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and systems according to various embodiments of this disclosure. It should be noted that each block in a flowchart or block diagram may represent a module, segment, or portion of code, which may include one or more executable instructions for implementing the logical functions specified in the various embodiments. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutively represented blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, may be implemented using a dedicated hardware-based system that performs the specified functions or operations, or using a combination of dedicated hardware and computer instructions.

[0026] In the scenario of monitoring dock structures, there are n different types of sensors. Each sensor uses a different sampling frequency. Collect data, data format is The problem of multi-source heterogeneous data processing and alignment synchronization in a digital twin system for wharf structure monitoring can be described as follows: Data access and routing: The sources of multi-source data are scattered and the types vary greatly (including structured sensor data such as strain and vibration, as well as unstructured data such as video). The lack of standardized access interfaces and accurate classification and routing mechanisms leads to low data aggregation efficiency, high difficulty in parsing and adaptation, and an inability to provide a unified input basis for subsequent processing. Data synchronization: The response of the wharf structure is temporally correlated with the load and environmental excitation. It is necessary to eliminate the clock deviation between different sensors, solve the problem of time misalignment of multi-sampling frequency data, achieve time alignment, and meet the time consistency requirements of structural dynamics analysis. Data cleaning: The dockside operating environment has unique noise sources such as structural vibration, electromagnetic interference, and salt spray corrosion, which can easily cause data distortion and data anomalies due to sensor malfunctions. Traditional methods rely solely on a single statistical threshold to mark anomalies without considering the physical characteristics of the associated structures, making it difficult to distinguish between the true structural response and interference signals, and thus failing to provide high-quality data support for subsequent analysis.

[0027] This invention summarizes the above problems into three key tasks: standardized access to multi-source heterogeneous data, high-quality cleaning, and high-fidelity time-series alignment. It solves these problems in an integrated manner by designing an "MQTT+RTSP" dual-protocol access architecture, a "two-stage cleaning mechanism," and a "PCHIP dynamic window alignment method," thereby providing a highly reliable and consistent data foundation for the port digital twin system.

[0028] Example 1: This embodiment provides a method for processing and aligning heterogeneous multi-source data from a digital twin for monitoring wharf structures, including the following steps: S1. The "MQTT+RTSP" dual-protocol architecture is adopted for unified access and aggregation of multi-source heterogeneous data. The three-level hierarchical MQTT topic routing mechanism of "region-structure-sensor type" is used to classify structured sensor data and standardize the RTSP access of unstructured video streams. At the same time, differentiated QoS levels and disconnection retransmission mechanisms are configured according to the importance of data to generate standardized data streams to be processed. To achieve efficient aggregation and standardized access of multi-source heterogeneous data at the port, and to provide a unified data input foundation for subsequent time synchronization and data cleaning in this invention, this invention constructs a "MQTT+RTSP" dual-protocol architecture: using the MQTT protocol to transmit sensor monitoring data and the RTSP protocol to transmit real-time monitoring video streams. At the same time, it designs a hierarchical topic routing mechanism, standardized data field specifications, new equipment access procedures, data / video disconnection and loss solutions, and transmission frequency / bitrate requirements to achieve decoupled transmission, accurate adaptation, and stable transmission of heterogeneous data and video streams.

[0029] This interface is built on the dual protocols of "MQTT+RTSP" and integrates the design of MQTT topic routing, RTSP video stream configuration, data specifications, and anomaly protection into the entire process of new device (sensor / camera) access.

[0030] Step S1.1: Interface communication configuration. The core of this step is to complete the basic adaptation of the sensor / camera and system interface, clarify the "communication rules" for data transmission, including key interface parameters such as topic routing and QoS level, and establish a basic channel for data transmission.

[0031] 1) Input device metadata and generate a globally unique identifier. Sensors follow the rule of "sensor type_structure ID_serial number" (e.g., "strain_pile005_001"), and cameras follow the rule of "camera_structure ID_serial number" (e.g., "camera_berth03_002"). Simultaneously, complete information such as deployment location and monitoring / surveillance range, and finalize the registration to achieve unique device identification. After information input, match the device identifier with model components to achieve the goal of unique device identification and model association.

[0032] 2) Differentiated transmission paths are designed for different device types to ensure accurate transmission of data and video streams. Sensor data adopts a three-level MQTT topic structure of "region-structure-sensor type," with the following format: The system generates unique topics based on the sensor's installation location and type. The number of topics subscribed to by the digital twin system client is limited to 10 to reduce resource consumption and communication overhead. This can be achieved by using topic wildcards (such as " / region / berth03 / structure / #") or device group associations, allowing a single topic to cover multiple types of sensor data within the same region / structure, avoiding the need to subscribe to individual sensor topics one by one. Real-time video streaming uses a standardized access address based on the RTSP protocol, with the following format: The camera deployment location is determined by address level, and the area / structure identification is kept consistent with the MQTT topic.

[0033] Differentiated protection mechanisms are configured based on the characteristics of the transmitted content. Sensor data is assigned a QoS level for MQTT based on its value: critical data such as structural security is set to QoS2 (exactly once) to ensure no duplication or loss; ordinary data such as temperature and humidity are set to QoS1 (at least once) to balance reliability and efficiency, and the interface automatically matches subscription / publishing parameters; real-time video streams are constructed based on the RTSP protocol to build an "RTP+RTCP" transmission system: RTP is responsible for real-time transmission of video frames, and RTCP is responsible for transmission quality feedback. A packet loss retransmission threshold is configured (retransmission is triggered when the packet loss rate is >5%), and a dynamic bitrate adjustment range (e.g., 2Mbps~8Mbps) is set to adapt to different bandwidth scenarios, as shown in Table 1.

[0034] Table 1 Transmission Strategy Configuration Table By employing differentiated transmission strategies, the integrity requirements of sensor data and the real-time requirements of video streams are met respectively, ensuring that the dual-protocol architecture is adaptable to various business scenarios.

[0035] S1.2: Sensor Data Format Adaptation. This step requires completing sensor data encapsulation and camera video stream parameter configuration according to interface standardization specifications to ensure that both structured data and unstructured video streams can be parsed and utilized. The interface establishes a unified field standard for sensor data; new sensors need to have all fields added before transmission via MQTT.

[0036] Table 2 shows the packaging format as follows. For the new sensor monitoring dimensions, specific rules are entered into the system: the data precision of data_value is specified (1-2 decimal places are retained) to facilitate subsequent data cleaning; to ensure data transmission efficiency and system load balance, basic transmission frequency requirements are set according to sensor type as the core parameters for data encapsulation and transmission. Key monitoring sensors (strain, vibration) need to ensure high-frequency acquisition, while environmental monitoring sensors can use low-frequency transmission. The basic frequency range and default value need to be configured in advance in the system and can be dynamically adjusted according to the scenario.

[0037] The camera needs to be configured with RTSP transmission core parameters according to the interface requirements to ensure that the video stream can be accessed by the system in a standardized manner; Table 3 shows the camera parameter specifications as follows. Step S1.3: Transmission anomaly handling mechanism. For possible disconnection anomalies during transmission, the MQTT "persistent session" function is enabled. After disconnection, the digital twin client subscription relationship and unacknowledged messages of QoS1 / 2 level are automatically retained. The reconnection mechanism adopts "exponential backoff optimization": the first disconnection triggers reconnection after 3 seconds. If it fails, it will retry at intervals of 5 seconds and 8 seconds (maximum 30 seconds). After successful reconnection, the unacknowledged data before the disconnection is immediately retransmitted through the QoS retransmission mechanism (QoS1 ensures at least once, QoS2 ensures exactly once).

[0038] To address potential data loss during transmission, the receiving end verifies the continuity of timestamps using `collect_ts` (an alarm is triggered if the time difference between adjacent data is greater than 1.5 times the sampling period). Upon triggering a packet loss alarm, the receiving end immediately sends a "data traceability request" (carrying the sensor_id and timestamp range of the missing data) to the Broker. Based on the locally stored QoS 1 / 2 level message logs, the Broker issues a retransmission command to the sensor. The sensor retransmits the data according to the specified timestamp range, and the retransmitted data still uses the original QoS level to ensure reliability. After the retransmission is completed, the system verifies the timestamp continuity to ensure data integrity.

[0039] S2. Perform a two-stage cleaning process on the standardized data stream, specifically including: using a sliding time window to obtain the mean and standard deviation of single sensor data, marking suspected anomalies according to the 3σ criterion; dividing sensors with physical location associations into association groups, constructing a trend association model based on the majority consensus principle, and verifying the changing trend of the suspected anomalies to distinguish between real structural anomalies and sensor fault data. like Figure 1 As shown, the data cleaning algorithm of this invention employs a two-stage anomaly detection mechanism: the first stage is based on a sliding window... The criteria mark suspected abnormal data; in the second stage, the numerical trend correlation model of the associated sensor group is combined to perform physical consistency verification on the suspected abnormal data, thereby distinguishing between real structural anomalies and sensor noise or faults.

[0040] S2.1: Anomaly detection based on the 3σ criterion using a sliding time window; For each sensor data stream, a time window W is set, the size of which is determined based on the fluctuation period of the different sensor data. Within the window, the mean μ and standard deviation σ of the data are calculated. For each data point within the window If the following conditions are met: The data point is then marked as a suspected outlier and proceeds to the second stage of verification.

[0041] S2.2: Trend verification of associated sensors based on the majority consensus principle; Sensors deployed in the same structural part or functionally related area are grouped together; Pile foundation monitoring team: Strain sensor (pile body) + Inclination sensor (pile top) + Vertical level (adjacent pile cap) Crossbeam monitoring group: Strain sensor (mid-span) + Tilt sensor (same row of supports) Panel monitoring group: strain sensor (mid-span of panel) + vertical level (panel support). Environmental correlation group: Temperature sensor + Strain sensor (same area) + Humidity sensor Table 4 Sensor Trend Verification Table The sensors within each associated group are physically related, and their monitoring data should show a consistent trend under normal operating conditions.

[0042] For suspected abnormal data points The sensor to which it belongs is denoted as Find the associated sensor group G. For each sensor within the group... Analyze its changing patterns within the corresponding time window: ① To ensure the statistical reliability of the trend slope in the regression calculation, it is stipulated that each subinterval must contain at least 5 consecutive valid data points; ②Based on sensor sampling frequency Total duration of sliding window Calculate the total number of data points in the window. ; ③ Preliminary calculation of the number of sub-intervals ( (This is a function for rounding down). ④ The effective range for limiting the number of sub-intervals is: If the calculated value is less than 2, take 2 to avoid having too few sub-intervals and being unable to capture data fluctuations; if the calculated value is greater than 8, take 8 to avoid having too many sub-intervals and causing trend fragmentation.

[0043] After determining the initial number of sub-intervals according to the above rules, fine-tuning is made in conjunction with data inflection points: if the calculated sub-interval boundary point happens to be located at the inflection point where the data trend changes significantly, then the boundary point is retained; if the boundary point is located in the stable data segment, then two adjacent sub-intervals are merged to ensure that the data trend in each sub-interval is relatively simple and to avoid trend misjudgment caused by calculating the slope across trend segments.

[0044] For each sensor (Association group), obtain the average slope of each sub-interval, and generate a trend sequence. slope Linear regression calculations show that positive values ​​represent growth (+1), negative values ​​represent decrease (-1), and values ​​close to zero represent stability (0). For example, if the data within the window first increases and then decreases, then... .

[0045] For each sub-interval The symbolic slope distribution of all sensors within the statistical group in this sub-interval is analyzed, and the mode is taken as the mainstream direction of this sub-interval. Where sign represents the sign (+1, 0, -1). Integrating the main directions of all sub-intervals yields the association group. Mainstream trend sequence .

[0046] Locating suspected abnormal data points In the sliding time window sub-interval position (Right now (Sub-interval number); extract sensor In this sub-interval trend direction (Right now The (elements); comparison The mainstream direction of the corresponding sub-interval in the mainstream trend sequence Perform a consistency check; like The data point is determined to be an anomaly in structural response; the data is retained and the structural response anomaly early warning mechanism is triggered. like The data point is determined to be an abnormal sensor malfunction. The data is retained, and the structural monitoring sensor abnormality early warning mechanism is triggered.

[0047] S3. Perform time-series alignment and synchronization on the cleaned data, determine the standard time axis and alignment point of the digital twin system, construct a dynamic sliding window for monitoring data, and obtain the window slope by using the central difference weighted average based on the piecewise cubic Hermit interpolation PCHIP algorithm, map the monitoring data sequences with different sampling frequencies to the standard time axis, and generate a time-consistent fusion dataset.

[0048] This invention addresses the alignment and synchronization problem of digital twin systems caused by different sampling times of multi-source heterogeneous sensors at a dock. It proposes a PCHIP interpolation-based method for aligning and synchronizing multi-source heterogeneous data, ensuring high-precision alignment and synchronization of sensor data on a unified time axis, thus meeting the temporal consistency requirements of structural dynamic analysis of the digital twin system. The specific steps are as follows: S3.1: Determine the data alignment point on the standard time axis and update the dynamic sliding window of the monitoring data. 1) Select the data dynamic update frequency of the digital twin system based on the accuracy requirements of structural dynamic analysis and the sensor sampling frequency, calculate the data update time interval Δt, for example, 50ms, and calculate the next data alignment standard timestamp based on the current standard clock of the digital twin system. : ,in The current standard clock; 2) Construct a dynamic sliding window for each monitoring data sequence. ,in To monitor data timestamps, and , ; These are the monitoring data values ​​for the corresponding time points. Align the data with standard timestamps. The dynamic sliding window data for each monitoring data sequence is based on... The changes are dynamically updated.

[0049] S3.2: Data Interpolation Alignment Based on Cubic Hermitian Interpolation PCHIP 1) Calculate the corresponding slope window for each dynamic sliding window of the monitoring data sequence. ,in and Sliding windows and The slope of the monitoring data at the corresponding time point is calculated using the central difference weighted average. 2) Calculate (update) the data alignment standard timestamp for each monitoring data sequence. Corresponding alignment data value ,as follows: in, The basis functions are: Experimental analysis was conducted on the multi-source heterogeneous data processing and alignment synchronization method for the port digital twin system proposed in this invention. An experimental scheme was designed based on a large-scale port structural engineering project. The port has a total shoreline length of 2.5 kilometers and is equipped with 200 monitoring points, including strain sensors, displacement sensors, vibration sensors, acceleration sensors, and high-definition surveillance cameras (25fps). The network environment is a typical 4G / 5G hybrid network for the port, exhibiting periodic jitter (delay variance ±10ms). The main experimental parameters are shown in Table 5.

[0050] Table 5 Experimental parameters The experiment compared the key performance of the method of this invention with a general multi-source heterogeneous data processing synchronization scheme (hereinafter referred to as the "reference method"). The reference method adopts a general MQTT topic access without hierarchical design, a single 3σ criterion for data cleaning, and a time-series regularization scheme using linear interpolation. The experiment statistically analyzed indicators such as data cleaning reliability and time synchronization quality, and the results are shown in Table 6.

[0051] Table 6 Comparison of Key Performance Indicators Experimental Analysis: In anomaly detection, this invention accurately identified 48 anomalies through structural physical correlation and trend consistency verification, reducing the false alarm rate to 4% and improving the reliability of the monitoring system. Regarding data alignment and synchronization, the method of this invention reduces the data alignment error to 1.2%, improving accuracy by more than 70% compared to the reference method. This invention improves data aggregation efficiency and transmission reliability through a unified access mechanism, significantly enhances anomaly detection accuracy through a two-stage cleaning mechanism, and ensures data temporal consistency through the PCHIP dynamic alignment and synchronization method. It provides a high-precision, high-consistency, and high-reliability data foundation for the digital twin system of wharf structure monitoring, ensuring the real-time performance and effectiveness of wharf structure health monitoring.

[0052] Example 2: Based on the aforementioned unified access interface for multi-source heterogeneous data, two-stage data cleaning algorithm, and PCHIP interpolation alignment and synchronization algorithm, this invention constructs a digital twin multi-source heterogeneous data processing and alignment synchronization system for wharf structure monitoring. This system achieves standardized access and precise alignment and synchronization of heterogeneous data, providing highly consistent and accurate data support for wharf structure safety monitoring. Figure 2 As shown, it includes: The data access module adopts a dual-protocol architecture of "MQTT+RTSP" for unified access and aggregation of multi-source heterogeneous data. It uses a three-level hierarchical MQTT topic routing mechanism of "region-structure-sensor type" to classify structured sensor data and standardizes RTSP access for unstructured video streams. At the same time, it configures differentiated QoS levels and disconnection retransmission mechanisms according to the importance of data to generate standardized data streams to be processed. The data cleaning module performs two-stage cleaning processing on the standardized data stream, specifically including: obtaining the mean and standard deviation of single sensor data using a sliding time window, marking suspected anomalies according to the 3σ criterion; dividing sensors with physical location associations into association groups, constructing a trend association model based on the majority consensus principle, and verifying the changing trend of the suspected anomalies to distinguish between real structural anomalies and sensor fault data. The data alignment and synchronization module performs time-series alignment and synchronization on the cleaned data, determines the standard time axis and alignment point of the digital twin system, constructs a dynamic sliding window for monitoring data, and uses the PCHIP algorithm of piecewise cubic Hermit interpolation to obtain the window slope by using the central difference weighted average, mapping monitoring data sequences with different sampling frequencies to the standard time axis, and generating a time-consistent fused dataset.

[0053] The above modules can be deployed on the same device or distributed devices; the division of modules is only a functional logic description and does not limit the specific physical boundaries or implementation order.

[0054] Example 3: An electronic device is provided for running the aforementioned "Method for Processing and Aligning Heterogeneous Data from a Digital Twin for Monitoring a Dock Structure". The electronic device includes a processor, a memory, and optional communication interfaces / display devices / input devices, etc.; the memory stores a computer program that can run on the processor, and when the processor executes the program, it implements steps S1 to S3 of the method described in Embodiment 1, specifically including but not limited to: S1. The "MQTT+RTSP" dual-protocol architecture is adopted for unified access and aggregation of multi-source heterogeneous data. The three-level hierarchical MQTT topic routing mechanism of "region-structure-sensor type" is used to classify structured sensor data and standardize the RTSP access of unstructured video streams. At the same time, differentiated QoS levels and disconnection retransmission mechanisms are configured according to the importance of data to generate standardized data streams to be processed. S2. Perform a two-stage cleaning process on the standardized data stream, specifically including: using a sliding time window to obtain the mean and standard deviation of single sensor data, marking suspected anomalies according to the 3σ criterion; dividing sensors with physical location associations into association groups, constructing a trend association model based on the majority consensus principle, and verifying the changing trend of the suspected anomalies to distinguish between real structural anomalies and sensor fault data. S3. Perform time-series alignment and synchronization on the cleaned data, determine the standard time axis and alignment point of the digital twin system, construct a dynamic sliding window for monitoring data, and obtain the window slope by using the central difference weighted average based on the piecewise cubic Hermit interpolation PCHIP algorithm, map the monitoring data sequences with different sampling frequencies to the standard time axis, and generate a time-consistent fusion dataset.

[0055] The electronic device hardware can be one of a server, personal computer, workstation, industrial controller, edge computing device, or mobile terminal; the processor can be a general-purpose CPU, GPU, NPU, FPGA, or a combination thereof; the memory can be RAM, ROM, flash memory, or disk array. The device can interact with local / remote data storage (acquiring observation data and outputting inversion results) through a communication interface. The above hardware configuration does not constitute a limitation of the present invention.

[0056] Example 4: A computer-readable storage medium storing a computer program, which, when run on a processor of an electronic device, causes the program to execute the method steps S1 to S3 described in Embodiment 1; the storage medium may be a disk, optical disk, flash memory, solid-state drive, read-only memory, random access memory, or any combination of the above media.

[0057] Those skilled in the art will understand that the modules or steps described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, which can then be stored in a storage device for execution by a computer device. Alternatively, they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. This disclosure is not limited to any particular combination of hardware and software.

[0058] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0059] While the specific embodiments of this disclosure have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this disclosure are still within the scope of protection of this disclosure.

Claims

1. A method for processing and aligning heterogeneous multi-source data from a digital twin for monitoring wharf structures, characterized in that, Includes the following steps: The system adopts a dual-protocol architecture of "MQTT+RTSP" for unified access and aggregation of multi-source heterogeneous data. It uses a three-level hierarchical MQTT topic routing mechanism of "region-structure-sensor type" to classify structured sensor data and standardizes RTSP access for unstructured video streams. At the same time, it configures differentiated QoS levels and disconnection retransmission mechanisms according to the importance of data to generate standardized data streams to be processed. The standardized data stream undergoes a two-stage cleaning process, specifically including: obtaining the mean and standard deviation of single-sensor data using a sliding time window, and then... Criteria are used to mark suspected anomalies; sensors with physical location associations are divided into association groups, and a trend association model based on the majority consensus principle is constructed to verify the changing trends of the suspected anomalies in order to distinguish between real structural anomalies and sensor failure data. The cleaned data is time-series aligned and synchronized to determine the standard time axis and alignment point of the digital twin system. A dynamic sliding window for monitoring data is constructed, and the window slope is obtained by using the central difference weighted average based on the piecewise cubic Hermit interpolation PCHIP algorithm. Monitoring data sequences with different sampling frequencies are mapped to the standard time axis to generate a time-series consistent fusion dataset.

2. The method for processing and aligning heterogeneous multi-source data of a digital twin for monitoring wharf structures according to claim 1, characterized in that, A three-tiered MQTT topic routing mechanism based on "region-structure-sensor type" is used to classify structured sensor data and to standardize RTSP access for unstructured video streams, specifically including: Obtain device metadata and generate a globally unique identifier; among them, the unique identifier of the sensor follows the naming rule of "sensor type_structure ID_serial number", and the unique identifier of the camera follows the naming rule of "camera_structure ID_serial number". The hierarchical MQTT topic format is used to build ,in Corresponding area code, The corresponding unique identifier of the structure Corresponding sensor type; To construct a standardized RTSP access address format By locating the camera deployment position through the address hierarchy, the region and structure identifiers of the video stream address are kept consistent with the MQTT topic.

3. The method for processing and aligning heterogeneous multi-source data of a digital twin for monitoring wharf structures according to claim 1, characterized in that, Configure differentiated QoS levels and retransmission mechanisms based on data importance, specifically including the following steps: For sensor data transmitted via the MQTT protocol, differentiated QoS levels are configured based on the data value: critical monitoring data involving structural safety are configured as QoS level 2 to ensure data transmission without duplication or loss; ordinary environmental data such as temperature and humidity are configured as QoS level 1 to balance transmission reliability and efficiency. The sensor data is encapsulated using a unified field standard. The encapsulation format includes at least the following: a millisecond-level Unix timestamp collect_ts, a unique sensor ID sensor_id following the "type_structure_serial number" rule, and a monitoring value data_value matching the corresponding physical unit. For real-time video streams transmitted via the RTSP protocol, an "RTP+RTCP" transmission system is constructed, where RTP is responsible for real-time transmission of video frames and RTCP is responsible for transmission quality feedback; a packet loss retransmission threshold and a dynamic bitrate adjustment range are configured. When the packet loss rate exceeds the set threshold, retransmission is triggered, and the video bitrate is dynamically adjusted within the range of 2Mbps to 8Mbps according to the network bandwidth. Establish a transmission exception handling mechanism: when a connection is detected to be broken, start an MQTT persistent session and use the exponential backoff optimization algorithm to reconnect; After a successful reconnection, the continuity of the collect_ts timestamps is checked to determine if there is any data loss. If the time difference between adjacent data is greater than 1.5 times the sampling period, it is determined to be a packet loss, and a data traceability request carrying the missing data sensor_id and timestamp range is sent to the Broker. The Broker sends a retransmission command to the sensor based on the locally retained message logs, and the sensor retransmits the data according to the specified timestamp range and the original QoS level.

4. The method for processing and aligning heterogeneous multi-source data of a digital twin for monitoring wharf structures according to claim 1, characterized in that, Sensors with physically related locations are divided into association groups, and a trend association model based on the majority consensus principle is constructed, specifically including: Based on the physical topology and functional coupling relationship of the wharf structure, various preset sensor association group types are defined; The pile foundation monitoring group includes strain sensors deployed on the pile body, tilt sensors deployed on the pile top, and a vertical level deployed near the pile cap, used to verify the consistency between the tilt angle change when the pile foundation tilts and the settlement trend detected by the level. The beam monitoring group includes strain sensors deployed in the middle of the span and tilt sensors deployed in the same row of frames, used to verify the consistency between the increase in strain and the change in tilt angle in the same direction when the beam bends. The panel monitoring group includes strain sensors deployed in the middle of the panel span and vertical level instruments deployed at the panel supports, used to verify the consistency between the changes in panel bending strain and the settlement trend at the supports; The environmental correlation group includes temperature sensors, humidity sensors, and strain sensors in the same area, used for data verification based on the strain thermal effect caused by temperature changes.

5. The method for processing and aligning heterogeneous multi-source data of a digital twin for monitoring wharf structures according to claim 1, characterized in that, The verification of the changing trend of the suspected anomalies is as follows: Divide the sliding time window W into multiple continuous sub-intervals, and then, for the associated groups... Each sensor inside The average slope of each sub-interval is obtained, and the trend of change is quantified into a symbolic sequence based on the sign of the slope. Positive values ​​indicate growth, negative values ​​indicate decrease, and values ​​close to zero indicate stability. Number of subintervals Indicates the average slope; For each sub-interval The symbolic slope distribution of all sensors within the correlation group in this sub-interval is statistically analyzed, and the mode is taken as the mainstream direction of this sub-interval. ,in Symbolic function; integrate the main directions of all subintervals to obtain the association group. Mainstream trend sequence ; Locating suspected abnormal data points The position of the sub-interval Extract its corresponding sensor Trend direction in this sub-interval ; Direction of the trend The mainstream direction of the corresponding sub-interval in the mainstream trend sequence Compare: If and If they are consistent, then the data point is determined to be a structural anomaly reflecting the true structural response; if... and If there is a discrepancy, the data point is determined to be sensor fault data.

6. The method for processing and aligning heterogeneous multi-source data of a digital twin for monitoring wharf structures according to claim 1, characterized in that, The method for constructing a dynamic sliding window for monitoring data is as follows: Based on the accuracy requirements of structural dynamic analysis and the sensor sampling frequency, a data dynamic update frequency is set, and the data update time interval Δt is obtained; based on the current standard clock. Get the next data alignment standard timestamp ; For each monitoring data sequence, a dynamic sliding window containing four data points is constructed. ,in The timestamps of the monitoring data are arranged in ascending order of time. The corresponding monitoring data value; When constructing the sliding window, constrain the next data to align with the standard timestamp. The second data point timestamp within the window With the third data point timestamp Between, that is, satisfying and followed The monitoring data is updated dynamically within the push window.

7. The method for processing and aligning heterogeneous multi-source data of a digital twin for monitoring wharf structures according to claim 1, characterized in that, Based on the PCHIP algorithm with piecewise cubic Hermitian interpolation, the window slope is obtained by using the central difference weighted average. Monitoring data sequences with different sampling frequencies are mapped to a standard time axis to generate a time-consistent fused dataset. The specific steps include: Dynamic sliding window for each monitoring data sequence [ Get the corresponding slope window [ ,in and Sliding windows and The slope of the monitoring data at the corresponding time point is calculated using the central difference weighted average method; The next data alignment standard timestamp is obtained using the cubic Hermit interpolation algorithm. Corresponding alignment data value ,as follows: in, For normalization parameters, , , , The basis functions are: The obtained alignment data values As this monitoring data sequence on the standard time axis The mapping values ​​at each time point are summed to generate a time-consistent fused dataset.

8. A digital twin multi-source heterogeneous data processing and alignment synchronization system for wharf structure monitoring, characterized in that, include: The data access module adopts a dual-protocol architecture of "MQTT+RTSP" for unified access and aggregation of multi-source heterogeneous data. It uses a three-level hierarchical MQTT topic routing mechanism of "region-structure-sensor type" to classify structured sensor data and standardizes RTSP access for unstructured video streams. At the same time, it configures differentiated QoS levels and disconnection retransmission mechanisms according to the importance of data to generate standardized data streams to be processed. The data cleaning module performs two-stage cleaning processing on the standardized data stream, specifically including: obtaining the mean and standard deviation of single sensor data using a sliding time window, marking suspected anomalies according to the 3σ criterion; dividing sensors with physical location associations into association groups, constructing a trend association model based on the majority consensus principle, and verifying the changing trend of the suspected anomalies to distinguish between real structural anomalies and sensor fault data. The data alignment and synchronization module performs time-series alignment and synchronization on the cleaned data, determines the standard time axis and alignment point of the digital twin system, constructs a dynamic sliding window for monitoring data, and uses the PCHIP algorithm of piecewise cubic Hermit interpolation to obtain the window slope by using the central difference weighted average, mapping monitoring data sequences with different sampling frequencies to the standard time axis, and generating a time-consistent fused dataset.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and running thereon, characterized in that, When the processor executes the program, it implements the method for processing and aligning heterogeneous digital twin data for monitoring wharf structures as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the method for processing and aligning heterogeneous data from a digital twin for monitoring a wharf structure as described in any one of claims 1-7.

Images