Online abnormal data preprocessing method and system of bridge lifeline monitoring system

Abstract

The invention discloses an online abnormal data preprocessing method and system for a bridge lifeline monitoring system, and the method comprises the steps: collecting time sequence monitoring data of a bridge sensor, and carrying out the data alignment and downsampling according to a fixed time interval; calculating a sliding median and a median absolute deviation, then calculating a robust Z-score of each sampling point, and when the robust Z-score is greater than a preset threshold value, determining that the point is an abnormal point; correcting the abnormal points by adopting iterative updating; the sampling time point lacking the monitoring data is judged as a missing point, and missing point data is repaired; and outputting the cleaned monitoring data, and carrying out abnormal point visual labeling and curve drawing. According to the method, online anomaly detection, missing value repair and iterative optimization processing can be carried out on multi-type sensor data in a bridge lifeline monitoring system, so that the integrity, continuity and reliability of bridge monitoring data are ensured.

Description

Technical Field

[0001] This invention belongs to the field of bridge structural health monitoring technology, and particularly relates to an online abnormal data preprocessing method for a bridge lifeline monitoring system, as well as a system using the preprocessing method. Background Technology

[0002] With the continuous expansion of large-scale bridge construction projects in my country, the safety and reliability of bridge structures have become a key focus of project management. Structural health monitoring systems, by deploying sensors at critical locations on bridges, can continuously collect multi-dimensional data including displacement, strain, cable tension, deflection, temperature and humidity, and wind load, providing fundamental data for bridge operational status assessment, anomaly warning, and maintenance decisions. However, several problems exist in actual monitoring processes that affect the validity and usability of the data.

[0003] First, sensors are susceptible to environmental interference during long-term operation, such as temperature changes, wind load impacts, or human-induced vibrations, leading to sudden anomalies in the collected data. These anomalies may deviate from the true measured values ​​by several times, and if left unaddressed, will severely impact structural condition analysis. Second, communication link interruptions, equipment failures, or insufficient power can cause intermittent or discontinuous gaps in data acquisition, resulting in gaps in the monitoring sequence and affecting overall data continuity and trend analysis.

[0004] Furthermore, traditional threshold-based anomaly detection methods are sensitive to extreme data, lack robustness, and cannot effectively distinguish between real anomalies and noise interference. In high-frequency sampled data, transient anomalies may be misclassified as structural anomalies, increasing the false alarm rate. Moreover, existing data cleaning methods are mostly offline, which cannot meet the real-time and iterative repair capabilities required by bridge lifeline monitoring systems. Summary of the Invention

[0005] The first objective of this invention is to provide an online abnormal data preprocessing method for a bridge lifeline monitoring system that can ensure the integrity, continuity, and reliability of bridge monitoring data.

[0006] The first objective of this invention is achieved through the following technical measures: an online abnormal data preprocessing method for a bridge lifeline monitoring system, characterized by comprising the following steps:

[0007] S1. Collect time-series monitoring data from bridge sensors and perform data alignment and downsampling at fixed time intervals, wherein the downsampling method is to perform median statistical downsampling at certain intervals;

[0008] S2. Calculate the moving median and median absolute deviation of the data obtained from step S1. Then calculate the robust Z-score of each sampling point based on the moving median and median absolute deviation. When the robust Z-score is less than or equal to the preset threshold, proceed to step S4. When the robust Z-score is greater than the preset threshold, determine that the point is an outlier and proceed to step S3.

[0009] S3. Set the data identified as outliers to null, then calculate the moving median and median absolute deviation of the other data obtained from step S1. Then calculate the robust Z-score for each sampling point based on the moving median and median absolute deviation. When the robust Z-score is less than or equal to the preset threshold, proceed to step S4. When the robust Z-score is greater than the preset threshold, the point is identified as an outlier. Repeat this step until the number of outliers converges or the maximum number of iterations is reached, then proceed to step S4.

[0010] S4. Identify sampling time points with missing monitoring data as missing points and repair the missing data:

[0011] (1) When the missing point is located between two known points ( , )and( , The set of missing points between ) is denoted as Interval linear interpolation is used to identify missing points. Repair value for:

[0012] ;

[0013] (2) When the missing point appears at the beginning of the sequence, and the first known point is ( , The set of missing points is Backfilling interpolation is used, that is, filling in the missing points with the nearest known value to the right. Repair value for:

[0014] ;

[0015] (3) When the missing point is located at the end of the sequence and only the left neighboring known point exists ( , When ), the set of missing points is Forward padding interpolation is used, which fills in the missing points with the nearest known value to the left. Repair value for:

[0016] ;

[0017] S5. Output the monitoring data after cleaning, and perform visualization annotation of anomalies and curve plotting.

[0018] This invention enables online anomaly detection, missing value repair, and iterative optimization of multi-type sensor data collected in a bridge lifeline monitoring system, thereby ensuring the integrity, continuity, and reliability of bridge monitoring data and providing an accurate basis for bridge structural condition analysis and early warning.

[0019] In step S3 of this invention, the convergence condition for the number of outliers is:

[0020] ;

[0021] Among them, O k ϵ represents the number of outliers detected in the k-th iteration, and ϵ is the convergence threshold, which is usually set to 1.

[0022] In step S3 of this invention, the maximum number of iterations is 3 to 10.

[0023] In step S2 of this invention, the formula for calculating the median absolute deviation (MAD) is:

[0024] ;

[0025] In the formula, To monitor data points, This indicates the calculation of the median.

[0026] In step S4(1) of the present invention, interpolation for:

[0027] ;

[0028] in,( , )and( , () represents the known points at both ends of the missing segment. This is a time index for missing points.

[0029] In step S2 of this invention, the robust Z-score calculation formula is as follows:

[0030] ;

[0031] when When the value is greater than T, the point is determined to be an outlier, where T is a preset threshold, typically ranging from 2.5 to 4.

[0032] The time-series monitoring data of the bridge sensor described in this invention includes static monitoring data such as cable force, strain, displacement, deflection, crack width, temperature and humidity, and average wind load.

[0033] The anomaly visualization annotation and curve drawing described in this invention includes a comparison of curves between the original data and the cleaned data, an anomaly location annotation map, and an anomaly statistics table. The anomaly statistics table contains anomaly timestamps and corresponding anomaly values, which facilitates analysis and decision-making by engineers.

[0034] A second objective of this invention is to provide a system for preprocessing online abnormal data using the aforementioned bridge lifeline monitoring system.

[0035] The second objective of this invention is achieved through the following technical measures: a system for preprocessing online abnormal data using the above-mentioned bridge lifeline monitoring system, characterized by comprising:

[0036] The data acquisition module is used to acquire bridge sensor data in real time and perform data alignment and downsampling at fixed time intervals;

[0037] The anomaly detection module is used to calculate the moving median and median absolute deviation of the data processed by the data acquisition module, and then calculate the robust Z-score of each sampling point based on the moving median and median absolute deviation. When the robust Z-score exceeds the preset threshold, the point is determined to be an anomaly.

[0038] The iterative optimization module is used to correct outliers through iterative updates. In each iteration, the data identified as outliers are set to null, the moving median and the absolute deviation of the median are recalculated, and outlier detection is performed again until the number of outliers converges or the maximum number of iterations is reached.

[0039] The missing value repair module is used to detect missing points and repair the missing value values ​​using comprehensive interpolation and two-way filling.

[0040] The data visualization module is used to generate cleaned data curves, outlier annotations, and statistical tables, and is integrated into single-bridge monitoring systems, bridge group monitoring systems, and urban lifeline monitoring systems.

[0041] The data acquisition module, anomaly detection module, iterative optimization module, missing value repair module, and data visualization module are connected sequentially.

[0042] Compared with the prior art, the present invention has the following significant effects:

[0043] (1) In the real-time processing of bridge monitoring data, this invention performs iterative robust detection of monitoring data by setting a sliding window and threshold, emptying outliers and correcting them through interpolation until convergence. Among them, the sliding median and MAD methods are used, which have strong robustness and low sensitivity to extreme values ​​and noise. The iterative correction mechanism ensures reliable convergence of outlier detection and repair results. The continuity and smoothness of the sequence are achieved through time interpolation and bidirectional filling, so that smooth and reliable monitoring data can be obtained in the end. At the same time, this invention supports graphical visualization, which can intuitively display the original data, outliers and cleaned data, which facilitates engineers to quickly analyze and make decisions, and assists in the online assessment and early warning of bridge operation status.

[0044] (2) This invention has the advantages of strong robustness, good real-time performance and high versatility. It also supports integration into online platforms such as single bridge monitoring systems, bridge group monitoring systems and urban lifeline monitoring systems. It is suitable for real-time preprocessing of various types of bridge static sensor data such as strain, deflection, temperature, displacement and crack width. Attached Figure Description

[0045] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0046] Figure 1 This is a block diagram of the pretreatment method flow and system composition of the present invention;

[0047] Figure 2 This is a curve diagram of abnormal detection and preprocessing of strain monitoring data of a bridge main beam in Example 1 of this invention;

[0048] Figure 3 This is a curve of abnormal detection and preprocessing of temperature monitoring data of a bridge in Example 2 of the present invention;

[0049] Figure 4 This is a curve diagram of abnormal detection and preprocessing of deflection monitoring data of a bridge in Example 3 of the present invention;

[0050] Figure 5 This is a graph showing the abnormal detection and preprocessing of crack monitoring data in the main beam of a bridge in Example 4 of this invention. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are also within the scope of protection of the present invention.

[0052] like Figures 1-5As shown, the present invention provides an online abnormal data preprocessing method for a bridge lifeline monitoring system, comprising the following steps:

[0053] S1. Collect time-series monitoring data from bridge sensors, including static monitoring data such as cable force, strain, displacement, deflection, crack width, temperature and humidity, and average wind load; then perform data alignment and downsampling at fixed time intervals. The downsampling method is to use median statistical downsampling at certain intervals to ensure the representativeness of the data and its resistance to anomalies.

[0054] Data alignment refers to the process of establishing a standard time series with a unified time step Δt (e.g., 1 minute or 5 minutes) to address situations where different sensors have inconsistent sampling frequencies or timestamp discrepancies. Then, the raw data from each sensor are mapped to this unified time axis using timestamp matching. For missing or offset time points, linear interpolation or spline interpolation is used to fill in the gaps, ensuring that all monitored quantities are synchronously aligned at the same time node, thereby guaranteeing consistency and comparability among multi-source data.

[0055] S2. Calculate the moving median and median absolute deviation (MAD) of the data obtained in step S1. Then, calculate the robust Z-score for each sampling point based on the moving median and median absolute deviation. When the robust Z-score is less than or equal to a preset threshold, proceed to step S4. When the robust Z-score is greater than the preset threshold, determine that the point is an outlier and proceed to step S3.

[0056] The formula for calculating the median absolute deviation (MAD) is:

[0057] ; In the formula, To monitor data points, This indicates the calculation of the median.

[0058] The robust Z-score calculation formula is:

[0059] ;

[0060] when When the value is greater than T, the point is determined to be an outlier, where T is a preset threshold, typically ranging from 2.5 to 4.

[0061] An iterative method based on the sliding median and median absolute deviation (MAD) is used to detect outliers. The standardized residual value at each time step is calculated and it is determined whether it exceeds a preset threshold. The robust Z-score method is used to detect outliers in the time series. The detected outliers are marked, and an iterative update strategy is used to continuously remove extreme values ​​and re-estimate the baseline level of the series until the results converge or the maximum number of iterations is reached.

[0062] The anomaly detection operation corresponds to step S2 of this invention and is a core processing step of this method. In this step, robust Z-score calculation is used to identify abnormal fluctuations in the time series in real time, and the detected anomaly information is passed as input to the subsequent step S3 for data repair and reconstruction.

[0063] S3. Set the data identified as outliers to null, then calculate the moving median and median absolute deviation of the other data obtained from step S1. Then calculate the robust Z-score for each sampling point based on the moving median and median absolute deviation. When the robust Z-score is less than or equal to the preset threshold, proceed to step S4. When the robust Z-score is greater than the preset threshold, the point is identified as an outlier. Repeat this step until the number of outliers converges or the maximum number of iterations is reached, then proceed to step S4.

[0064] This step uses iterative updates to correct outliers. In each iteration, the data identified as outliers are cleared, the moving median and MAD value are recalculated, and outlier detection is performed again until the number of outliers converges. That is, the difference in the number of outliers between two iterations is less than a set threshold, or the maximum number of iterations Nmax = 3 to 10 is reached.

[0065] The convergence condition for the number of outliers is:

[0066] ;

[0067] Among them, O k ϵ represents the number of outliers detected in the k-th iteration, and ϵ is the convergence threshold, which is usually set to 1.

[0068] S4. Identify sampling time points with missing monitoring data as missing points, and repair the missing data using comprehensive interpolation and two-way imputation:

[0069] (1) When the missing point is located between two known points ( , )and( , The set of missing points between ) is denoted as Interval linear interpolation is used to identify missing points. Repair value for:

[0070] ;

[0071] (2) When the missing point appears at the beginning of the sequence, and the first known point is ( , The set of missing points is Backfilling interpolation is used, that is, filling in the missing points with the nearest known value to the right. Repair value for:

[0072] ;

[0073] (3) When the missing point is located at the end of the sequence and only the left neighboring known point exists ( , When ), the set of missing points is Forward padding interpolation is used, which fills in the missing points with the nearest known value to the left. Repair value for:

[0074] ;

[0075] In the formula, This is a time index for missing points.

[0076] The detected outliers are cleared and repaired using time interpolation methods. These methods combine interval linear interpolation, forward time interpolation, and backward time interpolation to ensure the continuity and smoothness of the sequence in the missing intervals and at the boundaries.

[0077] The above combination of methods ensures that the data sequence can be continuously and smoothly repaired in terms of internal gaps and beginning and end boundaries.

[0078] S5. Output the cleaned monitoring data and perform visualization annotation and curve plotting of anomalies, including a comparison of the curves of the original data and the cleaned data, an anomaly location annotation map, and an anomaly statistics table, in which the anomaly statistics table includes anomaly timestamps and corresponding anomaly values.

[0079] A system for online abnormal data preprocessing using the above-mentioned bridge lifeline monitoring system includes a data acquisition module, an anomaly detection module, an iterative optimization module, a missing value repair module, and a data visualization module connected in sequence.

[0080] The data acquisition module is used to acquire bridge sensor data in real time and perform data alignment and downsampling at fixed time intervals; this module performs step S1.

[0081] The anomaly detection module is used to calculate the moving median and median absolute deviation of the data processed by the data acquisition module, and then calculate the robust Z-score of each sampling point based on the moving median and median absolute deviation. When the robust Z-score exceeds a preset threshold, the point is determined to be an anomaly point; this module proceeds to step S2.

[0082] The iterative optimization module is used to correct outliers through iterative updates. In each iteration, the data identified as outliers are cleared, the moving median and the absolute deviation of the median are recalculated, and outlier detection is performed again until the number of outliers converges or the maximum number of iterations is reached. This module performs step S3.

[0083] The missing value repair module is used to detect missing points and repair the missing value values ​​using comprehensive interpolation and two-way filling; this module performs step S4.

[0084] The data visualization module generates cleaned data curves, outlier annotations, and statistical tables, and integrates them into the single-bridge monitoring system, bridge group monitoring system, and urban lifeline monitoring system. This module performs step S5;

[0085] Example 1:

[0086] This example demonstrates the anomaly detection and preprocessing of strain monitoring data for the main beam.

[0087] Strain sensors were installed at key sections of the main beam.

[0088] S1. Data Acquisition and Downsampling: The original sampling frequency of the strain sensor is about 3Hz. To reduce the influence of instantaneous noise, a low-frequency representative sequence is obtained by downsampling the median at 1s intervals.

[0089] S2. Anomaly Detection: Robust Z-score was calculated based on the moving median and MAD method, and a large number of anomalies were initially detected.

[0090] S3. Iterative Correction: Set outliers to null, recalculate the moving median and MAD value, and repeat the iteration until the number of outliers converges or the maximum number of iterations is reached.

[0091] S4. Missing value repair: Linear interpolation is used for internal missing or outlier points, and forward or backward filling is used for missing points at the beginning or end of the sequence.

[0092] S5. Result Visualization: Outputs the strain curve after cleaning, and marks outliers. (See below) Figure 2 .

[0093] This embodiment demonstrates that even with frequent high-frequency anomalies, the method of the present invention can reliably eliminate anomalies and repair data.

[0094] Partial strain data for this example are shown in Table 1:

[0095] time Downsampled data Data after anomaly detection and processing Outlier 2024 / 4 / 1 3:16:16 3.5 3.5 0 2024 / 4 / 1 3:16:19 3.3 3.3 0 2024 / 4 / 1 3:16:27 3.5 3.5 0 2024 / 4 / 1 3:16:30 3.1 3.1 0 2024 / 4 / 1 3:16:34 3.5 3.5 0 2024 / 4 / 1 3:16:41 3.5 3.5 0 2024 / 4 / 1 3:16:44 3.1 3.1 0 2024 / 4 / 1 3:16:47 3.5 3.5 0 2024 / 4 / 1 3:16:49 3.1 3.1 0 2024 / 4 / 1 3:16:52 3.1 3.1 0 2024 / 4 / 1 3:16:59 3.1 3.1 0 2024 / 4 / 1 3:17:02 3.1 3.1 0 2024 / 4 / 1 3:17:10 2.8 2.8 0 2024 / 4 / 1 3:17:12 2.9 2.9 0 2024 / 4 / 1 3:17:16 2.1 2.1 0 2024 / 4 / 1 3:17:20 2.6 2.6 0 2024 / 4 / 1 3:17:24 3.5 3.5 0 2024 / 4 / 1 3:17:26 3.1 3.1 0 2024 / 4 / 1 3:17:30 2.6 2.6 0 2024 / 4 / 1 3:17:35 4 4 0 2024 / 4 / 1 3:17:38 4.5 3.663648476 1

[0096] (Table 1)

[0097] The anomaly data detected in this example are shown in Table 2 below:

[0098] time outliers 2024 / 4 / 1 0:03:48 8.1 2024 / 4 / 1 0:05:28 7.9 2024 / 4 / 1 2:38:50 4.8 2024 / 4 / 1 3:04:51 1.2 2024 / 4 / 1 3:05:02 10.4 2024 / 4 / 1 3:05:05 0.7 2024 / 4 / 1 3:17:38 4.5 2024 / 4 / 1 3:17:40 1.7 2024 / 4 / 1 3:35:43 5.4 2024 / 4 / 1 4:10:27 4.6 2024 / 4 / 1 4:13:51 -1 2024 / 4 / 1 4:13:53 -1 2024 / 4 / 1 4:13:57 3.9 2024 / 4 / 1 4:14:03 4.3

[0099] (Table 2)

[0100] Example 2:

[0101] This example demonstrates anomaly detection and preprocessing in temperature monitoring data.

[0102] S1. Data Acquisition and Downsampling: The original sampling frequency of the temperature sensor is variable. After processing, it is downsampled to low-frequency data with an hourly interval. There are fewer outliers.

[0103] S2. Anomaly Detection: Calculate the moving median and MAD Z-score to detect a small number of outliers.

[0104] S3. Iterative correction: Outliers are iteratively removed, leading to rapid convergence.

[0105] S4. Missing value repair: Linear interpolation and beginning / end padding strategies are used to ensure the continuity and smoothness of the sequence.

[0106] S5. Result Visualization: Outputs the temperature curve and anomaly point annotation diagram after cleaning. (See below) Figure 3 The verification method is suitable for low-frequency long-term data.

[0107] Example 3:

[0108] This example demonstrates anomaly detection and preprocessing of deflection monitoring data.

[0109] S1. Data Acquisition and Downsampling: Deflection sensors are deployed on the main beam of the bridge. The original sampling frequency is variable, and the data is downsampled to low-frequency data at 1-hour intervals. The number of anomalies is moderate, mainly occurring during peak traffic periods.

[0110] S2. Anomaly Detection: The sliding median and MAD methods are used to identify a small number of outliers.

[0111] S3. Iterative correction: Iteratively remove outliers, and the sequence converges and becomes stable.

[0112] S4. Missing value repair: linear interpolation and beginning / end padding.

[0113] S5. Result Visualization: Outputs the deflection curve and anomaly point annotation diagram after cleaning. (See below) Figure 4 The verification method is effective for data with a moderate frequency of anomalies.

[0114] Example 4:

[0115] This example demonstrates the anomaly detection and preprocessing of main beam crack monitoring data:

[0116] S41. Data Acquisition and Downsampling: The original sampling frequency of the crack width monitoring sensor is variable. Downsampling is performed using low-frequency data at 1-hour intervals, resulting in almost no anomalies.

[0117] S2. Anomaly detection: Detection using the moving median and MAD methods, with almost no outliers, and iterative correction is completed quickly.

[0118] S3. Iterative correction: Iteratively remove outliers, and the sequence converges and becomes stable.

[0119] S4. Result Visualization: Outputs the crack data curve after cleaning. Anomaly point annotations show almost no anomalies. See [link / reference] Figure 5 The verification method can maintain the authenticity of low-abnormal data.

Claims

1. An online abnormal data preprocessing method for a bridge lifeline monitoring system, characterized in that Includes the following steps: S1. Collect time-series monitoring data from bridge sensors and perform data alignment and downsampling at fixed time intervals; S2. Calculate the moving median and median absolute deviation of the data obtained from step S1. Then, calculate the robust Z-score for each sampling point based on the moving median and median absolute deviation. When the robust Z-score is less than or equal to the preset threshold, proceed to step S4. When the robust Z-score is greater than the preset threshold, determine that the point is an outlier and proceed to step S3. S3. Set the data identified as outliers to null, then calculate the moving median and median absolute deviation of the other data obtained from step S1. Then calculate the robust Z-score for each sampling point based on the moving median and median absolute deviation. When the robust Z-score is less than or equal to the preset threshold, proceed to step S4. When the robust Z-score is greater than the preset threshold, the point is identified as an outlier. Repeat this step until the number of outliers converges or the maximum number of iterations is reached, then proceed to step S4. S4. Identify sampling time points with missing monitoring data as missing points and repair the missing data: (1) When the missing point is located between two known points ( , ) and ( , ), the missing point set is denoted as , and the repair value of the missing point is obtained by interval linear interpolation. ； (2) When the missing point appears at the beginning of the sequence, and the first known point is ( , The set of missing points is Backfilling interpolation is used, that is, filling in the missing points with the nearest known value to the right. Repair value for: ； (3) When the missing point is located at the tail of the sequence and there is only a left adjacent known point ( , ), the missing point set is , and the forward filling interpolation is adopted, that is, the nearest left known value is filled, and the repair value of the missing point is : ； In the formula: is a missing point time index; S5. Output the monitoring data after cleaning, and perform visualization annotation of anomalies and curve plotting.

2. The online abnormal data preprocessing method of the bridge lifeline monitoring system according to claim 1, characterized in that: In step S3, the convergence condition for the number of outliers is: ； where O k represents the number of abnormal points detected in the kth iteration, and ε is the convergence threshold, usually taking a value of 1.

3. The method of online abnormal data preprocessing for bridge lifeline monitoring system according to claim 1, characterized in that: In step S3, the maximum number of iterations is 3 to 10.

4. The online abnormal data preprocessing method of the bridge lifeline monitoring system according to any one of claims 1-3, characterized in that: In step S2, the formula for calculating the median absolute deviation (MAD) is: ； In the formula, To monitor data points, Indicates median calculation.

5. The online abnormal data preprocessing method of the bridge lifeline monitoring system according to claim 4, characterized in that: In step S2, the robust Z-score calculation formula is: ； when When the value is greater than T, the point is determined to be an outlier, where T is a preset threshold with a value range of 2.5 to 4.

6. The online abnormal data preprocessing method for the bridge lifeline monitoring system according to claim 5, characterized in that: The time-series monitoring data of the bridge sensors include static monitoring data of cable force, strain, displacement, deflection, crack width, temperature and humidity, and average wind load.

7. The online abnormal data preprocessing method for the bridge lifeline monitoring system according to claim 6, characterized in that: In step S5, the visualization and curve drawing of outliers includes a comparison of curves between the original data and the cleaned data, an outlier location annotation map, and an outlier statistics table, wherein the outlier statistics table includes outlier timestamps and corresponding outlier values.

8. A system for preprocessing online abnormal data using the bridge lifeline monitoring system of claim 1, characterized in that... include: The data acquisition module is used to acquire bridge sensor data in real time and perform data alignment and downsampling at fixed time intervals; The anomaly detection module is used to calculate the moving median and median absolute deviation of the data processed by the data acquisition module, and then calculate the robust Z-score of each sampling point based on the moving median and median absolute deviation. When the robust Z-score exceeds the preset threshold, the point is determined to be an anomaly. The iterative optimization module is used to correct outliers through iterative updates. In each iteration, the data identified as outliers are set to null, the moving median and the absolute deviation of the median are recalculated, and outlier detection is performed again until the number of outliers converges or the maximum number of iterations is reached. The missing value repair module is used to detect missing points and repair the missing value values ​​using comprehensive interpolation and two-way filling. The data visualization module is used to generate cleaned data curves, outlier annotations, and statistical tables, and is integrated into single-bridge monitoring systems, bridge group monitoring systems, and urban lifeline monitoring systems. The data acquisition module, anomaly detection module, iterative optimization module, missing value repair module, and data visualization module are connected sequentially.

Images