Bridge monitoring data cleaning and quality evaluation method based on mask reconstruction

Abstract

The application discloses a bridge monitoring data cleaning and quality evaluation method based on mask reconstruction, belongs to the field of structure health monitoring and time series processing, and solves the problem that long-term monitoring data such as deflection and strain are abnormal in quality and are difficult to predict and evaluate due to noise, drift and sudden disturbance. The method comprises the following steps: acquiring deflection and strain sequences and segmenting by sliding window, constructing noise, drift, outlier and sudden pollution in the training stage and generating a mask, constructing a time series convolution network trained by physically guiding mask reconstruction, and repairing the loss containing the pollution position, keeping the un-polluted and the second-order difference consistency physical regularity. In the inference stage, the cleaned sequence is output, the quality score is obtained from the residual error, and the quality prompt is output in combination with the quantile threshold and the time-varying threshold grouped by hours. The application realizes the repair of multiple types of abnormalities and the interpretable prompt by mask-guided repair and un-polluted keeping and the introduction of physical consistency joint optimization, and improves the data availability and reliability.

Description

Technical Field

[0001] This invention belongs to the field of structural health monitoring and time series data processing technology, and specifically relates to a method for cleaning and quality assessment of bridge monitoring data based on mask reconstruction. Background Technology

[0002] Bridge health monitoring systems typically collect structural response data such as deflection and strain continuously over long periods at fixed sampling intervals, providing a data foundation for structural condition assessment, trend analysis, and early warning decisions. In actual engineering projects, monitoring data is characterized by its long-term continuity, strong non-stationarity, and numerous sources of disturbance. On the one hand, it is affected by external conditions such as traffic loads, temperature changes, and weather conditions. On the other hand, under long-term service and complex environmental interference, the monitoring link is susceptible to factors such as sensor failures, interference in the acquisition circuit, communication link jitter, and maintenance operations, leading to data quality anomalies such as high-frequency noise, slow baseline shifts, outlier spikes, and abnormal fluctuations in local segments. These quality anomalies often stem from measurement quality issues rather than actual structural degradation. If used directly as input for subsequent predictive modeling, condition assessment, or threshold warnings without processing, it can easily cause feature contamination and error accumulation, reducing the model's generalization ability and stability, and even leading to false alarms and missed alarms, affecting the reliability of operation and maintenance decisions.

[0003] Existing cleaning methods mostly rely on simple thresholds, smoothing filters, or strategies to minimize overall reconstruction errors. On the one hand, they struggle to cover multiple types of anomalies when anomaly labels are scarce. On the other hand, overall reconstruction is prone to over-rewriting and over-smoothing of normal segments, resulting in damage to trend and cyclical patterns. Furthermore, in situations with significant daily cycles and non-stationary conditions, using fixed thresholds for quality indicators is also difficult to adapt to the normal fluctuation levels across different time periods.

[0004] Therefore, it is necessary to provide a bridge monitoring data processing method that can perform targeted repair of various types of quality anomalies such as noise, drift, and mutation under weak supervision or self-supervision, and provide interpretable quality scores and prompts while outputting cleaned data, so as to improve the stability and reliability of monitoring data for subsequent predictive modeling and state assessment. Summary of the Invention

[0005] To address the aforementioned shortcomings, the present invention aims to provide a bridge monitoring data cleaning and quality assessment method based on mask reconstruction. Under conditions of scarce anomaly labels, non-stationary data, and diverse types of quality anomalies, this method effectively repairs quality anomalies and provides interpretable prompts for data quality, thereby improving the usability and reliability of monitoring data for subsequent prediction and status assessment.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: The technical solution of this invention is a method for cleaning and quality assessment of bridge monitoring data based on mask reconstruction, comprising the following steps: Step 1: Acquire bridge monitoring data and visualize it in a window; Step 2: Construct training contamination and generate contamination mask: Step 3: Construct a temporal convolutional network for physical guidance mask reconstruction and train and optimize it; Step 4: Use the trained model to output cleaned and reconstructed sequences and provide quality feedback.

[0007] Preferably, in step one, the deflection data and strain data are windowed with a window length of L, and are trained and inferred using cleaning and reconstruction models with the same structure but independent parameters.

[0008] Preferably, in step two, the pollution types include noise pollution, drift pollution, and outlier mutation pollution, and can be applied to the window sample individually or in combination.

[0009] Preferably, in step two, the contamination mask is a binary matrix M with the same dimension as the window sample, where M=1 represents the contaminated position, M=0 represents the uncontaminated position, and M=1 is corresponding to the interval of the mutation segment.

[0010] Preferably, in step three, both the repair loss and the retention loss are element-wise error aggregations, with the repair loss calculated only at position M=1 and the retention loss calculated only at position M=0.

[0011] Preferably, in step three, the physical consistency regularization term calculates the second-order difference between the input sequence and the cleaned and reconstructed sequence and constrains the difference between them, so that the cleaned and reconstructed sequence is consistent with the input in terms of rate of change and curvature shape.

[0012] Preferably, in step four, the quantile threshold includes a globally fixed threshold and a time-varying threshold grouped by hour, and the quality score is a window-level score, which is calculated from the reconstructed residuals and obtained through robust statistical aggregation, wherein the robust statistics are median aggregation.

[0013] Preferably, in step four, the quality score is a window-level score, which is obtained by calculating the reconstructed residuals and then performing robust statistical aggregation, wherein the robust statistics are median aggregation.

[0014] Compared with the prior art, the present invention has the following advantages: This invention designs a bridge monitoring data cleaning and quality assessment method based on mask reconstruction. It employs an efficient temporal convolutional network, using contamination construction and mask guidance during the training phase to achieve targeted repair of typical quality anomalies. The invention introduces physical consistency constraints to reduce excessive smoothing and morphological distortion during the cleaning process, making the cleaning results more consistent with the dynamic characteristics of the structural response. Furthermore, the invention constructs window-level quality scores based on reconstruction residuals and uses robust aggregation to output quality prompts, achieving integrated output of cleaning reconstruction and quality judgment, thereby improving the usability and engineering reliability of the monitoring data. Attached Figure Description

[0015] Figure 1 This is a flowchart of an embodiment of the present invention; Figure 2 This is an illustration of the network structure in an embodiment of the present invention; Figure 3 This is a schematic diagram of the contamination structure and mask generation in an embodiment of the present invention; Figure 4 This is a schematic diagram of the mask joint loss and physical consistency constraint module in an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the quality score calculation and threshold prompting in an embodiment of the present invention; Figure 6 This is a visual diagram illustrating the cleaning and reconstruction effect of an embodiment of the present invention. Detailed Implementation

[0016] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments. The present invention provides a method for cleaning and quality assessment of bridge monitoring data based on mask reconstruction, and the corresponding flowchart is shown below. Figure 1 As shown, the overall network structure is as follows Figure 2 As shown, the contamination structure and mask generation are as follows: Figure 3 As shown, the mask joint loss and physical consistency constraint are as follows: Figure 4 As shown, the quality score and threshold prompts are as follows: Figure 5 As shown, the cleaning and reconstruction effect is visualized as follows: Figure 6 As shown.

[0017] Figure 1 The diagram shows a flowchart of a bridge monitoring data cleaning and quality assessment method based on mask reconstruction according to an embodiment of the present invention. The specific steps are as follows: Step 1: Acquire the deflection and strain time series data collected by the bridge monitoring system, and perform sliding windowing processing on the time series to obtain window samples; Step 2: Contaminate the window samples during the training phase and generate a contamination mask with the same dimension as the window samples. Step 3: Construct a mask-based cleansing and reconstruction network and train and optimize the network parameters, adjusting the model weights through backpropagation of the loss function; Step 4: Input the data to be processed into the trained model, output the cleaned and reconstructed sequence, and generate quality scores and quality tips based on the reconstructed residuals.

[0018] Figure 2 The diagram shown is a model structure diagram of a bridge monitoring data cleaning and quality assessment method based on mask reconstruction according to an embodiment of the present invention. The specific steps are as follows: Step 1: Construct a cleaning and reconstruction network, and form an overall structure from contamination construction to joint constraints and then to scoring prompts; First, construct the cleaning and reconstruction network. Input is a sample from the bridge monitoring window. The output is the same-dimensional cleaning and reconstruction result. The mapping relationship is as follows:

[0019]

[0020] Where L is the window length, C is the number of channels, and x can be the deflection window or the strain window. Deflection and strain are trained and inferred using models with the same structure but independent parameters.

[0021] The window sequence is first input into a cleaning and reconstruction network with a temporal convolutional autoencoder as its backbone. This network consists of two parts: an encoder and a decoder. The encoder extracts temporal representations within the window and learns typical response structures under healthy conditions. The decoder maps the latent representations back to the original channel space and outputs a reconstruction result with the same dimension as the input. The model reconstruction output is then directly output as the cleaning and reconstruction sequence, thus forming an end-to-end data quality control process. This structure enables the main link to suppress anomalous perturbations while preserving the main trends and periodic patterns, thereby aligning with the unified cleaning objective of three types of quality anomalies: noise contamination, slow drift, and outlier mutations.

[0022] The contamination construction and mask generation module, such as Figure 3 As shown, this is used to construct noise contamination, drift contamination, and outlier mutation contamination on window samples during the training phase, and to generate a binary contamination mask. M=1 represents a contaminated location and M=0 represents an uncontaminated location. When the contamination is a sudden change segment, M=1 is applied to the interval corresponding to the sudden change segment.

[0023] The mask joint loss and physical consistency constraint module, such as Figure 4 As shown, the training target is partitioned based on the contamination mask to achieve targeted repair of contaminated areas and preservation of normal areas, and excessive smoothing and morphological distortion are suppressed by physical consistency regularization terms.

[0024] The quality score calculation and threshold prompt module, such as Figure 5 As shown, a window-level quality score is calculated based on the residual between the input and the cleaned and reconstructed output during the inference phase, and a quality hint result is output in combination with a quantile threshold strategy.

[0025] Step 2: Construct a mask joint loss and physical consistency constraints, and train and optimize network parameters to form a training mechanism from targeted repair to normal preservation and then to physical morphology constraints: Constructing the joint loss function in step one Including losses from remediation of contaminated sites Loss of maintaining uncontaminated locations and physical consistency regularization terms Its form is:

[0026] in , , These are the weighting coefficients.

[0027]

[0028] The site remediation loss is calculated only at the location M=1 to enhance targeted remediation of the contaminated area.

[0029] Repair loss at uncontaminated locations is calculated only at M=0 to suppress unnecessary rewriting of normal segments:

[0030] in, This is the element-wise error function, which can be either absolute error or squared error.

[0031] To maintain the dynamic morphological rationality of the cleaned and reconstructed sequence and reduce the risk of over-smoothing and morphological distortion, a physical consistency regularization term is constructed, such as... Figure 4 As shown, the second-order difference between the cleaned and reconstructed sequence and the input sequence is based on the second-order difference constraint in the time dimension:

[0032]

[0033] By calculating the joint loss function and performing backpropagation, the network parameters are updated using an optimization algorithm to complete the model training optimization.

[0034] Step 3: The inference stage outputs the cleaning results and completes the quality score and threshold prompts, forming an application chain from residual calculation to robust scoring to threshold discrimination, and visualization output.

[0035] The bridge monitoring data to be processed is input into the model trained in step two in a windowed manner. A cleaned and reconstructed window of the same dimension as the input is output, and the reconstruction residual between the input window and the cleaned and reconstructed window is calculated. A window-level quality score is constructed based on the reconstruction residual. To reduce the impact of individual spikes or short-term strong disturbances on the score, the quality score uses a robust statistical method to aggregate the residuals, such as median aggregation, so that the score can stably represent the overall quality deviation of the window.

[0036] A quantile threshold is estimated on the score set of the training or health segment, and the window-level quality score obtained during the inference phase is compared with the threshold to generate a quality alert result. This quality alert result is used to identify the window locations in the monitoring sequence where quality anomalies such as noise contamination, drift contamination, or outlier mutations may exist. The quantile threshold can be a globally fixed threshold or a time-varying threshold grouped by hours to adapt to normal fluctuation differences in different time periods under daily cycle conditions, such as... Figure 5 As shown.

[0037] Furthermore, the original monitoring sequences, cleaned and reconstructed sequences, and quality indication results are jointly visualized, such as... Figure 6 As shown. (Through) Figure 6 This can be intuitively verified that during periods of quality anomalies, the cleaning and reconstruction sequence can effectively repair or suppress abnormal disturbances, while maintaining the trend and periodicity of the original sequence without significant alteration during uncontaminated periods. The quality alert results are consistent with the anomaly segments in terms of time location, enabling rapid localization and interpretation of anomaly periods. This forms a closed loop of "cleaning and reconstruction output—quality score quantification—threshold discrimination alert—visual verification," demonstrating the effectiveness and engineering applicability of the method for repairing quality anomalies in bridge monitoring data, and providing stable and reliable data input for subsequent predictive modeling and condition assessment.

[0038] The specific embodiments described herein are merely illustrative of the spirit and technical solutions of the invention and do not constitute any limitation on the scope of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

[0039] This invention provides a method applicable to the processing of multi-channel time-series data acquired by a bridge health monitoring system. The time-series data preferably includes deflection and strain data. The method input is a sliding window sample of the monitoring window, and the method output includes at least: a cleaned and reconstructed sequence of the same dimension as the input, along with corresponding quality scores and / or quality feedback results, thereby achieving integrated processing of the cleaned monitoring data output and quality assessment.

[0040] The method of the present invention can be implemented by a data quality assessment and reconstruction repair device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the bridge monitoring data cleaning and quality assessment method of the present invention. Furthermore, the method of the present invention can also be implemented by a computer program stored on a computer-readable storage medium, which, when executed by a processor, implements the method of the present invention.

Claims

1. A method for cleaning and quality assessment of bridge monitoring data based on mask reconstruction, characterized in that... Includes the following steps: Step 1: Acquire multi-channel time series data collected by the bridge health monitoring system, and perform sliding windowing processing on the multi-channel time series data to obtain window samples; Step 2: During the training phase, construct contaminated data for the window samples and generate a contaminated mask; Step 3: Construct a cleaning and reconstruction network and train and optimize it so that the cleaning and reconstruction network outputs a cleaning and reconstruction sequence of the same dimension for the input window samples, and the training and optimization adopts a joint objective function that includes mask constraints and physical consistency constraints; Step 4: During the inference phase, the data to be processed is input into the trained cleaning and reconstruction network to output the cleaning and reconstruction sequence. The quality score is calculated based on the reconstruction residual between the input window and the cleaning and reconstruction window, and the quality prompt result is output in combination with the threshold strategy.

2. The method for cleaning and quality assessment of bridge monitoring data based on mask reconstruction according to claim 1, characterized in that, Step two includes at least the following steps: The training contamination is constructed on the window samples obtained in step one, and the training contamination includes one or more of noise contamination, drift contamination and outlier mutation contamination. A contamination mask M with the same dimension as the window sample is generated based on the location of the contamination injection. The contamination mask M is a binary matrix, where M=1 represents the contamination location and M=0 represents the uncontamination location. When the training contamination is a mutation segment, the contamination mask M takes the value of 1 within the time interval corresponding to the mutation segment.

3. The method for cleaning and quality assessment of bridge monitoring data based on mask reconstruction according to claim 1, characterized in that, The training optimization in step three includes a mask constraint module and a physical consistency constraint module: The mask constraint module performs partition constraints on the training target based on the pollution mask M obtained in step two. The physical consistency constraint module cleans and reconstructs the difference between the input sequence and the time-dimensional second-order difference constraint.

4. The method for cleaning and quality assessment of bridge monitoring data based on mask reconstruction according to claim 3, characterized in that, The second-order difference definition and regularization term of the physical consistency constraint module are as follows: For input window sequence With cleaning and reconstructing window sequence Calculate the second difference along the time dimension separately: And based on the second-order difference, a physical consistency regularization term is constructed: Where L is the window length and C is the number of channels. It is used to suppress excessive smoothing during the cleaning and reconstruction process and to maintain the consistency of the sequence dynamic morphology.

5. The method for cleaning and quality assessment of bridge monitoring data based on mask reconstruction according to claim 3, characterized in that, The mask constraint module includes contaminated site repair loss and uncontaminated site preservation loss, and its calculation method is as follows: in, It is an element-wise error function, and it is either an absolute error or a squared error. Calculated only at position M=1 Calculated only at the position M=0.

6. The method for cleaning and quality assessment of bridge monitoring data based on mask reconstruction according to claim 3, characterized in that, The joint objective function is optimized using weighted joint optimization. in, To repair the damage at the contaminated site, To preserve the loss in uncontaminated locations, This is a physical consistency regularization term. , , These are the weighting coefficients.

7. The method for cleaning and quality assessment of bridge monitoring data based on mask reconstruction according to claim 1, characterized in that, The cleaning and reconstruction network is a temporal convolutional autoencoder network, which includes an encoder and a decoder. The encoder is used to extract the temporal features of the window samples, and the decoder is used to output the cleaning and reconstruction sequence of the same dimension.

8. The method for cleaning and quality assessment of bridge monitoring data based on mask reconstruction according to claim 7, characterized in that, The temporal convolutional autoencoder network employs dilated causal convolution and residual connections.

9. The method for cleaning and quality assessment of bridge monitoring data based on mask reconstruction according to claim 1, characterized in that, When restoring the window-level cleaning and reconstruction results to a full-time cleaning sequence, the overlapping areas of adjacent windows are fused. The fusion process involves taking the average or median of the overlapping areas point by point.

10. The method for cleaning and quality assessment of bridge monitoring data based on mask reconstruction according to claim 1, characterized in that, The quality scoring and threshold prompting in step four include at least the following steps: Calculate input window samples With cleaning and reconstructing window samples Reconstruction residuals between ,in ; Robust statistical aggregation of the reconstructed residuals yields a window-level quality score. .

11. The method for cleaning and quality assessment of bridge monitoring data based on mask reconstruction according to claim 10, characterized in that, The robust statistics are median aggregations: 。 12. The method for cleaning and quality assessment of bridge monitoring data based on mask reconstruction according to claim 10, characterized in that, Scoring set in the training or health phase Calculate the quantile threshold : in Indicates quantile as quantiles.

13. The method for cleaning and quality assessment of bridge monitoring data based on mask reconstruction according to claim 10, characterized in that, The quality score With the threshold Compare the output quality prompts. ,in: when hour This indicates a quality anomaly. when hour This indicates a quality issue.

Images