A method for abnormal diagnosis and repair of bridge monitoring data

Abstract

The present invention discloses a method for diagnosing and repairing abnormal bridge monitoring data. The method first collects and preprocesses multi-point sensor data, and normalizes the data to eliminate interference from the absolute value of the amplitude. Then, the abnormal feature of the 11-dimensional feature vector is extracted from the normalized data; then, a BiLSTM neural network model is constructed, and the network is trained. When the prediction accuracy is greater than or equal to 95%, the network is saved for subsequent abnormal identification and positioning of real-time data. The real-time data is input into the model to identify and locate the abnormal data segment to complete the first stage of data abnormality identification. Finally, the training data of the adversarial network model is generated based on the data of appropriate length before the abnormality as a condition, and the multi-sensor correlation model obtained by model training is used. The normal sensor data in the abnormal data period is used as input, and the data of the abnormal sensor in this period is predicted as a filler replacement for the abnormal data segment, thereby realizing the full process automatic identification and repair of the abnormal trend of bridge monitoring data.

Description

Technical Field

[0001] The present invention relates to a method for diagnosing and repairing abnormal bridge monitoring data, and belongs to the technical field of cleaning and repairing bridge structural health monitoring data. Background Art

[0002] The premise for the bridge monitoring system to function is to obtain various data that truly reflect the structural performance. However, during the processes of data measurement, collection, transmission, storage, etc., it is inevitably affected by various adverse factors such as sensor aging faults, environmental noise interference, transmission packet loss, and compression distortion, resulting in distortion and anomalies in the monitoring data in terms of amplitude, distribution, and trend. The main types of data anomalies include missing, jump points, noise, drift, and trend anomalies. Among them, the first four can be effectively identified by comparing with the historical data of the sensor itself. However, for trend anomalies, it is often a phenomenon that the data is disturbed by long-term periods, causing large-area continuous deviation from the normal value. Due to the continuous abnormal temperature effect on the structure, it is possible for the response data to show continuous deviation. Therefore, this type of anomaly is difficult to directly distinguish only from its own historical data, and such data usually appears continuously in a large area, and it is extremely difficult to perform real-time repair only from the perspective of autoregression. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a method for diagnosing and repairing abnormal bridge monitoring data, which first uses a supervised learning paradigm to realize the identification and positioning of abnormal data, and on this basis, establishes a generative adversarial neural network model to predict and replace abnormal data segments, completing the full-process automatic identification and repair of sensor data anomalies.

[0004] The present invention adopts the following technical solutions to solve the above technical problems:

[0005] A method for diagnosing and repairing abnormal bridge monitoring data includes the following steps:

[0006] Step 1, sensors are set at each key section of the bridge, and the types of sensors set at all key sections are the same. The sensors are numbered, and the data of each sensor is collected and preprocessed.

[0007] Step 2, the preprocessed sensor data is normalized to obtain normalized data.

[0008] Step 3, for the normalized data, taking minutes as the time scale, an 11-dimensional feature vector of each normalized data is extracted. The 11-dimensional feature vector includes: average value, median, standard deviation, root mean square, amplitude, difference value, 20th percentile value, 80th percentile value, ratio of maximum value to 80th percentile value, ratio of average value to amplitude, and ratio of average value to 80th percentile value.

[0009] Step 4: Construct a bidirectional long short-term memory neural network model, and train the bidirectional long short-term memory neural network model using the pre-labeled tag data to obtain a trained bidirectional long short-term memory neural network model; input the 11-dimensional feature vector extracted in Step 3 into the trained bidirectional long short-term memory neural network model for diagnosing abnormal data. When abnormal data is diagnosed, proceed to Step 5;

[0010] Step 5: Blank out the data corresponding to the time period with abnormal data and fill it with nan. Use the data of all sensors in the 3 hours before the occurrence of the abnormality as the dataset to train a conditional generative adversarial network model, and obtain the non-linear relationship between the data of the abnormal sensor before the abnormality and the data of other normal sensors in the same time period. Input the data with abnormal data time periods filled with nan and the data of other normal sensors in the abnormal time period into the conditional generative adversarial network model to predict the data of the abnormal sensor in the abnormal time period, and replace the nan with the predicted data to obtain the repaired data.

[0011] As a preferred embodiment of the present invention, the key cross-sections described in Step 1 include the 1 / 4 cross-section, the mid-span cross-section, the 3 / 4 cross-section, and the top cross-section of the main tower of the cable-stayed bridge.

[0012] As a preferred embodiment of the present invention, the preprocessing of the sensor data described in Step 1 is specifically as follows: intercept the data collected by each sensor to ensure that the data lengths corresponding to each sensor are the same and do not contain nan values.

[0013] As a preferred embodiment of the present invention, for the normalization processing described in Step 2, the method used is the deviation normalization method, and the specific formula is as follows:

[0014]

[0015] where x (i,j) * represents x (i,j) the normalized data, x (i,j) represents the preprocessed data of the sensor numbered j at the i-th key cross-section, x i represents the set of the preprocessed data of all sensors.

[0016] As a preferred embodiment of the present invention, the bidirectional long short-term memory neural network model described in Step 4 includes a first bidirectional LSTM layer, a second bidirectional LSTM layer, a third bidirectional LSTM layer, a fully connected layer, and a Softmax layer connected in sequence.

[0017] As a preferred embodiment of the present invention, the loss function of the bidirectional long short-term memory neural network model described in Step 4 uses the cross-entropy loss function, specifically as follows:

[0018]

[0019] Among them, Loos represents the cross-entropy loss function; y is the sample label. If the sample belongs to the positive example, that is, the data is abnormal, its value is 1; otherwise, the data is normal, and its value is 0. is the probability that the model predicts the sample as a positive example.

[0020] When training the bidirectional long short-term memory neural network model, save the model when its classification accuracy is greater than 95%. As the trained bidirectional long short-term memory neural network model, the classification accuracy is specifically:

[0021] Accuracy = (TP + TN) / (ALL)

[0022] Among them, Accuracy represents the classification accuracy, TP represents the number of true positive classifications, TN represents the number of true negative classifications, and ALL represents the total number of labels in the pre-labeled tag data.

[0023] As a preferred solution of the present invention, for the conditional generative adversarial network model described in step 5, the objective function for its network optimization is expressed as:

[0024]

[0025] Among them, G and D respectively represent the generator and the discriminator, V represents the objective function, E represents the distribution expectation, f represents the conditional vector, t represents the input, p data (t) represents the real sample, z represents the initial generated noise, p z (z) represents the prior noise distribution.

[0026] Compared with the prior art, the present invention adopts the above technical solutions and has the following technical effects:

[0027] 1. Based on the multi-sensor data correlation, the present invention identifies the abnormal sensors and accurately locates the abnormal data segments at the minute scale. Thanks to the network architecture proposed by the present invention and the sensitive characteristics of the BiLSTM network to features of different time lengths, it can accurately identify different types of data anomalies at different time scales, such as jump point and trend anomalies. Then, using the appropriate length of data before the anomaly as the training data for the conditional generative adversarial network model, and based on the multi-sensor correlation model obtained from model training, using the normal sensor data during the abnormal data period as the input, predict the data of the abnormal sensor during this period as the filling and replacement of the abnormal data segment, so as to realize the full-process automatic identification and repair of the trend anomaly of bridge monitoring data.

[0028] 2. The present invention proposes a two-stage monitoring data repair method based on data correlation, using a bidirectional LSTM model and a conditional generative adversarial network model, which realizes the unification of data anomaly identification and automatic repair, and forms a full-process automated data cleaning method. Moreover, thanks to the given feature extraction method and correlation repair method, it can effectively identify and repair anomaly types such as trend anomalies that are difficult to distinguish only relying on the historical data of the data itself traditionally.

[0029] 3. For large-area data anomalies, the present invention constructs a generative adversarial neural network model, which can effectively learn the multi-sensor correlation mapping relationship. Based on this model and the data of other sensors during the abnormal period, it can make good predictions for the abnormal data segment, thereby completing the repair of the large-area anomaly situation, and the repaired data for the large area is still reliable and accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flowchart of a method for diagnosing and repairing abnormal bridge monitoring data according to the present invention;

[0031] Figure 2 is a BiLSTM classification model for anomaly identification according to the present invention;

[0032] Figure 3 is a conditional generative adversarial network model for data repair according to the present invention;

[0033] Figure 4 is the simulation verification result of the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] The following details the embodiments of the present invention, and the examples of the embodiments are shown in the drawings. The embodiments described below with reference to the drawings are exemplary and are only used to explain the present invention, and cannot be construed as a limitation of the present invention.

[0035] Since the sensors at different positions of the same bridge structure have significant correlations in the amplitude and variation law of the data, there is a good theoretical basis for judging data trend anomalies based on data correlation. Based on data correlation, using a long short-term neural memory network to capture the abnormal features of data signals in the time-frequency domain has the technical prospect of quickly realizing the anomaly judgment of a large amount of data. The feasibility of this method is also verified through the statistical analysis of a large amount of data. At the same time, the generative adversarial neural network can obtain the correlation relationship model between the concerned sensor and other sensors before the anomaly occurs. With the help of this model and the data of multiple sensors during this period except the concerned sensor after the anomaly occurs, the true data of the abnormal sensor in the abnormal data segment can be accurately predicted. This provides a reliable basis for the identification and repair of data anomalies in monitoring data.

[0036] Such as Figure 1As shown in the figure, the present invention proposes a method for abnormal diagnosis and repair of bridge monitoring data based on a data correlation conditional generative adversarial network, which mainly includes the following steps:

[0037] (1) Data collection and processing. Collect the data of multiple sensors at multiple positions of the bridge, number the sensor data, and perform preprocessing on the data collected by the sensors;

[0038] The preprocessing work includes intercepting the data collected by the sensors to ensure that the length of each sensor data is the same and does not contain nan values.

[0039] (2) Normalize all the above sensor data. The deviation normalization method is used for normalization to eliminate the error caused by the absolute value. The specific formula is as follows:

[0040]

[0041] where, x (i,j) * represents x (i,j) the normalized data, x (i,j) represents the preprocessed data of the sensor numbered j at the i-th key cross-section, x i represents the set of all preprocessed sensor data.

[0042] (3) Extract features, calculate an 11-dimensional feature vector at a scale of one minute, including: mean value, median value, standard deviation, root mean square, amplitude (the difference between the maximum value and the minimum value), difference value, 20th percentile value, 80th percentile value, the ratio of the maximum value to the 80th percentile value, the ratio of the mean value to the amplitude, and the ratio of the mean value to the 80th percentile value, for dimensionality reduction and feature extraction of the original data; the 11-dimensional features proposed by the present invention are key feature parameters that are extremely sensitive to abnormal bridge monitoring data after repeated screening based on a 40-statistical feature index library, and have significant benefits for improving the model accuracy and reducing the training cost;

[0043] (4) Locate abnormal data. As Figure 2 shown, construct a bidirectional long short-term memory neural network model BiLSTM containing 3 bidirectional LSTM layers, a fully connected layer, and a Softmax layer for classifying abnormal data. This network architecture is the exploratory attempt result of the long short-term memory neural network model in the data anomaly classification task, which not only meets the training set accuracy but also takes into account preventing overfitting. Use the label data to train the model, and save the model when the classification accuracy reaches more than 95% for automatic anomaly recognition of new data;

[0044] The loss function of the BiLSTM model uses the cross-entropy loss function:

[0045]

[0046] Among them, Loos represents the cross-entropy loss function; y is the sample label. If the sample belongs to the positive example, that is, the data is abnormal, its value is 1; otherwise, the data is normal, and its value is 0. is the probability that the model predicts the sample as a positive example.

[0047] When training the bidirectional long short-term memory neural network model, save the model when its classification accuracy is greater than 95% for the prediction classification of new data. The specific classification accuracy is:

[0048] Accuracy = (TP + TN) / (ALL)

[0049] Among them, Accuracy represents the classification accuracy, TP represents the number of true positive classifications, TN represents the number of true negative classifications, and ALL represents the total number of labels in the pre-labeled tag data.

[0050] (5) Data repair. Hollow out the data in the abnormal period and fill it with nan. Use all the data in the data volume of the 3 hours before the anomaly as the dataset training condition to generate an adversarial network (CGAN) model, and obtain the non-linear relationship between the abnormal sensor and other normal sensor data before the anomaly. Then fill the data in the abnormal data period of the sensor with nan, and use it together with the data of all other normal similar sensors in the same period as the model input data to predict the hollowed-out section data of the abnormal sensor. Finally, fill the hollowed-out section data with the predicted data to obtain the repaired data;

[0051] Such as Figure 3 shown, in the generator G, the prior noise distribution P z (z) and the conditional vector y are combined as the input of the first layer of the neural network. Usually, the adversarial training framework has considerable flexibility for the composition of this implicit representation; in the discriminator D, the input t from the real sample p data (t) and the conditional vector f are used as the input of the discriminant function (usually a multi-layer perceptron). The objective function for its network optimization can be expressed as:

[0052]

[0053] Among them, G and D represent the generator and the discriminator respectively, E represents the distribution expectation, f represents the conditional vector, t represents the input, and z represents the initial generated noise.

[0054] Embodiment

[0055] Next, take the discrimination and repair of the GPS data trend anomaly of the cross-river bridge as an example to illustrate the specific implementation process of the present invention.

[0056] (1) Data acquisition and processing. Data at each key section of the bridge is collected, including the positions of the 1 / 4, mid-span, 3 / 4 sections and the top section of the main tower of the cable-stayed bridge. There are 2 sensors at each section, for a total of 8 sensors numbered 1 - 8.

[0057] (2) Normalization processing. Normalize the data of all 8 GPS sensors of the same type for one day to eliminate the errors caused by the absolute values of the coordinate positions at different interfaces.

[0058] (3) Divide the basic recognition scale by the data volume per hour. Then, extract 11-dimensional characteristic parameters of each sensor data with a time scale of minutes, including: mean value, median, standard deviation, root mean square, amplitude (the difference between the maximum and minimum values), difference value, 20th percentile value, 80th percentile value, the ratio of the maximum value divided by the 80th percentile value, the ratio of the mean value to the amplitude (the difference between the maximum and minimum values), and the ratio of the mean value to the 80th percentile value.

[0059] (4) Data anomaly identification and classification. Based on the data anomaly classification model saved by the above-mentioned modeling method, perform automated classification and positioning of data anomalies. Input the characteristic matrix corresponding to each sensor to identify the data anomaly sensors and the time when the data anomalies occur.

[0060] (5) Data repair. Hollow out the data during the abnormal period and fill it with nan. Train a conditional generative adversarial network model with all the data of the same type in the 3 hours before the anomaly occurs to obtain the non-linear relationship between the data of the abnormal sensor before the anomaly and the data of all other sensors. Then, use the sensor data filled with nan and the data of all other data of the same type during this period as the input data of the model to predict the data of the hollowed-out section of the abnormal sensor. Finally, fill the hollowed-out section with the predicted data to obtain the repaired data. Figure 4 This is the result of simulation verification.

[0061] The above embodiments are only used to illustrate the technical idea of the present invention, and the protection scope of the present invention cannot be limited thereby. Any changes made on the basis of the technical solution according to the technical idea proposed by the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for abnormal diagnosis and repair of bridge monitoring data, characterized in that It includes the following steps: Step 1: Set sensors at each key section of the bridge. All sensors set at key sections are of the same type. Number the sensors, collect data from each sensor, and preprocess the sensor data; Step 2: Perform normalization processing on the preprocessed sensor data to obtain normalized data; Step 3: For the normalized data, with minutes as the time scale, extract the 11-dimensional feature vectors of each normalized data. The 11-dimensional feature vectors include: mean value, median value, standard deviation, root mean square, amplitude value, difference value, 20th percentile value, 80th percentile value, ratio of maximum value to 80th percentile value, ratio of mean value to amplitude, and ratio of mean value to 80th percentile value; Step 4: Construct a bidirectional long short-term memory neural network model. Use the pre-labeled tag data to train the bidirectional long short-term memory neural network model to obtain a trained bidirectional long short-term memory neural network model; Input the 11-dimensional feature vectors extracted in Step 3 into the trained bidirectional long short-term memory neural network model for diagnosing abnormal data. When abnormal data is diagnosed, go to Step 5; The loss function of the bidirectional long short-term memory neural network model adopts the cross-entropy loss function, specifically: Among them, Loos represents the cross-entropy loss function; y is the sample label. If the sample belongs to the positive example, that is, the data is abnormal, its value is 1; otherwise, the data is normal, and its value is 0. is the probability that the model predicts the sample as a positive example. When training the bidirectional long short-term memory neural network model, save the model when its classification accuracy is greater than 95% as the trained bidirectional long short-term memory neural network model. The classification accuracy is specifically: Accuracy=(TP + TN) / (ALL) Where, Accuracy represents the classification accuracy, TP represents the number of true positive classifications, TN represents the number of true negative classifications, and ALL represents the total number of tags in the pre-labeled tag data; Step 5: Hollow out the data corresponding to the abnormal data period and fill it with nan. Use the data of all sensors in the 3 hours before the abnormality as the dataset to train the conditional generative adversarial network model, obtain the non-linear relationship between the data of the abnormal sensor before the abnormality and the data of other normal sensors in the same period. Input the data with abnormal data period filled with nan and the data of other normal sensors in the abnormal period into the conditional generative adversarial network model to predict the data of the abnormal sensor in the abnormal period, and replace nan with the predicted data to obtain the repaired data; For the conditional generative adversarial network model, the objective function for network optimization is expressed as: Among them, G and D represent the generator and discriminator respectively, V represents the objective function, E represents the distribution expectation, f represents the conditional vector, and t represents the input. pdata(t) represents the real sample, and z represents the initial generated noise. pz(z) represents the prior noise distribution.

2. The abnormal diagnosis and repair method for bridge monitoring data according to claim 1, characterized in that, The key sections in Step 1 include the 1 / 4 section, mid-span section, 3 / 4 section, and the top section of the main tower of the cable-stayed bridge.

3. The abnormal diagnosis and repair method of bridge monitoring data according to claim 1, characterized in that The preprocessing of the sensor data in Step 1 is specifically: Perform data interception on the data collected by each sensor to ensure that the data length corresponding to each sensor is the same and does not contain nan values.

4. The abnormal diagnosis and repair method for bridge monitoring data according to claim 1, characterized in that The normalization processing in Step 2 adopts the method of deviation standardization, and the specific formula is as follows: where x (i,j) * represents x (i,j) the normalized data, x (i,j) represents the preprocessed data of the sensor numbered j at the i-th key section, x i represents the set of the preprocessed data of all sensors.

5. The abnormal diagnosis and repair method for bridge monitoring data according to claim 1, characterized in that, The bidirectional long short-term memory neural network model in Step 4 includes a first bidirectional LSTM layer, a second bidirectional LSTM layer, a third bidirectional LSTM layer, a fully connected layer, and a Softmax layer connected in sequence.

Images