Dam monitoring data processing method and system

Abstract

The invention relates to the technical field of monitoring data processing, in particular to a dam monitoring data processing method and system. The method comprises the following steps: firstly, obtaining and preprocessing dam historical environment data and dam historical deformation data, and constructing a monitoring data set; secondly, reading the monitoring data set by using a deep FAR-LSTM network, and performing network training; then, reading real-time monitoring data by using a deep FAR-LSTM network, predicting dam deformation parameters, generating a report, and sending the report to a worker for confirmation; and if the dam deformation parameter is abnormal, adding an early warning signal in the report to prompt a worker. And finally, the deep FAR-LSTM network performs real-time updating of the model by using continuously updated monitoring data and deformation data to ensure that a change rule of parameters of an environment where the dam is located is met. According to the invention, the processing efficiency and accuracy of dam monitoring data can be improved.

Description

Technical Field

[0001] The invention relates to the technical field of monitoring data processing, and in particular to a dam monitoring data processing method and system. Background Art

[0002] With the development of social economy, the scale of water conservancy engineering buildings has been expanding. As a water retaining structure, the dam plays a series of outstanding functions in flood control and water supply. Therefore, reliability monitoring of the dam is of great significance to the evaluation of the safe operation of the dam.

[0003] In the prior art, there are many methods for processing dam monitoring data, such as using deep learning methods to predict the external forces that cause the dam body to deflect, in order to prevent the generation of false warning signals; and using graph attention networks to detect anomalies in dam monitoring data, in order to improve the accuracy of anomaly detection. For time series data such as dam monitoring data, the LSTM network is the most suitable, because LSTM can effectively capture long-term dependencies in time series data through its special memory units and gating mechanisms. This is especially important for dam monitoring data, because the state changes of the dam may be affected by long-term environmental and structural conditions. LSTM can be used to predict future trends in monitoring data, such as water level changes or stress distribution, so as to identify potential problems and detect abnormal situations in advance, such as sudden structural deformation or unexpected water level changes, which is crucial for dam safety. However, in the prior art, the LSTM neural network used for monitoring dam-related data and outputting abnormal warning signals is relatively simple, and it does not take into account that the model should attach different importance to data, which may cause redundant data in different data to interfere with the results of model prediction, resulting in inaccurate prediction and low efficiency.

[0004] Therefore, a dam monitoring data processing method and system are proposed. Summary of the invention

[0005] The object of the present invention is to provide a dam monitoring data processing method and system for processing dam monitoring data. First, the historical environmental data and historical deformation data of the dam are obtained; the historical environmental data and historical deformation data of the dam are preprocessed to construct a monitoring data set. Then, a deep FAR-LSTM network is used to read the monitoring data set and perform network training; the output predicted by the deep FAR-LSTM network is the dam deformation parameter. Next, the trained deep FAR-LSTM network is used to read the real-time monitoring data to predict the dam deformation parameters, and a report is generated and sent to the staff; if the dam deformation parameters are abnormal, an early warning is issued to the staff. Finally, the deep FAR-LSTM network uses the continuously updated monitoring data and deformation data to update the model in real time to ensure that it complies with the changing laws of the environmental parameters of the dam.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A dam monitoring data processing method and system, comprising:

[0008] Acquire the historical environmental data of the dam, including air temperature, rainfall, dam water level, flow velocity, water temperature data, and the historical deformation data of the dam, including horizontal and vertical displacement data of the dam, concrete strain data, and concrete stress data; pre-process the historical environmental data of the dam and the historical deformation data of the dam to construct a monitoring data set;

[0009] Using a deep FAR-LSTM network to read the monitoring data set and perform network training; the output predicted by the deep FAR-LSTM network is a dam deformation parameter;

[0010] The deep FAR-LSTM network includes multiple layers of NBiLSTM, each layer of NBiLSTM includes multiple BiLSTM;

[0011] The fast Fourier attention layer is used between each layer of NBiLSTM to transform the time-frequency features and capture the frequency-prominent features;

[0012] Using a residual block after the fast Fourier attention layer to add a nonlinear transformation function to capture deeper feature levels;

[0013] The trained deep FAR-LSTM network is used to read real-time monitoring data to predict dam deformation parameters, and a report is generated and sent to staff for confirmation; if the dam deformation parameters are abnormal, an early warning signal is added to the report to prompt the staff;

[0014] The deep FAR-LSTM network uses the continuously updated monitoring data and deformation data to perform real-time updates of the model to ensure compliance with the changing laws of the environmental parameters of the dam.

[0015] Among them, the preprocessing includes data cleaning, outlier processing and missing value filling to ensure that the data format of each data point is consistent and that the data is valid; the preprocessing also includes time alignment to ensure that all extracted data has an accurate timestamp; all data are aligned and integrated according to time points, and a time window is selected to intercept the aligned data to construct a monitoring data set.

[0016] Furthermore, the fast Fourier attention layer first reads the input features, performs fast Fourier transform on the input features, converts the input features from the time domain to the frequency domain, and obtains frequency domain features; further extracts features from the frequency domain features, captures prominent frequency features through Linear and tanh layers, and also pays attention to secondary features; converts the frequency domain features into weight vectors through the softmax layer, and multiplies them with the frequency domain features to achieve weight distribution; finally, the frequency domain features are converted to the time domain using the inverse fast Fourier transform to obtain output features:

[0017]

[0018] Where X[k] is the kth frequency component of the frequency domain signal; x[n] is the nth sample point of the time domain signal; is a complex exponential function, representing the basis function in the fast Fourier transform; j is an imaginary unit, satisfying j 2 = -1;

[0019]

[0020] in, represents the frequency domain features after weight distribution; σ() represents the linear layer operation; δ() represents the tanh layer operation; ζ() represents the softmax operation;

[0021]

[0022] Among them, Y[n] represents the output feature after the inverse fast Fourier transform, that is, the output of the fast Fourier attention layer.

[0023] Furthermore, the residual block includes a convolutional layer, a batch normalization layer and an activation function, and these nonlinear transformation functions are used to establish long-term dependencies in the time series:

[0024] Y c [n] = Conv(Y[n]);

[0025] Among them, Conv() represents the convolution layer operation; Y c [n] represents the output after the convolution layer operation;

[0026] Y N [n] = BatchNorm(Y c [n]);

[0027] Among them, BatchNorm() is the batch normalization layer operation; Y N [n] is the output of the batch normalization layer;

[0028] Y[n]=softmax(Y N [n]);

[0029] Among them, softmax() is the activation function; Y[n] is the output of the residual block.

[0030] Furthermore, the continuously updated monitoring data is the real-time environmental data read by the deep FAR-LSTM network; the continuously updated deformation data is the correct real deformation data used by the staff to verify the dam deformation parameters predicted by the deep FAR-LSTM network; the real-time environmental data and the real deformation data are preprocessed to obtain new data; the deep FAR-LSTM network performs incremental learning based on the new data to update the model parameters, and the specific steps are as follows:

[0031] Initialization: setting a buffer and a memory bank in the hard disk, the buffer bank is used to store the new data, and the memory bank is used to store the old data for a long time;

[0032] Data storage: storing the new data in the buffer and periodically transferring the new data in the buffer to the memory bank;

[0033] Data fusion: During the training process, a portion of historical data is taken out from the memory bank and fused with the new data to perform incremental updates of the model;

[0034] Model training: Use the fused data to update the deep FAR-LSTM network, retaining the memory of the old data and adapting to the new data.

[0035] A dam monitoring data processing system, comprising:

[0036] A monitoring data set construction unit is used to obtain historical environmental data and historical deformation data of the dam; pre-process the historical environmental data and historical deformation data of the dam, including data cleaning, outlier processing, missing value filling and time alignment, select a time window, intercept the aligned data, and construct a monitoring data set;

[0037] The model building unit builds a deep FAR-LSTM network, wherein the deep FAR-LSTM network includes three parts, namely, a main part, a frequency domain feature extraction part and a deep feature extraction part; the main part includes multiple layers of NBiLSTM, and each layer of NBiLSTM includes multiple BiLSTMs; the frequency domain feature extraction part uses a fast Fourier attention layer to convert time-frequency features to capture frequency-prominent features; the deep feature extraction part uses a residual block to perform nonlinear transformation on features to capture deeper feature levels;

[0038] A model training unit, which trains the deep FAR-LSTM network based on the monitoring data set;

[0039] The parameter prediction and early warning unit reads the real-time monitoring data based on the deep FAR-LSTM network, predicts the dam deformation parameters, and generates a report to send to the staff for confirmation; if the dam deformation parameters are abnormal, a warning signal is added to the report to prompt the staff;

[0040] The model updating unit obtains the continuously updated monitoring data and deformation data, and updates the network parameters of the deep FAR-LSTM network to ensure that the network parameters conform to the changing rules of the environmental parameters of the dam.

[0041] The preprocessing ensures that the data are all valid and have accurate timestamps.

[0042] Furthermore, the fast Fourier attention layer first reads the input features, performs fast Fourier transform on the input features, converts the input features from the time domain to the frequency domain, and obtains frequency domain features; further extracts features from the frequency domain features, captures prominent frequency features through Linear and tanh layers, and also pays attention to secondary features; converts the frequency domain features into weight vectors through the softmax layer, and multiplies them with the frequency domain features to achieve weight distribution; finally, the frequency domain features are converted to the time domain using the inverse fast Fourier transform to obtain output features:

[0043]

[0044] Where X[k] is the kth frequency component of the frequency domain signal; x[n] is the nth sample point of the time domain signal; is a complex exponential function, representing the basis function in the fast Fourier transform; j is an imaginary unit, satisfying j 2 = -1;

[0045]

[0046] in, represents the frequency domain features after weight distribution; σ() represents the linear layer operation; δ() represents the tanh layer operation; ζ() represents the softmax operation;

[0047]

[0048] Among them, Y[n] represents the output feature after the inverse fast Fourier transform, that is, the output of the fast Fourier attention layer.

[0049] Furthermore, the residual block includes a convolutional layer, a batch normalization layer and an activation function, and these nonlinear transformation functions are used to establish long-term dependencies in the time series:

[0050] Y c [n] = Conv(Y[n]);

[0051] Among them, Conv() represents the convolution layer operation; Y c [n] represents the output after the convolution layer operation;

[0052] Y N [n] = BatchNorm(Y c [n]);

[0053] Among them, BatchNorm() is the batch normalization layer operation; Y N [n] is the output of the batch normalization layer;

[0054]

[0055] Among them, softmax() is the activation function; is the output of the residual block.

[0056] Furthermore, the continuously updated monitoring data is the real-time environmental data read by the deep FAR-LSTM network; the continuously updated deformation data is the correct real deformation data used by the staff to verify the dam deformation parameters predicted by the deep FAR-LSTM network; the real-time environmental data and the real deformation data are preprocessed to obtain new data; the deep FAR-LSTM network performs incremental learning based on the new data to update the model parameters, and the specific steps are as follows:

[0057] Initialization: setting a buffer and a memory bank in the hard disk, the buffer bank is used to store the new data, and the memory bank is used to store the old data for a long time;

[0058] Data storage: storing the new data in the buffer and periodically transferring the new data in the buffer to the memory bank;

[0059] Data fusion: During the training process, a portion of historical data is taken out from the memory bank and fused with the new data to perform incremental updates of the model;

[0060] Model training: Use the fused data to update the deep FAR-LSTM network, retaining the memory of the old data and adapting to the new data.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] 1. The fast Fourier attention layer used in the present invention can effectively capture the periodicity and frequency components in the data by converting the time domain features to the frequency domain, which is particularly important for monitoring the periodic changes in the data, such as seasonal water level changes. In addition, the frequency domain conversion can highlight the important frequency features in the data, which helps to reduce the impact of noise on the model and help the model better identify and utilize these features, thereby improving the ability to understand complex time series data and making the model more robust to environmental changes and data noise.

[0063] 2. The residual block used in the present invention makes the gradient flow more easily during the back propagation process, thereby alleviating the gradient vanishing problem in the deep network and making it easier to train a deeper network structure. In addition, after the introduction of the residual block, a nonlinear change function is added to the network model, which can help the model better fit complex data patterns, thereby capturing more complex and deep feature levels and improving the expressiveness of the model.

[0064] 3. The present invention uses the continuously updated monitoring data and deformation data to update the model in real time. This method enables the model to continuously adapt to the changing environmental data and deformation data and maintain high prediction accuracy. Moreover, the fusion of historical data and new data can maintain the model's understanding of long-term trends, while introducing new data to cope with recent changes, enhancing the stability of the model and making the model more accurate and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 A flow chart of a dam monitoring data processing method of the present invention;

[0066] Figure 2 This is a network architecture diagram of the deep FAR-LSTM network of the present invention;

[0067] Figure 3 It is a structural diagram of the fast Fourier attention layer of the present invention;

[0068] Figure 4 is a structural diagram of the residual block of the present invention;

[0069] Figure 5 It is a structural diagram of a dam monitoring data processing system of the present invention. DETAILED DESCRIPTION

[0070] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] In order to improve the processing efficiency and accuracy of dam monitoring data, the present invention proposes a dam monitoring data processing method and system. In order to illustrate the function of the present invention, the effectiveness of the present invention will be described from the following examples.

[0072] Embodiment 1

[0073] This embodiment specifically analyzes a dam monitoring data processing method.

[0074] Reference Figure 1 In step S01, historical environmental data of the dam and historical deformation data of the dam are obtained; the historical environmental data of the dam and the historical deformation data of the dam are preprocessed to construct a monitoring data set.

[0075] Among them, the historical environmental data of the dam include temperature, rainfall, dam water level, flow rate, and water temperature data; the historical deformation data of the dam include horizontal and vertical displacement data of the dam, concrete strain data, and concrete stress data. Temperature data can be obtained from the local meteorological station to obtain temperature trend data; rainfall data is monitored by precipitation radar near the dam to obtain higher resolution and real-time rainfall data; dam water level data is monitored by ultrasonic sensors, which use the time difference between the propagation of ultrasonic waves between air and water or other media to measure the distance, thereby determining the height of the water surface; flow rate data is monitored by ultrasonic flow meters, which are installed upstream of the dam at a certain distance from the dam; water temperature data is provided by underwater temperature sensors; dam horizontal and vertical displacement data are statically observed using the global positioning system (GNSS); concrete strain data is obtained using strain gauges, and then concrete stress data is calculated using Hooke's law.

[0076] Furthermore, the collected historical environmental data and historical deformation data of the dam are preprocessed, and the preprocessing includes data cleaning, outlier processing and missing value filling to ensure that the data format of each data point is consistent and that the data is valid; the preprocessing also includes time alignment to ensure that all extracted data have accurate timestamps; align and integrate all data according to time points, and select a time window, such as a week, a month, or a year; intercept the aligned data, generate time series data, and construct a monitoring data set. The preprocessing methods are all prior art and will not be described in detail.

[0077] Data cleaning and format unification ensure data consistency and reduce errors or misunderstandings caused by inconsistent data formats. In addition, outlier processing and missing value filling improve data quality, and time alignment ensures accurate timestamps for all data points, making time series analysis and model training more effective. By selecting a time window and data interception, the relevance and usefulness of the monitoring data set are ensured, improving the training effect of the model.

[0078] Further, refer to Figure 1 In step S02, a deep FAR-LSTM network is used to read the monitoring data set and perform network training; the output predicted by the deep FAR-LSTM network is the dam deformation parameter.

[0079] Among them, the network architecture of the deep FAR-LSTM network is as follows Figure 2 As shown in the figure, the deep FAR-LSTM network includes multiple layers of NBiLSTM, each of which includes multiple BiLSTMs. The fast Fourier attention layer is used between each layer of NBiLSTM to convert time-frequency features and capture frequency-prominent features. The residual block is used after the fast Fourier attention layer to add a nonlinear change function to capture deeper feature levels.

[0080] Among them, the structure diagram of the fast Fourier attention layer is as follows Figure 3 As shown. The fast Fourier attention layer first reads the input features, performs a fast Fourier transform on the input features, and converts the input features from the time domain to the frequency domain:

[0081]

[0082] Where X[k] is the kth frequency component of the frequency domain signal; x[n] is the nth sample point of the time domain signal; is a complex exponential function, representing the basis function in the fast Fourier transform; j is an imaginary unit, satisfying j 2 = -1;

[0083] Then, the frequency domain features are further extracted, and the prominent frequency features are captured through the Linear and tanh layers, while paying attention to the secondary features. Then, the frequency domain features are converted into weight vectors through the softmax layer, and multiplied with the frequency domain features to achieve the distribution of weights:

[0084]

[0085] in, represents the frequency domain features after weight distribution; σ() represents the linear layer operation; δ() represents the tanh layer operation; ζ() represents the softmax operation;

[0086] Finally, the frequency domain features are transformed into the time domain using the inverse fast Fourier transform to obtain the output features:

[0087]

[0088] Among them, Y[n] represents the output feature after the inverse fast Fourier transform, that is, the output of the fast Fourier attention layer.

[0089] The fast Fourier attention layer can effectively extract frequency domain features from time domain data, identify periodicity and frequency information in the data, and reveal important change patterns in the data, thereby improving the accuracy of predictions. By using the fast Fourier attention layer to transform time-frequency features, the model can better understand and capture complex time series patterns. In addition, the fast Fourier transform has high computational efficiency, which can accelerate the feature extraction process, reduce training time, and optimize computational efficiency.

[0090] In addition to the fast Fourier attention layer, the deep FAR-LSTM network also uses a residual block. The structure of the residual block is shown in the figure Figure 4 As shown. The residual block includes convolutional layers, batch normalization layers, and activation functions, and these nonlinear transformation functions are used to establish long-term dependencies in time series:

[0091] Y c [n] = Conv(Y[n]);

[0092] Among them, Conv() represents the convolution layer operation; Y c [n] represents the output after the convolution layer operation;

[0093] Y N [n] = BatchNorm(Y c [n]);

[0094] Among them, BatchNorm() is the batch normalization layer operation; Y N[n] is the output of the batch normalization layer;

[0095]

[0096] Among them, softmax() is the activation function; is the output of the residual block.

[0097] The paths in the residual block provide a favorable gradient flow, allowing the deep network to converge faster, thereby shortening the training time. In addition, the residual block combines the original input and the transformed features, allowing the model to learn the complex relationship between the high-level features and low-level features of the input data, and better capture the long-term dependencies in the data. More importantly, the introduction of convolutional layers, batch normalization layers, and activation functions in the residual block enables the network to model more complex nonlinear relationships, thereby improving the model's ability to express time series data.

[0098] For the training of the deep FAR-LSTM network, the following parameters are set: the learning rate is set to 0.001, and a lower learning rate helps to stabilize the training process; the optimizer uses the Adam optimizer, which has an adaptive learning rate and momentum correction and can effectively handle gradient sparsity and imbalance; the number of training rounds is set to 500, and the model is saved every 50 rounds, and the model performance is verified on the test set; the batch size is set to 64, and a larger batch size can speed up training; the time series length of the input network can be set to 7, 30, 90, 180, and 360. In the FAR-LSTM network, the sequence length refers to the number of time steps of the input sequence. Here, 7, 30, 90, 180, and 360 represent the environmental and deformation data of the past day, month, quarter, half a year, and year, respectively; the hidden layer unit is set to 128; the Dropout rate is set to 0.2 to prevent overfitting; the loss function uses the mean square error loss function.

[0099] Further, refer to Figure 1 In step S03, the trained deep FAR-LSTM network is used to read the real-time monitoring data to predict the dam deformation parameters, and a report is generated and sent to the staff; if the dam deformation parameters are abnormal, an early warning is issued to the staff. The generated report includes the predicted dam deformation parameters, covering the change trends of the dam horizontal and vertical displacement data, concrete strain data and concrete stress data, and visualizes them; the visualization method includes but is not limited to statistical charts, trend analysis charts and other forms.

[0100] Furthermore, thresholds and abnormal standards are defined to determine whether the dam deformation parameters are abnormal. The judgment of the predicted dam horizontal displacement or vertical displacement data is as follows:

[0101]

[0102] Among them, Alert displacement It is the early warning signal generated for displacement data; displacement It represents the displacement data predicted by the network model; Threshold represents the set displacement threshold. The specific displacement threshold setting is shown in Table 1.

[0103] Table 1 Displacement threshold setting of dam

[0104]

[0105] The judgment on the concrete strain data of the predicted dam is as follows:

[0106]

[0107] Among them, Alert strain It is the early warning signal generated for concrete strain data; strain It represents the concrete strain data predicted by the network model; 0.005 is the strain threshold, that is, if the strain value of the concrete exceeds 0.005 (ie 0.5%), it is considered abnormal.

[0108] The judgment on the concrete stress data for predicting the dam is as follows:

[0109]

[0110] Among them, Alert stress It is the early warning signal generated for concrete stress data; stress represents the concrete strain data predicted by the network model; 0.2 is the stress change rate threshold; and Represent the stress values ​​at the current moment and the previous moment respectively.

[0111] The traditional LSTM, GRU and CNN-LSTM are selected for comparison with the deep FAR-LSTM network of this embodiment to prove the effectiveness of the deep FAR-LSTM network. The comparative experimental results are shown in Table 2.

[0112] Table 2 Comparative experimental results

[0113]

[0114] As can be seen from Table 2, the deep FAR-LSTM network has the lowest mean square error of 0.015, and the prediction time reaches 0.21 seconds, which is the most efficient. At the same time, the network model has strong generalization and the highest robustness. This comparative experiment proves the effectiveness of the deep FAR-LSTM network in processing dam monitoring data, which can improve the efficiency and accuracy of prediction.

[0115] Further, refer to Figure 1 In step S04, the deep FAR-LSTM network uses the continuously updated monitoring data and deformation data to update the model in real time to ensure that it conforms to the changing laws of the environmental parameters of the dam.

[0116] Among them, the continuously updated monitoring data is the real-time environmental data read by the deep FAR-LSTM network; the continuously updated deformation data is the correct real deformation data used by the staff to verify the dam deformation parameters predicted by the deep FAR-LSTM network; after data preprocessing, the real-time environmental data and the real deformation data are obtained to obtain new data; the deep FAR-LSTM network performs incremental learning based on the new data to update the model parameters. The specific steps are as follows:

[0117] Initialization: A buffer and a memory are set up in the hard disk. The buffer is used to store the new data, and the memory is used to store the old data for a long time. The purpose of the buffer is to temporarily store the new data that arrives until it is processed and can be transferred to long-term storage. The role of the memory is to save historical data for use in model training. The memory has a large storage capacity and is used to store a large amount of historical data.

[0118] Data storage: The new data is stored in the buffer and the new data in the buffer is transferred to the memory bank periodically. When new data arrives, they are first stored in the buffer and temporarily saved for subsequent processing. Then, according to the preset time interval or data volume, the new data in the buffer is regularly transferred to the memory bank, which ensures that the data in the memory bank is kept updated and avoids buffer overflow. At the same time, the data may need to be processed and cleaned during the transfer process to adapt to the storage format of the memory bank.

[0119] Data fusion: During the training process, a portion of historical data is taken out from the memory bank, which is usually the input and label data of the previous training of the model to provide stable historical information. Then, it is fused with the new data to perform incremental updates of the model. The purpose of data fusion is to combine historical data and new data so that the model can remember old knowledge and adapt to new information at the same time. Fusion methods can include data splicing, weighted averaging or other fusion techniques.

[0120] Model training: Use the fused data to update the deep FAR-LSTM network, retaining the memory of the old data and adapting to the new data.

[0121] This method of updating model parameters enables the model to quickly adapt to changes in environmental parameters and maintain the accuracy and relevance of predictions by updating and integrating new data in real time. The memory bank saves historical data, allowing the model to use past data for long-term trend analysis, thereby avoiding forgetting key historical information and the need to retrain the entire model, greatly improving training efficiency and the use of computing resources. The combination of buffer and memory bank helps smooth the model's learning process and reduce overfitting or instability caused by short-term data fluctuations.

[0122] In summary, the fast Fourier attention layer used in this embodiment captures key time-frequency patterns through frequency features, improving sensitivity to periodic features; in addition, the residual block allows information to be transmitted more effectively in the deep network, and can capture more complex nonlinear relationships; not only that, the real-time update of model parameters ensures that the model adapts to environmental changes and improves prediction accuracy. This dam detection data processing method improves the efficiency and accuracy of dam deformation prediction and enhances real-time monitoring and early warning capabilities.

[0123] Embodiment 2

[0124] This embodiment specifically analyzes a dam monitoring data processing system.

[0125] The structure diagram of a dam monitoring data processing system is as follows: Figure 5 As shown, it includes a monitoring data set construction unit, which is used to obtain the historical environmental data of the dam and the historical deformation data of the dam; pre-process the historical environmental data of the dam and the historical deformation data of the dam, including data cleaning, outlier processing, missing value filling and time alignment, select a time window, intercept the aligned data, and construct a monitoring data set.

[0126] The data cleaning method includes, first, checking and deleting duplicate records to avoid data redundancy; then identifying and correcting unreasonable numerical values ​​or format problems; and then unifying the date and time format and numerical unit format of the data;

[0127] The outlier processing method is, first, to detect outliers in the data using the statistical method IQR, and to identify values ​​exceeding 1.5 times the IQR using a box plot; then, to replace outliers with the median;

[0128] The missing value filling method is to first identify the missing parts in the data and represent them with null values; then, linearly interpolate the missing values ​​and estimate the missing values ​​based on adjacent data.

[0129] A dam monitoring data processing system also includes a model building unit for building a deep FAR-LSTM network, wherein the deep FAR-LSTM network includes three parts, namely a main part, a frequency domain feature extraction part and a deep feature extraction part; the main part includes multiple layers of NBiLSTM, each layer of NBiLSTM includes multiple BiLSTMs; the frequency domain feature extraction part uses a fast Fourier attention layer to convert time-frequency features to capture frequency-prominent features; the deep feature extraction part uses a residual block to perform nonlinear transformation on features to capture deeper feature levels.

[0130] Among them, the fast Fourier attention layer first reads the input features, performs fast Fourier transform on the input features, converts the input features from the time domain to the frequency domain, and obtains frequency domain features; further extracts features from the frequency domain features, captures prominent frequency features through Linear and tanh layers, and also pays attention to secondary features; converts the frequency domain features into weight vectors through the softmax layer, and multiplies them with the frequency domain features to achieve weight distribution; finally, the frequency domain features are converted to the time domain using the inverse fast Fourier transform to obtain the output features:

[0131]

[0132] Where X[k] is the kth frequency component of the frequency domain signal; x[n] is the nth sample point of the time domain signal; is a complex exponential function, representing the basis function in the fast Fourier transform; j is an imaginary unit, satisfying j 2 = -1;

[0133]

[0134] in, represents the frequency domain features after weight distribution; σ() represents the linear layer operation; δ() represents the tanh layer operation; ζ() represents the softmax operation;

[0135]

[0136] Among them, Y[n] represents the output feature after the inverse fast Fourier transform, that is, the output of the fast Fourier attention layer.

[0137] Furthermore, the residual block includes a convolutional layer, a batch normalization layer and an activation function, and these nonlinear transformation functions are used to establish long-term dependencies in the time series:

[0138] Y c [n] = Conv(Y[n]);

[0139] Among them, Conv() represents the convolution layer operation; Y c [n] represents the output after the convolution layer operation;

[0140] Y N [n] = BatchNorm(Y c [n]);

[0141] Among them, BatchNorm() is the batch normalization layer operation; Y N [n] is the output of the batch normalization layer;

[0142]

[0143] Among them, softmax() is the activation function; is the output of the residual block.

[0144] A dam monitoring data processing system also includes a model training unit for training the deep FAR-LSTM network based on the monitoring data set.

[0145] Among them, for the training of the deep FAR-LSTM network, the following parameters are set: the learning rate is set to 0.001, and a lower learning rate helps to stabilize the training process; the optimizer uses the Adam optimizer, which has an adaptive learning rate and momentum correction, and can effectively handle gradient sparsity and imbalance; the number of training rounds is set to 500, and the model is saved every 50 rounds, and the model performance is verified on the test set; the batch size is set to 64, and a larger batch size can speed up training; the time series length of the input network can be set to 7, 30, 90, 180, 360. In the FAR-LSTM network, the sequence length refers to the number of time steps of the input sequence. Here, 7, 30, 90, 180, and 360 represent the environmental and deformation data of the past day, month, quarter, half a year, and year, respectively; the hidden layer unit is set to 128; the Dropout rate is set to 0.2 to prevent overfitting; the loss function uses the mean square error loss function.

[0146] A dam monitoring data processing system also includes a parameter prediction and early warning unit, which reads real-time monitoring data based on the deep FAR-LSTM network, predicts dam deformation parameters, and generates a report to send to staff for confirmation; if the dam deformation parameters are abnormal, a warning signal is added to the report to prompt the staff.

[0147] The generated report includes the predicted dam deformation parameters, covering the changing trends of the dam's horizontal and vertical displacement data, concrete strain data and concrete stress data, and presents them visually; the visualization methods include but are not limited to statistical charts, trend analysis charts and other forms.

[0148] Furthermore, thresholds and abnormal standards are defined to determine whether the dam deformation parameters are abnormal. The judgment of the predicted dam horizontal displacement or vertical displacement data is as follows:

[0149]

[0150] Among them, Alert displacement It is the early warning signal generated for displacement data; displacement represents the displacement data predicted by the network model; Threshold represents the set displacement threshold, and the specific displacement threshold setting is shown in Table 1 in the embodiment.

[0151] The judgment on the concrete strain data of the predicted dam is as follows:

[0152]

[0153] Among them, Alert strain It is the early warning signal generated for concrete strain data; strain It represents the concrete strain data predicted by the network model; 0.005 is the strain threshold, that is, if the strain value of the concrete exceeds 0.005 (ie 0.5%), it is considered abnormal.

[0154] The judgment on the concrete stress data for predicting the dam is as follows:

[0155]

[0156] Among them, Alert stress It is the early warning signal generated for concrete stress data; stress represents the concrete strain data predicted by the network model; 0.2 is the stress change rate threshold; and Represent the stress values ​​at the current moment and the previous moment respectively.

[0157] A dam monitoring data processing system also includes a model updating unit, which obtains continuously updated monitoring data and deformation data, and updates the network parameters of the deep FAR-LSTM network to ensure that it complies with the changing laws of the environmental parameters of the dam.

[0158] Among them, the continuously updated monitoring data is the real-time environmental data read by the deep FAR-LSTM network; the continuously updated deformation data is the correct real deformation data used by the staff to verify the dam deformation parameters predicted by the deep FAR-LSTM network; after data preprocessing, the real-time environmental data and the real deformation data are obtained to obtain new data; the deep FAR-LSTM network performs incremental learning based on the new data to update the model parameters. The specific steps are as follows:

[0159] Initialization: setting a buffer and a memory bank in the hard disk, the buffer bank is used to store the new data, and the memory bank is used to store the old data for a long time;

[0160] Data storage: storing the new data in the buffer and periodically transferring the new data in the buffer to the memory bank;

[0161] Data fusion: During the training process, a portion of historical data is taken out from the memory bank and fused with the new data to perform incremental updates of the model;

[0162] Model training: Use the fused data to update the deep FAR-LSTM network, retaining the memory of the old data and adapting to the new data.

[0163] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A dam monitoring data processing method, characterized in that: include: Acquire the historical environmental data of the dam, including air temperature, rainfall, dam water level, flow velocity and water temperature data, and the historical deformation data of the dam, including horizontal and vertical displacement data of the dam, concrete strain data and concrete stress data; pre-process the historical environmental data of the dam and the historical deformation data of the dam to construct a monitoring data set; Using a deep FAR-LSTM network to read the monitoring data set and perform network training; the output predicted by the deep FAR-LSTM network is a dam deformation parameter; The deep FAR-LSTM network includes multiple layers of NBiLSTM, each layer of NBiLSTM includes multiple BiLSTM; The fast Fourier attention layer is used between each layer of NBiLSTM to transform the time-frequency features and capture the frequency-prominent features; Using a residual block after the fast Fourier attention layer to add a nonlinear transformation function to capture deeper feature levels; The trained deep FAR-LSTM network is used to read real-time monitoring data to predict dam deformation parameters, and a report is generated and sent to staff for confirmation; if the dam deformation parameters are abnormal, an early warning signal is added to the report to prompt the staff; The deep FAR-LSTM network uses the continuously updated monitoring data and deformation data to perform real-time updates of the model to ensure compliance with the changing laws of the environmental parameters of the dam.

2. A dam monitoring data processing method according to claim 1, characterized in that: The preprocessing includes data cleaning, outlier processing and missing value filling to ensure that the data format of each data point is consistent and that the data is valid; the preprocessing also includes time alignment to ensure that all extracted data has an accurate timestamp; all data are aligned and integrated according to time points, and a time window is selected to intercept the aligned data to construct a monitoring data set.

3. A dam monitoring data processing method according to claim 1, characterized in that: The fast Fourier attention layer first reads the input features, performs fast Fourier transform on the input features, converts the input features from the time domain to the frequency domain, and obtains frequency domain features; The frequency domain features are further extracted, and the prominent frequency features are captured through the Linear and tanh layers, while the secondary features are also given attention; the frequency domain features are converted into weight vectors through the softmax layer, and multiplied with the frequency domain features to achieve the distribution of weights; finally, the frequency domain features are converted to the time domain using the inverse fast Fourier transform to obtain the output features: Where X[k] is the kth frequency component of the frequency domain signal; x[n] is the nth sample point of the time domain signal; is a complex exponential function, representing the basis function in the fast Fourier transform; j is an imaginary unit, satisfying j 2 = -1; in, represents the frequency domain features after weight distribution; σ() represents the linear layer operation; δ() represents the tanh layer operation; ζ() represents the softmax operation; Among them, Y[n] represents the output feature after the inverse fast Fourier transform, that is, the output of the fast Fourier attention layer.

4. A dam monitoring data processing method according to claim 1, characterized in that: The residual block includes convolutional layers, batch normalization layers, and activation functions, which use these nonlinear transformation functions to establish long-term dependencies in time series: AND c [n]=Conv(Y[n]); Among them, Conv() represents the convolution layer operation; Y c [n] represents the output after the convolution layer operation; Y N [n]=BatchNorm(Y c [n]); Among them, BatchNorm() is the batch normalization layer operation; Y N [n] is the output of the batch normalization layer; Among them, softmax() is the activation function; is the output of the residual block.

5. A dam monitoring data processing method according to claim 1, characterized in that: The continuously updated monitoring data is the real-time environmental data read by the deep FAR-LSTM network; the continuously updated deformation data is the correct real deformation data after the staff verifies the dam deformation parameters predicted by the deep FAR-LSTM network; After data preprocessing of the real-time environment data and the real deformation data, new data is obtained; the deep FAR-LSTM network performs incremental learning according to the new data to update the model parameters, and the specific steps are as follows: Initialization: setting a buffer and a memory bank in the hard disk, the buffer bank is used to store the new data, and the memory bank is used to store the old data for a long time; Data storage: storing the new data in the buffer and periodically transferring the new data in the buffer to the memory bank; Data fusion: During the training process, a portion of historical data is taken out from the memory bank and fused with the new data to perform incremental updates of the model; Model training: Use the fused data to update the deep FAR-LSTM network, retaining the memory of the old data and adapting to the new data.

6. A dam monitoring data processing system, characterized in that: include: Monitoring data set construction unit, to obtain the dam's historical environmental data and dam's historical deformation data; Preprocessing the historical environmental data of the dam and the historical deformation data of the dam, including data cleaning, outlier processing, missing value filling and time alignment, selecting a time window, intercepting the aligned data, and constructing a monitoring data set; The model building unit builds a deep FAR-LSTM network, wherein the deep FAR-LSTM network includes three parts, namely, a main part, a frequency domain feature extraction part and a deep feature extraction part; the main part includes multiple layers of NBiLSTM, and each layer of NBiLSTM includes multiple BiLSTMs; the frequency domain feature extraction part uses a fast Fourier attention layer to convert time-frequency features to capture frequency-prominent features; the deep feature extraction part uses a residual block to perform nonlinear transformation on features to capture deeper feature levels; A model training unit, which trains the deep FAR-LSTM network based on the monitoring data set; The parameter prediction and early warning unit reads the real-time monitoring data based on the deep FAR-LSTM network, predicts the dam deformation parameters, and generates a report to send to the staff for confirmation; if the dam deformation parameters are abnormal, a warning signal is added to the report to prompt the staff; The model updating unit obtains the continuously updated monitoring data and deformation data, and updates the network parameters of the deep FAR-LSTM network to ensure that the network parameters conform to the changing rules of the environmental parameters of the dam.

7. A dam monitoring data processing system according to claim 6, characterized in that: The data cleaning method is to first check and delete duplicate records to avoid data redundancy; then identify and correct unreasonable numerical values ​​or format problems; and then unify the date and time format and numerical unit format of the data; The outlier processing method is, first, to detect outliers in the data using the statistical method IQR, and to identify values ​​exceeding 1.5 times the IQR using a box plot; then, to replace outliers with the median; The missing value filling method is to first identify the missing parts in the data and represent them with null values; then, linearly interpolate the missing values ​​and estimate the missing values ​​based on adjacent data.

8. A dam monitoring data processing system according to claim 6, characterized in that: The fast Fourier attention layer first reads the input features, performs fast Fourier transform on the input features, converts the input features from the time domain to the frequency domain, and obtains frequency domain features; The frequency domain features are further extracted, and the prominent frequency features are captured through the Linear and tanh layers, while the secondary features are also given attention; the frequency domain features are converted into weight vectors through the softmax layer, and multiplied with the frequency domain features to achieve the distribution of weights; finally, the frequency domain features are converted to the time domain using the inverse fast Fourier transform to obtain the output features: Where X[k] is the kth frequency component of the frequency domain signal; x[n] is the nth sample point of the time domain signal; is a complex exponential function, representing the basis function in the fast Fourier transform; j is an imaginary unit, satisfying j 2 = -1; in, represents the frequency domain features after weight distribution; σ() represents the linear layer operation; δ() represents the tanh layer operation; ζ() represents the softmax operation; Among them, Y[n] represents the output feature after the inverse fast Fourier transform, that is, the output of the fast Fourier attention layer.

9. A dam monitoring data processing system according to claim 6, characterized in that: The residual block includes convolutional layers, batch normalization layers, and activation functions, which use these nonlinear transformation functions to establish long-term dependencies in time series: AND c [n]=Conv(Y[n]); Among them, Conv() represents the convolution layer operation; Y c [n] represents the output after the convolution layer operation; Y N [n]=BatchNorm(Y c [n]); Among them, BatchNorm() is the batch normalization layer operation; Y N [n] is the output of the batch normalization layer; Among them, softmax() is the activation function; is the output of the residual block.

10. A dam monitoring data processing system according to claim 6, characterized in that: The continuously updated monitoring data is the real-time environmental data read by the deep FAR-LSTM network; the continuously updated deformation data is the correct real deformation data used by the staff to verify the dam deformation parameters predicted by the deep FAR-LSTM network; After data preprocessing of the real-time environment data and the real deformation data, new data is obtained; the deep FAR-LSTM network performs incremental learning according to the new data to update the model parameters, and the specific steps are as follows: Initialization: setting a buffer and a memory bank in the hard disk, the buffer bank is used to store the new data, and the memory bank is used to store the old data for a long time; Data storage: storing the new data in the buffer and periodically transferring the new data in the buffer to the memory bank; Data fusion: During the training process, a portion of historical data is taken out from the memory bank and fused with the new data to perform incremental updates of the model; Model training: Use the fused data to update the deep FAR-LSTM network, retaining the memory of the old data and adapting to the new data.

Images