A water supply pipeline operation data anomaly detection method

Abstract

A kind of water supply pipeline operation data anomaly detection method, specific steps are as follows: S1: data acquisition and preprocessing;S2: construct CNN-LSTM model based on Attention mechanism;S3: training and verifying model;S4: anomaly detection.The method of the application combines the characteristics of CNN and LSTM time series data anomaly detection model, can learn the correlation between the space of multiple features of data through CNN, and LSTM can effectively avoid the problem of gradient vanishing or explosion by its internal gating mechanism, so as to process longer span time series data, while introducing Attention mechanism, weight distribution is carried out to input sequence, so that some important features in sequence are more easily captured, thereby further reducing the error of model.

Description

Technical Field

[0001] This invention relates to the field of water supply pipeline transportation, and specifically to a method for detecting abnormal operation data of water supply pipelines. Background Technology

[0002] Currently, the most commonly used methods for anomaly detection in water supply pipeline operation data are machine learning and deep learning methods. In the former, LOF, OC-SVM, and SVDD are commonly used anomaly detection methods. However, these methods can only detect spatial anomalies, that is, the distribution of abnormal data and normal data is significantly different in the numerical value range, without utilizing temporal information. In the latter, such as the traditional RNN model, it can capture short-term dependencies in time series data very well. However, accurate anomaly detection often requires obtaining dependencies with a long span of time series data. At this time, RNN will face problems such as a sharp increase in computational complexity, gradient vanishing or exploding. Summary of the Invention

[0003] The technical problem to be solved by this invention is to provide a method for detecting anomalies in water supply pipeline operation data, addressing the shortcomings mentioned in the background art. This method combines the characteristics of CNN (Convolutional Neural Network) and LSTM (Long Short-Term Memory) time-series data anomaly detection models. It can learn the spatial correlation between multiple features of data through CNN, and LSTM, with its internal gating mechanism, can effectively avoid the problems of gradient vanishing or exploding, thus enabling the processing of time-series data with a longer span. At the same time, the introduction of the Attention mechanism to assign weights to the input sequence makes it easier to capture certain important features in the sequence, thereby further reducing the model error.

[0004] The technical solution adopted by this invention to solve its technical problem is: a method for detecting abnormal operation data of water supply pipelines, the specific steps of which are as follows:

[0005] S1: Data acquisition and preprocessing;

[0006] Water supply pipeline operation data is collected by the online data acquisition and transmission unit and sent to the data center. The data is then processed sequentially, including time series missing value imputation, time series denoising, outlier detection, and analysis of major correlation factors.

[0007] S2: Construct a CNN-LSTM model based on the Attention mechanism;

[0008] The CNN-LSTM model based on the attention mechanism is an unsupervised anomaly detection model that combines the characteristics of CNN (Convolutional Neural Network) and LSTM (Long Short-Term Memory) time-series data anomaly detection models. It can learn the spatial correlations between multiple features of data through CNN, and capture the temporal dependencies of data using LSTM. Furthermore, the introduction of the attention mechanism assigns weights to the input sequence, making it easier to capture certain important features, thereby further reducing the model's error. Its structure is as follows:

[0009] The first layer is the input layer, which defines the format of the input data: batch size, time steps, and feature dimension. The batch size is set to 1 by default, the time steps are denoted as t, and the feature dimension is denoted as n. Then, a sample can be represented as a real number sequence matrix. ,remember for Vector representation of the data at the i-th time step;

[0010] The second layer is the CNN layer. The CNN layer can learn the spatial correlation between multiple features of the data, making up for the shortcomings of LSTM in capturing the spatial components of the data. At the same time, the features it extracts still have temporal sequence. When the sample data enters the CNN layer, it will undergo convolution, pooling and fully connected operations in sequence.

[0011] The third layer is a multi-layer LSTM. LSTM has a memory function, which can extract the temporal change information of the nonlinear data of the pipeline operation. It introduces input gate, forget gate, and output gate, and also adds candidate state, cell state and hidden state. Cell state stores long-term memory, which can alleviate gradient vanishing, while hidden state stores short-term memory. This model uses a multi-layer LSTM. The output of the previous LSTM is the input of the next layer, and it is passed down layer by layer. The output of the last LSTM hidden layer will enter the attention layer for further processing.

[0012] The fourth layer is the Attention layer. Attention can enhance the role of important time steps in the LSTM, thereby further reducing the model's prediction error. Attention is essentially the weighted average sum of the output vectors of the last LSTM layer.

[0013] The fifth layer is the output layer, which specifies the prediction time step. The final output is the predicted time step. The prediction results within;

[0014] S3: Training and validating the model;

[0015] The dataset used is the NAB dataset, which is divided into training and testing sets in an 8:2 ratio. It consists of more than 50 labeled real-world and artificial time series data files, such as AWS server metrics, cloud server CPU utilization, and industrial equipment operating parameter records. The NAB dataset is an open-source dataset from Numenta used to evaluate streaming time series anomaly detection algorithms. For each time series, there is a Boolean outlier label, which helps us determine whether it is an outlier.

[0016] The evaluation metrics for an anomaly detection model include precision, recall, and F1-score. Precision, also known as accuracy, is the proportion of correctly identified anomalous samples out of all predicted anomalous samples. Recall is the proportion of correctly identified anomalous samples out of the total number of anomalous samples in the original data. Precision and recall are interdependent; ideally, both should be high, but in reality, they are mutually constraining. F1-score is the combined evaluation result of precision and recall, used in practical applications to evaluate the quality of the model; a higher F1-score indicates a more effective detection model.

[0017] S4: Abnormal data detection;

[0018] By inputting preprocessed real data into the model, the anomaly detection results for the anomaly detection interval time period are obtained.

[0019] Furthermore, in step S1, the operating data of the water supply pipeline includes data such as flow rate, flow velocity, water pressure, water temperature, and water level.

[0020] Furthermore, in step S2, during the construction of the second CNN layer, this model employs one-dimensional convolution, with the convolution kernel performing convolution only along a single temporal direction; the number of convolution kernels is r, and the size is set to k. for The real matrix from the i-th time step to the (i+k-1)-th time step, with a sliding step size of 1; weight matrix It is a k×n real matrix; feature extraction is performed on the sequence vector at every k time steps to obtain a feature vector. The calculation formula is as follows:

[0021] ,

[0022] It is a non-linear activation function. ∈R is a bias; after a convolutional kernel extracts the sequence data of a sample, it will obtain a feature map o of shape (t-k+1)×1, calculated as follows:

[0023] ,

[0024] These r feature maps are the features extracted by the CNN layer. They are reduced to a real vector of length r*(t-k+1) / 2, which preserves the spatial relationship between different feature values ​​in the sample data. This vector is then input into the LSTM layer for further processing.

[0025] Furthermore, in step S2, during the construction of the fourth Attention layer, the output vector of the LSTM hidden layer is used as the input to the attention layer. It is trained through a fully connected layer, and then the output of the fully connected layer is normalized using the softmax function to obtain the assigned weights for each hidden layer vector. The weight magnitude represents the importance of the hidden state at each time step to the prediction result. The weight training process is as follows:

[0026] ,

[0027] ,

[0028] Then, the weighted average sum of the hidden layer output vectors is calculated using the trained weights, and the result is as follows:

[0029] ,

[0030] in This is the output of the last LSTM hidden layer. The score output for each hidden layer. These are the weighting coefficients. The result is the weighted sum, and softmax is the activation function.

[0031] Furthermore, in step S3, the calculation formulas for the model evaluation metrics precision, recall, and F1-score are as follows:

[0032] ,

[0033] ,

[0034] ,

[0035] Among them, TP (True Class) means that an instance is a positive class and is also judged as a positive class; FP (False Positive Class) means that an instance is actually a false class but is judged as a positive class; FN (False Negative Class) means that an instance is actually a positive class but is judged as a false class. In the anomaly detection effect, if the anomaly is regarded as a positive class, then the other points are regarded as false classes.

[0036] This invention enriches the features of the original data through data acquisition and preprocessing, then constructs a CNN-LSTM model based on the Attention mechanism. The model is trained and validated using the NAB dataset, which consists of real-world and artificial time-series data files. Finally, the preprocessed real data is input into the model to obtain anomaly detection results for the anomaly detection interval. This invention combines the characteristics of Convolutional Neural Networks (CNN) and Long Short-Term Memory Networks (LSTM) time-series anomaly detection models. CNN can learn the spatial correlations between multiple features of the data, while LSTM, with its internal gating mechanism, can effectively avoid the vanishing or exploding gradient problem, thus enabling the processing of longer-span time-series data. Furthermore, the introduction of the Attention mechanism assigns weights to the input sequence, making it easier to capture certain important features, thereby further reducing model error. Attached Figure Description

[0037] Figure 1 This is a flowchart of a method for detecting anomalies in water supply pipeline operation data based on a CNN-LSTM model with an Attention mechanism, provided in an embodiment of the present invention.

[0038] Figure 2 This is a diagram of the CNN-LSTM model structure based on the Attention mechanism. Detailed Implementation

[0039] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0040] Reference Figure 1 An anomaly detection method for water supply pipeline operation data based on a CNN-LSTM model with an attention mechanism is presented, with the following specific steps:

[0041] S1: Data acquisition and preprocessing;

[0042] Water supply pipeline operation data is collected and sent to the data center through online data acquisition and transmission units (such as RTU, PLC and other equipment), including data such as flow rate, flow velocity, water pressure, water temperature and water level. The data is then processed in sequence, including time series missing value imputation, time series denoising, outlier detection and main correlation factor analysis.

[0043] S2: Construct a CNN-LSTM model based on the Attention mechanism;

[0044] The CNN-LSTM model based on the attention mechanism is an unsupervised anomaly detection model that combines the characteristics of CNN (Convolutional Neural Network) and LSTM (Long Short-Term Memory) time-series anomaly detection models. It leverages the spatial correlations between multiple features in the data through CNN, while LSTM, with its internal gating mechanism, effectively avoids the vanishing or exploding gradient problem, thus enabling the processing of longer-span time-series data. Furthermore, the introduction of the attention mechanism assigns weights to the input sequence, making it easier to capture certain important features, thereby further reducing the model's error. Its structure is as follows: Figure 2 As shown:

[0045] The first layer is the input layer, which defines the format of the input data: batch size, time steps, and feature dimension. The batch size is set to 1 by default, the time steps are denoted as t, and the feature dimension is denoted as n. Then, a sample can be represented as a real number sequence matrix. ,remember for Vector representation of the data at the i-th time step;

[0046] The second layer is the CNN layer. The CNN layer can learn the spatial correlations between multiple features of the data, compensating for the limitation of LSTM in capturing the spatial components of the data, while still retaining the temporal sequence of the extracted features. Sample data entering the CNN layer undergoes convolution, pooling, and fully connected operations sequentially. This model uses one-dimensional convolution, with the convolution kernel performing convolution only along a single temporal direction. The number of convolution kernels is r, and the size is set to k. for The real matrix from the i-th time step to the (i+k-1)-th time step, with a sliding step size of 1; weight matrix It is a k×n real matrix; feature extraction is performed on the sequence vector at every k time steps to obtain a feature vector. The calculation formula is as follows:

[0047] ,

[0048] It is a non-linear activation function. ∈R is a bias; after a convolutional kernel extracts the sequence data of a sample, it will obtain a feature map o of shape (t-k+1)×1, calculated as follows:

[0049] ,

[0050] These r feature maps are the features extracted by the CNN layer. They are reduced to a real vector of length r*(t-k+1) / 2, which stores the spatial relationship between different feature values ​​in the sample data. Then, they are input into the LSTM layer for further processing.

[0051] The third layer is a multi-layer LSTM. LSTM has a memory function, which can extract the temporal change information of the nonlinear data of the pipeline operation. It introduces input gate, forget gate, and output gate, and also adds candidate state, cell state and hidden state. Cell state stores long-term memory, which can alleviate gradient vanishing, while hidden state stores short-term memory. This model uses a multi-layer LSTM. The output of the previous LSTM is the input of the next layer, and it is passed down layer by layer. The output of the last LSTM hidden layer will enter the attention layer for further processing.

[0052] The fourth layer is the Attention layer. Attention enhances the role of important time steps in the LSTM, thereby further reducing the model's prediction error. Attention essentially involves calculating the weighted average of the output vectors of the last LSTM layer. The output vectors of the LSTM hidden layers serve as the input to the Attention layer, are trained through a fully connected layer, and then the output of the fully connected layer is normalized using the softmax function to determine the weights assigned to each hidden layer vector. The magnitude of the weights represents the importance of the hidden state at each time step to the prediction result. The weight training process is as follows:

[0053] ,

[0054] ,

[0055] Then, the weighted average sum of the hidden layer output vectors is calculated using the trained weights, and the result is as follows:

[0056] ,

[0057] in This is the output of the last LSTM hidden layer. The score output for each hidden layer is represented by the weight coefficients. The result is the weighted sum, and softmax is the activation function.

[0058] The fifth layer is the output layer, which specifies the prediction time step. The final output is the predicted time step. The prediction results within;

[0059] S3: Training and validating the model;

[0060] The dataset used is the NAB dataset, which is divided into training and testing sets in an 8:2 ratio. It consists of more than 50 labeled real-world and artificial time series data files, such as AWS server metrics, cloud server CPU utilization, and industrial equipment operating parameter records. The NAB dataset is an open-source dataset from Numenta used to evaluate streaming time series anomaly detection algorithms. For each time series, there is a Boolean outlier label, which helps us determine whether it is an outlier.

[0061] The evaluation metrics for an anomaly detection model include precision, recall, and F1-score. Precision, also known as accuracy, is the proportion of correctly identified anomaly samples out of all predicted anomaly samples. Recall is the proportion of correctly identified anomaly samples out of the total number of anomaly samples in the original data. Precision and recall are interdependent; ideally, both should be high, but in reality, they are mutually restrictive. The F1-score is a composite score of precision and recall, used to evaluate the model's performance; a higher F1-score indicates a more effective detection model. The formulas for calculating precision, recall, and F1-score are as follows:

[0062] ,

[0063] ,

[0064] ,

[0065] Among them, TP (True Class) represents an instance that is a positive class and is also judged as a positive class; FP (False Positive Class) represents an instance that is actually a false class but is judged as a positive class; FN (False Negative Class) represents an instance that is actually a positive class but is judged as a false class; in the anomaly detection effect, if the anomaly is regarded as a positive class, then the other points are regarded as false classes.

[0066] S4: Abnormal data detection;

[0067] By inputting preprocessed real data into the model, the anomaly detection results for the anomaly detection interval time period are obtained.

Claims

1. A water supply pipeline operation data anomaly detection method characterized by, The specific steps are as follows: S1: data acquisition and pretreatment; The water supply pipeline operation data are collected by the online data acquisition and transmission unit and sent to the data center, and the data are sequentially subjected to time series missing value interpolation, time series denoising, outlier detection and main correlation factor analysis; S2: construction of a CNN-LSTM model based on Attention mechanism; The CNN-LSTM model based on Attention mechanism is an unsupervised anomaly detection model combining the characteristics of CNN and LSTM time series data anomaly detection model. It can learn the spatial correlation between multiple features of data through CNN, and effectively avoid the problem of gradient vanishing or explosion through its internal gating mechanism, so as to process longer span time series data. Meanwhile, the Attention mechanism is introduced to assign weights to the input sequence, so that some important features in the sequence can be captured more easily, thereby further reducing the error of the model. The structure is as follows: The first layer is an input layer, which defines the format of the input data: batch size, time step number, and feature dimension. The batch size is set to 1 by default, the time step number is denoted as t, and the feature dimension is denoted as n. A sample can be represented as a real number sequence matrix , where is the vector representation of the i-th time step data in the sequence . The second layer is the CNN layer, which can learn the spatial correlation between multiple features of data, making up for the disadvantage of LSTM that cannot capture the spatial components of data. Meanwhile, the features extracted by it still have time series; sample data entering the CNN layer will be subjected to convolution, pooling and full connection operations in turn; The third layer is a multi-layer LSTM layer. LSTM has memory function and can extract the time series change information of nonlinear pipeline operation data. It introduces input gate, forget gate and output gate, and also adds candidate state, cell state and hidden state. Cell state stores long-term memory and can alleviate gradient vanishing, and hidden state stores short-term memory. This model uses multi-layer LSTM, and the output of the previous layer is the input of the next layer, which is transmitted layer by layer, and the output of the hidden layer of the last LSTM layer will enter the attention layer for further processing; The fourth layer is the Attention layer. Attention can improve the role of important time steps in LSTM, thereby further reducing the prediction error of the model. Attention is essentially the weighted average sum of the output vector of the last LSTM layer; The fifth layer is the output layer, which specifies the predicted time step for which the final output is the prediction result. ​ S3: training and validating the model; The data set uses NAB data set, which is divided into training set and test set in the ratio of 8:

2. It consists of more than 50 labeled real-world and artificial time series data files. NAB data set is an open data set for evaluating streaming time series anomaly detection algorithms opened by Numenta company. For each time series, it contains a Boolean anomaly value label, which is used to help us judge whether it is an anomaly value. The evaluation indexes of the model include: precision, recall and F1-score; precision is the precision rate, also known as the precision rate, that is, the proportion of the abnormal samples correctly identified by the anomaly detection model in all the samples predicted as abnormal samples; recall is the recall rate, that is, the proportion of the abnormal samples correctly identified in the total abnormal samples in the original data samples; the precision rate and the recall rate affect each other, and in the ideal state, both are high, but in the actual situation, they are mutually "restricted"; F1-score is the F1 score, which is the evaluation result of the precision rate and the recall rate, and is used to evaluate the model in practical application, the higher the value, the more effective the model used for detection; S4: Abnormal data detection; The preprocessed real data is input into the model to obtain the abnormal detection result of the abnormal detection interval time period.

2. The water supply line operational data anomaly detection method of claim 1, wherein: In step S1, the operation data of the water supply pipeline includes flow, flow rate, water pressure, water temperature and water level.

3. The water supply line operational data anomaly detection method according to claim 1 or 2, characterized in that: In step S2, in the construction of the second layer CNN layer, one-dimensional convolution is adopted, and the convolution kernel only performs convolution in a single time domain direction; the number of convolution kernels is r, and the size is set to k, so that is a real matrix from the i-th time step to the i+k-1 time step, and the sliding step is 1; the weight matrix is a k*n real matrix; feature extraction is performed on each k time step sequence vector to obtain a feature , and the calculation formula is as follows: ， is a non-linear activation function, ∈ ℝ is a bias; when a convolution kernel extracts a sequence data of a sample, a feature map o of a (t-k+1) x 1 shape is obtained, and the calculation formula is as follows: ， The r feature maps are the features extracted by the CNN layer, which are reduced to a real number vector with a length of r*(t-k+1) / 2, which saves the spatial relationship between different feature values in the sample data, and is input into the LSTM layer for further processing.

4. The water supply line operational data anomaly detection method according to claim 1 or 2, characterized by: In step S2, in the construction of the fourth layer Attention layer, the LSTM hidden layer output vector is used as the input of the attention layer, which is trained through a fully connected layer, and then the output of the fully connected layer is normalized by using the softmax function to obtain the allocation weight of each hidden layer vector, and the weight size represents the importance of the hidden state of each time step to the prediction result; the weight training process is as follows: ， ， Then, the weighted average of the hidden layer output vector is calculated using the trained weight, and the calculation result is as follows: ， wherein is the output of the last LSTM hidden layer, is the score of each hidden layer output, is the weight coefficient, is the result of the weighted sum, and softmax is the activation function.

5. The water supply line operational data anomaly detection method according to claim 1 or 2, characterized by: In step S3, the calculation formulas of the model evaluation indexes precision, recall and F1-score are as follows: ， ， ， Wherein, TP represents a positive class when an instance is positive and is also determined as positive; FP represents that an instance is false class but is determined as positive; FN represents that an instance is positive class but is determined as false class; in the abnormal detection effect, if the abnormal point is regarded as positive class, then the other points are regarded as false class.

Images