Anomaly detection system and method for industrial control systems

Abstract

The present invention discloses an anomaly detection system and method for industrial control systems, relating to the field of anomaly detection. The system comprises a data acquisition module, a data preprocessing module, an anomaly detection model training module, a threshold setting module, an anomaly detection module, and a result output module. The method comprises: step 1, acquiring training data; step 2, performing data preprocessing; step 3, learning time-dimensional features; step 4, learning time series correlations; step 5, reconstructing the multidimensional time series data; step 6, setting anomaly thresholds; step 7, performing real-time online detection; and step 8, outputting detection results. The present invention can effectively model time series, learn the periodic patterns of normal sequences, and effectively perform robust modeling of time series data contaminated by anomalies. It uses the extreme value theorem to learn reconstruction errors during training and automatically sets thresholds, avoiding the inconvenience of setting thresholds based on empirical values.

Description

Technical Field

[0001] The present invention relates to the field of anomaly detection, and in particular to an anomaly detection system and method for industrial control systems. Background Art

[0002] Industrial control systems are widely used in industrial sectors and critical infrastructure, such as electrical, water and wastewater, oil and gas, chemicals, transportation, pharmaceuticals, pulp and paper, food and beverage, and discrete manufacturing (such as automotive, aerospace, and durable goods). Timely and effective anomaly detection for industrial control systems can ensure the long-term stable operation of industrial production.

[0003] In actual production processes, labeled data is often difficult to obtain, so anomaly detection is often based on unsupervised methods. The complex structure of industrial control systems makes single-variable time series anomaly detection difficult to apply. However, industrial production processes typically involve a large number of sensors monitoring the current state in real time, and actuators operating the current system. Therefore, multivariate time series data collected from sensors and actuators is an important research topic in the field of anomaly detection.

[0004] While traditional machine learning-based anomaly detection methods offer good interpretability, they rely on expert experience to build complex feature engineering for time series data. With the advancement of technology and increased computing power, anomaly detection methods based on deep learning have begun to attract widespread attention. Recurrent neural networks can capture long-term dependencies in time series data, but they do not account for correlations between different time series. Therefore, they are not ideal for modeling multidimensional time series data with potential sequence correlations. Furthermore, in real-world industrial scenarios, workload fluctuations and environmental noise are inevitable, resulting in poor model robustness. Furthermore, the setting of anomaly detection thresholds often relies on experience and lacks flexibility.

[0005] Jin Yaohui et al. provided a multidimensional time series anomaly detection method and detection system in their Chinese invention patent application “A multidimensional time series anomaly detection method and detection system” (application number: 202011060906.4), including: reconstructing the sampled low-dimensional variables into a multidimensional time series through a recurrent neural network autoencoder, optimizing the model using a regularization method based on the Markov smoothing assumption of the time series, and calculating the time series anomalies based on the probability distribution of the reconstructed time series. However, this method only learns the laws of the time series in the time dimension and ignores the potential correlation between time series. The lack of potential correlation between time series will lead to low model accuracy and low detection performance.

[0006] Zhao Peihai et al. proposed a multidimensional time series anomaly detection method and detection system in their Chinese invention patent application “Real-time anomaly detection method for multidimensional time series data using unsupervised deep neural network” (application number: 202110848400.8), including: calculating the correlation feature matrix SFM of the collected data, inputting the feature matrix sequence into a feature extraction and reconstruction data module with a four-layer convolutional neural network as a feature extractor, adding a layer of LSTM network structure to each layer of convolutional neural network, reconstructing the feature extraction matrix output by each layer of LSTM network structure to obtain a reconstruction matrix, and all reconstructed matrices constitute a reconstruction matrix sequence, which is used as the input of a linear regression. The output of the linear regression is the predicted collected data of the n-order square matrix PSFM, calculating the difference between PSFM and SFM to obtain the anomaly score sc, and judging whether the anomaly score sc reaches the abnormal range according to the given threshold δ. Although this method takes into account the correlation between time series, its correlation calculation is limited to linear relationships, and it is difficult to achieve good results for systems with complex relationships.

[0007] In their Chinese invention patent application “A method and system for detecting anomalies in multidimensional time series data” (application number: 202111371649.0), Pi Dechang et al. provide a method and system for detecting anomalies in multidimensional time series data, comprising the following steps: first, preprocessing the satellite telemetry data to obtain coded additional data and time scale fusion features; then, fusing the coded additional data and fusion features to obtain fused input information; subsequently, inputting the fused input information into the Transformer variational autoencoder for encoding and decoding to reconstruct the result, and calculating the reconstruction error; then, using the weighted moving average method to smooth the reconstruction error, when the smoothed error exceeds the threshold range, it is determined that the satellite telemetry data to be detected is anomaly, and the abnormal time point is recorded. This method can effectively capture the correlation between time series, but the threshold setting for anomaly detection still uses empirical values, which lacks flexibility and adaptability.

[0008] Therefore, those skilled in the art are committed to developing a new anomaly detection system and method for industrial control systems to overcome the above-mentioned defects in the prior art. Summary of the Invention

[0009] In view of the above-mentioned defects of the prior art, the technical problem to be solved by the present invention is how to overcome the defects in the prior art, such as the lack of learning of the potential relationship between time series, resulting in low model performance, overfitting of the anomaly detection deep learning model in the presence of noise in the training data set, and the reliance on experience in setting the anomaly threshold.

[0010] To achieve the above objectives, the present invention provides a multidimensional time series anomaly detection system and method for industrial control systems. The system adopts a deep learning method to learn the normal periodic laws of industrial multidimensional time series, and detects time series anomalies based on the reconstructed time series probability distribution, which is beneficial to improving the accuracy and stability of anomaly detection and enhancing the reliability of the detection model.

[0011] The present invention provides a multi-dimensional time series anomaly detection system for industrial control systems, comprising:

[0012] A data acquisition module, wherein the data acquisition module records multi-dimensional time series data of the industrial control system;

[0013] A data preprocessing module, connected to the data acquisition module, preprocesses the acquired multidimensional time series data to obtain several batches of multidimensional time series subsequences;

[0014] an anomaly detection model training module, connected to the data preprocessing module, receiving the batches of multidimensional time series subsequences, and constructing and training a neural network model for anomaly detection, referred to as an anomaly detection model; the output of the anomaly detection model training module is reconstructed data of the input multidimensional time series data;

[0015] a threshold setting module, connected to the anomaly detection model training module, calculating the error between the reconstructed data and the input multidimensional time series data, referred to as the reconstruction error, using the reconstruction error as sample data, using the extreme value theorem for learning, and automatically setting an anomaly threshold;

[0016] An anomaly detection module, which inputs the real-time collected time series data into the trained anomaly detection model after passing it through the data preprocessing module, calculates the reconstruction error, uses the reconstruction error as an anomaly score, and compares the anomaly score with the anomaly threshold; when the anomaly score is less than the anomaly threshold, it is considered that the real-time collected time series data has an anomaly;

[0017] A result output module is connected to the anomaly detection module and outputs an anomaly detection result for the real-time collected time series data of the detected anomaly.

[0018] Furthermore, the time series data collected in real time is also collected by the data collection module.

[0019] Furthermore, in the data preprocessing module, the data preprocessing first uses a sliding window technology to perform segmentation, and then the segmented subsequences are batched and normalized.

[0020] Furthermore, in the anomaly detection model training module, the anomaly detection model uses a one-dimensional convolutional neural network to learn the temporal features of the time series, uses a random recurrent neural network to learn the correlation between time series, and reconstructs the input multidimensional time series data through the idea of ​​variational autoencoder; constructs a loss function through the reconstruction error, the posterior distribution of the latent space vector and the KL divergence of the assumed prior distribution; performs model training by optimizing the loss function, and saves the parameters of the anomaly detection model when the loss function reaches the minimum.

[0021] Furthermore, in the result output module, the abnormality detection result includes the abnormality occurrence location, occurrence time, and duration length.

[0022] The present invention also provides a multi-dimensional time series anomaly detection method for industrial control systems, the method comprising the following steps:

[0023] Step 1: Continuously sample a sensor and b actuator of the industrial control system at a preset frequency f, with a sampling time length of T, to obtain a sample X of multidimensional time series data, with a size of X∈R N×M , where N is the sampling data length, which is calculated by the frequency f and the sampling time length T; M is the sampling data dimension, M=a+b;

[0024] Step 2: Perform data preprocessing and set the sliding time window, including the start time s t and the end time e t ; The length of the sliding time window is w=e t -s t , the width is the sampling data dimension M of the multidimensional time series data; slide the sliding time window on the multidimensional time series data until the sequence data ends; set the step size of each sliding to s, and divide the multidimensional time series data into several subsequence segments x with a sampling time length of w and a sampling data dimension of M, x∈R w×M ; When the sampling time length is less than the length of the sliding time window, the fragment is directly used as a subsequence; the batch size is selected as b s , the segmented subsequence fragments are divided into multiple segments of size b s The subsequence size of each batch is (b s ,w,M);

[0025] Step 3: Learning time dimension features: First, use several one-dimensional convolutional neural networks to perform one-dimensional convolution on the input multidimensional time series data along its time dimension to learn a low-dimensional representation z1 of the time series features. Then, deconvolute the low-dimensional representation z1 of the time series features, and output d;

[0026] Step 4: Learn the time series correlation. First, input d into the variational encoding network of the random recurrent neural network to learn the correlation between time series and obtain a low-dimensional representation z2. Then, z2 is subjected to the realNVP flow to obtain an enhanced low-dimensional representation. Steps 3 and 4 constitute the approximate inference network in the structure of the variational autoencoder:

[0027]

[0028] Specifically:

[0029]

[0030] in:

[0031] f(·) and f -1 (·) represents one-dimensional convolution and deconvolution operations, Represents the gated recurrent unit GRU;

[0032] Step 5: reconstructing the multidimensional time series data;

[0033] Step 6: Set the abnormal threshold;

[0034] Step 7: Real-time online detection;

[0035] Step 8: Output the test results.

[0036] Furthermore, in step 5, the low-dimensional representation z1 of the time series feature is deconvolved to obtain the input e of the decoding network, and the time series information contained in e is used as external input and input into the random recurrent neural network to construct and train a neural network model for anomaly detection, which is called anomaly detection model. At the same time, the low-dimensional representation z2 between time series is combined to realize the reconstruction of the original input multidimensional time series data to obtain reconstructed data, which can be expressed as:

[0037] p θ (x,z1,z2)=p θ (x|z1z,2)p θ (z2|z1)

[0038] Specifically:

[0039]

[0040] Where: g(·) represents the deconvolution operation on z1, and constructs a loss function to jointly optimize the approximate model and the generative model;

[0041] According to the optimization function of the variational autoencoder:

[0042] in the form of

[0043] The structure optimization function is:

[0044]

[0045] The optimization methods include using Monte Carlo sampling, SGVB estimator and reparameterization trick.

[0046] Furthermore, in the step 6, calculate the error between the reconstructed data and the original input multi-dimensional time series data, and obtain a series of reconstruction errors error={er1, er2...};

[0047] Taking the reconstruction error as a sample, use the extreme value theorem to automatically set the threshold as threshold. From the extreme value theorem:

[0048]

[0049] The calculation formula of the anomaly threshold is:

[0050]

[0051] where, th is the initially set threshold, is the parameter to be learned, q is the set probability size, N is the number of input samples, N t is the number of samples greater than the initial threshold, and at the same time, add the real-time calculated anomaly score to iteratively update the detection threshold.

[0052] Furthermore, in the step 7, for the time series data collected in real time at time t, input it into the anomaly detection model, and obtain the anomaly score score. score is the reconstruction probability of the input data by the anomaly detection model; when score < threshold, it is considered that an anomaly occurs, otherwise, it is considered normal.

[0053] Furthermore, in the step 8, for the detected abnormal time series segment, calculate the anomaly possibility of all dimensions of the input data at this moment, sort them from high to low, select the first k dimensions as anomalies, and output the names of the sensors or actuators corresponding to the abnormal dimensions, the time when the anomaly occurs, and the duration length.

[0054] The multi-dimensional time series anomaly detection system and method for industrial control systems provided by the present invention at least have the following technical effects:

[0055] 1. The technical solution provided by the present invention is based on the strategy of hierarchical variational autoencoders. It not only learns the temporal relationship of multidimensional time series, but also captures the potential correlation between different time series. It can effectively model time series and learn the periodic laws of normal series.

[0056] 2. The technical solution provided by the present invention reconstructs the input to filter out anomalies in the original data through a one-dimensional convolutional network and an inverse one-dimensional convolution operation, and can effectively perform robust modeling on time series data contaminated by anomalies;

[0057] 3. The technical solution provided by the present invention is to use the extreme value theorem to learn the reconstruction error during the training process and automatically set the threshold, avoiding the inconvenience of setting according to empirical values.

[0058] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is an overall framework diagram of a preferred embodiment of the present invention;

[0060] Figure 2 for Figure 1 Schematic diagram of multidimensional time series and sliding window of an embodiment;

[0061] Figure 3 for Figure 1 The random recurrent neural network of the embodiment;

[0062] Figure 4 for Figure 1 Diagram of the overall architecture of the anomaly detection model of the embodiment. DETAILED DESCRIPTION

[0063] The following describes several preferred embodiments of the present invention with reference to the accompanying drawings to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0064] The technical problem to be solved by the present invention is how to overcome the defects in the prior art, such as the lack of learning about the potential relationship between time series, which leads to low model performance, overfitting of the deep learning model for anomaly detection in the presence of noise in the training data set, and the reliance on experience in setting the anomaly threshold. In order to solve the above technical problems, the present invention adopts one-dimensional convolution to learn the temporal relationship of multidimensional time series, and at the same time captures the potential correlation between different time series through random variational autoencoders, reconstructs data based on the temporal relationship and correlation information, and trains the model based on the strategy of variational autoencoders, which can effectively model the time series and learn the periodic laws of normal sequences. In addition, the extreme value theorem is used to learn the reconstruction error generated during the training process, automatically set the anomaly detection threshold, and avoid the inconvenience of setting according to empirical values.

[0065] The present invention provides a multidimensional time series anomaly detection system and method for industrial control systems. The system uses a deep learning method to learn the normal periodic laws of industrial multidimensional time series and detects time series anomalies based on the reconstructed time series probability distribution. This is beneficial to improving the accuracy and stability of anomaly detection and enhancing the reliability of the detection model.

[0066] like Figure 1 As shown, the present invention provides a multi-dimensional time series anomaly detection system for industrial control systems, including:

[0067] The data acquisition module records the multi-dimensional time series data of the industrial control system. The data acquisition module records the current status information of sensors and actuators obtained by continuous sampling at a fixed frequency during the normal operation of the industrial control system, generating multi-dimensional time series data. During the model training phase of the anomaly detection system, the data acquisition module constructs the data set required for neural network modeling. During the model operation phase of the anomaly detection system, the data acquisition module collects system status data in real time for anomaly detection.

[0068] The data preprocessing module is connected to the data acquisition module to preprocess the collected multidimensional time series data to obtain several batches of multidimensional time series subsequences; specifically, the data preprocessing module preprocesses the collected multidimensional time series data, uses the sliding window technology to segment the multidimensional time series, and then divides the segmented subsequences into batches and normalizes them to obtain several batches of multidimensional time series subsequences.

[0069] The anomaly detection model training module is connected to the data preprocessing module, receives several batches of multidimensional time series subsequences, and constructs and trains a neural network model for anomaly detection, called the anomaly detection model. The output of the anomaly detection model training module is the reconstruction data of the input multidimensional time series data. Specifically, the anomaly detection model uses a one-dimensional convolutional neural network to learn the temporal characteristics of the time series, a random recurrent neural network to learn the correlations between time series, and reconstructs the input multidimensional time series data using the concept of a variational autoencoder. The loss function is constructed by reconstructing the error, the posterior distribution of the latent space vector, and the KL divergence of the hypothesized prior distribution. The model is trained by optimizing the loss function, and when the loss function reaches a minimum, the parameters of the anomaly detection model are saved.

[0070] The threshold setting module is connected to the anomaly detection model training module. It calculates the error between the reconstructed data and the input multi-dimensional time series data, called the reconstruction error. The reconstruction error is used as sample data and the extreme value theorem is used for learning to automatically set the anomaly threshold.

[0071] The anomaly detection module inputs the real-time collected time series data into the trained anomaly detection model after passing it through the data preprocessing module to calculate the reconstruction error. The reconstruction error is used as the anomaly score and compared with the anomaly threshold. When the anomaly score is less than the anomaly threshold, it is considered that the real-time collected time series data is anomaly.

[0072] Among them, the real-time collected time series data is also collected by the data acquisition module.

[0073] The result output module is connected to the anomaly detection module and outputs the anomaly detection results for the real-time collected time series data of the detected anomaly. The anomaly detection results include the location, time, and duration of the anomaly.

[0074] The present invention also provides a multi-dimensional time series anomaly detection method for industrial control systems. The method first measures and collects sensor and actuator signals in an industrial production process to generate a multi-dimensional time series. The collected multi-dimensional time series is segmented into time series segments according to a sliding window. The low-dimensional features of the time series are hierarchically learned. A one-dimensional convolutional neural network is used to learn the time series features along the time dimension of the time series. A random recurrent neural network is used to learn the potential correlation between the time series along the feature dimension of the time series. The input time series is reconstructed using the learned time series features and correlation information. A loss function is constructed using a variational autoencoder strategy to train and optimize the model. The error between the reconstructed data and the original input is calculated as a reconstruction error. The reconstruction error is learned, an anomaly detection threshold is set, and the real-time collected data is input into the anomaly detection model. The anomaly score is calculated and a threshold judgment is performed. If it is judged to be abnormal, an anomaly alarm is output and the anomaly is located.

[0075] Specifically, the method includes the following steps:

[0076] Step 1: Continuously sample a sensor and b actuator of the industrial control system at a preset frequency f, with a sampling time length of T, to obtain a sample X of multidimensional time series data, with a size of X∈R N×M , where N is the sampling data length, which is calculated by the frequency f and the sampling time length T; M is the sampling data dimension, M = a + b;

[0077] Step 2: Preprocess the data, such as Figure 2 As shown, set the sliding time window, including the start time s t and the end time e t ; The length of the sliding time window is w = e t -s t , the width is the sampling data dimension M of the multidimensional time series data; slide the sliding time window on the multidimensional time series data until the sequence data ends; set the step size of each sliding to s, and divide the multidimensional time series data into several subsequence segments x with a sampling time length of w and a sampling data dimension of M, x∈R w×M ; When the sampling time length is less than the length of the sliding time window, the fragment is directly used as a subsequence; the batch size is selected as b s , the segmented subsequence fragments are divided into multiple segments of size b s The subsequence size of each batch is (b s ,w,M), and then sequentially input into the anomaly detection model for training;

[0078] Step 3: Learn the time dimension features. First, use several one-dimensional convolutional neural networks to perform one-dimensional convolution on the input multi-dimensional time series data along its time dimension to learn the low-dimensional representation z1 of the time series features. Then, deconvolve the low-dimensional representation z1 of the time series features, and output d. The purpose of the deconvolution operation is to filter out abnormal data noise in the training data to ensure the accuracy of the model and the consistency of subsequent time series correlation learning.

[0079] Step 4: Learn time series correlation, such as Figure 3 As shown in the figure, first, d is input into the variational encoding network of the random recurrent neural network to learn the correlation between time series and obtain a low-dimensional representation z2. Then, z2 is passed through the realNVP flow to obtain an enhanced low-dimensional representation; steps 3 and 4 constitute the approximate inference network in the structure of the variational autoencoder:

[0080]

[0081] Specifically:

[0082]

[0083] in:

[0084] f(·) and f -1 (·) represents one-dimensional convolution and deconvolution operations, Represents the gated recurrent unit GRU;

[0085] Step 5: Reconstruct the multidimensional time series data. Deconvolve the low-dimensional representation z1 of the time series features to obtain the input e of the decoding network. The time series information contained in e is used as external input and input into the random recurrent neural network to build and train a neural network model for anomaly detection, called the anomaly detection model. At the same time, the low-dimensional representation z2 between time series is combined to reconstruct the original input multidimensional time series data. The reconstructed data can be expressed as:

[0086] p θ (x,z1,z2)=p θ (x|z1,z2)p θ (z2|z1)

[0087] Specifically:

[0088]

[0089] Where: g(·) represents the deconvolution operation on z1, and constructs a loss function to jointly optimize the approximate model and the generative model;

[0090] According to the optimization function of the variational autoencoder:

[0091] in the form of

[0092] The construction optimization function is:

[0093]

[0094] The optimization methods include using Monte Carlo sampling, SGVB estimator and reparameterization trick;

[0095] Step 6, set the anomaly threshold, such as Figure 4 As shown, calculate the error between the reconstructed data and the multi-dimensional time series data of the original input, and obtain a series of reconstruction errors error = {er1, er2...};

[0096] Taking the reconstruction error as a sample, use the extreme value theorem to automatically set the threshold as threshold. By the extreme value theorem:

[0097]

[0098] The calculation formula for the anomaly threshold is:

[0099]

[0100] where th is the initially set threshold, is the parameter to be learned, q is the set probability size, N is the number of input samples, N t is the number of samples greater than the initial threshold, and at the same time, add the anomaly score calculated in real time to iteratively update the detection threshold;

[0101] Step 7, real-time online detection. For the time series data collected in real time at time t, input it into the anomaly detection model to obtain the anomaly score score, and score is the reconstruction probability of the input data by the anomaly detection model; when score < threshold, it is considered that an anomaly occurs, otherwise, it is considered normal;

[0102] Step 8, output the detection result. For the detected abnormal time series segment, calculate the anomaly possibility of all dimensions of the input data at this moment, sort them from high to low, select the top k dimensions as anomalies, and output the names of the sensors or actuators corresponding to the abnormal dimensions, the time when the anomaly occurs, and the duration length.

[0103] The above has described in detail the preferred specific embodiments of the present invention. It should be understood that those of ordinary skill in the art can make many modifications and variations according to the concept of the present invention without creative labor. Therefore, all technical solutions that can be obtained by those skilled in the art in the technical field based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art should be within the protection scope determined by the claims.

Claims

1. A multi-dimensional time series anomaly detection system for industrial control systems, characterized by: include: A data acquisition module, wherein the data acquisition module records multi-dimensional time series data of the industrial control system; A data preprocessing module, connected to the data acquisition module, preprocesses the acquired multidimensional time series data to obtain multiple batches of multidimensional time series subsequences; An anomaly detection model training module, connected to the data preprocessing module, receives multiple batches of multidimensional time series subsequences, constructs and trains a neural network model for anomaly detection, referred to as an anomaly detection model, and outputs the reconstructed data of the input multidimensional time series data; a threshold setting module, connected to the anomaly detection model training module, calculating the error between the reconstructed data and the input multidimensional time series data, referred to as the reconstruction error, using the reconstruction error as sample data, using the extreme value theorem for learning, and automatically setting an anomaly threshold; An anomaly detection module, which inputs the real-time collected time series data into the trained anomaly detection model after passing it through the data preprocessing module, calculates the reconstruction error, uses the reconstruction error as an anomaly score, and compares the anomaly score with the anomaly threshold; when the anomaly score is less than the anomaly threshold, it is considered that the real-time collected time series data has an anomaly; a result output module, the result output module being connected to the anomaly detection module and outputting an anomaly detection result for the real-time collected time series data of the detected anomaly; In the anomaly detection model training module, the anomaly detection model uses a one-dimensional convolutional neural network to learn the temporal features of the time series, uses a random recurrent neural network to learn the correlation between time series, and reconstructs the input multidimensional time series data through a variational autoencoder; constructs a loss function based on the reconstruction error, the posterior distribution of the latent space vector, and the KL divergence of the assumed prior distribution; performs model training by optimizing the loss function, and saves the parameters of the anomaly detection model when the loss function reaches a minimum; The system also includes a multi-dimensional time series anomaly detection method for industrial control systems, the method comprising the following steps: Step 1: Industrial control system sensors and The actuators are driven at a pre-set frequency Continuous sampling, the sampling time length is , get samples of multidimensional time series data , the size is ,in, is the sampling data length, determined by the frequency and the sampling time length Calculated; is the sampling data dimension, ; Step 2: Perform data preprocessing and set the sliding time window, including the start time and end time ; The length of the sliding time window is , the width is the sampling data dimension of the multidimensional time series data ; Slide the sliding time window on the multidimensional time series data until the sequence data ends; set the step size of each sliding to , split the multidimensional time series data into multiple subsequence segments , ; When the length of the remaining sequence of the multidimensional time series data after continuous sampling is less than the length of the sliding time window, the fragment is directly used as a subsequence; the batch size is selected as , the segmented subsequence fragments are divided into multiple sizes of The subsequence size of each batch is ( ; Step 3: Learn the time dimension features. First, use multiple one-dimensional convolutional neural networks to perform one-dimensional convolution on the input multi-dimensional time series data along its time dimension to learn and obtain the low-dimensional representation of the time series features. , then, the low-dimensional representation of the temporal features Perform deconvolution and the output is ; Step 4: Learn time series correlation. First, Input the variational encoding network of the random recurrent neural network to learn the correlation between time series and obtain a low-dimensional representation , then, go through Obtain an enhanced low-dimensional representation; Step 3 and Step 4 constitute an approximate inference network in the structure of the variational autoencoder: in, Expressed as input data, Indicates output, represents the probability distribution function, Indicates that the input data is Output under the premise of The probability distribution of Indicates that the input is Output under the premise of The probability distribution of Indicates that the input is Under the premise of The distribution function of is the intermediate output, Represents the final output; Specifically: ， in: Indicates the moment, Indicates Data collected at all times , and Respectively represent the input The output obtained after learning is Indicates input The output of the one-dimensional convolution operation is , represents a one-dimensional convolution operation, Indicates that through The output of the deconvolution operation , represents the deconvolution operation, middle Represents the gated recurrent unit GRU operation, Respectively expressed in and The feature parameters generated in the moment recurrent neural network operation, Indicates that the Characteristic parameters of the moment and Learned parameters As input, we get Time parameters The probability distribution function of Step 5: reconstructing the multidimensional time series data; Step 6: Set the abnormal threshold; Step 7: Real-time online detection; Step 8: Output the test results; In step 5, the low-dimensional representation of the time series feature Perform deconvolution to get the input of the decoding network ,Will The contained time series information is used as external input and input into the random recurrent neural network to build and train a neural network model for anomaly detection, called anomaly detection model. At the same time, the low-dimensional representation between time series is combined To reconstruct the original input multidimensional time series data, the reconstructed data can be expressed as: Specifically: ， in: Express Perform deconvolution operations and construct a loss function to jointly optimize the approximate model and the generative model; According to the optimization function of the variational autoencoder: in the form of The optimization function is constructed as: ， Optimization methods include Monte Carlo sampling, SGVB estimator and reparameterization techniques; In step 6, the error between the reconstructed data and the original input multidimensional time series data is calculated to obtain the reconstruction error ; Taking the reconstruction error as a sample, the extreme value theorem is used to automatically set the threshold as , by the extreme value theorem: ， The calculation formula for the abnormal threshold is: ， in, is the initial threshold value, are the parameters that need to be learned, is the set probability size, is the number of input samples, The detection threshold is iteratively updated by adding the real-time calculated anomaly score for the number of samples greater than the initial threshold.

2. The multidimensional time series anomaly detection system for industrial control systems according to claim 1, characterized in that: The time series data collected in real time is also collected by the data collection module.

3. The multidimensional time series anomaly detection system for industrial control systems according to claim 1, characterized in that: In the data preprocessing module, the data preprocessing first uses the sliding window technology to perform segmentation, and then the segmented subsequences are batched and normalized.

4. The multidimensional time series anomaly detection system for industrial control systems according to claim 1, characterized in that: In the result output module, the abnormality detection result includes the abnormality occurrence location, occurrence time, and duration length.

5. The multidimensional time series anomaly detection system for industrial control systems according to claim 1, characterized in that: In step 7, for The time series data collected in real time is input into the anomaly detection model to obtain the anomaly score , That is, the reconstruction probability of the input data by the anomaly detection model; when When , it is considered abnormal, otherwise, it is considered normal.

6. The multidimensional time series anomaly detection system for industrial control systems according to claim 1, characterized in that: In step 8, for the detected abnormal time series fragments, the abnormal probability of all dimensions of the input data at that moment is calculated, and the abnormal probability is sorted from high to low, and the top abnormal probability is selected. Dimensions are taken as anomalies, and the name of the sensor or actuator corresponding to the abnormal dimension, the time when the abnormality occurred, and the duration length are output.

Images