Reconstruction and prediction-based time sequence anomaly detection method

Abstract

The invention discloses a time series anomaly detection method based on reconstruction and prediction, and particularly relates to the field of time series data analysis of the Internet of Things, comprising the following steps: step 1, data preprocessing; 2, constructing a reconstruction network based on an auto-encoder architecture and a time convolution network; step 3, constructing a prediction network based on LSTM and an attention mechanism; and step 4, end-to-end model training is carried out, and anomaly detection performance is evaluated. According to the method, a reconstruction network based on an auto-encoder architecture is constructed, a time convolution network TCN is taken as a core, multi-scale features are extracted by using expansion causal convolution, and input data are reconstructed to obtain reconstruction errors. A prediction network is constructed to capture a long-term dependency relationship in time sequence data, and weights of different time steps are dynamically allocated, so that the modeling capability of a model to a complex mode is enhanced. By integrating the reconstruction error and the prediction error, an abnormal score is constructed, and a threshold value is automatically determined by using a POT algorithm, so that accurate discrimination of abnormal data points is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of time series data analysis of the Internet of Things, and in particular to a time series anomaly detection method based on reconstruction and prediction. Background Art

[0002] In recent years, the Internet of Things has become a hot topic in the field of information technology. The Internet of Things refers to connecting physical devices, vehicles, household appliances, and various other items through the network, enabling them to collect and exchange data and achieve intelligent control and management. With the rapid development of artificial intelligence, mobile computing, and 5G technology, the Internet of Everything is providing society with more convenient production and living conditions. The application fields of the Internet of Things are wide-ranging, including smart homes, smart cities, healthcare, agriculture, transportation, etc. The data generated in production and life in various fields is growing exponentially. Among all data types, time series data has always been an important research direction in academia and industry.

[0003] In the IoT environment, the importance of anomaly detection in time series data cannot be ignored. This detection capability is essential for timely detection of system failures, improving operational safety, optimizing resource utilization, and predicting potential risks. In many fields such as industrial production, equipment monitoring, medical health, and environmental monitoring, accurate anomaly detection can help decision makers take timely intervention measures to avoid major losses and improve overall system efficiency.

[0004] Existing time series data anomaly detection methods can be mainly divided into the following categories: statistical methods, which identify anomalies based on the statistical characteristics of data. Common methods include moving average, exponential smoothing, ARIMA model, etc., which have low computational complexity, but are very limited in their ability to capture anomalies in nonlinear and complex patterns; traditional machine learning methods, which include support vector machines, decision trees, random forests, clustering algorithms, etc., which can handle slightly more complex data patterns, but cannot effectively capture long-term dependencies; deep learning methods, prediction-based methods include recurrent neural networks, long short-term memory networks, graph neural networks, etc., and reconstruction-based methods include autoencoders, Transformers, generative adversarial networks, etc. Summary of the invention

[0005] The purpose of the present invention is to provide a time series anomaly detection method based on reconstruction and prediction to solve the problems raised in the above background technology.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a time series anomaly detection method based on reconstruction and prediction, comprising the following steps:

[0007] Step 1: Data preprocessing: Remove the feature dimensions that are not related to anomaly detection in the collected IoT data to ensure that the data set only contains information related to the detection task. Use interpolation to fill in missing data for the remaining IoT data to avoid model performance degradation caused by missing values. Use the Min-Max normalization method to standardize each feature data, and then use the sliding window technology to construct a time series sample to capture time dependencies and ensure that the model input can reflect the time series characteristics and context information, thereby providing high-quality input data for subsequent anomaly detection.

[0008] Step 2: Construct a reconstruction network based on the autoencoder architecture and the temporal convolutional network. The reconstruction network adopts the autoencoder architecture, which includes an encoder part and a decoder part. The encoder part uses the temporal convolutional network TCN to extract the low-dimensional features of the input data. The decoder part also uses the temporal convolutional network TCN to map the low-dimensional hidden representation back to the original dimension of the input data layer by layer, thereby reconstructing an output similar to the input data.

[0009] Step 3: Construct a prediction network based on LSTM and attention mechanism; the prediction network is composed of LSTM and attention mechanism. LSTM captures the long-term dependencies in sequence data. The LSTM can effectively retain important information and forget unnecessary information by setting forget gate, input gate and output gate. The attention mechanism assigns different weights to each time step, so that the model can focus on the importance of different time steps, effectively improving the modeling ability of complex time series patterns. This dual detection mechanism combining reconstruction error and prediction error enhances the model's sensitivity to abnormal situations.

[0010] Step 4: End-to-end model training to evaluate anomaly detection performance; construct anomaly scores by combining reconstruction error and prediction error, automatically determine thresholds using the POT algorithm, and compare anomaly scores with thresholds to determine whether a data point is abnormal. During the training process, the model uses part of the data for training, and the rest of the data is used for verification and testing to ensure the generalization ability of the model. At each stage of model training, multiple indicators such as precision, recall, F1 score, and accuracy are used to evaluate the anomaly detection performance of the model, providing a basis for further model optimization.

[0011] In a preferred embodiment, the formula for normalization using the Min-Max normalization method in step 1 is:

[0012]

[0013] Among them, xi is the collected value of the i-th sensor, min(xtrain,i) is the minimum collected value of the i-th sensor in the training set, max(xtrain,i) is the maximum collected value of the i-th sensor in the training set, and x'i is the collected value of the i-th sensor after standardization.

[0014] In a preferred embodiment, the Min-Max normalization method is used in step 1 to perform normalization processing to scale the data values ​​to a range of 0 to 1, thereby effectively alleviating the impact of numerical differences between different features.

[0015] In a preferred embodiment, the temporal convolutional network TCN in the step 2 introduces dilated causal convolution, and the main variable parameters are the convolution kernel size k and the dilation factor d. A temporal convolutional network TCN encoder is composed of three TCN residual blocks, each of which contains two dilated causal convolution layers, two WeightNorm layers, two ReLU activation function layers, two dropout layers and a residual connection. The convolution kernel size k of the convolution layer in the residual block is 3, and the dilation factors d are 1, 2, and 4 respectively. In the last layer of the encoder part, it is mapped to the hidden space through a fully connected layer.

[0016] In a preferred embodiment, the decoder part in step 2 maps the potential representation back to the high-dimensional space through a fully connected layer, and then performs a transposed convolution operation to restore it to the same dimension as the input data layer by layer.

[0017] In a preferred embodiment, in step three, at each time step, the attention score of the input sequence is calculated, and the attention weight is generated using the softmax function.

[0018] In a preferred embodiment, the output of the decoder part in step 4 is the reconstructed data of the input data, and the reconstruction error is obtained by calculating the mean square error of the reconstructed data.

[0019] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0020] The present invention combines the two strategies of reconstruction and prediction to effectively identify complex abnormal patterns. By constructing a reconstruction network based on the autoencoder architecture, with the temporal convolutional network TCN as the core, the dilated causal convolution is used to extract multi-scale features, and the input data is reconstructed to obtain the reconstruction error. By constructing a prediction network based on LSTM and attention mechanism to capture long-term dependencies in time series data and dynamically assign weights to different time steps, the model's ability to model complex patterns is enhanced. Finally, by comprehensively analyzing the reconstruction error and prediction error, an anomaly score is constructed, and the POT algorithm is used to automatically determine the threshold to achieve accurate discrimination of abnormal data points, providing important support for real-time monitoring and decision support of IoT data. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0022] Figure 1 It is a flow chart of a time series anomaly detection method based on reconstruction and prediction of the present invention;

[0023] Figure 2 This is an overall solution architecture diagram of a time series anomaly detection method based on reconstruction and prediction of the present invention;

[0024] Figure 3 It is a schematic diagram of a TCN residual block of a time series anomaly detection method based on reconstruction and prediction of the present invention. DETAILED DESCRIPTION

[0025] The following specific embodiments illustrate the implementation of the present invention, and the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] Please refer to Figure 1 and Figure 2 As shown, the present invention provides a time series anomaly detection method based on reconstruction and prediction, comprising the following steps:

[0027] Step 1: Data preprocessing; remove the feature dimensions that are not related to anomaly detection in the collected IoT data to ensure that the data set only contains information related to the detection task. Use interpolation to fill in missing data for the remaining IoT data to avoid the degradation of model performance due to missing values. Use the Min-Max normalization method to standardize each feature data. Use the Min-Max normalization method to scale the data values ​​to the range of 0 to 1, effectively alleviating the impact of numerical differences between different features. Then use the sliding window technology to construct time series samples to capture time dependencies and ensure that the model input can reflect time series features and context information, thereby providing high-quality input data for subsequent anomaly detection.

[0028] The formula for standardization using the Min-Max normalization method is:

[0029]

[0030] Among them, xi is the collected value of the i-th sensor, min(xtrain,i) is the minimum collected value of the i-th sensor in the training set, max(xtrain,i) is the maximum collected value of the i-th sensor in the training set, and x'i is the collected value of the i-th sensor after standardization.

[0031] Step 2: Construct a reconstruction network based on the autoencoder architecture and the temporal convolutional network. The reconstruction network adopts the autoencoder architecture, which includes an encoder part and a decoder part. The encoder part uses the temporal convolutional network TCN to extract the low-dimensional features of the input data. The decoder part also uses the temporal convolutional network TCN to map the low-dimensional hidden representation back to the original dimension of the input data layer by layer, thereby reconstructing an output similar to the input data.

[0032] For details, please refer to Figure 3 As shown in the figure, the temporal convolutional network TCN introduces dilated causal convolution, and the main variable parameters are the convolution kernel size k and the dilation factor d. A temporal convolutional network TCN encoder is composed of three TCN residual blocks, each of which contains two dilated causal convolution layers, two WeightNorm layers, two ReLU activation function layers, two dropout layers and a residual connection. The convolution kernel size k of the convolution layer in the residual block is 3, and the dilation factors d are 1, 2, and 4 respectively. In the last layer of the encoder part, it is mapped to the hidden space through the fully connected layer to obtain a low-dimensional representation D;

[0033] The decoder also uses the structure of a temporal convolutional network. First, the potential representation is mapped back to a high-dimensional space through a fully connected layer, and then a transposed convolution operation is performed to restore it layer by layer to the same dimension as the input data. After each convolution layer, a ReLU activation function is used, followed by batch normalization. The output of the decoder is the reconstructed data X' of the input data. The reconstruction error Lr is obtained by calculating the mean square error MSE. The calculation formula is as follows:

[0034] L r =||XX′||2

[0035] Step 3: Construct a prediction network based on LSTM and attention mechanism; the prediction network is composed of LSTM and attention mechanism. LSTM captures the long-term dependencies in sequence data. The LSTM can effectively retain important information and forget unnecessary information by setting forget gate, input gate and output gate. The attention mechanism assigns different weights to each time step, so that the model can focus on the importance of different time steps, effectively improving the modeling ability of complex time series patterns. This dual detection mechanism combining reconstruction error and prediction error enhances the model's sensitivity to abnormal situations.

[0036] Specifically, the input Y of the prediction network is composed of the reconstruction error Lr and the potential representation D encoded by the temporal convolutional network TCN, which can make full use of the temporal characteristics and contextual information of the time series data. When designing the prediction network, the long short-term memory network is used to capture the long-term dependencies in the sequence data and improve the model's ability to learn complex time patterns. The specific calculation formula is as follows:

[0037] f t =σ(W f ·[h t-1 , Y t ]+b f )

[0038] i t =σ(W i ·[h t-1 , Y t ]+b i )

[0039]

[0040]

[0041] o t =σ(W o ·[h t-1 ,Y t ]+b0)

[0042] ht =o t *tanh(C t )

[0043] Where: ft represents the forget gate, it represents the input gate, ot represents the output gate, Ct represents the cell state at the current moment, ht represents the hidden state at the current moment, σ represents the sigmoid function, W and b represent the weight and bias.

[0044] After the LSTM layer, the attention mechanism is introduced to assign different weights to each time step, by calculating the attention score of each time step in the input sequence, as shown below:

[0045] E t =σ(W a ·h t +b a )

[0046] The attention weights are calculated using the softmax activation function as follows:

[0047]

[0048] Combining the weighted representation with the current LSTM hidden state, we get the prediction error as follows:

[0049]

[0050] Step 4: End-to-end model training to evaluate anomaly detection performance; construct anomaly scores by combining reconstruction error and prediction error, automatically determine thresholds using the POT algorithm, and compare anomaly scores with thresholds to determine whether a data point is abnormal. During the training process, the model uses part of the data for training, and the rest of the data is used for verification and testing to ensure the generalization ability of the model. At each stage of model training, multiple indicators such as precision, recall, F1 score, and accuracy are used to evaluate the anomaly detection performance of the model, providing a basis for further model optimization.

[0051] In specific use, the final anomaly score is calculated from the reconstruction error and the prediction error, and the threshold is automatically calculated by POT:

[0052] score t =λ*L rt +(1-λ)*L pt Performance evaluation can be performed using four indicators: precision, recall, F1 score, and accuracy. The calculation formulas for each indicator are as follows:

[0053]

[0054] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A time series anomaly detection method based on reconstruction and prediction, characterized by: The following steps are involved: Step 1: Data preprocessing: Remove the feature dimensions that are not related to anomaly detection in the collected IoT data, use interpolation to fill in the missing data for the remaining IoT data, use the Min-Max normalization method to standardize each feature data, and then use the sliding window technology to construct a time series sample; Step 2: Construct a reconstruction network based on the autoencoder architecture and the temporal convolutional network. The reconstruction network adopts the autoencoder architecture, which includes an encoder part and a decoder part. The encoder part uses the temporal convolutional network TCN to extract the low-dimensional features of the input data. The decoder part also uses the temporal convolutional network TCN to map the low-dimensional hidden representation back to the original dimension of the input data layer by layer, thereby reconstructing an output similar to the input data. Step 3: Construct a prediction network based on LSTM and attention mechanism; the prediction network is composed of LSTM and attention mechanism. LSTM captures the long-term dependencies in sequence data. The LSTM can effectively retain important information and forget unnecessary information by setting forget gate, input gate and output gate. The attention mechanism assigns different weights to each time step, so that the model can focus on the importance of different time steps. Step 4: End-to-end model training to evaluate anomaly detection performance; construct anomaly scores by comprehensively considering reconstruction error and prediction error, use the POT algorithm to automatically determine the threshold, and compare the anomaly score with the threshold to determine whether the data point is abnormal.

2. The time series anomaly detection method based on reconstruction and prediction according to claim 1, characterized in that: In the step 1, the formula for normalization using the Min-Max normalization method is: Among them, xi is the collected value of the i-th sensor, min(xtrain,i) is the minimum collected value of the i-th sensor in the training set, max(xtrain,i) is the maximum collected value of the i-th sensor in the training set, and x'i is the collected value of the i-th sensor after standardization.

3. The time series anomaly detection method based on reconstruction and prediction according to claim 2 is characterized in that: In the step 1, the Min-Max normalization method is used for standardization processing to scale the data values ​​to the range of 0 to 1, which effectively alleviates the impact of numerical differences between different features.

4. The time series anomaly detection method based on reconstruction and prediction according to claim 1 is characterized in that: In the step 2, the temporal convolutional network TCN introduces dilated causal convolution, and the main variable parameters are the convolution kernel size k and the dilation factor d. A temporal convolutional network TCN encoder is composed of three TCN residual blocks, each of which contains two dilated causal convolution layers, two WeightNorm layers, two ReLU activation function layers, two dropout layers and a residual connection. The convolution kernel size k of the convolution layer in the residual block is 3, and the dilation factors d are 1, 2, and 4 respectively. In the last layer of the encoder part, it is mapped to the hidden space through a fully connected layer.

5. The time series anomaly detection method based on reconstruction and prediction according to claim 4 is characterized in that: In the step 2, the decoder part maps the potential representation back to the high-dimensional space through a fully connected layer, and then performs a transposed convolution operation to restore it to the same dimension as the input data layer by layer.

6. The time series anomaly detection method based on reconstruction and prediction according to claim 1, characterized in that: In step 3, at each time step, the attention score of the input sequence is calculated, and the attention weight is generated using the softmax function.

7. The time series anomaly detection method based on reconstruction and prediction according to claim 1 is characterized by: The output of the decoder part in step 4 is the reconstructed data of the input data, and the reconstruction error is obtained by calculating the mean square error of the reconstructed data.

Images