Time sequence anomaly detection method based on time-frequency fusion

Abstract

The invention provides a time sequence anomaly detection method based on time-frequency fusion, and belongs to the field of time sequence data anomaly detection. The invention provides a time sequence anomaly detection method based on time-frequency fusion aiming at the problems that potential correlation exists among variables in multivariable time sequence data and frequency domain characteristics are not considered in a traditional method. According to the method, time domain data and frequency domain data in two different modes are innovatively calculated through Spearman correlation coefficients to obtain a time domain similarity matrix and a frequency domain similarity matrix, then fusion is carried out in a point multiplication mode, and potential relations between different variables are mined by using a graph neural network. The time sequence diagram neural network is used for capturing the time dependence and the space relation at the same time, and the features of the time sequence data can be better extracted. The network model is applied to anomaly detection of multivariable time series data, the purpose of improving the accuracy rate can be achieved, and the effectiveness of the method is verified through experimental results.

Description

Technical field:

[0001] The present invention relates to the technology of unsupervised data anomaly detection, and in particular to a time series anomaly detection method based on time-frequency fusion. Background technology:

[0002] Driven by the rapid development of information technology, the era of big data has arrived rapidly, making people's lives gradually digitalized, and the amount of data accumulated is also increasing. Among these data, time series data, as a common information carrier, is particularly prominent. Time series data is composed of numerical values ​​​​expanded in chronological order, and it records the key information of each moment in the work process in a real and detailed manner. In many application scenarios such as industrial production, intelligent manufacturing, the Internet of Things, and intelligent transportation, time series data serves as an important source of basic information.

[0003] However, there are often some outliers in time series data. These outliers do not conform to the distribution law of normal data and are often the precursors of potential failures or the manifestation of actual failures. Because of this, detecting and identifying these abnormal data has become an important means to ensure the stable operation of the system and improve the safety and reliability of the system. With the widespread application of Cyber-Physical Systems (CPS), the number of sensors and interconnected devices has exploded in vehicle systems, industrial automation systems, data centers and other fields, and the demand for real-time monitoring of these systems has also increased dramatically. The security of these key areas depends on the ability to detect anomalies in a timely manner, and anomaly detection plays a vital role in this.

[0004] At present, many methods have been proposed for anomaly detection of multivariate time series data. Most of these methods only focus on the time domain characteristics of the data, that is, anomaly detection is performed through information in the time dimension, while ignoring the potential value of frequency domain information in anomaly detection. At the same time, there are also some methods that only use frequency domain information without fully combining the characteristics of time domain and frequency domain. In fact, time domain information and frequency domain information each have their own advantages: time domain information can reflect the temporal dynamic changes of data, while frequency domain information helps to reveal potential periodic or frequency-related anomalies. Therefore, how to effectively combine time domain and frequency domain information to improve the accuracy and robustness of detection is an issue worthy of in-depth study.

[0005] To solve the above problems, we designed an anomaly detection method based on time-frequency fusion. This method not only makes full use of the dynamic information of time series data in the time dimension, but also conducts in-depth mining from the frequency dimension, combining time domain and frequency domain information, which significantly improves the performance of anomaly detection. On this basis, we also optimized the characteristics of multivariate time series data.

[0006] Multivariate time series data is usually collected and generated by multiple sensors. The relationship between these sensors does not exist independently, but they often have certain correlations. For example, in a vehicle system, the data of different sensors may reflect different aspects of the same component, so there may be strong correlation or coupling between the data of these sensors. However, most current anomaly detection methods do not make full use of this correlation and ignore the complex relationship between multiple sensors. Therefore, relying only on the data of individual sensors for anomaly detection may lead to missed detection or false detection.

[0007] To address this problem, we use the Spearman correlation coefficient for correlation analysis, aiming to more accurately describe the relationship between different sensors. By performing correlation analysis on sensor data, we can extract and quantify the interdependencies between sensors and use them in anomaly detection. To further improve the quality of feature representation, we combine Graph Neural Network (GNN) to capture the complex relationships between sensors by building a graph structure for sensor data. With the help of GNN, we can perform deeper feature extraction on multivariate data, which greatly improves the accuracy and generalization ability of anomaly detection.

[0008] In summary, our time-frequency fusion anomaly detection method combines time domain and frequency domain information, uses graph neural networks to model the relationship between sensors, and achieves efficient anomaly detection of multivariate time series data. This method not only provides new ideas in theory, but also provides strong technical support for the security of actual systems. Summary of the invention:

[0009] The purpose of the present invention is to solve the problem of potential correlation between variables in multivariate time series data and the problem that traditional time series anomaly detection methods often only consider the time domain characteristics of time series data without fully considering the frequency domain characteristics, and propose a time series anomaly detection method based on time-frequency fusion.

[0010] (I) Technical solution

[0011] A time series anomaly detection method based on time-frequency fusion, the method comprises the following steps:

[0012] Step 1: Data preprocessing, including data standardization and data set division.

[0013] Step 2: By graphing the time series data, we obtain the adjacency matrix of the time series data, which is then used as the input of the time series graph neural network.

[0014] Step 3: Convert the training data and test data into frequency domain data through wavelet transform, then calculate the similarity matrix of frequency domain data and time domain data respectively, and finally obtain the final feature adjacency matrix through fusion as the input of the time series graph neural network model, and train it using the prediction error based method.

[0015] Step 4: In the model testing phase, the trained model is used to test the test data.

[0016] The data preprocessing module in step 1 has the following specific steps:

[0017] Step 1-1 Load the training data, test data, and test set label files.

[0018] Step 1-2 Data normalization. Use the z-score normalization method to normalize the data of the training set and the test frame. The z-score is normalized based on the mean and standard deviation of the original data. The formula is as follows:

[0019]

[0020] where x i represents the i-th value of the time series data, represents the mean of the time series data, and s represents the variance of the time series data. This process is performed on each time series data of the training data and the test data.

[0021] Steps 1-3 divide the training data set into training set, validation set, and test set. The training set is used to train the model, the validation set is used to adjust the model parameters, and the test set is used to test the final performance of the model.

[0022] In step 2, the adjacency matrix of the time series data is obtained. The specific steps are:

[0023] Step 2-1 divides the time series data into multiple different windows, and the next time point of each window is used as the predicted label.

[0024] Step 2-2 connects each node in the window with all other nodes to obtain the adjacency matrix of the fully connected time series data.

[0025] In step 3, the time series graph neural network is trained. The specific steps are as follows:

[0026] Step 3-1 transforms the time series data into frequency domain data through wavelet transform.

[0027] Step 3-2 uses the Spearman coefficient to calculate the similarity of the frequency domain data and obtains the similarity matrix of the frequency domain data.

[0028] Step 3-3 initializes an embedding and uses the Spearman coefficient to calculate the similarity matrix as the similarity matrix of the time series data.

[0029] Step 3-4 multiplies the similarity matrix of the time series data with the similarity matrix of the frequency domain data to obtain the fused similarity matrix, and obtains the final feature adjacency matrix through Topk.

[0030] Step 3-5: Input the feature adjacency matrix and time series data into the feature graph attention network of the time series graph neural network to obtain the feature representation of the time series data. Then input the obtained feature representation and the time series adjacency matrix into the time series graph attention network of the time series graph neural network. By capturing the time dependency, the feature representation of the time series data is further obtained. Finally, it is input into the fully connected layer to obtain the predicted value of the time series data. Then, the predicted value and the true value are used as the prediction error loss to train the model. The prediction loss uses L MSE , the formula is as follows:

[0031]

[0032] Among them, N represents the number of samples, M represents the number of features, and y ij represents the true value of the jth feature of the i-th sample, Represents the predicted value of the jth feature of the ith sample predicted by the model. This formula calculates the squared difference between the predicted value and the true value of each feature in each sample and averages all samples and features to get the overall mean square error. The smaller the value of MSE, the better the prediction effect of the model.

[0033] Back propagation process:

[0034] According to the error loss back propagation, the parameters are updated layer by layer. The parameter update process is as follows:

[0035]

[0036] Among them, θ represents the parameter set, θ' represents the updated parameter set, and α is the learning rate.

[0037] Steps 3-6 continuously iterate the time series graph neural network model to obtain an optimized time series graph neural network model.

[0038] The model testing module in step 4 has the following specific steps:

[0039] Step 4-1 Load the test data.

[0040] Step 4-2 inputs the test data into the optimized time series graph neural network model to obtain the predicted label.

[0041] In step 4-3, the predicted label is compared with the actual label to obtain the prediction error. The prediction error is compared with the selected threshold, and the F1-score, AUC, and AP of the test set are calculated to verify the performance of the model.

[0042] (II) Beneficial effects

[0043] 1. The present invention solves the problem of potential correlation between variables in multivariate time series data, and uses graph neural networks to deeply mine the relationship between multiple variables.

[0044] 2. The present invention fully considers the time domain features and frequency domain features through time-frequency fusion. Compared with the traditional time series anomaly detection method, it can extract features more effectively, thereby improving the effect of anomaly detection.

[0045] 3. The present invention uses a time series graph neural network, which can process time series data and graph structure data simultaneously, capture temporal dynamic changes and spatial relationships, thereby realizing the prediction of time series data and improving the effect of anomaly detection. Description of the drawings:

[0046] Figure 1 It is the main program flow chart of the present invention.

[0047] Figure 2 This is a model diagram described in the present invention.

[0048] Figure 3 This is a detailed structural diagram of the temporal neural network described in the present invention. Specific implementation method:

[0049] In order to explain the technical solution of the present invention more clearly and completely, the present invention is further explained below in conjunction with the accompanying drawings and examples.

[0050] The following is a water treatment plant data set SWaT data set obtained from the Singapore University of Technology and Design. It uses 51 sensors in the water treatment plant, 496,800 normal time points in the training set, and 449,919 abnormal time points in the test set as an example. Figure 1 As shown, the present invention provides a time series anomaly detection method based on time-frequency fusion, comprising the following steps:

[0051] Step 1 Figure 1 As shown in the method flow chart, the first step of the method is to input data and perform data preprocessing. The specific steps of data preprocessing are as follows:

[0052] Step 1-1 loads the csv data files corresponding to the training set, test set, and test set labels in the data folder. After loading, the training set has 496,800 data and the test set has 449,919 data. Since the sensor data does not change in a short period of time, in order to reduce the amount of calculation of the model, sampling is performed every 100 time points to obtain 4,968 training data and 4,499 test data. Since the data collected by the sensor was not stable at the beginning, the first 2,160 training data were discarded and 2,808 data were used as training data.

[0053] Step 1-2 uses the z-score standardization method to normalize each data in the training set, validation set, and test set. For example, for the time series data x=(-0.00367,-0.00016,-0.00005.-0.00020,0.00163,...) with a length of 2808, its mean is 0.0008 and the standard deviation is 0.0019. Substitute each value of the sequence x into the formula Then we get a new sequence x=(-2.37456,-0.49505,-0.43354,-0.51591,0.46342,...) with a length of 2808.

[0054] Steps 1-3 divide the data into training set, validation set and test set. The number of data items in the training set is 2527, the validation set is 281, and the test set is 4499. The length of each data item is 51.

[0055] Step 2: Training data model training phase, training model such as Figure 2 As shown in the figure, the time domain data is first converted to the frequency domain, and then graphed in the feature dimension, time-frequency fusion is performed to obtain the fused adjacency matrix. At the same time, the time series data is graphed in the time dimension, and the original time series data, the adjacency matrix of the feature dimension, and the adjacency matrix of the time dimension are simultaneously input into the time series graph neural network, and the model is trained through the prediction error. The specific steps are as follows:

[0056] Step 2-1 Convert 7307 time domain data X T Divide into multiple windows, the next time point S of each window t As the predicted label, the formula is as follows:

[0057] X T =[S t-w ,S t-w+1 ...,S t-1 ]

[0058] Step 2-2 connects each node in the window with all other nodes to obtain an adjacency matrix of the fully connected time domain data.

[0059] Step 2-3: Convert the time domain data X T The frequency domain data X is obtained by wavelet transform F The frequency domain data is calculated by the Spearman coefficient to obtain the frequency domain similarity matrix e F .

[0060] Step 2-4 initializes an embedding and uses the Spearman coefficient to calculate the similarity matrix as the similarity matrix e of the time domain data T .

[0061] Step 2-5 fuses the time domain similarity matrix and the frequency domain similarity matrix by point multiplication, and then obtains the feature adjacency matrix by TopK. The formula is as follows:

[0062] A ji =1{j∈TopK({e ki :k∈C i})}

[0063] Step 2-6 inputs the feature adjacency matrix of time domain data and time-frequency fusion into the feature graph attention network for feature extraction, and inputs the extracted features and the adjacency matrix of time domain data into the time series graph attention network for time series feature extraction. Finally, the dependency relationship of time domain data is captured from the front and back directions through the bidirectional LSTM, as shown in Figure 3 As shown, the time domain prediction is finally performed through the fully connected layer.

[0064] Step 2-7 Predict the value through the time domain With the true value S t The loss is calculated by MSE:

[0065]

[0066] Steps 2-8 continuously optimize the model through back propagation.

[0067] Step 3: Input the test data into the optimized time series graph neural network model, and then calculate the F1-score, AUC, and AP. The specific steps are as follows:

[0068] Step 3-1 Load 4499 test data.

[0069] Step 3-2 Test data passed Figure 2 The model in is used to obtain the time domain prediction value.

[0070] Step 3-3 compares the time domain prediction value with the true value to obtain the prediction error. Then, the prediction error is compared with the selected threshold. If the prediction error is greater than the threshold, it is judged as abnormal. The performance of the model is verified by calculating the F1-score, AUC, and AP of the test set. The F1-score formula is as follows:

[0071]

[0072] Precision is the accuracy, which indicates the proportion of abnormal samples in the detection results. Recall is the recall rate, which indicates the proportion of abnormal classes that are correctly identified. By changing the value of α, a compromise between precision and recall can be achieved. In general, the value is 0.5.

[0073] The data set used in this example comes from a water treatment plant data set SWaT data set from the Singapore University of Technology and Design. The harmonic mean of precision and recall F1-score, AUC, and average precision AP are selected as the performance indicators of the algorithm. Compared with the traditional time series anomaly detection method, the three indicators of the present invention have been improved. The comparison of evaluation indicators is shown in Table 1, where TF-MTAD represents the time series anomaly detection method based on time-frequency fusion proposed by the present invention. In order to verify the stability of the model, 5 random seeds were used, and the three indicators were represented by the average and variance of the 5 random seeds.

[0074] Table 1

[0075] Models AUC (%) AP(%) F1(％) AnoGAN 72.07±0.05 72.4±0.04 63.34±0.03 ALAD 73.32±0.79 71.27±0.66 62.5±0.68 DAGMM 58.08±10.26 60.85±1.71 61.33±1.99 USAD 70.12±0.16 62.79±2.13 64.44±0.07 GDN 83.80±0.27 69.9±1.29 71.61±0.47 STGAT 86.51±0.19 76.39±0.18 76.59±0.16 TF-MTAD 90.51±1.08 80.92±1.02 77.13±1.29

[0076] The above is a detailed description of the embodiments of the present invention in conjunction with the accompanying drawings, and the specific implementation methods herein are only used to help understand the method of the present invention. For those of ordinary skill in the art, according to the idea of ​​the present invention, changes and modifications can be made in the specific implementation methods and application scopes, so the present invention should not be understood as limiting the present invention.

Claims

1. A time series anomaly detection method based on time-frequency fusion, characterized in that: The method comprises the following steps: Step 1: Data preprocessing, including data standardization and data set division. Step 2: Graph the time series data, generate an adjacency matrix and use it as the input of the time series graph neural network. Step 3: Convert the training data and test data into frequency domain data through wavelet transform, calculate the similarity matrix of frequency domain and time domain data respectively, and generate the final feature adjacency matrix as the input of the time series graph neural network through fusion. Training is performed through the prediction error-based method. Step 4: In the model testing phase, the trained model is used to test the test data.

2. The time series anomaly detection method based on time-frequency fusion according to claim 1 is characterized in that: The data preprocessing module in step 1 comprises the following specific steps: Step 1-1 Load the training data, test data, and test set label files. Step 1-2 Data normalization. Use the z-score normalization method to normalize the data of the training set and the test frame. The z-score is normalized based on the mean and standard deviation of the original data. The formula is as follows: Steps 1-3 divide the training data set into a training set and a validation set with a ratio of 9:

1.

3. The time series anomaly detection method based on time-frequency fusion according to claim 1 is characterized in that: The adjacency matrix of the time series data is obtained in step 2, and the specific steps are: Step 2-1 divides the time series data into multiple windows, and the next time point of each window is used as the prediction label. Step 2-2 establishes a fully connected relationship between each node and other nodes in the window and generates a fully connected adjacency matrix.

4. The time series anomaly detection method based on time-frequency fusion according to claim 1 is characterized in that: In step 3, the time series graph neural network is trained, and the specific steps are as follows: Step 3-1 converts the time series data into frequency domain data through wavelet transform. Step 3-2 uses the Spearman coefficient to calculate the similarity of the frequency domain data and obtains the similarity matrix of the frequency domain data. Step 3-3 initializes an embedding and uses the Spearman coefficient to calculate the similarity matrix as the similarity matrix of the time series data. Step 3-4 multiplies the similarity matrix of the time series data with the similarity matrix of the frequency domain data to obtain the fused similarity matrix, and obtains the final feature adjacency matrix through Topk. Step 3-5: Input the feature adjacency matrix and time series data into the feature graph attention network of the time series neural network to obtain the feature representation of the time series data. Then input the obtained feature representation and the time series adjacency matrix into the time series attention network of the time series neural network. By capturing the time dependency, the feature representation of the time series data is further obtained. Finally, it is input into the fully connected layer to obtain the predicted value of the time series data. Then add the predicted value and the true value as the prediction error loss. The model is trained, and the prediction loss uses MSE, the formula is as follows: Among them, N represents the number of samples, M represents the number of features, and y ij represents the true value of the jth feature of the i-th sample, Represents the predicted value of the jth feature of the ith sample predicted by the model. This formula calculates the squared difference between the predicted value and the true value of each feature in each sample and averages all samples and features to get the overall mean square error. The smaller the value of MSE, the better the prediction effect of the model. Back propagation process: According to the error loss back propagation, the parameters are updated layer by layer. The parameter update process is as follows: Among them, θ represents the parameter set, θ' represents the updated parameter set, and α is the learning rate. The time series graph neural network model is continuously iterated to obtain an optimized time series graph neural network model.

5. The time series anomaly detection method based on time-frequency fusion according to claim 1 is characterized in that: The model testing module in step 4 includes the following specific steps: Step 4-1 Load the test data. Step 4-2 inputs the test data into the optimized time series graph neural network model to obtain the predicted label. In step 4-3, the predicted label is compared with the actual label to obtain the prediction error. The prediction error is compared with the selected threshold, and the F1-score, AUC, and AP of the test set are calculated to verify the performance of the model.

Images