An unsupervised domain adaptation method for anomaly detection in multivariate time series data

Abstract

The present invention discloses an unsupervised domain-adaptive multivariate time series data anomaly detection method, which comprises the following steps: in an offline training phase, a backbone network is trained through an unsupervised multi-task learning framework consisting of a prediction task and a reconstruction task; the training set multivariate time series data is input into the backbone network, and domain adaptation window normalization, significant anomaly error set scanning, and POT threshold selection operations are sequentially performed to confirm the optimal threshold; in an online detection phase, the multivariate time series data to be tested is input into the backbone network, and domain adaptation window normalization and significant anomaly error set scanning operations are sequentially performed to obtain anomaly scores and significant anomaly error sets, and the anomaly scores are compared with the optimal threshold to output anomaly detection results. The unsupervised domain-adaptive multivariate time series data anomaly detection method proposed by the present invention solves the domain drift problem of multivariate time series data anomaly detection, and has the advantages of lightweight model, superior performance, and good interpretability.

Description

Technical Field

[0001] The present invention relates to the technical field of signal processing and anomaly detection, and in particular to an unsupervised domain-adaptive multivariate time series data anomaly detection method. Background Art

[0002] Many real-world devices, such as servers, spacecraft, and automobile engines, are equipped with numerous sensors that collect state signals from multivariate time series. Automatic anomaly detection in time series can promptly detect anomalies and alert operators to prevent irreversible losses. With the development of large-scale equipment, the increasing number of sensors has brought greater challenges to the effective detection of anomalies in multivariate time series data. Traditional distance-based, clustering, or classification-based methods are unable to effectively model complex multivariate time series data, resulting in poor detection results. Currently, most methods use deep learning to model multivariate time series data. For example, recurrent neural networks are used to capture time series features and then use the error between predicted and true results as an anomaly score. Generative adversarial networks are also used to enhance the robustness of predictions. Other work uses graph neural networks to model relationships between variables and thus improve prediction accuracy. These methods require learning a decision threshold for anomaly detection. When the given anomaly score exceeds the decision threshold, the moment is considered an anomaly. However, due to the dynamic and complex nature of multivariate time series data in real life, the distribution of anomaly scores between training and test sets can differ significantly. If researchers generalize the decision thresholds used in the training set to the test set, detection performance can degrade significantly. This problem, also known as domain drift, exists in most existing methods. Furthermore, the lack of labeled datasets and the significant cost of data annotation in real life, as well as the complex models and poor interpretability of deep learning methods, are major challenges in detecting anomaly in multivariate time series data. Summary of the Invention

[0003] The purpose of the present invention is to solve the above-mentioned defects in the prior art and provide an unsupervised domain-adaptive multivariate time series data anomaly detection method. The backbone network is trained through an unsupervised multi-task learning framework to solve the problem that multivariate time series data lacks labels and the data annotation cost is huge; a new domain-adaptive window normalization method is proposed to reduce the distribution difference of anomaly scores between the training set and the test set, solve the domain drift problem, and effectively improve the generalization ability of the detection model; at the same time, a non-parametric statistical scanning method is used to select a subset of significant anomaly errors, which reduces the side effects of dynamic noise and highlights the significant anomaly parts of the time series data, thereby bringing better detection effects and interpretability.

[0004] The purpose of the present invention can be achieved by taking the following technical solutions:

[0005] A multivariate time series data anomaly detection method based on unsupervised domain adaptation, comprising the following steps:

[0006] S1, offline training stage, offline sensors installed on devices such as servers, spacecraft and automobile engines collect signals to obtain multivariate time series data of the training set, preprocess the multivariate time series data of the training set, and input the preprocessed data into the backbone network. The backbone network first extracts deep time series features through the dilated convolutional autoencoder, and then inputs the deep time series features into the prediction network and reconstruction network respectively to obtain prediction results and reconstruction results. The backbone network is trained through an unsupervised multi-task learning framework consisting of prediction tasks and reconstruction tasks;

[0007] S2. Input the preprocessed training set multivariate time series data in S1 into the trained backbone network to obtain prediction results and reconstruction results, and calculate the error between the results and the true value to obtain the prediction error and reconstruction error. The prediction error and reconstruction error are sequentially subjected to domain adaptation window normalization operations and significant anomaly error set scanning operations to obtain the training set anomaly score. The training set anomaly score is subjected to POT threshold selection operations to obtain the optimal threshold.

[0008] S3, in the online detection stage, the multivariate time series data to be tested is preprocessed and then input into the trained backbone network to obtain the prediction result and reconstruction result, and the error is calculated by comparing the result with the true value to obtain the prediction error and reconstruction error. The prediction error and reconstruction error are subjected to domain adaptation window normalization and significant anomaly error set scanning operations in turn to obtain the significant anomaly error set and anomaly score. The anomaly score is compared with the optimal threshold in S2 to output the anomaly detection result.

[0009] Furthermore, the process of step S1 is as follows:

[0010] S11. Denoise and normalize the training set multivariate time series data to obtain the processed training set multivariate time series data x. Process the training set multivariate time series data x into the training set multivariate time series window data X, which is expressed as:

[0011] x={x1,x2,...x t ,...,x M},x t ∈R k

[0012] X={X1,X2,...X t ,...,X M}

[0013] X t =[x t-T ,x t-T+1 ,…,x t-1 ]∈Rk×T

[0014] Where M represents the total number of moments of the multivariate time series data in the training set, k represents the feature dimension of the multivariate time series data, R represents the real number domain, and x t represents the multivariate time series data of the training set at time t, T represents the length of the multivariate time series window data at any time, X t The multivariate time series window data of the training set at time t, including the multivariate time series data of the training set from time tT to time t-1;

[0015] S12. For each moment t in the multivariate time series data of the training set, t Input the dilated convolutional autoencoder to extract the deep temporal feature Z at time t t =[z t-T ,z t-T+1 ,…,z t-1 ], the dilated convolutional autoencoder consists of L layers of TemporalBlocks connected in sequence. The TemporalBlocks are connected from the input layer to the output layer in sequence as follows: dilated convolution layer Conv1d-1, dilated convolution layer Conv1d-2, padding layer Chomp1d-3, padding layer Chomp1d-4, BN layer BatchNorm1d-5, BN layer BatchNorm1d-6, ReLU layer ReLU-7, ReLU layer ReLU-8, Dropout layer Dropout-9 , Dropout layer Dropout-10, dilated convolution layer Conv1d-11, dilated convolution layer Conv1d-12, padding layer Chomp1d-13, padding layer Chomp1d-14, BN layer BatchNorm1d-15, BN layer BatchNorm1d-16, ReLU layer ReLU-17, ReLU layer ReLU-18, Dropout layer Dropout-19, Dropout layer Dropout-20, ReLU layer ReLU-21;

[0016] Furthermore, the dilated convolution in the dilated convolutional autoencoder in step S12 was proposed by Yu and Koltun in their 2015 paper "Multi-Scale Context Aggregation by Dilated Convolutions." It is a significant invention in the field of convolution-related technologies, primarily used in fields such as speech recognition, computer vision, and sequence modeling. Dilated convolution injects holes into the standard convolution kernel to increase the model's receptive field. Compared to the original normal convolution operation, in addition to the convolution kernel size, stride, and padding, the dilated convolution has an additional dilation factor, which defines the spacing between parameters in the convolution kernel. By increasing the dilation factor of the dilated convolution layer by layer, the receptive field of the dilated convolutional autoencoder gradually increases to cover the entire multivariate time series window data, thereby effectively modeling the relationship between each moment of the multivariate time series window data and extracting more robust deep time series features.

[0017] S13, Z t The deep temporal feature z at time t-1 t-1 As the input of the prediction network, z t-1 Copy T copies as the input of the reconstructed network to obtain the prediction result at time t and reconstructing multivariate time series window data Contains the reconstruction results from time tT to time t-1, where the prediction network is a fully connected network, and the expression formula is:

[0018]

[0019] Among them, W is the weight parameter of the fully connected network, b is the bias parameter of the fully connected network, the reconstruction network is a reverse dilated convolutional decoder, and the connection from the input layer to the output layer is: L layer is connected in sequence: TemporalBlock, linear layer Linear-1, linear layer Linear-2, ReLU layer ReLU-3, Dropout layer Dropout-4, linear layer Linear-5, ReLU layer ReLU-6, Dropout layer Dropout-7, linear layer Linear-8, ReLU layer ReLU-9, Dropout layer Dropout-10, linear layer Linear-11;

[0020] Furthermore, the reconstructed network in step S13 is an inverse dilated convolution decoder. Unlike the dilated convolution encoder in S12, the dilation factor of the dilated convolution in the inverse dilated convolution decoder decreases layer by layer. The purpose is to reconstruct the multivariate time series window data from coarse granularity to fine granularity, thereby obtaining a better reconstruction effect.

[0021] S14. For each moment t in the multivariate time series data of the training set, and Respectively with x t and X t Calculate the bi-norm error, construct the joint loss function, and optimize the backbone network by gradient descent. The expression is as follows:

[0022]

[0023]

[0024] Loss total =(1-α)Loss recon +αLoss pre

[0025] Among them, Loss pre Represents the prediction loss function, ||||2 represents the vector two norm, Loss recon Represents the reconstruction loss function, Loss total is the joint loss function, and α is the balancing factor.

[0026] Furthermore, in step S14, gradient descent is used to optimize the joint loss function. By minimizing the joint loss function, the backbone network can achieve accurate prediction and reconstruction, while simultaneously modeling global and local information. Since more than 90% of the data in the training set is normal data, the unsupervised multi-task learning framework enables the backbone network to model the normal data distribution, thus enabling anomaly detection capabilities.

[0027] Furthermore, the process of step S2 is as follows:

[0028] S21, for each moment t in the multivariate time series data of the training set, replace X in the multivariate time series window data X in the training set in S1 t and X t+1 Input the trained backbone network to obtain the prediction result at time t and the reconstructed multivariate time series window data at time t+1 Pick Reconstruction results at time t Will and and the training set multivariate time series data x at time t t Calculate the absolute value error to get the prediction error ε at time t t and reconstruction error r t , the calculation formula is as follows:

[0029]

[0030]

[0031] Among them, || means taking the absolute value;

[0032] S22, the prediction error ε at each moment t in the training set multivariate time series data t and reconstruction error r t Perform domain adaptation window normalization operation to obtain the error vector e at time t t The specific process is as follows:

[0033] S221, the prediction error ε for each moment t in the training set multivariate time series data t and reconstruction error r t , take the prediction error window data E of length w before time t t and reconstruction error window data R t , the expression formula is as follows:

[0034] E t =[ε t-w ,ε t-w+1 ,…,ε t-1 ]

[0035] R t =[r t-w ,r t-w+1 ,…,r t-1 ]

[0036] S222, ε t and r t Perform domain adaptation window normalization operation to obtain the prediction error at time t after normalization and reconstruction error Then and The error vector e at time t is obtained by splicing t , the expression formula is as follows:

[0037]

[0038]

[0039]

[0040] Among them, μ(E t ) represents the prediction error window data E t The mean value, μ(R t ) represents the reconstruction error window data R t The mean value, σ(E t ) represents the prediction error window data E t The variance, σ(R t ) represents the reconstruction error window data R tThe variance of , ⊕ represents splicing along the feature dimension;

[0041] Furthermore, the domain adaptation window normalization operation in step S22 normalizes the prediction error and reconstruction error at the current moment by the mean and variance of the prediction error window data and the reconstruction error window data, adaptively normalizing the prediction error and reconstruction error to the same scale, and the relative size relationship between the errors does not change, thereby improving the domain adaptation capability.

[0042] S23, the training set error vector D obtained in S22 train ={e1,e2,...e t ,...e M}Save as background error set for nonparametric statistical scanning;

[0043] S24, the error vector e at each moment t in the multivariate time series data of the training set t Perform a significant abnormal error set scanning operation to obtain the significant abnormal error set S at time t t , and calculate the anomaly score at time t The specific process is as follows:

[0044] S241. Calculate e for each t moment in the training set multivariate time series data. t The p-value p of the error for each feature dimension t,j , get the p-value set p at time t t ={p t,1 ,p t,2 ,...p t,j ,...p t,k}, and its calculation formula is as follows:

[0045]

[0046] Among them, p t,j Represents the error vector e at time t t The p-value of the error of the j-th dimension feature, D train represents the background error set, e i represents the i-th error vector in the background error set, e i,j Represents the i-th error vector e in the background error set i The error of the j-th dimension feature, e t,j Represents the error vector e at time t t The error of the j-th dimension feature, I() represents the counting function, which is 1 if the conditions in the brackets are met, otherwise it is 0;

[0047] S242, filter p t The significance is greater than the significance threshold αmax The p-value is obtained Meanwhile, for sorting is performed to obtain the sorted p-value set at time t The expression formula is as follows:

[0048]

[0049]

[0050] Where represents the filtered p-value set at time t, Sort() represents the sorting function, which sorts the data from smallest to largest, represents the p-value set with the number n after sorting at time t;

[0051] S243. By performing non-parametric statistical scanning on the significant abnormal p-value set at time t is obtained where n * represents the number of p-values in the significant abnormal p-value set, and n * << n, mapping the p-values in to the error vector e at time t, t the significant abnormal error set S at time t is obtained by taking the error corresponding to the feature dimension in t , and finally, the abnormal score at time t is calculated through S t The calculation formula is as follows: The calculation formula is as follows:

[0052] Furthermore, the non-parametric statistical scanning method in step S243 was proposed by Neill D B and Lingwall J in the 2007 paper "A nonparametric scan statistic for multivariate disease surveillance". Since the non-parametric statistical scanning method does not require assumptions about the data distribution, it is widely used in pattern detection tasks. The principle of the non-parametric statistical scanning method is to regard the pattern detection task as a search task for finding a significant abnormal set, and it can find the significant abnormal set with the largest deviation degree. By performing non-parametric statistical scanning on the sorted p-value set the p-value that is most significant from the background error set is obtained, and the significant abnormal p-value set Finally, a set of significant anomaly errors is obtained through mapping. Through non-parametric statistical scanning, the side effects of dynamic noise in multivariate time series data are reduced, and the significant anomalies in the multivariate time series data are highlighted, thereby achieving better detection results and interpretability.

[0053] S25, the training set anomaly score obtained in S24 Perform POT threshold selection operation to obtain the optimal threshold th * .

[0054] Furthermore, the POT threshold selection operation described in step S25 was proposed by Siffer A in 2017 and published in the paper "Anomaly Detection in Streams with Extreme Value Theory" presented at the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. The basic idea of ​​this method is to apply extreme value theory to fit the extreme value distribution of the data using a generalized Pato distribution. The parameters of the generalized Pato distribution are then estimated using maximum likelihood estimation to determine the optimal threshold. The POT threshold selection operation fits the distribution of extreme values ​​in the anomaly scores of the training set and determines the optimal threshold.

[0055] Furthermore, the process of step S3 is as follows:

[0056] S31, in the online detection stage, the multivariate time series data to be tested collected by the online sensor is collected, denoised and normalized to obtain the processed multivariate time series data to be tested x′, and then x′ is processed into the multivariate time series window data to be tested X′, which is expressed as:

[0057] x′={x1′,x2′,...x t ′,...,x N ′},x t ′∈R k

[0058] X′={X1′,X2′,...X t ′,...,X N ′}

[0059] X t ′=[x t-T ′,x t-T+1 ′,…,x t-1 ′]∈R k×T

[0060] Among them, x t ′ represents the multivariate time series data to be measured at time t, X t' represents the multivariate time series window data to be tested at time t, including the multivariate time series data to be tested from time tT to time t-1, and N represents the total number of time points of the multivariate time series data to be tested;

[0061] S32, at each time t in the multivariate time series data to be tested, the X in the multivariate time series window data X′ to be tested in S31 is converted to t ′ and X t+1 ' Input the trained backbone network to obtain the prediction result at time t and the reconstructed multivariate time series window data at time t+1 Pick Reconstruction results at time t Will and and the multivariate time series data x at time t t 'Calculate the absolute value error to get the prediction error ε at time t t ′ and reconstruction error r t ';

[0062] S33, the prediction error ε at each time t in the multivariate time series data to be tested t ′ and reconstruction error r t ′ performs domain adaptation window normalization operation to obtain the error vector e at time t t ';

[0063] S34, the error vector e at each moment t in the multivariate time series data to be measured t ′ Perform a significant abnormal error set scanning operation to obtain the significant abnormal error set S at time t t ′, and calculate the anomaly score at time t

[0064] S35. Anomaly score at each time t in the multivariate time series data to be tested and the optimal threshold th obtained in S2 * For comparison, if Greater than th * , then the detection result y at time t t If it is 1, it means abnormal, otherwise the detection result y at time t t 0 means normal, and finally output the detection result y at time t t , anomaly score And the set of significant abnormal errors S t ′.

[0065] The present invention has the following advantages and effects compared to the prior art:

[0066] (1) This paper proposes a novel unsupervised multi-task learning framework that can robustly model multivariate time series data by jointly optimizing a backbone network consisting of a dilated convolutional autoencoder, a fully connected prediction network, and a reverse dilated convolutional decoder reconstruction network.

[0067] (2) This paper further proposes a new domain-adaptive window normalization method, which normalizes the error at the current moment using historical error window data to reduce the distribution difference of anomaly scores in the training set and test set, solves the domain drift problem, and effectively improves the generalization ability of the detection model;

[0068] (3) The present invention simultaneously utilizes the significant abnormal error set scanning to select the significant abnormal error set, thereby reducing the side effects of dynamic noise in the multivariate time series data and highlighting the significant abnormal parts in the multivariate time series data, thereby bringing better detection effect and interpretability. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0070] Figure 1 It is a flowchart of the unsupervised domain-adaptive multivariate time series data anomaly detection method disclosed in the present invention;

[0071] Figure 2 It is a schematic diagram of the backbone network structure disclosed in the present invention;

[0072] Figure 3 This is a detection effect diagram of the unsupervised domain-adaptive multivariate time series data anomaly detection method disclosed in the present invention. DETAILED DESCRIPTION

[0073] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0074] Example 1

[0075] Figure 1 The flowchart of the unsupervised domain-adaptive multivariate time series data anomaly detection method disclosed in the present invention is given. Taking the data collected by the server sensor of an Internet company as an example, the method specifically includes the following steps:

[0076] In the offline training phase, the data collected by the sensors of a server of an Internet company is used as the training set. The multivariate time series data of the training set is preprocessed and the preprocessed data is input into the backbone network. The backbone network structure is as follows: Figure 2 As shown in the figure, the backbone network first extracts deep time series features through a dilated convolutional autoencoder, and then inputs the deep time series features into the prediction network and reconstruction network to obtain prediction results and reconstruction results respectively. The backbone network is trained through an unsupervised multi-task learning framework consisting of prediction tasks and reconstruction tasks.

[0077] The specific implementation of step S1 is:

[0078] S11. The data from the server sensors of an Internet company is used as the training set. The total number of moments in the training set is 708,400, and the feature dimension is 38. The multivariate time series data of the training set is denoised and normalized to obtain the processed multivariate time series data x of the training set. The multivariate time series data x of the training set is processed into the multivariate time series window data X of the training set, which is expressed as:

[0079] x={x1,x2,...x t ,...,x M},x t ∈R k

[0080] X={X1,X2,...X t ,...,X M}

[0081] X t =[x t-T ,x t-T+1 ,…,x t-1 ]∈R k×T

[0082] Where M represents the total number of moments in the multivariate time series data of the training set, which is 708400 here, k represents the feature dimension of the multivariate time series data, which is 38 here, R represents the real number domain, and x t represents the multivariate time series data of the training set at time t, T represents the length of the multivariate time series window data at any time, where the length T is 100, X t The multivariate time series window data of the training set at time t, including the multivariate time series data of the training set from time tT to time t-1;

[0083] S12. For each moment t in the multivariate time series data of the training set, t Input the dilated convolutional autoencoder to extract the deep temporal feature Z at time t t =[z t-T ,z t-T+1 ,…,z t-1], the dilated convolutional autoencoder consists of 7 layers of TemporalBlock connected in sequence. The TemporalBlock is connected from the input layer to the output layer in sequence as follows: dilated convolution layer Conv1d-1, dilated convolution layer Conv1d-2, padding layer Chomp1d-3, padding layer Chomp1d-4, BN layer BatchNorm1d-5, BN layer BatchNorm1d-6, ReLU layer ReLU-7, ReLU layer ReLU-8, Dropout layer Dropout-9 , Dropout layer Dropout-10, dilated convolution layer Conv1d-11, dilated convolution layer Conv1d-12, padding layer Chomp1d-13, padding layer Chomp1d-14, BN layer BatchNorm1d-15, BN layer BatchNorm1d-16, ReLU layer ReLU-17, ReLU layer ReLU-18, Dropout layer Dropout-19, Dropout layer Dropout-20, ReLU layer ReLU-21;

[0084] S13, Z t The deep temporal feature z at time t-1 t-1 As the input of the prediction network, z t-1 Copy T copies as the input of the reconstructed network to obtain the prediction result at time t and reconstructing multivariate time series window data Contains the reconstruction results from time tT to time t-1, where the prediction network is a fully connected network, and the expression formula is:

[0085]

[0086] Among them, W is the weight parameter of the fully connected network, b is the bias parameter of the fully connected network, the reconstruction network is a reverse dilated convolutional decoder, and the connection from the input layer to the output layer is: 7 layers of TemporalBlock connected in sequence, linear layer Linear-1, linear layer Linear-2, ReLU layer ReLU-3, Dropout layer Dropout-4, linear layer Linear-5, ReLU layer ReLU-6, Dropout layer Dropout-7, linear layer Linear-8, ReLU layer ReLU-9, Dropout layer Dropout-10, linear layer Linear-11;

[0087] In order to meet the receptive field of the multivariate time series window data length of 100, the convolution kernel size of the dilated convolution in S12 and S13 is 3 and the dilation factor is 2.

[0088] S14. For each moment t in the multivariate time series data of the training set, and Respectively with x t and X t Calculate the bi-norm error, construct the joint loss function, and optimize the backbone network by gradient descent. The expression is as follows:

[0089]

[0090]

[0091] Loss total =(1-α)Loss recon +αLoss pre

[0092] Among them, Loss pre Represents the prediction loss function, ||||2 represents the vector two norm, Loss recom Represents the reconstruction loss function, Loss total is the joint loss function, and α is the balancing factor.

[0093] The S14 training process uses 8 1080Ti NVIDIA graphics cards for training, with 30 training rounds; the Adam optimizer is used for optimization, the learning rate is set to 0.001, and the early stopping strategy is used; the balance factor α is set to 0.5.

[0094] S2. Input the preprocessed training set multivariate time series data in S1 into the trained backbone network to obtain prediction results and reconstruction results, and calculate the error between the results and the true value to obtain the prediction error and reconstruction error. The prediction error and reconstruction error are sequentially subjected to domain adaptation window normalization operations and significant anomaly error set scanning operations to obtain the training set anomaly score. The training set anomaly score is subjected to POT threshold selection operations to obtain the optimal threshold.

[0095] The specific implementation of step S2 is:

[0096] S21, for each moment t in the multivariate time series data of the training set, replace X in the multivariate time series window data X in the training set in S1 t and X t+1 Input the trained backbone network to obtain the prediction result at time t and the reconstructed multivariate time series window data at time t+1 Pick Reconstruction results at time t Will and and the training set multivariate time series data x at time t t Calculate the absolute value error to get the prediction error ε at time tt and reconstruction error r t , the calculation formula is as follows:

[0097]

[0098]

[0099] Among them, || means taking the absolute value;

[0100] S22, the prediction error ε at each moment t in the training set multivariate time series data t and reconstruction error r t Perform domain adaptation window normalization operation to obtain the error vector e at time t t The specific process is as follows:

[0101] S221, the prediction error ε for each moment t in the training set multivariate time series data t and reconstruction error r t , take the prediction error window data E of length w before time t t and reconstruction error window data R t , the expression formula is as follows:

[0102] E t =[ε t-w ,ε t-w+1 ,…,ε t-1 ]

[0103] R t =[r t-w ,r t-w+1 ,…,r t-1 ]

[0104] S222, ε t and r t Perform domain adaptation window normalization operation to obtain the prediction error at time t after normalization and reconstruction error Then and The error vector e at time t is obtained by splicing t , the expression formula is as follows:

[0105]

[0106]

[0107]

[0108] Among them, μ(E t ) represents the prediction error window data E t The mean value, μ(R t) represents the reconstruction error window data R t The mean value, σ(E t ) represents the prediction error window data E t The variance, σ(R t ) represents the reconstruction error window data R t The variance of , ⊕ represents splicing along the feature dimension;

[0109] In order to ensure stable statistical characteristics, the length w of the prediction error window data and the reconstruction error window data is 100.

[0110] S23, the training set error vector D obtained in S22 train ={e1,e2,...e t ,...e M}Save as background error set for nonparametric statistical scanning;

[0111] S24, the error vector e at each moment t in the multivariate time series data of the training set t Perform a significant abnormal error set scanning operation to obtain the significant abnormal error set S at time t t , and calculate the anomaly score at time t The specific process is as follows:

[0112] S241. Calculate e for each t moment in the training set multivariate time series data. t The p-value p of the error for each feature dimension t,j , get the p-value set p at time t t ={p t,1 ,p t,2 ,...p t,j ,...p t,k}, and its calculation formula is as follows:

[0113]

[0114] Among them, p t,j Represents the error vector e at time t t The p-value of the error of the j-th dimension feature, D train represents the background error set, e i represents the i-th error vector in the background error set, e i,j Represents the i-th error vector e in the background error set i The error of the j-th dimension feature, e t,j Represents the error vector e at time t i The error of the j-th dimension feature, I() represents the counting function, which is 1 if the conditions in the brackets are met, otherwise it is 0;

[0115] S242, filter pt The p-value with medium significance greater than the significance threshold α max is obtained Meanwhile, for sorting is performed to obtain the sorted p-value set at time t The expression formula is as follows:

[0116]

[0117]

[0118] Where represents the filtered p-value set at time t, Sort() represents the sorting function, which sorts the data from small to large, represents the p-value set with a quantity of n sorted at time t;

[0119] S243. By performing non-parametric statistical scanning on the significant abnormal p-value set at time t is obtained Where n * represents the quantity of p-values in the significant abnormal p-value set, and n * << n, mapping the p-values in to the error vector e at time t t the corresponding error of the feature dimension in to obtain the significant abnormal error set S at time t t , and finally through S t calculate the anomaly score at time t The calculation formula is as follows:

[0120]

[0121] S25. The anomaly scores of the training set obtained in S24 perform the POT threshold selection operation to obtain the optimal threshold th * , and the optimal threshold obtained here is 4,072.

[0122] S3. In the online detection stage, the measured multivariate time series data is preprocessed and then input into the trained backbone network to obtain the prediction result and the reconstruction result, and the errors are calculated between the results and the true values to obtain the prediction error and the reconstruction error. The prediction error and the reconstruction error are sequentially subjected to domain adaptation window normalization and significant abnormal error set scanning operations to obtain the significant abnormal error set and the anomaly score. The anomaly score is compared with the optimal threshold in S2 to output the anomaly detection result.

[0123] The specific implementation manner of step S3 is:

[0124] S31, in the online detection stage, the multivariate time series data to be tested collected by the online sensor is collected, denoised and normalized to obtain the processed multivariate time series data to be tested x′, and then x′ is processed into the multivariate time series window data to be tested X′, which is expressed as:

[0125] x′={x1′,x2′,...x t ′,...,x N ′},x t ′∈R k

[0126] X′={X1′,X2′,...X t ′,...,X N ′}

[0127] X t ′=[x t-T ′,x t-T+1 ′,…,x t-1 ′]∈R k×T

[0128] Among them, x t ′ represents the multivariate time series data to be measured at time t, X t ' represents the multivariate time series window data to be tested at time t, including the multivariate time series data to be tested from time tT to time t-1, and N represents the total number of time points of the multivariate time series data to be tested;

[0129] The total number of collected multivariate time series data moments N is 1000.

[0130] S32, at each time t in the multivariate time series data to be tested, the X in the multivariate time series window data X′ to be tested in S31 is converted to t ′ and X t+1 ' Input the trained backbone network to obtain the prediction result at time t and the reconstructed multivariate time series window data at time t+1 Pick Reconstruction results at time t Will and and the multivariate time series data x at time t t 'Calculate the absolute value error to get the prediction error ε at time t t ′ and reconstruction error r t ';

[0131] S33, the prediction error ε at each time t in the multivariate time series data to be tested t ′ and reconstruction error r t ′ performs domain adaptation window normalization operation to obtain the error vector e at time t t ';

[0132] S34, the error vector e at each moment t in the multivariate time series data to be measured t ′ Perform a significant abnormal error set scanning operation to obtain the significant abnormal error set S at time t t ′, and calculate the anomaly score at time t

[0133] S35. Anomaly score at each time t in the multivariate time series data to be tested and the optimal threshold th obtained in S2 * For comparison, if Greater than th * , then the detection result y at time t t If it is 1, it means abnormal, otherwise the detection result y at time t t 0 means normal, and finally output the detection result y at time t t , anomaly score And the set of significant abnormal errors S t ′.

[0134] After S34 and S35, the abnormal scores at t = {285 to 308}, a total of 24 times, are obtained. Compare with the optimal threshold to get the detection result 5y 285 ~y 308}{51,0,0,1,0,1,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0}, and at the same time, we get the set of significant abnormal errors selected at each of the 24 moments t={285~308}. In actual production, abnormalities occur continuously, and it is very important to be able to detect and locate abnormalities in time within the abnormal segment. At the abnormal start time t=285, the detection result y 285 =1, which means that abnormalities can be discovered in time.

[0135] Figure 3 The left subfigure shows the selected significant anomaly error sets for each of the 24 time points t = {285-308}. The right subfigure shows the result of mapping these selected significant anomaly error sets to the corresponding multivariate time series data features under test, with t = {285-308} marked with a dashed box. The results show that the multivariate time series data corresponding to the selected significant anomaly error sets exhibits significant anomalies, and the significant anomaly error sets can be used to locate the anomalous sensor.

[0136] The above embodiments are preferred implementations of the present invention, but the implementations of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A multivariate time series data anomaly detection method with unsupervised domain adaptation, characterized in that: The multivariate time series data anomaly detection method comprises the following steps: S1, offline training stage, offline sensors installed on devices such as servers, spacecraft and automobile engines collect signals to obtain multivariate time series data of the training set, preprocess the multivariate time series data of the training set, and input the preprocessed data into the backbone network. The backbone network first extracts deep time series features through the dilated convolutional autoencoder, and then inputs the deep time series features into the prediction network and reconstruction network respectively to obtain prediction results and reconstruction results. The backbone network is trained through an unsupervised multi-task learning framework consisting of prediction tasks and reconstruction tasks; S2. Input the preprocessed training set multivariate time series data in S1 into the trained backbone network to obtain prediction results and reconstruction results, and calculate the error between the results and the true value to obtain the prediction error and reconstruction error. The prediction error and reconstruction error are sequentially subjected to domain adaptation window normalization operation and significant abnormal error set scanning operation to obtain the training set anomaly score. The training set anomaly score is subjected to POT threshold selection operation to obtain the optimal threshold. The process of step S2 is as follows: S21, for each moment t in the multivariate time series data of the training set, replace X in the multivariate time series window data X in the training set in S1 t and X t+1 Input the trained backbone network to obtain the prediction result at time t and the reconstructed multivariate time series window data at time t+1 Pick Reconstruction results at time t Will and and the training set multivariate time series data x at time t t Calculate the absolute value error to get the prediction error ε at time t t and reconstruction error r t , the calculation formula is as follows: Among them, || means taking the absolute value; S22, the prediction error ε at each moment t in the training set multivariate time series data t and reconstruction error r t Perform domain adaptation window normalization operation to obtain the error vector e at time t t ; S23, the training set error vector D obtained in S22 train ={e1,e2,...e t ,...e M }Save as background error set for nonparametric statistical scanning; S24, the error vector e at each moment t in the multivariate time series data of the training set t Perform a significant abnormal error set scanning operation to obtain the significant abnormal error set S at time t t , and calculate the anomaly score at time t S25, the training set anomaly score obtained in S24 Perform POT threshold selection operation to obtain the optimal threshold th * ; S3, in the online detection stage, the multivariate time series data to be tested is preprocessed and then input into the trained backbone network to obtain the prediction result and reconstruction result, and the error is calculated by comparing the result with the true value to obtain the prediction error and reconstruction error. The prediction error and reconstruction error are subjected to domain adaptation window normalization and significant anomaly error set scanning operations in turn to obtain the significant anomaly error set and anomaly score. The anomaly score is compared with the optimal threshold in S2 to output the anomaly detection result.

2. The unsupervised domain adaptive multivariate time series data anomaly detection method according to claim 1, characterized in that: The process of step S1 is as follows: S11. Denoise and normalize the training set multivariate time series data to obtain the processed training set multivariate time series data x. Process the training set multivariate time series data x into the training set multivariate time series window data X, which is expressed as: x={x1,x2,...x t ,…,x M },x t ∈R k X={X1,X2,...X t ,…,X M } X t =[x t-T ,x t-T+1 ,…,x t-1 ]∈R k×T Where M represents the total number of moments of the multivariate time series data in the training set, k represents the feature dimension of the multivariate time series data, R represents the real number domain, and x t represents the multivariate time series data of the training set at time t, T represents the length of the multivariate time series window data at any time, X t The multivariate time series window data of the training set at time t, including the multivariate time series data of the training set from time tT to time t-1; S12. For each moment t in the multivariate time series data of the training set, t Input the dilated convolutional autoencoder to extract the deep temporal feature Z at time t t =[z t-T ,z t-T+1 ,…,z t-1 ], the dilated convolutional autoencoder consists of L layers of TemporalBlocks connected in sequence. The TemporalBlocks are connected from the input layer to the output layer in sequence as follows: dilated convolution layer Conv1d-1, dilated convolution layer Conv1d-2, padding layer Chomp1d-3, padding layer Chomp1d-4, BN layer BatchNorm1d-5, BN layer BatchNorm1d-6, ReLU layer ReLU-7, ReLU layer ReLU-8, Dropout layer Dropout-9 , Dropout layer Dropout-10, dilated convolution layer Conv1d-11, dilated convolution layer Conv1d-12, padding layer Chomp1d-13, padding layer Chomp1d-14, BN layer BatchNorm1d-15, BN layer BatchNorm1d-16, ReLU layer ReLU-17, ReLU layer ReLU-18, Dropout layer Dropout-19, Dropout layer Dropout-20, ReLU layer ReLU-21; S13, Z t The deep temporal feature z at time t-1 t-1 As the input of the prediction network, z t-1 Copy T copies as the input of the reconstructed network to obtain the prediction result at time t and reconstructing multivariate time series window data Contains the reconstruction results from time tT to time t-1, where the prediction network is a fully connected network, and the expression formula is: Among them, W is the weight parameter of the fully connected network, b is the bias parameter of the fully connected network, the reconstruction network is a reverse dilated convolutional decoder, and the connection from the input layer to the output layer is: L layer is connected in sequence: TemporalBlock, linear layer Linear-1, linear layer Linear-2, ReLU layer ReLU-3, Dropout layer Dropout-4, linear layer Linear-5, ReLU layer ReLU-6, Dropout layer Dropout-7, linear layer Linear-8, ReLU layer ReLU-9, Dropout layer Dropout-10, linear layer Linear-11; S14. For each moment t in the multivariate time series data of the training set, and Respectively with x t and X t Calculate the bi-norm error, construct the joint loss function, and optimize the backbone network by gradient descent. The expression is as follows: Loss total =(1-α)Loss recon +αLoss pre Among them, Loss pre Represents the prediction loss function, ||||2 represents the vector two norm, Loss recon Represents the reconstruction loss function, Liss total is the joint loss function, and α is the balancing factor.

3. The unsupervised domain adaptive multivariate time series data anomaly detection method according to claim 1, characterized in that: Step S22 in the step S2 further includes the following process: S221, the prediction error ε for each moment t in the training set multivariate time series data t and reconstruction error r t , take the prediction error window data E of length w before time t t and reconstruction error window data R t , the expression formula is as follows: E t =[e t-w ,he t-w+1 ,…,eh t-1 ] R t ＝[r t-w ,r t-w+1 ,…,r t-1 ] S222, ε t and r t Perform domain adaptation window normalization operation to obtain the prediction error at time t after normalization and reconstruction error Then and The error vector e at time t is obtained by splicing t , the expression formula is as follows: Among them, μ(E t ) represents the prediction error window data E t The mean value, μ(R t ) represents the reconstruction error window data R t The mean value, σ(E t ) represents the prediction error window data E t The variance, σ(R t ) represents the reconstruction error window data R t The variance of Represents splicing along the feature dimension; Step S24 also includes the following process: S241. Calculate e for each t moment in the training set multivariate time series data. t The p-valuep of the error for each feature dimension t,j , get the p-value set p at time t t ={p t,1 ,p t,2 ,...p t,j ,...p t,k }, and its calculation formula is as follows: Among them, p t,j Represents the error vector e at time t t The p-value of the error of the j-th dimension feature, D train represents the background error set, e i represents the i-th error vector in the background error set, e i,j Represents the i-th error vector e in the background error set i The error of the j-th dimension feature, e t,j Represents the error vector e at time t t The error of the j-th dimension feature, I() represents the counting function, which is 1 if the conditions in the brackets are met, otherwise it is 0; S242, filter p t The significance is greater than the significance threshold α max The p-value is obtained At the same time Sort to get the sorted p-value set at time t The expression formula is as follows: in Represents the p-value set after filtering at time t, Sort() represents the sorting function, which sorts the data from small to large. Represents the p-value set of n sorted at time t; S243. By performing a non-parametric statistical scan on to obtain the set of significant anomaly p-values at time t where n * represents the number of p-values in the set of significant anomaly p-values, and n * << n, map the p-values in to the error vector e at time t t to obtain the set of significant anomaly errors S at time t corresponding to the errors in the corresponding feature dimensions t , and finally calculate the anomaly score at time t through S t The calculation formula is as follows: The calculation formula is as follows:

4. The unsupervised domain adaptive multivariate time series data anomaly detection method according to claim 1, characterized in that: The process of step S3 is as follows: S31, in the online detection stage, the multivariate time series data to be tested collected by the online sensor is collected, denoised and normalized to obtain the processed multivariate time series data to be tested x′, and then x′ is processed into the multivariate time series window data to be tested X′, which is expressed as: x′={x1′,x2′,…x t ′,…,x N ′},x t ′∈R k X′={X1′,X2′,…X t ′,…,X N ′} X t ′=[x t-T ′,x t-T+1 ′,…,x t-1 ′]∈R k×T Among them, x t ′ represents the multivariate time series data to be measured at time t, X t ' represents the multivariate time series window data to be tested at time t, including the multivariate time series data to be tested from time tT to time t-1, and N represents the total number of time points of the multivariate time series data to be tested; S32, at each time t in the multivariate time series data to be tested, the X in the multivariate time series window data X′ to be tested in S31 is converted to t ′ and X t+1 ' Input the trained backbone network to obtain the prediction result at time t and the reconstructed multivariate time series window data at time t+1 Pick Reconstruction results at time t Will and and the multivariate time series data x at time t t 'Calculate the absolute value error to get the prediction error ε at time t t ′ and reconstruction error r t '; S33, the prediction error ε at each time t in the multivariate time series data to be tested t ′ and reconstruction error r t ′ performs domain adaptation window normalization operation to obtain the error vector e at time t t '; S34, the error vector e at each moment t in the multivariate time series data to be measured t ′ Perform a significant abnormal error set scanning operation to obtain the significant abnormal error set S at time t t ′, and calculate the anomaly score at time t S35. Anomaly score at each time t in the multivariate time series data to be tested and the optimal threshold th obtained in S2 * For comparison, if Greater than th * , then the detection result y at time t t If it is 1, it means abnormal, otherwise the detection result y at time t t 0 means normal, and finally output the detection result y at time t t , anomaly score And the set of significant abnormal errors S t ′.

Images