Time sequence anomaly detection method based on similarity auto-encoder

Abstract

The invention relates to a time sequence anomaly detection method based on a similarity auto-encoder, and the method comprises the steps: collecting multi-measurement-point monitoring data of a system in a normal operation state, carrying out the preprocessing, and constructing a data set; constructing a parameterized auto-encoder of an encoder-decoder architecture; inputting the data set into an auto-encoder for similarity auto-encoding; designing a composite loss function, and optimizing model parameters by adopting a gradient descent algorithm; and calculating the mean square error of the training sample through the composite loss function in the step S4. According to the time sequence anomaly detection method based on the similarity auto-encoder, the similarity modeling process is deeply integrated to the auto-encoding architecture, induction bias based on the similarity principle is introduced for the representation learning process, and the detection precision of the model is improved. And data self-encoding and similarity modeling are organically fused, so that abnormal data are promoted to generate remarkably distinguishable error offset in a reconstruction space, and generalization in a time sequence anomaly detection task is improved.

Description

Technical Field

[0001] The present invention belongs to the field of data processing technology, and in particular relates to a time series anomaly detection method based on a similarity autoencoder. Background Art

[0002] Data-driven anomaly detection technology holds significant application value in modern industrial systems, particularly hydropower plants. Its core goal is to provide early warning of equipment anomalies through industrial monitoring data analysis, thereby preventing major safety incidents and economic losses. Industrial monitoring data is continuously collected through a multi-point sensor network deployed at key equipment locations, including vibration, pressure, temperature, oil level, and flow. Distributed monitoring systems then process these raw signals in a time-series fashion. Industrial systems operate in a long-term, normal state, and anomaly data within these monitoring data is highly sparse.

[0003] Among deep learning methods, time series anomaly detection based on autoencoders uses an encoder-decoder autoencoder architecture to establish a dual mapping for input data, from the input space to the representation space and then from the representation space to the output space. This allows for data reconstruction and prediction, and uses the reconstruction error as the basis for anomaly scoring. However, such methods have inherent flaws: the training objective of the autoencoder is positioned on the surrogate task of data reconstruction, rather than directly targeting the anomaly detection task itself. Model optimization focuses on data reconstruction accuracy rather than anomaly separability, which can easily lead to overfitting of non-critical features in the data, resulting in limited generalization of the resulting representation. During the testing phase, the reconstruction error distributions of normal and anomaly data overlap significantly, seriously affecting the model's anomaly detection accuracy. Summary of the Invention

[0004] The technical problem of the present invention is: the present invention introduces the inductive bias based on the similarity principle into the autoencoder architecture through the similarity autoencoder, organically integrates data autoencoding and similarity modeling, prompts abnormal data to produce significantly distinguishable error offsets in the reconstruction space, improves the generalization in the time series anomaly detection task, and trains a high-performance time series anomaly detection model.

[0005] The purpose of the present invention is to solve the above problems. The present invention proposes a time series anomaly detection method based on similarity autoencoder, comprising the following steps: S1, collect monitoring data from multiple measuring points of the system under normal operating conditions, and construct a data set after preprocessing; S2, constructing a parameterized autoencoder with an encoder-decoder architecture; S3. Input the data set into the autoencoder for similarity autoencoding; S4. Design a composite loss function and use the gradient descent algorithm to optimize the model parameters; Furthermore, in step S1, the preprocessing includes normalizing the monitoring data of multiple measuring points, performing sliding window segmentation according to a preset window length, and constructing a training data set from the segmented training samples.

[0006] Preferably, the training data set also includes selecting some typical samples to form a support set, inputting the training set into the encoder to obtain the inherent latent representation of the training data set, and using the K-means clustering method to cluster the training samples corresponding to the cluster centers to construct the support set.

[0007] Furthermore, in step S2, the autoencoder includes an encoder and a decoder composed of a deep convolutional neural network, including a one-dimensional convolution layer, a batch normalization layer, a nonlinear activation layer, a downsampling layer and an upsampling layer connected in series.

[0008] Preferably, in step S2, the encoder maps the data from the input space to the representation space; and the decoder maps the data from the representation space back to the output space.

[0009] Furthermore, in step S3, similarity self-encoding includes the following sub-steps: 1) The training samples and support set are synchronously input into the encoder to obtain their respective inherent representations; 2) Based on the similarity modeling process between the training sample and the inherent representation of the support set, the predicted representation of the training sample is obtained; 3) The inherent representation and predicted representation of the training sample are synchronously input into the encoder to obtain the reconstructed value and predicted value of the training sample.

[0010] Preferably, the inherent representation includes 4 inherent skip connection representations and an inherent latent representation; the inherent skip connection representation is used to retain data detail information to improve the autoencoding performance; the predicted representation of the training sample includes 4 predicted skip connection representations and a predicted latent representation.

[0011] Preferably, the composite loss function in step S4 consists of three parts: prediction loss, reconstruction loss and alignment loss; the prediction loss is calculated as the mean square error between the predicted value and the true value of the training sample; the reconstruction loss is calculated as the mean square error between the reconstructed value and the true value of the training sample; the alignment loss is calculated as the mean square error between the inherent latent representation of the training sample and the predicted latent representation.

[0012] Another aspect of the present invention provides a method for calculating anomaly scores and thresholds based on a similarity autoencoder. First, the above method is used to detect multi-point monitoring data, and then the following steps are performed: The training set and support set are simultaneously input into the autoencoder for similarity autoencoding process. The predicted value of each training sample in the training set is obtained, and the mean square error between the true value and the predicted value is calculated. The mean square error is then used as the anomaly score, and the threshold exceedance method is used to calculate the anomaly score threshold. Furthermore, the anomaly score calculation process needs to be performed for each measurement point, that is, the mean square error between the true value and the predicted value of each measurement point is calculated separately as the anomaly score of the measurement point.

[0013] Furthermore, the calculation process of the anomaly score threshold also needs to be implemented independently for each measurement point.

[0014] Preferably, the threshold crossing method comprises the following sub-steps: 1) Set the extreme threshold for the training set anomaly score; 2) Model the Pareto distribution for the training set anomaly scores that exceed the extreme value threshold and determine the distribution parameters through maximum likelihood estimation; 3) Determine the anomaly score threshold by combining the extreme value threshold and Pareto distribution parameters.

[0015] A method for anomaly detection and adaptive updating of anomaly score thresholds, in which test samples and support sets undergo a similarity self-encoding process and anomaly score calculation process, including: comparing the test sample anomaly score with the anomaly score threshold and determining the abnormal state; if an anomaly occurs and the test sample anomaly score exceeds the extreme value threshold, it is included in an anomaly score set initialized to the training set anomaly score and continuously updated; then the threshold is updated according to the anomaly score threshold calculation process in the previous step for subsequent monitoring of data streams.

[0016] Preferably, the length of the first-in-first-out queue is consistent with the window length described in the first aspect.

[0017] Preferably, the test samples need to be normalized, and the processing parameters thereof completely match the normalization standards of the training set in the first aspect.

[0018] Furthermore, the three steps of anomaly score calculation, anomaly score comparison, and anomaly score threshold update need to be performed independently for each measurement point.

[0019] Compared with the prior art, the present invention has the following beneficial effects: 1) This paper proposes a time series anomaly detection method based on similarity autoencoder. By deeply integrating the similarity modeling process into the autoencoder architecture, an inductive bias based on the similarity principle is introduced into the representation learning process, thereby improving the detection accuracy of the model.

[0020] 2) The present invention proposes a time series anomaly detection method based on similarity autoencoder, which organically integrates data autoencoding and similarity modeling, prompting abnormal data to produce significantly distinguishable error offsets in the reconstruction space, thereby improving the generalization of time series anomaly detection tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The present invention will be further described below with reference to the accompanying drawings and examples.

[0022] Figure 1 Schematic diagram of the structure of a time series anomaly detection model based on a similarity autoencoder according to an embodiment of the present invention; Figure 2 Schematic diagram of the structure of the encoder and decoder model according to an embodiment of the present invention; Figure 3 This is a flowchart of a time series anomaly detection model training based on a similarity autoencoder according to an embodiment of the present invention; Figure 4 This is a flowchart of time series anomaly detection scoring and threshold calculation according to an embodiment of the present invention; Figure 5 Flowchart of a timing anomaly detection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The following will further describe the technical solutions implemented in the present invention in detail with reference to the accompanying drawings. It should be understood that the embodiments described are preferred embodiments of the present invention rather than all embodiments, and the protection scope of this description is not limited thereto.

[0024] like Figure 1 As shown in FIG, a time series anomaly detection method based on similarity autoencoder includes the following steps: Collect monitoring data from multiple measuring points when the system is in normal operation, and construct a data set after preprocessing.

[0025] S101: Record the acquired monitoring data as , where N and L represent the number of measurement points and time step, respectively. Represents the real number domain, and the monitoring data is represented as a real number matrix containing N measurement points and L time steps .

[0026] Preprocessing, including monitoring data The data of each measuring point in the data set are normalized separately, and a window with a length of T is used as the sliding window step. The number of training samples obtained is L-T+1. Each training sample can be represented as a real number matrix containing N measurement points and T time steps. . Use the above training samples to build a training set , where the superscript i represents the sample index, which is a positive integer between 1 and L-T+1.

[0027] S102: The training data set also includes selecting some typical samples to form a support set, inputting the training set into the encoder to obtain the inherent latent representation of the training data set, and using the K-means clustering method to cluster the training samples corresponding to the cluster centers to construct the support set.

[0028] Build parameterized autoencoders with encoder-decoder architectures; The autoencoder includes an encoder and a decoder composed of a deep convolutional neural network, which includes a one-dimensional convolution layer, a batch normalization layer, a nonlinear activation layer, a downsampling layer and an upsampling layer connected in series.

[0029] The encoder maps data from the input space to the representation space; the decoder maps the representation space back to the output space.

[0030] like Figure 2 The autoencoder shown includes: S001: The structure of the autoencoder, 9*1 Conv1D, 16, BN, ReLU represents a one-dimensional convolutional neural network with a convolution kernel length of 9 and the number of output channels of 16, with a batch normalization layer and ReLU activation function, and Maxpool represents the maximum pooling operation.

[0031] S002:Maxpool, \2” represents the maximum pooling layer with a stride of 2, which aims to downsample the time dimension of the input features; S003: The terminal layer of the encoder does not have a batch normalization layer and a ReLU activation function, and is used to output latent representation features; S004: The encoder synchronously outputs the intermediate layer features through 4 skip connections to provide more feature details for the decoding process; Represent the sample as , the encoder is represented as , the latent representation is expressed as , the four skip connections are represented as four skip connections respectively 、 、 and In order to adapt to the prediction characteristics of the similarity autoencoder, it is described by two words, namely, inherent skip connection and inherent latent representation.

[0032] Input the dataset into the autoencoder for similarity autoencoding; S103: Randomly sample training samples from the training set for small batch training, sampling one training sample each time, expressed as .

[0033] S104: Synchronously input the training samples and support set into the encoder to obtain their respective inherent representations: Synchronously input the support set and training samples into the encoder for encoding to obtain support set jump connections 1 to 4, support set inherent latent representations, training sample jump connections 1 to 4 and training sample inherent latent representations.

[0034] S105: Based on the similarity modeling process between the training sample and the inherent representation of the support set, the predicted representation of the training sample is obtained. The calculation formula of the similarity modeling process is: ; ; ; Where, represents the covariance matrix, represents the inverse matrix of the covariance matrix, represents a K-dimensional vector, represents a K-dimensional matrix, represents a K-dimensional vector, which represents the weight component of the inherent latent representation of the training sample compared to the inherent latent representation of the support set. represents the Cauchy kernel function.

[0035] The Cauchy kernel function is less sensitive to hyperparameters, and its heavy-tailed nature can effectively suppress the overfitting of the autoencoder to non-critical information in the data. The calculation formula is: ; Where, represents the hyperparameter, represents the Frobenius matrix norm, a and b represent the parameters of the Cauchy kernel function.

[0036] The weight vector is sparse and normalized, and the expression is: , ; Where, Represents an assignment operation, express No. elements, express All elements in Percentile value, represents the 1-norm, and k represents the counting unit.

[0037] S106: Based on the weight vector output by the similarity modeling process, weighted combinations are performed on the support set skip links 1 to 4 and the support set inherent latent representation to obtain the predicted skip links 1 to 4 and the predicted latent representation of the training sample. The calculation formula for the predicted skip links 1 to 4 and the predicted latent representation of the training sample is: ; ; Where, represents the training sample, 、 、 、 and Represent the predicted skip connections 1 to 4 and the predicted latent representation respectively.

[0038] The inherent representation and predicted representation of the training sample are synchronously input into the encoder to obtain the reconstructed value and predicted value of the training sample.

[0039] S107: The structure of the decoder is as follows Figure 1 As shown, the predicted jump connections 1 to 4 and the predicted implicit representation are synchronously input into the decoder for decoding. The decoding calculation formula is: ; Where, Indicates the result of decoding.

[0040] The inherent representation includes 4 inherent skip connection representations and an inherent latent representation; the inherent skip connection representation is used to retain data detail information to improve the autoencoding performance; the predicted representation of the training sample includes 4 predicted skip connection representations and a predicted latent representation.

[0041] Design a composite loss function and use the gradient descent algorithm to optimize the model parameters.

[0042] S108: Based on the difference between the input value of the training sample and its predicted value, a prediction loss function is constructed, which is expressed as: ; The prediction loss function is used to describe the similarity between the training data representation and the typical normal representation, and is the main part of the similarity autoencoder composite loss function.

[0043] The composite loss function consists of three parts: prediction loss, reconstruction loss, and alignment loss. The prediction loss is calculated as the mean squared error between the predicted value and the true value of the training sample. The reconstruction loss is calculated as the mean squared error between the reconstructed value and the true value of the training sample. The alignment loss is calculated as the mean squared error between the inherent latent representation of the training sample and the predicted latent representation.

[0044] S109: Input the inherent skip connections 1 to 4 and the inherent latent representation of the training sample directly into the decoder for decoding to obtain the reconstructed value of the training sample, which is expressed as: ; Where, Represents the result of reconstructing a value.

[0045] Based on the difference between the input value of the training sample and its reconstructed value, a reconstruction loss function is constructed. The calculation formula of the loss function is: ; Where, Represents the calculation result of the loss function.

[0046] S110: Based on the difference between the inherent latent representation of the training sample and its predicted latent representation, an alignment loss function is constructed. The alignment loss function is used to ensure the consistency between the inherent latent representation of the training sample and the predicted latent representation. The calculation formula of the alignment loss function is: ; Where, Represents the calculation result of the alignment loss function.

[0047] The formula for the composite loss function of the autoencoder is: ; Where, and denote the weight hyperparameters of the reconstruction loss and alignment loss respectively.

[0048] like Figure 3 As shown, the optimization of the model includes: constructing a training set according to S101, and constructing a support set according to S102, and then executing S103 to S107 in sequence to perform the similarity autoencoder process, constructing a loss function according to S108 to S110 and using the gradient descent method to optimize the parameters of the autoencoder, and repeatedly iterating S102 to S110 until the maximum number of iterations is reached.

[0049] Example 2: Another aspect of the present invention provides a method for calculating anomaly scores and thresholds based on a similarity autoencoder. First, the above method is used to detect multi-point monitoring data, and then the following steps are performed: S201: Perform similarity autoencoding process on each sample in the training set according to S104 to S107 to obtain the predicted value of the training set.

[0050] S202: Calculate an anomaly score independently for each measurement point in the training set.

[0051] For training samples Measuring point , calculate the mean square error between the true value and the predicted value as its anomaly score, the expression is: ; Where, Represents the measurement points of the training samples.

[0052] S203: Based on the anomaly scores of each measurement point in the training set, an anomaly score threshold is calculated for each measurement point using a threshold exceeding method.

[0053] The threshold exceeding method assumes that the probability distribution of extreme events is independent of the distribution of the events themselves and uses the Pareto distribution to describe extreme events.

[0054] Test points for the training set The abnormal score of , takes its 98% quantile as the extreme value threshold of the measurement point, recorded as , the anomaly score that exceeds the extreme threshold called extreme points.

[0055] The abnormal scores of each point in the training set cannot completely cover all extreme points. Among all extreme points, only a part of them are considered abnormal points. Assume that the proportion of abnormal points is , this ratio is also called the risk level coefficient. Since the training set is all normal samples, It is usually set to a small value, such as 1e-4. Assume that the total amount of anomaly scores is ,in The number of , abnormal threshold The analytical expression is: ; Where, Indicates the proportion of abnormal points. Since the training set consists of normal samples, It is usually set to a small value, such as 1e-4; represents the total amount of anomaly scores, The number of , the calculation formula of the abnormal threshold is: ; Where, represents the abnormal threshold, and They represent the unknown parameters of the Pareto distribution, which need to be estimated based on the maximum likelihood function.

[0056] and The calculation formula of the likelihood function is: ; ; Where, for The cardinality of the set is .

[0057] Find the partial derivative of the above formula and set it to 0, and we can easily get and The optimal value of and The following equations must be satisfied, and the GrimShaw technique must be used to find the numerically approximate optimal solution. The calculation formula is: ;

[0058] Where, and Substitute the above In the analytical expression of The anomaly scoring threshold.

[0059] It is necessary to execute S202 to S203 for each measurement point in the training set to obtain the abnormality score threshold of each measurement point. .

[0060] Example 3 like Figure 5 As shown in FIG, a method for adaptively updating anomaly detection and anomaly scoring threshold is provided. The test sample and the support set undergo a similarity self-encoding process and anomaly scoring calculation process, including: S301: Construct a queue of length T to store the multi-point sensing data at the current moment and the data at the previous T-1 historical moments. The sensing data at each moment is an N-dimensional vector.

[0061] The queue follows the first-in-first-out principle, that is, whenever new data is pushed into the queue, the oldest data at the head of the queue is automatically removed. Arrange all the data in the queue in chronological order and build The matrix of dimension is used as the test sample, denoted as Then, referring to the step in S101, normalization is performed on the test sample, wherein the maximum / minimum value of each measurement point data used in the normalization process should be consistent with the normalization benchmark in S101.

[0062] S302: Perform the similarity self-encoding process on the test sample according to S104 to S107 to obtain the test sample prediction value, and then calculate the abnormality score of each test point of the test sample according to S202, which is recorded as .

[0063] S303: Calculate the abnormality score of each measurement point in the training set in the same way as S302.

[0064] Construct an anomaly score set for each measurement point and The set of anomaly scores is denoted as Initialized as training set points The anomaly score of the test sample is dynamically incorporated into the anomaly score of the test sample.

[0065] S304: Calculate the abnormality score threshold of each measurement point with reference to S203. The calculation formula is: ; , ; ; ; Where, Indicates the extreme value threshold in the second embodiment.

[0066] S305: Compare the abnormality score of each test point of the test sample with the corresponding abnormality score threshold to determine whether the test point is abnormal. ,if , then the measuring point is detected as abnormal, otherwise it is normal.

[0067] S306: The abnormality score of each measurement point of the test sample is included in the abnormality score set of the corresponding measurement point. The expression is: ; Where, S307: Repeat S301 to S306 and obtain new sensing data each time.

[0068] The test sample anomaly score is compared with the anomaly score threshold to determine the abnormal state. If an anomaly occurs and the test sample anomaly score exceeds the extreme value threshold, it is included in an anomaly score set that is initialized to the training set anomaly score and continuously updated. The threshold is then updated according to the anomaly score threshold calculation process in the previous step for subsequent monitoring of data streams.

Claims

1. A time series anomaly detection method based on similarity autoencoder, characterized in that: The following steps are involved: S1, collect monitoring data from multiple measuring points of the system under normal operating conditions, and construct a data set after preprocessing; S2, constructing a parameterized autoencoder with an encoder-decoder architecture; S3. Input the data set into the autoencoder for similarity autoencoding; S4. Design a composite loss function and use the gradient descent algorithm to optimize the model parameters.

2. The method for detecting anomalies in a time series based on a similarity autoencoder according to claim 1, wherein: In step S1, the preprocessing includes normalizing the monitoring data of multiple measuring points, performing sliding window segmentation according to a preset window length, and constructing a training data set from the segmented training samples.

3. The method for detecting anomalies in a time series based on a similarity autoencoder according to claim 2, wherein: The training data set also includes selecting some typical samples to form a support set, inputting the training set into the encoder to obtain the inherent latent representation of the training data set, and using the K-means clustering method to cluster the training samples corresponding to the cluster centers to construct the support set.

4. The method for detecting anomalies in a time series based on a similarity autoencoder according to claim 1, wherein: In step S2, the autoencoder includes an encoder and a decoder composed of a deep convolutional neural network, including a one-dimensional convolution layer, a batch normalization layer, a nonlinear activation layer, a downsampling layer and an upsampling layer connected in series.

5. The method for detecting anomalies in a time series based on a similarity autoencoder according to claim 3, wherein: In step S2, the encoder maps data from the input space to the representation space; the decoder maps data from the representation space back to the output space.

6. The method for detecting anomalies in a time series based on a similarity autoencoder according to claim 1, wherein: In step S3, the similarity self-encoding includes the following sub-steps: 1) The training samples and support set are synchronously input into the encoder to obtain their respective inherent representations; 2) Based on the similarity modeling process between the training sample and the inherent representation of the support set, the predicted representation of the training sample is obtained; 3) The inherent representation and predicted representation of the training sample are synchronously input into the encoder to obtain the reconstructed value and predicted value of the training sample.

7. The method for detecting anomalies in a time series based on a similarity autoencoder according to claim 6, wherein: The inherent representation includes 4 inherent skip connection representations and an inherent latent representation; the inherent skip connection representation is used to retain data detail information to improve autoencoding performance; the predicted representation of the training sample includes 4 predicted skip connection representations and a predicted latent representation.

8. The method for detecting anomalies in a time series based on a similarity autoencoder according to claim 1, wherein: In step S4, the composite loss function consists of three parts: prediction loss, reconstruction loss and alignment loss; the prediction loss is calculated as the mean square error between the predicted value and the true value of the training sample; the reconstruction loss is calculated as the mean square error between the reconstructed value and the true value of the training sample; the alignment loss is calculated as the mean square error between the inherent latent representation of the training sample and the predicted latent representation.

9. A method for calculating anomaly scores and thresholds based on similarity autoencoders, characterized in that: First, the method for detecting time series anomalies according to any one of claims 1 to 8 is used to detect monitoring data at multiple measuring points, and then the following steps are performed: The training set and support set are simultaneously input into the autoencoder for similarity autoencoding process. The predicted value of each training sample in the training set is obtained, and the mean square error between the true value and the predicted value is calculated. The mean square error is then used as the anomaly score, and the threshold exceedance method is used to calculate the anomaly score threshold.

10. A method for adaptively updating anomaly detection and anomaly scoring threshold, characterized in that: The test sample and the support set go through a similarity self-encoding process and anomaly score calculation process, including: comparing the test sample anomaly score with the anomaly score threshold and determining the abnormal state. If an anomaly occurs and the test sample anomaly score exceeds the extreme value threshold, it will be included in an anomaly score set initialized to the training set anomaly score and continuously updated. Then, the threshold is updated according to the anomaly score threshold calculation process in the previous step for subsequent monitoring of data streams.

Images