Unsupervised anomaly sensing system and anomaly detection method of multivariate time series

Abstract

The invention provides an unsupervised anomaly sensing system for a multivariate time sequence, which is used for detecting whether the multivariate time sequence is abnormal or not, and comprises a data acquisition module used for acquiring an original multivariate time sequence; the anomaly perception model is used for coding and decoding the original multivariate time sequence to obtain a reconstructed multivariate time sequence; and the anomaly evaluation module is used for calculating an anomaly degree quantitative score of the original multivariate time sequence according to a preset evaluation mode based on the original multivariate time sequence and the reconstructed multivariate time sequence, and evaluating whether the original multivariate time sequence is abnormal or not according to a preset evaluation rule based on the anomaly degree quantitative score. According to the invention, through the lightweight architecture design, the parameter quantity and the calculation complexity are greatly reduced, and the method is suitable for a resource-limited environment; and the abnormal data mode is effectively identified through the feature representation of the normal multivariate time sequence stored in the memory slot, so that the accuracy and robustness of anomaly detection are improved.

Description

Technical Field

[0001] The present invention relates to the field of data anomaly perception, and more specifically, to an anomaly detection technology in the field of data anomaly perception, and more specifically, to an unsupervised anomaly perception system and an anomaly detection method for multivariate time series. Background Art

[0002] In the field of unsupervised anomaly detection in multivariate time series, with the increase in network scale and model complexity, existing technologies have made significant progress in processing complex multivariate time series. Existing technologies usually rely on deep learning models to achieve anomaly detection in multivariate time series, such as long short-term memory networks (LSTM), variational autoencoders (VAE), and Transformer networks based on attention mechanisms. These models can effectively detect anomalies or abnormal events in multivariate time series by capturing complex temporal correlations in multivariate time series. However, as the scale of the model increases, the computing resource requirements of deep learning models also increase, making these models difficult to apply in resource-constrained environments (such as edge computing, mobile devices, and embedded systems).

[0003] Specifically, LSTM models excel in processing multivariate time series, especially for long-term dependency problems. However, such models usually contain a large number of parameters, have high training and inference overhead, and are difficult to avoid overfitting when processing high-dimensional multivariate time series. In addition, the anomaly detection capability of LSTM models depends on reconstruction errors and may not fully capture global patterns.

[0004] VAE models and GAN models, these generative models can generate realistic samples of normal multivariate time series by modeling the latent space of normal multivariate time series, and detect anomalies through reconstruction errors. However, these models have high computational overhead and strong assumptions about multivariate time series. For example, VAE models usually assume that the data distribution is Gaussian, which is not applicable to non-Gaussian data. Moreover, in practical applications, the inference speed of these generative models is slow and it is difficult to meet the requirements of real-time detection.

[0005] The Transformer model, which effectively captures long-term dependencies through the self-attention mechanism, has been shown to have good results in anomaly detection. Especially on multivariate time series data, the Transformer model can automatically pay attention to the dependencies between different time steps and variables. However, the computational complexity of the Transformer model increases quadratically with the increase in the time length of the multivariate time series, so the computing resource consumption is extremely high in long time series or high-frequency data scenarios. This makes the Transformer model difficult to apply in scenarios with limited resources.

[0006] Traditional machine learning methods: In addition to deep learning models, traditional methods such as Isolation Forest, Local Outlier Factor (LOF) and Support Vector Machine (SVM) are also used in anomaly detection. These methods rely on the local characteristics and statistical laws of the data and are suitable for low-dimensional, feature-clear multivariate time series. However, as the dimension and complexity of the data increase, the performance of traditional methods gradually decreases and cannot adapt to high-dimensional, complex multivariate time series.

[0007] In summary, although the existing technology has made significant progress in processing complex multivariate time series, the following problems still exist: First, the model complexity is high: the existing technology relies on large-scale networks and a large number of parameters and cannot be applied to environments with limited computing resources; second, the scene adaptability is poor: most of the existing technologies are designed for specific scenarios and lack adaptability to cross-domain tasks; third, the anomaly judgment criteria are limited: the existing models usually rely on one-sided anomaly judgment criteria (such as reconstruction error only), which cannot capture distribution drift such as progressive anomalies, affecting the anomaly perception performance.

[0008] It should be noted that this background technology is only used to introduce the relevant information of the present invention to help understand the technical solution of the present invention, but it does not mean that the relevant information is necessarily the prior art. If there is no evidence that the relevant information has been disclosed before the application date of the present invention, the relevant information shall not be regarded as the prior art. Summary of the invention

[0009] Therefore, the purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide an unsupervised anomaly perception system for multivariate time series and an anomaly detection method for multivariate time series.

[0010] The objectives of the present invention are achieved through the following technical solutions.

[0011] According to a first aspect of the present invention, an unsupervised anomaly perception system for a multivariate time series is provided, which is used to detect whether a multivariate time series has an anomaly. The system comprises: a data acquisition module, which is used to acquire an original multivariate time series, wherein the multivariate time series contains data of multiple dimensions at the same time; an anomaly perception model, which is used to encode and decode the original multivariate time series to obtain a reconstructed multivariate time series; wherein the anomaly perception model is obtained by performing multiple rounds of iterative training using a multivariate time series as input and a reconstructed multivariate time series as output using a preset objective function, and the anomaly perception model comprises an encoder, a sparse contrast memory module and a decoder, wherein: the encoder is used to perform feature extraction on the multivariate time series to obtain a feature vector representation of the multivariate time series; the sparse contrast memory module comprises multiple Memory slots, each memory slot stores a feature representation of a normal multivariate time series, the sparse comparison module is configured to: calculate the similarity between the feature vector representation of the multivariate time series and the feature representation stored in each memory slot, and obtain a new feature vector representation of the multivariate time series according to a preset calculation method based on the feature representation stored in each memory slot and each calculated similarity; the decoder is used to decode the new feature vector representation of the multivariate time series to obtain a reconstructed multivariate time series; an abnormality assessment module is used to calculate the abnormality degree quantification score of the original multivariate time series according to a preset assessment method based on the original multivariate time series and the reconstructed multivariate time series, and evaluate whether the original multivariate time series has an abnormality based on the abnormality degree quantification score according to a preset assessment rule.

[0012] In some embodiments of the present invention, the sparse contrast memory module is configured to calculate the similarity between the feature vector representation of the multivariate time series and the feature representation stored in each memory slot as follows:

[0013]

[0014] Among them, s i represents the similarity between the feature vector representation of the multivariate time series and the feature representation stored in the i-th memory slot, z represents the feature vector representation of the multivariate time series, sim(·) represents the similarity metric function, τ represents the temperature coefficient, n represents the number of memory slots, and m i represents the i-th memory slot, m j represents the jth memory slot.

[0015] In some embodiments of the present invention, the preset calculation method is:

[0016]

[0017] Among them, Z ′A new feature vector representation representing the multivariate time series.

[0018] In some embodiments of the present invention, the preset objective function is:

[0019]

[0020] in,

[0021]

[0022] in,

[0023]

[0024] in,

[0025]

[0026] in,

[0027]

[0028] Among them, L totally represents the objective function, represents the dynamic weighted reconstruction loss, L sparse represents the sparse similarity loss, L contrast represents the memory contrast loss, λ s represents the first coefficient, λ c represents the second coefficient, T represents the number of multivariate time series, F represents the number of dimensions contained in the multivariate time series, and w t,f Represents a multivariate time series at time t The weight of the fth dimension in , ε t,f Represents a multivariate time series at time t The reconstruction error of the data in the fth dimension, ε t′,f Indicates t ′ Multivariate time series of moments The reconstruction error of the data in the fth dimension in , Represents a multivariate time series at time t The data of the fth dimension in Represents the reconstructed multivariate time series at time t The data of the fth dimension, ∈ represents the preset value, z t The eigenvector representation of the multivariate time series at time t, represents all memory slots with the feature vector z t The closest positive memory slot, represents all memory slots with the feature vector z t The next closest negative memory slot, α represents the boundary, [·]+ represents the symbolic function, m k represents the kth memory slot.

[0029] In some embodiments of the present invention, the preset evaluation method is:

[0030]

[0031] in,

[0032]

[0033] in,

[0034]

[0035] in, Represents a multivariate time series at time t The quantitative score of abnormality, ω t represents the first index parameter, ψ t represents the second index parameter, Represents a multivariate time series at time t The reconstruction error of the data in the fth dimension in , Indicates t ′ The reconstruction error of the data in the fth dimension in the multivariate time series at time, Represents a multivariate time series at time t The Euclidean distance to the nearest memory slot among all memory slots, f enc (·) represents the encoder, θ enc represents the encoder weight, Represents the multivariate time series at time t in all memory slots The closest memory slot.

[0036] In some embodiments of the present invention, the preset evaluation rule is: if the quantitative score of the abnormality degree of the original multivariate time series is less than or equal to a preset threshold, the original multivariate time series is normal data, otherwise, it is abnormal data.

[0037] In some embodiments of the present invention, the encoder and the decoder both use a single fully connected layer.

[0038] According to a second aspect of the present invention, a method for detecting anomalies in a multivariate time series is provided, the method comprising: step S1, obtaining a multivariate time series to be processed; step S2, performing anomaly detection on the multivariate time series to be processed using the system as described in the first aspect of the present invention.

[0039] Compared with the prior art, the advantages of the present invention are: (1) the unsupervised anomaly perception system greatly reduces the number of parameters and computational complexity through lightweight architecture design, and is suitable for resource-constrained environments; (2) a memory slot is introduced into the anomaly perception model, so that the anomaly perception model can effectively identify complex abnormal data patterns based on the feature representation of normal multivariate time series stored in the memory slot, thereby improving the accuracy and robustness of anomaly detection; (3) a dynamic weighted reconstruction loss is introduced when training the anomaly perception model, so that the anomaly perception model can better handle the dependencies in the time series, and sparse similarity loss and memory contrast loss are also introduced, so that the model can distinguish different normal multivariate time series while paying more attention to global features, which is conducive to improving the accuracy of anomaly detection; (4) the anomaly evaluation module considers the information of reconstruction space, time space and latent space to calculate a comprehensive quantitative score of the degree of anomaly, further improving the comprehensiveness and accuracy of anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:

[0041] Figure 1 A schematic diagram of the composition of an unsupervised anomaly perception system according to an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of the composition of an abnormality perception model according to an embodiment of the present invention;

[0043] Figure 3 Schematic diagram of examples of the training phase and the detection phase of the unsupervised anomaly perception system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0045] As mentioned in the background technology section, although the existing technology has made significant progress in processing complex multivariate time series, the following problems still exist: First, the model complexity is high: the existing technology relies on large-scale networks and a large number of parameters and cannot be applied to environments with limited computing resources; second, the scene adaptability is poor: most of the existing technologies are designed for specific scenarios and lack adaptability to cross-domain tasks; third, the anomaly judgment criteria are limited: the existing models usually rely on one-sided anomaly judgment criteria (such as reconstruction error only), which cannot capture distribution drift such as progressive anomalies, affecting the anomaly perception performance.

[0046] In order to solve the above problems, the inventors have studied and analyzed the existing technologies and found that the existing large-scale models (such as LSTM, Transformer, etc.) rely on a large number of parameters and complex structures to capture the complex temporal correlations in multivariate time series, thereby realizing anomaly detection. Although the existing models can achieve good detection results in resource-rich environments, they cannot be applied in scenarios with limited computing resources. Although the model complexity can be reduced by reducing the number of network layers and neurons, this direct reduction in the model size often leads to a significant decrease in model performance, making it impossible for the model to effectively learn the important features in the multivariate time series, affecting the anomaly detection effect. In addition to the problem of excessively large model parameter scale, the inventors also found that most of the existing deep learning models are designed for specific scenarios and lack adaptability to cross-domain tasks. For example, the VAE model usually assumes that the data distribution is Gaussian and is not suitable for non-Gaussian data. In addition to the problems of excessively large model parameter scale and poor scene adaptability, the inventors also found that in multivariate time series, anomalies may manifest as changes in certain global patterns, rather than just deviations in local features. For example, for a multivariate time series composed of medical data in multiple dimensions, anomalies may not only manifest as anomalies in medical data of a single dimension, but may also be changes in the complex temporal relationship between medical data in multiple dimensions. These global abnormal patterns are often difficult to detect using models that only consider local features (such as traditional isolation forests or local window-based models). In addition, most existing models only focus on reconstruction errors or similarities in local time or feature dimensions when dealing with anomaly detection, which makes it impossible for the model to effectively capture the global correlation features in the multivariate time series. In particular, when normal patterns and abnormal patterns are similar in local features, existing models are prone to missed detections or false detections (some abnormal patterns may be highly similar to normal patterns in the short term, but will show significant differences when observed globally).

[0047] Based on the above analysis, the inventor proposes an anomaly perception system that can be applied to resource-constrained scenarios, which includes a data acquisition module, an anomaly perception model and an anomaly assessment module. Among them, the data acquisition module is used to collect the original multivariate time series. The anomaly perception model is used to encode and decode the original multivariate time series data to obtain the reconstructed multivariate time series, wherein the anomaly perception model reduces the model complexity by reducing the number of network layers and the number of neurons, thereby reducing the computing resources required for model encoding and decoding processing; at the same time, multiple memory slots with extremely small parameter spaces are introduced into the anomaly perception model to store the feature representation of the normal multivariate time series, and the feature representation of the normal multivariate time series stored in each memory slot is similarly calculated with the original multivariate time series to retrieve the normal data pattern in the original multivariate time series, and the multivariate time series is reconstructed based on the similarity calculation results, and then the difference between the reconstructed multivariate time series and the original multivariate time series is analyzed to realize anomaly detection; the introduction of memory slots in the anomaly perception model can not only make up for the performance loss caused by the reduction of the parameters of the anomaly perception model, but also effectively improve the adaptability and robustness of the anomaly perception model in resource-constrained scenarios. The anomaly assessment module is used to calculate the quantitative score of the degree of anomaly of the original multivariate time series based on the original multivariate time series and the reconstructed multivariate time series according to a preset evaluation method, and analyze whether there is an anomaly in the original multivariate time series based on the quantitative score of the degree of anomaly. Taking into account the defects of the prior art, the anomaly assessment module fuses information from multiple different dimensional spaces when calculating the quantitative score of the degree of anomaly, thereby effectively identifying global anomalies and avoiding missed anomalies due to focusing only on local features.

[0048] In summary, if Figure 1As shown, the present invention provides an unsupervised anomaly perception system for multivariate time series, which is used to detect whether there is an anomaly in the multivariate time series. The system includes: a data acquisition module, which is used to acquire the original multivariate time series, wherein the multivariate time series contains data of multiple dimensions at the same time; an anomaly perception model, which is used to encode and decode the original multivariate time series to obtain a reconstructed multivariate time series; wherein the anomaly perception model is obtained by performing multiple rounds of iterative training using the multivariate time series as input and the reconstructed multivariate time series as output using a preset objective function, and the anomaly perception model includes an encoder, a sparse contrast memory module and a decoder, wherein: the encoder is used to extract features from the multivariate time series to obtain a feature vector representation of the multivariate time series; the sparse contrast memory module includes multiple memories The memory slots each store a feature representation of a normal multivariate time series, the sparse comparison module is configured to: calculate the similarity between the feature vector representation of the multivariate time series and the feature representation stored in each memory slot, and obtain a new feature vector representation of the multivariate time series according to a preset calculation method based on the feature representation stored in each memory slot and each calculated similarity; the decoder is used to decode the new feature vector representation of the multivariate time series to obtain a reconstructed multivariate time series; the abnormality assessment module is used to calculate the abnormality degree quantification score of the original multivariate time series according to a preset assessment method based on the original multivariate time series and the reconstructed multivariate time series, and evaluate whether the original multivariate time series has an abnormality based on the abnormality degree quantification score according to a preset assessment rule.

[0049] In order to better understand the present invention, each component of the unsupervised anomaly perception system is described in detail below in conjunction with specific embodiments.

[0050] 1. Data acquisition module

[0051] The data acquisition module is used to collect the original multivariate time series. It should be noted that the data acquisition module can be configured to use a multivariate sensor to acquire and collect the multivariate time series, and transmit the acquired multivariate time series to the abnormality perception model through a readable and writable storage medium or a communicative data transmission medium; it can also be configured to only receive the multivariate time series. It should also be noted that a multivariate time series can be represented as X t =[x 1,t ,x 2,t ,…,x n,t ], where X t represents the multivariate time series at time t, x 1,t Represents the data of the first dimension of the multivariate time series at time t, x 2,t Represents the data of the second dimension of the multivariate time series at time t, x n,trepresents the data of the nth dimension of the multivariate time series at time t. The dimension here refers to the data collected by different sensors. For example, suppose X t is a multivariate time series representing meteorological changes, x 1,t It can be expressed as the temperature data collected by the temperature sensor at time t, x 2,t It can be expressed as the humidity data collected by the humidity sensor at time t, x n,t It can be expressed as the humidity data collected by the wind direction sensor at time t.

[0052] 2. Abnormal Perception Model

[0053] The anomaly perception model is used to encode and decode the original multivariate time series to obtain a reconstructed multivariate time series. The anomaly perception model is obtained by performing multiple rounds of iterative training using a preset objective function with a multivariate time series as input and a reconstructed multivariate time series as output. Figure 2 As shown, the anomaly perception model includes an encoder, a sparse contrast memory module and a decoder. The following describes each component of the anomaly perception model one by one.

[0054] The encoder is used to extract features from multivariate time series to obtain feature vector representations of multivariate time series. The encoder is responsible for compressing the multidimensional multivariate time series into a low-dimensional latent vector representation (feature vector representation). This process not only reduces the data dimension and computational complexity, but also retains the key information in the multivariate time series. The feature extraction process of the encoder can be expressed as: Z = f enc (X), where f enc (·) represents the mapping function of the encoder, X represents the multivariate time series, and Z represents the compressed latent vector representation (feature vector representation). It should be noted that in order to ensure that the abnormal perception model can be applied to resource-constrained scenarios and has efficient encoding and decoding processing capabilities, the encoder adopts a single-layer fully connected layer, and other network layers suitable for resource-constrained scenarios can also be used as encoders, and the present invention does not impose any special restrictions.

[0055] The sparse contrast memory module includes multiple memory slots, each memory slot stores a normal feature representation of a multivariate time series, and the sparse contrast module is configured to: calculate the similarity between the feature vector representation of the multivariate time series and the feature representation stored in each memory slot, and obtain a new feature vector representation of the multivariate time series according to a preset calculation method based on the feature representation stored in each memory slot and each calculated similarity. The sparse contrast memory module combines the ideas of memory network and sparse learning, and can efficiently and accurately retrieve the feature representation of the normal multivariate time series stored in the memory slot that is most similar to the multivariate time series; and the memory slots in the sparse contrast memory module can be trained, that is, the memory slots can learn the normal patterns of different multivariate time series in iterative training. Among them, the multiple memory slots in the sparse contrast memory module can be expressed as: M = [m1, m2,…, m n ], M represents a set of memory slots, m1 represents the first memory slot, m2 represents the second memory slot, and m n Represents the nth memory slot, each memory slot includes data of multiple dimensions.

[0056] It should be noted that the original intention of setting up memory slots in the sparse contrast memory module is to overcome the shortcomings of large-scale models. By introducing multiple memory slots with controllable capacity and differentiable to store the feature representation of normal multivariate time series, the anomaly perception model can accurately retrieve the feature representation of normal multivariate time series stored in the memory slot that is most similar to the multivariate time series based on the memory slot, thereby realizing accurate identification of abnormal data. However, how to choose the appropriate memory capacity (i.e., the number of memory slots) is a key issue. Too few memory slots may not effectively cover the diversity of normal multivariate time series, resulting in a decrease in the detection accuracy of the anomaly perception model; while too many memory slots will increase the computational burden of the anomaly perception model, which is contrary to the original intention of lightweight design. In order to ensure that the anomaly perception model can maintain a high detection accuracy without bringing too much computational burden, the number of memory slots is set to be less than or equal to the number of spatial dimensions represented by the feature vector of the multivariate time series. For example, assuming that the number of spatial dimensions represented by the feature vector of the multivariate time series is 7, the number of memory slots must be less than or equal to 7.

[0057] According to one embodiment of the present invention, the sparse contrast memory module is configured to calculate the similarity between the feature vector representation of the multivariate time series and the feature representation stored in each memory slot in the following manner:

[0058]

[0059] Among them, s irepresents the similarity between the feature vector representation of the multivariate time series and the feature representation stored in the i-th memory slot, z represents the feature vector representation of the multivariate time series, sim(·) represents the similarity metric function, τ represents the temperature coefficient, n represents the number of memory slots, and m i represents the i-th memory slot, m j represents the jth memory slot. The similarity measurement function can be the cosine similarity function sim(z,m i )=z·m i T , where m i T Represents the transposition of the i-th memory slot (the transposition of the feature representation stored in the memory slot), and other similarity functions may also be used, such as inner product similarity, which is not specifically limited in the present invention.

[0060] According to one embodiment of the present invention, the preset calculation method is:

[0061]

[0062] Among them, Z ′ A new feature vector representation representing the multivariate time series.

[0063] According to one embodiment of the present invention, the preset objective function is:

[0064]

[0065] Among them, L totally represents the objective function, represents the dynamic weighted reconstruction loss, L sparse represents the sparse similarity loss, L contrast represents the memory contrast loss, λ s represents the first coefficient, λ c represents the second coefficient. The following is an introduction to each loss.

[0066] Before introducing the dynamic weighted reconstruction loss, let’s first introduce the traditional target loss L rec , which is expressed as follows:

[0067]

[0068] Where T represents the number of multivariate time series, F represents the number of dimensions contained in the multivariate time series, and ε t,f Represents the multivariate time series X at time t t The reconstruction error of the data in the fth dimension, X t,f Represents a multivariate time series X t The data of the fth dimension in Represents the reconstructed multivariate time series at time t The data of the fth dimension in . In the traditional target loss L rec In the paper, only the difference between the original multivariate time series and the reconstructed multivariate time series is focused on, ignoring the temporal correlation and potential characteristic structure in the multivariate time series.

[0069] Compared with the traditional target loss L rec Different from the above, the dynamic weighted reconstruction loss proposed in this invention Considering the error contribution of different dimensions in the multivariate time series at each moment, and assigning different weights to the error contribution of different dimensions in the multivariate time series at each moment, the anomaly perception model can better handle the dependencies in the time series. It is expressed as follows:

[0070]

[0071] Where T represents the number of multivariate time series, F represents the number of dimensions contained in the multivariate time series, and w t,f Represents a multivariate time series at time t The weight of the fth dimension in , ε t,f Represents a multivariate time series at time t The reconstruction error of the data in the fth dimension, ε t′,f Indicates t ′ Multivariate time series of moments The reconstruction error of the data in the fth dimension in , Represents a multivariate time series at time t The data of the fth dimension in Represents the reconstructed multivariate time series at time t The data of the fth dimension in .

[0072] Furthermore, in order to encourage the anomaly perception model to compare and remember the feature representations of different normal multivariate time series, a sparse similarity loss L is introduced into the objective function. sparse , specifically expressed as follows:

[0073]

[0074] Among them, ∈ represents a preset value, and ∈ is a very small value used to ensure numerical stability.

[0075] Furthermore, in order to distinguish the feature representations of different normal multivariate time series stored in the memory slots, a memory contrast loss L is introduced into the objective function. contrast , specifically expressed as follows:

[0076]

[0077] in,

[0078]

[0079] Among them, z t The eigenvector representation of the multivariate time series at time t, represents all memory slots with the feature vector z t The closest positive memory slot, represents all memory slots with the feature vector z t The next closest negative memory slot, α represents the boundary, [·] + represents a symbolic function, if [·] + If the number inside is positive, it returns the original value, otherwise it returns zero. k Represents the kth memory slot. It should be noted that the memory contrast loss distinguishes normal multivariate time series stored in different memory slots by reducing the distance between the feature vector representation of the multivariate time series and its nearest memory slot, and maximizing the distance between the feature vector representation of the multivariate time series and its second nearest memory slot.

[0080] Based on the preset objective function proposed in the present invention, multiple rounds of iterative training are performed on the anomaly perception model, which can enable the trained anomaly perception model to efficiently and quickly retrieve normal data patterns in the multivariate time series to reconstruct the multivariate time series during the process of reconstructing the multivariate time series, thereby achieving accurate identification of abnormal data; at the same time, it also pays attention to the local and global features of the multivariate time series, further improving the accuracy of anomaly detection.

[0081] The decoder is used to decode the new feature vector representation of the multivariate time series to obtain the reconstructed multivariate time series. It should be noted that in order to ensure that the abnormal perception model can be applied to resource-constrained scenarios and has efficient encoding and decoding processing capabilities, the decoder also uses a single-layer fully connected layer, and other network layers applicable to resource-constrained scenarios can also be used as decoders, and the present invention does not impose special restrictions.

[0082] In order to better understand the anomaly perception model, the training process of the model is introduced below. It should be noted that the anomaly perception model can be trained separately and then configured in the unsupervised anomaly perception system, or it can be trained directly in the unsupervised anomaly perception system. Figure 3 The training process is explained by the light yellow framed part.

[0083] Depend on Figure 3It can be seen that the training process of the abnormal perception model is: first obtain the normal multivariate time series (historical data) collected previously; then pass the historical data to the data acquisition module, and the data acquisition module passes the historical data to the abnormal perception model to perform multiple rounds of iterative training, wherein in each round of iteration, the parameters of the abnormal perception model are updated with the preset objective function described in the above embodiment until the model converges. It should be noted that during the training process, the abnormal perception model will continuously learn the data pattern of the normal multivariate time series and store the learned data pattern in the model.

[0084] 3. Abnormal Assessment Module

[0085] The anomaly assessment module is used to calculate the abnormality degree quantitative score of the original multivariate time series based on the original multivariate time series and the reconstructed multivariate time series according to a preset assessment method, and to assess whether the original multivariate time series has an abnormality based on the abnormality degree quantitative score according to a preset assessment rule.

[0086] Before explaining the preset evaluation method, let's first introduce the traditional anomaly evaluation method, which is specifically expressed as follows:

[0087]

[0088] in, Represents a multivariate time series at time t The reconstruction error is , F represents the number of dimensions contained in the multivariate time series, Represents a multivariate time series at time t The data of the fth dimension in Represents the reconstructed multivariate time series at time t The traditional anomaly assessment method only considers the information of the reconstructed space, lacks the fusion of temporal space and latent space information, and is prone to missed detection or false detection.

[0089] Different from the traditional anomaly assessment method, the assessment method proposed in the present invention performs anomaly detection in three different dimensions: reconstruction space, time space, and latent space. This multi-space detection method can significantly improve the anomaly detection capability and reduce the missed detection caused by only considering the information of the reconstruction space. According to one embodiment of the present invention, the preset assessment method is:

[0090]

[0091] in,

[0092]

[0093] in,

[0094]

[0095] in, Represents a multivariate time series at time t The quantitative score of abnormality, ω t represents the first index parameter, ψ t represents the second index parameter, Represents a multivariate time series at time t The reconstruction error of the data in the fth dimension in , Indicates t ′ Multivariate time series of moments The reconstruction error of the data in the fth dimension in , Represents a multivariate time series at time t The Euclidean distance to the nearest memory slot among all memory slots, f enc (·) represents the encoder, θ enc represents the encoder weight, Represents the multivariate time series at time t in all memory slots The memory slot with the closest distance. Different from the traditional anomaly evaluation method, the preset evaluation method integrates the reconstruction space, time space, and latent space to calculate a comprehensive anomaly degree quantitative score AS, which can better detect anomalies.

[0096] According to one embodiment of the present invention, the preset evaluation rule is: if the abnormality quantification score of the original multivariate time series is less than or equal to the preset threshold, the original multivariate time series is normal data, otherwise, it is abnormal data. Specifically, if the abnormality quantification score AS≤δ (δ is the preset threshold), the original multivariate time series is judged to be normal data; if the abnormality quantification score AS>δ, the original multivariate time series is judged to be abnormal data.

[0097] In order to better understand the anomaly detection process of the unsupervised anomaly perception system, Figure 3 The detection process is explained by the purple box in Figure 1. Figure 3 It can be seen that the anomaly detection process is as follows: the data acquisition module obtains the current multivariate time series (original online data) and passes the original online data to the anomaly perception model; the anomaly perception model then encodes and decodes the original online data to obtain the reconstructed online data; finally, the anomaly perception module calculates the quantitative score of the degree of anomaly based on the original online data and the reconstructed online data according to the preset evaluation method, and determines whether the original online data has anomalies based on the quantitative score of the degree of anomaly.

[0098] Based on the contents described in the above embodiments, it can be known that the unsupervised anomaly perception system is a lightweight system suitable for resource-constrained environments, such as embedded platforms such as Jeston Nano. In order to adapt to resource-constrained environments, the unsupervised anomaly perception system is configured with a simple anomaly perception model (the number of parameters can be compressed to 0.1M). The model introduces a memory slot to store the feature representation of normal multivariate time series to achieve efficient and fast retrieval of normal mode data, which can not only improve the accuracy of anomaly detection, but also reduce the computational overhead of the model. At the same time, in order to avoid missed detection and missed detection, the unsupervised anomaly perception system is also configured with an anomaly assessment module, which considers the information of reconstruction space, time space and potential space to calculate a comprehensive quantitative score of the degree of anomaly, which can detect anomalies more accurately.

[0099] Based on the unsupervised anomaly perception system described in the aforementioned embodiment, the present invention also proposes an anomaly detection method for a multivariate time series, the method comprising: step S1, obtaining a multivariate time series to be processed; step S2, using the unsupervised anomaly perception system described in the aforementioned embodiment to perform anomaly detection on the multivariate time series to be processed.

[0100] In order to verify the performance of the unsupervised anomaly perception system proposed in the present invention, the anomaly perception model in the unsupervised anomaly perception system is replaced with OCSVM, LOF, IForest, U-Time, DAGMM, ITAD, VAR, MMPCACD, CL-MPPCA, TS-CP2, BeatGAN, BOCPD, Deep-SVDD, LSTM-VAE, LSTM or OmniAnomaly model, and then a comparative experiment is conducted on the real data sets PSM, SAMP, SWaT, MSL, and SMD to obtain the experimental results shown in Tables 1-3. Among them, Table 1 shows the experimental results on the data sets SMD and MSL, Table 2 shows the experimental results on the data sets SAMP and SWaT, and Table 3 shows the experimental results on the data set PSM and the mean of the experimental results on the data sets PSM, SAMP, SWaT, MSL and SMD. In order to control the variables, the experimental settings are completely consistent, and the experimental environment is: Ubuntu 18.04LTS, Python3.1.0, and the graphics card uses NVIDIA RTX 3090.

[0101] It can be seen from Tables 1-3 that compared with the abnormal perception system after replacing other models, the unsupervised abnormal perception system proposed in the present invention shows better performance in the evaluation indicators of each data set. Among them, the evaluation index P represents precision, which is used to measure the reliability of the prediction of positive samples; the evaluation index R represents the recall rate, which is used to measure the ability to find positive samples; the evaluation index F1 represents the F1 score, which is used to comprehensively evaluate the precision and recall rate, and is the harmonic mean of the two.

[0102] Table 1

[0103]

[0104] Table 2

[0105]

[0106]

[0107] Table 3

[0108]

[0109]

[0110] The beneficial effects of the present invention are as follows: (1) The unsupervised anomaly perception system greatly reduces the number of parameters and computational complexity through a lightweight architecture design, and is suitable for resource-constrained environments; (2) A memory slot is introduced into the anomaly perception model, so that the anomaly perception model can effectively identify complex abnormal data patterns based on the feature representation of the normal multivariate time series stored in the memory slot, thereby improving the accuracy and robustness of anomaly detection; (3) A dynamic weighted reconstruction loss is introduced when training the anomaly perception model, so that the anomaly perception model can better handle the dependencies in the time series, and sparse similarity loss and memory contrast loss are also introduced, so that the model can distinguish different normal multivariate time series while paying more attention to global features, which is conducive to improving the accuracy of anomaly detection; (4) The anomaly evaluation module considers the information of the reconstruction space, time space and latent space to calculate a comprehensive quantitative score of the degree of anomaly, further improving the comprehensiveness and accuracy of anomaly detection.

[0111] It should be noted that although the above describes the various steps in a specific order, it does not mean that the various steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.

[0112] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0113] Computer readable storage medium can be a tangible device that holds and stores instructions used by an instruction execution device. Computer readable storage medium can include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0114] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. An unsupervised anomaly perception system for multivariate time series, used to detect whether there is anomaly in multivariate time series, characterized in that: The system comprises: A data collection module is used to collect original multivariate time series, where the multivariate time series contains data of multiple dimensions at the same time; The anomaly perception model is used to encode and decode the original multivariate time series to obtain the reconstructed multivariate time series; wherein the anomaly perception model is obtained by performing multiple rounds of iterative training using the multivariate time series as input and the reconstructed multivariate time series as output using a preset objective function. The anomaly perception model includes an encoder, a sparse contrast memory module and a decoder, wherein: The encoder is used to extract features from the multivariate time series to obtain a feature vector representation of the multivariate time series; The sparse contrast memory module includes a plurality of memory slots, each memory slot storing a normal feature representation of a multivariate time series, and the sparse contrast module is configured to: calculate the similarity between the feature vector representation of the multivariate time series and the feature representation stored in each memory slot, and obtain a new feature vector representation of the multivariate time series according to a preset calculation method based on the feature representation stored in each memory slot and each calculated similarity; The decoder is used to decode the new feature vector representation of the multivariate time series to obtain a reconstructed multivariate time series; The anomaly assessment module is used to calculate the quantitative score of the degree of abnormality of the original multivariate time series based on the original multivariate time series and the reconstructed multivariate time series according to a preset assessment method, and to evaluate whether the original multivariate time series has anomalies based on the quantitative score of the degree of abnormality according to preset assessment rules.

2. The system according to claim 1, characterized in that The sparse contrastive memory module is configured to compute the similarity between the feature vector representation of the multivariate time series and the feature representation stored in each memory slot as follows: Among them, s i represents the similarity between the feature vector representation of the multivariate time series and the feature representation stored in the i-th memory slot, z represents the feature vector representation of the multivariate time series, sim(·) represents the similarity metric function, τ represents the temperature coefficient, n represents the number of memory slots, and m i represents the i-th memory slot, m j represents the jth memory slot.

3. The system according to claim 2, characterized in that The default calculation method is: Among them, Z′ represents the new feature vector representation of the multivariate time series.

4. The system according to claim 3, characterized in that The preset objective function is: in, in, in, in, Among them, L totally represents the objective function, represents the dynamic weighted reconstruction loss, L sparse represents the sparse similarity loss, L contrast represents the memory contrast loss, λ s represents the first coefficient, λ c represents the second coefficient, T represents the number of multivariate time series, F represents the number of dimensions contained in the multivariate time series, and w t，f Represents a multivariate time series at time t The weight of the fth dimension in , ε t，f Represents a multivariate time series at time t The reconstruction error of the data in the fth dimension, ε t′，f Represents the multivariate time series at time t′ The reconstruction error of the data in the fth dimension in , In the multivariate time series at time t The data of the fth dimension, Represents the reconstructed multivariate time series at time t The data of the fth dimension in ∈ represents the preset value, z t The eigenvector representation of the multivariate time series at time t, represents all memory slots with the feature vector z t The closest positive memory slot, represents all memory slots with the feature vector z t The next closest negative memory slot, α represents the boundary, [·] + represents the symbolic function, m k represents the kth memory slot.

5. The system according to claim 4, characterized in that The preset evaluation method is: in, in, in, Represents a multivariate time series at time t The quantitative score of abnormality, ω t represents the first index parameter, ψ t represents the second index parameter, Represents a multivariate time series at time t The reconstruction error of the data in the fth dimension in , Represents the multivariate time series at time t′ The reconstruction error of the data in the fth dimension in , Represents a multivariate time series at time t The Euclidean distance to the nearest memory slot among all memory slots, f enc (·) represents the encoder, θ enc represents the encoder weight, Represents the multivariate time series at time t in all memory slots The closest memory slot.

6. The system according to claim 5, characterized in that The preset evaluation rule is: if the quantitative score of the abnormality degree of the original multivariate time series is less than or equal to the preset threshold, the original multivariate time series is normal data, otherwise, it is abnormal data.

7. The system according to claim 6, characterized in that The encoder and decoder both use a single fully connected layer.

8. A method for detecting anomalies in multivariate time series, characterized in that: The method comprises: Step S1, obtaining a multivariate time series to be processed; Step S2: Use the system as described in any one of claims 1 to 7 to perform anomaly detection on the multivariate time series to be processed.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of the method described in claim 8.

10. An electronic device, characterized in that: include: one or more processors; as well as A memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method of claim 8 by executing the executable instructions.

Images