Time series data anomaly detection method, device and equipment and storage medium

Abstract

The embodiment of the application provides a time series data anomaly detection method, device and equipment and a storage medium, and relates to the cross technical field of artificial intelligence and industrial intelligent manufacturing. Multivariate time series data is input into an encoder of an anomaly detection model to obtain a latent feature vector, the latent feature vector and a memory matrix are input into a memory module of the anomaly detection model, attention weight is obtained by performing memory item addressing operation according to the latent feature vector and the memory matrix, aggregation features are obtained by performing weighted aggregation operation according to the attention weight and the memory matrix, the aggregation features are input into a decoder of the anomaly detection model to perform time series reconstruction to obtain reconstructed data, reconstruction error is obtained according to the reconstructed data and the multivariate time series data, and then an anomaly detection result is obtained. The latent features are sent into the decoder after memory addressing and weighted aggregation by the memory module, the defect of the identity mapping of the autoencoder is broken, the reconstruction error is enlarged, and the missed detection and false detection are reduced.

Description

Technical Field

[0001] This application relates to the interdisciplinary field of artificial intelligence and intelligent industrial manufacturing, and in particular to methods, apparatus, equipment and storage media for detecting anomalies in time series data. Background Technology

[0002] In real-world scenarios such as industrial IoT, smart manufacturing, aerospace telemetry and control, and data center operation and maintenance, it is necessary to perform accurate and real-time anomaly detection on multivariate time series data collected by multiple types of sensors, so as to achieve equipment fault early warning, system safety monitoring, and production quality control.

[0003] In related technologies, due to the low occurrence rate of anomalies in industrial scenarios, the scarcity of anomaly samples, and the high cost of manual annotation, autoencoders are commonly used to achieve unsupervised anomaly detection of multivariate time series. However, conventional autoencoders directly feed the latent features output by the encoder into the decoder, and the network is prone to converge to an identity transformation. Anomaly data can also be reconstructed with high precision. The reconstruction error difference between anomaly samples and normal samples is small, which can easily lead to missed detections and false detections of anomalies. Summary of the Invention

[0004] The main objective of this application is to propose a method, apparatus, device, and storage medium for anomaly detection in time series data, thereby improving the accuracy of anomaly detection in multivariate time series data.

[0005] To achieve the above objectives, a first aspect of this application proposes a method for detecting anomalies in time series data, comprising: Acquire the multivariate time series data to be detected, and input the multivariate time series data into the encoder of the anomaly detection model for encoding to obtain the latent feature vector; The latent feature vector and memory matrix are input into the memory module of the anomaly detection model. A memory item addressing operation is performed based on the latent feature vector and the memory matrix to obtain attention weights. A weighted aggregation operation is performed based on the attention weights and the memory matrix to obtain aggregated features. The memory matrix stores the temporal features of the normal operating state. The aggregated features are input into the decoder of the anomaly detection model for temporal reconstruction to obtain reconstructed data. The decoder and the encoder have symmetrical structures. The reconstruction error is obtained based on the reconstructed data and the multivariate time series data, and the anomaly detection result is obtained based on the reconstruction error and the preset quantile threshold.

[0006] In some embodiments, the step of performing memory item addressing based on the latent feature vector and the memory matrix to obtain attention weights includes: Read-only addressing is performed on all memory items in the memory matrix, and the cosine similarity between the projected latent feature vector and each memory item is calculated to obtain the original attention weights; The original attention weights are normalized using Softmax to obtain the attention weights.

[0007] In some embodiments, the weighted aggregation operation based on the attention weights and the memory matrix to obtain aggregated features includes: The attention weights and the memory items in the memory matrix are multiplied element by element to obtain the contribution features corresponding to each memory item; The aggregated feature is obtained by summing all the contribution features along the memory item dimension.

[0008] In some embodiments, the encoder includes at least one cascaded encoding unit, which sequentially includes a base convolutional layer and a downsampling layer. The process of inputting the multivariate time series data into the encoder of the anomaly detection model to obtain a latent feature vector includes: The input vector is input into the encoding unit, and the data is processed sequentially using the basic convolutional layer and the downsampling layer to obtain the output vector. The initial value of the input vector is the multivariate time series data, and the last output vector is the latent feature vector. The basic convolutional layer comprises a 3×3 one-dimensional convolutional layer, a batch normalization layer, and an activation layer cascaded in sequence. The downsampling layer is a standard convolutional layer with a 4×4 convolutional kernel and a stride of 2. The decoder comprises at least one cascaded decoding unit, which comprises an upsampling layer and the basic convolutional layer in sequence. The upsampling layer is a transposed convolutional layer with a 4×4 convolutional kernel and a stride of 2.

[0009] In some embodiments, the training process of the anomaly detection model includes at least the following steps: Multiple normal samples containing only the normal operating status of the equipment are obtained from the original time series data, and the normal samples constitute a training set without any abnormal data; The normal samples in the training set are input into the anomaly detection model for data processing. The encoder outputs training latent features, the memory module outputs training aggregate features corresponding to the training latent features, and the decoder performs temporal reconstruction on the training aggregate features to obtain reconstructed samples. The reconstructed sample is input into the encoder for encoding to obtain the reconstructed encoded vector; The total loss value is calculated based on the normal sample, the training latent features, the reconstructed sample, the reconstructed encoding vector, and the memory matrix. The anomaly detection model is trained based on the total loss value, and the memory matrix is ​​updated synchronously until the training is completed, resulting in the trained anomaly detection model.

[0010] In some embodiments, calculating the total loss value based on the normal sample, the training latent features, the reconstructed sample, the reconstructed encoding vector, and the memory matrix includes: Calculate the reconstruction loss based on the normal samples and the reconstructed samples; Calculate the first distance between the current training latent feature and the memory item with the largest cosine similarity in the memory matrix, and the second distance between the feature and other memory items in the memory matrix. Calculate the feature loss based on the distance difference between the first distance and the second distance, and sum all the feature losses to obtain N pairs of feature losses. The memory matrix is ​​orthogonally constrained to obtain the memory separation loss; The semantic loss is calculated based on the training latent features and the corresponding reconstructed encoding vectors, and all the semantic losses are summed to obtain the encoding loss; The total loss value is obtained by weighted summing of the reconstruction loss, the N-pair feature loss, the memory separation loss, and the encoding loss.

[0011] In some embodiments, the synchronous update of the memory matrix includes: Before training begins, the memory matrix is ​​initialized to a standard Gaussian distribution and then added to the learnable parameters of the anomaly detection model. Calculate the gradient of the total loss value with respect to all the learnable parameters, and feed the gradient values ​​back to the encoder, the decoder, and the memory matrix to update the corresponding parameters, thereby obtaining the updated memory matrix.

[0012] In some embodiments, the preset quantile threshold is obtained in advance according to the following steps: Input all normal samples in the training set into the trained anomaly detection model to obtain the corresponding reconstructed samples, and calculate the reconstruction error based on the corresponding normal samples and the reconstructed samples; The statistical distribution of the reconstruction error is fitted using a Gaussian mixture distribution to obtain a probability density function, and the cumulative distribution function is obtained based on the probability density function. The preset quantile threshold is determined based on the cumulative distribution function and the preset confidence quantile.

[0013] To achieve the above objectives, a second aspect of this application provides a time series data anomaly detection device, comprising: Data encoding module: used to acquire multivariate time series data to be detected, input the multivariate time series data into the encoder of the anomaly detection model for encoding, and obtain latent feature vectors; Memory operation module: A memory module used to input the latent feature vector and memory matrix into the anomaly detection model, perform memory item addressing operation based on the latent feature vector and memory matrix to obtain attention weights, and perform weighted aggregation operation based on the attention weights and memory matrix to obtain aggregated features. The memory matrix stores the temporal features of the normal operation state. Data decoding module: used to input the aggregated features into the decoder of the anomaly detection model for temporal reconstruction to obtain reconstructed data; the decoder and the encoder have symmetrical structures. Anomaly detection module: used to obtain the reconstruction error based on the reconstructed data and the multivariate time series data, and to obtain the anomaly detection result based on the reconstruction error and a preset quantile threshold.

[0014] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0015] To achieve the above objectives, a fourth aspect of the present application provides a storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0016] The time-series data anomaly detection method, apparatus, device, and storage medium proposed in this application acquire multivariate time-series data to be detected, input the multivariate time-series data into the encoder of the anomaly detection model for encoding to obtain a latent feature vector, input the latent feature vector and memory matrix into the memory module of the anomaly detection model, perform memory item addressing operation based on the latent feature vector and memory matrix to obtain attention weights, perform weighted aggregation operation based on the attention weights and memory matrix to obtain aggregated features, wherein the memory matrix stores the time-series features of normal operation, input the aggregated features into the decoder of the anomaly detection model for time-series reconstruction to obtain reconstructed data, the decoder and encoder have symmetrical structures, obtain the reconstruction error based on the reconstructed data and multivariate time-series data, and obtain the anomaly detection result based on the reconstruction error and a preset quantile threshold. This application does not directly send the latent feature vector output by the encoder into the decoder, but instead uses the memory module to complete addressing and weighted aggregation to generate aggregated features before inputting them into the decoder. This avoids the autoencoder learning identity mapping from an architectural perspective, making it difficult for anomalous samples to match the normal patterns represented by the memory matrix, increasing the reconstruction error, and effectively reducing the probability of anomaly missed detection and false detection. In addition, the memory matrix can explicitly represent diverse normal data patterns under various industrial operating conditions, and can improve the detection range when facing multivariate time-series scenarios with frequent changes in operating conditions. Attached Figure Description

[0017] Figure 1 This is a flowchart of the time series data anomaly detection method provided in the embodiments of this application.

[0018] Figure 2 This is a schematic diagram of the anomaly detection model provided in the embodiments of this application.

[0019] Figure 3 This is a flowchart provided in an embodiment of the present application for performing memory item addressing operations based on latent feature vectors and memory matrices to obtain attention weights.

[0020] Figure 4 This is a flowchart of an embodiment of the present application that shows how to perform weighted aggregation operations based on attention weights and memory matrices to obtain aggregated features.

[0021] Figure 5 This is a flowchart of the training process of the anomaly detection model provided in the embodiments of this application.

[0022] Figure 6 This is a flowchart provided in an embodiment of the present application for calculating the total loss value based on normal samples, training latent features, reconstructed samples, reconstructed encoding vectors, and memory matrices.

[0023] Figure 7 This is a flowchart of the synchronous update of the memory matrix provided in the embodiments of this application.

[0024] Figure 8 This is a flowchart illustrating the calculation of the preset quantile threshold provided in the embodiments of this application.

[0025] Figure 9 This is a visualization of the anomaly detection results provided in the embodiments of this application.

[0026] Figure 10 This is a structural block diagram of a time series data anomaly detection device provided in another embodiment of this application.

[0027] Figure 11 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0029] It should be noted that although functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0031] In real-world scenarios such as the Industrial Internet of Things (IIoT), intelligent manufacturing, aerospace telemetry and control, and data center operations and maintenance, multivariate time-series data collected by various types of sensors comprehensively records the dynamic processes of equipment operation, system status, and environmental changes. Accurate and real-time anomaly detection of this type of high-dimensional time-series data is a key technological support for achieving equipment fault early warning, system safety monitoring, and production quality control.

[0032] Due to the low anomaly rate, scarcity of anomaly samples, and high cost of manual annotation in industrial scenarios, autoencoders are commonly used to achieve unsupervised anomaly detection for multivariate time series data. This eliminates the need for labeled data and allows model learning solely from normal time series data. Unsupervised anomaly detection techniques for multivariate time series data can be broadly categorized according to their technical principles as follows.

[0033] The first type is time-series anomaly detection methods based on traditional statistical models. These methods identify data segments that deviate from normal statistical patterns by constructing a statistical distribution model of the time-series data. Typical methods include autoregressive moving average models, exponential smoothing, Gaussian mixture models, and isolation forests. While these methods are simple in principle and have low computational cost, they can only capture linear time-series dependencies and shallow statistical features. They cannot model the nonlinear coupling relationships between multiple sensors. In complex industrial time-series data characterized by strong nonlinearity, non-stationarity, and multiple operating conditions, they are prone to numerous missed and false detections, making it difficult to meet the requirements for high-precision detection.

[0034] The second type is the temporal reconstruction detection method based on recurrent neural networks. Recurrent autoencoder structures, represented by LSTM-AE and GRU-AE, utilize the modeling ability of recurrent neural networks to represent temporal dependencies. The encoder maps temporal data to a latent space, and the decoder reconstructs the original sequence, using the reconstruction error as the criterion for anomaly detection. This type of method can capture long-term temporal dependencies, but recurrent structures suffer from problems such as a large number of parameters, slow inference speed, and long-term vanishing gradients. Furthermore, during training, the network tends to converge to an identity transformation, allowing anomalous data to be reconstructed with high accuracy. The reconstruction error difference between anomalous and normal samples is small, easily leading to missed or false detections of anomalies.

[0035] The third type is temporal feature extraction methods based on convolutional and graph neural networks. To improve the efficiency of temporal feature extraction, these methods use one-dimensional convolutional neural networks to extract local temporal patterns, or use graph neural networks to model the topological relationships between sensors, such as MTAD-GAT and TranAD. Compared with recurrent networks, these methods have faster inference speeds and can extract local temporal features in parallel. However, their convolutional structures are mostly shallow, single-scale designs, making it difficult to take into account multi-scale temporal dependencies. At the same time, these methods are still based on traditional encoder-decoder architectures, without applying effective constraints to the latent feature space, and still suffer from identity mapping problems. Furthermore, they cannot explicitly represent the diverse normal temporal patterns under multiple operating conditions in industrial scenarios, and their generalization ability is poor in scenarios with frequent operating condition changes.

[0036] Based on this, embodiments of this application provide a method, apparatus, device, and storage medium for anomaly detection in time series data. Instead of directly feeding the latent feature vector output by the encoder into the decoder, it uses a memory module to perform addressing and weighted aggregation to generate aggregated features before inputting them into the decoder. This architectural approach avoids the autoencoder learning identity mappings, making it difficult for anomalous samples to match the normal patterns represented by the memory matrix, thus increasing reconstruction errors and effectively reducing the probability of missed and false detections. Furthermore, the memory matrix can explicitly represent diverse normal data patterns under various industrial operating conditions, improving the detection's adaptability in multi-variable time series scenarios with frequent changes in operating conditions.

[0037] This application provides a method, apparatus, device, and storage medium for detecting anomalies in time series data. The specific details are illustrated in the following embodiments. First, the method for detecting anomalies in time series data in this application is described.

[0038] The time series data anomaly detection method provided in this application relates to the interdisciplinary field of artificial intelligence and intelligent industrial manufacturing. This method can be applied to a terminal, a server, or a computer program running on either a terminal or a server. For example, the computer program can be a native program or software module in an operating system; it can be a native application (APP), i.e., a program that needs to be installed in the operating system to run, such as a client that supports time series data anomaly detection, i.e., a program that only needs to be downloaded to a browser environment to run; or it can be a small program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module, or plugin. The terminal communicates with the server via a network. This time series data anomaly detection method can be executed by the terminal or the server, or by the terminal and the server working together.

[0039] In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, or smartwatch, etc. The server can be a standalone server, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms; it can also be a service node in a blockchain system, where the service nodes form a peer-to-peer (P2P) network. The P2P protocol is an application layer protocol running on top of the Transmission Control Protocol (TCP). The terminal and server can connect via Bluetooth, Universal Serial Bus (USB), or a network, etc., and this embodiment does not impose any limitations.

[0040] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0041] The following describes a time series data anomaly detection method in the embodiments of this application.

[0042] Figure 1 This is an optional flowchart of the time series data anomaly detection method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps 110 to 140. It is also understood that this embodiment... Figure 1 The order of steps 110 to 140 is not specifically limited. The order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0043] Step 110: Obtain the multivariate time series data to be detected, input the multivariate time series data into the encoder of the anomaly detection model for encoding, and obtain the latent feature vector.

[0044] In one embodiment, reference is made to Figure 2 , Figure 2 This is a schematic diagram of the anomaly detection model provided in this embodiment. The anomaly detection model includes an encoder, a memory module, and a decoder, with the encoder and decoder having symmetrical structures. Specifically, to overcome the shortcomings of conventional convolutional structures such as insufficient extraction of local temporal details, inadequate modeling of long and short-term temporal dependencies, and easy feature loss due to upsampling and downsampling, this embodiment optimizes the structure of the encoder and decoder. Both the encoder and decoder are composed of multi-layered stacked basic units.

[0045] The encoder comprises at least one cascaded encoding unit, which sequentially includes a basic convolutional layer and a downsampling layer. The basic convolutional layer consists of a 3×3 one-dimensional convolutional layer, a batch normalization layer, and a LeakyReLU activation layer connected in series. Correspondingly, the decoder comprises at least one cascaded decoding unit, which sequentially includes an upsampling layer and a basic convolutional layer. This embodiment abandons the upsampling and downsampling methods of max pooling and bilinear interpolation, and uses a standard convolutional layer with a 4×4 kernel and a stride of 2 as the downsampling layer, and a transposed convolutional layer with a 4×4 kernel and a stride of 2 as the upsampling layer, avoiding detail compression and information loss when the feature dimension is transformed. Figure 2 The encoder is configured with 4 encoding units and the decoder is configured with 4 decoding units. This is only an illustrative example and is not intended to limit the number of stacking layers in this embodiment.

[0046] In one embodiment, the encoder can also be replaced by a bidirectional LSTM network, a GRU network, a temporal Transformer network, or a graph convolutional network, etc., which, together with the memory module, realizes the potential spatial constraints and is suitable for scenarios with different temporal lengths and topological structures. The standard convolutional layer can also be replaced by dilated convolutions, depthwise separable convolutions, or multi-scale receptive field convolutions, etc., to enhance the ability to model long temporal dependencies while maintaining lightweight design.

[0047] In one embodiment, for each encoding unit of the encoder, the input is an input vector and the output is an output vector. In actual processing, the input vector is input to the encoding unit, and data processing is performed sequentially using a basic convolutional layer and a downsampling layer to obtain the output vector. Figure 2 The structure is such that the input vector of the first encoding unit is the initial value of the input vector, specifically multivariate time series data, and the output vector of the last encoding unit is the latent feature vector output by the encoder.

[0048] Next, the latent feature vectors are processed in the memory module.

[0049] Step 120: Input the latent feature vector and memory matrix into the memory module of the anomaly detection model, perform memory item addressing operation based on the latent feature vector and memory matrix to obtain attention weights, and perform weighted aggregation operation based on attention weights and memory matrix to obtain aggregated features.

[0050] In one embodiment, during the inference phase, the memory module performs two operations: read-only addressing and weighted aggregation, which will be described below.

[0051] In one embodiment, reference is made to Figure 3 , Figure 3 This is a flowchart of an embodiment of the present application providing a memory item addressing operation based on latent feature vectors and a memory matrix to obtain attention weights, specifically including the following steps: Step 310: Perform read-only addressing on all memory items in the memory matrix, calculate the cosine similarity between the projected latent feature vector and each memory item, and obtain the original attention weights.

[0052] In one embodiment, the memory matrix is ​​a pre-learned matrix that stores the temporal features of normal operating conditions. Each temporal feature is a memory item. The dimension of the memory matrix is ​​used to limit the capacity of memory items to adapt to the number of industrial multi-condition modes. It can be 64×128-dimensional, that is, containing 64 independent memory items. The feature dimension corresponding to each memory item is fixed at 128 dimensions. 64 dimensions can cover the number of typical normal operating conditions in most industrial scenarios, while avoiding feature redundancy and a surge in computation caused by too many memory items. 128 dimensions effectively control the overall number of model parameters while ensuring the accuracy of normal mode representation.

[0053] In one embodiment, the latent feature vector output by the encoder is first used as the query vector. Then, parallel read-only addressing is performed on all memory items in the memory matrix to calculate the cosine similarity between the latent feature vector and each memory item, thereby obtaining the original attention weight for each memory item. The cosine similarity calculation result ranges from [-1, 1]. The closer the value is to 1, the higher the pattern matching degree between the two feature vectors. The closer the value is to -1, the greater the pattern difference. The original attention weights corresponding to all memory items together constitute the original attention weight set.

[0054] It is important to note that during the entire addressing and calculation process, only the feature values ​​stored in the memory matrix are read, and no parameters of the memory matrix are modified, thereby ensuring the stability and consistency of the addressing process.

[0055] This embodiment employs a parallel computing mode to simultaneously process the similarity calculations of all memory items, rather than sequentially reading and calculating them. This significantly improves the computing speed and meets the performance requirements of real-time anomaly detection in industrial settings. Furthermore, cosine similarity is chosen instead of Euclidean distance for similarity calculation because cosine similarity measures the directional consistency of two feature vectors in a high-dimensional space. It focuses only on the pattern similarity represented by the features and is unaffected by the overall magnitude of the feature vectors. Therefore, it can accurately identify the same normal operating condition pattern under different loads and operating intensities, effectively avoiding the mismatch problem caused by the sensitivity of Euclidean distance to feature magnitude.

[0056] Additionally, it is understandable that since the latent feature vector output by the encoder has a dimension of 256, a 1×1 convolutional layer is used to adapt and transform the latent feature vector to achieve similarity calculation between the latent feature vector and the memory item.

[0057] Step 320: Perform Softmax normalization on the original attention weights to obtain the attention weights.

[0058] In one embodiment, Softmax normalization is performed on all the obtained original attention weights to generate attention weights with values ​​ranging from [0,1], and the sum of all weight values ​​is 1. Softmax normalization amplifies the similarity differences between different memory items through an exponential function, converting the original attention weights into probabilistic contribution percentages. In the attention weights, a higher weight value for a memory item indicates a higher degree of matching between the normal operating condition mode stored in that memory item and the operating condition mode represented by the current input latent feature vector, resulting in a greater contribution to the subsequent generation of aggregated features. Conversely, a lower weight value indicates a lower matching degree and a smaller contribution. The normalized attention weights highlight the memory items with the highest matching degree of normal patterns while suppressing the influence of irrelevant memory items, effectively reducing interference from irrelevant features and strengthening the constraint effect of the memory module on the latent feature space. This constraint effect forces subsequent weighted aggregation operations to combine only the normal pattern features stored in the memory matrix, ensuring that abnormal inputs, unable to match the corresponding normal pattern memory items, produce significantly different aggregated features, generating reconstruction errors much larger than those of normal data.

[0059] Next, refer to Figure 4 , Figure 4 This is a flowchart of an embodiment of the present application providing a weighted aggregation operation based on attention weights and a memory matrix to obtain aggregated features, specifically including the following steps: Step 410: Multiply the attention weights and memory items in the memory matrix element by element to obtain the contribution features corresponding to each memory item.

[0060] In one embodiment, the attention weights are multiplied element-wise with all memory items stored in the memory matrix to generate contribution features for each memory item. Each weight value in the attention weights represents the percentage contribution of the corresponding memory item to the anomaly detection of the current input data. This method fuses the contribution allocation of the attention weights with the feature information of the memory items, thereby quantifying the contribution of each normal pattern to the reconstruction of the current input multivariate time series data.

[0061] Step 420: Sum all contributing features along the memory item dimension to obtain aggregated features.

[0062] In one embodiment, the contribution features corresponding to all memory items are accumulated and merged to obtain aggregated features. These aggregated features are weighted combinations of all normal pattern base features in the memory matrix according to attention weights, containing only normal temporal pattern information stored in memory and not any direct transmission of original latent features. The generated aggregated features are then upscaled to the dimension required for the decoder input through convolutional layers and fed into the decoder to complete temporal reconstruction. Specifically, for normal input data, the aggregated features are dominated by one or several normal pattern features with the highest matching degree, and are highly similar to the corresponding latent feature vectors, ensuring subsequent reconstruction accuracy. For abnormal input data, since no matching normal pattern can be found in the memory matrix, the aggregated features can only be composed of a mixture of multiple unrelated normal pattern features, thus showing significant differences from the corresponding latent feature vectors.

[0063] The memory module in this embodiment completely severs the direct information path from the encoder to the decoder through two operations: read-only addressing and weighted aggregation, forcing the decoder to perform time-series reconstruction only based on the normal patterns in the memory matrix.

[0064] Step 130: Input the aggregated features into the decoder of the anomaly detection model for temporal reconstruction to obtain reconstructed data.

[0065] In one embodiment, dimension-adapted aggregated features are input into the decoder of the anomaly detection model for temporal reconstruction to obtain reconstructed data. (Refer to...) Figure 2 Assuming the input multivariate time series data is 64-dimensional, the encoder and decoder employ a symmetrical stepped channel configuration, with the overall channel variation following a hierarchical arrangement of 64-dimensional -> 128-dimensional -> 256-dimensional -> 128-dimensional -> 64-dimensional. Specifically, the encoder constructs the data layer by layer from the 64-dimensional multivariate time series data to a 256-dimensional latent feature vector. During this process, the number of convolutional channels increases from 64-dimensional to 128-dimensional, and then further expands to 256-dimensional, with the channel dimension increasing progressively. Similarly, the decoder constructs the data layer by layer from the 256-dimensional latent feature vector to the 64-dimensional reconstructed data. The number of convolutional channels decreases from 256-dimensional to 128-dimensional, and finally returns to 64-dimensional, with the channel dimension decreasing progressively, forming a symmetrical stepped channel scaling structure.

[0066] This embodiment utilizes a hierarchical channel design to progressively expand the receptive field of the convolution, leveraging multi-layered stacked coding units to fully exploit long-range temporal dependencies within multivariate time-series data. Conversely, the hierarchical channel design gradually restores detailed information during the decoding stage, fully preserving minute-by-minute time-series features corresponding to high-frequency sensor vibrations and abrupt changes in equipment status. This channel arrangement enhances the model's feature representation capabilities while effectively managing the overall network parameter count and real-time inference computation, achieving both lightweight network construction and high-precision feature extraction. It strengthens the anomaly detection model's ability to capture weak and abrupt anomalies from the source of feature extraction, reducing the probability of missing minor faults.

[0067] The training process of the anomaly detection model in the embodiments of this application is described below.

[0068] In one embodiment, reference is made to Figure 5 , Figure 5 This is a flowchart of the training process of the anomaly detection model provided in this application embodiment, which includes at least the following steps: Step 510: Filter from the original time series data to obtain multiple normal samples that only contain the normal operating status of the equipment.

[0069] In one embodiment, raw time-series data synchronously output from multiple sensors in an industrial setting is collected. The raw time-series data is continuously segmented according to a preset fixed window length, generating multiple time-series input segments of equal length. A min-max normalization method is used to preprocess all time-series input segments to eliminate dimensional and numerical range differences between different sensors. From all preprocessed time-series input segments, segments containing only the normal operating state of the equipment are selected as normal samples. All normal samples together constitute a training set without any abnormal data. The remaining segments containing both normal and abnormal operating states constitute a test set. The training and test sets are strictly divided according to time order, without random shuffling. Furthermore, the normalization process only uses the statistical parameters corresponding to the training set data, without introducing any statistical information from the test set. The test set does not participate in any model training, parameter updates, or threshold calibration processes; it is only used for model performance evaluation and effect verification before deployment.

[0070] Step 520: Input normal samples from the training set into the anomaly detection model for data processing, use the encoder to output training latent features, use the memory module to output training aggregate features corresponding to the training latent features, and use the decoder to perform temporal reconstruction of the training aggregate features to obtain reconstructed samples.

[0071] In one embodiment, normal samples from the training set are input into the anomaly detection model in training batches for a complete training forward propagation. For a given normal sample, it is first input into the encoder, processed by multiple cascaded encoding units, and the output is a training latent feature. The training latent feature fully encodes the multi-scale temporal dependency information and multi-sensor correlation information of the corresponding normal sample. Then, after dimensionality adaptation transformation, the training latent feature undergoes read and weighted aggregation operations in the memory module to generate the corresponding training aggregated feature. Next, the training aggregated feature undergoes dimensionality adaptation transformation through a convolutional layer and is input into the decoder for temporal reconstruction. The decoder adopts a structure symmetrical to the encoder, and after processing by multiple cascaded decoding units, it outputs a reconstructed sample with the same dimensions as the input normal sample.

[0072] Step 530: Input the reconstructed sample into the encoder for encoding to obtain the reconstructed encoded vector.

[0073] In one embodiment, the reconstructed sample is input into the encoder again and subjected to the same encoding process as the normal sample to obtain a reconstructed encoding vector. Here, the reconstructed encoding vector has the same dimension and semantic space as the training latent features corresponding to the normal sample, which is used to constrain the semantic consistency between the original encoding and the reconstructed encoding and improve the matching degree of the model's high-level features.

[0074] Step 540: Calculate the total loss value based on normal samples, training latent features, reconstructed samples, reconstructed encoding vectors, and memory matrix. Train the anomaly detection model based on the total loss value and update the memory matrix synchronously until training is complete, thus obtaining the trained anomaly detection model.

[0075] In one embodiment, the process of calculating the total loss value is first described. (Refer to...) Figure 6 , Figure 6 This is a flowchart provided in an embodiment of the present application for calculating the total loss value based on normal samples, training latent features, reconstructed samples, reconstructed encoding vectors, and memory matrices, specifically including the following steps: Step 610: Calculate the reconstruction loss based on the normal samples and the reconstructed samples.

[0076] In one embodiment, a reconstruction loss is calculated based on the input normal samples and the corresponding reconstructed samples. The calculation formula is:

[0077] in, This indicates normal samples in this batch. This represents the corresponding reconstructed sample, where mw is the total number of elements in the sample. The reconstruction loss is calculated using mean squared error. The squared difference between the corresponding values ​​of the normal sample and the reconstructed sample is calculated element-wise. The average of all squared differences is then used to obtain the single-sample reconstruction loss. Finally, the average of the reconstruction losses for all samples in the entire batch is calculated. This reconstruction loss drives the anomaly detection model to learn the reconstruction ability of normal temporal patterns, ensuring that normal samples can be accurately reconstructed by the model.

[0078] Step 620: Calculate the first distance between the current training latent feature and the memory item with the highest cosine similarity in the memory matrix, and the second distance between the feature and other memory items in the memory matrix. Calculate the feature loss based on the distance difference between the first and second distances, and sum all feature losses to obtain N pairs of feature losses.

[0079] In one embodiment, in a training batch, N pairs of feature losses are performed. The calculation process is expressed as follows:

[0080] Where L is the number of samples in the training batch. Let i be the training latent features corresponding to the i-th sample. For memory matrix and The memory term with the highest cosine similarity. For memory matrix, except All other memory items besides N, where N is the total number of memory items in the memory matrix. Indicates the first distance. Indicates the second distance. Let represent the feature loss corresponding to the i-th sample.

[0081] Specifically, for each training latent feature corresponding to a sample, the memory item with the highest cosine similarity to the most similar memory item is first found in the memory matrix. The L2 Euclidean distance between this training latent feature and the most similar memory item is calculated as the first distance. Then, the L2 Euclidean distance between this training latent feature and all other memory items in the memory matrix is ​​calculated as the second distance. Next, based on the differences between the first distance and all the second distances, the feature loss of the current sample is calculated through exponential amplification and logarithmic smoothing. The feature losses of all samples are accumulated to obtain N pairs of feature losses. The N pairs of feature losses are used to drive the clustering of similar memory items and the separation of dissimilar memory items, thereby improving the discriminative power of memory items.

[0082] Step 630: Apply orthogonal constraints to the memory matrix to obtain the memory separation loss.

[0083] In one embodiment, memory separation loss The calculation formula is:

[0084] in, For memory matrix, It is the transpose of the memory matrix. The operation of extracting the upper triangular portion of the matrix is ​​used to exclude diagonal elements during computation. First, the product of the memory matrix and its transpose is calculated to obtain the autocorrelation matrix of the memory matrix. Then, the upper triangular portion of the autocorrelation matrix is ​​extracted, excluding diagonal elements. Finally, the L2 norm of the upper triangular portion is calculated to obtain the memory separation loss. The memory separation loss is used to drive the orthogonality between different memory terms, eliminate feature redundancy between memory terms, and improve the memory matrix's ability to represent diverse normal patterns.

[0085] Step 640: Calculate the semantic loss based on the training latent features and the corresponding reconstructed encoding vectors, and sum all the semantic losses to obtain the encoding loss.

[0086] In one embodiment, coding loss The calculation formula is:

[0087] in, Let be the reconstructed encoding vector corresponding to the i-th sample. For each sample, the mean squared error is calculated as the semantic loss of the current normal sample. The semantic losses of all normal samples in the same training batch are summed to obtain the encoding loss. The encoding loss is used to constrain the semantic consistency between the original training latent features and the reconstructed encoding vector, avoiding the loss of high-level semantic information during the reconstruction process.

[0088] Step 650: Perform a weighted sum of the reconstruction loss, N-pair feature loss, memory separation loss, and encoding loss to obtain the total loss value.

[0089] In one embodiment, the reconstruction loss N pairs of feature loss Memory separation loss and encoding loss Weighted summation is performed to obtain the total loss value. Joint constraints are applied to the latent feature space and the memory matrix. Combined with the memory orthogonal constraint mechanism, the redundancy of memory features is effectively reduced, and the discriminative power of latent features and the ability to represent normal patterns are improved.

[0090] The weighting ratio of the weighted summation can be determined through extensive industrial scenario experiments. For example, the reconstruction loss weight can be set to 1, the N-pair feature loss weight to 0.1, the memory separation loss weight to 0.05, and the encoding loss weight to 0.2. This weighting ratio can balance the constraint strength of different loss terms, enabling the anomaly detection model to simultaneously enhance the discriminativeness and orthogonality of memory terms and eliminate memory feature redundancy while ensuring accurate reconstruction of normal samples.

[0091] In one embodiment, the total loss value can also be adjusted by removing some of the constraint terms or adding adversarial loss, temporal smoothing loss, etc., as needed.

[0092] After the total loss value is calculated, the parameters are updated using the Adam optimizer with an initial learning rate of 1e-3 and a batch size of 32. A cosine annealing learning rate scheduling strategy is introduced to dynamically adjust the learning rate during training, accelerate model convergence, and enable collaborative optimization of the memory module, matrix, encoder, and decoder, thereby further improving the detection stability under various working conditions.

[0093] In one embodiment, the above training process can be extended to a semi-supervised training mode on the basis of pure unsupervised training, introducing a small number of abnormal samples to fine-tune the discrimination threshold, taking into account both small sample scenarios and high accuracy requirements.

[0094] Next, refer to Figure 7 , Figure 7 This is a flowchart of the synchronous update of the memory matrix provided in an embodiment of this application, which specifically includes the following steps: Step 710: Initialize the memory matrix to a standard Gaussian distribution before training begins, and add the memory matrix to the learnable parameters of the anomaly detection model.

[0095] In one embodiment, a one-time initialization operation is performed before model training begins, generating a 64×128 memory matrix in which all elements follow a standard Gaussian distribution with a mean of 0 and a variance of 1. The initialized memory matrix is ​​used as a learnable parameter of the anomaly detection model and is added to the model's computation graph and the optimizer's parameter update list along with the convolutional layer parameters of the encoder and decoder. This embodiment uses a standard Gaussian distribution for initialization to ensure natural orthogonality and diversity among the memory items in the initial state, preventing convergence of all memory items in the early stages of training and facilitating the differentiation of normal modes under various operating conditions later. It is understood that this initialization operation is only performed once at the start of training and is not re-initialized during training.

[0096] Step 720: Calculate the gradient of the total loss with respect to all learnable parameters, and backpropagate the gradient values ​​to the encoder, decoder, and memory matrix to update the corresponding parameters, thus obtaining the updated memory matrix.

[0097] In one embodiment, after the forward propagation of each training batch is completed and the total loss value is calculated, the gradient of the total loss value with respect to all learnable parameters of the model is automatically calculated using the automatic differentiation mechanism of the deep learning framework. The calculated gradient values ​​are simultaneously fed back to the encoder, decoder, and memory matrix, and the parameter updates of the three are performed synchronously without any order, thereby achieving collaborative optimization of the encoder, decoder, and memory matrix.

[0098] The gradient of the total loss value with respect to the memory matrix is ​​composed of the gradients of multiple loss terms. Specifically, the gradients of N pairs of feature losses drive memory terms with high matching degrees closer to their corresponding query vectors, increasing the feature clustering of similar normal patterns. Conversely, the gradient of the memory separation loss drives different memory terms away from each other, eliminating feature redundancy between memory terms. After multiple rounds of iterative training, the memory matrix gradually converges, ultimately forming a basic feature library capable of storing diverse normal operating modes in industrial scenarios. During the model inference phase, the memory matrix maintains its parameters unchanged after training convergence and does not perform any parameter update operations.

[0099] Unlike related technologies where attention networks only use basic similarity constraints for memory terms, which suffer from high redundancy, poor feature discriminative power, and inability to cover multiple normal modes, leading to misjudgments of normal modes and missed detections of abnormal modes in temporal anomaly detection scenarios, this embodiment uses a memory module to explicitly read, aggregate, and update the basic feature components of normal temporal modes in the vector space of the latent feature vectors output by the encoder. It also mechanically blocks the path for the encoder and decoder to form an identity mapping, ensuring that abnormal samples cannot be accurately reconstructed.

[0100] In one embodiment, the fixed-dimensional memory matrix can also be replaced with a dynamic incremental memory bank, a hash memory module, or a clustering-driven memory unit as needed. The corresponding memory update method can adopt strategies such as moving average and momentum update to expand the adaptability of multi-condition long-term scenarios.

[0101] Step 140: Obtain the reconstruction error based on the reconstructed data and multivariate time series data, and obtain the anomaly detection result based on the reconstruction error and the preset quantile threshold.

[0102] In one embodiment, a reconstruction error is calculated based on the reconstructed data and the input multivariate time series data. This reconstruction error is used as an anomaly score and compared with a preset quantile threshold to obtain the final anomaly detection result. The reconstruction error can be calculated using mean squared error. If, after comparison, the reconstruction error is greater than the preset quantile threshold, the multivariate time series sample is determined to be an anomaly sample; if the reconstruction error is less than or equal to the preset quantile threshold, it is determined to be a normal sample.

[0103] In one embodiment, the preset quantile threshold is determined once after the training phase and remains fixed during the inference phase, without needing to be dynamically adjusted based on the inference data. (Refer to...) Figure 8 , Figure 8 This is a flowchart illustrating the calculation of the preset quantile threshold provided in this application embodiment, specifically including the following steps: Step 810: Input all normal samples from the training set into the trained anomaly detection model to obtain the corresponding reconstructed samples, and calculate the reconstruction error based on the corresponding normal samples and reconstructed samples.

[0104] In one embodiment, after the anomaly detection model is trained and all parameters are fixed, a preset quantile threshold is calibrated. All normal samples from the training set are sequentially input into the trained anomaly detection model, and a forward propagation process identical to the inference phase is executed to obtain a reconstructed sample corresponding to each normal sample. The reconstruction error between each normal sample and its corresponding reconstructed sample is calculated using the mean squared error calculation method, ultimately resulting in a statistical set containing the reconstruction errors of all training normal samples. This statistical set only contains error data for normal samples and does not incorporate any anomaly sample information.

[0105] Step 820: Fit the statistical distribution of the reconstruction error using a Gaussian mixture distribution to obtain the probability density function, and then obtain the cumulative distribution function based on the probability density function.

[0106] In one embodiment, a Gaussian mixture distribution, rather than a single Gaussian distribution, is used to fit the statistical distribution of the aforementioned reconstruction error statistical set. The Gaussian mixture distribution, composed of a linear combination of multiple Gaussian components, can adaptively match the multi-peaked distribution characteristics of reconstruction errors under various industrial operating conditions, statistically analyzing the error distribution patterns of normal samples under different loads and operating states. Next, the expectation-maximization algorithm is used to estimate all parameters of the Gaussian mixture distribution, including the weight, mean, and variance of each Gaussian component, to obtain the probability density function of the reconstruction error. Then, integration is performed based on the probability density function to obtain the corresponding cumulative distribution function, where the cumulative distribution function represents the probability that the reconstruction error is less than or equal to a certain value.

[0107] Step 830: Determine the preset quantile threshold based on the cumulative distribution function and the preset confidence quantile.

[0108] In one embodiment, the corresponding preset quantile threshold is calculated based on the obtained cumulative distribution function and the preset confidence quantile. The preset confidence quantile can be 95%, 98%, or 99%, flexibly selected according to the cost of false positives and false negatives in the industrial scenario. For example, the 95% quantile is suitable for safety-critical scenarios, prioritizing a reduction in the false negative rate; the 98% quantile is suitable for general industrial scenarios, balancing the false positive and false negative rates; and the 99% quantile is suitable for scenarios with high false positive costs, prioritizing a reduction in the false positive rate.

[0109] The process of determining the preset quantile threshold involves finding the reconstruction error value that satisfies the cumulative distribution function value equal to the preset confidence quantile. This value is the final preset quantile threshold. This preset quantile threshold is automatically determined by the statistical distribution of normal samples in the training set, eliminating the need for manual assignment. This effectively reduces false positives and false negatives caused by changes in operating conditions, improving the detection accuracy of the anomaly detection model in complex industrial real-world scenarios.

[0110] In one embodiment, anomaly scoring can also use kernel density estimation, isolated forest, or threshold adaptive learning strategies to replace the above Gaussian distribution fitting process, depending on the actual needs.

[0111] In one embodiment, the time-series data anomaly detection method of this embodiment is validated on four real industrial time-series anomaly detection datasets, including: SMD dataset, SMAP dataset, MSL dataset, and CWRU dataset. Specifically, the SMD dataset contains time-series data of an internet server cluster, used for data center operation and maintenance anomaly detection; the SMAP and MSL datasets contain sensor data from NASA satellites and Mars rovers, respectively, used for aerospace equipment condition monitoring; and the CWRU dataset contains bearing vibration data, used for rotating machinery fault diagnosis.

[0112] Among related technologies, DAGMM, LSTM-AE, OmniAnomaly, USAD, and MTAD-GAT were selected for comparison. These models represent probabilistic models, recurrent neural networks, autoencoders, and graph attention networks, respectively. The time series data anomaly detection method in this embodiment is denoted as MASCAE. The selected validation metrics are: precision (P), recall (R), F1 score, accuracy (Acc), and overall performance (AUC). The validation results are shown in Table 1 below.

[0113]

[0114] Table 1. Schematic diagram of verification results As shown in Table 1, on the SMD dataset, this embodiment leads in both recall and AUC. The recall rate reaches 0.9780, effectively minimizing business interruptions caused by undetected server failures. The AUC reaches 0.9632, indicating the model has the strongest overall classification ability across different thresholds. The F1 score is 0.9271, ranking first, demonstrating a balance between precision and recall. On the SMAP dataset, this embodiment leads in all five metrics: precision (0.9255), recall (0.9300), F1 score (0.9146), accuracy (0.9091), and AUC (0.9296), proving the significant advantages of this method in long-term, multivariate, and noisy scenarios such as aerospace telemetry. On the MSL dataset, this embodiment also leads in all five metrics. On the CWRU dataset, the method in this embodiment performs best in terms of precision and overall performance. Specifically, the precision is 0.9787, the false positive rate is the lowest, the F1 score is 0.9485, the accuracy is 0.9528, and the AUC is 0.9647, all of which rank first. Although the recall rate of 0.9200 is slightly lower than USAD and MTAD-GAT, the higher precision achieves better overall performance, making it more suitable for industrial production scenarios where the cost of false positives is high.

[0115] In one embodiment, reference is made to Figure 9 , Figure 9 This is a visualization of the anomaly detection results provided in an embodiment of this application. The figure illustrates the input multivariate time series data, reconstructed data, anomaly score, preset quantile threshold, actual anomaly interval, and predicted anomaly interval predicted by the anomaly detection model.

[0116] The first six rows of the figure show the comparison curves between multivariate time series data and corresponding reconstructed data from six different sensor channels. The seventh row shows the unified anomaly scoring curve, the preset quantile threshold, and the true anomaly markers and predicted anomaly markers. During normal operation outside the true anomaly interval, the input multivariate time series data and reconstructed data for each channel highly overlap, indicating that the anomaly detection model can accurately reconstruct normal time series patterns. Within the true anomaly interval, the input multivariate time series data shows significant pattern deviation, and the reconstructed data cannot accurately fit the anomaly pattern, resulting in a significant difference between the two. This shows that this embodiment blocks the identity mapping through the memory module, increasing the reconstruction error of anomaly samples. The anomaly scoring curve calculated based on the reconstruction error shows a significant peak within the true anomaly interval, significantly exceeding the preset quantile threshold. The predicted anomaly interval determined by the anomaly detection model is highly matched with the true anomaly interval, completely covering all true anomaly periods. Simultaneously, no significant false detections occur during normal operation, directly verifying the high recall and low false detection performance of the method in this embodiment.

[0117] As described above, the overall architecture of this application adopts an end-to-end unsupervised training method, which only requires normal samples to complete training and does not require abnormal data annotation. This can significantly widen the reconstruction error between normal and abnormal samples, reduce the false negative and false positive rates of anomalies, and improve the detection stability under various working conditions. At the same time, the lightweight design of stacked convolutions significantly reduces the number of model parameters and greatly improves the inference speed, enabling deployment on edge devices to achieve real-time anomaly detection.

[0118] In one embodiment, the anomaly detection method of this embodiment is applicable to various industrial scenarios such as server maintenance, aerospace equipment monitoring, rotating machinery fault diagnosis, and intelligent manufacturing production lines. Its overall detection performance surpasses that of existing mainstream unsupervised anomaly detection methods. In engineering applications, it can reduce the cost of anomaly data annotation in a single scenario and the R&D investment in a universal architecture for multiple scenarios. It can provide early warnings of potential equipment anomalies, reducing unplanned downtime, minimizing equipment failures and production losses, and lowering the cost of computing hardware deployment and long-term maintenance energy consumption. It possesses strong engineering practicality and promotional value.

[0119] For example, in timing anomaly detection for data center server clusters, it is suitable for early identification of hardware failures, load anomalies, and potential service crashes. It uses the publicly available server maintenance dataset SMD, which includes multi-dimensional sensor timing data for server CPU, memory, disk, network, etc., and anomaly types include hardware failures, traffic spikes, and process freezes. In equipment status monitoring for space probes, it is suitable for unattended monitoring in the extreme environment of space, using NASA's publicly available space datasets SMAP and MSL, which include high-dimensional timing data such as probe temperature, voltage, attitude, and fuel flow. In bearing vibration fault diagnosis for rotating machinery, it is suitable for detecting vibration anomalies in rotating machinery bearings such as fans, motors, and machine tools, enabling early warning of faults such as wear, cracks, and broken teeth. It uses the CWRU bearing vibration dataset, which includes vibration timing signals from the drive end and fan end, and fault types include inner ring faults, outer ring faults, and rolling element faults. When detecting anomalies in welding production lines for automobile manufacturing, this technology is suitable for multi-sensor time-series monitoring of welding robots in intelligent manufacturing production lines. It identifies abnormal welding current, mechanical jamming, and posture deviation. It uses a real welding production line dataset from a car manufacturer, which includes multi-variable time-series data such as welding current, voltage, temperature, and position.

[0120] The technical solution provided in this application involves acquiring multivariate time-series data to be detected, inputting the multivariate time-series data into the encoder of an anomaly detection model for encoding to obtain a latent feature vector, inputting the latent feature vector and a memory matrix into the memory module of the anomaly detection model, performing memory item addressing operations based on the latent feature vector and the memory matrix to obtain attention weights, and performing weighted aggregation operations based on the attention weights and the memory matrix to obtain aggregated features. The memory matrix stores the temporal features of the normal operating state. The aggregated features are input into the decoder of the anomaly detection model for temporal reconstruction to obtain reconstructed data. The decoder and encoder have symmetrical structures. The reconstruction error is obtained based on the reconstructed data and the multivariate time-series data, and the anomaly detection result is obtained based on the reconstruction error and a preset quantile threshold. This application does not directly send the latent feature vector output by the encoder into the decoder, but instead uses the memory module to complete addressing and weighted aggregation to generate aggregated features before inputting them into the decoder. This avoids the autoencoder learning identity mappings from an architectural perspective, making it difficult for anomalous samples to match the normal patterns represented by the memory matrix, thus increasing the reconstruction error and effectively reducing the probability of missed and false detections of anomalies. In addition, the memory matrix can explicitly represent diverse normal data patterns under various industrial operating conditions, and can improve the detection range when facing multivariate time-series scenarios with frequent changes in operating conditions.

[0121] This application also provides a time series data anomaly detection device, which can implement the above-described time series data anomaly detection method, referring to... Figure 10 The device includes: Data encoding module 1010: Used to acquire the multivariate time series data to be detected, input the multivariate time series data into the encoder of the anomaly detection model for encoding, and obtain the latent feature vector.

[0122] Memory Operation Module 1020: This module is used to input the latent feature vector and memory matrix into the anomaly detection model. It performs memory item addressing operation based on the latent feature vector and memory matrix to obtain attention weights. It then performs weighted aggregation operation based on the attention weights and memory matrix to obtain aggregated features. The memory matrix stores the temporal features of the normal operating state.

[0123] Data decoding module 1030: Used to temporally reconstruct the decoder of the anomaly detection model by inputting aggregated features into it, and obtain reconstructed data. The decoder and encoder have symmetrical structures.

[0124] Anomaly detection module 1040: used to obtain the reconstruction error based on the reconstructed data and multivariate time series data, and to obtain the anomaly detection result based on the reconstruction error and the preset quantile threshold.

[0125] The specific implementation of the time series data anomaly detection device in this embodiment is basically the same as the specific implementation of the time series data anomaly detection method described above, and will not be repeated here.

[0126] This application also provides an electronic device, including: At least one memory; At least one processor; At least one program; The program is stored in a memory, and the processor executes the at least one program to implement the time-series data anomaly detection method described above in this application. The electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0127] Please see Figure 11 , Figure 11 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 1101 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 1102 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1102 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1102 and is called and executed by the processor 1101 to execute the time-series data anomaly detection method of the embodiments of this application. Input / output interface 1103 is used to implement information input and output; The communication interface 1104 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 1105 transmits information between various components of the device (e.g., processor 1101, memory 1102, input / output interface 1103, and communication interface 1104); The processor 1101, memory 1102, input / output interface 1103 and communication interface 1104 are connected to each other within the device via bus 1105.

[0128] This application also provides a storage medium storing a computer program that, when executed by a processor, implements the above-described time series data anomaly detection method.

[0129] Memory, as a non-transitory storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0130] The time-series data anomaly detection method, apparatus, device, and storage medium proposed in this application acquire multivariate time-series data to be detected, input the multivariate time-series data into the encoder of the anomaly detection model for encoding to obtain a latent feature vector, input the latent feature vector and memory matrix into the memory module of the anomaly detection model, perform memory item addressing operation based on the latent feature vector and memory matrix to obtain attention weights, perform weighted aggregation operation based on the attention weights and memory matrix to obtain aggregated features, wherein the memory matrix stores the time-series features of normal operation, input the aggregated features into the decoder of the anomaly detection model for time-series reconstruction to obtain reconstructed data, the decoder and encoder have symmetrical structures, obtain the reconstruction error based on the reconstructed data and multivariate time-series data, and obtain the anomaly detection result based on the reconstruction error and a preset quantile threshold. This application does not directly send the latent feature vector output by the encoder into the decoder, but instead uses the memory module to complete addressing and weighted aggregation to generate aggregated features before inputting them into the decoder. This avoids the autoencoder learning identity mapping from an architectural perspective, making it difficult for anomalous samples to match the normal patterns represented by the memory matrix, increasing the reconstruction error, and effectively reducing the probability of anomaly missed detection and false detection. In addition, the memory matrix can explicitly represent diverse normal data patterns under various industrial operating conditions, and can improve the detection range when facing multivariate time-series scenarios with frequent changes in operating conditions.

[0131] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0132] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0133] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0134] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0135] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0136] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0137] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0138] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0139] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0140] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0141] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for anomaly detection in time series data, characterized in that, include: The process involves acquiring multivariate time-series data to be detected, inputting the multivariate time-series data into the encoder of an anomaly detection model for encoding to obtain a latent feature vector. The encoder includes at least one cascaded encoding unit, which in turn includes a basic convolutional layer and a downsampling layer. The basic convolutional layer includes a cascaded 3×3 one-dimensional convolutional layer, a batch normalization layer, and an activation layer. The downsampling layer is a standard convolutional layer with a 4×4 convolutional kernel and a stride of 2. The multivariate time-series data includes server CPU signals, memory signals, disk signals, and network signals in server operation and maintenance monitoring scenarios; detector temperature, voltage, attitude, and fuel flow in detector equipment status monitoring scenarios; vibration timing signals of the drive end and fan end in bearing vibration fault diagnosis scenarios; and welding current, voltage, temperature, and position in welding production line anomaly monitoring scenarios. The latent feature vector and memory matrix are input into the memory module of the anomaly detection model. A memory item addressing operation is performed based on the latent feature vector and the memory matrix to obtain attention weights. A weighted aggregation operation is performed based on the attention weights and the memory matrix to obtain aggregated features. The memory matrix stores the temporal features of the normal operating state. The aggregated features are input into the decoder of the anomaly detection model for temporal reconstruction to obtain reconstructed data. The decoder and the encoder have symmetrical structures and adopt a symmetrical stepped channel configuration. The decoder includes at least one cascaded decoding unit. The decoding unit includes an upsampling layer and the basic convolutional layer in sequence. The upsampling layer is a transposed convolutional layer with a 4×4 convolutional kernel and a stride of 2. The reconstruction error is obtained based on the reconstructed data and the multivariate time series data, and the anomaly detection result is obtained based on the reconstruction error and the preset quantile threshold. The training process of the anomaly detection model involves calculating the total loss value based on normal samples, training latent features, reconstructed samples, reconstructed encoding vectors, and a memory matrix. This includes: calculating the reconstruction loss based on the normal samples and the reconstructed samples; calculating the first distance between the current training latent feature and the memory item with the highest cosine similarity in the memory matrix, and the second distance between the current training latent feature and other memory items in the memory matrix; calculating the feature loss based on the distance difference between the first and second distances; summing all the feature losses to obtain N pairs of feature losses; applying orthogonal constraints to the memory matrix to obtain a memory separation loss; calculating the semantic loss based on the training latent feature and the corresponding reconstructed encoding vector; summing all the semantic losses to obtain an encoding loss; and weighted summing the reconstruction loss, the N pairs of feature losses, the memory separation loss, and the encoding loss to obtain the total loss value.

2. The time series data anomaly detection method according to claim 1, characterized in that, The step of performing memory item addressing based on the latent feature vector and the memory matrix to obtain attention weights includes: Read-only addressing is performed on all memory items in the memory matrix, and the cosine similarity between the projected latent feature vector and each memory item is calculated to obtain the original attention weights; The original attention weights are normalized using Softmax to obtain the attention weights.

3. The time series data anomaly detection method according to claim 2, characterized in that, The step of performing a weighted aggregation operation based on the attention weights and the memory matrix to obtain aggregated features includes: The attention weights and the memory items in the memory matrix are multiplied element by element to obtain the contribution features corresponding to each memory item; The aggregated feature is obtained by summing all the contribution features along the memory item dimension.

4. The time series data anomaly detection method according to claim 1, characterized in that, The step of inputting the multivariate time series data into the encoder of the anomaly detection model for encoding to obtain a latent feature vector includes: The input vector is input into the encoding unit, and the data is processed sequentially using the basic convolutional layer and the downsampling layer to obtain the output vector. The initial value of the input vector is the multivariate time series data, and the last output vector is the latent feature vector.

5. The time series data anomaly detection method according to claim 1, characterized in that, The training process of the anomaly detection model includes at least the following steps: Multiple normal samples containing only the normal operating status of the equipment are obtained from the original time series data, and the normal samples constitute a training set without any abnormal data; The normal samples in the training set are input into the anomaly detection model for data processing. The encoder outputs training latent features, the memory module outputs training aggregate features corresponding to the training latent features, and the decoder performs temporal reconstruction on the training aggregate features to obtain reconstructed samples. The reconstructed sample is input into the encoder for encoding to obtain the reconstructed encoded vector; The total loss value is calculated based on the normal sample, the training latent features, the reconstructed sample, the reconstructed encoding vector, and the memory matrix. The anomaly detection model is trained based on the total loss value, and the memory matrix is ​​updated synchronously until the training is completed, resulting in the trained anomaly detection model.

6. The time series data anomaly detection method according to claim 5, characterized in that, The synchronous update of the memory matrix includes: Before training begins, the memory matrix is ​​initialized to a standard Gaussian distribution and then added to the learnable parameters of the anomaly detection model. Calculate the gradient of the total loss value with respect to all the learnable parameters, and feed the gradient values ​​back to the encoder, the decoder, and the memory matrix to update the corresponding parameters, thereby obtaining the updated memory matrix.

7. The time series data anomaly detection method according to claim 1, characterized in that, The preset quantile threshold is obtained in advance according to the following steps: Input all normal samples in the training set into the trained anomaly detection model to obtain the corresponding reconstructed samples, and calculate the reconstruction error based on the corresponding normal samples and the reconstructed samples; The statistical distribution of the reconstruction error is fitted using a Gaussian mixture distribution to obtain a probability density function, and the cumulative distribution function is obtained based on the probability density function. The preset quantile threshold is determined based on the cumulative distribution function and the preset confidence quantile.

8. A time series data anomaly detection device, characterized in that, include: Data encoding module: used to acquire multivariate time series data to be detected, input the multivariate time series data into the encoder of the anomaly detection model for encoding, and obtain latent feature vectors. The encoder includes at least one cascaded encoding unit, which includes a basic convolutional layer and a downsampling layer in sequence. The basic convolutional layer includes a cascaded 3×3 one-dimensional convolutional layer, a batch normalization layer and an activation layer. The downsampling layer is a standard convolutional layer with a 4×4 convolutional kernel and a stride of 2. The multivariate time series data includes server CPU signals, memory signals, disk signals and network signals in server operation and maintenance monitoring scenarios, or detector temperature, voltage, attitude and fuel flow in detector equipment status monitoring scenarios, or vibration timing signals of the drive end and fan end in bearing vibration fault diagnosis scenarios, or welding current, voltage, temperature and position in welding production line anomaly monitoring scenarios. Memory operation module: A memory module used to input the latent feature vector and memory matrix into the anomaly detection model, perform memory item addressing operation based on the latent feature vector and memory matrix to obtain attention weights, and perform weighted aggregation operation based on the attention weights and memory matrix to obtain aggregated features. The memory matrix stores the temporal features of the normal operation state. The data decoding module is used to input the aggregated features into the decoder of the anomaly detection model for temporal reconstruction to obtain reconstructed data. The decoder and the encoder have symmetrical structures and adopt a symmetrical stepped channel configuration. The decoder includes at least one cascaded decoding unit, which in turn includes an upsampling layer and a basic convolutional layer. The upsampling layer is a transposed convolutional layer with a 4×4 convolutional kernel and a stride of 2. The anomaly detection module is used to obtain the reconstruction error based on the reconstructed data and the multivariate time series data, and to obtain the anomaly detection result based on the reconstruction error and a preset quantile threshold. The training process of the anomaly detection model involves calculating the total loss value based on normal samples, training latent features, reconstructed samples, reconstructed encoding vectors, and a memory matrix. This includes: calculating the reconstruction loss based on the normal samples and the reconstructed samples; calculating the first distance between the current training latent feature and the memory item with the highest cosine similarity in the memory matrix, and the second distance between the current training latent feature and other memory items in the memory matrix; calculating the feature loss based on the distance difference between the first and second distances; summing all the feature losses to obtain N pairs of feature losses; applying orthogonal constraints to the memory matrix to obtain a memory separation loss; calculating the semantic loss based on the training latent feature and the corresponding reconstructed encoding vector; summing all the semantic losses to obtain an encoding loss; and weighted summing the reconstruction loss, the N pairs of feature losses, the memory separation loss, and the encoding loss to obtain the total loss value.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the time series data anomaly detection method according to any one of claims 1 to 7.

10. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the time series data anomaly detection method according to any one of claims 1 to 7.

Images