Unsupervised multivariate time sequence anomaly detection method based on adaptive memory comparison

Abstract

The invention relates to the technical field of time sequence anomaly detection software, in particular to an unsupervised multivariate time sequence anomaly detection method based on adaptive memory comparison, and the method comprises the steps: constructing a feature extractor based on fast Fourier transform and 1D Swin Transform, and extracting the features of time sequence data from a time domain and a frequency domain to obtain time sequence features; constructing a self-adaptive dynamic memory updating algorithm based on hierarchical reference attention comparison, and obtaining attention weights, performing memory item updating and performing feature updating by using a reference attention mechanism respectively; constructing the memory updated features into a full connection graph G, performing feature reconstruction based on a KAN embedded graph attention network, and outputting reconstruction data; and calculating the difference between the original input and the reconstructed data on a plurality of frequency bands by using a frequency domain discrete wavelet transform loss function, and carrying out anomaly judgment in a time domain and a frequency domain. According to the method, the abnormal data can be accurately and effectively detected.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of time series anomaly detection software, and particularly relates to an unsupervised multivariate time series anomaly detection method based on adaptive memory comparison. BACKGROUND

[0002] Point anomalies, contextual anomalies and collective anomalies that may occur in the fields of detecting equipment and industrial processes, financial fraud transaction identification, traffic congestion and vehicle behavior, power grid data, etc. These abnormal data need to be detected to ensure normal production, financial security, smooth traffic and stable power supply, etc. The maturity of the current deep learning framework and the improvement of computing power have greatly promoted the development and application of time series anomaly detection technology. However, the time series in the above fields contain complex dynamic patterns (such as trends, cycles, fluctuations), and their non-stationary characteristics limit the effectiveness of methods that rely on raw data or static thresholds. Effective detection requires the construction of a robust normal behavior model, and the key lies in learning deep representations from data - this involves extracting statistical features, identifying potential patterns, or using transform domain analysis (wavelet, Fourier) and representation learning techniques to capture their internal structure and evolution law. However, the high nonlinearity and non-stationary evolution of time series in practical scenarios, combined with the diversity and sparsity of abnormal patterns (point / context / group mode deviation), severely restrict the ability of existing models to characterize normal patterns and their generalization performance, resulting in a low detection rate for unknown or complex anomalies.

[0003] In the prior art, methods capable of depicting complex features and patterns of time series have been used for anomaly detection. For example, traditional methods based on statistical models (such as ARIMA, exponential smoothing), distance or density-based unsupervised methods (such as LOF, One-Class SVM), matrix decomposition (such as PCA for dimensionality reduction and reconstruction error), and deep learning-based models such as autoencoders, generative adversarial networks (GANs), long short-term memory networks (LSTM), temporal convolutional networks (TCN), and Transformer architecture. These deep models can capture more complex temporal dependencies by learning high-dimensional representations or reconstructing input sequences. Existing time series anomaly detection includes normal pattern modeling, feature reconstruction, and anomaly scoring. In terms of feature extraction, the Transformer structure is used as the feature extractor of the entire model to capture information at different temporal and spatial scales, and further extract high-dimensional latent features in multiple time windows. Although this scheme can achieve feature extraction of data at periodic scales, the feature extraction capability is limited, especially when facing non-stationary data, which generally affects the subsequent feature aggregation and the incomplete information saved, resulting in reduced accuracy of anomaly discrimination. In terms of memory item learning update, a memory item is formed by combining a attention weight matrix with the features extracted by the extractor. Although this scheme can aggregate local feature information using the attention mechanism, it is also limited by the inherent network architecture, which cannot dynamically modify the memory item based on previously extracted feature information. In terms of loss function and anomaly discrimination, a multi-layer perceptron is used to reconstruct the features processed by the learning memory item. The reason for using only a simple multi-layer perceptron for feature reconstruction is to avoid over-reconstruction, which would reconstruct abnormal data and result in undetectable anomalies. Although this scheme can better distinguish anomalies during reconstruction, it is limited by its structure, and this reconstruction method often misjudges normal data as abnormal.

[0004] Therefore, it is necessary to provide a more accurate time series anomaly detection method for identifying anomalies. SUMMARY

[0005] The purpose of the present application is to provide an unsupervised multivariate time series anomaly detection method based on adaptive memory comparison, which can accurately and effectively detect abnormal data.

[0006] To achieve this purpose, the present application adopts the following technical solutions:

[0007] An unsupervised multivariate time series anomaly detection method based on adaptive memory comparison, comprising the following steps:

[0008] S1: Feature Extraction: Construct a feature extractor based on Fast Fourier Transform and 1D Swin Transformer to extract features from time-series data in the time and frequency domains to obtain time-series features;

[0009] S2: Memory item update: Construct an adaptive dynamic memory update algorithm based on hierarchical reference attention comparison, which obtains attention weights, updates memory items, and updates features using the reference attention mechanism respectively;

[0010] S3: Feature Reconstruction: Construct a fully connected graph G from the updated features, and perform feature reconstruction based on the KAN embedding graph attention network to output the reconstructed data;

[0011] S4: Anomaly Detection: Combine the frequency domain discrete wavelet transform loss function to calculate the difference between the original input and the reconstructed data in multiple frequency bands, and perform anomaly detection in the time domain and frequency domain.

[0012] Furthermore, in step S1, the method for extracting temporal features based on the Fast Fourier Transform and the 1D Swin Transformer feature extractor includes:

[0013] S110: The input time-series data is converted from time-domain data to frequency-domain features using Fast Fourier Transform, and the main frequency components are selected based on the amplitude of the spectrum, and reconstructed into a combination of period and period number.

[0014] S120: The reconstructed data is then input into a 1D Swin Transformer, which extracts high-dimensional features from the reconstructed frequency domain data. The extracted features are then converted back to time domain features using an inverse fast Fourier transform, restoring them to the original input dimension.

[0015] Furthermore, step S110 includes:

[0016] S111: The input time series data is defined as x∈R B×D×T B is the batch size, T is the time step, and D is the feature dimension;

[0017] S112: Convert time-domain data into frequency-domain representation using Fast Fourier Transform. k = 0, 1, ..., T-1, where x(t) is the time-domain signal, X FFT (k) is the frequency domain representation, e -j2πkt / T It is a complex exponential basis function, where k represents the frequency domain component index;

[0018] S113: Calculate the amplitude of the spectrum Based on the energy ratio η, select the k frequencies whose cumulative energy ratio reaches η, and retain these frequencies using a mask to obtain the filtered frequency domain data X. filtered= X FFT • M, M is a binary mask;

[0019] S114: Separate the real and imaginary parts of the frequency domain data X filtered , respectively, and perform segmentation along the time dimension: expand with length P and step S to obtain a tensor with shape [B, D, Np, P]; concatenate the real and imaginary part tensors along the feature dimension to form z ∈ R B·Np×D×2P .

[0020] Further, the step S120 comprises:

[0021] S121: Input z in ∈ R B·Np×D×2P , map the feature dimension from 2P to the embedding dimension D embed by linear projection, divide the time window along the sequence dimension, and calculate the window self-attention:

[0022] where Q, K, V ∈ R B · Np×W×Dembed , W is the window size in 1D SwinTransformer, d k = D embed / H;

[0023] S122: Extract high-order features by multi-layer window self-attention, output feature z trans ∈ R B·Np×D×Dembed , adjust the feature z trans dimension to 2P by linear layer, and reshape to [B, N p , D, 2P];

[0024] S123: Separate the real and imaginary parts of the feature z trans , and apply the inverse fast Fourier transform: Reshape the feature x recon (t) to [B, D embed , T], and then restore the feature shape to [B, T, D embed ] by dimension transformation, complete the feature extraction.

[0025] Further, the step S2 memory item updating method comprises:

[0026] S210: Calculate the attention weight of the input feature and the memory item by the reference attention mechanism;

[0027] S220: The memory module generates the initial memory item by random initialization, calculates the candidate memory item based on the attention weight, multiplies the variable parameter with the input feature to generate the update gate memory item, calculates the update gate by the sigmoid function, and updates the memory item by the update gate.

[0028] S230: Calculate the feature increment by referring to the attention mechanism, and fuse the feature increment with the previous feature in residual to obtain new features.

[0029] Further, the step S210 comprises:

[0030] S211: Initialize the memory module as a matrix with a set number of memory items, and train and test the memory module, and the trained memory module generates initial memory items through random initialization;

[0031] S212: The extracted feature z∈R B×T×Dembed , which is changed to z∈R L×Dembed , the attention weight of the input feature and the memory item is calculated by referring to the attention mechanism:

[0032] i=1,2,...,N,D embed , z t is the feature at time step t, and α i,t represents the attention weight of the i-th memory item M i .

[0033] S213: Weighted sum of the memory items with the attention weight, and then add the original input feature to enhance the original feature.

[0034] Further, the step S220 comprises:

[0035] S221: Refer to the attention mechanism, take the memory item as the query vector Q, and take the feature as the keyword K and the value vector V at the same time, calculate the memory item increment m' t based on the reference attention mechanism, m' t = matmul(α i,t , z t ).

[0036] S222: Use the changeable parameter W∈R N×T to generate the update gate memory item m i = matmul(W, z t ) for each extracted feature.

[0037] S223: Calculate the update gate ψ of the memory item using the intermediate memory item: ψ = sigmoid(m i ), and the memory item update formula is: M i,new = (1-ψ)·M i +ψ·m' t , M i represents the memory item that has not been updated.

[0038] The feature update in S230 includes: the new reference attention mechanism will feature z t As the query vector Q, the original memory item is taken as the keyword K and the value vector V at the same time; based on the new reference attention mechanism, the incremental part of the feature is calculated:

[0039]

[0040] Finally, the feature incremental part is combined with the feature z t , and the combined feature is:

[0041] Further, the method of feature reconstruction based on the KAN embedding graph attention network in step S3 includes:

[0042] S310: the updated memory feature is constructed into a fully connected graph G;

[0043] S320: the node is updated by the attention mechanism of the graph attention network, wherein the initial adjacency matrix and the learnable weight are used to obtain the adjacency matrix, and the KAN layer maps the feature by the adjustable activation function to obtain the reconstructed data.

[0044] Further, the step S310 includes:

[0045] S311: input feature z'∈R B×T×Dembed is regarded as the node feature of the graph, and a fully connected graph G=(V, E) is constructed, wherein the number of nodes |V| = D embed , and the node feature is the time dimension T;

[0046] S312: the initial adjacency matrix A∈R Dembed×Dembed is set as a full 1 matrix, and the variable parameter W A is used for updating: A'=softmax(W A ·A);

[0047] The step S320 includes:

[0048] S321: attention coefficient: The graph attention network updates the node feature by the attention mechanism:

[0049] W q ,W k ,W h (l) ∈R Dembed×Dembed is a variable parameter, z' i is the node feature before updating, N is the total number of nodes, and j is the current node;

[0050] S322: the KAN layer maps the features through an adjustable activation function: wherein φ k is a B-spline basis function, w k is a weight, is output data;

[0051] The KAN layer adopts Fourier series approximation of complex functions, and performs nonlinear transformation on the features through a weighted sum of sine and cosine functions; the multi-head attention mechanism of the graph attention network performs parallel calculation on multiple attention heads.

[0052] Further, the step S4 comprises:

[0053] S401: define a frequency domain discrete wavelet transform loss function:

[0054] wherein Wl(·) represents the lth layer wavelet decomposition coefficient, and L represents the decomposition layer number;

[0055] S402: on the basis of the reconstruction loss, the difference between the original input x and the reconstructed output is calculated on multiple frequency bands by combining the frequency domain discrete wavelet transform loss function, to form a time-frequency domain joint optimization target: L total = L rec + alpha L freq , L rec is the reconstruction loss, and alpha is a weight;

[0056] S403: an abnormal score system: anomal_score = latent x (recon_score + beta L freq ), beta is a weight, recon_score is a reconstruction error term, and latent is a latent feature term.

[0057] The technical solution provided by the application can include the following beneficial effects:

[0058] The method of the present application mainly comprises four steps: the first step is to construct a feature extractor based on fast Fourier transform (FFT) and 1D Swin Transformer, which is suitable for internal information representation of complex nonlinear data, and utilizes the cooperation of FFT and 1D Swin Transformer to extract global periodic trend and local non-stationary features in the data, so as to ensure the adaptability and stability of the extracted features; the second step is to construct an adaptive dynamic memory updating algorithm based on hierarchical attention comparison, which can efficiently fuse high-dimensional features and realize adaptive saving and updating of memory items with sequence input, solving the defect that the existing attention weight cannot be updated with the sequence; the third step is to use KAN enhanced graph attention network (GAT), which can better adaptively adjust the weight and capture more complex inter-node relationships based on effective aggregation of inter-node information through the powerful nonlinear fitting and dynamic correlation learning ability of KAN; the fourth step is to calculate the difference between the original input and the reconstructed data on multiple frequency bands based on the DWT loss function, optimize the reconstruction quality, realize accurate anomaly discrimination of high-dimensional complex data, overcome the limitations of traditional MLP, and improve the anomaly discrimination accuracy. Therefore, the method of the present application can accurately and effectively detect the anomaly of time series data, especially in the typical scenario of data label scarcity (very few abnormal samples), can distinguish subtle normal fluctuations from real anomalies, and can adapt to different data characteristics and application scenarios (such as industrial monitoring, financial fraud, IT operation and maintenance, and medical diagnosis) to obtain a robust anomaly detection model. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 is a flow chart of an unsupervised multivariate time series anomaly detection method based on adaptive memory comparison according to an embodiment of the present application;

[0060] Figure 2 is an architecture diagram of the method of the present application;

[0061] Figure 3 is a training and testing flow chart of the algorithm of the method of the present application. DETAILED DESCRIPTION

[0062] The embodiments of the present application will be described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as a limitation of the present application.

[0063] The embodiments of the present application will be described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as a limitation of the present application. Figure 1 , Figure 2 and Figure 3The application discloses an unsupervised multivariate time series anomaly detection method based on adaptive memory comparison.

[0064] S1: feature extraction: a feature extractor based on fast Fourier transform and 1D Swin Transformer is constructed to extract features of time series data from time domain and frequency domain, and time series features are obtained;

[0065] S2: memory item updating: an adaptive dynamic memory updating algorithm based on hierarchical reference attention comparison is constructed, reference attention mechanism is used to obtain attention weights, memory items are updated, and features are updated;

[0066] S3: feature reconstruction: the features updated by memory are constructed into a full connection graph G, feature reconstruction is performed based on a KAN embedded graph attention network, and reconstructed data are outputted;

[0067] S4: anomaly discrimination: a difference between original input and reconstructed data is calculated on multiple frequency bands in combination with a frequency domain discrete wavelet transform loss function, and anomaly discrimination is performed in time domain and frequency domain.

[0068] The method of the application mainly includes four steps: the first step is to construct a feature extractor based on fast Fourier transform (FFT) and 1D Swin Transformer, FFT captures the periodicity of data through frequency domain analysis, and 1D Swin Transformer focuses on local dynamic changes through window attention mechanism, while extracting global periodicity and local non-stationary characteristics in data, ensuring that the extracted features have wide adaptability and stability; the second step is to construct an adaptive dynamic memory updating algorithm based on hierarchical attention comparison, which efficiently fuses high-dimensional features and realizes adaptive saving and updating of memory items with sequence input, solving the defect that existing attention weights cannot be updated with sequences; the third step is to use KAN enhanced graph attention network (GAT), which can better adaptively adjust weights to capture more complex inter-node relationships based on effective aggregation of inter-node information through the powerful nonlinear fitting and dynamic correlation learning ability of KAN, thus realizing accurate reconstruction of high-dimensional data and significantly improving the discrimination accuracy of anomaly detection; the fourth step is to score the anomaly based on the DWT loss function, calculate the difference between the original input and the reconstructed data in multiple frequency bands, optimize the reconstruction quality, realize accurate anomaly discrimination of high-dimensional complex data, overcome the limitations of traditional MLP, and improve the anomaly discrimination accuracy. Therefore, the method of the application can accurately and effectively detect anomalies in time series data, especially in typical scenarios where data labels are scarce (few abnormal samples), can distinguish subtle normal fluctuations from real anomalies, and can adapt to robust anomaly detection models for different data characteristics and application scenarios (such as industrial monitoring, financial fraud, IT operation, medical diagnosis).

[0069] Further, in the step S1, the method for extracting time series features based on fast Fourier transform and 1D Swin Transformer includes:

[0070] S110: The input time series data is converted from time domain data to frequency domain features by fast Fourier transform, the main frequency components are selected according to the amplitude of the frequency spectrum, and the combination of period and period number is reconstructed;

[0071] S120: The reconstructed data is then input into the 1D Swin Transformer, high-dimensional features are extracted from the reconstructed frequency domain data, the extracted features are converted from frequency domain features to time domain features by inverse fast Fourier transform, and the dimension of the original input is restored.

[0072] The feature extractor of the application combines the fast Fourier transform and the 1D Swin Transformer, aiming to map the input data into frequency domain features to fully exploit the deep information of the data. Specifically, the fast Fourier transform is responsible for extracting frequency domain characteristics and capturing the periodicity of the data; while the 1D Swin Transformer analyzes local features through its window attention mechanism, adapting to the dynamic changes of non-stationary data. The combination of the two can comprehensively depict the potential patterns of multivariate data and improve the robustness of the model in various scenarios.

[0073] In step S110, the input data is first converted from time domain to frequency domain by fast Fourier transform (FFT), the frequency distribution is analyzed, and the main frequency components with high amplitude are selected to determine the key periodic information. The fast Fourier transform reveals the periodicity and frequency characteristics of the data by decomposing the input signal in the frequency domain, and is particularly suitable for data with obvious frequency characteristics and relatively stationary. The reconstructed data is then input into the 1D Swin Transformer for further processing.

[0074] In step S120, the 1D Swin Transformer divides the data into multiple local windows through its sliding window mechanism and calculates self-attention within each window to capture the correlation between local features. Compared with the traditional global attention mechanism, the window design of Swin Transformer significantly reduces the computational complexity, and realizes the fusion of global and local features through information interaction across windows. Finally, the features processed by the 1D Swin Transformer are sent to the inverse fast Fourier transform (iFFT) module to convert the frequency domain features back to the time domain, restoring the dimension of the original input. This step ensures the integrity of the feature extraction process while retaining the advantages of frequency domain analysis. By combining frequency domain features with time domain representations, the application can more comprehensively represent the internal structure of the data. The iFFT not only restores the time domain form of the data, but also smooths the noise that may be introduced in the feature extraction process to some extent, thereby providing more robust and reliable feature representation for subsequent anomaly detection tasks.

[0075] The step S110 includes S111-S114, specifically as follows:

[0076] S111: The input time series data is defined as x∈R B×D×T , B is the batch size, T is the time step, and D is the feature dimension.

[0077] S112: Convert the time domain data into frequency domain representation by fast Fourier transform k = 0, 1, …, T-1, where x(t) is the time domain signal, X FFT(k) is the frequency domain representation, e -j2πkt / T is the complex exponential basis function, k represents the frequency domain component index. The summation is from t = 0 to T - 1, corresponding to the periodicity assumption of the discrete signal, where X FFT ∈ C B×D×T is a complex tensor, representing the frequency domain features.

[0078] S113: Calculate the amplitude of the spectrum According to the energy ratio η (usually set to 0.8), the top k frequencies with cumulative energy ratio reaching η are selected, and these frequencies are retained by masking, obtaining the filtered frequency domain data X filtered = X FFT · M, M is a binary mask. This step is a dimension reduction and noise reduction process. In the actual signal, most of the energy is usually concentrated in a small number of frequencies (such as 80% of the energy concentrated in 20% of the frequencies), and the remaining frequencies may be noise or secondary components. Retaining the top k frequencies can greatly reduce the data volume while suppressing noise, achieving the effect of highlighting the core features. For example, in mechanical fault diagnosis, fault features are usually concentrated in specific frequencies (such as the characteristic frequency of bearing fault), and these frequencies have higher energy. By energy screening, these key frequencies can be effectively focused on. Dimension reduction and noise reduction also improve computational efficiency, as only k frequencies need to be processed in subsequent analysis (such as segmentation operation, modeling), reducing computational cost.

[0079] S114: Separate the real part and the imaginary part of the frequency domain data X filtered , and perform segmentation operation along the time dimension respectively: expand with length P (patch length) and step S, to obtain a tensor with shape [B, D, Np, P]; concatenate the real part and the imaginary part tensors along the feature dimension to form z ∈ R B·Np×D×2P .

[0080] This scheme assumes that the correlation inherent in this periodic view of time evolution remains stable in the time step dimension. This assumption guides us to observe the dynamic properties of the inter-sequence and intra-sequence correlations learned by the model. To further optimize feature extraction, the converted frequency domain data is then processed through a frequency domain filter that selects the top k frequency components with the largest amplitude from the spectrum. These high-amplitude frequencies correspond to periods that usually contain more energy and information, effectively representing the core characteristics of the data. Based on these selected key periods, the time step dimension of the input data is reconstructed into a combination of periods and period numbers. Specifically, the original time series is segmented by the selected period, with each segment containing one period of data, thereby forming a new two-dimensional structure where the row number corresponds to the number of periods and the column number corresponds to the period length. This reconstruction not only preserves the frequency domain characteristics, but also lays the foundation for subsequent local feature analysis.

[0081] In step S120, the 1D Swin Transformer processes the patch data through a window self-attention mechanism. Step S120 includes S121-S123:

[0082] S121: Input is z in ∈R B·Np×D×2P The feature dimension is mapped from 2P to the embedding dimension D through linear projection. embed Divide the time window along the sequence dimension (length D) and calculate the self-attention within the window:

[0083] Where Q, K, V ∈ R B · Np×W×Dembed W is the window size in a 1D SwinTransformer, and d k =D embed / H (H is the number of attention heads);

[0084] S122: Extract high-order features and output feature z through multi-layer window self-attention. trans ∈R B·Np×D×Dembed The feature z is processed through a linear layer. trans The dimensions are adjusted to 2P and reshaped to [B, N]. p [,D,2P];

[0085] S123: Add feature z trans Separating the real and imaginary parts, we apply the inverse fast Fourier transform: feature x recon (t) is reshaped into [B, D] embed [B, T], and then the feature shape is restored to [B, T, D] through dimensional transformation. embed ], complete feature extraction.

[0086] This method significantly improves the robustness and expressiveness of features by combining frequency domain filtering and local attention. In this invention, the window size and stride are adaptively adjusted according to the characteristics of the data to ensure that the model can capture both local details and cover global trends. Furthermore, the multi-level structure of the 1D Swin Transformer allows it to progressively extract higher-order features, thus adapting to the dynamic changes of non-stationary data. Through this process, the extractor can extract more representative high-dimensional features from the reconstructed frequency domain data, further enhancing the expressiveness of the features.

[0087] Furthermore, the method for updating the memory item in step S2 includes:

[0088] S210: Calculate the attention weights between input features and memory items using a reference attention mechanism;

[0089] S220: The memory module generates the initial memory items through random initialization, calculates the candidate memory items based on the attention weight, multiplies the input features with the variable parameters to generate the update gate memory items, and calculates the update gate using the sigmoid function to update the memory items;

[0090] S230: Calculate the feature increment by referring to the attention mechanism, and fuse the feature increment with the previous feature to obtain new features.

[0091] In this scheme, the memory module (Memory Module) is initialized as a matrix of a certain number of memory items, which represent typical patterns in the data. In the training phase, the memory items are generated through random initialization, and normalization processing is used to ensure the stability of the initial value; while in the test phase, the pre-trained memory items are directly loaded, which are usually formed by continuously comparing and updating the features extracted in the training data to ensure the consistency of the inference process.

[0092] For the input feature sequence (query), the algorithm first calculates the similarity between the input feature sequence and the memory items through the reference attention mechanism, and enhances the features. In the memory update process, the algorithm introduces a variable parameter W and another reference attention mechanism to calculate the memory increment. In order to control the amplitude of the update, the algorithm uses the sigmoid function to calculate an update gate to determine the fusion ratio of the memory increment and the old memory. Finally, the memory items are updated by weighted average: the new memory items are composed of the weighted sum of the old memory items and the memory increment, and the weight is dynamically adjusted by the update gate. This mechanism enables the memory items to adaptively retain historical information while incorporating new patterns. This scheme aims to efficiently save and update the time series features, thereby significantly improving the model's processing capability for high-dimensional time series data. The memory module generates N memory items through random initialization, which represent typical feature patterns in the data, providing a reliable foundation for subsequent feature updating and inference.

[0093] Specifically, in S220 and S230, the input features are used as the query vector Q, and the memory items are used as the keyword K and the value vector V. Dot product operation is performed on Q and K; then the softmax function is used for normalization to obtain the attention weight. This weight reflects the relevance of the input features to each memory item. Then, the algorithm uses these weights to perform weighted summation on the memory items (value vector V) to generate an enhanced representation, and adds it to the original input features to obtain a more rich feature representation. This process is similar to "reading" relevant information from memory to supplement the current input.

[0094] Specifically, the step S210 includes S211-S212:

[0095] S211: initialize the memory module as a matrix of a set number of memory items, and train and test the memory module, the trained memory module generates initial memory items through random initialization. The memory module is initialized as a matrix of a set number of memory items as initial memory items, in the training stage, the memory items are generated through random initialization and normalization; in the test stage, the pre-trained memory item matrix is loaded. During the training process, the memory items are updated step by step through the interaction with the input features to capture the typical patterns in the data.

[0096] S212: the extracted feature z∈R B×T×Dembed , which is changed to z∈R L×Dembed , the attention weight of the input feature and the memory item is calculated by referring to the attention mechanism:

[0097] i=1,2,...,N,D embed , which represents the embedding dimension, z t is the feature at time step t, and α i,t represents the attention weight of the i-th memory item M i . The similarity between the feature and the generated memory item is calculated, where z t ∈R L×Dembed is the feature at time step t.

[0098] The step S220 includes S221-S223:

[0099] S221: referring to the attention mechanism, taking the memory item as the query vector Q, and taking the feature as the keyword K and the value vector V at the same time, based on the reference attention mechanism, the candidate memory item increment m' t is calculated, m' t =matmul(α i,t ,z t );

[0100] S222: for each extracted feature, use the variable parameter W∈R N×T , the variable parameter W and the input feature generate the update gate memory item: m i =matmul(W,z t ). This design not only ensures the stability and adaptability of the update gate, but also effectively captures the potential patterns in the data through the variable parameter;

[0101] S223: calculate the update gate ψ of the memory item using the intermediate memory item: ψ=sigmoid(m i ); the memory item update formula is: M i,new =(1-ψ)·M i +ψ·m' t , M iThe memory item that has not been updated yet is represented by z, and the update gate is used to control the update amplitude of the memory item;

[0102] The feature update includes: the new reference attention mechanism calculates the feature increment part z t As the query vector Q, the original memory item is simultaneously used as the keyword K and the value vector V; based on the new reference attention mechanism, the feature increment part z

[0103]

[0104] Finally, the feature increment part z is added to the original feature z t to perform residual fusion, and the fused feature is: By adding the enhanced feature to the original feature, the expression ability of the original feature is effectively enhanced, which is similar to “reading” relevant information from the memory to supplement the current input.

[0105] When updating the feature, a reference attention mechanism based on the feature (as the query Q) and the memory item (as the key K and the value V) is designed, while when updating the memory item, the memory item is used as Q, and the feature is used as K and V), another reference attention mechanism is designed, thereby forming a hierarchical reference attention mechanism. This unique design based on hierarchical reference attention brings significant advantages: it actively builds a two-way information bridge between the feature space and the memory space, ensuring that information can flow dynamically according to the current context during feature update (reading memory) and memory item update (writing memory); the model relies on reference attention scores to drive the update process, accurately weighting the memory item to supplement the current feature during feature update, and similarly accurately weighting the updated feature to calculate the increment during memory item update, achieving adaptive updating of features and memory items in context perception. The core advantage of reference attention is particularly reflected in its strong dynamic modeling capability, by continuously calculating the instantaneous relevance (attention score) between the feature and the memory item, the model can sensitively perceive and respond to changing trends in time series data, so that the content of the memory item can accurately reflect the key information of the current input feature and the emerging new patterns in the data stream, thereby more effectively capturing and representing the dynamic changing trends and key features of time series data.

[0106] In the test phase, the model directly loads the memory items generated by the update in the training phase. This approach ensures consistency between the inference process and the training phase, while improving the reliability of the model in practical applications. By combining the reference attention mechanism, this method effectively enhances the modeling capability of complex high-dimensional time series data while maintaining computational efficiency, providing an innovative and practical solution for dynamic memory updating.

[0107] In addition, in the method of the present application, the feature extractor considers batch processing and dynamic adjustment of sequence length in actual application, ensuring good adaptability to input data of different scales. In the adaptive dynamic memory updating algorithm based on hierarchical reference attention comparison, the updating of the memory items is gradually optimized through gradient descent in the training process, so that it gradually converges to the typical mode of the data; in the test stage, the memory items are fixed to ensure consistency. This design not only enhances the modeling ability of the model for long-time dependence, but also significantly improves the adaptability and accuracy of time series anomaly detection by dynamically adjusting the memory content.

[0108] Further, the step S3 of the feature reconstruction method based on the KAN embedding graph attention network comprises:

[0109] S310: constructing the memory-updated features into a fully connected graph G;

[0110] S320: updating the nodes through the attention mechanism of the graph attention network, wherein the initial adjacency matrix and the learnable weight are used to obtain the adjacency matrix, and the KAN layer maps the features through the adjustable activation function to obtain the reconstructed data.

[0111] When the features are enhanced by the memory items, the present application proposes a KAN embedding graph attention feature reconstruction to realize accurate reconstruction of high-dimensional time series data and thus to distinguish abnormal data. This algorithm combines the sustainable and efficient nonlinear learning ability of KAN and the adaptive feature aggregation characteristics of the graph attention network, can effectively map the obtained features to the node space, and transmit and integrate information between different nodes, so as to effectively capture the complex dependence relationship in the data and improve the accuracy and stability of feature reconstruction.

[0112] Specifically, the step S310 comprises:

[0113] S311: inputting the feature z'∈R B×T×Dembed regarding the feature as the node feature of the graph, constructing a fully connected graph G=(V, E), wherein the number of nodes |V| = D embed , and the node feature is the time dimension T;

[0114] S312: updating the initial adjacency matrix A∈R Dembed×Dembed , setting it as a full 1 matrix, and updating the adjacency matrix A' through the changeable parameter W A ; updating: A' = softmax(W A ·A); the fully connected graph G provides an initial complete connectivity topology (the adjacency matrix is a full 1 matrix), and the dynamically generated adjacency matrix A' gives importance weights to each edge in the graph through the graph attention mechanism, guiding the selective attention to the key time dependence relationship during feature aggregation. The updated matrix A' in S312 is used as the calculation of the attention coefficient a ijThe basic weight matrix directly participates in the distribution of the relationship weight between nodes to guide the feature aggregation.

[0115] The step S320 comprises:

[0116] S321: Attention coefficient: The graph attention network updates the node feature through the attention mechanism:

[0117] W q ,W k ,W h (l) ∈R Dembed×Dembed is a variable parameter, z' i is the node feature before updating, N is the total number of nodes, and j is the index of the neighbor node;

[0118] S322: The KAN layer maps the feature through the adjustable activation function: Wherein φ k is a B-spline basis function, w k is a weight, is output data;

[0119] Wherein, the KAN layer adopts Fourier series to approximate complex functions, and the feature is nonlinearly transformed through the weighted sum of sine and cosine functions, and such a design enables the model to flexibly capture the periodic patterns and high-frequency changes in the data, and is particularly suitable for processing time series data with complex dynamics. The multi-head attention mechanism of the graph attention network (GAT) enhances the ability of the model to capture different feature relationships by parallel computing multiple attention heads, each head independently learns specific association patterns between nodes, thereby improving the comprehensiveness of feature aggregation. The variable parameter is initially randomly generated and changes constantly as the model is continuously trained, representing the characteristic information of the commonness of normal patterns.

[0120] In actual application, the model dynamically adjusts the graph structure through a learnable adjacency matrix, so that it adapts to the characteristics of the input data, avoiding the limitations of traditional fixed graph structures. The introduction of residual connection further optimizes the training stability of the deep network, reduces information loss, and ensures that the reconstruction result not only retains the global structure but also captures local details. In addition, the model gradually improves the reconstruction accuracy by optimizing the weights of the KAN layer and the attention parameters of the GAT during the training process; in the inference stage, the fixed parameters are used to ensure consistency.

[0121] To enhance the model's ability to perceive complex fluctuation patterns, the frequency domain discrete wavelet transform (DWT) loss function is innovatively introduced to improve the quality of anomaly discrimination through multi-scale frequency domain constraints. Specifically, the step S4 comprises:

[0122] S401: Define the frequency domain discrete wavelet transform loss function:

[0123] where DWT l denotes the l-th layer wavelet decomposition coefficient, and L is the number of decomposition layers.

[0124] S402: On the basis of the reconstruction loss, the difference between the original input x and the reconstruction output is calculated in multiple frequency bands by combining the frequency domain discrete wavelet transform loss function, forming a time-frequency domain joint optimization target: L total = L rec + aL freq , L rec is the reconstruction loss, a is the weight, and L rec is obtained by mean square error (MSE) calculation.

[0125] S403: Abnormal scoring system: anomal_score = latent x (recon_score + bLfreq), b is the weight, recon_score is the reconstruction error term, and latent is the latent feature term, that is, the similarity weight ratio of the input feature and the training memory item. b controls the weight of the frequency domain difference in the score. The time domain reconstruction error and the frequency domain loss value are fused to construct a double discrimination mechanism: the time domain score reflects the overall reconstruction deviation, and the frequency domain loss captures specific frequency fluctuation anomalies. This design enables the model to detect both transient high-frequency anomalies and periodic low-frequency anomalies, significantly improving the recognition sensitivity to complex fluctuation patterns. In the abnormal scoring stage, recon_score (reconstruction loss) is obtained by calculating the mean square error of the input and the reconstruction output, and latent (query-memory similarity weight) is generated by Softmax normalization on the distance between the query vector and the memory library prototype.

[0126] By combining the nonlinear fitting ability of KAN, the adaptive aggregation mechanism of GAT, and the constraint of the frequency domain DWT loss function, the method realizes accurate feature reconstruction in high-dimensional complex data, significantly reduces the misjudgment rate of normal data, and improves the discrimination performance of anomaly detection.

[0127] Taking industrial production as an example, time series anomaly detection is the "dynamic monitoring sentinel" of industrial production: by continuously tracking the time series signals of multiple sensors, it identifies various implicit and dynamic anomalies from the "time dimension" and "multi-parameter correlation dimension" (such as "whether the current changes synchronously when the temperature rises"), covering all aspects of equipment, process, materials, environment, and operation. For example, to identify process execution anomalies in production: temperature curve anomalies, thickness variable anomalies, current variable anomalies, material annual change anomalies, pressure change anomalies, etc.

[0128] The method of the present application, when applied, acquires time series data with time characteristics from multiple sensors in the production process, takes the time series data as input, and performs time series anomaly judgment to identify whether the production process deviates from normal operation.

[0129] Other configurations and operations of the unsupervised multivariate time series anomaly detection method based on adaptive memory comparison according to the embodiments of the present application are known to those skilled in the art, and will not be described in detail here.

[0130] In the description of the present specification, the description referring to the terms "embodiment", "example", and the like means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0131] Although the embodiments of the present application have been shown and described, those skilled in the art can understand that various changes, modifications, replacements, and variations can be made to the embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. An unsupervised multivariate time series anomaly detection method based on adaptive memory comparison, characterized in that, The method comprises the following steps: S1: feature extraction: a feature extractor based on fast Fourier transform and 1D Swin Transformer is constructed to extract features of time series data from time domain and frequency domain, and time series features are obtained; S2: memory item updating: an adaptive dynamic memory updating algorithm based on hierarchical reference attention comparison is constructed to obtain attention weights by a reference attention mechanism, update memory items, and update features; S3: feature reconstruction: the features updated by memory are constructed into a fully connected graph G, feature reconstruction is performed based on a KAN embedded graph attention network, and reconstructed data is output; S4: anomaly discrimination: a difference between the original input and the reconstructed data is calculated on multiple frequency bands by combining a frequency domain discrete wavelet transform loss function, and anomaly discrimination is performed in time domain and frequency domain.

2. The method of claim 1, wherein, In the step S1, the method for extracting time series features by the feature extractor based on fast Fourier transform and 1D Swin Transformer comprises: S110: input time series data is converted from time domain data to frequency domain features by fast Fourier transform, main frequency components are selected according to the amplitudes of the frequency spectrum, and are reconstructed into a combination of periods and period numbers; S120: the reconstructed data is then input into the 1D Swin Transformer, high-dimensional features are extracted from the reconstructed frequency domain data, the extracted features are converted from frequency domain features to time domain features by inverse fast Fourier transform, and the dimension is restored to the original input.

3. The method of claim 2, wherein, The step S110 comprises: S111: The input time series data is defined as x ∈ R B×D×T , B is the batch size, T is the time step, and D is the feature dimension. S112: convert the time-domain data into a frequency-domain representation by a fast Fourier transform k = 0, 1,..., T - 1, where x(t) is a time-domain signal, X FFT (k) is a frequency-domain representation, e -j2πkt / T is a complex exponential basis function, and k denotes a frequency-domain component index; S113: Calculate the amplitude of the spectrum Select the first k frequencies whose cumulative energy proportion reaches η according to the energy proportion η, reserve these frequencies by mask, and obtain the filtered frequency domain data X filtered = X FFT · M, M is a binary mask S114: separate the real part and the imaginary part of the frequency domain data X filtered , respectively, and perform segmentation operation along the time dimension: expand with length P and step S to obtain a tensor with shape [B, D, Np, P]; concatenate the real part and the imaginary part tensors along the feature dimension to form z∈R B·Np×D×2P .

4. The method of claim 3, wherein, The step S120 comprises: S121: input is z in ∈ R B·Np×D×2P , map the feature dimension from 2P to embedding dimension D by linear projection embed , split the time window along the sequence dimension, compute the self-attention within the window: where Q, K, V ∈ R B Np×W×Dembed W is the window size in 1D SwinTransformer, d k embed = D / H;​ S122: Extract high-order features through multi-layer window self-attention, output feature z trans ∈R B·Np×D×Dembed , the feature z trans is adjusted to 2P dimension and reshaped to [B, N p , D, 2P]; S123: Separate the real and imaginary parts of the feature z trans , apply the inverse fast Fourier transform: Reshape the feature x recon (t) to [B, D embed , T], then restore the feature shape to [B, T, D embed ] by dimension transformation, complete feature extraction.

5. The method of claim 1, wherein, The method for memory item updating in the step S2 comprises: S210: attention weights of input features and memory items are calculated by a reference attention mechanism; S220: the memory module generates initial memory items by random initialization, calculates candidate memory items based on the attention weights, multiplies the input features by variable parameters to generate update gate memory items, calculates update gates by a sigmoid function, and updates the memory items by the update gates; S230: feature increments are calculated by the reference attention mechanism, the feature increments are fused with the previous features by residual fusion to obtain new features, and feature updating is completed.

6. The method of claim 5, wherein, The step S210 comprises: S211: the memory module is initialized as a matrix with a set number of memory items, and the memory module is trained and tested, and the trained memory module generates initial memory items by random initialization; S212: The extracted feature z e R B×T×Dembed , which is changed to z e R L×Dembed , the attention weight of the input feature and the memory item is calculated by referring to the attention mechanism: i = 1, 2,..., N, D embed denotes the embedding dimension, z t is the feature at time step t, a i,t denotes the attention weight of the i-th memory item M i .

7. The method of claim 6, wherein, The step S220 comprises: S221: reference attention mechanism takes the memory item as the query vector Q, takes the feature as the keyword K and the value vector V at the same time, and calculates the memory item increment m' based on the reference attention mechanism t , m' t = matmul (a i,t , z t ) S222: use a variable parameter W e R for each extracted feature N×T The variable parameter generates an update gate memory term: m i = matmul(W, z t ); S223: calculate the update gate of the memory item using the intermediate memory item m: ψ = sigmoid(m i ); the memory item update formula is: M i,new = (1 - ψ) · M i + ψ · m' t , M i denotes the memory item that has not been updated; The feature update in S230 includes: the new reference attention mechanism will update the feature z t As the query vector Q, the original memory item is simultaneously taken as the keyword K and the value vector V; based on the new reference attention mechanism, the incremental part of the feature is calculated: Finally, the feature increment part is added to the feature z t to perform residual fusion, and the fused feature is:

8. The method of claim 1, wherein, The method for feature reconstruction based on the KAN embedded graph attention network in the step S3 comprises: S310: the features updated by memory are constructed into a fully connected graph G; S320: node updating is performed by a graph attention network through an attention mechanism, an adjacency matrix is obtained by an initial adjacency matrix and a learnable weight, and the KAN layer obtains reconstructed data by mapping features through a tunable activation function.

9. The method of claim 8, wherein, The step S310 comprises: S311: input feature z' ∈ R B×T×Dembed A fully connected graph G = (V, E) is constructed, where the number of nodes |V| = D embed The node feature is a time dimension T; S312: initial adjacency matrix A ∈ R Dembed×Dembed Set to all ones matrix, through a variable parameter W A Update: A' = softmax(W A · A); The step S320 comprises: S321: attention coefficient: The graph attention network updates the node features through the attention mechanism: W q ,W k ,W h (l) ∈R Dembed×Dembed for a variable parameter, z' i is the node feature before update, N is the total number of nodes, and j is the index of the neighbor node. S322: The KAN layer maps the features through an adjustable activation function: where φ k is a B-spline basis function, w k is a weight, is the output data; The KAN layer adopts Fourier series to approximate complex functions, and performs nonlinear transformation on features through a weighted sum of sine and cosine functions; and the multi-head attention mechanism of the graph attention network performs parallel calculation on multiple attention heads.

10. The method of claim 9, wherein, The step S4 comprises: S401: define a frequency domain discrete wavelet transform loss function: wherein Wl(·) denotes the l-th layer wavelet decomposition coefficients and L is the number of decomposition levels; S402: On the basis of the reconstruction loss, the difference between the original input x and the reconstruction output is calculated on multiple frequency bands by combining the frequency domain discrete wavelet transform loss function, forming a time-frequency domain joint optimization target: L total = L rec + aL freq , L rec is the reconstruction loss, and a is the weight;​ S403: Anomaly scoring system: anomal_score = latent x (recon_score + β · L freq ), β is a weight, recon_score is a reconstruction error term, and latent is a latent feature term.

Images