Time-frequency dual-domain isolation time sequence anomaly detection method based on Mamba-self-attention

Abstract

The invention discloses a time-frequency double-domain isolation time sequence anomaly detection method based on Mamba-self-attention, which can be applied to the fields of industrial manufacturing, medical equipment and the like and can detect anomaly by quantifying time-frequency difference. The method comprises the steps of extracting a multivariable time sequence sample from business data, and obtaining multivariable data through reversible instance normalization; the input time domain representation module is used for independently inputting a Mama network according to a natural time sequence and an inversion time sequence, capturing forward and reverse time features and fusing the forward and reverse time features into time features; a frequency domain representation module is input, seasonal variables are extracted, frequency features are extracted in combination with discrete cosine transform and an attention mechanism, and the frequency features are reconstructed to a time domain through inverse discrete cosine; the time and frequency characteristics are input into a time-frequency difference module, and the inconsistency is quantified through Kullback-Leible (KL) divergence so as to compare and learn a similarity loss function training model; and generating an anomaly score and setting a hyper-parameter to judge anomaly. The method is based on a bidirectional Mama and self-attention time-frequency double-domain isolation architecture, mode specificity discrimination features masked by traditional fusion are reserved, the consistency of normal mode domains is high, the correlation of abnormal performance is collapsed, and the detection accuracy and reliability are improved.

Description

Technical Field

[0001] This invention relates to the fields of industrial manufacturing, medical devices, and finance, and particularly to a time-frequency dual-domain isolated time series anomaly detection method based on Mamba-self-attention. Background Technology

[0002] Time series anomaly detection algorithms have wide and crucial applications in many fields, and their accuracy and effectiveness directly affect the quality and efficiency of decision-making. For example, in industry, efficient anomaly detection can scan production line equipment operating data (such as vibration, temperature, and current) in real time, capturing potential fault signals such as "abnormally increased vibration" and "sudden temperature changes" in the first instance, driving equipment from "passive maintenance" to "predictive maintenance," significantly reducing the probability of unplanned downtime and equipment maintenance costs. In the medical equipment field, accurate time series anomaly detection can monitor physiological signals (such as ECG waveforms, blood oxygen concentration, and body temperature timing) collected by monitors and implantable devices at the millisecond level, identifying critical characteristics such as "increased arrhythmia" and "sudden drop in blood oxygen concentration" in the first instance, securing the golden window for treatment, and significantly improving clinical treatment efficiency and patient survival probability. In the financial field, time series anomaly detection can keenly capture risk characteristics such as "abnormal transaction frequency" and "sudden changes in fund flows," providing decision-making basis for real-time payment interception and stock market anomaly warnings, strengthening the defense line for fund security and preventing systemic financial risks. Furthermore, with the explosive growth of IoT terminals and the increasing real-time nature of data streams, traditional detection methods have revealed their shortcomings in generalization ability and real-time performance. Therefore, there is an urgent need for more advanced methods to process complex time-series data.

[0003] Current time series anomaly detection algorithms based on deep learning, especially those based on autoencoders, variational autoencoders (VAEs), and Transformer architectures, still have fundamental limitations, despite their advantages in feature extraction and pattern modeling. These methods struggle to handle cross-domain feature complementarity, address inherent defects in reconstructed targets, and efficiently model long-range time dependencies.

[0004] Time series anomaly detection schemes based on reconstruction paradigms (such as Autoencoder and VAE) generally suffer from the "identity shortcut" problem—the model tends to converge to trivial solutions that directly map the input to the output, severely weakening the discriminative power of anomaly reconstruction errors. While anomaly detection methods using the Transformer architecture can capture long-range correlations, their computational complexity increases quadratically with the sequence length (O(n)). 2In industrial-grade long-term scenarios, real-time performance is severely limited. Furthermore, traditional methods often focus on single-domain (time or frequency) analysis, neglecting the inherent complementarity of time and frequency features. This results in a persistently high false negative rate for cross-domain composite anomalies, such as time-domain burst anomalies combined with frequency-domain periodic distortion anomalies. These shortcomings have created technical bottlenecks, making it difficult for existing solutions to meet the demands of "low latency and high robustness" in complex industrial production lines and financial transaction scenarios. There is an urgent need to overcome the technical constraints of single-domain bias, reconstruction dependence, and inefficient long-range modeling. Summary of the Invention

[0005] The embodiments of the present invention provide a time-frequency dual-domain isolated time series anomaly detection method based on Mamba-self-attention, in order to solve the problem of inefficient modeling of long-distance dependencies that related technologies, such as single-domain time series anomaly detection models, ignore the inherent complementarity between temporal dynamics and frequency features.

[0006] To achieve the above objectives, the technical solution adopted in this application is as follows:

[0007] The method provided in the embodiments of the present invention includes:

[0008] S1. Extract multivariate time series data from business data and perform reversible instance normalization to obtain multivariate data.

[0009] The business data includes, but is not limited to, sensor data from industrial production lines, physiological signal data output by medical devices, and transaction behavior data from the financial sector. For example, the sampled data is uniformly sampled, and the sampling frequency includes, but is not limited to, 1 millisecond, 1 minute, and 1 hour. In the industrial manufacturing sector, the business data consists of multi-dimensional sensor sequences of production line equipment (such as vibration, temperature, and current time series), which are represented as M-channel sensor data sampled L times within a certain monitoring period (such as vibration waveforms of machine tools on an assembly line and temperature change curves of furnaces). In the medical device sector, the business data consists of physiological signal time series collected by monitors and wearable devices (such as electrocardiogram waveforms, blood oxygen concentration, and body temperature fluctuation sequences), which are represented as M-type physiological indicator sequences sampled L times within a certain period (such as the trend of heart rate variability in postoperative patients and respiratory rate fluctuations in intensive care). In the financial transaction sector, the business data consists of time series data of market placement behavior recorded by the trading system (such as stock prices, futures trading volume, and cryptocurrency trading frequency sequences), which are represented as M-type financial indicator sequences sampled L times within a certain period (such as the dynamic order volume of cryptocurrencies and the exchange rate fluctuation trajectory in the foreign exchange market).

[0010] S2. Input the multivariate data in S1 into the time domain representation module, and input them into the Mamba network independently in natural time order and reverse time dimension order to capture the forward time features and reverse time features of the time series. Then, fuse these complementary features as time features.

[0011] S3. Input the multivariate data in S1 into the frequency domain representation module, extract the seasonal variables, combine discrete cosine transform and attention mechanism to perform frequency analysis to extract frequency features, and then use inverse discrete cosine transform to reconstruct the frequency features to the time domain.

[0012] S4. Input the time features and frequency features obtained from S2 and S3 into the time-frequency difference module, measure the inconsistency between time features and frequency features using Kullback-Leible (KL) divergence, and train the time series anomaly detection model using a similarity loss function based on contrastive learning to obtain the trained time series model.

[0013] S5. The time series anomaly detection model obtained in S4 generates anomaly scores, and a hyperparameter δ is set to determine whether a certain time point is an anomaly. The time series anomaly detection model can identify abnormal time points through input real-time or historical business data. For example, in the industrial manufacturing sector, it can promote the shift from "passive maintenance" to "predictive maintenance," proactively intercepting the risk of unplanned downtime; in the medical equipment sector, it can secure a golden response window for the treatment of severe illnesses such as acute myocardial infarction and respiratory failure; in the financial transaction sector, it can provide decision-making basis for real-time payment interception and stock market anomaly warnings, strengthening the financial security defense line.

[0014] The present invention provides a time-frequency dual-domain isolated time series anomaly detection method based on Mamba-self-attention, comprising: performing reversible instance normalization on the input multivariate time series to obtain multivariate data; inputting the multivariate data into a time domain representation module, independently inputting it into a Mamba network according to natural time order and reverse time dimension to capture the forward and reverse time features of the time series, and then fusing these complementary features as time features; inputting the multivariate data into a frequency domain representation module, extracting seasonal variables, combining discrete cosine transform and attention mechanism to perform frequency analysis to extract frequency features, and then reconstructing the frequency features into the time domain using inverse discrete cosine transform; inputting the time features and frequency features into a time-frequency difference module, using Kullback-Leible (KL) divergence to measure the inconsistency between the time features and frequency features, and training a time series anomaly detection model using a similarity loss function based on contrastive learning to obtain a trained time series model; generating anomaly scores through the time series anomaly detection model, and setting hyperparameter δ to determine whether a certain time point is an anomaly. This invention can model time series from both time and frequency domain perspectives and design a domain isolation architecture, thereby solving the technical constraints of single-domain bias, reconstruction dependency, and inefficiency in long-range modeling. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art are briefly introduced below. Obviously, the drawings in the following description are only some specific embodiments of the present invention. For those skilled in the art, other drawings can be derived from these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of a possible implementation of the time-frequency dual-domain isolated time series anomaly detection method based on Mamba-self-attention provided in the embodiments of the present invention, wherein (a) is a time-domain representation module, (b) is a Mamba module, (c) is a frequency-domain representation module, and (d) is a self-attention module.

[0017] Figure 2 This is a schematic diagram of the method flow provided in an embodiment of the present invention.

[0018] Figure 3 This is a schematic diagram of the time-frequency domain difference UMAP visualization on the SWaT dataset provided in an embodiment of the present invention.

[0019] Figure 4 The image shows the detection results for various anomaly types provided in this embodiment of the invention, including global point anomalies, context point anomalies, seasonal anomalies, grouping anomalies, and trend anomalies. The red dots and areas represent anomaly points and areas. Detailed Implementation

[0020] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Embodiments of the present invention will be described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in the specification of the present invention means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or couplings. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items. It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the meaning consistent with their meaning in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0021] Compared to traditional time series anomaly detection schemes based on Transformer and RNN, the State Space Model (SSM) has emerged as a more promising technological paradigm. By mapping an infinite-dimensional time series history to a finite-dimensional latent space, it overcomes the bottleneck of long-range dependency modeling in traditional recursive architectures. While RNN / LSTM-based anomaly detection methods can only effectively model approximately 50 tokens, SSM, leveraging the characteristics of linear time-invariant systems, can accurately capture long-range dependencies at hundreds or even thousands of time steps. In this embodiment, Mamba introduces an "input-feature-based SSM parameterization method" and combines it with a "selective state transition mechanism" to address the inefficiency of traditional SSM in selectively capturing time-frequency dual-domain features. By assigning independent projection parameters to each time step and employing attention-enhanced feature interactions for the frequency-domain periodic components, it enables the deep application of SSM in time series anomaly detection with time-frequency dual-domain isolation. Based on this, the asymmetric bi-branch structure of "bidirectional Mamba time modeling + self-attention frequency analysis" solves the problems of "high computational complexity, fuzzy abnormal patterns in single-domain feature fusion, and identity shortcuts induced by reconstruction target" in the traditional Transformer architecture. It achieves efficient modeling of long-range time dependence and global frequency interaction, while retaining specific discriminative features in the time and frequency domain to support accurate quantification of cross-domain inconsistencies of anomalies.

[0022] The invention provides a time-frequency dual-domain isolated time series anomaly detection method based on Mamba-self-attention, such as... Figure 1 , 2 As shown, it includes:

[0023] S1. Extract multivariate time series data from business data and perform reversible instance normalization to obtain multivariate data; wherein, the types of business data include at least: industrial production line sensor data, physiological signal data output by medical devices, and financial transaction behavior data. The data in the time series samples can be extracted from industrial production line sensor data, physiological signal data output by medical devices, financial transaction behavior data, or other data; this embodiment does not limit the types of data.

[0024] S2. Input the multivariate data in S1 into the time domain representation module, and input them into the Mamba network independently in natural time order and reverse time dimension order to capture the forward time features and reverse time features of the time series. Then, fuse these complementary features as time features.

[0025] S3. Input the multivariate data in S1 into the frequency domain representation module, extract the seasonal variables, combine discrete cosine transform and attention mechanism to perform frequency analysis to extract frequency features, and then use inverse discrete cosine transform to reconstruct the frequency features to the time domain.

[0026] S4. Input the time features and frequency features obtained from S2 and S3 into the time-frequency difference module, measure the inconsistency between time features and frequency features using Kullback-Leible (KL) divergence, and train the time series anomaly detection model using a similarity loss function based on contrastive learning to obtain the trained time series model.

[0027] S5. The time series anomaly detection model obtained from S4 generates anomaly scores, and the hyperparameter δ is set to determine whether a certain time point is an anomaly.

[0028] Specifically, in S1 of this embodiment, the steps include: acquiring time series data, normalizing the input multivariate time series to obtain multivariate data, and scaling the data to a range with a mean of 0 and a variance of 1, so that the data distribution approximates a normal distribution. The time series sample is obtained by sorting multiple data samples according to their acquisition time, and the sequence segment sample includes at least one data sample. This is achieved through the following formula: extracting L data samples at preset times from the business data and arranging them in the preset time sequence to obtain the time series sample, where L is a positive integer and L≥2; the specific implementation steps for normalizing the time series sample are as follows: X raw The original input time series sample, μ is the mean of the data, σ is the variance of the data, and X is the standardized data. The data of multiple variables are standardized into the set of multivariate data. During the normalization process, the multivariate data is scaled to a range with a mean of 0 and a variance of 1.

[0029] For example Figure 1 As shown, the input multivariate time series is processed by reversible instance normalization to obtain multivariate data. On one hand, the multivariate data is input into the time domain representation module, and independently into the Mamba network according to the natural time order and the reverse time dimension order to capture the forward and reverse time features of the time series. These complementary features are then fused as time features. On the other hand, the multivariate data is input into the frequency domain representation module to extract seasonal variables. Frequency features are extracted by combining discrete cosine transform and self-attention mechanism. The frequency features are then reconstructed into the time domain using inverse discrete cosine transform. The inconsistency between time features and frequency features is then quantified by Kullback-Leible (KL) divergence. A time series anomaly detection model is trained by a similarity loss function based on contrastive learning to obtain the trained time series model. Finally, anomaly scores are generated by the time series anomaly detection model, and a hyperparameter δ is set to determine whether a certain time point is an anomaly.

[0030] It should be noted that traditional time series anomaly detection methods in related technologies, such as Autoregressive Integrated Moving Average (ARIMA) and Vector Autoregression (VAR), use statistical methods to capture patterns, trends, and seasonality in time series data to identify anomalies. However, real-world time series data often exhibit complex nonlinear relationships (such as sudden failures in industrial equipment, abnormal electrocardiograms in medical settings, and anomalous financial transactions) and long-range anomaly dependencies (such as hidden signs a few hours before a failure). The linear assumptions and short-term memory limitations of traditional methods make it difficult to accurately identify such complex anomalies—missing the gradual trend of chronic faults in real-time industrial production line monitoring and misjudging isolated electrocardiogram mutations as normal fluctuations in critical medical condition warnings, resulting in severely insufficient generalization and accuracy in complex scenarios. For the Transformer model in related technologies, its self-attention mechanism models relationships at a token-level granularity, naturally ignoring the strict sequentiality of time series (such as the continuous causality of 1-second time series in industrial sensors and the physiological logic of millisecond-level waveforms in medical electrocardiograms). While introducing positional encoding mitigates this problem, it cannot fully restore the inherent dependency structure of time series. Furthermore, traditional methods often focus on extracting features from a single domain (time or frequency only) or simply concatenate time-frequency features, blurring the domain-specific discriminative nature of "time-domain burst point anomalies" and "frequency-domain periodic distortion anomalies," thus missing cross-domain composite anomalies of "time abrupt changes + frequency distortion." In addition, reconstruction schemes based on autoencoders tend to converge to the identity shortcut of directly mapping the input to the output, severely weakening the distinguishability of reconstruction errors between abnormal and normal samples, and failing to support the stringent requirements of low false negatives and low false negatives in industrial and medical scenarios. In this embodiment, the input multivariate time series is processed by reversible instance normalization to obtain multivariate data; in the time domain, a bidirectional Mamba network is used to process the data independently according to natural time order and inverse dimension, capturing complementary time features of causal and anti-causal trends; in the frequency domain, after decomposing the time series into trends, the frequency features of seasonal components are extracted by discrete cosine transform, and then the global frequency interaction is modeled by a self-attention mechanism, and the data is reconstructed and aligned with the time domain features by inverse discrete cosine transform; the inconsistency between time features and frequency features is quantified by Kullback-Leible (KL) divergence, and a time series anomaly detection model is trained by a similarity loss function based on contrastive learning to obtain a trained time series model; finally, anomaly scores are generated by the time series anomaly detection model, and a hyperparameter δ is set to determine whether a certain time point is an anomaly.

[0031] The data in the time series samples can be sensor data from industrial production lines, physiological signal data output by medical devices, transaction behavior data in the financial field, or other data. This embodiment does not limit the type of data.

[0032] For example Figure 3 As shown, representational analysis based on UMAP visualizes the feature distributions in the time domain, frequency domain, and cross-domain difference space, revealing the fundamental limitations of single-domain methods in distinguishing normal and anomalous patterns. The fundamental properties of normal patterns across domains allow for the learning of transformation-invariant representations through contrastive alignment, while anomalous patterns inherently violate this invariance due to their domain-specific performance biases. Crucially, normal data points share a consistent feature space across both domains, while amplifying cross-domain differences when anomalous patterns disrupt the natural time-frequency correspondence. Single-domain projection reveals significant overlap between normal and anomalous clusters, confirming the inherent ambiguity of unimodal representations. The cross-domain difference space derived through time-frequency similarity scoring demonstrates superior separation trends, establishing an anomaly detection mechanism based on violations of the time-frequency consistency principle.

[0033] Specifically, in S2 of this embodiment, the process includes: inputting multivariate data into the time-domain representation module, independently inputting it into the Mamba network according to the natural time order and reverse time order dimensions to capture the forward and reverse time features of the time series, and then fusing these complementary features as time features. For example, S2 may include the following sub-steps (steps 2-1 to 2-4):

[0034] Step 2-1: Map the data to the latent space. The mapping operation is implemented using the following formula:

[0035] X t =Embedding(X)

[0036] Where X is the input data, Embedding(·) is the mapping operation, X t This is the mapped data.

[0037] Step 2-2: Process the time series using a Mamba network in natural chronological order to capture causal relationships, achieved through the following formula:

[0038] T forward =Mamba(X) t )

[0039] Among them, T forward This represents the forward temporal feature representation trained on a Mamba network, where Mamba(·) is the Mamba module, and X... t This refers to the input data.

[0040] Steps 2-3 involve reversing the time series along the time dimension and extracting the anti-causal pattern using a Mamba network, achieved through the following formula:

[0041] T backward =Mamba(Reverse(X) t))

[0042] Among them, T backward This is the reverse time feature representation trained on a Mamba network, where Mamba(·) represents the Mamba module, Reverse(·) represents the operation to reverse the time dimension, and X... t This refers to the input data.

[0043] Steps 2-4 involve adding these complementary features element-wise, and finally performing a dimensionality transformation on the fused features using a linear layer. This operation is implemented using the following formula:

[0044]

[0045] Among them, T forward T is the forward temporal feature representation trained on a Mamba network. backward This is the inverse temporal feature representation trained by the Mamba network, where Linear(·) is the linear layer and T is the fused temporal feature.

[0046] Furthermore, the Mamba module, as a state-space model, maps the input sequence to the output through the latent space, which is achieved through the following formula:

[0047]

[0048] y(t)=Ch(t)

[0049] Where A is the state matrix transformation parameter, B and C are the mapping parameters, x(t) is the input function, h(t) is the hidden space, y(t) is the output function, and t is the function independent variable.

[0050] The discrete form of SSM is discretized using the time-scale parameter Δ to transform the continuous parameters A, B, and C, and is achieved through the following formula:

[0051]

[0052] in, Let x be the discretized parameter, k represents the k-th sampling point after discretization, and x be the parameter. k Let y be the k-th value of the discretized input sequence. k h represents the k-th value of the output sequence after discretization. k Let A be the discretized hidden space at the k-th step, A be the state matrix transformation parameters, B and C be the mapping parameters, and I be the identity matrix. Discrete recurrent SSMs are practically untrainable due to their sequential nature. Since continuous convolutions are linear time-invariant systems, to obtain an effective representation, continuous convolutions are modeled as discrete convolutions. The previous step is vectorized into a convolution, achieved through the following formula:

[0053]

[0054] in, This represents a single convolution operation, where x0 is the first value of the discretized input sequence, and x... k Let be the k-th value of the discretized input sequence, x be the discretized input sequence, y be the discretized output sequence, and L be the length of the discretized input sequence.

[0055] The Mamba module uses a selection mechanism to parameterize the input of SSM. For different time features, there are unique and different B and C parameters, which enables Mamba to effectively solve the content-aware problem.

[0056] Specifically, in S3 of this embodiment, the steps include: inputting multivariate data into the frequency domain representation module, extracting seasonal variables, performing frequency analysis using discrete cosine transform and attention mechanism to extract frequency features, and then reconstructing the frequency features to the time domain using inverse discrete cosine transform. For example, S3 may include the following sub-steps (steps 3-1 to 3-4):

[0057] Step 3-1: Average pooling of the time series data reveals the trend variable and extracts the seasonal variable. This extraction is performed using the following formula:

[0058] X trend =AvgPool(X)

[0059] X seasonal =XX trend

[0060] Among them, X trend Let X be the trend variable, AvgPool(X) be the average pooling variable, and X be the input data. seasonal It is a seasonal variable.

[0061] Step 3-2: The seasonal variable is mapped to the frequency domain through discrete cosine transform, and then embedded in the latent space using the following formula:

[0062] F f =Embedding(DCT(X) seasonal ))

[0063] Among them, F f For frequency domain embedding features, Embedding(·) is the feature embedding operation, DCT(·) is the discrete cosine transform, and X... seasonal It is a seasonal variable.

[0064] Step 3-3: The interaction relationship between global frequency components is modeled using a self-attention mechanism, implemented through the following formula:

[0065] Q, K, V = Linear(F)f )

[0066]

[0067] Where Q, K, and V are the query, key, and value features, respectively, and Linear(·) is the linear transformation. f For frequency domain features, F′ is the frequency feature after attention optimization, Softmax(·) is the Softmax function, and d k For key feature dimensions.

[0068] Steps 3-4 involve reconstructing the frequency features back to the time domain using inverse discrete cosine transform. Finally, a linear layer completes the dimensional transformation of the frequency features, achieved through the following formula:

[0069] F = Linear(iDCT(F′))

[0070] Where F represents the frequency feature, iDCT(·) represents the inverse discrete cosine transform, F′ represents the attention-optimized frequency domain feature, and Linear(·) represents the linear transformation operation.

[0071] Furthermore, the cosine transform is a special variant within the framework of the Fourier transform. While the Fourier transform decomposes a function into sine and cosine components, the cosine transform focuses only on the cosine components and operates only on real numbers, making it more suitable for modern deep learning techniques. The discrete cosine transform converts time-domain signals (such as time series or image pixels) into a frequency-domain representation. The discrete cosine transform (DCT) for channel sequences is defined as follows:

[0072]

[0073] The inverse discrete cosine transform (IDCT) converts a frequency domain signal back to the time domain, and is defined as follows:

[0074]

[0075] Where, x n Represents a sequence The nth element, where L is the length of X, x k This represents the k-th frequency component of the DCT frequency domain coefficients.

[0076] Specifically, in S4 of this embodiment, the steps include: quantifying the inconsistency between time features and frequency features using Kullback-Leible (KL) divergence, and training a time series anomaly detection model using a similarity loss function based on contrastive learning to obtain a trained time series model. For example, S4 may include the following sub-steps (steps 4-1 to 4-4):

[0077] Step 4-1: Perform Softmax normalization on the time and frequency features dimension by dimension, using the following formula:

[0078] P time =softmax(T)

[0079] P freq =softmax(F)

[0080] Among them, P time P represents the probability distribution of time-domain features, softmax(·) is the softmax function, and T is the time-domain feature; freq Let F be the probability distribution of the frequency characteristics, and F be the frequency domain characteristics.

[0081] Step 4-2: Construct a bidirectional difference loss using the KL divergence framework to quantify the inherent inconsistency between the two types of features and obtain the KL divergence, achieved through the following formula:

[0082]

[0083] in, For two-way difference loss, For the desired operation, P time P is the probability distribution of time characteristics. freq Let be the probability distribution of the frequency characteristics, and ∈ be the numerical stability coefficient.

[0084] Step 4-3: Use this divergence as the loss function to drive model optimization, and alternately perform gradient separation operations between time-frequency branches during the training phase, as achieved by the following formula:

[0085]

[0086] in, The gradient of the time branch parameters, For the partial derivative operation of the time branch parameters, Let P be the Kullback-Leibler divergence. time P is the probability distribution of time characteristics. freq ·detach represents the frequency characteristic probability distribution after gradient truncation; The gradient of the frequency branch parameters, For the partial derivative operation of the frequency branch parameter, P freq P is the probability distribution of frequency characteristics. time ·detach represents the temporal probability distribution after gradient truncation.

[0087] Specifically, in S5 of this embodiment, the steps include: generating anomaly scores using a time series anomaly detection model, and setting a hyperparameter δ to determine whether a certain time point is an anomaly, achieved through the following formula:

[0088]

[0089] Among them, S t Let σ(·) be the anomaly score at time t, and let σ(·) be the activation function. Let P be the Kullback-Leibler divergence at time t. time P is the probability distribution of time characteristics. freq The probability distribution represents the frequency characteristics.

[0090] Based on the method flow of the above embodiments, this embodiment also provides a time-frequency dual-domain isolated time series anomaly detection system based on Mamba-self-attention, including:

[0091] The preprocessing module is configured to: acquire time series data and normalize the input multivariate time series data to obtain multivariate data.

[0092] The time-domain representation module is configured to independently input multivariate data into the Mamba network in natural time order and reverse time dimension order to capture the forward and reverse time features of the time series, and then fuse these complementary features as time features.

[0093] The frequency domain representation module is configured to: input multivariate data into the frequency domain representation module, extract seasonal variables, combine discrete cosine transform and attention mechanism to perform frequency analysis to extract frequency features, and then use inverse discrete cosine transform to reconstruct the frequency features to the time domain, so as to provide frequency features for the model;

[0094] The time-frequency difference module is configured to: measure the inconsistency between time features and frequency features using Kullback-Leible (KL) divergence, and train a time series anomaly detection model using a similarity loss function based on contrastive learning to obtain a trained time series model;

[0095] The anomaly detection module is configured to generate anomaly scores using a time series anomaly detection model and set hyperparameter δ to determine whether a certain time point is an anomaly.

[0096] Based on the method flow of the above embodiments, this embodiment also provides an electronic device, including a memory for storing computer program instructions and a processor for executing the program instructions. When the computer program instructions are executed by the processor, the electronic device is triggered to execute the time-frequency dual-domain isolation time series anomaly detection method based on Mamba-self-attention described in the embodiment. This invention provides a computer-readable storage medium, which includes a stored program. During program execution, the device containing the computer-readable storage medium is controlled to execute the time-frequency dual-domain isolation time series anomaly detection method based on Mamba-self-attention described in the embodiment. The electronic device may include a processor, a memory, and a communication unit. These components communicate through one or more buses. Those skilled in the art will understand that the structure of the electronic device does not constitute a limitation on the embodiments of this invention; it can be a bus topology, a star topology, or a combination of certain components, or different component arrangements. The communication unit is used to establish a communication channel, thereby enabling the electronic device to communicate with other devices. It receives user data sent by other devices or sends user data to other devices. The processor, serving as the control center of the electronic device, connects various parts of the device via interfaces and lines. It executes software programs and / or modules stored in the memory and retrieves data stored in the memory to perform various functions and / or process data. The processor can be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor may consist only of a central processing unit (CPU). In this embodiment, the CPU may have a single processing core or include multiple processing cores. The memory stores the processor's execution instructions. The memory can be implemented using any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. When the execution instructions in the memory are executed by the processor, the electronic device is able to perform some or all of the method steps in this embodiment.

[0097] The solution in this embodiment, based on Mamba-self-attention time-frequency dual-domain isolated time series anomaly detection, can be applied to most time series analysis scenarios, such as industrial manufacturing, medical equipment, and finance. This invention can detect anomalies based on the quantification of inherent time-frequency differences within the time series. Specifically, the process involves reversibly normalizing the input multivariate time series data to obtain multivariate data. This multivariate data is then input into a time-domain representation module, independently fed into a Mamba network in both natural and reverse time-dimensional order to capture the forward and reverse time features of the time series. These complementary features are then fused to form the time features. Next, the multivariate data is input into a frequency-domain representation module to extract seasonal variables. Frequency features are extracted using discrete cosine transform (DCT) and an attention mechanism, and then reconstructed in the time domain using inverse DCT. The time and frequency features are input into a time-frequency difference module, where the inconsistency between the time and frequency features is quantified using Kullback-Leible (KL) divergence. A time-series anomaly detection model is trained using a similarity loss function based on contrastive learning, resulting in a well-trained time-series model. Anomaly scores are generated using the time-series anomaly detection model, and a hyperparameter δ is set to determine whether a given time point is an anomaly. This invention, based on a time-frequency dual-domain architecture using Mamba and a self-attention mechanism, achieves state-of-the-art results while maintaining a lightweight design.

[0098] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The above descriptions are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for time-frequency dual-domain isolated time series anomaly detection based on Mamba-self-attention, characterized in that, include: S1. Extract multivariate time series data from business data as samples, and obtain multivariate data after reversible instance normalization. S2. Input the multivariate data in S1 into the time domain representation module, and input them into the Mamba network independently in natural time order and reverse time dimension order to capture the forward time features and reverse time features of the time series. Then, fuse these complementary features as time features. S3. Input the multivariate data in S1 into the frequency domain representation module, extract the seasonal variables, combine discrete cosine transform and attention mechanism to perform frequency analysis to extract frequency features, and then use inverse discrete cosine transform to reconstruct the frequency features to the time domain. S4. Input the time features and frequency features obtained from S2 and S3 into the time-frequency difference module, measure the inconsistency between time features and frequency features using Kullback-Leible (KL) divergence, and train the time series anomaly detection model using a similarity loss function based on contrastive learning to obtain the trained time series model. S5. The time series anomaly detection model obtained from S4 generates anomaly scores, and the hyperparameter δ is set to determine whether a certain time point is an anomaly.

2. The method according to claim 1, characterized in that, S1 includes: Extract L data samples at preset times from the business data and arrange them in the preset time sequence to obtain a time series sample, where L is a positive integer and L≥2, and the t-th time point contains M variables; The time series samples are normalized, including: X raw The original input time series sample, μ is the mean of the data, σ is the variance of the data, X is the standardized data, and the standardized set of data for multiple variables is used as the multivariate data.

3. The method according to claim 1 or 2, characterized in that, The types of business data include at least: industrial production line sensor data, physiological signal data output by medical devices, and transaction behavior data in the financial field.

4. The method according to claim 1, characterized in that, In S2, the time-domain representation module includes: mapping data to a latent space, employing a bidirectional processing strategy, using a Mamba network to process the time series in natural time order to capture causal relationships, then reversing the time series along the time dimension, using a Mamba network to extract anti-causal patterns, then adding these complementary features element-wise, and finally performing dimensional transformation on the fused features through a linear layer to provide time features for the model.

5. The method according to claim 1, characterized in that, In S3, the frequency domain representation module includes: after performing trend decomposition on the time series to obtain seasonal variables, mapping the seasonal variables to the frequency domain through discrete cosine transform, embedding them in the latent space, modeling the interaction relationship between global frequency components using a self-attention mechanism, reconstructing the frequency features back to the time domain through inverse discrete cosine transform, and finally completing the dimensional transformation of the frequency features by a linear layer to provide frequency features for the model.

6. The method according to claim 1, characterized in that, In S4, the time-frequency difference module includes: performing Softmax normalization on time features and frequency features in each dimension, constructing a bidirectional difference loss through the KL divergence framework, quantifying the inherent inconsistency between the two types of features to obtain KL divergence, using the divergence as a loss function to drive model optimization, and alternately performing gradient separation operations between the time-frequency branches during the training phase, finally obtaining a time series anomaly detection model.

Images