Abnormity detection method and system based on context residual learning

Abstract

The invention belongs to the technical field of anomaly detection, and provides an anomaly detection method and system based on context residual learning, and the method comprises the steps: obtaining detection data of a target domain, enabling the detection data to be a time sequence, carrying out the sliding segmentation according to a given window length, selecting a plurality of context prompt windows for each to-be-detected query window, obtaining a curvature perception basic residual error by using a pre-trained time sequence anomaly detection model, and determining abnormal data based on a comparison result of the curvature perception basic residual error and a set threshold value; during training, based on the time attention representation and the frequency domain attention representation, constructing context residuals of a time domain and a frequency domain; and training by using a multi-term loss function. By capturing the relative difference mode between the query window and the corresponding normal reference window, the method has high stability in a cross-domain scene.

Description

Technical Field

[0001] This invention belongs to the field of anomaly detection technology, specifically relating to an anomaly detection method and system based on contextual residual learning. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Temporal anomaly detection plays a crucial role in numerous real-world scenarios, encompassing industrial IoT devices, autonomous platforms, and large-scale service infrastructure. In these applications, there is a strong demand for a single detector capable of reliably operating across heterogeneous domains, enabling one-time pre-training and universal deployment without domain-specific adaptation. However, multivariate time series collected from different scenarios and environments often exhibit significant differences in sampling rate, channel dimension, noise characteristics, and temporal patterns (e.g., seasonality or frequency structure).

[0004] Existing Time Series Anomaly Detection (TSAD) methods encompass a wide range of paradigms, including density-based scoring, clustering and single-class classification, supervised label-driven models, reconstruction-based methods, and contrastive learning frameworks. Density and clustering methods heavily rely on local neighborhood structures or domain-specific feature distributions, making them highly sensitive to scale and noise shifts. Reconstruction-based models, whether statistical or deep, assume that normal patterns can be faithfully recovered through domain-specific encoder-decoder pairs, leading to learned representations that overfit to domain-specific temporal amplitude and frequency features. While contrastive learning variants reduce intra-domain noise, they still rely on a domain-consistent view or augmentation, which fails in heterogeneous sensor configurations. Therefore, although these methods perform well on single datasets, they struggle to transfer across domains with varying channel dimensions, sampling rates, noise mechanisms, and temporal dynamics, rendering them unsuitable for unified anomaly detection. Summary of the Invention

[0005] To address the aforementioned problems, this invention proposes an anomaly detection method and system based on contextual residual learning. This invention abandons the learning of absolute signal amplitude and instead uses a contextual learning paradigm to capture the relative difference pattern between the query window and its corresponding normal reference window, exhibiting high stability in cross-domain scenarios.

[0006] According to some embodiments, the present invention adopts the following technical solution: An anomaly detection method based on contextual residual learning includes the following steps: The detection data of the target domain is obtained. The detection data is a time series. It is slid segmented according to a given window length. For each query window to be detected, several context prompt windows are selected. The curvature-aware basic residual is obtained using a pre-trained time series anomaly detection model. Based on the comparison results of the curvature-aware basic residual and the set threshold, the abnormal data is identified. The training process of the time series anomaly detection model includes: Detection data from multiple domains is acquired, and the detection data is a time series, forming a training dataset set; The time series of each dataset in the set is slidably segmented according to a given window length to obtain a series of subsequence windows. The windows of each dataset are standardized and a channel mask is constructed for each window. The masked window is used to extract temporal attention representation using a temporal coding branch and frequency domain attention representation using a frequency coding branch. Based on the temporal and frequency domain attention representations, context residuals in the temporal and frequency domains are constructed. The second-order residual of the context residual is calculated. The curvature residual trajectory is obtained by fusing and normalizing the second-order residuals in the time domain and frequency domain. The absolute value of the first-order residual is averaged to construct the basic amplitude residual. The curvature residual trajectory and the basic amplitude residual are then normalized and weighted by interpolation to obtain the curvature-aware basic residual. Training is performed using the task loss, which constrains the difference between the attention distribution in the time and frequency domains and the mask target, and the second-order consistency loss, which constrains the normalized shape of the curvature residual to remain consistent with the basic residual, as loss functions.

[0007] As an alternative implementation, the windows of each dataset are standardized, and the process of constructing a channel mask for each window includes: recording the maximum number of channels in all datasets. And the windows of each dataset are uniformly expanded to the maximum value in the channel dimension. For non-existent channel positions, fill with 0 to construct a channel mask for each window. ,in Channels with a value of 1 represent true, valid channels, while channels with a value of 0 represent filler channels.

[0008] As an alternative implementation, the process of extracting temporal attention representations using a time-domain coding branch and extracting frequency-domain attention representations using a frequency-domain coding branch includes: sending the masked windows to the time-domain coding branch and the frequency-domain coding branch respectively, wherein the masked windows include a query window and one or more contextual prompt windows selected from normal data in the same dataset; the time-domain coding branch extracts a temporal attention representation mainly based on local fluctuations from the masked windows, and the frequency-domain coding branch extracts a frequency-domain attention representation mainly based on smooth trends and periodic structures; each attention representation carries a channel mask.

[0009] As an alternative implementation, the process of constructing the context residuals in the time domain and frequency domain based on the time domain attention representation and the frequency domain attention representation includes: introducing lightweight window-level adapters on the time domain coding branch and the frequency domain coding branch respectively, for mapping the extracted attention representations to one-dimensional trajectories; The time domain processing result of the query window is as follows: The time domain processing result of the context prompt window is The frequency domain processing result of the query window is The frequency domain processing result of the context prompt window is Calculate the time-domain context residual Calculate the frequency domain context residual ; The results obtained and It is a time-varying first-order context residual trajectory used to reflect the degree of deviation of the current window from the normal reference window from both time and frequency perspectives.

[0010] As an alternative implementation, the process of calculating the second-order residual of the context residual and obtaining the curvature residual trajectory by fusing and normalizing the second-order residuals in the time and frequency domains includes: [The text abruptly ends here, likely due to an incomplete translation or a missing section.] With frequency domain context residual Treating it as a discrete-time curve, the second-order residual is calculated using the central second-order difference operator. By fusing and normalizing the second-order residuals in the time and frequency domains, the curvature residual trajectory is obtained. ; The fundamental magnitude residuals are constructed by averaging the absolute values ​​of the first-order residuals. Then, the basic amplitude residuals and curvature residual trajectory Normalization and weighted interpolation are performed to obtain the basic residuals for curvature sensing. It is used to reflect amplitude deviation and includes information on curvature changes.

[0011] As an alternative implementation, the time series anomaly detection model further includes a mask-aware low-rank feature trajectory projection module, which is used to project the trajectory in a unified channel dimension. Below, multiple windows are sampled from multi-domain data to form a matrix. The matrix is ​​centered, and the mean vector is denoted as μ. Singular value decomposition is performed on the centered matrix, and the first K principal components whose cumulative explained variance reaches the preset threshold are selected to form the basis matrix. For any new window vector x, calculate its projection into the subspace. Reconstruction Obtain the window-level residual vector By combining the channel mask M with a weighted average only on the effective channels, the low-rank residual trajectory is obtained. This serves as a teacher signal to guide the model residuals during training.

[0012] As an alternative implementation, the process of using a task loss to constrain the differences between the time-domain and frequency-domain attention distributions and the mask target, and a second-order consistency loss to constrain the normalized shape of the curvature residuals to remain consistent with the fundamental residuals, as loss functions includes: employing a task loss based on Kullback–Leibler divergence. By constraining the differences between the attention distribution in the time domain and the mask target in the frequency domain, the time series anomaly detection model is guided to focus on more critical time points and channels; By constructing a second-order consistency loss The constrained curvature residuals maintain a consistent normalized shape with the basic residuals, enabling the time series anomaly detection model to form a stable correspondence on the first-order and second-order geometric information. Constituting the total loss Joint pre-training is performed on a multi-domain training dataset to obtain a trained time series anomaly detection model.

[0013] As a further implementation, the total loss also includes a low-rank consistency loss. Low-rank consistency loss The underlying residual trajectory output by the time series anomaly detection model. With the generated residual trajectory Alignment, suppressing instability caused by noise and inter-domain scale differences, total loss , These are the weighting coefficients.

[0014] As an alternative implementation, the process of determining anomalous data by using a pre-trained time-series anomaly detection model to obtain curvature-aware basic residuals and comparing these residuals with a set threshold includes: obtaining curvature-aware basic residuals using a pre-trained time-series anomaly detection model. Combined with low-rank residuals The two are linearly fused according to the weight parameters to obtain the final anomaly score trajectory. The threshold value is determined based on the preset anomaly ratio or the estimated quantile. For each time point Identify abnormal score trajectories Does it exceed the set threshold? If the time exceeds the limit, an exception label will be assigned to the corresponding time point.

[0015] An anomaly detection system based on contextual residual learning includes: The anomaly detection module is configured to acquire detection data in the target domain. The detection data is a time series and is slid segmented according to a given window length. For each query window to be detected, several context prompt windows are selected. Using a pre-trained time series anomaly detection model, the curvature-aware basic residual is obtained. Based on the comparison result between the curvature-aware basic residual and a set threshold, the abnormal data is identified. The model training module is configured to acquire detection data from multiple domains, wherein the detection data is a time series and constitutes a training dataset set. The time series of each dataset in the set is slidably segmented according to a given window length to obtain a series of subsequence windows. The windows of each dataset are standardized and a channel mask is constructed for each window. The masked window is used to extract temporal attention representation using a temporal coding branch and frequency domain attention representation using a frequency coding branch. Based on the temporal and frequency domain attention representations, context residuals in the temporal and frequency domains are constructed. The second-order residual of the context residual is calculated. The curvature residual trajectory is obtained by fusing and normalizing the second-order residuals in the time domain and frequency domain. The absolute value of the first-order residual is averaged to construct the basic amplitude residual. The curvature residual trajectory and the basic amplitude residual are then normalized and weighted by interpolation to obtain the curvature-aware basic residual. Training is performed using the task loss, which constrains the difference between the attention distribution in the time and frequency domains and the mask target, and the second-order consistency loss, which constrains the normalized shape of the curvature residual to remain consistent with the basic residual, as loss functions.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention proposes a method for judging anomalies in target domain detection data based on a unified anomaly detection model. By training multi-domain data and combining three complementary forms of residual evidence to construct domain-invariant representations, a unified residual learning model is formed through collaborative work. This model can jointly suppress variability in specific domains while preserving the intrinsic characteristics of anomalous behavior.

[0017] This invention introduces a contextual residual formula that conditions each input window with a reference normal window, enabling the model to measure anomalies through contextual differences rather than absolute magnitudes. This contextual conditioning suppresses domain-dependent scale, bias, and trend effects, and can generate stable difference signals even when the channel layout or sampling features differ between datasets.

[0018] To obtain a concept of robust normality that remains consistent across domains, this invention introduces a mask-aware low-rank feature trajectory (EigenTrajectory, ET) teacher module. By capturing the dominant temporal structure shared between heterogeneous datasets through low-rank priors, it filters out sensor-dependent noise patterns. The ET residuals serve as both pseudo-supervision during training and auxiliary cues during inference, guiding the model toward a domain-independent residual space.

[0019] This invention enhances characterization by introducing a curvature-aware second-order residual module. While first-order differences primarily capture amplitude shifts, second-order differences provide geometric cues such as inflection points, accelerations, and abrupt local deformations. These cues are much less sensitive to domain-specific scale, level, or frequency variations. These shape-based features carry strong anomaly definition information and can generalize well across different sensors and environments.

[0020] In summary, this invention compares each input window with a reference normal window using context learning, generating domain-invariant difference patterns that can generalize across heterogeneous datasets. It also enables noise-resistant residual supervision and serves as an auxiliary signal during inference, aligning the learned residual space with domain-independent normal behavior. Furthermore, it can capture geometric deformation cues that remain stable under scale, noise, and temporal frequency variations, thereby achieving robust cross-domain anomaly representation.

[0021] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0022] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0023] Figure 1 This is a schematic diagram of an anomaly detection process in one embodiment; Figure 2This is a visualization of anomaly scores on a test dataset in one embodiment, where the blue line represents the anomaly scores obtained by the model, and the pink area marks the anomaly intervals. (a) is a schematic diagram of the anomaly scores of MSL, (b) is a schematic diagram of the anomaly scores of SWaT, (c) is a schematic diagram of the anomaly scores of SMD, and (d) is a schematic diagram of the anomaly scores of SMAP. Figure 3 This is an ET visualization result on the MSL dataset in one embodiment; Figure 4 This is a visualization of the impact of outlier scores in one embodiment; Figure 5 This is a schematic diagram of unified anomaly detection results on a real-world dataset in one embodiment; Figure 6 This is a schematic diagram of the combined ablation experiment results on three datasets in one embodiment. Detailed Implementation

[0024] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0025] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0026] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0027] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0028] Example 1 An anomaly detection method based on contextual residual learning includes the following steps: The detection data of the target domain is obtained. The detection data is a time series. It is slid segmented according to a given window length. For each query window to be detected, several context prompt windows are selected. The curvature-aware basic residual is obtained using a pre-trained time series anomaly detection model. Based on the comparison results of the curvature-aware basic residual and the set threshold, the abnormal data is identified. Among them, such as Figure 1 As shown, the training process of the time series anomaly detection model includes: Detection data from multiple domains is acquired, and the detection data is a time series, forming a training dataset set; The time series of each dataset in the set is slidably segmented according to a given window length to obtain a series of subsequence windows. The windows of each dataset are standardized and a channel mask is constructed for each window. The masked window is used to extract temporal attention representation using a temporal coding branch and frequency domain attention representation using a frequency coding branch. Based on the temporal and frequency domain attention representations, context residuals in the temporal and frequency domains are constructed. The second-order residual of the context residual is calculated. The curvature residual trajectory is obtained by fusing and normalizing the second-order residuals in the time domain and frequency domain. The absolute value of the first-order residual is averaged to construct the basic amplitude residual. The curvature residual trajectory and the basic amplitude residual are then normalized and weighted by interpolation to obtain the curvature-aware basic residual. Training is performed using the task loss, which constrains the difference between the attention distribution in the time and frequency domains and the mask target, and the second-order consistency loss, which constrains the normalized shape of the curvature residual to remain consistent with the basic residual, as loss functions.

[0029] To make the solution provided in this embodiment clearer to those skilled in the art, the specific application process described above, namely, acquiring the detection data of the target domain, the detection data being a time series, sliding segmentation according to a given window length, selecting several context prompt windows for each query window to be detected, using a pre-trained time series anomaly detection model to obtain curvature-aware basic residuals, and determining abnormal data based on the comparison results of curvature-aware basic residuals and a set threshold, is described as the model inference process.

[0030] The following is a detailed description.

[0031] First, this embodiment provides a unified time series anomaly detection method based on in-context residual learning. It abandons the learning of absolute signal amplitude and instead uses the context learning paradigm to capture the relative difference pattern between the query window and its corresponding normal reference window (Prompt). This difference pattern has high stability in cross-domain scenarios.

[0032] To address the issue of inconsistent channel counts in heterogeneous datasets, this invention first designs a mask-aware channel unification mechanism. This mechanism extends all input sequences to a unified target dimension and introduces a channel mask matrix to label valid variables. This channel mask is used throughout the entire process of attention calculation, feature aggregation, and residual calculation, ensuring that the model can effectively handle inputs of any dimension while preventing false correlations introduced by zero-padding from polluting the domain-invariant representation.

[0033] Specifically, the first step is to construct a multi-domain pre-trained dataset set. The i-th dataset is denoted as . in, Indicates the length of time. Let N represent the number of channels, where N is the total number of pre-training datasets. For each dataset, the original sequence is slidably divided according to a given window length W to obtain a series of sub-sequence windows. The maximum number of channels in all datasets is denoted as . And the windows of each dataset are uniformly expanded along the channel dimension. Non-existent channel positions are filled with 0s, and a channel mask is constructed for each window. In M, channels with a value of 1 represent true valid channels, and channels with a value of 0 represent filler channels.

[0034] To ensure that those skilled in the art fully understand the solutions provided by this invention, the description focuses on industrial IoT devices and power detection as the multiple domains provided in this embodiment. A set of pre-trained datasets for the industrial IoT devices, power detection, and smart sensing domains is obtained. The pre-trained dataset for the industrial IoT devices domain can be production line process detection data, such as time-series data of pressure, temperature, flow rate, air pressure, stress, and intensity. The pre-trained dataset for the power detection domain can be time-series data of voltage, current, and temperature. Of course, in other embodiments, the data in the above-mentioned training datasets can be flexibly adjusted according to the application domain.

[0035] Based on a unified channel representation, this embodiment addresses each query window to be detected. Select one or more context prompt windows from normal data in the same dataset. W is the window length, C is the channel dimension, and both are processed in a unified channel expansion manner as described above.

[0036] Then, channel-by-channel filtering is performed using a mask M. During the encoding stage, this embodiment sends the masked windows to the time-domain encoding branch and the frequency-domain encoding branch, respectively. The time-domain encoding branch (also known as the time-domain encoder) extracts a time-attention representation dominated by local fluctuations from the query window X and the context prompt window P. The frequency-domain encoding branch (also known as the frequency-domain encoder) extracts a frequency-domain attention representation dominated by smooth trends and periodic structures through frequency-domain transformation and other methods. The attention tensors mentioned above all carry the channel mask M in subsequent calculations to ensure that the filler value channels do not interfere with the attention distribution.

[0037] Based on this, this embodiment adopts a dual-domain context residual modeling strategy. The time series anomaly detection model (hereinafter referred to as the model) does not directly store the absolute form of normal data, but instead uses a time-domain encoder and a frequency-domain encoder in parallel to extract features of the input window and the reference prompt window in terms of local fluctuations and long-term trends, respectively. By calculating the difference between the query features and the prompt features, context residuals are generated.

[0038] To obtain scalar residual trajectories usable for cross-domain comparisons, this embodiment introduces lightweight window-level adapters in both the time and frequency domain branches, mapping the attention tensor to a one-dimensional trajectory. That is, the time-domain processing result of the query window is... The time domain processing result of the context prompt window is The frequency domain processing result of the query window is The frequency domain processing result of the context prompt window is Calculate the time-domain context residual Calculate the frequency domain context residual ; The results obtained and It is a time-varying first-order context residual trajectory, reflecting the degree of deviation of the current window from the normal reference window from both time and frequency perspectives. Because this residual is based on relative differences rather than absolute amplitudes, it is more stable across different sampling rates, channel configurations, and noise levels.

[0039] Furthermore, to capture geometric cues insensitive to scale changes, this embodiment also incorporates a curvature-aware second-order residual module. This module extracts shape features such as inflection points and accelerations by calculating the second-order central difference of the residual trajectory. Specifically, it... and Treating it as a discrete-time curve, the second-order residual is calculated using the central second-order difference operator. By fusing and normalizing the second-order residuals in the time and frequency domains, the curvature residual trajectory is obtained. Simultaneously, the absolute values ​​of the first-order residuals are averaged to construct the fundamental amplitude residuals. And then and Normalization and weighted interpolation are performed to obtain the basic residuals for curvature sensing. It reflects both amplitude deviation and curvature change information, and can produce a sharper response near anomaly points, which is beneficial for accurate boundary positioning.

[0040] To further enhance robustness, this embodiment introduces a mask-aware low-rank feature trajectory (EigenTrajectory, ET) projection module as the teacher signal. A cross-domain shared low-rank subspace is constructed using singular value decomposition, the input is projected and reconstructed, and the reconstruction residual is used to suppress sensor-specific noise.

[0041] Specifically, in the unified channel dimension Below, a matrix is ​​formed by sampling a large number of windows from multi-domain data. The matrix is ​​centered, with the mean vector denoted as μ. Singular value decomposition is then performed on the centered matrix, and the top K principal components whose cumulative explained variance reaches a preset threshold are selected to form the basis matrix. For any new window vector... First, calculate its projection in the subspace. Reconstruct the vector , thus obtaining the window-level residual vector By combining the channel mask M with a weighted average only on the effective channels, a low-rank residual trajectory is obtained. This trajectory mainly reflects the low-frequency trend of cross-domain sharing and is not sensitive to small fluctuations in single-domain noise. It can be used as a teacher signal to guide the model residuals during training and fused with the model output during the inference stage.

[0042] The above process achieves accurate anomaly localization by fusing first-order fundamental residuals, second-order geometric residuals, and ET noise-resistant residuals.

[0043] During model training, this embodiment uses joint optimization of multiple loss functions to ensure that the information in different residual spaces remains consistent in both numerical value and shape.

[0044] On the one hand, a task loss based on Kullback–Leibler divergence is adopted. By constraining the differences between the attention distribution in the time and frequency domains and the masking target, the model is guided to focus on more critical time points and channels; on the other hand, a second-order consistency loss is constructed. The constrained curvature residuals maintain a consistent normalized shape with the basic residuals, enabling the model to form a stable correspondence on the first-order and second-order geometric information.

[0045] In some embodiments, low-rank consistency loss is also used. The basic residual trajectory output by the model The residual trajectory generated by the mask-aware low-rank feature trajectory (EigenTrajectory, ET) projection module Alignment helps suppress instability caused by noise and inter-domain scale differences.

[0046] In this embodiment, the total loss is calculated by combining all constraints. , For weight coefficients, in a multi-domain pre-trained dataset set Joint pre-training is performed on the model to obtain a unified time series anomaly detection model.

[0047] During the inference phase, this embodiment does not require retraining on the target domain.

[0048] Given a target domain time series, windows are segmented and context-sensitive windows are selected in the same manner as during the training phase. These windows are then input into the pre-trained model to obtain the curvature-aware basic residuals. Optionally, some embodiments may further obtain low-rank residuals by combining the constructed mask-aware low-rank feature trajectory subspace. The two are linearly fused according to the weight parameters to obtain the final anomaly score trajectory. .

[0049] The threshold is determined based on a preset anomaly ratio or an estimated quantile on the validation set. For each time point judge Does it exceed This will output the anomaly label for that time point.

[0050] Because the model relies on the residual pattern between the query window and the context window, rather than the absolute signal pattern in a fixed domain, it can still maintain good uniform detection performance across various time series datasets with significant variations in the number of channels, sampling rate, and noise statistics.

[0051] To verify the solution provided in this embodiment, an experiment was conducted.

[0052] The experimental configuration conditions for this embodiment are as follows: the Transformer structure based on masked autoencoder is constructed, and the original single dataset single model paradigm is transformed into a unified time series anomaly detection model (hereinafter referred to as TSAD) by joint pre-training on heterogeneous time series corpora.

[0053] All sequences are preprocessed using the mask-aware channel unified method proposed in this embodiment, and fixed-length windows are sampled according to a unified protocol. During the inference phase, anomaly scores are obtained by aggregating the residual trajectories of overlapping windows, and a threshold is then selected according to the evaluation protocol. The entire framework does not contain any domain-specific modules; a unified model only needs to be pre-trained once and can be directly applied to all evaluation datasets without fine-tuning.

[0054] Following a unified pre-training protocol, a diverse multi-domain corpus was constructed. The pre-training set includes a subset of the Monash repository (processed with synthetic anomaly injection) and 11 real-world anomaly detection datasets: the American Heart Association dataset (AHA), the EXATHLON high-dimensional time series benchmark for interpretable anomaly detection, the AIOps multi-task / multi-scenario dataset (GAIA), the synthetic chaotic sequence anomaly benchmark (MGAB), the MIT-BIH Arrhythmia Database (MITDB), the NYC taxi passenger flow dataset (NYCTaxi), the Ozone Monitoring Instrument (OMI), the Opportunity dataset for human activity recognition, the SKAB benchmark for industrial installations / sensors anomalies, and the MIT-BIH Supraventricular Arrhythmia Database (SVDB). These datasets cover multiple domains, including industrial control, physiological signals, human activity, environmental sensing, and network / server logs.

[0055] In the evaluation phase, following the practices of previous unified TSAD studies, five widely used benchmark datasets were employed: Server Machine Dataset (SMD), Mars Science Laboratory (MSL), Soil Moisture Active Passive (SMAP), Secure Water Treatment (SWaT), and Pooled Server Metrics (PSM). Importantly, none of these datasets were included in the pre-training pool, ensuring a rigorous cross-domain generalization setting.

[0056] Recent studies have shown that point adjustment (PA) can lead to erroneous performance evaluations. Therefore, this widely used method, which marks an entire outlier segment as correct if any single point is detected, has been abandoned, thus exaggerating performance. Instead, we employ association-based F1 scores (A_F1), calculating the directed distance between predicted and actual outliers to evaluate precision (A_P) and recall (A_R). This metric penalizes both oversegmentation and misalignment and has become a recognized reliable evaluation standard in the TSAD field.

[0057] We compare the unified model with 15 representative methods, covering reconstructional, probabilistic, and Transformer paradigms: Autoencoder (AE), Deep Autoencoding Gaussian Mixture Model (DAGMM), Long Short-Term Memory (LSTM), Convolutional Autoencoder Ensemble (CAE-Ensemble), Generative Adversarial Network Temporal Anomaly Detection (BeatGAN), Robust Multivariate Temporal Anomaly Detection (OmniAnomaly), Anomaly Transformer (AT), Memory-guided Transformer (MEMTO), Dual-attention Contrastive Detector (DCdetector), Dynamic Decomposition with Diffusion Reconstruction (D3R), Pre-trained Language Model-Driven Temporal Series Methods (GPT4TS), and Modern Temporal Convolutional Networks (Modern). The methods include Temporal Convolutional Network (ModernTCN), an anomaly detection method that enhances sensitivity to normal patterns (SensitiveHUE), the Temporal-Frequency Masked Autoencoder (TFMAE), and the General Time Series Anomaly Detector (DADA). All methods are evaluated using a unified benchmarking pipeline: running on the same sliding window, generating anomaly scores at the same timestamps, and using the same post-processing and threshold selection procedures. All reconstructive and probabilistic benchmark models are independently retrained on each target dataset according to their recommended configurations. For DADA, since its exact pre-training data, preprocessing pipeline, and pre-training weights are not publicly available, we use its reported results as a unified TSAD benchmark.

[0058] Key experimental results: On five heterogeneous benchmark datasets, our unified model is compared with the high-performance TSAD benchmark model for the target domain. Despite not undergoing any target domain training, our method consistently achieves competitive or better results, demonstrating its strong cross-domain generalization ability.

[0059] Quantitative performance comparison: Figure 5 The proposed solution provides association precision (A_P), association recall (A_R), and association F1 score (A_F1) on five standard benchmark datasets (SMD, MSL, SMAP, SWAT, and PSM). Despite not being trained on the target domain, the solution presented in this embodiment achieves competitive or better results on all datasets. In particular, the solution achieves the highest macro-average F1 score of 80.29%, surpassing the previous state-of-the-art model DADA by 1.22 percentage points. The solution also sets new best recall scores on all datasets (SMD 98.35%, MSL 97.02%, SMAP 99.65%, SWAT 97.24%, PSM 95.47%), demonstrating its high sensitivity to anomaly boundaries and variations. These results validate the effectiveness of the solution presented in this embodiment: curvature-aware second-order residuals enhance early anomaly response and boundary localization capabilities, while the low-rank teacher of the EigenTrajectory (ET) suppresses false positives by filtering noise and domain-specific biases. These components work together to enable the unified model to narrow, and in some cases even surpass, the performance gap with domain-specific TSAD models.

[0060] Qualitative performance comparison: Figure 2 Examples of anomaly scores on test sequences from four datasets—MSL, SWAT, SMD, and SMAP—are presented. The red curve represents the predicted anomaly score, the blue area represents the true anomaly interval, and the dashed line represents the decision threshold. On all four datasets, scores are significantly higher within most anomaly intervals, while scores for most normal segments remain below the threshold. On MSL and SMAP, the solution presented in this embodiment not only highlights long anomaly segments but also responds quickly to short transient anomalies. On the anomaly-dense and highly correlated SWAT dataset, the solution presented in this embodiment consistently delivers high scores throughout the attack, demonstrating its robustness to rapid state changes.

[0061] Comprehensive ablation experiments were also conducted to evaluate the role of each core component in the unified anomaly detector. The results are summarized as follows: Figure 6 As shown, the visualization analysis is as follows: Figure 3 and Figure 4 As shown.

[0062] The Role of Contextual Residual Modeling: To evaluate the advantages of contextual residual modeling, we compare the full model with variants that remove the prompt or disable residual differencing. Figure 6As shown in the first group, removing contextual cues leads to a consistent performance degradation on the SMD, MSL, and SMAP datasets. This confirms that comparing the window to its reference normal cue provides a domain-invariant residual signal.

[0063] The purpose of mask-aware channel uniformity: Since datasets in a uniform setting have different numbers of channels, zero-padding is required to match dimensions. However, zero-padding can introduce misleading attention relevance. Figure 6 As shown, enabling the channel mask provided in this embodiment significantly improves performance by preventing invalid dimensions from affecting the representation. The mask reduces false alarms and stabilizes threshold selection, making it a key component for learning from heterogeneous domains.

[0064] The role of Feature Trajectories (ET): To evaluate the model performance with and without low-rank teachers with ET. For example... Figure 6 As shown in the second and third lines, removing ET simultaneously reduces both precision and F1 score, demonstrating its effectiveness in suppressing false positives caused by noise. Figure 3 The cumulative explained variance curve was plotted, and the principal component dimension K required to reach the variance retention threshold (var_keep) was marked. Figure 4 The ET residual trajectory is shown, which maintains an extremely low noise floor in the normal segment while producing significant peaks at the anomaly. These results collectively demonstrate that ET provides a stable global normal subspace and enhances the separability of anomalies.

[0065] The role of ET in the training process: Further research was conducted on the effects of using ET only during inference or also during training. Figure 6 The results show that applying the ET consistency loss during training brings significant performance improvement, confirming that ET provides a useful supervisory signal that aligns the predicted residuals with the low-rank normal manifold.

[0066] Visualization: To further illustrate the effectiveness of each module in our method, a 70-second time period was randomly selected for visualization. Figure 4 This diagram shows the anomaly scores and true labels (pink areas) for each timestamp within the MSL dataset for this time period. Basic is derived from window-level anomaly scores; SO (second-order) represents the anomaly scores when only second-order calculations are performed; ET is the score calculated from the reconstruction residuals obtained through window-level low-rank principal component analysis; and Fused is the fusion result of the three. It can be seen that: SO approaches zero in stationary segments, forms peaks at anomalous abrupt changes, and is sensitive to anomaly boundaries; ET has a lower noise floor and denser peaks, making it more sensitive to global bias; Fused achieves a better balance between peak amplitude and noise floor, further improving the alignment between anomaly peaks and true anomaly regions.

[0067] Example 2 An anomaly detection method based on contextual residual learning includes the following steps: Data preprocessing and window construction: The original time series in each source dataset are first standardized (e.g., z-score normalization), and then segmented according to the preset window length and sliding step size to obtain a series of fixed-length sliding windows; the number of channels in each dataset is recorded, all windows are uniformly expanded to the maximum number of channels in the channel dimension, non-existent channels are filled with zero values, and a corresponding channel mask is generated for each window to mark the valid channels and the filled channels.

[0068] Contextual cue sampling: During the unified pre-training phase, for each query window to be detected, one or more windows are randomly selected from the normal window set of the same dataset or selected according to a strategy as contextual cue windows. This makes the query window and the cue window representative and diverse in terms of time, working conditions, etc., so that the model can learn stable contextual residual patterns.

[0069] Time-frequency dual-domain encoding: The query window and context prompt window are input into the feature extraction backbone network. The backbone network includes a time domain encoding branch and a frequency domain encoding branch. The time domain branch is used to capture short-term fluctuations and local dependencies, while the frequency domain branch is used to characterize periodic structures and smooth trends. During the encoding process, a channel mask is introduced to suppress the influence of filling channels. Finally, one-dimensional feature trajectories that change over time are obtained on the two branches respectively.

[0070] Context residual and curvature residual calculation: Based on the one-dimensional feature trajectory in the time domain and frequency domain, the difference between the query window and the context prompt window is calculated to obtain the first-order context residual, which is used to represent the degree of deviation of the current window from the normal reference window. On this basis, a second-order difference operation is applied to the residual trajectory to construct the curvature residual. The amplitude residual and the curvature residual are normalized and weighted and fused to obtain the curvature-aware basic residual that reflects both the deviation magnitude and is sensitive to the turning point.

[0071] Mask-aware low-rank feature trajectory (EigenTrajectory, ET) subspace construction and update: During multi-domain pre-training, windows that have undergone unified channel processing are periodically sampled from each dataset, expanded into vectors to form a sample matrix, and singular value decomposition is performed on the sample matrix. Principal components whose cumulative explained variance reaches a set threshold are selected to form a low-rank temporal subspace. For any window, projection and reconstruction are first performed in this subspace, and then the error between the original window and the reconstructed window is calculated. Combined with the channel mask, the error is aggregated on the effective channels to form a noise-robust low-rank residual trajectory, which serves as a teacher signal or auxiliary score.

[0072] Joint loss training: During the model training phase, a joint loss function is constructed, which includes the attention distribution task loss, the consistency loss between the basic residual and the curvature residual, and the consistency loss between the basic residual and the low-rank residual. The total loss is formed by weighted summation, and the backbone network and related modules are optimized end-to-end. The optimization algorithm can adopt the existing common gradient descent method until it converges on the validation set, thereby obtaining a unified time series anomaly detection model.

[0073] Unified inference process: In the inference stage, the target dataset that was not used in training is directly standardized and windowed in the same way, and a small number of normal windows are selected as contextual prompts without retraining the model. The query window and the prompt window are input into the trained model to obtain the curvature-aware basic residual and the optional low-rank residual. The residuals are fused to obtain the final anomaly score. The scores of the overlapping windows are aligned and summarized by time. A unified threshold is set according to the preset anomaly ratio or empirical quantile, and the anomaly detection results of each time point of the target time series can be output.

[0074] Example 3 An anomaly detection system based on contextual residual learning includes: The anomaly detection module is configured to acquire detection data in the target domain. The detection data is a time series and is slid segmented according to a given window length. For each query window to be detected, several context prompt windows are selected. Using a pre-trained time series anomaly detection model, the curvature-aware basic residual is obtained. Based on the comparison result between the curvature-aware basic residual and a set threshold, the abnormal data is identified. The model training module is configured to acquire detection data from multiple domains, wherein the detection data is a time series and constitutes a training dataset set. The time series of each dataset in the set is slidably segmented according to a given window length to obtain a series of subsequence windows. The windows of each dataset are standardized and a channel mask is constructed for each window. The masked window is used to extract temporal attention representation using a temporal coding branch and frequency domain attention representation using a frequency coding branch. Based on the temporal and frequency domain attention representations, context residuals in the temporal and frequency domains are constructed. The second-order residual of the context residual is calculated. The curvature residual trajectory is obtained by fusing and normalizing the second-order residuals in the time domain and frequency domain. The absolute value of the first-order residual is averaged to construct the basic amplitude residual. The curvature residual trajectory and the basic amplitude residual are then normalized and weighted by interpolation to obtain the curvature-aware basic residual. Training is performed using the task loss, which constrains the difference between the attention distribution in the time and frequency domains and the mask target, and the second-order consistency loss, which constrains the normalized shape of the curvature residual to remain consistent with the basic residual, as loss functions.

[0075] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of one or more computer-usable storage media (including, but not limited to, disk storage, etc.) containing computer-usable program code. CD - ROM It takes the form of a computer program product implemented on (such as optical memory, etc.).

[0076] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0077] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0078] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0079] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made by those skilled in the art without creative effort within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An anomaly detection method based on contextual residual learning, characterized in that, Includes the following steps: The detection data of the target domain is obtained. The detection data is a time series. It is slid segmented according to a given window length. For each query window to be detected, several context prompt windows are selected. The curvature-aware basic residual is obtained using a pre-trained time series anomaly detection model. Based on the comparison results of the curvature-aware basic residual and the set threshold, the abnormal data is identified. The training process of the time series anomaly detection model includes: Detection data from multiple domains is acquired, and the detection data is a time series, forming a training dataset set; The time series of each dataset in the set is slidably segmented according to a given window length to obtain a series of subsequence windows. The windows of each dataset are standardized and a channel mask is constructed for each window. The masked window is used to extract temporal attention representation using a temporal coding branch and frequency domain attention representation using a frequency coding branch. Based on the temporal and frequency domain attention representations, context residuals in the temporal and frequency domains are constructed. The second-order residual of the context residual is calculated. The curvature residual trajectory is obtained by fusing and normalizing the second-order residuals in the time domain and frequency domain. The absolute value of the first-order residual is averaged to construct the basic amplitude residual. The curvature residual trajectory and the basic amplitude residual are then normalized and weighted by interpolation to obtain the curvature-aware basic residual. Training is performed using the task loss, which constrains the difference between the attention distribution in the time and frequency domains and the mask target, and the second-order consistency loss, which constrains the normalized shape of the curvature residual to remain consistent with the basic residual, as loss functions.

2. The anomaly detection method based on contextual residual learning as described in claim 1, characterized in that, The process of standardizing the windows of each dataset and constructing a channel mask for each window includes: recording the maximum number of channels across all datasets. And the windows of each dataset are uniformly expanded to the maximum value in the channel dimension. For non-existent channel positions, fill with 0 to construct a channel mask for each window. ,in Channels with a value of 1 represent true, valid channels, while channels with a value of 0 represent filler channels.

3. The anomaly detection method based on contextual residual learning as described in claim 1, characterized in that, The process of extracting temporal attention representations using the time-domain coding branch and extracting frequency-domain attention representations using the frequency-domain coding branch includes: feeding the masked windows into the time-domain coding branch and the frequency-domain coding branch respectively. The masked windows include a query window and one or more contextual prompt windows selected from normal data in the same dataset. The time-domain coding branch extracts a temporal attention representation based on local fluctuations from the masked windows, while the frequency-domain coding branch extracts a frequency-domain attention representation based on smooth trends and periodic structures. Each attention representation carries a channel mask.

4. The anomaly detection method based on contextual residual learning as described in claim 1, characterized in that, The process of constructing the temporal and frequency domain context residuals based on temporal and frequency domain attention representations includes: introducing lightweight window-level adapters on the temporal and frequency domain coding branches respectively to map the extracted attention representations into one-dimensional trajectories; The time domain processing result of the query window is as follows: The time domain processing result of the context prompt window is The frequency domain processing result of the query window is The frequency domain processing result of the context prompt window is Calculate the time-domain context residual Calculate the frequency domain context residual ; The results obtained and It is a time-varying first-order context residual trajectory used to reflect the degree of deviation of the current window from the normal reference window from both time and frequency perspectives.

5. The anomaly detection method based on contextual residual learning as described in claim 1, characterized in that, The process of calculating the second-order residual of the context residual, and obtaining the curvature residual trajectory by fusing and normalizing the second-order residuals in the time and frequency domains, includes: fusing the second-order residuals in the time domain... With frequency domain context residual Treating it as a discrete-time curve, the second-order residual is calculated using the central second-order difference operator. By fusing and normalizing the second-order residuals in the time and frequency domains, the curvature residual trajectory is obtained. ; The fundamental magnitude residuals are constructed by averaging the absolute values ​​of the first-order residuals. Then, the basic amplitude residuals and curvature residual trajectory Normalization and weighted interpolation are performed to obtain the basic residuals for curvature sensing. It is used to reflect amplitude deviation and includes information on curvature changes.

6. An anomaly detection method based on contextual residual learning as described in claim 1 or 5, characterized in that, The time series anomaly detection model also includes a mask-aware low-rank feature trajectory projection module, which is used to project the trajectory in a unified channel dimension. Below, multiple windows are sampled from multi-domain data to form a matrix. The matrix is ​​centered, and the mean vector is denoted as μ. Singular value decomposition is performed on the centered matrix, and the first K principal components whose cumulative explained variance reaches the preset threshold are selected to form the basis matrix. For any new window vector x, calculate its projection into the subspace. Reconstruction Obtain the window-level residual vector By combining the channel mask M with a weighted average only on the effective channels, the low-rank residual trajectory is obtained. This serves as a teacher signal to guide the model residuals during training.

7. The anomaly detection method based on contextual residual learning as described in claim 1, characterized in that, The process of using a task loss to constrain the differences between the attention distribution in the time and frequency domains and the mask target, and a second-order consistency loss to constrain the normalized shape of the curvature residuals to remain consistent with the basic residuals, as loss functions includes: employing a task loss based on Kullback–Leibler divergence. By constraining the differences between the attention distribution in the time domain and the mask target in the frequency domain, the time series anomaly detection model is guided to focus on more critical time points and channels; By constructing a second-order consistency loss The constrained curvature residuals maintain a consistent normalized shape with the basic residuals, enabling the time series anomaly detection model to form a stable correspondence on the first-order and second-order geometric information. Constituting the total loss Joint pre-training is performed on a multi-domain training dataset to obtain a trained time series anomaly detection model.

8. The anomaly detection method based on contextual residual learning as described in claim 7, characterized in that, The total loss also includes low-rank consistency loss. Low-rank consistency loss The underlying residual trajectory output by the time series anomaly detection model. With the generated residual trajectory Alignment, suppressing instability caused by noise and inter-domain scale differences, total loss , These are the weighting coefficients.

9. The anomaly detection method based on contextual residual learning as described in claim 1, characterized in that, The process of identifying anomalous data by using a pre-trained time-series anomaly detection model to obtain curvature-aware basic residuals and comparing these residuals with a set threshold includes: obtaining curvature-aware basic residuals using the pre-trained time-series anomaly detection model. Combined with low-rank residuals The two are linearly fused according to the weight parameters to obtain the final anomaly score trajectory. The threshold value is determined based on the preset anomaly ratio or the estimated quantile. For each time point Identify abnormal score trajectories Does it exceed the set threshold? If the time exceeds the limit, an exception label will be assigned to the corresponding time point.

10. An anomaly detection system based on contextual residual learning, characterized in that, include: The anomaly detection module is configured to acquire detection data in the target domain. The detection data is a time series and is slid segmented according to a given window length. For each query window to be detected, several context prompt windows are selected. Using a pre-trained time series anomaly detection model, the curvature-aware basic residual is obtained. Based on the comparison result between the curvature-aware basic residual and a set threshold, the abnormal data is identified. The model training module is configured to acquire detection data from multiple domains, wherein the detection data is a time series and constitutes a training dataset set. The time series of each dataset in the set is slidably segmented according to a given window length to obtain a series of subsequence windows. The windows of each dataset are standardized and a channel mask is constructed for each window. The masked window is used to extract temporal attention representation using a temporal coding branch and frequency domain attention representation using a frequency coding branch. Based on the temporal and frequency domain attention representations, context residuals in the temporal and frequency domains are constructed. The second-order residual of the context residual is calculated. The curvature residual trajectory is obtained by fusing and normalizing the second-order residuals in the time domain and frequency domain. The absolute value of the first-order residual is averaged to construct the basic amplitude residual. The curvature residual trajectory and the basic amplitude residual are then normalized and weighted by interpolation to obtain the curvature-aware basic residual. Training is performed using the task loss, which constrains the difference between the attention distribution in the time and frequency domains and the mask target, and the second-order consistency loss, which constrains the normalized shape of the curvature residual to remain consistent with the basic residual, as loss functions.

Images