Method, device, processor and medium for relieving misinformation of time sequence anomaly detection caused by distribution offset based on decoupling and causal inference

Abstract

The invention relates to a method for relieving a time sequence anomaly detection false alarm caused by distribution offset based on decoupling and causal inference. The method comprises the following steps: a main branch performs anomaly detection on original data based on prediction; the auxiliary branch divides the original data into two parts which are influenced by distribution offset and are invariable in environment, and a model structure isomorphic with the main branch is adopted to carry out anomaly detection on the invariable part; identifying and rejecting a misinformation sample, and obtaining an anomaly detection result after the misinformation is removed; the double-branch training process is accelerated through the knowledge distillation technology. According to the method and the device for relieving the time sequence anomaly detection false alarm caused by the distribution offset based on the decoupling and causal inference, the processor and the computer readable storage medium thereof, the data set association is fully extracted based on a double-attention mechanism; a distribution offset data invariant component separation method and an interpretable multivariate anomaly detection method eSPOT based on data enhancement, comparative learning and course learning are innovatively proposed, and the model efficiency is remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of neural network technology, and more particularly to the field of anomaly detection. Specifically, it relates to a method, apparatus, processor, and computer-readable storage medium for mitigating false alarms in time-series anomaly detection caused by distribution shift based on decoupling and causal inference. Background Technology

[0002] Time series anomaly detection (TSAD) has important applications in finance, industry, meteorology, and other fields, and is crucial for system stability, resource optimization, and risk identification. Due to the scarcity of anomalous data, existing TSAD methods are mainly based on unsupervised learning, directly learning normal patterns from unlabeled data and identifying points that significantly deviate from these patterns as anomalies. For example, reconstruction-based methods encode time series data into a low-dimensional space and reconstruct them; larger reconstruction errors correspond to higher anomaly probabilities. Conversely, prediction-based methods learn to predict future values, identifying moments with large prediction errors as anomalies.

[0003] However, traditional time series analysis typically assumes a stationary data distribution, while real-world time series often experience distribution shifts due to environmental changes or system dynamics. Unsupervised anomaly detection methods face significant challenges in handling distribution shifts: because distribution shifts manifest as changes in data patterns, the static patterns learned from historical data do not match the shifted distribution characteristics, making it difficult for models to distinguish between genuine anomalies and biases caused by the shift. This increases the false alarm rate and reduces detection effectiveness. For example, in traffic monitoring systems, models trained on historical data may misjudge holiday traffic fluctuations (distribution shifts) as abnormal congestion, while ignoring real accidents (genuine anomalies), leading to unnecessary emergency response costs or safety hazards.

[0004] Existing research addressing the TSAD problem under distribution shift typically assumes that the contextual information of the distribution shift is known, or that the data can be explicitly decomposed into shift-related variable components and stable, invariant components. However, these methods rely on the assumption that "invariant features are sufficient to characterize anomalies," while real-world scenarios are far more complex. Furthermore, artificially dividing features may disrupt systematic associations among multiple variables, which could contain crucial anomaly information. Therefore, existing methods lack fine-grained analysis of how distribution shift affects anomaly detection, as well as systematic mechanistic studies. The key to eliminating the impact of distribution shift lies in understanding its mechanism of action on anomaly detection, rather than simply relying on shift labels or blindly removing shifted data. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, apparatus, processor and computer-readable storage medium for mitigating false alarms in timing anomaly detection caused by distribution offset based on decoupling and causal inference, which is characterized by high accuracy, high automation and wide applicability.

[0006] To achieve the above objectives, the present invention provides a method, apparatus, processor, and computer-readable storage medium for mitigating false alarms in timing anomaly detection caused by distribution offset based on decoupling and causal inference, as well as the following: The method for mitigating false alarms in time-series anomaly detection caused by distribution shift, based on decoupling and causal inference, is characterized by the following steps: (1) The main branch is related to the original data. Anomaly detection based on prediction; (2) The auxiliary branch will take the original data It is divided into two parts: one affected by distribution shift and the other by environmental invariance. A model structure isomorphic to the main branch is used for the invariant part. Perform anomaly detection; (3) Combining the anomaly detection results of the main branch and the auxiliary branch , The system identifies and removes false positives caused by distribution shifts, obtaining anomaly detection results after removing false positives. ; (4) Accelerate the bi-branch training process through knowledge distillation technology.

[0007] Preferably, step (1) specifically includes the following steps: (1.1) For the original dataset where there are no outliers in the training set Feature extraction and model building are performed. (1.2) Using the trained model Based on historical data in the original data It predicts future data and calculates the error between the predicted and actual values. ; (1.3) Use the eSPOT method to obtain the detection tags output by the main branch. .

[0008] Preferably, in step (1.2), the error between the predicted value and the true value is calculated. Specifically: Calculate the error between the predicted value and the actual value using the following formula. : ; in, For error calculation function, Historical data from the original data set. This is the trained model.

[0009] Preferably, step (1.1) specifically includes the following steps: (1.1.1) Use attention mechanisms to extract the temporal features of each variable as it changes over time; (1.1.2) Use another attention mechanism to extract the changes in the correlation between variables over time; (1.1.3) Combine the features extracted in steps (1.1.1) and (1.1.2) to obtain the feature extraction model.

[0010] Preferably, step (1.3) specifically includes the following steps: (1.3.1) Calculate the first Prediction error of individual variables Information entropy ; (1.3.2) Assign corresponding weights to each variable based on the entropy value of the prediction error of each variable, normalize the weights, and ensure that the sum of the weights is 1; (1.3.3) Multiply the entropy values ​​of the prediction errors of each variable by their corresponding weights and sum them to obtain the anomaly score; (1.3.4) Using the SPOT method to score anomalies Perform calculations and automatically select anomaly score thresholds. If abnormal scores Greater than the abnormal score threshold If the sample is predicted as abnormal, then the sample is predicted as normal; otherwise, the sample is predicted as normal.

[0011] Preferably, step (2) specifically includes the following steps: (2.1) Through Separate the offset-related changes from the environmentally invariant components in the time series data to obtain... Environmentally invariant components in the raw data ; (2.2) Anomaly detection is performed on environmentally unchanging components to obtain the detection tags output by the auxiliary branch. .

[0012] Preferably, step (2.1) specifically includes the following steps: (2.1.1) Randomly select one of the three data processing strategies: permutation, scaling, and dithering. As weak enhancement methods, the remaining two are... and Combining as a strong enhancement method; (2.1.2) Use generate Corresponding weak augmentation data ,use and Generate separately The augmented data is combined using random one-hot encoding to obtain a strongly augmented dataset. Strong and weak augmentation datasets are denoted as ; (2.1.3) Random selection Sample window in As an anchor sample, Window at the same time As positive samples Window with a long time interval As negative samples Construct positive and negative pairs; (2.1.4) Narrowing the distance between positive sample pairs during training. At the same time, it increases the distance between negative sample pairs. ,right Conduct training.

[0013] Preferably, step (3) specifically includes the following steps: (3.1) The confusion and association of auxiliary branches are eliminated by adjusting the backdoor, and anomaly detection is performed; (3.2) Based on the main branch detection results ,exist Among the samples marked as anomalous, the detection results of the auxiliary branch are compared. Perform verification, if If the sample is determined to be normal, then the final test result for that sample is... If the sample is a false positive, then the final test result for that sample is... These are genuine anomalous samples; (3.3) Based on real labels With predictive labels Calculation performance metrics.

[0014] Preferably, step (4) specifically includes the following steps: (4.1) The trained main branch feature extraction module parameters Distillation to auxiliary branch In the corresponding module; (4.2) The distribution of the predicted values ​​of the main branch for future data is used as the soft objective, and the actual values ​​are used as the hard objective. The model is updated by combining soft and hard objectives, and the auxiliary branches are trained.

[0015] Preferably, the training of the auxiliary branch in step (4.2) specifically involves: The auxiliary branch is trained according to the following formula: in, Indicates the data distribution. These are hyperparameters used to control the weights of the corresponding items. This represents the KL divergence between A and B.

[0016] The device for mitigating false alarms in timing anomaly detection caused by distribution shift based on decoupling and causal inference is characterized in that the device comprises: A processor is configured to execute computer-executable instructions; The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the steps of the method described above for mitigating false alarms in timing anomaly detection caused by distribution offset based on decoupling and causal inference.

[0017] The processor for mitigating false alarms in timing anomaly detection caused by distribution offset based on decoupling and causal inference is characterized in that the processor is configured to execute computer-executable instructions, which, when executed by the processor, implement the various steps of the aforementioned method for mitigating false alarms in timing anomaly detection caused by distribution offset based on decoupling and causal inference.

[0018] The computer-readable storage medium is characterized in that it stores a computer program that can be executed by a processor to implement the steps of the above-described method for mitigating false alarms in timing anomaly detection caused by distribution offset based on decoupling and causal inference.

[0019] This invention employs a method, apparatus, processor, and computer-readable storage medium based on decoupling and causal inference to mitigate false positives in temporal anomaly detection caused by distribution shift. Through a dual-branch collaborative detection architecture combined with a voting mechanism, it effectively eliminates false positives in temporal data anomaly detection caused by distribution shift. This method fully extracts dataset associations based on a dual-attention mechanism and innovatively proposes a distribution shift-invariant component separation method based on data augmentation, contrastive learning, and curriculum learning, as well as an interpretable multivariate anomaly detection method, eSPOT. In terms of knowledge distillation, it inputs knowledge from the main branch into the auxiliary branch from both parameter and loss function dimensions, significantly improving model efficiency. Attached Figure Description

[0020] Figure 1 This is a schematic diagram illustrating the enhanced data generation method of the present invention for mitigating false alarms in the detection of temporal anomalies caused by distribution offset based on decoupling and causal inference.

[0021] Figure 2The flowchart of the method for mitigating false alarms in time-series anomaly detection caused by distribution offset based on decoupling and causal inference of the present invention is as follows: the training, separation and dual-branch detection of the original data separation environment invariant component separator are carried out through a voting mechanism to obtain the final detection result.

[0022] Figure 3 This is a schematic diagram of the network structure of the separation module in the method for mitigating false alarms in timing anomaly detection caused by distribution offset based on decoupling and causal inference according to the present invention.

[0023] Figure 4 This is a confused causal relationship diagram of the method for mitigating false alarms in timing anomaly detection caused by distribution offset based on decoupling and causal inference according to the present invention.

[0024] Figure 5 This is a visualization of the portability and plug-in effect of the method for mitigating false alarms in timing anomaly detection caused by distribution offset based on decoupling and causal inference in this invention.

[0025] Figure 6 This is a visualization of the knowledge distillation time efficiency and detection effect of the method for mitigating false alarms in time-series anomaly detection caused by distribution offset based on decoupling and causal inference, as presented in this invention. Detailed Implementation

[0026] To more clearly describe the technical content of the present invention, the following description is provided in conjunction with specific embodiments.

[0027] The method for mitigating false alarms in time-series anomaly detection based on decoupling and causal inference of the present invention includes the following steps: (1) The main branch is related to the original data. Anomaly detection based on prediction; (2) The auxiliary branch will take the original data It is divided into two parts: one affected by distribution shift and the other by environmental invariance. A model structure isomorphic to the main branch is used for the invariant part. Perform anomaly detection; (3) Combining the anomaly detection results of the main branch and the auxiliary branch , The system identifies and removes false positives caused by distribution shifts, obtaining anomaly detection results after removing false positives. ; (4) Accelerate the bi-branch training process through knowledge distillation technology.

[0028] In a preferred embodiment of the present invention, step (1) specifically includes the following steps: (1.1) For the original dataset where there are no outliers in the training set Feature extraction and model building are performed. (1.2) Using the trained model Based on historical data in the original data It predicts future data and calculates the error between the predicted and actual values. ; (1.3) Use the eSPOT method to obtain the detection tags output by the main branch. .

[0029] In a preferred embodiment of the present invention, step (1.2) involves calculating the error between the predicted value and the actual value. Specifically: Calculate the error between the predicted value and the actual value using the following formula. : ; in, For error calculation function, Historical data from the original data set. This is the trained model.

[0030] In a preferred embodiment of the present invention, step (1.1) specifically includes the following steps: (1.1.1) Use attention mechanisms to extract the temporal features of each variable as it changes over time; (1.1.2) Use another attention mechanism to extract the changes in the correlation between variables over time; (1.1.3) Combine the features extracted in steps (1.1.1) and (1.1.2) to obtain the feature extraction model.

[0031] In a preferred embodiment of the present invention, step (1.3) specifically includes the following steps: (1.3.1) Calculate the first Prediction error of individual variables Information entropy ; (1.3.2) Assign corresponding weights to each variable based on the entropy value of the prediction error of each variable, normalize the weights, and ensure that the sum of the weights is 1; (1.3.3) Multiply the entropy values ​​of the prediction errors of each variable by their corresponding weights and sum them to obtain the anomaly score; (1.3.4) Using the SPOT method to score anomalies Perform calculations and automatically select anomaly score thresholds. If abnormal scores Greater than the abnormal score threshold If the sample is predicted as abnormal, then the sample is predicted as normal; otherwise, the sample is predicted as normal.

[0032] In a preferred embodiment of the present invention, step (2) specifically includes the following steps: (2.1) Through Separate the offset-related changes from the environmentally invariant components in the time series data to obtain... Environmentally invariant components in the raw data ; (2.2) Anomaly detection is performed on environmentally unchanging components to obtain the detection tags output by the auxiliary branch. .

[0033] In a preferred embodiment of the present invention, step (2.1) specifically includes the following steps: (2.1.1) Randomly select one of the three data processing strategies: permutation, scaling, and dithering. As weak enhancement methods, the remaining two are... and Combining as a strong enhancement method; (2.1.2) Use generate Corresponding weak augmentation data ,use and Generate separately The augmented data is combined using random one-hot encoding to obtain a strongly augmented dataset. Strong and weak augmentation datasets are denoted as ; (2.1.3) Random selection Sample window in As an anchor sample, Window at the same time As positive samples Window with a long time interval As negative samples Construct positive and negative pairs; (2.1.4) Narrowing the distance between positive sample pairs during training. At the same time, it increases the distance between negative sample pairs. ,right Conduct training.

[0034] In a preferred embodiment of the present invention, step (3) specifically includes the following steps: (3.1) The confusion and association of auxiliary branches are eliminated by adjusting the backdoor, and anomaly detection is performed; (3.2) Based on the main branch detection results ,exist Among the samples marked as anomalous, the detection results of the auxiliary branch are compared. Perform verification, if If the sample is determined to be normal, then the final test result for that sample is... If the sample is a false positive, then the final test result for that sample is... These are genuine anomalous samples; (3.3) Based on real labels With predictive labels Calculation performance metrics.

[0035] In a preferred embodiment of the present invention, step (4) specifically includes the following steps: (4.1) The trained main branch feature extraction module parameters Distillation to auxiliary branch In the corresponding module; (4.2) The distribution of the predicted values ​​of the main branch for future data is used as the soft objective, and the actual values ​​are used as the hard objective. The model is updated by combining soft and hard objectives, and the auxiliary branches are trained.

[0036] In a preferred embodiment of the present invention, the training of the auxiliary branch in step (4.2) specifically includes: The auxiliary branch is trained according to the following formula: in, Indicates the data distribution. These are hyperparameters used to control the weights of the corresponding items. This represents the KL divergence between A and B.

[0037] The present invention provides an apparatus for mitigating false alarms in the detection of temporal anomalies caused by distribution shifts based on decoupling and causal inference, wherein the apparatus comprises: A processor is configured to execute computer-executable instructions; The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the steps of the method described above for mitigating false alarms in timing anomaly detection caused by distribution offset based on decoupling and causal inference.

[0038] The processor of the present invention, which mitigates false alarms in timing anomaly detection caused by distribution offset based on decoupling and causal inference, is configured to execute computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the various steps of the above-described method for mitigating false alarms in timing anomaly detection caused by distribution offset based on decoupling and causal inference.

[0039] The computer-readable storage medium of the present invention stores a computer program that can be executed by a processor to implement the various steps of the method described above for mitigating false alarms in timing anomaly detection caused by distribution offset based on decoupling and causal inference.

[0040] The purpose of this invention is to overcome the shortcomings of the prior art and provide a transferable, label-independent, two-branch time-series anomaly detection method based on decoupling and causal inference. This method eliminates false alarms caused by distribution shifts and separates invariant features. It obtains two-branch collaborative detection results through a voting mechanism, accelerates model training using knowledge distillation, and proposes an entropy-based multivariate data anomaly detection method, eSPOT.

[0041] This invention relates to a method for mitigating false positives in time-series anomaly detection caused by distribution shift based on decoupling and causal inference. The method employs a dual-branch architecture: the main branch directly detects anomalies in the original data; the auxiliary branch divides the original data into two components: those affected by distribution shift and those invariant to the environment. Anomaly detection is performed on the invariant component using a model structure isomorphic to the main branch. Combined with backdoor adjustments in causal inference, the association confusion caused by distribution shift is eliminated. The main branch's detection results are dominant, while the auxiliary branch's detection results are used to effectively identify and eliminate false positive samples caused by distribution shift through a voting mechanism. Knowledge distillation technology is used to accelerate the dual-branch training process. This invention also relates to an environment-invariant component separation technique, based on data augmentation, contrastive learning, and curriculum learning strategies, to accurately separate shift-related changes and environment-invariant components in time-series data without the need for distribution labels, providing support for subsequent operations.

[0042] Please see Figure 2 As shown, the present invention provides a portable plug-in method for eliminating false alarms in time series data anomaly detection caused by distribution offset based on decoupling and causal inference. The method includes the following steps: (1) The main branch directly performs anomaly detection on the original data based on prediction; (2) The auxiliary branch divides the original data into two components: one affected by distribution shift and the other unaffected by the environment. It adopts a model structure isomorphic to the main branch based on the unaffected part. Perform anomaly detection; (3) By effectively combining the anomaly detection results of the main and auxiliary branches through the voting mechanism, false alarm samples caused by distribution shift are identified and eliminated to obtain the anomaly detection results after removing false alarms; (4) Knowledge distillation technique is used to accelerate the bi-branch training process.

[0043] Furthermore, step (1) specifically includes the following steps: (1.1) For the original dataset where there are no outliers in the training set Feature extraction and model building are performed. (1.2) Using the trained model Based on historical data in the original data It predicts future data and calculates the error between the predicted and actual values. : in For the error calculation function, MSE or MAE can be selected in this invention; (1.3) Use the eSPOT method to obtain the detection tags output by the main branch. .

[0044] Furthermore, step (1.1) specifically includes the following steps: (1.1.1) Use attention mechanisms to extract the temporal features of each variable as it changes over time; (1.1.2) Use another attention mechanism to extract the changes in the correlation between variables over time; (1.1.3) Integrate the features extracted in steps (1.1.1) and (1.1.2) to obtain a more comprehensive and reliable feature extraction model.

[0045] Furthermore, step (1.3) specifically includes the following steps: (1.3.1) Calculate the first Prediction error of individual variables Information entropy The smaller the entropy value, the more reliable the error of the variable; (1.3.2) Assign corresponding weights to each variable based on the entropy value of the prediction error of each variable in the following manner: Normalize the weights to ensure that the sum of the weights is 1; (1.3.3) The anomaly score is obtained by summing the entropy values ​​of the prediction errors of each variable after multiplying them by their corresponding weights. ; (1.3.4) Using the SPOT method to score anomalies The SPOT method automatically selects the anomaly score threshold during calculation. , The sample is predicted as abnormal if it is not abnormal, and as normal if it is not abnormal. Furthermore, step (2) specifically includes the following steps: (2.1) Based on data augmentation, contrastive learning, and curriculum learning strategies, without the need for distributed labels, through To achieve accurate separation of offset-related changes and environmentally invariant components in time-series data, based on... Get Environmentally invariant components in the raw data ; (2.2) Using the same steps and network structure as in step (1), for Perform anomaly detection and obtain the detection labels output by the auxiliary branch. .

[0046] Furthermore, step (2.1) specifically includes: (2.1.1) Randomly select one of the three data processing strategies: permutation, scaling, and dithering. As weak enhancement methods, the remaining two are... and Combining as a strong enhancement method; (2.1.2) Use generate Corresponding weak augmentation data ,use and Generate separately The augmented data is combined using random one-hot encoding to obtain a strongly augmented dataset. Strong and weak augmentation datasets are denoted as ; (2.1.3) Random selection Sample window in As an anchor sample, Window at the same time As positive samples Window with a long time interval As negative samples (in ), construct positive and negative pairs; (2.1.4) Narrowing the distance between positive sample pairs during training At the same time, it increases the distance between negative sample pairs. Follow the steps below Conduct training: in, The distance between positive sample pairs. The distance between negative sample pairs is denoted by , and Relu(·) is the activation function. (2.1.5) In addition, Based on course learning strategy training, first use Train until convergence, then use Perform the same operations to ensure the stability and reliability of the training process.

[0047] Furthermore, step (3) specifically includes: (3.1) The auxiliary branch eliminates the confusion caused by the distribution offset through backdoor adjustment, and performs anomaly detection on this basis; (3.2) Based on the main branch detection results As the leader, in In samples marked as anomalous, further comparisons were made of the detection results of the secondary branches. Perform verification. When If the sample is determined to be normal, then the final test result for that sample is... If the sample is a false positive, then the final test result for that sample is... This is a genuine anomalous sample.

[0048] (3.3) Based on the actual label in the following manner With predictive labels Computational performance metrics: , , Among the true examples ( ), false positives ( ), false negatives ( ), true negative examples ( ) represent the number of samples that the model predicted was positive and actually was positive, predicted was positive but actually was negative, predicted was negative but actually was positive, and predicted was negative and actually was negative, respectively.

[0049] Furthermore, step (4) specifically includes: (4.1) Regarding model parameters, the trained main branch feature extraction module is processed using weighted interpolation in the following manner. parameters Distillation to auxiliary branch In the corresponding module: in express The initial weights; (4.2) Regarding the loss function, the distribution of the predicted values ​​of the main branch for future data is used as the soft objective, and the true values ​​are used as the hard objective. The model is updated by combining soft and hard objectives, and the auxiliary branches are trained in the following manner: in , This indicates the distribution of data.

[0050] In a specific embodiment of this invention, during the anomaly detection process of time-series data under distribution shift, the existence of distribution shift may cause the model to misjudge the changes brought about by the distribution shift as anomalous samples. This technical solution proposes a transferable plug-in method based on decoupling and causal inference to eliminate false positives in time-series data anomaly detection caused by distribution shift. By combining a dual-branch collaborative detection architecture with a voting mechanism, the method effectively eliminates false positives in time-series data anomaly detection caused by distribution shift. This method fully extracts the correlation of the dataset based on a dual attention mechanism and innovatively proposes a distribution shift-based data invariant component separation method and an interpretable multivariate anomaly detection method, eSPOT, based on data augmentation, contrastive learning, and curriculum learning. In terms of knowledge distillation, the knowledge from the main branch is input into the auxiliary branch from two dimensions: parameters and loss function, significantly improving model efficiency.

[0051] This method for eliminating false alarms in time-series data anomaly detection caused by distribution shift, based on invariant feature separation and causal inference, mainly includes the following three stages: (1) It has a module for separating environmentally invariant components in distributed migration data; (2) It has a dual-branch anomaly detection module; (3) It has a voting mechanism module.

[0052] 1. Separation of environmentally invariant components in distribution migration data This invention employs a data augmentation, contrastive learning, and curriculum learning strategy to accurately separate offset-related changes from environmentally invariant components in time-series data without the need for distributed labels. The data augmentation process is as follows: Figure 1 As shown, the structure and training process of the separation module are as follows: Figure 2 As shown.

[0053] The first step is to randomly select one of the three data processing strategies: permutation, scaling, and dithering. As weak enhancement methods, the remaining two are... and Combining as a strong enhancement method; The second step is to use... generate Corresponding weak augmentation data , and Generate separately The two augmented data sets are combined using random one-hot encoding to obtain a strongly augmented dataset. Strong and weak augmentation datasets are denoted as ; The third step is to randomly select... Sample window in As anchor samples, Window at the same time As positive samples Window with a long time interval As negative samples (in ), construct positive and negative pairs; The fourth step is to narrow the distance between positive sample pairs during training. At the same time, it increases the distance between negative sample pairs. Follow the steps below Conduct training: The distance between positive sample pairs. The distance between negative sample pairs is denoted by , and Relu(·) is the activation function. In addition, the separation module is trained based on the course learning strategy, first using Train until convergence, then use Perform the same operations to ensure the stability and reliability of the training process.

[0054] 2. Dual-branch anomaly detection The main and auxiliary branches respectively use a dual attention mechanism to extract and predict features from the original data and environment-invariant components, obtaining prediction errors. These prediction errors are then input into eSPOT to obtain the anomaly detection results for each branch. The specific process is as follows: Figure 2 As shown.

[0055] The first step is for the main branch to directly process the original data. Anomaly detection yields predicted labels. ; The second step is to use a separator. Separate Environmentally invariant components ; The third step involves auxiliary branches that are environmentally invariant components. Anomaly detection yields predicted labels. .

[0056] 3. Voting Mechanism Distribution offset indirectly affects anomaly detection bias through intermediate paths, which is the root cause of the increased false positive rate. Collaborative anomaly detection via main and auxiliary branches can reduce false positives caused by the above reasons. The voting mechanism process is as follows: The first step is to use the main branch detection results. To take the lead and find Samples marked as anomalous; The second step is to further compare the detection results of the auxiliary branches. Verification will be performed. Specifically, when... If the sample is determined to be normal, then the final test result for that sample is... If the sample is a false positive, then the final test result for that sample is... This is a genuine anomalous sample.

[0057] To more clearly demonstrate the effects of this invention, the main branch for detecting anomalies in the original data is denoted as Base, and the auxiliary branch for detecting anomalies in invariant components is denoted as Cake.

[0058] The anomaly detection performance of this invention in distributed offset scenarios is as follows: Table 1 shows the performance of this invention on three datasets: real-world and synthetic datasets, and strong and weak distribution offset datasets. The Water Treatment Plant dataset (SWaT) and the Server Machine dataset (SMD) are two widely used real-world datasets in the field of industrial anomaly detection. In addition, we introduced the penicillin fermentation process simulation dataset (Pensim) to validate the model's performance in simulated industrial environments.

[0059] The portability and plug-in effects of this invention are as follows: Figure 5 The transferability of the plugin of this invention across autoencoder (AE), convolutional neural network (CNN), long short-term memory network (LSTM), and gated recurrent neural network (GRU) frameworks is visually demonstrated. Results show that the plugin of this invention can effectively mitigate misjudgments in anomaly detection caused by distribution shift across different anomaly detection models, specifically by improving precision and F1 score while maintaining relatively small fluctuations in the original recall rate.

[0060] The training acceleration effect brought about by knowledge distillation is as follows: Table 2 shows a comparison of anomaly detection time for the plugin in this invention on different models before and after knowledge distillation. Furthermore, to further demonstrate the balance between time efficiency and detection performance before and after knowledge distillation, Figure 6 The model training time and training effect were compared, and the results showed that the present invention maintained the detection effect while significantly reducing the training time.

[0061] The model is evaluated as follows: In terms of technical performance indicators, this invention demonstrates excellent false alarm elimination capabilities in anomaly detection under various distribution shift conditions. Specific experimental data show that it achieves an F1-score of 0.8972 on the SGaT dataset with strong distribution shift, representing a performance improvement of 8.02 percentage points; and an F1-score of 0.9475 on the Pensim dataset, representing an improvement of 0.42 per thousand. For the SMD dataset with weak distribution shift, its performance difference from the optimal solution is only 1.57 percentage points. Cross-dataset tests show that the average F1-score of this solution remains consistently high at 0.9371, fully demonstrating its robustness and generalization ability under different distribution shift conditions.

[0062] The technological innovation lies in the design of the core plugin, which possesses significant architecture-independent characteristics. Experimental results show that when applied to the AE architecture, it achieves a 28.78 percentage point improvement in accuracy and a 12.35 percentage point improvement in F1-score on the SWAT dataset; and a 48.21 percentage point improvement in accuracy and a 24.76 percentage point improvement in F1-score on the Pensim dataset. For the LSTM architecture, it achieves a 7.42 percentage point improvement in accuracy and a 3.19 percentage point improvement in F1-score on the SMD dataset. These data confirm the effectiveness of this module in eliminating spurious correlations and reducing false positive rates.

[0063] In terms of implementation efficiency, the knowledge distillation technique employed in this scheme significantly optimizes the training process. Experimental data shows that a single round of distillation training can achieve the performance level of four rounds of patience from the traditional early stopping method. Specifically, the training time on the SMD dataset Base architecture is reduced from 2803.80 seconds to 287.27 seconds, a reduction of 89.75 percentage points. This technique not only significantly improves training efficiency but also effectively suppresses overfitting, ensuring the stability and reliability of the model.

[0064] In summary, this invention has advantages in adaptability to strong distribution shifts, versatility across model architectures, and efficiency in the training process. Experimental data fully verify the technical advantages of this scheme in improving detection accuracy, reducing false alarm rate, and optimizing computational efficiency, providing reliable technical support for anomaly detection in industrial environments. Therefore, the proposed method for mitigating false alarms in temporal anomaly detection caused by distribution shifts based on decoupling and causal inference is effective and practically significant.

[0065] This module is used to separate invariant components in distributed offset data, wherein the module includes: Layer normalization is used for initialization to provide a stable numerical basis for subsequent processing. There are two attention mechanisms, namely Flash Attention and ordinary attention mechanism. Flash Attention performs efficient feature extraction on the original input and captures long-range dependencies. Then, ordinary attention mechanism is used to further extract features from its output. Dual-stream feature transformation: the global stream transforms features as a whole through linear projection, emphasizing global correlation; the local stream uses one-dimensional convolution with Dropout to focus on fine-tuning local features. The dual-stream outputs are combined for feature fusion to achieve enhanced interaction between global and local information. The output of the separation module is multiplied by the original input data to obtain the environmentally invariant components; The module maintains training stability through multiple layers of normalization and residual connections, avoiding gradient vanishing or exploding.

[0066] This invention analyzes the impact mechanism of distribution shift on anomaly detection using causal reasoning. During the generation of time-series data, the data itself directly influences the generation of anomaly labels, and indirectly affects the anomaly judgment result through the mediating variable of distribution shift. Traditional anomaly detection models, when learning directly from historical data, confuse the influence of data distribution itself and distribution shift on anomaly judgment, leading to the misclassification of normal distribution changes as anomalies. To accurately measure the true causal relationship between data and anomaly labels, this invention constructs a counterfactual reasoning framework. By fixing specific time-series data and determining the values ​​of mediating variables, it quantifies the indirect impact of distribution shift on anomaly judgment. Research reveals that during anomaly detection, distribution shift indirectly affects anomaly judgment bias through mediating paths, which is the root cause of the increased false positive rate. Based on this, this invention proposes to effectively distinguish between true anomalies and normal changes caused by distribution shift by identifying and eliminating the indirect impact of distribution shift during model training and inference, thereby reducing the false positive rate. This method reveals the intrinsic mechanism by which distribution shift affects anomaly detection from a causal perspective, providing a new technical approach to solving the false positive problem.

[0067] The Nonparametric Anomaly Detection Method (SPOT) based on extreme value optimization theory is a univariate anomaly detection method. This method achieves adaptive threshold optimization through dynamic sequential analysis, independent of data distribution assumptions, and can automatically set thresholds, resulting in more accurate and automated anomaly detection. Although SPOT is designed for univariate detection, it establishes a rigorous theoretical framework for univariate anomaly detection and provides a new methodological foundation for building interpretable multidimensional detection systems.

[0068] For the specific implementation scheme of this embodiment, please refer to the relevant descriptions in the above embodiments, which will not be repeated here.

[0069] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0070] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means at least two.

[0071] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0072] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution device. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0073] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The corresponding program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0074] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0075] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0076] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0077] This invention employs a method, apparatus, processor, and computer-readable storage medium based on decoupling and causal inference to mitigate false positives in temporal anomaly detection caused by distribution shift. Through a dual-branch collaborative detection architecture combined with a voting mechanism, it effectively eliminates false positives in temporal data anomaly detection caused by distribution shift. This method fully extracts dataset associations based on a dual-attention mechanism and innovatively proposes a distribution shift-invariant component separation method based on data augmentation, contrastive learning, and curriculum learning, as well as an interpretable multivariate anomaly detection method, eSPOT. In terms of knowledge distillation, it inputs knowledge from the main branch into the auxiliary branch from both parameter and loss function dimensions, significantly improving model efficiency.

[0078] In this specification, the invention has been described with reference to specific embodiments thereof. However, it will be apparent that various modifications and variations can be made without departing from the spirit and scope of the invention. Therefore, the specification and drawings should be considered illustrative rather than restrictive.

Claims

1. A method for mitigating false alarms in time-series anomaly detection caused by distribution shift based on decoupling and causal inference, characterized in that, The method includes the following steps: (1) The main branch is related to the original data. Anomaly detection based on prediction; (2) The auxiliary branch will take the original data It is divided into two parts: one affected by distribution shift and the other by environmental invariance. A model structure isomorphic to the main branch is used for the invariant part. Perform anomaly detection; (3) Combining the anomaly detection results of the main branch and the auxiliary branch , The system identifies and removes false positives caused by distribution shifts, obtaining anomaly detection results after removing false positives. ; (4) Accelerate the bi-branch training process through knowledge distillation technology.

2. The method for mitigating false alarms in time-series anomaly detection based on decoupling and causal inference according to claim 1, characterized in that, Step (1) specifically includes the following steps: (1.1) For the original dataset where there are no outliers in the training set Feature extraction and model building are performed. (1.2) Using the trained model Based on historical data in the original data It predicts future data and calculates the error between the predicted and actual values. ; (1.3) Use the eSPOT method to obtain the detection tags output by the main branch. .

3. The method for mitigating false alarms in time-series anomaly detection based on decoupling and causal inference according to claim 2, characterized in that, In step (1.2), the error between the predicted value and the actual value is calculated. Specifically: Calculate the error between the predicted value and the actual value using the following formula. : ； in, For error calculation function, Historical data from the original data. This is the trained model.

4. The method for mitigating false alarms in time-series anomaly detection based on decoupling and causal inference according to claim 2, characterized in that, Step (1.1) specifically includes the following steps: (1.1.1) Use attention mechanisms to extract the temporal features of each variable as it changes over time; (1.1.2) Use another attention mechanism to extract the changes in the correlation between variables over time; (1.1.3) Combine the features extracted in steps (1.1.1) and (1.1.2) to obtain the feature extraction model.

5. The method for mitigating false alarms in time-series anomaly detection based on decoupling and causal inference according to claim 2, characterized in that, Step (1.3) specifically includes the following steps: (1.3.1) Calculate the first Prediction error of individual variables Information entropy ; (1.3.2) Assign corresponding weights to each variable based on the entropy value of the prediction error of each variable, normalize the weights, and ensure that the sum of the weights is 1; (1.3.3) Multiply the entropy values ​​of the prediction errors of each variable by their corresponding weights and sum them to obtain the anomaly score; (1.3.4) Using the SPOT method to score anomalies Perform calculations and automatically select anomaly score thresholds. If abnormal scores Greater than the abnormal score threshold If the sample is predicted as abnormal, then the sample is predicted as normal; otherwise, the sample is predicted as normal.

6. The method for mitigating false alarms in time-series anomaly detection based on decoupling and causal inference according to claim 1, characterized in that, Step (2) specifically includes the following steps: (2.1) Through Separate the offset-related changes from the environmentally invariant components in the time series data to obtain... Environmentally invariant components in the raw data ; (2.2) Anomaly detection is performed on environmentally unchanging components to obtain the detection tags output by the auxiliary branch. .

7. The method for mitigating false alarms in time-series anomaly detection based on decoupling and causal inference according to claim 6, characterized in that, Step (2.1) specifically includes the following steps: (2.1.1) Randomly select one of the three data processing strategies: permutation, scaling, and dithering. As weak enhancement methods, the remaining two are... and Combining as a strong enhancement method; (2.1.2) Use generate Corresponding weak augmentation data ,use and Generate separately The augmented data is combined using random one-hot encoding to obtain a strongly augmented dataset. Strong and weak augmentation datasets are denoted as ; (2.1.3) Random selection Sample window in As an anchor sample, Window at the same time As positive samples Window with a long time interval As negative samples Construct positive and negative pairs; (2.1.4) Narrowing the distance between positive sample pairs during training. At the same time, it increases the distance between negative sample pairs. ,right Conduct training.

8. The method for mitigating false alarms in time-series anomaly detection based on decoupling and causal inference according to claim 1, characterized in that, Step (3) specifically includes the following steps: (3.1) The confusion and association of auxiliary branches are eliminated by adjusting the backdoor, and anomaly detection is performed; (3.2) Based on the main branch detection results ,exist Among the samples marked as anomalous, the detection results of the auxiliary branch are compared. Perform verification, if If the sample is determined to be normal, then the final test result for that sample is... If the sample is a false positive, then the final test result for that sample is... These are genuine anomalous samples; (3.3) Based on real labels With predictive labels Calculation performance metrics.

9. The method for mitigating false alarms in time-series anomaly detection based on decoupling and causal inference according to claim 1, characterized in that, Step (4) specifically includes the following steps: (4.1) The trained main branch feature extraction module parameters Distillation to auxiliary branch In the corresponding module; (4.2) The distribution of the predicted values ​​of the main branch for future data is used as the soft objective, and the actual values ​​are used as the hard objective. The model is updated by combining soft and hard objectives, and the auxiliary branches are trained.

10. The method for mitigating false alarms in time-series anomaly detection based on decoupling and causal inference according to claim 9, characterized in that, The training of the auxiliary branch in step (4.2) is specifically as follows: The auxiliary branch is trained according to the following formula: in, Indicates the data distribution. These are hyperparameters used to control the weights of the corresponding items. This represents the KL divergence between A and B.

11. A device for mitigating false alarms in timing anomaly detection caused by distribution shift based on decoupling and causal inference, characterized in that, The device includes: A processor is configured to execute computer-executable instructions; The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the steps of the method for mitigating false alarms in timing anomaly detection based on decoupling and causal inference as described in any one of claims 1 to 10.

12. A processor for mitigating false alarms in timing anomaly detection caused by distribution offset based on decoupling and causal inference, characterized in that, The processor is configured to execute computer-executable instructions, which, when executed by the processor, implement the steps of the method for mitigating false alarms in timing anomaly detection caused by distribution offset based on decoupling and causal inference as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, It stores a computer program that can be executed by a processor to implement the steps of the method for mitigating false alarms in timing anomaly detection caused by distribution offset based on decoupling and causal inference, as described in any one of claims 1 to 10.

Images