Skip Learning for Multivariate Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing anomaly detection methods for multivariate time-series data struggle to effectively apply to online streaming settings with thousands of sensors, leading to high false-alarm rates and lower precision and recall due to the inability to capture and explain relationships between different time-series data.
Innovation Solution
The implementation of a Multivariate Anomaly Detection (MVAD) approach using Isolation Median Absolute Deviation Forests with explainability (IMFx) and an innovative skip learning technique, which trains machine learning models to detect anomalies in time-series data by skipping anomalous values and utilizing rolling windows for continuous learning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional anomaly detection methods are used for multivariate time-series data from thousands of sensors, then the system can process streaming data, but the false-alarm rate increases and detection precision decreases due to inability to capture relationships between time-series data
Solution Approach 1:
The patent combines multiple time-series data sources into a unified multivariate anomaly detection model. The system integrates data from thousands of sensors across different equipment devices, capturing interrelationships between variables. This merging approach allows the model to distinguish true anomalies from normal variations, thereby improving detection precision and reducing false alarms simultaneously.
Solution Approach 2:
The system implements continuous feedback loops where detection results are fed back into the model for ongoing refinement. The model learns from accumulated data patterns, adjusting its sensitivity and thresholds based on historical performance. This feedback mechanism enables the system to adapt to changing operational conditions while maintaining high precision and reliability.
2Measurement precision
If complex machine learning models are deployed to capture relationships between time-series data, then detection accuracy improves, but computational resources and processing time increase
Solution Approach 1:
The patent segments the anomaly detection process into distinct computational stages: data preprocessing, feature extraction, anomaly scoring, and validation. Each stage processes only necessary data with optimized algorithms. This segmentation reduces overall computational burden while preserving detection accuracy by focusing resources on critical analysis steps rather than processing entire datasets monolithically.
Solution Approach 2:
The system applies partial action by implementing progressive anomaly detection that processes data in batches rather than continuously analyzing every data point in real-time. The model performs comprehensive analysis on sampled subsets and applies lighter-weight detection rules to remaining data. This approach achieves high accuracy on critical cases while reducing overall computational resource consumption.
3Speed
If real-time processing is implemented for streaming sensor data, then low-latency detection is achieved, but the system cannot effectively learn from historical patterns
Solution Approach 1:
The patent implements preliminary action by pre-computing statistical baselines, normal operational patterns, and relationship models during periods when data accumulates. These pre-learned patterns are stored and rapidly applied during real-time detection. This allows the system to perform fast anomaly scoring using pre-established knowledge, achieving low latency while maintaining strong learning capabilities from historical data.
Solution Approach 2:
The system maintains continuous learning through incremental model updates that process incoming streaming data in real-time. Rather than batch-processing only, the model continuously adapts to new patterns while preserving learned historical relationships. This continuous action ensures both low-latency detection capability and ongoing learning improvement without sacrificing either objective.
Data Source
AI summary
Systems and methods are described for detecting anomalies within time series data using one or more online machine learning models. In one example, time series data may be obtained that spans a moving time period of a set length, to be used as training data. A machine learning model may be trained using the training data to identify anomalies in the time series data by upon detecting an estimation of an anomalous value in the time series data, pausing input of additional values of the training data into the machine learning model, and resuming input of the additional values of the training data into the machine learning model based on detecting at least one non-anomalous value. The trained machine learning model may then be used to detect a first anomaly in additional time series data by comparing the at least one additional value to the trained machine learning model.