Anomaly Detection Model Using State Inference for Time-Series Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Anomaly detection in complex dynamic systems with large volumes of time-series data is challenging due to the difficulty in identifying normal operating states and anomalies, leading to potential false negatives and false positives, which can result in operational failures and financial losses.
Innovation Solution
The technique employs state inference and machine learning to detect anomalies in time-series data by obtaining likelihoods, training an anomaly detection model, and selecting appropriate thresholds to separate normal and anomalous behavior, using a supervised learning approach that generalizes well across different settings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional anomaly detection methods are used in complex dynamic systems with large volumes of time-series data, then the system can process the data, but it produces false negatives and false positives due to difficulty in identifying normal operating states and anomalies
Solution Approach 1:
The patent segments the time-series data into distinct operational states using state inference algorithms before performing anomaly detection. By dividing the complex data stream into manageable state segments, the system can apply targeted detection methods to each state, improving both accuracy and reliability by reducing false positives from normal state variations and false negatives from anomalies within specific states.
Solution Approach 2:
The patent performs preliminary state inference and likelihood calculation before the actual anomaly detection step. By pre-processing the data to identify normal operating states and calculate likelihood scores, the system prepares the foundation for more accurate anomaly detection, allowing it to distinguish between normal variations and true anomalies more effectively.
2Adaptability or versatility
If manual intervention is used to adjust anomaly detection thresholds, then detection accuracy can be improved for specific cases, but the system cannot adapt quickly to new data patterns and requires frequent adjustments
Solution Approach 1:
The patent implements a self-adapting anomaly detection system that automatically adjusts thresholds based on learned patterns from training data. The machine learning model autonomously optimizes detection parameters without requiring manual intervention, enabling the system to adapt quickly to new data patterns and reducing the time loss associated with frequent threshold adjustments.
Solution Approach 2:
The patent incorporates feedback mechanisms where the anomaly detection model continuously learns from new data and adjusts its parameters accordingly. The system uses feedback from detection results and new operational data to refine its thresholds and improve adaptability, creating a closed-loop system that automatically evolves with changing conditions.
3Adaptability or versatility
If supervised learning techniques are used to train the anomaly detection model, then the model can generalize well across different settings, but the training process requires significant computational resources and time
Solution Approach 1:
The patent performs preliminary feature extraction and state inference before training the anomaly detection model. By pre-processing the data to extract relevant features and organize it into state-based structures, the training process becomes more efficient and requires fewer computational resources while maintaining strong generalization capabilities across different settings.
4Measurement precision
If the system uses multiple anomaly thresholds to improve detection coverage, then more anomalies can be detected, but the complexity of the detection system increases
Solution Approach 1:
The patent segments the anomaly detection process into state-specific detection steps, where each operational state has its own optimized threshold. This segmentation allows the system to use multiple thresholds effectively while managing complexity by organizing them according to distinct states rather than using a complex multi-threshold system for all data uniformly.
Data Source
AI summary
Techniques are provided for anomaly detection in time-series data using state inference and machine learning. An exemplary method comprises: obtaining detected states of a plurality of data samples in temporal data, wherein each data sample in the temporal data has a corresponding detected state; obtaining a likelihood that each of the data samples belongs to the corresponding detected state; obtaining a distribution of likelihoods of the data samples indicating a number of observations of each of a plurality of likelihood values; training, using a supervised learning technique, an anomaly detection model that, given the distribution of likelihoods and one or more anomaly thresholds, generates a quality score for each of the anomaly thresholds; and selecting at least one anomaly threshold based on the quality score, wherein the trained anomaly detection model is applied to detect anomalies in new temporal data samples using the selected at least one anomaly threshold.