Automated Time Series Model Selection for Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for selecting appropriate machine learning models for anomaly detection in timeseries data are inefficient and often lead to inaccurate results, particularly in real-time applications like cybersecurity and fraud detection, due to the manual or brute force approaches required to identify suitable models for processing trends, heteroskedasticity, seasonality, and other temporal traits.
Innovation Solution
An automated model selection system that determines temporal traits in timeseries datasets, such as trend, heteroskedasticity, or seasonality, to select and adjust appropriate anomaly detection models, optimizing model execution based on the identified traits and signals within the data, thereby improving accuracy and speed of anomaly detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual exploration of timeseries data is used to select models, then model selection accuracy may be improved, but time consumption and resource consumption increase significantly
Solution Approach 1:
The system enables automatic model selection through self-service mechanisms where the algorithm autonomously analyzes timeseries data characteristics and selects appropriate models without human intervention, resolving the contradiction between accuracy and time consumption
Solution Approach 2:
The system changes parameters by automatically detecting temporal traits (trend, seasonality, heteroskedasticity) and adjusting model selection based on these detected parameters, eliminating manual exploration while maintaining accuracy
2Reliability
If brute force approach is used to run data through every model, then comprehensive model evaluation is achieved, but computing resources and time requirements become unacceptable
Solution Approach 1:
The system extracts only the necessary models for evaluation by detecting temporal traits first, then selecting only those models appropriate for the detected traits, rather than running all available models through the data
Solution Approach 2:
The approach segments the model evaluation process into two stages: first detecting temporal traits, then selecting and evaluating only relevant models for those traits, reducing the overall computational burden while maintaining reliability
3Measurement precision
If multiple models are evaluated manually, then better model selection may be achieved, but the process becomes unacceptable for real-time applications
Solution Approach 1:
The system performs preliminary action by detecting temporal traits and pre-selecting appropriate models before actual anomaly detection, enabling fast real-time processing while maintaining high selection quality
Solution Approach 2:
The automatic model selection system performs self-service by autonomously matching detected temporal traits with appropriate models, eliminating manual analysis time and enabling real-time response while maintaining selection quality
Data Source
AI summary
Methods and systems are described herein for improving anomaly detection in timeseries datasets. Different machine learning models may be trained to process specific types of timeseries data efficiently and accurately. Thus, selecting a proper machine learning model for identifying anomalies in a specific set of timeseries data may greatly improve accuracy and efficiency of anomaly detection. Another way to improve anomaly detection is to process a multitude of timeseries datasets for a time period (e.g., 90 days) to detect anomalies from those timeseries datasets and then correlate those detected anomalies by generating an anomaly timeseries dataset and identifying anomalies within the anomaly timeseries dataset. Yet another way to improve anomaly detection is to divide a dataset into multiple datasets based on a type of anomaly detection requested.