Time Series Forecasting with Missing Values via Segmented Data Structures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing time series forecasting algorithms face challenges with circular dependency between anomaly detection and forecasting, especially with missing values, leading to imperfect forecasts and biased anomaly identification.
Innovation Solution
The use of two data structures, an anomalies detection data structure and a forecast data structure, along with a Random Cut Forests (RCF) approach, allows for anomaly detection and forecasting with missing values, filtering out sparse anomalies and adapting to distribution changes, providing explainable forecasts that do not affect anomaly detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional forecasting algorithms are used to handle missing values, then forecasting capability is improved, but circular dependency between anomaly detection and forecasting occurs leading to biased anomaly identification
Solution Approach 1:
The patent divides the data processing into two separate data structures: an anomaly detection data structure that excludes forecasted values, and a forecast data structure that includes them. This segmentation prevents the circular dependency where forecasted values would otherwise contaminate anomaly detection results.
Solution Approach 2:
The patent introduces an intermediary mechanism (the two separate data structures) that mediates between forecasting and anomaly detection. The anomaly detection data structure acts as a buffer that prevents forecasted values from directly influencing anomaly detection, while still allowing the forecasting function to operate on complete data.
2Measurement precision
If anomalies are removed before forecasting, then forecasting accuracy is improved, but data loss occurs reducing the robustness of the system
Solution Approach 1:
The patent applies local quality by treating different parts of the data differently: the anomaly detection data structure excludes anomalies to improve detection accuracy, while the forecast data structure includes them to maintain data completeness for forecasting. This localized differentiation resolves the contradiction between removing anomalies and preserving data.
3Loss of information
If missing values are imputed before anomaly detection, then data completeness is improved, but false anomalies are introduced reducing detection reliability
Solution Approach 1:
The patent segments the data handling process so that imputation and anomaly detection occur in separate data structures. The forecast data structure receives imputed values for completeness, while the anomaly detection data structure uses only original values, preventing false anomalies from contaminating detection results.
Data Source
AI summary
Techniques for seasonality-based anomaly detection and forecast are described. For example, a method of receiving a request to generate forecast for received time series data; performing a seasonality-based anomaly detection and forecast for the received time series data based upon the received request, the seasonality-based anomaly detection and forecasting to utilize a second data structure that reflect anomalies found in a first data structure on the input from the received time series data; and providing a result of the performed seasonality-based anomaly detection and forecast is described.