Hierarchical Factor Forecasting for Scalable High-Dimensional Time Series
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing time-series forecasting techniques struggle with high-dimensional datasets, failing to capture complex data patterns, scale poorly, and lack interpretability, leading to inaccurate and uncertain forecasts, especially in domains like retail and cloud computing.
Innovation Solution
A system and method for hierarchical factor-based forecasting that converts time-series data into distance vectors, performs hierarchical clustering, extracts commonalities using Gaussian process decomposition, and applies random forest classifiers for precise forecasting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing time-series forecasting techniques are used, then forecasting can be performed, but they fail to capture complex data patterns and scale poorly with high-dimensional datasets
Solution Approach 1:
The patent segments the high-dimensional time-series dataset into multiple clusters using hierarchical clustering algorithms. This segmentation allows the system to process and analyze subsets of data independently, improving scalability while maintaining the ability to capture complex patterns within each cluster. The segmented approach enables parallel processing and reduces computational complexity.
Solution Approach 2:
The patent transforms the original time-series data into a new dimensional space by creating distance vectors between time-series points and applying hierarchical clustering. This dimensionality change allows the system to handle high-dimensional datasets more effectively by organizing data in a tree-like hierarchical structure, improving both scalability and pattern recognition capabilities.
2Ease of operation
If existing forecasting methods are applied, then forecasts can be generated, but they lack interpretability and provide uncertainty in estimations
Solution Approach 1:
The patent introduces distance vectors as an intermediary representation between the original time-series data and the forecasting model. These distance vectors capture the relationships and patterns in the data while maintaining interpretability. The hierarchical clustering of these vectors provides a structured view that enhances understanding of forecast uncertainty and improves reliability.
3Measurement precision
If traditional forecasting tools are used, then training can be performed, but they are not readily scalable to big datasets containing millions of time-series
Solution Approach 1:
The patent divides the large-scale forecasting problem into smaller, manageable segments through hierarchical clustering. By organizing millions of time-series into a hierarchical tree structure, the system can process segments independently and combine results, achieving scalability without sacrificing forecast precision. This segmented approach reduces memory requirements and enables distributed computing.
Solution Approach 2:
The patent applies dimensionality change by transforming the original high-dimensional time-series data into distance vectors and organizing them in a hierarchical clustering space. This transformation reduces the computational complexity associated with processing millions of time-series while maintaining the ability to generate precise forecasts through the structured hierarchical representation.
Data Source
AI summary
The present invention provides for a system and a method for optimised time series forecasting. A time-series dataset is converted corresponding to a system, for which forecast data is to be determined, into data embeddings in the form of a distance vector. A hierarchical clustering of values of the distance vector is performed, wherein the hierarchical clustering comprises creating a high-level cluster by combining two or more local clusters. A hierarchical tree is created based on the hierarchical clustering, wherein the hierarchical tree represents a first level cluster and a second level cluster. A plurality of factors is extracted from each node of the tree and a gaussian process decomposition is applied on the extracted factors from each node of the tree to determine decomposed factors. The decomposed factors represent interpretable components of the extracted factors and a forecast data is determined for system based on decomposed factors.