DeepTIMe Time-Series Forecasting via Continuous Time Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing time-series forecasting models, such as auto-regressive architectures like Transformer-based models, are limited by their complex parameterization relying on discrete time steps, which is not suitable for continuous time-series signals, leading to inefficiencies in memory and compute usage and reduced accuracy for longer horizons.
Innovation Solution
The DeepTIMe framework uses a meta-learning approach with a time-index model that takes a normalized time index as input, employing a ridge regressor to produce vector representations and forecast time-series data, allowing for efficient training and accurate predictions by optimizing parameters in a bi-level paradigm.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If Transformer-based models are used for time-series forecasting, then forecasting capability is provided, but memory and compute requirements increase and accuracy decreases for longer horizons
Solution Approach 1:
The patent changes the fundamental parameter representation from discrete time steps to continuous time embeddings. By representing time as continuous vectors rather than discrete indices, the model achieves better generalization for long-term forecasting while reducing computational complexity. The continuous time embeddings allow the model to interpolate and extrapolate more effectively across time horizons.
Solution Approach 2:
The patent introduces a new dimensional representation by mapping time to a continuous vector space rather than using scalar time steps. This dimensional transformation enables the model to capture temporal patterns more efficiently, improving forecasting accuracy for long horizons while reducing the computational burden associated with traditional Transformer architectures.
2Device complexity
If discrete time step parameterization is used, then model structure is simplified, but accuracy for continuous time-series signals decreases
Solution Approach 1:
The patent transforms the time parameter from discrete integer steps to continuous vector representations. This parameter change allows the model to better represent continuous time-series signals while maintaining a relatively simple architectural structure. The continuous time embeddings capture temporal relationships more effectively than discrete indices.
Solution Approach 2:
The patent replaces the mechanical discrete time step indexing system with a continuous embedding space. Instead of using integer-based time positions, the model uses continuous vector representations that can be smoothly interpolated, better matching the continuous nature of real-world time-series data while keeping the overall model structure efficient.
Data Source
AI summary
Embodiments described herein provide a time-index model for forecasting time-series data. The architecture of the model takes a normalized time index as an input, uses a model, g_φ, to produce a vector representation of the time-index, and uses a “ridge regressor” which takes the vector representation and provides an estimated value. The model may be trained on a time-series dataset. The ridge regressor is trained for a given g_φ to reproduce a given lookback window. g_φ is trained over time-indexes in a horizon window, such that g_φ and the corresponding ridge regressor will accurately predict the data in the horizon window. Once g_φ is sufficiently trained, the ridge regressor can be updated based on that final g_φ over a lookback window comprising the time-indexes with the last known values. The final g_φ together with the updated ridge regressor can be given time-indexes past the known values, thereby providing forecasted values.