Scalable Time Series Forecasting via Parallel Seasonal Encoders
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning-based time series forecasting methods face challenges in effectively incorporating seasonal correlations, leading to issues such as vanishing gradients and poor multi-step predictive ability, particularly due to the use of large dimensional combined vectors that increase model and processing complexity.
Innovation Solution
A scalable deep neural network architecture is proposed, comprising multiple parallel encoders, a neural network layer, and a decoder with sequential units. This architecture processes seasonal time series data by generating summary vectors that encode seasonal correlations, and uses a neural network layer to create a feature vector independent of the order of seasonality, making the model more context-dependent and efficient.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple parallel encoders are used to capture seasonal correlations of different orders, then the predictive ability for multi-step time series forecasting is improved, but the dimensionality of the combined vector increases leading to increased model complexity and processing time
Solution Approach 1:
The patent extracts the seasonal correlation information from the time series data and encodes it into summary vectors using multiple parallel encoders. Each encoder processes a specific seasonal lag group, extracting relevant patterns without including unnecessary information from other seasons. This extraction approach allows the model to capture multi-step predictive ability while keeping each encoder's processing scope manageable.
Solution Approach 2:
The patent segments the time series data into multiple groups based on seasonal lags (e.g., same day of week, same day of month, same day of year). Each group is processed by a separate encoder, dividing the complex task of capturing all seasonal correlations into multiple simpler, parallel sub-tasks. This segmentation reduces the dimensionality burden on any single encoder while maintaining comprehensive seasonal coverage.
2Measurement precision
If seasonal correlations are incorporated using traditional encoder-decoder approaches, then multi-step predictive ability is improved, but vanishing gradients occur reducing training effectiveness
Solution Approach 1:
The patent implements feedback mechanisms through the recurrent nature of the decoder units, where each decoder unit receives feedback from previous time steps and seasonal patterns. The summary vectors from encoders provide continuous feedback about seasonal correlations, enabling the model to adjust its predictions iteratively. This feedback loop prevents gradient vanishing by maintaining strong temporal and seasonal connections throughout the sequence.
Solution Approach 2:
The patent performs preliminary encoding of seasonal correlation patterns before the main prediction process. By pre-computing summary vectors that capture seasonal relationships, the model prepares relevant information in advance, reducing the computational burden during training and preventing gradient vanishing that would occur if seasonal correlations were learned simultaneously with predictions.
3Adaptability or versatility
If many-to-many RNN architecture is used for multi-step prediction, then the model can process sequential data, but multi-step predictive ability remains poor
Solution Approach 1:
The patent introduces a new dimension by processing time series data through multiple parallel encoders that operate on different seasonal lag groups simultaneously. This dimensional expansion allows the model to capture multi-step dependencies without relying solely on sequential RNN processing. The encoders operate in a different dimensional space (seasonal lag groups) rather than purely temporal sequences, enabling better multi-step prediction.
Solution Approach 2:
The patent merges the outputs from multiple parallel encoders with the RNN decoder architecture, combining the sequential processing capability of RNNs with the parallel seasonal correlation extraction of encoders. This merging creates a hybrid architecture where encoders handle seasonal pattern recognition in parallel while RNNs manage temporal sequences, achieving both adaptability and predictive accuracy.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The disclosure herein relates to a method and system for time series forecasting incorporating seasonal correlations using scalable architecture. The scalable architecture comprises parallel encoders, a neural network layer and a decoder. The neural network layer is either an attention layer or RNN layer. Each of the encoders and the decoder comprises multiple sequential encoder and decoder units, respectively. The parallel encoders encode seasonal correlations in a time series to generate summary vectors which are then processed along with state of a previous decoder unit by the neural network layer to generate a feature vector whose size is independent of order of seasonality of the time series. The feature vector is then processed by a next decoder unit to forecast a seasonal time series data at a subsequent time step. This process is repeated for all the decoder units to train the deep neural network model for time series forecasting.