EC-DARNNS for Multivariate Time Series Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multivariate time series prediction methods, such as linear techniques, probabilistic models, and deep neural networks, fail to capture nonlinear relationships and structural information in hidden features, and cannot effectively handle multiple types of historical data.
Innovation Solution
An Ensemble of Clustered dual-stage attention-based Recurrent Neural Networks (EC-DARNNS) is employed to decompose time series into raw, shape, and trend components, with attention-based encoders selecting relevant driving series and clustering hidden features using a temporal attention-based decoder to predict future time steps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If linear methods such as ARMA or ARIMA models are used for multivariate time series prediction, then the model structure is simple and easy to implement, but the model cannot capture the underlying nonlinear relationships in the data
Solution Approach 1:
The patent segments the time series data into multiple components (trend component, seasonal component, and residual component) using decomposition techniques. This allows the model to handle different characteristics of the data separately, capturing nonlinear relationships in each component while maintaining overall model interpretability and managing complexity.
Solution Approach 2:
The patent employs a hybrid composite model that integrates multiple modeling approaches: decomposition techniques, nonlinear autoregressive models with exogenous variables (NARX), and neural networks. This composite structure combines the strengths of different methods to capture nonlinear relationships while maintaining manageable complexity through modular design.
2Reliability
If probabilistic models such as Gaussian processes are used for multivariate time series prediction, then the model can handle uncertainty, but the model employs a pre-defined nonlinear form that may not be flexible enough to capture complex relationships
Solution Approach 1:
The patent uses dynamic neural networks with adaptive learning mechanisms that can adjust their internal parameters and structures based on the input data characteristics. This dynamic adaptability allows the model to capture complex nonlinear relationships without being constrained by pre-defined functional forms, while still providing uncertainty estimates through the probabilistic nature of neural network predictions.
3Adaptability or versatility
If existing deep neural networks such as LSTM are used for multivariate time series prediction, then the model can capture nonlinear relationships, but the model does not consider the structural info of hidden features and cannot handle multiple types of historical data
Solution Approach 1:
The patent introduces attention mechanisms that add a new dimension to the LSTM architecture by computing attention scores across different time steps and feature dimensions. This attention dimension allows the model to selectively focus on relevant structural information in the hidden features, preventing information loss while maintaining the nonlinear capture capability of LSTMs. The attention weights provide an additional layer of information processing that highlights important structural patterns.
4Productivity
If existing deep neural networks are used for multivariate time series prediction, then the model can process large amounts of data, but the model cannot handle multiple types of historical data effectively
Solution Approach 1:
The patent designs a universal input layer architecture that can accept multiple types of historical data (e.g., past time series values, aggregated historical statistics, external covariates) through a unified interface. This universal structure uses embedding layers and feature transformation modules that adaptively process different data types, allowing the model to handle diverse historical data effectively while maintaining high data processing capacity through efficient batch operations.
Data Source
AI summary
A method for multivariate time series prediction is provided. Each time series from among a batch of multiple driving time series and a target time series is decomposed into a raw component, a shape component, and a trend component. For each decomposed component, select a driving time series relevant thereto from the batch and obtain hidden features of the selected driving time series, by applying the batch to an input attention-based encoder of an Ensemble of Clustered dual-stage attention-based Recurrent Neural Networks (EC-DARNNS). Automatically cluster the hidden features in a hidden space using a temporal attention-based decoder of the EC-DARNNS. Each Clustered dual-stage attention-based RNN in the Ensemble is dedicated and applied to a respective one of the decomposed components. Predict a respective value of one or more future time steps for the target series based on respective prediction outputs for each of the decomposed components by the EC-DARNNS.