DNN Mixer for Non-Linear Multi-Variate Time-Series Forecasting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional forecasting models are limited by their reliance on linear transformations and fixed procedures, which fail to accurately capture the complex relationships between multi-variate time-series data, leading to suboptimal prediction performance.
Innovation Solution
The implementation of a Deep Neural Network (DNN) mixer that learns to combine multi-variate time-series inputs in a non-linear manner, adjusting weights based on dataset-specific relationships and using stochastic optimization to improve forecasting accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If linear transformation techniques (PCA, VAR) are used to reduce dimensionality of multi-variate time-series data, then the input size is reduced and processing is simplified, but the forecasting accuracy deteriorates because these methods cannot capture complex non-linear relationships between variables
Solution Approach 1:
The patent replaces traditional mechanical linear transformation methods (PCA, VAR) with a neural network-based system that performs non-linear transformations. The neural network learns complex relationships between multiple time-series variables through training, substituting the rigid linear algebraic operations with adaptive non-linear function approximation, thereby capturing intricate patterns that linear methods miss while maintaining computational feasibility.
Solution Approach 2:
The patent changes the fundamental parameter of transformation from linear to non-linear. By using neural network activation functions and multiple layers, the system transforms the input data through non-linear operations that can model complex relationships. This parameter change allows the system to adapt to different data distributions and capture higher-order interactions between variables, significantly improving forecasting accuracy over linear methods.
2Ease of operation
If fixed procedure transformations are applied to all dataset instances, then the processing method is simple and consistent, but the adaptability to dataset-specific relationships deteriorates, resulting in suboptimal forecasts
Solution Approach 1:
The patent introduces dynamics into the transformation process by using a trained neural network that adapts its parameters based on the specific dataset. Instead of applying a fixed transformation procedure, the system learns optimal transformation parameters through training on the actual data, allowing it to adapt to dataset-specific relationships while maintaining a consistent processing framework during deployment.
Solution Approach 2:
The neural network performs self-adjustment by automatically learning the appropriate transformation parameters from the data itself. The system serves itself by identifying patterns and relationships within the dataset and configuring its internal parameters accordingly, eliminating the need for manual configuration or fixed procedures while maintaining operational consistency through the learned model.
3Productivity
If matrix-based dimensionality reduction with fixed weights is used, then the transformation is computationally efficient, but the ability to learn dataset-specific relationships is lost, reducing forecasting performance
Solution Approach 1:
The patent applies preliminary action by training the neural network offline before deployment. The computationally intensive learning process is performed in advance to establish the optimal transformation parameters, after which the trained model can be efficiently applied to new data. This separates the heavy computational burden of learning from the efficient application phase, maintaining both accuracy and operational efficiency.
Solution Approach 2:
The patent creates a copied representation of the complex relationships through the trained neural network weights. Once the network learns the optimal transformation during training, this learned transformation can be copied and applied repeatedly to new datasets with the same characteristics, achieving both the accuracy of learned relationships and the efficiency of applying fixed parameters during deployment.
Data Source
AI summary
Deep Neural Networks (DNNs) for forecasting future data are provided. In one embodiment, a non-transitory computer-readable medium is configured to store computer logic having instructions that, when executed, cause one or more processing devices to receive, at each of a plurality of Deep Neural Network (DNN) forecasters, an input corresponding to a time-series dataset of a plurality of input time-series datasets. The instructions further cause the one or more processing devices to produce, from each of the plurality of DNN forecasters, a forecast output and provide the forecast output from each of the plurality of DNN forecasters to a DNN mixer for combining the forecast outputs to produce one or more output time-series datasets.