Multi-Layer Perceptron Time Series Mixer Architecture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional time series forecasting models, such as ARIMA and univariate models, are inadequate for handling complex real-world data with multiple interdependent covariates and additional information like static and time-varying features, often suffering from overfitting and inefficiency compared to multivariate models like recurrent neural networks and transformers.
Innovation Solution
A time series mixer system based on multi-layer perceptrons (MLPs) that alternates between time-domain and feature-domain operations, using time-dependent weights to capture temporal patterns and leverage cross-variate information, while applying two-dimensional normalization to maintain scale across features and time steps, reducing computational complexity and model size.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multivariate models like recurrent neural networks and transformers are used to handle complex real-world data with multiple interdependent covariates, then the ability to leverage cross-variate information is improved, but the risk of overfitting increases and computational complexity rises
Solution Approach 1:
The patent segments the processing of multivariate time series data into distinct time-domain and feature-domain operations. By alternating between these two types of operations, the model processes temporal patterns and cross-variate relationships separately, reducing the risk of overfitting while maintaining the ability to leverage cross-variate information. This segmentation allows the model to handle complex real-world data with multiple interdependent covariates more reliably.
2Measurement precision
If multivariate models like recurrent neural networks and transformers are used to model complex temporal dependencies, then the ability to capture temporal patterns is improved, but the computational cost and model size increase
Solution Approach 1:
The patent divides the computational process into time-domain MLP operations for capturing temporal patterns and feature-domain MLP operations for processing cross-variate information. This segmentation reduces computational complexity by avoiding the heavy machinery of recurrent neural networks and transformers while maintaining the ability to model complex temporal dependencies through efficient alternating operations.
Solution Approach 2:
The patent changes the parameters of the model by using multi-layer perceptrons with alternating time-domain and feature-domain operations instead of traditional recurrent neural network or transformer architectures. This parameter change reduces model size and computational cost while preserving the ability to capture temporal patterns in multivariate time series data.
3Device complexity
If traditional univariate models like ARIMA are used for time series forecasting, then the model simplicity is maintained, but the ability to handle multiple interdependent covariates and additional information is limited
Solution Approach 1:
The patent creates a universal model that can handle both univariate and multivariate time series forecasting by using multi-layer perceptrons with alternating time-domain and feature-domain operations. This universal approach maintains model simplicity while gaining the ability to handle multiple interdependent covariates and additional information like static and time-varying features, effectively bridging the gap between traditional univariate models and complex multivariate models.
Data Source
AI summary
The present disclosure provides an architecture for time series forecasting. The architecture is based on multi-layer perceptrons (MLPs), which involve stacking linear models with non-linearities between them. In this architecture, the time-domain MLPs and feature-domain MLPs are used to perform both time-domain and feature-domain operations in a sequential manner, alternating between them. In some examples, auxiliary data is used as input, in addition to historical data. The auxiliary data can include known future data points, as well as static information that does not vary with time. The alternation of time-domain and feature-domain operations using linear models allows the architecture to learn temporal patterns while leveraging cross-variate information to generate more accurate time series forecasts.