Masked Multi-Step Forecasting for Time Series Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-step multivariate time series forecasting methods fail to accurately incorporate known future information, leading to errors in predictions due to recursive structures and difficulties in training direct methods, especially for long forecasting horizons.
Innovation Solution
The Masked Multi-Step Multivariate Forecasting (MMMF) system uses a self-supervised learning framework that integrates both past and future information by applying a masking technique to neural network models, allowing for flexible training and inference of multi-step forecasts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If recursive methods are used for multi-step forecasting, then the forecasting framework can be implemented, but errors accumulate over long forecasting horizons
Solution Approach 1:
The patent segments the forecasting task into multiple independent direct prediction steps, where each step predicts a specific future time step directly from historical data without relying on previous predictions. This segmentation prevents error accumulation by eliminating the recursive dependency chain that causes errors to propagate and amplify over longer forecasting horizons.
Solution Approach 2:
The patent introduces an intermediary training mechanism using masked future information during the training phase. By masking certain future time steps and training the model to predict them, the system creates an intermediate learning stage that improves the model's ability to make accurate direct predictions without the harmful effects of recursion during actual forecasting.
2Reliability
If direct methods are used for multi-step forecasting, then forecasting accuracy can be maintained, but training becomes harder especially for large forecast horizons
Solution Approach 1:
The patent applies preliminary action by performing masked training on future information before actual forecasting. During training, certain future time steps are masked and the model learns to predict them using historical data and unmasked future information. This preliminary training phase prepares the model to make accurate direct predictions during inference without requiring complex training procedures, as the model has already learned the prediction task in advance.
Solution Approach 2:
The patent implements feedback mechanisms during training by comparing model predictions with actual masked future values. This feedback loop allows the model to learn from its prediction errors and continuously improve its direct prediction capability, making training more effective and less difficult even for large forecast horizons.
3Reliability
If future information is incorporated into forecasting, then prediction accuracy improves, but the forecasting framework complexity increases
Solution Approach 1:
The patent extracts and utilizes only the necessary future information (predictor variables) that are needed for improved predictions, while keeping the forecasting framework simple. By selectively taking out and using relevant future predictor variables without incorporating entire future sequences, the system achieves accuracy improvements without proportionally increasing framework complexity.
Solution Approach 2:
The patent creates a universal forecasting framework that can handle both traditional recursive forecasting and the new direct forecasting with future information using the same model architecture. This multi-functional approach allows the system to incorporate future information when available while maintaining simplicity, as the same framework serves multiple forecasting purposes without requiring separate complex systems.
Data Source
AI summary
A system includes a computing device including at least one processor in communication with at least one memory. The at least one processor is programmed to (a) store a plurality of historical time series data; (b) randomly select a sequence; (c) randomly select a mask length for a mask for the selected sequence; (d) apply the mask to the selected sequence, wherein the mask is applied to the plurality of forecast variables in the selected sequence; (e) execute a model with the masked selected sequence to generate predictions for the masked forecast variables; (f) compare the predictions for the masked forecast variables to the actual forecast variables in the selected sequence; (g) determine if convergence occurs based upon the comparison; and (h) if convergence has not occurred, update one or more parameters of the model and return to step b.


