Masked Multi-Step Forecasting for Time Series Accuracy

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing multi-step multivariate time series forecasting methods fail to accurately incorporate known future information, leading to errors in predictions due to recursive structures and difficulties in training direct methods, especially for long forecasting horizons.

Innovation Solution

The Masked Multi-Step Multivariate Forecasting (MMMF) system uses a self-supervised learning framework that integrates both past and future information by applying a masking technique to neural network models, allowing for flexible training and inference of multi-step forecasts.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If recursive methods are used for multi-step forecasting, then the forecasting framework can be implemented, but errors accumulate over long forecasting horizons

Engineering Contradiction:
Improveforecasting accuracyVSAvoidforecasting horizon
Core Design Contradiction:
ReliabilityVSDuration of action of moving object

Solution Approach 1:

The patent segments the forecasting task into multiple independent direct prediction steps, where each step predicts a specific future time step directly from historical data without relying on previous predictions. This segmentation prevents error accumulation by eliminating the recursive dependency chain that causes errors to propagate and amplify over longer forecasting horizons.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent introduces an intermediary training mechanism using masked future information during the training phase. By masking certain future time steps and training the model to predict them, the system creates an intermediate learning stage that improves the model's ability to make accurate direct predictions without the harmful effects of recursion during actual forecasting.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Reliability

If direct methods are used for multi-step forecasting, then forecasting accuracy can be maintained, but training becomes harder especially for large forecast horizons

Engineering Contradiction:
Improveforecasting accuracyVSAvoidtraining difficulty
Core Design Contradiction:
ReliabilityVSEase of manufacture

Solution Approach 1:

The patent applies preliminary action by performing masked training on future information before actual forecasting. During training, certain future time steps are masked and the model learns to predict them using historical data and unmasked future information. This preliminary training phase prepares the model to make accurate direct predictions during inference without requiring complex training procedures, as the model has already learned the prediction task in advance.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent implements feedback mechanisms during training by comparing model predictions with actual masked future values. This feedback loop allows the model to learn from its prediction errors and continuously improve its direct prediction capability, making training more effective and less difficult even for large forecast horizons.

Inventive Principle:
Principle #23Feedback

3Reliability

If future information is incorporated into forecasting, then prediction accuracy improves, but the forecasting framework complexity increases

Engineering Contradiction:
Improveprediction accuracyVSAvoidforecasting framework complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent extracts and utilizes only the necessary future information (predictor variables) that are needed for improved predictions, while keeping the forecasting framework simple. By selectively taking out and using relevant future predictor variables without incorporating entire future sequences, the system achieves accuracy improvements without proportionally increasing framework complexity.

Inventive Principle:
Principle #2Taking out (Extraction)

Solution Approach 2:

The patent creates a universal forecasting framework that can handle both traditional recursive forecasting and the new direct forecasting with future information using the same model architecture. This multi-functional approach allows the system to incorporate future information when available while maintaining simplicity, as the same framework serves multiple forecasting purposes without requiring separate complex systems.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Data Source

PatentUS20240054348A1Systems and method for masked multi-step multivariate time series power forcasting and estimation
Publication Date: 2024.02.15 GENERAL ELECTRIC CO
  • US20240054348A1 patent drawing
  • US20240054348A1 patent drawing
  • US20240054348A1 patent drawing

AI summary

A system includes a computing device including at least one processor in communication with at least one memory. The at least one processor is programmed to (a) store a plurality of historical time series data; (b) randomly select a sequence; (c) randomly select a mask length for a mask for the selected sequence; (d) apply the mask to the selected sequence, wherein the mask is applied to the plurality of forecast variables in the selected sequence; (e) execute a model with the masked selected sequence to generate predictions for the masked forecast variables; (f) compare the predictions for the masked forecast variables to the actual forecast variables in the selected sequence; (g) determine if convergence occurs based upon the comparison; and (h) if convergence has not occurred, update one or more parameters of the model and return to step b.