Time-Series Prediction Model With Adaptive Forgetting Rate
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing time-series prediction models face challenges with non-stationary data, leading to overfitting or underfitting due to inaccurate forgetting factors and inefficient parameter optimization, especially in industries like agriculture and finance, where environmental variations and data complexity are high.
Innovation Solution
The method employs an adaptive forgetting rate and hyper-forgetting rate, using recursive pseudo-inversion to update model parameters efficiently, thereby avoiding overfitting and underfitting by determining the pseudo-inverse of the Hessian of the loss function, which facilitates accurate and efficient optimization of model parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional fixed forgetting rate is used in time-series prediction models, then the model structure is simple and computation is fast, but the model cannot adapt to non-stationary data leading to overfitting or underfitting
Solution Approach 1:
The patent implements dynamic forgetting rates that adapt over time based on data characteristics. The forgetting rate is no longer fixed but evolves dynamically to match the non-stationary nature of the time-series data, allowing the model to automatically adjust between forgetting old data and retaining historical patterns based on current data conditions
Solution Approach 2:
The patent changes the forgetting rate parameter dynamically during the learning process. By adjusting the forgetting rate parameter based on the observed data patterns and model performance, the system adapts to non-stationary conditions without requiring complex architectural changes to the underlying model structure
2Measurement precision
If more hyper-parameters are tuned to improve model accuracy, then prediction accuracy improves, but the optimization process becomes computationally expensive and time-consuming
Solution Approach 1:
The patent performs preliminary identification and selection of the most influential hyper-parameters before the main optimization process. By pre-identifying which parameters have the greatest impact on model performance, the system focuses computational resources on tuning only those critical parameters, avoiding wasted computation on less important parameters
Solution Approach 2:
The system implements self-adjusting mechanisms where the model automatically tunes its own hyper-parameters based on performance feedback. The forgetting rate and other parameters are adjusted automatically during training based on observed data patterns, reducing the need for extensive manual hyper-parameter search and experimentation
3Adaptability or versatility
If recursive parameter updates are performed frequently to adapt to changing data patterns, then model adaptability improves, but computational cost and error accumulation increase
Solution Approach 1:
The patent implements periodic updates of the forgetting rate and model parameters rather than continuous updates. By updating parameters at specific intervals or triggered by certain conditions (such as significant changes in data patterns), the system maintains adaptability to environmental variations while reducing the computational burden of frequent parameter recalculations
Data Source
AI summary
Systems and methods for modelling time-series data includes testing a testing model with a plurality of hyper-forgetting rates to select a best performing hyper-forgetting rate. A model optimization is tested using the best performing hyper-forgetting rate with the testing model to test combinations of hyper-parameters to select a best performing combination of hyper-parameters. An error of the model is determined using the model optimization. Model parameters are recursively updated according to the least squares regression by determining a pseudo-inverse of a Hessian of the least squares regression at a current time stamp according to a projection of the time-series data at the current time stamp and the pseudo-inverse of the Hessian at a previous time-stamp to determine an optimum model parameter. A next step behavior of the time-series data is predicted using the optimum model parameter. The next step behavior is stored in a database for access by a user.