Prediction Model Training via Periodic Step Size Reset
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Prediction models trained on historical data struggle to accurately reflect recent data distributions due to gradual learning over long time durations, leading to outdated knowledge in rapidly changing environments.
Innovation Solution
A method for managing prediction models by dividing the training procedure into time periods, obtaining gradient information for each time slot, determining an offset, and using it to periodically reset the step size for updating model parameters, emphasizing recent data and reducing historical data impact.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the prediction model is trained gradually on historical data covering a long time duration, then the model accumulates comprehensive knowledge, but the model cannot accurately reflect recent data distributions
Solution Approach 1:
The patent implements periodic resetting of the step size parameter at fixed time intervals during training. This periodic action allows the model to periodically shift focus from historical data to recent data, enabling it to adapt to changing data distributions while maintaining the benefit of long-term training. The reset mechanism creates a rhythmic pattern of learning that balances comprehensive knowledge accumulation with responsiveness to recent trends.
Solution Approach 2:
The step size parameter is made dynamic rather than static, changing its value based on the current time slot and offset within the training period. This dynamic adjustment allows the training process to adapt its learning rate and focus differently at different stages, prioritizing recent data when the offset indicates newer time slots while still incorporating historical data during other phases, thus resolving the contradiction between comprehensive knowledge and recent adaptability.
2Adaptability or versatility
If the model uses fixed step size for parameter updates, then the training process is simple and stable, but the model cannot emphasize recent data over historical data
Solution Approach 1:
The patent changes the step size parameter dynamically based on the time slot offset during training. By modifying this key parameter according to the current position within the training period, the model can emphasize recent data when the offset corresponds to newer time slots. This parameter change approach adds adaptability without requiring fundamental changes to the training architecture, thus balancing complexity with functionality.
Solution Approach 2:
The patent pre-defines the step size reset strategy and offset calculations before training begins. The periodic reset pattern and offset computations are established in advance, allowing the training process to automatically emphasize recent data without requiring complex real-time decisions during training. This preliminary setup reduces runtime complexity while achieving the desired data prioritization.
Data Source
AI summary
There are proposed methods, devices, and computer program products for prediction model management. In the method, gradient information associated with the prediction model is obtained based on sample data for a time slot in a predetermined time period. An offset of the time slot in the predetermined time period is acquired. A step size is determined for updating a parameter of the prediction model based on the gradient information, the offset, and historical gradient information that is determined based on historical sample data for a group of historical time slots before the time slot. With these implementations, the whole training procedure may be divided into multiple time period and each time period may further include multiple time slots. During each time period, the offset may be used to control the importance of the historical gradient information and the gradient information in determining the step size.