ML Model Generation Using Date Split Distribution Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge in generating machine learning models lies in inadvertently causing distributional shifts across training, test, and validation datasets due to sub-optimal choices of time split variables, which degrades model performance across different customers and geographies.
Innovation Solution
The solution involves automatically determining the optimal date variable for time splits by analyzing distribution differences between training and testing/validation datasets using vector markers and distance metrics, such as Manhattan or Euclidean distances, to minimize distribution shifts and enhance model performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If time dependent data is split using conventional methods, then the training and testing process can be completed, but distributional shifts occur between training and testing datasets degrading model performance
Solution Approach 1:
The patent applies parameter changes by systematically varying the date variable used for splitting the time dependent data. Instead of using a fixed date variable, the method evaluates multiple different date variables to determine which one minimizes distributional shifts between training and testing sets, thereby optimizing model performance while maintaining distribution consistency
Solution Approach 2:
The patent implements feedback by calculating distributional shift metrics between training and testing datasets for each date variable split, then using this feedback to select the optimal date variable. This iterative process of splitting, evaluating distribution consistency, and selecting the best parameter ensures reliable model performance
2Adaptability or versatility
If multiple date variables are evaluated to minimize distributional shifts, then model generalization improves, but the complexity of data processing increases
Solution Approach 1:
The patent applies self-service by automatically evaluating and selecting the optimal date variable through computational processes. The system autonomously calculates distributional shifts for multiple date variables and selects the best one without requiring manual intervention, thereby improving model generalization while managing processing complexity through automation
Solution Approach 2:
The patent implements preliminary action by pre-evaluating multiple date variables and their resulting distributional shifts before final model training. This advance analysis identifies the optimal splitting strategy in advance, allowing the model to be trained with confidence that distributional consistency will be maintained, thus improving generalization while containing complexity through upfront computation
Data Source
AI summary
Embodiments generate a machine learning (“ML”) model. Embodiments receive training data, the training data including time dependent data and a plurality of dates corresponding to the time dependent data. Embodiments date split the training data by two or more of the plurality of dates to generate a plurality of date split training data. For each of the plurality of date split training data, embodiments split the date split training data into a training dataset and a corresponding testing dataset using one or more different ratios to generate a plurality of train/test splits. For each of the train/test splits, embodiments determine a difference of distribution between the training dataset and the corresponding testing dataset. Embodiments then select the train/test split with a smallest difference of distribution and train and test the ML model using the selected train/test split.