Parallel Predictive Model Training with Convergence Termination
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing predictive modeling techniques require large volumes of training data and often struggle with efficiently selecting and training effective models, especially when dealing with diverse input data types, which can lead to resource-intensive processes and prolonged runtime.
Innovation Solution
A computer-implemented method that receives training data and executes multiple processes in parallel to generate multiple trained predictive models, determining convergence status and effectiveness scores to select the most effective model, allowing for dynamic model updates and resource optimization by terminating non-convergent processes and extending runtime for promising ones.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple training functions are executed in parallel to generate multiple predictive models, then model selection accuracy is improved, but computing resource consumption and memory usage increase
Solution Approach 1:
The patent applies partial action by executing multiple training functions in parallel but terminating processes that are unlikely to converge based on convergence status determination. This allows the system to benefit from parallel execution for improved model selection accuracy while avoiding the waste of computing resources on processes that will not produce useful results.
Solution Approach 2:
The system implements feedback mechanisms by determining convergence status of each training process and using this information to dynamically terminate non-convergent processes. The effectiveness score generation provides feedback on model quality, enabling the system to select the best model while managing resource consumption efficiently.
2Reliability
If training processes are allowed to run for extended runtime to ensure convergence, then model effectiveness is improved, but overall processing time increases
Solution Approach 1:
The patent applies preliminary action by determining convergence status early in the training process and terminating processes that are unlikely to converge. This preliminary assessment prevents wasteful extension of runtime for non-convergent processes while allowing sufficient time for processes that show convergence potential, thus balancing model effectiveness with processing time.
Solution Approach 2:
The system dynamically adjusts the runtime of training processes based on their convergence status. Rather than applying a static runtime to all processes, the system terminates non-convergent processes early and allows convergent processes to continue, creating a dynamic runtime management strategy that optimizes both effectiveness and time efficiency.
3Adaptability or versatility
If multiple predictive models are trained simultaneously, then model selection capability is improved, but memory usage increases
Solution Approach 1:
The patent applies the extraction principle by selectively retaining only the most effective predictive model based on effectiveness scores, while discarding other trained models. This allows the system to benefit from training multiple models simultaneously for improved selection capability, but reduces memory usage by storing only the best model rather than all trained models.
4Productivity
If convergence determination is performed frequently to identify non-convergent processes, then resource optimization is improved, but computational overhead increases
Solution Approach 1:
The system applies self-service by using the training processes themselves to generate convergence status information that determines their own continuation or termination. The convergence determination leverages information already being computed during training (such as loss function values or parameter changes) rather than requiring separate, complex convergence analysis, thus optimizing resources while minimizing additional computational overhead.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on one or more computer storage devices, for receiving training data for predictive modeling and executing multiple processes simultaneously to generate multiple trained predictive models using the training data and training functions. After executing the processes for an initial runtime, a convergence status of each process is determined that indicates a likelihood that the training function being executed will converge on the training data. Based on the determination, training functions are identified that are not likely to converge and processes that are executing these training functions are terminated. After an ultimate runtime has expired, processes that are still executing training functions that have not yet converged are terminated. An effectiveness score is generated for each of the trained predictive models that were successfully generated and a trained predictive model is selected based on the effectiveness scores.