Dynamic Predictive Model Repository with Incremental Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing predictive models require large volumes of training data and expertise to manage and update, making it challenging for client systems to efficiently train and maintain effective models, especially when data changes over time.
Innovation Solution
A computer-implemented system that dynamically updates a repository of trained predictive models using a training data queue and repository of training functions, allowing for incremental or batch data updates, and selecting the most effective models based on effectiveness scores, enabling continuous model improvement without requiring extensive client-side resources or expertise.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If large volumes of training data are used to train predictive models, then model accuracy and effectiveness are improved, but the complexity and resource requirements for managing and updating models increase
Solution Approach 1:
The system segments the training data into a queue structure and divides the model repository into static and updateable models. This segmentation allows incremental processing of data and selective updating of only relevant models, reducing overall management complexity while maintaining accuracy.
Solution Approach 2:
The system introduces dynamic characteristics by allowing predictive models to be updated incrementally as new data arrives, rather than requiring complete retraining. The model repository transitions from static to dynamic, with models automatically updated based on incoming training data while maintaining others as static references.
2Reliability
If predictive models are updated frequently with new data, then model effectiveness is maintained, but the time and resources required for retraining increase
Solution Approach 1:
The system performs preliminary actions by pre-processing and queuing training data before it is needed for model updates. Training data is accumulated in a queue structure, allowing the system to prepare updates in advance and apply them efficiently when triggered, reducing real-time retraining time.
Solution Approach 2:
The system maintains continuous model effectiveness through incremental updates rather than periodic complete retraining. The useful action of model updating continues progressively as data arrives, ensuring models remain effective without the time loss of repeated full retraining cycles.
3Adaptability or versatility
If multiple types of predictive models are maintained in the repository, then adaptability to different data types is improved, but the complexity of managing and selecting models increases
Solution Approach 1:
The system implements universality by creating a unified model repository structure that handles multiple predictive model types through a common interface. The repository manages diverse models (static and updateable, different algorithms) using consistent operations, reducing management complexity while maintaining adaptability.
Solution Approach 2:
The system applies local quality by treating different model types with appropriate specialized handling within the unified repository. Static models are managed differently from updateable models, and different predictive algorithms are optimized for their specific requirements, while maintaining overall system simplicity through standardized access patterns.
4Reliability
If training data is stored indefinitely to maintain model accuracy, then model reliability is improved, but the storage requirements and data management overhead increase
Solution Approach 1:
The system implements discarding and recovering by selectively retaining only the training data needed for model updates while discarding redundant historical data. The queue mechanism allows the system to recover and reuse recent training data for incremental updates, reducing overall storage requirements while maintaining accuracy.
Solution Approach 2:
The system applies partial action by storing and processing only the portion of training data that is actually needed for model updates, rather than retaining all historical data indefinitely. This selective data management reduces storage volume while maintaining sufficient data for model accuracy.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on one or more computer storage devices, for training and retraining predictive models. A series of training data sets are received and added to a training data queue. In response to a first condition being satisfied, multiple retrained predictive models are generated using the training data queue, multiple updateable trained predictive models obtained from a repository of trained predictive models, and multiple training functions. In response to a second condition being satisfied, multiple new trained predictive models are generated using the training data queue, at least some training data stored in a training data repository and training functions. The new trained predictive models include static trained predictive models and updateable trained predictive models. The repository of trained predictive models is updated with at least some of the retrained predictive models and new trained predictive models.