Prediction Model Replacement via Inference Data Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Prediction models in machine learning systems degrade in accuracy over time due to changes in input data characteristics, such as location or time, leading to a need for periodic retraining, which is time-consuming and results in using models with poor accuracy during the update period.
Innovation Solution
A system that includes a storage module for storing inference data, a retraining module for training a retraining model using inference data, and a determination module for comparing the performance of the prediction model and the retraining model, allowing for timely replacement of the prediction model with the retraining model when abnormalities are detected, thereby minimizing performance degradation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If periodic retraining is performed to update the prediction model, then the model accuracy can be maintained, but a considerable amount of time is required and the existing model with poor accuracy must be used during the update period
Solution Approach 1:
The patent applies preliminary action by continuously accumulating inference data and preparing retraining models in advance before the actual model replacement is needed. The system pre-processes data and builds candidate models during off-peak times, so when retraining is finally executed, the ready-to-deploy model is already prepared, minimizing the time the system operates with degraded accuracy.
Solution Approach 2:
The patent segments the retraining process into multiple parallel tracks: data collection continues independently, candidate models are trained in parallel using different datasets, and performance evaluation is conducted separately. This segmentation allows the system to prepare multiple model versions simultaneously rather than sequentially, reducing the total retraining time while maintaining accuracy.
2Adaptability or versatility
If the prediction model is updated using latest data, then the model can reflect current environment changes, but the existing model with poor accuracy has to be used during the update period
Solution Approach 1:
The system performs preliminary actions by continuously collecting and storing inference data in a database, preparing it in advance for retraining. When environment changes occur, the pre-collected data can be immediately used to train updated models without waiting for new data accumulation, enabling rapid adaptation while maintaining service continuity.
Solution Approach 2:
The patent creates a copy of the prediction model (the retraining model) that can be independently developed and tested against the same input data. This copy allows the system to prepare an updated version in parallel while the original model continues to serve production needs, enabling smooth transition when the updated model is deployed.
3Reliability
If inference data is stored and used for retraining, then the model can be updated with latest data, but the system complexity increases with additional modules for storage, retraining, and performance comparison
Solution Approach 1:
The patent implements multi-functionality by designing the retraining module to perform multiple functions: it trains new models, evaluates their performance against input data, compares them with the existing model, and prepares deployment decisions. The storage module serves both as a database for inference data and as a repository for historical training data. This consolidation reduces the need for separate specialized components.
Solution Approach 2:
The system performs self-service by automatically comparing the performance of the retraining model against the existing model using the same input data, and automatically deploying the superior model. The determination module autonomously evaluates performance metrics and makes deployment decisions without requiring manual intervention, reducing operational complexity while maintaining high accuracy.
Data Source
AI summary
A system for enhancing a prediction model according to an embodiment includes a storage module configured to receive and store inference data for input data from a prediction model, a retraining module configured to train a retraining model using retraining data including the inference data, and a determination module configured to compare performances of the prediction model and the retraining model and replace the prediction model with the retraining model according to the comparison result.