ML Model Retraining via Consistent Evaluation Metrics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The growing volume of data in enterprise IT centers makes traditional data analysis methods infeasible, and machine learning model training is hindered by inconsistencies between training approaches and the lack of comparable evaluation metrics for retraining decisions.
Innovation Solution
A method and system for machine learning model training that involves receiving a trained model, evaluating its quality using feedback data, and triggering a retraining process if below a threshold, with distinct phases of k-fold cross-validation to ensure comparable accuracy assessments across training and retraining iterations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional data analysis methods are used, then the analysis process is simple and easy to implement, but the methods become infeasible when analyzing large volumes of data
Solution Approach 1:
The patent replaces traditional mechanical data analysis methods with machine learning models that automatically learn patterns from data. The system uses trained ML models to perform complex analysis tasks without manual intervention, enabling the processing of large data volumes while maintaining feasibility through automated pattern recognition and prediction algorithms.
2Reliability
If machine learning model retraining is performed frequently to maintain accuracy, then model quality is improved, but computational resources and time are consumed
Solution Approach 1:
The patent implements a feedback mechanism where the system continuously monitors model performance using evaluation metrics on incoming data. When performance degradation is detected, the system automatically triggers retraining only when necessary, rather than performing frequent retraining regardless of actual need. This feedback-driven approach maintains model accuracy while minimizing unnecessary computational resource consumption.
Solution Approach 2:
The system dynamically adjusts the retraining frequency based on actual model performance and data characteristics. Rather than using a fixed retraining schedule, the system adapts its retraining behavior to match the actual needs of the model, triggering retraining only when performance thresholds are violated. This dynamic approach optimizes the balance between maintaining accuracy and reducing time loss.
3Adaptability or versatility
If inconsistent evaluation metrics are used across training and retraining phases, then the evaluation process is flexible and adaptable, but comparability of results is lost
Solution Approach 1:
The patent establishes consistent evaluation metrics across all model training and retraining phases, creating an equipotential evaluation framework. The same performance thresholds, evaluation criteria, and measurement standards are applied uniformly whether evaluating an initially trained model or a retrained model. This ensures that results are directly comparable and that decisions about model redeployment are based on consistent, reliable measurements rather than phase-dependent variations.
Data Source
AI summary
In a method for machine learning model training, the method includes one or more processors receiving a trained original machine learning model, including related parameters and a set of training data with which the machine learning model has been trained. The method further includes one or more processors determining an original quality evaluation value for the trained original machine learning model using a first set of feedback data. The method further includes one or more processors, in response to determining that the quality evaluation value is below a quality threshold value, triggering a retraining process for the original machine learning model, the retraining process comprising a first retraining phase for a first machine learning model and a second retraining phase for a second machine learning model.