Machine Learning Model Updating via Feature Distribution Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models face accuracy deterioration due to concept drift, where data distribution changes over time, necessitating updates that are often delayed, leading to continued use of inaccurate models during the update process.
Innovation Solution
A machine learning system that includes a deterioration detection unit to identify when a model's accuracy declines and swiftly updates it by generating a drift-compatible model based on recent data, using a neural network with a high durability learning technique to maintain model accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the machine learning model is regularly retrained to maintain accuracy, then the model accuracy is improved, but the system stability and operational continuity deteriorate due to frequent updates
Solution Approach 1:
The patent applies preliminary action by proactively detecting concept drift before it significantly degrades model accuracy. The drift detection mechanism monitors data distribution changes and triggers model updates in advance, preventing accuracy deterioration rather than reacting to it. This allows the system to maintain accuracy while avoiding unnecessary frequent retraining, thus preserving system stability.
Solution Approach 2:
The patent implements feedback through a closed-loop system that continuously monitors both data distribution changes and model performance. The drift detection unit provides feedback about concept drift to the model update mechanism, which then adjusts the model accordingly. This feedback loop ensures updates occur only when necessary, balancing accuracy maintenance with system stability.
2Measurement precision
If the model update frequency is increased to track concept drift, then the model accuracy is improved, but the time required for model maintenance and the complexity of the system increases
Solution Approach 1:
The feedback mechanism monitors data distribution changes in real-time and triggers model updates only when concept drift is detected. This conditional update approach avoids unnecessary retraining operations, reducing the time spent on model maintenance while ensuring accuracy is maintained when it becomes necessary.
Solution Approach 2:
The patent changes the parameter of model update frequency from a fixed schedule to a dynamic, condition-based frequency. The update frequency adapts to the actual drift conditions in the data, increasing updates only when drift is detected and maintaining lower frequency when data distribution remains stable. This optimizes the balance between accuracy and maintenance time.
3Measurement precision
If the model is updated using all available training data, then the model accuracy is improved, but the computational resources and processing time required increase significantly
Solution Approach 1:
The patent extracts only the necessary subset of training data for model updates by identifying and utilizing only the data that reflects the detected concept drift. Rather than retraining on all available data, the system selectively uses relevant data samples that capture the distribution changes, significantly reducing computational resources while maintaining accuracy improvement.
4Measurement precision
If the drift detection threshold is set to be highly sensitive, then the detection accuracy is improved, but the number of false positives increases leading to unnecessary model updates
Solution Approach 1:
The patent dynamically adjusts the drift detection threshold parameter based on the characteristics of the data and the severity of detected drift. Rather than using a fixed sensitive threshold that causes false positives, the system adapts the threshold to distinguish between normal data variations and genuine concept drift, maintaining high detection accuracy while avoiding unnecessary updates that would reduce productivity.
Data Source
AI summary
A storage medium storing a machine learning program that causes a computer to execute a process, the process includes specifying first distribution of a feature of data calculated by a second machine learning model; determining whether or not output accuracy of the second machine learning model decreases based on the first distribution; when the determining determines that the output accuracy decreases, selecting, from the plurality of machine learning models, a fourth machine learning model that has second distribution of a feature of data input that is the most similar with third distribution of a feature of the training data among the plurality of the machine learning model, and generating the third machine learning model by updating a parameter of the fourth machine learning model based on a certain piece of the training data labeled based on the feature of the data.