ML Model Retraining Trigger via Multivariate Data Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models struggle to determine when retraining is needed due to changes in inter-related data variables, as current methods focus on individual variable changes rather than collective changes across groups of variables.
Innovation Solution
Implement a system where a second machine learning model monitors collective changes across all data variables used in training, employing multivariate analysis and clustering algorithms to identify new scenarios or modifications to existing scenarios, thereby determining the need for retraining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If individual data variable monitoring is used, then implementation simplicity is maintained, but detection precision of model retraining needs deteriorates
Solution Approach 1:
The patent combines multiple individual data variable monitoring streams into a unified multivariate analysis framework. By merging the monitoring of inter-related data variables into a single collective analysis system, the patent achieves both implementation feasibility and improved detection precision for identifying when model retraining is needed.
Solution Approach 2:
The patent creates a universal monitoring system that handles multiple data variables simultaneously through multivariate analysis. This multi-functional approach allows the system to detect collective changes across different variables and their inter-relationships, providing comprehensive detection precision while maintaining a unified implementation structure.
2Measurement precision
If multivariate analysis with clustering algorithms is implemented, then detection precision of collective data changes improves, but device complexity increases
Solution Approach 1:
The patent segments the complex multivariate analysis into distinct functional components: data collection module, multivariate analysis module, clustering algorithm module, and alert generation module. This segmentation allows the sophisticated detection capabilities to be implemented through modular, manageable components that can be developed and maintained independently.
Solution Approach 2:
The patent introduces intermediary processing layers between raw data input and final retraining recommendations. The multivariate analysis acts as an intermediary that transforms raw data into meaningful patterns, and the clustering algorithms serve as another intermediary layer that identifies scenario changes. These intermediaries manage the complexity by breaking down the analysis into sequential, manageable steps.
3Reliability
If monitoring of all data variables is performed, then reliability of model performance assessment improves, but loss of computational resources increases
Solution Approach 1:
The patent implements partial monitoring through selective focus on inter-related data variables that have the most significant impact on model performance. Rather than continuously analyzing all possible variables at full depth, the system monitors key variables and their relationships, achieving reliable detection of retraining needs with reduced computational overhead.
Solution Approach 2:
The patent applies local quality by focusing computational resources on specific regions of the data space where changes are most likely to indicate retraining needs. The clustering algorithms identify specific scenarios or patterns that require attention, allowing the system to maintain high reliability in assessing model performance while concentrating computational effort where it is most needed rather than uniformly across all data.
Data Source
AI summary
A need to retrain a machine learning model is determined based on identification of either a new scenario (i.e., new grouping/population of data variables) or a modification to an existing/trained scenario (i.e., addition and/or deletion of data variables in an already trained scenario). Change occurring across all of the data variables used in training the machine learning model are monitored as opposed to monitoring only the data variables on an individual basis. When concurrent change occurs across multiple inter-related data variables, the need to add or modify a scenario (i.e., retrain a machine learning model) is identified.