ML Model Selection via Diversity Criteria
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Selecting a subset of automatically generated machine-learned models is challenging due to the trade-off between model correctness, performance, and diversity, and the complexity of optimizing these criteria effectively.
Innovation Solution
A system and method for selecting a subset of machine-learned models by comparing the output data of each pair of models to generate a diversity score, and then selecting models that collectively produce the most diverse outputs while considering correctness and performance metrics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple ML models are selected to improve correctness through ensemble methods, then model correctness is improved, but system complexity increases
Solution Approach 1:
The system automatically selects the optimal number of models for the ensemble by evaluating diversity scores and correctness metrics, eliminating the need for manual configuration of ensemble size and reducing system complexity while maintaining improved correctness
Solution Approach 2:
The system dynamically adjusts the number of models in the ensemble based on diversity criteria and performance metrics, optimizing the balance between correctness and complexity by changing the ensemble size parameter automatically
2Reliability
If more ML models are included in the ensemble, then model correctness is improved, but training time increases
Solution Approach 1:
The system selects a partial subset of available models that provides sufficient diversity and correctness improvement without requiring all models to be trained, reducing training time while maintaining adequate correctness through selective model inclusion based on diversity scores
3Reliability
If model diversity is increased to improve ensemble performance, then ensemble reliability is improved, but model selection complexity increases
Solution Approach 1:
The system uses diversity scores as feedback to automatically adjust model selection, comparing output data from different models and using the diversity metric to guide the selection process, thereby improving ensemble reliability while keeping the selection mechanism systematic and manageable
Solution Approach 2:
The system replaces manual model selection with an automated algorithm that computes diversity scores and selects models based on predefined criteria, substituting human judgment with a systematic computational approach that reduces selection complexity
Data Source
AI summary
Techniques for selecting machine-learned (ML) models using diversity criteria are provided. In one technique, for each ML model of multiple ML models, output data is generated based on input data to the ML model. Multiple pairs of ML models are identified, where each ML model in the multiple pairs is from the multiple ML models. For each pair of ML models in the multiple pairs of ML models: (1) first output data that was previously generated by a first ML model in the pair is identified; (2) second output data that was previously generated by a second ML model in the pair is identified; (3) a diversity value that is based on the first and second output data is generated; and (4) the diversity value is added to a set of diversity values. A subset of the multiple ML models is selected based on the set of diversity values.