AI Model Selection via Misclassification Probability Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI models vary in accuracy when processing the same input data, leading to inconsistent results due to differences in architecture, hyper-parameters, and training techniques, making it challenging to select the most appropriate model for optimal processing.
Innovation Solution
A method is developed to select an optimal AI model by identifying misclassified data from a training dataset, training an additional model on this data, and adjusting weight values based on misclassification probabilities to enhance processing accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple AI models with different architectures and training techniques are used, then processing accuracy varies, but model selection becomes complex and time-consuming
Solution Approach 1:
The patent changes the parameter of model evaluation from general accuracy metrics to specific misclassification probability parameters. By training an additional model on misclassified data and using it to calculate misclassification probabilities for each candidate model, the system transforms the model selection problem into a parameter-based comparison, resolving the complexity of selecting among multiple models with different architectures and training techniques.
Solution Approach 2:
The patent introduces an additional trained AI model as an intermediary between the candidate AI models and the selection process. This intermediary model processes misclassified data and generates misclassification probability assessments, serving as a mediator that objectively evaluates and compares different candidate models, thereby simplifying the selection complexity while maintaining accuracy differentiation.
2Reliability
If AI models are trained on complete training datasets, then general accuracy is achieved, but performance on difficult cases remains inconsistent
Solution Approach 1:
The patent segments the training process into two distinct phases: initial training on the complete training dataset to achieve general accuracy, and subsequent targeted training on misclassified data to improve difficult case performance. This segmentation allows the additional model to specialize in handling misclassified cases while the original models maintain their general competence, thereby improving overall classification reliability and misclassification accuracy.
Solution Approach 2:
The patent performs preliminary classification using multiple candidate models before final decision-making. By first obtaining misclassification probabilities from the additional model trained on difficult cases, the system can preliminarily assess which candidate model is most likely to succeed, thereby improving reliability before committing to a final classification decision.
3Productivity
If traditional accuracy metrics are used for model evaluation, then overall performance is measured, but specific case performance cannot be optimized
Solution Approach 1:
The patent changes the evaluation parameter from aggregate accuracy metrics to case-specific misclassification probabilities. By using the additional trained model to generate probability assessments for each candidate model on the specific input data, the system enables adaptability to specific cases while maintaining processing efficiency through automated probability-based selection rather than manual evaluation.
Solution Approach 2:
The patent implements a feedback mechanism where the additional trained model provides misclassification probability feedback about each candidate model's performance on the current input. This feedback loop allows the system to adaptively select the most appropriate model based on real-time probability assessments, enhancing versatility for different data cases while maintaining efficient automated processing.
Data Source
AI summary
A method of selecting an artificial intelligence (AI) model based on input data to select an AI model capable of correctly obtaining a result corresponding to data to be classified, e.g., a classification result indicating one of at least one class, from among a plurality of AI models, and a display device for performing the method.