Explainable Active Learning for Robust Vision Model Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI/ML systems in industrial applications face challenges in achieving industry-grade quality due to a lack of methods that holistically enhance model robustness, reliability, and generalization capabilities, despite significant effort in image dataset annotation, with traditional evaluation focusing on accuracy and F1 score rather than robustness and reliability.
Innovation Solution
A computer-implemented training method for machine-learned vision models that incorporates explainability-based active learning, using uncertainty sampling and insertion/deletion games to select and refine training data, generating saliency maps for improved model understanding and robustness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional active learning approaches using uncertainty or diversity-based metrics are employed, then functional performance (accuracy) is improved, but model robustness and reliability remain insufficient
Solution Approach 1:
The patent changes the selection criteria parameters from traditional uncertainty/diversity metrics to explainability-based metrics (SHAP values, LIME explanations, counterfactuals). This parameter change enables the selection of data points that reveal model behavior patterns and edge cases, thereby improving both accuracy and robustness simultaneously by focusing on instances that most impact model decision-making
Solution Approach 2:
The patent implements a feedback mechanism where model explanations and predictions are analyzed to identify uncertain or controversial cases, which are then selected for annotation. The annotated data is fed back into the training set, and the process repeats with updated model versions. This closed-loop feedback ensures continuous improvement of both functional performance and reliability through iterative refinement based on explainability insights
2Reliability
If extensive manual annotation is performed to improve model quality, then model robustness and reliability are enhanced, but time consumption and annotation costs increase significantly
Solution Approach 1:
The patent extracts and prioritizes only the most informative and challenging data points for manual annotation based on explainability metrics. By using SHAP values, LIME explanations, and counterfactual analysis, the system identifies a small subset of high-value samples that maximum impact model improvement, thereby reducing the overall annotation workload while maintaining or enhancing model robustness
Solution Approach 2:
The system performs self-service by automatically generating explanations, calculating uncertainty metrics, and prioritizing samples for annotation without requiring manual review of all data points. The explainability-infused active learning framework autonomously identifies which samples need human annotation, reducing the time and effort required for the annotation process
3Measurement precision
If focus is placed on achieving high accuracy through traditional metrics, then functional performance is improved, but non-functional characteristics (robustness, reliability, generalization) are neglected
Solution Approach 1:
The patent fundamentally changes the evaluation and selection parameters from accuracy-centric metrics to explainability-centric metrics. By using SHAP values, LIME explanations, and counterfactual analysis as selection criteria, the system identifies data points that reveal model behavior patterns, edge cases, and potential failure modes, thereby improving generalization capability and adaptability while maintaining accuracy
Solution Approach 2:
The patent adds a new dimension to the active learning process by incorporating explainability metrics alongside traditional accuracy metrics. This multi-dimensional approach considers not only whether predictions are correct but also how and why the model makes predictions, enabling selection of samples that improve both functional performance and non-functional characteristics like robustness and generalization
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Presented is a system (100) and a computer-implemented training method (200) for machine-learned (ML) vision models to perform training on a training dataset that comprises a plurality of unlabeled images and a plurality of labeled images. The method (200) comprising of training a machine-learned model using the plurality of unlabeled images and obtaining a respective label for each unlabeled datapoint to transform the unlabeled images into weak supervision labeled images. Then, calculating an uncertainty sampling metrics based on a confidence score associated with the training dataset and determining an area under an explainability metrics based on a change in the confidence score. Thereafter, obtaining a selection of the unlabeled datapoints for training of the ML model based on the confidence scores and a model explainability and retraining the machine-learned model based on the training dataset comprising of the plurality of labeled images, wherein the labeling of the images is supervised by an annotator.