Outlier Detection Model for Confident ML Data Samples
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models face challenges in maintaining confidence and accuracy when applied to unseen data, particularly in medical diagnosis where poor predictions can lead to incorrect clinical decisions due to the data-driven nature of these models.
Innovation Solution
A system that projects both training and unseen data onto a standard feature space, using an outlier detection model to classify data samples as either inliers or outliers, thereby determining confident data samples for ML models, which are expected to perform accurately, and filtering out unreliable predictions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If Bayesian dropout is used to estimate DNN model confidence, then prediction uncertainty can be quantified and segmentation accuracy can be improved, but training cost increases and model architecture becomes more complex
Solution Approach 1:
The patent extracts the confidence estimation function from the main DNN architecture by using a separate outlier detection model that operates independently. This allows confidence assessment without modifying the core model structure, resolving the contradiction between obtaining reliable confidence estimates and maintaining architectural simplicity.
Solution Approach 2:
The patent introduces an intermediary outlier detection model that mediates between the DNN model and the confidence assessment process. This intermediary component evaluates confidence by detecting outliers in feature space projections, providing reliable confidence estimates without integrating complex Bayesian dropout mechanisms into the main architecture.
2Measurement precision
If filtering strategies are used to remove low-confidence samples, then precision at given confidence level can be optimized, but the ability to handle visually indistinguishable unseen data deteriorates
Solution Approach 1:
The patent transitions from operating in the original data space to a projected feature space dimension. By projecting data onto a standard feature space and detecting outliers in this transformed dimension, the system can identify confident samples without being misled by visual similarities in the original space, thus maintaining precision while improving adaptability to unseen data.
Solution Approach 2:
The patent changes the parameter space in which confidence assessment occurs by using projected feature representations instead of raw data. This parameter transformation allows the outlier detection model to distinguish confident from uncertain samples based on their positional characteristics in feature space, enabling precise filtering while maintaining versatility for unseen data types.
Data Source
AI summary
Techniques are provided for determining confident data samples for machine learning (ML) models on unseen data. In one embodiment, a method is provided that comprises extracting, by a system comprising a processor, a feature vector for a data sample based on projection of the data sample onto a standard feature space. The method further comprises processing, by the system, the feature vector using an outlier detection model to determine whether the data sample is within a scope of a training dataset used to train a machine learning model, wherein the outlier detection model was trained using features extracted from the training dataset based on projection of data samples included in the training dataset onto the standard feature space.