Feature Extractor Confidence Measurement via Embedding Space
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing confidence measurement methods for deep neural network-based classification models face challenges in scalability and accuracy, particularly as the number of parameters and data increase, and struggle to effectively measure confidence for out-of-distribution data.
Innovation Solution
The method involves additional training of a feature extractor using a metric-learning-based loss function and cross-entropy-based loss function to ensure feature vectors for the same class are closer and different classes are farther apart in an embedding space, allowing for accurate confidence measurement of classification results by calculating distances in this space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional confidence measurement methods (iterative inference, Bayesian Neural Network, Monte-Carlo Dropout) are used, then confidence ranges can be presented, but computation time and resource requirements increase significantly as model parameters and data size increase
Solution Approach 1:
The patent extracts the confidence measurement function from the main classification model by using a separate, pre-trained feature extractor. This allows confidence measurement to be performed independently without requiring iterative inference or complex Bayesian computations on the full model, significantly reducing computation time while maintaining measurement accuracy.
Solution Approach 2:
The patent creates a simplified copy of the classification model's feature extraction capability through a separate feature extractor that is pre-trained on the same dataset. This copy can be used for confidence measurement without requiring the full computational overhead of the original model, enabling fast and accurate confidence assessment.
2Reliability
If confidence measurement uses in-distribution training data only, then the model is trained on intended classification data, but confidence measurement performance for out-of-distribution data cannot be guaranteed
Solution Approach 1:
The patent performs preliminary action by training the feature extractor on both in-distribution and out-of-distribution data before deployment. This pre-training on diverse data distributions enables the system to reliably measure confidence for both familiar and novel data types without requiring retraining when encountering out-of-distribution data.
Data Source
AI summary
A confidence measurement method according to an embodiment includes additionally training a feature extractor of a classification model trained using training data including a plurality of images, using the training data, such that feature vectors for images labeled with the same class among feature vectors for the plurality of images become closer to each other in an embedding space for the feature extractor, and measuring confidence of a classification result for an input image by the trained classification model using the additionally trained feature extractor.