Frozen Feature Extraction for Open-World Object Identification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models used in image processing can only identify objects that were labeled in the training data, failing to recognize objects not included in the training images, which limits their accuracy and applicability in real-world scenarios.
Innovation Solution
A method that employs a machine learning model comprising a feature extraction model and a classification scoring model, where the classification scoring model is updated based on ground truth classification scores, while preventing updates to the feature extraction model, allowing the model to distinguish between labeled and unlabeled objects and improve identification accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning models are trained using labeled images, then objects that have been labeled in the training data can be identified, but objects that have not been labeled in the training data cannot be identified
Solution Approach 1:
The machine learning model is segmented into two independent parts: a feature extraction model that processes image data and a classification scoring model that assigns labels. This segmentation allows the classification scoring model to be updated with new labels without retraining the entire model, enabling the system to adapt to new objects while maintaining existing identification accuracy.
Solution Approach 2:
The classification scoring model is designed to be dynamically updatable without requiring complete model retraining. By updating only the classification scoring model with new ground truth labels, the system can adapt to newly encountered objects in real-time while preserving the feature extraction capabilities learned from original training data.
2Adaptability or versatility
If the entire machine learning model is retrained when new objects are introduced, then identification accuracy for new objects improves, but training time and computational resources increase
Solution Approach 1:
The classification scoring model is extracted as a separate, independently updatable component from the feature extraction model. This extraction allows only the classification scoring model to be retrained when new objects are introduced, while the feature extraction model remains frozen. This significantly reduces training time and computational resources compared to retraining the entire model.
3Adaptability or versatility
If the feature extraction model is updated with new training data, then the model can adapt to new objects, but the model may forget previously learned object characteristics
Solution Approach 1:
The model is segmented into a feature extraction model that maintains stable, frozen parameters for consistent feature representation, and a classification scoring model that is updated with new data to adapt to diverse objects. This segmentation prevents the feature extraction model from changing its learned representations while allowing the classification system to adapt to new object categories.
Solution Approach 2:
Different parts of the system have different update characteristics: the feature extraction model maintains stable local quality for feature representation, while the classification scoring model has local quality optimized for adapting to new object classes. This local differentiation ensures that feature stability is preserved while classification adaptability is achieved.
Data Source
AI summary
A method, device, and medium are provided for processing an image using a machine learning model that identifies at least one candidate object from an image. The model comprises: a feature extraction model for describing an association between the image and a feature of the at least one candidate object; and a classification scoring model for describing an association between the feature and a classification score of the at least one candidate object. An update parameter associated with the classification scoring model is determined based on the classification score of the at least one candidate object and a ground truth classification score of at least one ground truth object in the image. The classification scoring model is updated based on the update parameter associated with the classification scoring model. The feature extraction model is prevented from being updated with the update parameter associated with the classification scoring model.