Generative Model for Novel Object Detection in Digital Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for object detection and semantic segmentation in digital images struggle to effectively classify novel or unknown objects, as they often require extensive retraining and are limited in handling open-set scenarios where new classes are introduced.
Innovation Solution
A method and device that utilize a classifier and generative model to determine classes for digital images by synthesizing objects or modifying parts of images using techniques like random noise sampling or latent space interpolation, allowing for the creation of augmented training data that includes novel objects, and employing pre-trained models like Projected GANs or MaskGIT for object manipulation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a classifier is trained on existing training data to detect known object classes, then detection accuracy for known classes is improved, but the ability to detect novel or unknown classes deteriorates
Solution Approach 1:
The system performs preliminary actions by generating synthetic training data containing novel objects before actual detection tasks. The generative model creates augmented training datasets with unseen object classes, allowing the classifier to be pre-trained on a broader range of object variations. This preliminary data preparation enables the classifier to better adapt to novel classes without requiring extensive retraining when new classes are introduced.
2Measurement precision
If extensive retraining is performed to handle new object classes, then classification accuracy for new classes is improved, but training time and computational resources increase
Solution Approach 1:
The system performs preliminary actions by generating synthetic training data containing novel objects before actual detection tasks. The generative model creates augmented training datasets with unseen object classes, allowing the classifier to be pre-trained on a broader range of object variations. This preliminary data preparation enables the classifier to better adapt to novel classes without requiring extensive retraining when new classes are introduced.
Solution Approach 2:
The system creates copies of existing training data through the generative model, which synthesizes new training examples by transforming and augmenting existing data. This copying approach generates diverse variations of objects including novel classes, providing the classifier with sufficient training examples without requiring collection and annotation of extensive new real-world data, thereby reducing training time and computational resources.
3Adaptability or versatility
If the training dataset is expanded to include more object classes, then the system's versatility is improved, but data annotation complexity and cost increase
Solution Approach 1:
The system creates copies of existing training data through the generative model, which synthesizes new training examples by transforming and augmenting existing data. This copying approach generates diverse variations of objects including novel classes, providing the classifier with sufficient training examples without requiring collection and annotation of extensive new real-world data, thereby reducing training time and computational resources.
Solution Approach 2:
The generative model performs self-service by automatically generating synthetic training data with novel object classes without requiring manual data collection and annotation. The model autonomously creates diverse object variations and augments the training dataset, eliminating the need for expensive and time-consuming manual annotation processes while expanding the number of detectable classes.
Data Source
Figure 1~2
AI summary
A device and a method for determining a class for at least a part of a digital image, wherein the method comprises providing (202) a classifier for a first class and a second class, determining (206) a digital image comprising an object of the second class in at least the part of the digital image, determining (208) the class for at least the part of the digital image with the classifier, wherein determining (206) the digital image comprises determining the object of the second class with a generative model depending on a label for the first class and/or depending on at least one pixel representing an object of the first class.