Coreset Generation for Vision Model Pretraining
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large vision models trained on generic images exhibit poor performance on domain-specific images due to the significant differences between Internet and domain-specific image datasets, leading to suboptimal performance in real-world applications.
Innovation Solution
A method for generating a training set using a coreset of feature vectors, where each vector summarizes an image, iteratively adding vectors with maximum distance to ensure diversity, and using this coreset to train a machine learning model for accurate feature prediction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If foundation models are trained on generic images from datasets like ImageNet and COCO, then the models achieve good performance on general tasks, but they exhibit poor performance on domain-specific images
Solution Approach 1:
The patent segments the training data into a coreset subset that is specifically curated for domain-specific images, separating it from the general training data. This coreset is generated by selecting representative images that capture the essential characteristics of the target domain, allowing the model to be fine-tuned on domain-specific data without retraining on all generic data from scratch.
Solution Approach 2:
The patent changes the parameter of data representation by converting images into feature vectors using a pre-trained model, then selecting a coreset based on the distances between these vectors. This transformation allows for efficient selection of diverse and representative images that best capture domain-specific characteristics, improving adaptability while managing data quantity.
2Reliability
If all images from a large dataset are used for training, then the model achieves comprehensive coverage, but the training time and computational resources required become excessive
Solution Approach 1:
The patent extracts a coreset subset from the full training dataset by selecting only the most representative images based on feature vector distances. This extraction process identifies and isolates the critical samples that provide the most valuable training information, allowing the model to achieve comprehensive coverage with a much smaller subset of data, thereby significantly reducing training time.
Solution Approach 2:
Instead of using all available training data (excessive action), the patent applies partial action by selecting only the necessary portion (coreset) that provides sufficient coverage. The coreset generation algorithm ensures that the selected subset is just large enough to achieve the desired training coverage, avoiding the waste of computational resources on redundant samples.
3Productivity
If images are selected randomly from the dataset, then the selection process is simple and fast, but the diversity and representativeness of the training set are insufficient
Solution Approach 1:
The patent replaces the mechanical/simple random selection process with a computational approach using feature vector calculations and distance metrics. By substituting random selection with a systematic method that computes distances between feature vectors and selects images based on their representativeness and diversity, the process achieves both speed and precision in training set construction.
Data Source
AI summary
An application generates feature vectors from a plurality of images, each feature vector summarizing an image of the plurality. The application selects a seed vector from the feature vectors and adds the seed vector to a coreset. The application computes a distance metrics, each distance metric measuring a distance between the seed vector and a given one of the plurality of feature vectors, and adds a feature vector having a largest distance metric relative to all other ones of the distance metrics to the coreset. The application determines a next feature vector to use as a next seed vector based on a nearest neighbor search, and iteratively adds additional ones of the feature vectors to the coreset until a predefined coverage is achieved. The application generates a training set using images of the coreset, the training set used to train a machine learning model to predict features of input images.