Latent Dataset Compression Using Similarity Graph Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional machine learning systems rely on full annotated datasets for training, leading to substantial computational overhead and inefficiencies in adapting to new or shifting target domains.
Innovation Solution
A structured selection process that reduces the size of annotated datasets by encoding images into latent vectors, constructing a similarity graph, identifying connected components, and selecting representative samples using a divergence classifier to form a reduced dataset for training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full annotated datasets are used for training, then model performance is maintained, but computational overhead increases substantially
Solution Approach 1:
The patent extracts only the most representative samples from the full annotated dataset by encoding images into latent vectors, constructing similarity graphs, identifying connected components, and selecting representative samples using a divergence classifier. This extraction process creates a reduced dataset that maintains model performance while substantially reducing computational overhead during training.
2Loss of information
If full annotated datasets are used for training, then representational value is preserved, but training efficiency decreases
Solution Approach 1:
The patent segments the full annotated dataset into connected components based on latent vector similarity, then selects representative samples from each component. This segmentation approach ensures that the reduced dataset maintains diverse representational value across different data clusters while significantly improving training efficiency by reducing the total number of samples that need to be processed.
3Quantity of substance
If dataset size is reduced, then computational resources are reduced, but adaptability to new domains may worsen
Solution Approach 1:
The patent changes the parameter of dataset composition by using a divergence classifier to select samples that best represent the target domain distribution. This parameter change ensures that the reduced dataset maintains high adaptability to new domains by prioritizing samples with distributional relevance rather than simply reducing dataset size arbitrarily.
4Loss of time
If representative samples are selected using divergence classifier, then training speed improves, but selection complexity increases
Solution Approach 1:
The patent performs preliminary actions by pre-computing latent vectors for all images and constructing the similarity graph before the actual sample selection process. This preliminary processing organizes the data in a way that makes the subsequent divergence-based selection more efficient, thereby improving training speed while managing selection complexity through structured preprocessing.
Data Source
AI summary
An example operation may include at least one of determining, by a transformer encoder trained on annotated image-text data, first latents for a first dataset stored in a memory, and second latents for a second dataset stored in the memory, generating a similarity matrix based on comparisons between the first latents and the second latents, constructing a graph comprising nodes corresponding to the first latents and edges based on pairwise similarity exceeding a threshold, identifying connected components in the graph and selecting, from each component, at least one latent having a highest score from a classifier trained to approximate divergence between the first dataset and the second dataset, forming a reduced dataset comprising the at least one latent, providing the reduced dataset to a model training module, and training an image classifier using the reduced dataset and the second dataset.