Active Sampler for Neural Network Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing methods for training feature descriptors in computer vision systems face challenges due to their dependency on initializations and selected matching/non-matching image pairs, leading to inefficient CPU usage and power consumption, as well as poor feature representation and slow convergence in training.
Innovation Solution
An active training method for neural networks that selects image pairs based on similarity values and gradually increases the threshold of viewpoint difference, using an active sampler to update weighting parameters and reduce CPU usage, power consumption, and improve feature representation by focusing on domain knowledge rather than relying on the current learned model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional model-based hard sample selection is used, then training focuses on difficult cases, but training efficiency decreases and CPU usage increases
Solution Approach 1:
The patent changes the selection criterion from model-based loss values to domain-based similarity values. By using viewpoint difference thresholds and similarity metrics inherent to the problem domain rather than training-dependent loss values, the system selects hard samples more efficiently without requiring iterative model feedback, thus improving training efficiency while maintaining effectiveness.
Solution Approach 2:
The patent introduces domain knowledge (viewpoint difference, similarity values) as an intermediary between the training data and the model. This intermediary provides an objective criterion for selecting hard samples that is independent of the model's current state, avoiding the inefficiency of model-dependent selection while ensuring reliable training focus on difficult cases.
2Measurement precision
If model-dependent sample selection is used, then samples are selected based on current model performance, but training becomes slower and convergence is delayed
Solution Approach 1:
The patent performs preliminary sample selection based on domain knowledge before training begins. By pre-categorizing samples according to viewpoint difference and similarity metrics, the system eliminates the need for iterative model-dependent selection during training, thereby reducing training time while maintaining accurate identification of hard samples through the predetermined thresholds.
3Ease of operation
If initialization-dependent training is used, then early training performance varies with initialization, but robustness decreases
Solution Approach 1:
The patent extracts the dependency on model initialization by removing the model from the sample selection process. By selecting hard samples based solely on domain-based similarity values and viewpoint differences, the system eliminates the coupling between initialization and sample selection, making training more robust to different initializations while maintaining operational simplicity.
4Productivity
If conventional batch training is used, then all samples are processed uniformly, but computational resources are wasted on easy samples
Solution Approach 1:
The patent applies local quality by differentiating sample processing based on their difficulty level. Instead of uniform processing, the system selectively focuses computational resources on hard samples identified through domain-based criteria, while easier samples receive less attention or are processed differently, thereby reducing overall CPU usage while maintaining training throughput.
Data Source
AI summary
A system for active-training a neural network includes an input interface to receive a set of images, a memory to store an active sampler, a feature generator and a subset of the images selected from the set of the images, the subset of the images having similarity values based on a predetermined feature domain information, at least one processor to train the feature generator by use of the active sampler. The active sampler is configured to perform first-selecting, from the subset of the images, a pair of images as matching pair images according to a threshold of similarity values, second-selecting a pair of images from another subset of the images, feeding the matching pair images and the unmatched pair images to the feature generator, updating weighting parameters and increasing the threshold according to preset values an output interface to render the weighting parameters of the feature generator.