Image Hallucination for Deep Neural Network Patch Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer vision methods require large numbers of carefully labeled patches to achieve good performance, and there is a lack of effective methods for training and learning patch descriptions, especially for learning high-dimensional embedding spaces where matching patches are closer than non-matching patches.
Innovation Solution
The system generates image hallucinations to train a deep neural network, using a curricular learning framework to progressively learn invariant representations without requiring large numbers of high-quality training data, by synthesizing views of patches from different random viewpoints and simulating transformations like rotations and illumination changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If structure-from-motion is used to create sparse 3D models with many views of a single 3D point, then large numbers of matching patches are generated, but the process becomes costly and requires large numbers of carefully labelled patches
Solution Approach 1:
The patent uses image hallucination to synthesize virtual views of patches from different viewpoints, creating synthetic training data copies that mimic real-world variations. This allows the system to generate large numbers of matching patches without requiring extensive manual labeling or complex structure-from-motion processes, as the synthetic patches are generated through computational transformation of existing patch data
Solution Approach 2:
The system pre-processes patches by extracting features and generating hallucinated views in advance to create a training dataset. This preliminary action prepares the training data before the actual neural network training begins, reducing the need for complex real-time processing during training and enabling efficient learning of patch matching
2Measurement precision
If neural networks are used to learn high-dimensional embedding spaces, then matching patches are learned to be closer than non-matching patches, but immensely large numbers of matching patches are required
Solution Approach 1:
The patent generates synthetic copies of patches through image hallucination, creating virtual viewpoints and transformations from existing patch data. This copying approach allows the neural network to learn from numerous synthetic examples without requiring proportionally large numbers of real labeled patches, as the synthetic data provides diverse training examples that preserve the geometric and semantic relationships needed for accurate matching
Solution Approach 2:
The system transforms patches by changing parameters such as viewpoint angle, illumination conditions, and geometric transformations to generate hallucinated views. These parameter changes create diverse training examples that help the neural network learn robust high-dimensional embeddings, enabling accurate patch matching with fewer real training patches by leveraging the variability introduced through parameter transformation
Data Source
AI summary
Computer vision systems and methods for machine learning using image hallucinations are provided. The system generates image hallucinations that are subsequently used to train a deep neural network to match image patches. In this scenario, the synthesized changes serve in the learning of feature-embedding that captures how a patch of an image might look like from a different vantage point. In addition, a curricular learning framework is provided which is used to automatically train the neural network to progressively learn more invariant representations.