Geometric Vision Pre-Training with Cross-View Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer vision systems face challenges in performing high-level geometric vision tasks due to the limited availability of annotated training data, making it difficult to effectively learn 3D geometry from unannotated data using unsupervised methods.
Innovation Solution
An unsupervised pre-training method is employed using cross-view alignment and cross-view completion techniques to train a pretext model on unannotated image pairs, learning spatial relationships between images from different viewpoints, followed by fine-tuning with annotated data to enhance performance on downstream geometric vision tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If supervised training approaches are used with annotated training data, then the machine learning model can achieve optimal outcomes for specific tasks, but the amount of available training data is limited due to the significant effort required for annotation
Solution Approach 1:
The patent applies preliminary action by performing unsupervised pre-training on large unannotated datasets before supervised fine-tuning on smaller annotated datasets. The encoder is first pre-trained to learn general image representations from unannotated data, then fine-tuned with annotated data for specific geometric vision tasks. This two-stage approach allows the model to benefit from both large unannotated datasets and smaller annotated datasets, resolving the contradiction between data quality and quantity.
2Quantity of substance
If unsupervised training approaches are used to leverage large unannotated data repositories, then the amount of training data increases, but the training is limited to tasks for which the optimal result can be determined without explicit annotation of ground-truth data
Solution Approach 1:
The patent segments the training process into two distinct phases: unsupervised pre-training phase and supervised fine-tuning phase. The encoder is first trained unsupervisedly on large unannotated datasets to learn general representations, then separately fine-tuned on smaller annotated datasets for specific geometric vision tasks. This segmentation allows the model to leverage large unannotated datasets while still achieving task-specific performance through the subsequent supervised phase.
Solution Approach 2:
The patent applies preliminary action by performing unsupervised pre-training on large unannotated datasets before supervised fine-tuning on smaller annotated datasets. The encoder is first pre-trained to learn general image representations from unannotated data, then fine-tuned with annotated data for specific geometric vision tasks. This two-stage approach allows the model to benefit from both large unannotated datasets and smaller annotated datasets, resolving the contradiction between data quality and quantity.
3Ease of manufacture
If a pretext machine learning model is trained unsupervisedly on unannotated image pairs, then the model can learn spatial relationships without annotation effort, but the model requires subsequent fine-tuning with annotated data to perform well on downstream geometric vision tasks
Solution Approach 1:
The patent applies preliminary action by performing unsupervised pre-training on large unannotated datasets before supervised fine-tuning on smaller annotated datasets. The encoder is first pre-trained to learn general image representations from unannotated data, then fine-tuned with annotated data for specific geometric vision tasks. This two-stage approach allows the model to benefit from both large unannotated datasets and smaller annotated datasets, resolving the contradiction between data quality and quantity.
Data Source
AI summary
A method includes: performing unsupervised pre-training of a model, the model including and a decoder including: obtaining a first image and a second image under different conditions or from different viewpoints; encoding, by the encoder, the first image into a representation of the first image and the second image into a representation of the second image; transforming the representation of the first image into a transformed representation; decoding, by the decoder, the transformed representation into a reconstructed image, where the transforming of the representation of the first image and the decoding of the transformed representation is based on the representation of the first image and the representation of the second image; and adjusting one or more parameters of at least one of the encoder and the decoder based on minimizing a loss; and fine-tuning the model, initialized with a set of task specific encoder parameters, for a geometric vision task.