Fully Convolutional Network for Visual Correspondence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing visual correspondence methods, particularly those using convolutional neural networks (CNNs), are inefficient as they rely on surrogate objective functions and patch-based similarity measures, leading to suboptimal feature extraction and high computational costs for tasks like stereo disparity and structure from motion, due to the need for O(n^2) feed-forward passes and lack of optimization for metric spaces.
Innovation Solution
A fully convolutional neural network (FCNN) with a correspondence contrastive loss and active hard negative mining is proposed, enabling efficient feature extraction and learning of metric spaces for visual correspondences, along with a convolutional spatial transformer for patch normalization, allowing for faster training and testing with reduced computational complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If patch-based similarity methods are used, then image patch similarity can be measured, but computational complexity increases to O(n^2) feed-forward passes
Solution Approach 1:
The patent segments the image into dense feature points and uses a fully convolutional network to extract features at each point independently, avoiding the need to compare all pairs of patches. This segmentation approach reduces computational complexity from O(n^2) to O(n) while maintaining measurement precision.
Solution Approach 2:
The patent replaces the mechanical patch-based comparison system with a neural network-based feature extraction system. By using convolutional neural networks to learn metric spaces directly, the system eliminates the need for exhaustive patch pairing and similarity computation, achieving both precision and efficiency.
2Productivity
If intermediate convolution features are used, then features can be extracted, but they are not optimized for visual correspondence tasks
Solution Approach 1:
The patent employs dynamic feature extraction by training the convolutional network to adaptively learn metric spaces through contrastive loss functions. The network dynamically adjusts feature representations based on correspondence relationships, ensuring both efficient extraction and high accuracy for visual correspondence tasks.
Solution Approach 2:
The patent changes the parameter space by transforming intermediate convolution features into optimized metric space features through contrastive loss training. This parameter transformation ensures that features are both efficiently extractable and precisely suitable for visual correspondence, resolving the contradiction between extraction speed and accuracy.
3Quantity of substance
If dense feature extraction is performed, then correspondence density increases, but computational cost increases
Solution Approach 1:
The patent creates a universal fully convolutional network that can extract dense features efficiently for multiple correspondence tasks simultaneously. The network's multi-functionality allows it to handle both geometric and semantic correspondences with a single model, increasing correspondence density without proportionally increasing computational energy consumption.
Data Source
AI summary
A computer-implemented method for training a convolutional neural network (CNN) is presented. The method includes extracting coordinates of corresponding points in the first and second locations, identifying positive points in the first and second locations, identifying negative points in the first and second locations, training features that correspond to positive points of the first and second locations to move closer to each other, and training features that correspond to negative points in the first and second locations to move away from each other.