Image Alignment via CNN Feature Maps and Lucas-Kanade Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image alignment techniques face challenges in efficiently estimating parametric motion models between relative images, particularly in applications like panoramic image stitching and visual odometry, where accuracy and convergence speed are critical.
Innovation Solution
The method employs a convolutional neural network to generate feature maps from input and template images, combined with an iterative Lucas-Kanade network to estimate motion parameters, enabling precise image alignment by minimizing the sum of squared errors between feature maps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional image alignment techniques are used to estimate parametric motion models, then the process can be completed, but the convergence speed is slow and the number of iterations required is large
Solution Approach 1:
The patent replaces traditional iterative optimization methods (mechanical/mathematical systems) with a convolutional neural network that has been trained to directly estimate motion parameters. The CNN learns the mapping from image patches to motion parameters through training data, substituting the iterative Lucas-Kanade optimization process with a single forward pass through the network, thereby achieving faster convergence in fewer iterations.
2Measurement precision
If traditional feature matching methods are used, then image alignment can be achieved, but the accuracy and precision of motion parameter estimation are insufficient
Solution Approach 1:
The patent substitutes traditional feature detection and matching algorithms with a convolutional neural network that directly regresses motion parameters from image patches. The CNN leverages learned feature representations and hierarchical processing to achieve higher precision in motion estimation, outperforming hand-crafted feature descriptors and matching algorithms in both accuracy and robustness.
3Reliability
If iterative optimization algorithms are used for motion estimation, then reasonable results can be obtained, but the computational complexity and processing time are high
Solution Approach 1:
The patent performs preliminary action by training the convolutional neural network offline on a large dataset of image patches and corresponding motion parameters. This pre-training phase captures the complex relationships between image content and motion, allowing the network to make accurate predictions during actual image alignment tasks without requiring iterative optimization at runtime, thus reducing computational complexity while maintaining reliability.
Data Source
AI summary
An image processing method includes generating, by a processing component, a first input feature map based on an input image using a first convolutional neural network; generating, by the processing component, a first template feature map based on a template image using the first convolutional neural network; generating, by the processing component, a first estimated motion parameter based on an initial motion parameter, the first input feature map and the first template feature map using an iterative Lucas-Kanade network; and performing, by the processing component, image alignment between the input image and the template image based on the first estimated motion parameter.