Deep Learning Image Stitching via Flow and Weight Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Image stitching technologies face challenges in addressing parallax distortion caused by varying distances between objects and capturing viewpoints, which affects the quality of stitched images.
Innovation Solution
A deep learning-based image stitching method that generates flow maps to move pixels of images, warping images based on these maps, and blends them using weight maps to correct parallax distortion, utilizing an L1 loss function and SSIM loss function for training to achieve robust stitching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional image stitching methods are used, then the stitching process is simple, but parallax distortion occurs due to distance differences between viewpoints
Solution Approach 1:
The patent introduces flow maps as an intermediary component that mediates between the input images and the stitching process. These flow maps encode pixel movement information and serve as a bridge to correct parallax distortion, allowing traditional simple stitching methods to achieve high-quality results by incorporating this intermediate representation layer
Solution Approach 2:
The patent transitions from traditional 2D image coordinate matching to a 3D flow map representation that incorporates depth information. By adding this dimensional aspect through flow maps that encode pixel displacement in multiple directions, the method resolves parallax distortion while maintaining stitching quality
2Manufacturing precision
If deep learning-based flow maps are generated to correct parallax, then stitching quality improves, but computational complexity increases
Solution Approach 1:
The patent performs preliminary action by pre-training the deep learning model on synthetic data with known ground truth flow maps. This pre-training phase prepares the model in advance to accurately predict flow maps for real images, reducing the need for complex iterative optimization during actual stitching operations
Solution Approach 2:
The patent uses synthetic copied images with known depth and flow information as training data. By creating artificial training pairs where the ground truth flow maps are known, the model learns to generalize this knowledge to real images, achieving high precision without requiring equally complex processing for each real image
3Measurement precision
If multiple loss functions are used for training, then model accuracy improves, but training time increases
Solution Approach 1:
The patent applies partial action by using a combination of L1 loss and SSIM loss functions during training, rather than exhaustively using all possible loss functions. This selective combination provides sufficient training signal to achieve accurate flow maps while avoiding the excessive computational burden of using multiple complex loss functions simultaneously
Data Source
AI summary
A deep learning-based image stitching method and apparatus are provided. The deep learning-based image stitching method includes receiving a plurality of images including a first image and a second image stitched to the first image, extracting a feature map of the plurality of images, generating a plurality of flow maps indicating movement information of pixels of the plurality of images, based on the feature map, generating a plurality of warping images in which the pixels of the plurality of images are moved, by applying the plurality of flow maps to the plurality of images, respectively, generating a plurality of weight maps for blending the plurality of warping images, based on the feature map, and generating a stitch image in which the first image and the second image are stitched, by blending the plurality of warping images based on the plurality of weight maps.