Planar Homography Free View Interpolation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image-based rendering techniques face challenges in achieving high-resolution free view interpolation, especially on mobile devices with limited computation resources and uncontrolled reference images, due to complexities in camera calibration and artifacts from depth and view disparities.
Innovation Solution
A novel Bayesian model applying Lie group theory to 3D space for geometric registration and photometric regulation, enabling full 6DOF registration and interpolation with limited input images, which treats significant disparities and allows for zoom-in, zoom-out, and rotation effects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If 3D reconstruction based rendering is applied to generate high resolution novel views, then viewing angle flexibility is improved, but camera calibration complexity increases significantly
Solution Approach 1:
The patent extracts the essential geometric transformation parameters (planar homography) from the complex 3D reconstruction process. Instead of performing full 3D scene reconstruction with multiple view geometry, the method extracts only the 2D planar transformation parameters needed for the specific application, significantly simplifying the calibration process while maintaining the ability to generate novel views.
Solution Approach 2:
The patent inverts the traditional approach by starting with 2D planar image transformations rather than 3D scene reconstruction. Instead of reconstructing 3D points and projecting them to new views, the method directly computes 2D homography transformations from reference images to generate novel views, reversing the conventional pipeline and reducing complexity.
2Use of energy by moving object
If image stitching based approaches are applied for free view interpolation, then computation resources are reduced, but resolution enhancement capability is lost
Solution Approach 1:
The patent merges the advantages of both 3D reconstruction and image stitching approaches. It combines the computational efficiency of image stitching with the resolution enhancement capability of super-resolution techniques by integrating planar homography-based image registration with multi-scale image fusion, achieving both low computational overhead and high output resolution.
Solution Approach 2:
The patent introduces a multi-scale dimension to the image stitching process. By processing images at multiple resolutions and fusing them in a pyramid structure, the method enhances the output resolution beyond that of individual input images, adding a dimensional aspect (scale/resolution) that transforms standard stitching into a super-resolution capability.
3Measurement precision
If structure-from-motion algorithms are applied for precise camera calibration, then registration accuracy is improved, but processing time increases
Solution Approach 1:
The patent applies partial action by computing only the necessary planar homography parameters rather than performing complete structure-from-motion reconstruction. This partial computation approach achieves sufficient registration accuracy for planar scenes without the excessive processing time required for full 3D reconstruction, extracting only the essential transformation parameters needed.
Solution Approach 2:
The patent changes the parameter space from 3D structure-from-motion parameters (camera positions, 3D point coordinates) to 2D planar homography parameters. This parameter transformation reduces the computational complexity and processing time while maintaining registration accuracy for planar structures, as fewer parameters need to be estimated and optimized.
4Ease of manufacture
If redundant information in overlapping areas is not utilized, then image stitching simplicity is maintained, but resolution enhancement is prevented
Solution Approach 1:
The patent ensures continuous utilization of redundant information across overlapping image regions through multi-scale image fusion. By processing images at multiple resolutions and continuously fusing information from overlapping areas in the pyramid structure, the method maintains stitching simplicity while continuously extracting useful information to enhance resolution, avoiding information waste.
Solution Approach 2:
The patent performs preliminary action by pre-processing images into a multi-scale pyramid structure before fusion. This preliminary organization of image data at different resolutions enables efficient utilization of redundant information during the fusion stage, preparing the data structure in advance to facilitate resolution enhancement without complicating the overall stitching process.
Data Source
AI summary
A method and system is proposed to create a generative model to interpolate any view of a planar scene given a sequence of reference views and a synthesis view that is optimized by the marginalization of photometric regulation, and geometric registration parameters. According to one aspect of the claimed subject matter, a technique is proposed to combine information from varying input camera poses. Planar homography based image super resolution in free view interpolation for planar structure is applied to the combined information. Non-redundant information is combined in such a manner that the high resolution and free view problems in traditional 2D based image-based rendering techniques are overcome.