Multi-view Synthesis Using Coordinate Alignment Matrix and Common Background Layer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 3D display technologies relying on polarization require glasses, which are inconvenient and limit widespread adoption, and existing methods for generating multiple viewpoints in autostereoscopic displays are either costly or computationally intensive, leading to inconsistencies and inefficiencies.
Innovation Solution
A method that generates a coordinate alignment matrix and a common background layer to efficiently repair occlusions and synthesize views, allowing for consistent and cost-effective generation of multiple virtual views from a limited camera array, using techniques like exemplar-based inpainting and placement matrices for hole filling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple cameras are used to capture each separate viewpoint, then viewpoint coverage is improved, but device complexity and cost increase significantly
Solution Approach 1:
The patent creates virtual views by copying and warping image data from a limited number of captured views. Instead of using multiple physical cameras, the system generates synthetic copies of scenes from different angles through coordinate transformation and warping operations, thereby achieving multiple viewpoint coverage without proportionally increasing camera quantity.
Solution Approach 2:
The patent introduces a coordinate alignment matrix as an intermediary that maps pixels between different views. This matrix serves as a mediator to transfer image information from captured views to synthesized views, enabling consistent multi-view generation without requiring direct capture from multiple camera positions.
2Manufacturing precision
If view synthesis methods process missing regions pixel-by-pixel or use patch-based inpainting, then manufacturing precision is improved, but processing time increases
Solution Approach 1:
The patent performs coordinate alignment and generates the coordinate alignment matrix before actual view synthesis. This preliminary organization of pixel correspondence relationships enables faster processing during the synthesis stage, as the mapping relationships are pre-established rather than computed in real-time during rendering.
Data Source
AI summary
An embodiment of the invention is a method that provides common data and enforces multiple view consistency. In a preferred embodiment, a coordinate alignment matrix is generated after initial warped views are formed. The matrix is used efficiently to repair holes/occlusions when synthesizing final views. In another preferred embodiment, a common background layer is generated based upon initial warped views. Holes are repaired on the common background layer, which can then be used for synthesizing final views. The cost of the latter method is sub-linear, meaning that additional views can be generated with lesser cost. Preferred methods of the invention also use a placement matrix to generate initial warped views, which simplifies warping to a look-up operation.