View Extrapolation Using Integrated Frames to Fill Boundary Holes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 3D image apparatuses face challenges in generating accurate and seamless views at points outside the photographed points, particularly due to discrepancies in frame properties and missing data in boundary regions during view extrapolation.
Innovation Solution
The apparatus includes an input view generation unit, an integrated frame generation unit, and an extrapolated view generation unit, which utilize motion estimation, seamless frame merging, and depth information to create integrated frames and extrapolated views by repositioning pixels based on distance and using chronological motion information, thereby compensating for frame and object boundary holes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If view extrapolation is performed using individual frames, then the extrapolated view can be generated, but frame boundary holes and discordance occur due to different properties between frames
Solution Approach 1:
The patent merges multiple successive frames into a single integrated frame before performing view extrapolation. This combining approach integrates information from multiple frames to fill boundary holes and reduce discordance, directly resolving the technical contradiction by eliminating frame boundary holes while maintaining extrapolation capability.
Solution Approach 2:
The patent performs preliminary frame integration and motion estimation before the actual view extrapolation process. By preparing an integrated frame that already has reduced boundary holes and improved consistency, the subsequent extrapolation operation produces higher quality results without suffering from the frame boundary problems.
2Reliability
If multiple successive frames are integrated, then boundary discordance is reduced, but processing complexity increases
Solution Approach 1:
The patent segments the frame integration process into distinct functional units: motion estimation unit, integrated frame generation unit, and seamless frame merging unit. This segmentation manages complexity by organizing the processing into modular, manageable stages, each handling a specific aspect of the integration task.
Solution Approach 2:
The patent applies different processing strategies to different regions of the frames. The seamless frame merging unit specifically targets boundary spots with discordance and applies localized correction, while other regions undergo standard integration. This local quality approach reduces overall processing complexity by focusing intensive operations only where needed.
3Productivity
If motion estimation is performed on individual frames, then processing speed is maintained, but accuracy of pixel repositioning decreases
Solution Approach 1:
The patent combines multiple frames into an integrated frame before performing motion estimation. This integration accumulates motion information across frames, improving the accuracy of motion vectors and subsequent pixel repositioning. The unified integrated frame provides more robust data for motion analysis, enhancing measurement precision without significantly impacting processing speed.
Data Source
AI summary
A view extrapolation apparatus and a view extrapolation method to generate images at a plurality of virtual points uses a relatively small number of input images. The view extrapolation apparatus and the view extrapolation method output a view at a reference point, the view at the reference point being formed of frames generated chronologically, integrate a plurality of successive frames of the view at the reference point to generate an integrated frame, and generate an extrapolated view at a virtual point using the integrated frame.