Stereo Image Pair Generation from 2D Video via Motion Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for converting 2D video to 3D stereo images often produce undesirable results due to assumptions about parallel camera motion, especially when capturing videos with abrupt rotations or small transitions, which are common in general 2D videos.
Innovation Solution
A method that analyzes a temporal sequence of digital images to produce stereo suitability scores, selects suitable image pairs based on motion consistency, and generates motion consistency maps to combine images into left and right views, effectively handling various capture conditions such as stationary or moving scenes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If MPEG motion estimation is used to extract stereo pairs from video sequences, then real-time 2D to 3D video conversion is achieved, but abrupt rotations and small transitions of camera produce undesirable results
Solution Approach 1:
The patent applies preliminary action by performing motion consistency analysis and camera motion detection before stereo pair extraction. The system pre-processes video sequences to identify frames with consistent motion patterns and detects camera rotation movements, selecting only suitable frames for stereo conversion. This preliminary filtering ensures that abrupt rotations and small transitions are identified and handled appropriately before the main stereo extraction process, preventing undesirable results while maintaining real-time performance.
2Device complexity
If a parallel camera assumption is made for stereo extraction, then the conversion process is simplified, but camera rotations and transitions violate the assumption and produce poor results
Solution Approach 1:
The patent applies dynamics by making the stereo extraction process adaptive rather than static. The system dynamically detects camera motion patterns and adjusts the extraction process based on detected conditions. When camera rotation or transition is detected, the system modifies its behavior to accommodate these movements, selecting appropriate frame pairs that maintain stereo consistency even under non-parallel camera conditions. This dynamic adaptation allows the system to handle diverse capture conditions while maintaining reasonable complexity.
3Productivity
If key frame extraction is used to represent video sequences, then processing efficiency is improved, but no method is provided for forming stereo images from video sequences
Solution Approach 1:
The patent applies the intermediary principle by introducing motion consistency maps as an intermediate representation between key frame extraction and stereo pair formation. The system extracts key frames efficiently, then generates motion consistency maps that encode motion information from the video sequence. These maps serve as intermediaries that preserve essential motion data needed for stereo formation, allowing the system to work with reduced frame sets while maintaining the information necessary for accurate stereo image construction.
Data Source
AI summary
A method of producing a stereo image from a temporal sequence of digital images, comprising: receiving a temporal sequence of digital images; analyzing pairs of digital images to produce corresponding stereo suitability scores, wherein the stereo suitability score for a particular pair of images is determined responsive to the relative positions of corresponding features in the particular pair of digital image; selecting a pair of digital images including a first image and a second image based on the stereo suitability scores; using a processor to analyze the selected pair of digital images to produce a motion consistency map indicating regions of consistent motion, the motion consistency map having an array of pixels; producing a stereo image pair including a left view image and a right view image by combining the first image and the second image responsive to the motion consistency map; and storing the stereo image pair in a processor-accessible memory.