Virtual 3D Space Generation from Real Video Depth Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for generating virtual three-dimensional game spaces using real video images restrict movement in the depth direction, causing strange feelings due to limitations in expressing depth.
Innovation Solution
A method that specifies the positional relation between a virtual screen and camera within real video image data, generates three-dimensional reference object data, and combines it with computer-generated data to create a virtual three-dimensional space, ensuring accurate depth representation by aligning boundaries and adjusting camera positions based on the view angle.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If real video images are used to generate virtual three-dimensional space, then the realism and visual fidelity are improved, but the ability to express depth direction and enable movement in the depth direction is worsened
Solution Approach 1:
The patent extracts depth information from two-dimensional real video images by analyzing parallax between multiple images taken from different positions. This converts 2D image data into 3D spatial information, enabling depth direction expression while maintaining the visual fidelity of real video images. The depth map generation process effectively adds a third dimension to the original 2D video data.
Solution Approach 2:
The patent introduces depth maps as an intermediary data structure that bridges real video images and virtual three-dimensional space. The depth maps encode depth information extracted from video images and serve as a mediator to enable character movement in the depth direction while maintaining compatibility with the original video data. This intermediary layer allows the system to utilize both the visual quality of real video and the spatial capabilities of 3D environments.
2Adaptability or versatility
If characters move in depth direction using real video images, then the immersion is improved, but strange feelings are caused due to depth expression limitations
Solution Approach 1:
The patent employs feedback mechanisms where the generated depth maps are continuously refined by comparing synthesized images with original video images. This feedback loop ensures that the depth information accurately reflects the actual spatial relationships in the scene, preventing strange feelings while enabling free movement in the depth direction. The system adjusts depth map values based on the consistency between synthesized and original images.
Solution Approach 2:
The patent performs preliminary depth extraction and depth map generation before character movement begins. By pre-processing the video images to extract accurate depth information and generate comprehensive depth maps in advance, the system ensures that depth perception accuracy is maintained throughout the entire movement process, preventing strange feelings that would arise from real-time depth calculation limitations.
Data Source
AI summary
Three-dimensional reference object data indicating a boundary of a reference object contained in a real video image is generated from first data containing the real video image based on a positional relation between a virtual screen position where the real video image is displayed and a virtual camera position corresponding to a camera position. Second data is generated, which indicates an object constituting a virtual three-dimensional CG space generated by reproducing an actual space where the real video image is taken. Third data is data in which the virtual camera position in the virtual three-dimensional CG space is determined in such a manner that the boundary of the reference object in the reference object data and a boundary of the object corresponding to the reference object in the second data are caused to coincide with each other.