3D Projection Image Generation Using Depth Value Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating 3D images in indoor and outdoor environments fail to provide realistic images corresponding to arbitrary user movement and rotation, limiting the user's ability to explore spaces beyond the original image capture route and restricting the field of view.
Innovation Solution
A projection image generating method that estimates multiple image and depth obtaining poses in a 3D space, projects depth values into corresponding images based on user pose, and uses supplementary images to fill lost viewing field regions, allowing for natural and realistic 3D image generation across arbitrary movements and rotations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If images and depth values are obtained from multiple continuous poses to enable natural 3D image display during user movement, then the realism of the 3D image is improved, but the system is limited to displaying images only along the specific capture route and cannot support arbitrary user movement
Solution Approach 1:
The patent transitions from a one-dimensional linear interpolation approach (mixing images from sequential poses) to a three-dimensional spatial reconstruction approach. By establishing a 3D coordinate system and mapping depth values to spatial positions, the system enables users to view images from arbitrary positions and angles, not just along the capture route.
Solution Approach 2:
The patent creates a virtual copy of the physical environment by reconstructing 3D spatial information from multiple images and depth values. This virtual 3D space allows users to explore the environment freely without being constrained to the original capture path, as the system synthesizes views from any position within the reconstructed space.
2Ease of manufacture
If images are captured only at specific locations and angles, then the data acquisition process is simple, but the field of view is limited and users cannot explore spaces beyond the capture route
Solution Approach 1:
The patent performs preliminary 3D spatial reconstruction and depth mapping during the data acquisition phase. By pre-processing images and depth values to establish a comprehensive 3D coordinate system and spatial relationships, the system enables flexible querying and viewing from any position later, without requiring additional capture operations.
3Adaptability or versatility
If images from multiple poses are mixed to display 3D images when users are at poses where no image was obtained, then coverage is improved, but the displayed images appear unnatural
Solution Approach 1:
The patent changes the parameter space from simple image blending weights to three-dimensional spatial coordinates and depth values. By mapping pixels to their true spatial positions and using depth information for accurate projection, the system generates natural-looking images even at poses where no original image was captured, avoiding the unnatural appearance of mixed images.
Data Source
Figure 1
Figure 2~3
Figure 4~6a
AI summary
Disclosed is a projection image generating method for a 3D space. An exemplary embodiment of the present disclosure provides a projection image generating method, including: estimating a plurality of image obtaining poses and a plurality of depth obtaining poses including obtaining positions and obtaining angles of a plurality of obtained images and a plurality of obtained depth values obtained in an actual 3D space, respectively, with respect to a reference coordinate system; obtaining a user pose including a location and an angle of the user in a virtual 3D space corresponding to the actual 3D space with respect to the reference coordinate system; and generating a projection image obtained by projecting the plurality of obtained depth values into at least one of the plurality of obtained images, based on the corresponding image obtaining pose corresponding to the user pose and at least one corresponding depth obtaining pose corresponding to the user pose.