Spherical Capture Methods for Real-Time 3D Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for capturing and processing 360-degree spherical images struggle to automatically convert two-dimensional data into three-dimensional data in real-time, especially in virtual reality applications, where user input such as head turns or eye gaze direction is not effectively utilized to generate immersive 3D content.
Innovation Solution
A computer-implemented method and system using a spherically shaped camera rig to capture images, where user input determines regions for 3D transformation, calculates depth values, generates spherical images, and applies texture mapping to create immersive 3D surfaces for display in head-mounted devices, while maintaining surrounding content in 2D format.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If 360-degree spherical images are captured and processed to provide immersive viewing, then user immersion and engagement are improved, but real-time automatic conversion of 2D to 3D data based on user input becomes computationally complex and resource-intensive
Solution Approach 1:
The spherical image is divided into multiple regions, with only the region of interest (determined by user input such as head turn or eye gaze) being converted to 3D data. This segmentation allows the system to provide immersive 3D experience where needed while keeping other areas in 2D format, reducing overall processing complexity.
Solution Approach 2:
The patent applies different quality levels to different parts of the image: the region of interest receives full 3D transformation with depth calculation and texture mapping, while surrounding areas remain in 2D format. This local quality approach optimizes resource allocation by concentrating computational efforts where the user is actually looking.
2Adaptability or versatility
If the entire spherical image is converted to three-dimensional data, then complete immersive experience is achieved, but bandwidth, processing, and storage requirements increase significantly
Solution Approach 1:
Instead of converting the entire spherical image to 3D data, the system performs partial conversion only on the region of interest determined by user input. This partial action principle reduces data volume for bandwidth, processing, and storage while still providing immersive experience in the critical viewing area.
Solution Approach 2:
The image is segmented into 3D-converted region of interest and 2D surrounding areas. This segmentation reduces the quantity of 3D data that needs to be transmitted and stored, optimizing bandwidth and storage requirements while maintaining immersive experience in the relevant region.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods are described for capturing spherical content. The systems and methods can include determining a region within a plurality of images captured with a plurality of cameras in which to transform two-dimensional data into three-dimensional data, calculating a depth value for a portion of pixels in the region, generating a spherical image, the spherical image including image data for the portion of pixels in the region, constructing, using the image data, a three-dimensional surface in three-dimensional space of a computer graphics object generated by an image processing system, generating, using the image data, a texture mapping to a surface of the computer graphics object, and transmitting the spherical image and the texture mapping for display in a head-mounted display device.