Tiled Light-Field Camera Array for Immersive VR Capture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current capture systems for virtual reality (VR) and augmented reality (AR) are limited by the number of physical camera viewpoints, leading to incomplete or inaccurate imagery when users change their head position or orientation, requiring algorithmic pixel generation or manual artistry to infer missing data.
Innovation Solution
A system using tiled arrays of light-field cameras to capture a light-field volume, allowing for the direct computation of physically correct virtual views from any position and viewing direction within the captured volume, enabling more immersive and accurate VR/AR experiences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a limited number of physical cameras are used to capture VR/AR imagery, then device complexity is reduced, but measurement precision and completeness of light-field data deteriorate when users change head position or orientation
Solution Approach 1:
The light-field camera is segmented into multiple independent lenslets arranged in a grid array, where each lenslet captures light rays from a specific directional perspective. This segmentation enables the system to capture complete light-field data (position and direction information) from a single device, resolving the contradiction between using fewer physical cameras and achieving high measurement precision for multiple viewpoints.
Solution Approach 2:
The patent adds a fourth dimension (angular/directional information) to the traditional two-dimensional image capture by incorporating microlens arrays that record both position and direction of light rays. This dimensional enhancement allows a single camera to capture data equivalent to multiple cameras positioned at different viewpoints, improving measurement precision without increasing device complexity.
2Device complexity
If algorithmic pixel generation is used to infer missing data from limited viewpoints, then device complexity remains low, but loss of information increases due to interpolation requirements
Solution Approach 1:
The light-field camera performs preliminary capture of complete light-field data (including angular information) at the moment of acquisition, storing directional information for all rays in the field of view. This preliminary action eliminates the need for subsequent interpolation or algorithmic generation of missing data, as all necessary information is already captured, thereby preventing information loss while maintaining simple device structure.
3Measurement precision
If tiled arrays of light-field cameras are used to capture complete light-field volume, then measurement precision and user freedom of motion are improved, but device complexity and manufacturing difficulty increase
Solution Approach 1:
The patent merges multiple lenslets and their associated optical paths into a single integrated light-field camera unit. By combining the functionality of multiple cameras into one device with a microlens array, the system achieves complete light-field volume capture without the complexity of coordinating multiple separate cameras, thus improving measurement precision while reducing device complexity.
Solution Approach 2:
The light-field camera is designed as a universal device that can capture light-field data from all directions within its field of view, making it multi-functional in terms of viewpoint coverage. This universal capture capability eliminates the need for multiple specialized cameras positioned at different locations, reducing device complexity while maintaining high measurement precision for virtual views from any position.
4Ease of manufacture
If conventional cameras are used with few viewpoints, then ease of manufacture is improved, but productivity of capturing immersive VR/AR data deteriorates due to insufficient coverage
Solution Approach 1:
The camera sensor is segmented into multiple regions, each corresponding to a specific lenslet in the microlens array. This segmentation allows each region to capture directional information independently, enabling complete light-field volume capture from a single device. The segmented structure maintains ease of manufacture using standard microlens array technology while significantly improving productivity by capturing all necessary data in one shot.
Solution Approach 2:
The patent enhances conventional two-dimensional image capture by adding angular dimension through microlens arrays. This dimensional change transforms the capture system into a light-field camera that records position and direction simultaneously, improving productivity by capturing complete immersive data from all viewpoints in a single capture event, while maintaining ease of manufacture through integration with standard imaging sensors.
Data Source
AI summary
A capture system may capture light-field data representative of an environment for use in virtual reality, augmented reality, and the like. The system may have a plurality of light-field cameras arranged to capture a light-field volume within the environment, and a processor. The processor may use the light-field volume to generate a first virtual view depicting the environment from a first virtual viewpoint. The light-field cameras may be arranged in a tiled array to define a capture surface with a ring-shaped, spherical, or other arrangement. The processor may map the pixels captured by the image sensors to light rays received in the light-field volume, and store data descriptive of the light rays in a coordinate system representative of the light-field volume.