Light Field VR via Camera Array View Interpolation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Traditional virtual reality solutions require extensive processing time and manual work to create 360-degree field of view content, often lacking motion parallax cues, which reduces immersion and introduces visual fatigue, and are cost-inefficient due to the need for expensive capture systems and dense scanning methods.

Innovation Solution

The system generates virtual reality content using a circular array of wide-angle cameras, interpolating views based on head position tracking to introduce motion parallax and reduce rendering complexity, eliminating the need for panorama stitching and 3D model creation, thereby enabling real-time, cost-effective, and immersive light field virtual reality experiences.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If traditional panorama stitching algorithms are used to create 360-degree VR content, then a complete field of view is achieved, but processing time increases dramatically and visual fatigue is introduced due to lack of motion parallax

Engineering Contradiction:

Improveprocessing timeVSAvoidimmersion quality

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent divides the scene into multiple overlapping fisheye images captured by an array of cameras positioned at different locations. Instead of stitching a single 360-degree panorama, the system segments the view into discrete camera perspectives that can be independently processed and selectively combined based on head position, eliminating the need for complex panorama stitching while preserving motion parallax information.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent transitions from 2D panorama stitching to 3D light field reconstruction by capturing images from multiple spatial positions. This dimensional transformation enables the system to reconstruct intermediate views through view interpolation, providing motion parallax cues that were absent in traditional 2D stitching approaches and significantly reducing processing complexity.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Measurement precision

If dense scanning methods with expensive capture systems are used, then high-quality light field data is obtained, but cost and system complexity increase

Engineering Contradiction:

Improvelight field data qualityVSAvoidcapture system cost

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent uses standard wide-angle fisheye cameras that can be obtained from consumer devices, making the capture system universal and cost-effective. These cameras perform multiple functions by capturing not only traditional panoramas but also light field information when arranged in an array, eliminating the need for specialized expensive capture equipment.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent creates virtual copies of camera views through view interpolation algorithms. Instead of physically placing cameras at every possible viewing position, the system computationally generates intermediate views from a limited set of captured images, providing dense light field coverage without the cost and complexity of dense physical camera arrays.

Inventive Principle:

Principle #26Copying

3Reliability

If 3D models are created using structure from motion techniques, then immersive 3D content is generated, but tremendous manual work and processing time are required

Engineering Contradiction:

Improve3D content qualityVSAvoidmanual work time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent replaces manual structure from motion processing with automatic view interpolation based on head position tracking. Instead of requiring manual 3D model creation and reconstruction, the system automatically generates appropriate views by interpolating between captured camera images based on real-time head position data, dramatically reducing manual work while maintaining 3D immersion quality.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

4Reliability

If intermediate views are interpolated from multiple corrected images, then motion parallax is introduced, but rendering complexity increases

Engineering Contradiction:

Improvemotion parallax qualityVSAvoidrendering complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent performs preliminary correction and alignment of fisheye images during the capture phase, preparing the data for efficient view interpolation. By pre-processing the images to establish proper geometric relationships and corrections, the system reduces the computational complexity of real-time view interpolation while maintaining high motion parallax quality.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11812009B2Generating virtual reality content via light fields

Publication Date: 2023.11.07 INTEL CORP

AI summary

An example system for generating light field content is described herein. The system includes a receiver to receive a plurality of images and a calibrator to intrinsically calibrate a camera. The system also includes a corrector and projector undistort the images and project the undistorted images to generate undistorted rectilinear images. An extrinsic calibrator may rectify and align the undistorted rectilinear images. Finally, the system includes a view interpolator to perform intermediate view interpolation on the rectified and aligned undistorted rectilinear images.