HMD View Synthesis via Tracking Cameras

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Head-mounted displays (HMDs) for virtual and augmented reality face challenges in providing a see-through view without the need for physical see-through cameras, which increases manufacturing costs and obstructs the user's direct view.

Innovation Solution

The use of multiple tracking cameras around the HMD to capture image data and depth information, allowing for view synthesis to generate a see-through effect as if viewed from the opposite side, eliminating the need for physical see-through cameras.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If physical see-through cameras are installed on the HMD to provide a see-through view, then the user can view the scene, but the manufacturing cost increases and the user's direct view is obstructed

Engineering Contradiction:

Improvesee-through view capabilityVSAvoidmanufacturing cost

Core Design Contradiction:

Ease of operationVSEase of manufacture

Solution Approach 1:

The patent uses tracking cameras positioned around the HMD to capture images of the scene, then synthesizes a see-through view through image processing algorithms. This creates a virtual copy of the see-through camera function without requiring physical see-through cameras to be mounted on the HMD, thereby reducing manufacturing costs while maintaining the see-through viewing capability

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent introduces tracking cameras as intermediary devices positioned around the HMD rather than directly on it. These tracking cameras capture scene information and the processor synthesizes the final see-through view, acting as intermediaries between the physical environment and the user's visual experience, thus avoiding direct obstruction and reducing manufacturing complexity

Inventive Principle:

Principle #24Intermediary (Mediator)

2Ease of operation

If physical see-through cameras are installed on the HMD to provide a see-through view, then the user can view the scene, but the HMD structure becomes more complex

Engineering Contradiction:

Improvesee-through view capabilityVSAvoidHMD structure

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The patent positions tracking cameras around the HMD in three-dimensional space rather than mounting them directly on the front surface. This spatial redistribution moves the camera system to another dimension (surrounding the HMD), simplifying the front HMD structure while still enabling see-through functionality through computational synthesis

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

3Ease of manufacture

If tracking cameras are used around the HMD for view synthesis, then manufacturing cost is reduced, but the view synthesis processing complexity increases

Engineering Contradiction:

Improvemanufacturing costVSAvoidview synthesis processing

Core Design Contradiction:

Ease of manufactureVSDevice complexity

Solution Approach 1:

The patent replaces the mechanical/optical system of physical see-through cameras with a computational system. Instead of using optical path splitting and physical camera mounting, the system uses image capture from tracking cameras combined with digital image processing and view synthesis algorithms to create the see-through effect, substituting mechanical complexity with computational processing

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS11792352B2Camera view synthesis on head-mounted display for virtual reality and augmented reality

Publication Date: 2023.10.17 MEDIATEK INC

AI summary

Various examples pertaining to camera view synthesis on head-mounted display (HMD) for virtual reality (VR) and augmented reality (AR) are described. A method involves receiving, from a plurality of tracking cameras disposed around a HMD, image data of a scene which is on a first side of the HMD. The method also involves performing, using the image data and depth information pertaining to the scene, view synthesis to generate a see-through effect of viewing the scene from a viewing position on a second side of the HMD opposite the first side thereof.