Immersive Video System With Time-of-Flight Sensor

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current interactive systems lack tools to seamlessly create complex, immersive 3D environments that allow users to interact based on their location and size, and do not easily support multi-user experiences with consistent UI elements across different applications.

Innovation Solution

An immersive video system incorporating a display, a Time-of-Flight sensor to determine user location, a projector to project images based on user data, and a processor to manipulate these projections, enabling tailored interactions and shared experiences among multiple users.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If simple hand tracking and 3D visualization are used in immersive environments, then the system complexity is reduced, but the ability to understand complex human factors issues and create engaging interactions is limited

Engineering Contradiction:

Improvesystem complexityVSAvoidinteraction capability

Core Design Contradiction:

Device complexityVSAdaptability or versatility

Solution Approach 1:

The system segments the interaction space into multiple depth planes (near, mid, far zones) and assigns different interaction modalities to each plane. Hand tracking is used for close-range interactions while gaze tracking handles distant selections, dividing the complex interaction problem into manageable segments that can be processed independently

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The immersive system integrates multiple sensing capabilities (hand tracking, gaze tracking, depth sensing) into a single unified platform that can handle diverse interaction types. The same system infrastructure supports both simple visualization and complex multi-modal interactions, making the system universally applicable across different use cases

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Adaptability or versatility

If gesture-based AR systems with multiple sensors and processing components are integrated, then the interaction capability and user engagement are improved, but the device complexity and integration difficulty increase

Engineering Contradiction:

Improveinteraction capabilityVSAvoidintegration complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent merges hand tracking and gaze tracking systems into a single integrated AR platform. The depth sensor, camera, and processing units are combined into one cohesive system rather than separate components, reducing integration complexity while maintaining full functionality of both interaction modalities

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The system introduces a depth sensor as an intermediary component that bridges the gap between simple 2D camera input and complex 3D interaction understanding. The depth information acts as a mediator that enables accurate hand tracking and spatial awareness without requiring overly complex processing of raw video data

Inventive Principle:

Principle #24Intermediary (Mediator)

3Adaptability or versatility

If sensors are used to detect user location and manipulate projections in real-time, then the location-aware interaction capability is improved, but the processing requirements and system resource consumption increase

Engineering Contradiction:

Improvelocation-aware interactionVSAvoidprocessing resource consumption

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The system applies partial processing by focusing computational resources only on the regions of interest in the 3D space. Instead of processing all projected content uniformly, the system selectively manipulates only those portions where user interaction is detected, reducing overall processing requirements while maintaining responsive location-aware interactions

Inventive Principle:

Principle #16Partial or excessive action

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This solution allows for immersive, location-aware interactions and shared experiences in 3D environments, enhancing user engagement and consistency across different applications by integrating sensors and processing systems for precise user feedback and projection mapping.

Implementation Method 1

a Time-of-Flight sensor to determine user location

Methodology Applied

Scientific EffectTime of Flight: Time of Flight

Data Source

PatentUS10462454B2Extensible authoring and playback platform for complex virtual reality interactions and immersive applications

Publication Date: 2019.10.29 MEMORY ENGINE

AI summary

An immersive video system includes a display, a sensor that provides information about a user's location relative to the display, a projector capable of projecting images onto the user, and a processor in communication with the display, the sensor, and the projector. The processor manipulates the images projected onto the user based on user location data from the sensor.