Adaptive Content Projection Onto 3D Objects

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Natural user interfaces, such as gesture controls, are limited by their reliance on computer monitors and lack of adaptability to the geometry and character of objects in their field of view, restricting their applicability and ease of use.

Innovation Solution

A system comprising a 2D/3D input device that uses a 3D camera and processor to create a depth map, allowing for adaptive content projection onto 3D objects, adjusting scale, distortion, and content based on object geometry and viewer parameters, enabling interactive projections on suitable objects or virtual surfaces.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If gesture controls rely on optical 3-dimensional mapping, then user interaction capability is improved, but adaptability to object geometry and character is reduced

Engineering Contradiction:

Improveuser interaction capabilityVSAvoidadaptability to object geometry

Core Design Contradiction:

Ease of operationVSAdaptability or versatility

Solution Approach 1:

The system dynamically adapts the projected interface to the detected object's geometry and characteristics. The projection device modifies the scale, distortion, and content of projected elements based on real-time depth map analysis, making the interface both easy to operate and adaptable to different objects.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes multiple parameters of the projected content including scale, distortion, focus, and content selection based on the object's geometric properties. This allows the same gesture control system to work effectively with objects of varying sizes, shapes, and characteristics.

Inventive Principle:

Principle #35Parameter changes

2Device complexity

If gesture controls relate largely to the computer monitor, then interface simplicity is maintained, but applicability and ease of use are reduced

Engineering Contradiction:

Improveinterface simplicityVSAvoidapplicability

Core Design Contradiction:

Device complexityVSAdaptability or versatility

Solution Approach 1:

The system makes the computer monitor serve multiple functions by projecting content onto both traditional screens and physical objects in the environment. This universal approach maintains interface simplicity while greatly expanding applicability to various contexts and objects.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The projection device acts as an intermediary between the computer monitor and physical objects, transferring digital content to the physical world. This mediator enables natural interaction with real-world objects while maintaining the simplicity of software-based control interfaces.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Manufacturing precision

If content projection adapts to object geometry, then projection quality is improved, but system complexity increases

Engineering Contradiction:

Improveprojection qualityVSAvoidsystem complexity

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

The system performs preliminary depth mapping and object recognition before projection to pre-calculate the necessary geometric transformations. This preliminary analysis simplifies the real-time projection process by preparing transformation matrices and distortion corrections in advance.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system replaces complex mechanical adjustment mechanisms with software-based geometric transformations. Instead of physically adjusting projection optics for each object, the system uses computational methods to calculate and apply the appropriate scale, distortion, and orientation corrections digitally.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

Enables enhanced user interaction by projecting content onto objects or virtual surfaces, providing a more natural and intuitive interface that adapts to the environment and viewer position, improving usability and applicability of natural user interfaces.

Implementation Method 1

a sensing element for acquiring a scene including a 2-dimensional camera and a 3-dimensional camera, a processor linked to the 3-dimensional camera and the 2-dimensional camera and programmed to produce a depth map of the scene using an output of the 3-dimensional camera

Methodology Applied

Scientific EffectOptical imaging: Photography

Data Source

PatentUS9158375B2Interactive reality augmentation for natural interaction

Publication Date: 2015.10.13 APPLE INC

AI summary

Embodiments of the invention provide apparatus and methods for interactive reality augmentation, including a 2-dimensional camera (36) and a 3-dimensional camera (38), associated depth projector and content projector (48), and a processor (40) linked to the 3-dimensional camera and the 2-dimensional camera. A depth map of the scene is produced using an output of the 3-dimensional camera, and coordinated with a 2-dimensional image captured by the 2-dimensional camera to identify a 3-dimensional object in the scene that meets predetermined criteria for projection of images thereon. The content projector projects a content image onto the 3-dimensional object responsively to instructions of the processor, which can be mediated by automatic recognition of user gestures.