3D Camera Touch Interface for Non-Occlusive Hand Input

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing touch-sensitive interfaces, such as those used in computing devices, often require users to physically interact with a display screen, which can be impractical for devices like desktop computers where the screen is at a distance, leading to occlusion issues and limited intuitive input methods.

Innovation Solution

A system utilizing a 3D camera to detect the position of users' hands relative to a touch surface, allowing for intuitive input without physical contact, enabling hover events and precise hand recognition for complex input functions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If a touch-sensitive display screen is used for user input, then the interface becomes more intuitive, but the user's hands occlude the display screen

Engineering Contradiction:

Improveintuitiveness of interfaceVSAvoidvisible display area

Core Design Contradiction:

Ease of operationVSArea of stationary object

Solution Approach 1:

The patent introduces a camera as an intermediary device to capture images of the user's hands and displays a processed representation (such as a shadow or reflection) on the screen. This mediator allows the system to detect hand position and gestures without requiring direct physical contact with the display, thus preventing occlusion while maintaining intuitive interaction.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent creates a visual copy or representation of the user's hand (through shadow, reflection, or processed image) and displays it on the screen. This copy serves as a proxy for the actual hand, allowing the user to interact with the display indirectly through the represented image, thereby avoiding occlusion of the actual display content.

Inventive Principle:

Principle #26Copying

2Length of stationary object

If the display screen is placed at a distance from the user (as in desktop computers), then the device structure is improved, but it becomes impractical for the user to reach out and touch the display

Engineering Contradiction:

Improvedistance between user and displayVSAvoidaccessibility of touch interface

Core Design Contradiction:

Length of stationary objectVSEase of operation

Solution Approach 1:

The patent replaces the mechanical touch interaction (requiring physical contact with the display surface) with an optical detection system. The camera captures images of the user's hands from a distance, and image processing algorithms detect hand position and gestures, substituting the need for mechanical contact with optical field detection.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent transitions from a two-dimensional touch surface interaction to a three-dimensional spatial interaction. By using camera imaging and depth analysis, the system can detect hand position in 3D space, allowing users to interact with the display from a distance through gestures in the third dimension (depth), rather than requiring contact with the 2D screen surface.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

3Adaptability or versatility

If a 3D camera is used to detect hand position, then non-contact input is enabled, but the system complexity increases

Engineering Contradiction:

Improveinput method flexibilityVSAvoidsystem structure

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent employs a camera, which is a common and versatile device, to perform multiple functions: capturing user input gestures, providing visual feedback representation, and enabling various interaction modes (touch, hover, gesture recognition). This multi-functional use of a single component reduces overall system complexity compared to dedicated sensors for each function.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system uses the camera to capture both the input signal (hand position and gestures) and to provide visual feedback (displaying the hand representation on screen). The same optical field and imaging system serve dual purposes, eliminating the need for separate feedback display mechanisms and reducing system complexity.

Inventive Principle:

Principle #25Self-service

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

Enables users to interact with electronic devices without occluding the display, providing intuitive and precise input methods, including touch and hover events, by rendering a transparent representation of hands on the screen and correlating hand positions to control elements.

Implementation Method 1

determining a three-dimensional (3D) position of an object relative to a surface based on 3D image data

Methodology Applied

Scientific Effect3D imaging:

Data Source

PatentEP2834723B1Touch sensitive user interface

Publication Date: 2017.12.27 MICROSOFT TECHNOLOGY LICENSING LLC

Figure 1A
Figure 1B
Figure 1C

AI summary

A system and method are disclosed for providing a touch interface for electronic devices. The touch interface can be any surface. As one example, a table top can be used as a touch sensitive interface. In one embodiment, the system determines a touch region of the surface, and correlates that touch region to a display of an electronic device for which input is provided. The system may have a 3D camera that identifies the relative position of a user's hands to the touch region to allow for user input. Note that the user's hands do not occlude the display. The system may render a representation of the user's hand on the display in order for the user to interact with elements on the display screen.