Virtual Interface Projection on Hand Contours for Input Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing user interfaces face challenges in accurately determining the location of input devices within a scene or environment, particularly when a portion of the input device covers a part of the user interface, making it difficult to identify the selection of input features.
Innovation Solution
The system projects a user interface onto a display object, such as a hand, and uses 3D imaging and structured light techniques to analyze the contours and curvature information of both the display object and the operating object, merging contours to estimate the location of the operating object's input portion that is not reflected in the merged contour, allowing for accurate selection of input features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional input devices like keyboards are used, then physical manipulation is required, but user motion detection and gesture recognition become difficult
Solution Approach 1:
The patent replaces traditional mechanical input devices (keyboards, mice) with a gesture-based system that uses optical sensors and image processing to detect user motions and gestures in 3D space, allowing users to interact with the interface through natural movements rather than physical device manipulation
Solution Approach 2:
The system introduces an intermediary processing layer that captures raw sensor data from multiple sources (cameras, depth sensors), processes this data through image analysis algorithms, and converts it into meaningful gesture commands, bridging the gap between physical motion and digital input recognition
2Adaptability or versatility
If the operating object covers the display object, then user interaction flexibility increases, but identification of the operating object's input portion becomes difficult
Solution Approach 1:
The system performs preliminary actions by capturing images of the display object at multiple known locations before the operating object covers them, storing these reference images for later comparison when the operating object is present, enabling accurate location determination even during overlap
Solution Approach 2:
The system uses feedback mechanisms by comparing the appearance of the display object at known locations (captured when no operating object is present) with the current image data, allowing it to infer the location of the operating object's input portion based on what is and isn't visible in the merged contour
3Measurement precision
If 3D imaging and contour merging techniques are used, then location determination accuracy improves, but system complexity increases
Solution Approach 1:
The patent segments the complex task of location determination into distinct processing stages: capturing images at multiple known locations, generating contours of the display object, merging contours when the operating object is present, and comparing merged contours with reference contours to determine location, making the overall system more manageable and implementable
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This method enables precise determination of the location of the operating object's input portion, ensuring accurate communication of user input to the system, even when the operating object overlaps with the display object, thereby enhancing the accuracy of user interface interactions.
Implementation Method 1
uses 3D imaging and structured light techniques to analyze the contours and curvature information
Implementation Method 2
analyze the contours and curvature information of both the display object and the operating object
Data Source
AI summary
A virtual user interface in a 3D environment is projected onto a display object, such as an open hand. An operating object, such as a stylus or a second hand with an index finger extended, may be used to select input features of the projected user interface. In one scenario, for example, a telephone keypad may be projected onto the open hand and the index finger may be used to select a particular number on the keypad. Images of the hands may be segmented to produce contours of the finger and open hand, and these contours are used to determine the location of the finger relative to the open hand.