Natural User Interface Using Angled Camera Triangulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for 3D control of electronic devices using a user's body or hands are complex, costly, and require lengthy calibration procedures or specialized equipment, limiting the effectiveness of natural user interfaces for intuitive interaction with electronic devices.
Innovation Solution
A natural user interface system utilizing a mobile device and a display device with cameras positioned at an angle to each other, allowing for the detection of a user's hand posture and movement to control elements on a display screen without the need for specialized hardware or calibration, using overlapping fields of view and triangulation techniques to determine 3D shape and motion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If two or more non-coaxial cameras are used to detect 3D gestures, then 3D control capability is achieved, but device complexity and cost increase
Solution Approach 1:
The patent applies universality by using a single camera to perform multiple functions: capturing 2D images for gesture detection and utilizing its known physical position and orientation to provide reference data for 3D reconstruction. This eliminates the need for multiple specialized cameras while achieving the same 3D control capability, thereby reducing device complexity and cost.
2Measurement precision
If specialized equipment is used for gesture-based 3D control, then detection precision is improved, but cost and availability worsen
Solution Approach 1:
The patent applies self-service by having the single camera's known physical characteristics (position and orientation) serve the dual purpose of capturing gesture images and providing reference data for 3D reconstruction. The system uses readily available consumer camera technology rather than specialized expensive equipment, achieving both precision and cost-effectiveness by making the existing camera resources work harder rather than adding more expensive hardware.
3Measurement precision
If large gestures and lengthy calibration procedures are used, then 3D control accuracy is improved, but ease of operation and time efficiency worsen
Solution Approach 1:
The patent applies preliminary action by pre-establishing the camera's physical position and orientation data before gesture detection begins. This reference information is prepared in advance and stored in the system, eliminating the need for lengthy calibration procedures during operation. Users can immediately perform natural gestures without extensive setup, while the system uses the pre-configured camera parameters to accurately reconstruct 3D motion.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables simple and intuitive control of electronic devices through natural hand gestures without specialized hardware, reducing computational load and eliminating the need for calibration, thus providing a cost-effective and efficient close-range user interface solution.
Implementation Method 1
A processor of the mobile device may determine a position of the user's hand in three dimensional space based on a position of the user's hand in a field of view of the mobile device camera and a position of the user's hand in a field of view of the display device camera
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
A natural user interface system and method of operation thereof including: a display device having a display screen and a display device camera; a mobile device having a mobile device camera, an optical axis of the mobile device camera positioned at an angle to an optical axis of the display device camera; wherein: the mobile device includes a first device pairing module for pairing the mobile device with the display device; the mobile device camera and the display device camera are for: detecting a user's hand, and determining posture and movement of the user's hand; and the display device includes a motion translation module for translating the posture and movement of the user's hand into a gesture for controlling an element of a user interface on the display screen.