Eye Gaze and Hand Pose Sensing for Intuitive Object Selection

Solution Overview

Problem

Conventional user interface methods require cumbersome three-step processes for selecting objects, leading to user fatigue and increased time, necessitating an improvement in intuitiveness and usability.

Innovation Solution

A user interface apparatus incorporating an eye gaze sensing unit to determine the position of an object and a hand pose sensing unit to calculate conformity factors, allowing for the selection of targets without additional manipulation by combining eye gaze and hand pose data using a discriminant to determine the intended object.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If conventional mouse or 6 DOF handheld input device is used for object selection, then the selection process is completed with established methods, but the user experiences fatigue and the process takes relatively more time

Engineering Contradiction:

Improveselection accuracyVSAvoidselection time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs preliminary actions by detecting eye gaze direction and hand pose in advance, calculating position factor and conformity factors before final target determination. This allows the system to pre-process selection criteria and rapidly identify the intended target without requiring sequential manual操作步骤

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent replaces mechanical input devices (mouse, 6 DOF handheld device) with a sensing system that detects physiological signals (eye gaze) and physical pose (hand position and orientation). This substitution eliminates the need for mechanical manipulation while maintaining or improving selection accuracy

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Reliability

If conventional mouse or 6 DOF handheld input device is used for object selection, then the selection process is completed with established methods, but the operation becomes cumbersome and less intuitive

Engineering Contradiction:

Improveselection accuracyVSAvoidoperation intuitiveness

Core Design Contradiction:

ReliabilityVSEase of operation

Solution Approach 1:

The system allows the user's natural physiological responses (eye gaze direction, hand pose) to automatically serve as the selection mechanism. The user simply looks at and position their hand toward the desired object, and the system self-determines the target based on these natural cues, eliminating the need for learned operational procedures

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent replaces mechanical input devices (mouse, 6 DOF handheld device) with a sensing system that detects physiological signals (eye gaze) and physical pose (hand position and orientation). This substitution eliminates the need for mechanical manipulation while maintaining or improving selection accuracy

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Reliability

If three-step selection mechanism is used, then target selection can be achieved with conventional devices, but user fatigue increases and time consumption increases

Engineering Contradiction:

Improveselection accuracyVSAvoidselection efficiency

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent merges multiple selection criteria (eye gaze position, eye gaze direction, hand pose, object geometry) into a unified target determination process. By combining these factors through the discriminant function, the system achieves accurate target selection in a single integrated step rather than through sequential three-step operations

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The system performs preliminary actions by detecting eye gaze direction and hand pose in advance, calculating position factor and conformity factors before final target determination. This allows the system to pre-process selection criteria and rapidly identify the intended target without requiring sequential manual操作步骤

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS9547371B2User interface apparatus and method for controlling the same

Publication Date: 2017.01.17 KOREA INST OF SCI & TECH

AI summary

The present disclosure relates to a multi-dimensional user interface apparatus that selects an object by sensing a motion of a user and a control method thereof.The user interface apparatus according to the present disclosure includes an eye gaze sensing unit to sense eye positions and eye gaze directions of a user, and a hand shape sensing unit to sense a hand shape of the user, wherein the user interface apparatus calculates a position factor related to a location of an object from the eye positions and the eye gaze directions, calculates at least one conformity factor related to a conformity between the hand shape and the object from the hand shape, and determines the object as a target selected by the user based on the calculated position factor and the at least one conformity factor.