Virtual Display Control via Eye Tracking and Hand Position Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional interactive display technologies require physical touch and lack effective methods for concurrently controlling objects at a distance and nearby within an infinitely extended virtual display, resulting in poor user experience and limited control capabilities.
Innovation Solution
A controlling system and method utilizing visual line tracking and space transformation technologies to track a user's visual line, form virtual display spaces, capture hand location, transform it into a cursor location, and control virtual displays, enabling remote object manipulation through a cursor.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If physical touch interface is used for control, then user can interact with display, but user experience deteriorates due to hand affecting object recognition and inability to control remote objects
Solution Approach 1:
The patent replaces the mechanical touch interface with a visual line tracking system that uses optical detection (camera-based eye tracking) to determine user gaze direction. This substitution eliminates the need for physical contact while maintaining control functionality, thereby improving user experience without sacrificing control versatility.
Solution Approach 2:
The patent introduces a cursor as an intermediary between the user's gaze and the virtual objects. The cursor translates eye movement into controllable pointer movement, enabling indirect control of both nearby and distant objects without physical contact, thus resolving the contradiction between ease of operation and control adaptability.
2Adaptability or versatility
If conventional cursor control is used, then nearby objects can be controlled, but concurrent control of distant and nearby objects is unavailable
Solution Approach 1:
The patent implements dynamic cursor behavior where the cursor automatically adjusts its movement speed and scaling based on the distance to the selected object. When the user gazes at distant objects, the cursor moves faster and scales appropriately, enabling efficient control of both nearby and distant objects within the same interface without requiring manual adjustment.
3Area of stationary object
If virtual display is infinitely extended, then more objects can be displayed, but effective cursor control for concurrent distant and nearby objects becomes unavailable
Solution Approach 1:
The patent extends the control mechanism from two-dimensional cursor movement to three-dimensional spatial awareness by incorporating depth information and distance-based scaling. The system perceives objects at different depths and adjusts cursor behavior accordingly, enabling effective control in an infinitely extended virtual space by adding the dimension of depth perception and dynamic scaling.
Data Source
AI summary
A controlling system and a controlling method for virtual display are provided. The controlling system for virtual display includes a visual line tracking unit, a space forming unit, a hand information capturing unit, a transforming unit and a controlling unit. The visual line tracking unit is used for tracking a visual line of a user. The space forming unit is used for forming a virtual display space according to the visual line. The hand information capturing unit is used for obtaining a hand location of the user's one hand in a real operation space. The transforming unit is used for transforming the hand location to be a cursor location in the virtual display space. The controlling unit is used for controlling the virtual display according to the cursor location.