Eye Tracking for Hands-Free Virtual Item Positioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional Near-Eye-Display (NED) devices are inadequate for users who need to position virtual items in a real-world environment while unable to perform hand gestures, such as when using tools or being restricted, as they require hand movements for controlling virtual item placement.
Innovation Solution
The implementation of an eye tracking system that interprets a user's gaze direction as input for hands-free positioning of virtual items, allowing users to adjust the position of virtual items by directing their gaze and performing specific facial gestures, such as closing an eye to 'grab' and moving their gaze to reposition, without needing hand gestures or dedicated input devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If hand gestures are used to control virtual item positioning, then positioning precision is improved, but ease of operation deteriorates when hands are occupied
Solution Approach 1:
The patent replaces the mechanical hand gesture control system with an optical eye tracking system. The eye tracking system uses cameras and image processing to detect eye movements and gaze direction, substituting the mechanical movement of hands with optical detection of eye movements. This allows users to control virtual item positioning without using their hands, resolving the contradiction between positioning precision and ease of operation when hands are occupied.
Solution Approach 2:
The patent introduces an intermediary mapping system that translates eye movement data into virtual item positioning commands. The eye tracking system captures raw eye movement data, and an intermediary algorithm maps these movements to control signals for virtual item manipulation. This intermediary layer enables precise control through eye movements, making the system usable when hands are occupied while maintaining positioning accuracy.
2Ease of operation
If eye tracking system is implemented, then ease of operation is improved for hands-free control, but device complexity increases
Solution Approach 1:
The patent integrates the eye tracking system into the existing NED device framework, allowing the same device to perform both traditional hand gesture control and new eye-based control. The eye tracking module shares processing resources and software infrastructure with other device functions, enabling hands-free operation capability without completely redesigning the device architecture, thus managing complexity while adding functionality.
Solution Approach 2:
The eye tracking system uses the device's existing cameras and processors to perform eye tracking functions. Rather than requiring entirely separate dedicated hardware, the system leverages existing components for image capture and processing, reducing the need for additional complex hardware and simplifying integration into the existing device.
3Productivity
If eye movements are used for control, then productivity is improved for tool-using tasks, but measurement precision may deteriorate due to eye movement limitations
Solution Approach 1:
The patent enhances eye movement control by incorporating head position and orientation data alongside eye tracking data. This multi-dimensional approach compensates for limitations in eye movement precision by adding spatial context from head tracking, allowing accurate virtual item positioning even when eye movements alone provide limited precision, thereby maintaining productivity for tool-using tasks.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables users to refine and anchor virtual item positions within the real-world environment without hand gestures, facilitating practical applications like inspecting industrial machinery without setting down tools, by dynamically tracking eye movements and correlating them with real-world coordinates.
Implementation Method 1
an eye tracking system to monitor a user's gaze direction
Data Source
Figure 1~2
Figure 3~4
Figure 5A
AI summary
A Near-Eye-Display (NED) device utilizes eye tracking to interpret gaze direction as user input for hands free positioning of virtual items. The NED device includes an eye tracking system to monitor gaze direction and a display component that renders virtual items within a user's view of a physical real-world environment. The NED device receives an indication that the user desires to adjust a position of the virtual item. In response, the NED device uses tracks the user's eye movements to dynamically change the position at which the display component is rendering the virtual item. In this way, a NED device may identify when a user desires to adjust a position at which a virtual item is being rendered and then may enable the user to make the desired adjustments simply by "dragging" the virtual item with deliberate and controlled eye movements.