3D Gaze Control for Robot Navigation and Object Manipulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current gaze-tracking technologies are limited to 2D tracking, restricting the number and type of commands that can be given, and require tedious eye gestures for control, making it cumbersome for users to control robotic devices in three-dimensional spaces.
Innovation Solution
The development of a 3D gaze tracking system that uses binocular head-mounted tracking with intelligent fuzzy algorithms and neural networks to differentiate intentional eye movements from normal behavioral movements, allowing users to control robotic devices with smooth and continuous navigation in 3D space based on gaze position.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If 2D gaze tracking is used on computer display, then the system complexity is reduced, but the control capability in three-dimensional space is limited
Solution Approach 1:
The patent transitions from 2D gaze tracking on a computer display to 3D gaze tracking in real-world space. The system uses multiple cameras to capture eye images from different viewpoints, enabling determination of gaze position in three-dimensional space rather than limited to two-dimensional screen coordinates. This dimensional expansion resolves the contradiction by maintaining system feasibility while dramatically improving control capability in 3D environments.
2Speed
If gaze control is implemented without differentiation of intentional eye movements, then the response speed is improved, but the accuracy of command recognition deteriorates due to the Midas touch problem
Solution Approach 1:
The system employs feedback mechanisms to differentiate intentional eye movements from normal behavioral movements. By analyzing eye movement patterns, fixation duration, and saccade characteristics, the system provides feedback to distinguish deliberate commands from incidental eye movements. This feedback loop enables accurate command recognition while maintaining relatively fast response times, resolving the contradiction between speed and precision.
3Measurement precision
If traditional eye gestures such as excessive blinks or deliberate dwell time are required for command confirmation, then the accuracy of command selection is improved, but the ease of operation deteriorates due to tedious eye gestures
Solution Approach 1:
The patent extracts and eliminates the need for tedious eye gestures such as excessive blinks or deliberate dwell time for command confirmation. By using 3D gaze tracking and pattern recognition algorithms, the system directly interprets intentional eye movements as commands without requiring additional confirmation gestures. This extraction of unnecessary gesture requirements dramatically improves ease of operation while maintaining accurate command selection through sophisticated eye movement analysis.
Data Source
AI summary
Disclosed herein are devices, methods, and systems for controlling a robot or assistive device that allows the robot or device to find and manipulate objects in a real world environment. The disclosed devices, methods, and systems may, in many cases, receive control input through monitoring, tracking, and analyzing the 3D gaze of a user/controller. Using the described 3D eye tracking, a user can directly and naturally look at an object of interest in the real world, while the system monitors, tracks, and records the user's 3D gaze position. This information, in many cases, is then translated into input commands for the robot or assistive.