Extended Reality Interaction via Eyelid Motion Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Extended reality (XR) systems face challenges in detecting user eye motion and focus within XR environments, leading to difficulties in determining the user's gaze and preferred interactions, especially with multiple objects in view, and current methods like hand gestures or joysticks are cumbersome.

Innovation Solution

The system detects eyelid motion to regenerate objects with modified detail levels, matches eyelid motions with stored identifiers to perform actions, and uses opacity-based indicators to reflect gaze shifts, allowing voice commands near objects to execute actions, enhancing user interaction and information access within XR environments.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If hand gestures or joysticks are used for navigation and interaction in XR environment, then user control capability is maintained, but user convenience deteriorates and immersion is reduced

Engineering Contradiction:

Improveuser convenienceVSAvoidinteraction complexity

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The patent replaces mechanical interaction systems (hand gestures, joysticks) with an optical-based eye tracking system. Sensors detect eye motions and eyelid movements to determine user gaze and intent, substituting physical mechanical controls with non-contact optical detection, thereby improving ease of operation while maintaining control capability

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system enables self-service interaction by automatically detecting user gaze direction and object selection through eye tracking without requiring explicit manual input. The XR system interprets natural eye movements and eyelid actions to perform navigation and object selection functions autonomously, reducing the cognitive and physical burden on the user

Inventive Principle:

Principle #25Self-service

2Measurement precision

If pupil dilation and constriction are monitored to determine gaze, then field of view information is obtained, but reliability deteriorates due to lack of user control over pupil

Engineering Contradiction:

Improvegaze detection accuracyVSAvoidgaze monitoring reliability

Core Design Contradiction:

Measurement precisionVSReliability

Solution Approach 1:

Instead of monitoring pupil dilation and constriction (which are involuntary and unreliable), the patent inverts the approach by monitoring eyelid movements (which are voluntary and controllable). The system detects eyelid position and motion to determine user intent, reversing the traditional physiological measurement approach to achieve reliable gaze tracking

Inventive Principle:

Principle #13The other way round (Inversion)

3Adaptability or versatility

If multiple objects are present in user field of view, then environment richness is improved, but object identification difficulty increases

Engineering Contradiction:

Improveenvironment complexityVSAvoidobject selection difficulty

Core Design Contradiction:

Adaptability or versatilityVSDifficulty of detecting and measuring

Solution Approach 1:

The patent introduces an intermediary selection mechanism where eye tracking data serves as a mediator between the user and multiple objects in the field of view. The system uses detected eye motions and eyelid actions to determine which specific object the user intends to interact with, resolving the ambiguity that arises when multiple objects are present and making object identification straightforward

Inventive Principle:

Principle #24Intermediary (Mediator)

4Loss of information

If detailed information about objects is always displayed, then information completeness is improved, but visual clutter increases

Engineering Contradiction:

Improveinformation accessibilityVSAvoidvisual clutter

Core Design Contradiction:

Loss of informationVSObject-generated harmful factors

Solution Approach 1:

The patent applies partial action by displaying information selectively rather than continuously. The system provides detailed object information only when the user's gaze is directed at a specific object and an information request is detected through eye tracking, avoiding constant display of all object information and thereby reducing visual clutter while maintaining information accessibility on demand

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS12026308B2Methods and systems of extended reality environment interaction based on eye motions

Publication Date: 2024.07.02 ADEIA GUIDES INC

AI summary

Systems and methods are described for extended reality environment interaction. An extended reality environment including an object is generated for display, and a table of eyelid motion identifiers and corresponding actions performable on the object in the extended reality environment is stored in memory. An eyelid motion is detected by using a sensor, and the detected eyelid motion is matched to one of the stored eyelid motion identifiers. An updated version of the extended reality environment, based on the action that corresponds to the matched eyelid motion, is generated for display, in response to matching the detected eyelid motion to one of the stored eyelid motion identifiers.