HMD Avatar Appearance Control During Device Removal
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques fail to adequately maintain user representation appearance when head-mounted devices (HMDs) are doffed or removed, leading to distorted or unnatural changes in user avatars due to interrupted sensor data.
Innovation Solution
Utilizing sensor data to detect HMD removal through motion and eye tracking, and applying algorithms or machine learning to maintain a neutral avatar appearance by freezing the current representation or transitioning to a pre-session neutral appearance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If sensor data is continuously used to generate user representation during HMD removal, then the user representation reflects real-time changes, but the appearance becomes distorted or unnatural
Solution Approach 1:
The system detects HMD removal in advance using motion sensors and eye tracking before the user representation becomes distorted. By identifying the removal event early (when the HMD starts moving from the eye box region), the system can proactively freeze the user representation to maintain natural appearance, preventing the distortion that would occur from continuing to use interrupted sensor data.
2Shape
If the user representation is frozen to maintain appearance during HMD removal, then the appearance remains natural, but the representation does not reflect current state
Solution Approach 1:
The system dynamically adjusts the user representation strategy based on HMD state. During normal use, the representation continuously reflects real-time sensor data. Upon detecting HMD removal, it transitions to a frozen state to maintain appearance quality. The system then dynamically switches between these modes based on whether the HMD is in the eye box region or being removed, optimizing both appearance quality and information accuracy for each operational phase.
3Speed
If eye tracking is used to detect HMD removal, then removal can be detected promptly, but the system complexity increases
Solution Approach 1:
The system uses eye tracking technology that serves multiple functions: it detects HMD removal, monitors user gaze for interaction purposes, and tracks eye position for display rendering. By making the eye tracking system multi-functional, the patent avoids adding separate dedicated removal detection sensors, thereby achieving fast removal detection without proportionally increasing system complexity.
Data Source
AI summary
Various implementations disclosed herein include devices, systems, and methods that use sensor data to recognize circumstances in which a user representation should be configured to account for head-mounted device (HMD) doffing or other removal from normal use position, e.g., ensuring that the user representation is not deformed or otherwise changed in an unnatural/undesirable way based on the images or other sensor data of the user's eyes and/or other face portions not being captured from expected capture positions as the device is doffed or otherwise removed from its normal use position. As examples, this may involve using motion sensor data to detect device changes in position and/or orientation and/or image sensor data to determine when the user's eye (whether open or closed) is present within the eye box region of the device.