Reverse Pass-Through Glasses for AR/VR Face Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AR and VR devices lack the ability for onlookers to accurately determine the user's state of mind or focus of attention, as they typically use two-dimensional displays that do not provide a realistic view of the user's facial features, leading to disconnection and potential hazards in interactions.
Innovation Solution
The implementation of a reverse pass-through feature using autostereoscopic external displays and light field technology to project a three-dimensional reconstruction of the user's face, allowing onlookers to see a realistic and accurate view of the user's facial features, including gaze direction and focus, through a combination of eye imaging systems, micro lens arrays, and light field displays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional two-dimensional displays are used for outward facing displays, then the device complexity is reduced, but the realism and accuracy of the user's facial features view is deteriorated
Solution Approach 1:
The patent transitions from traditional two-dimensional displays to light field displays that render three-dimensional images. This dimensional change allows onlookers to view the user's facial features with accurate depth and spatial information, resolving the contradiction by prioritizing measurement precision (accuracy of facial features view) while accepting increased device complexity through the use of advanced light field technology and micro lens arrays.
2Loss of information
If outward facing displays are implemented, then onlookers can see what the user is experiencing, but the onlookers cannot determine the user's state of mind or focus of attention
Solution Approach 1:
The patent segments the imaging function by adding dedicated eye imaging systems that capture images of the user's eyes separately from the main outward facing display. This segmentation allows the system to extract and display specific information about the user's gaze direction and focus of attention, resolving the information loss about user state of mind while managing device complexity through functional decomposition.
Solution Approach 2:
The patent introduces an intermediary processing system that receives images from both the outward facing display and the eye imaging system. This intermediary processes the eye images to determine gaze direction and focus of attention, then integrates this information into the light field display. This mediator resolves the information loss by synthesizing multiple data sources while managing overall system complexity.
3Reliability
If light field displays with three-dimensional reconstruction are used, then the realism of the user's face view is improved, but the device complexity and manufacturing difficulty increase
Solution Approach 1:
The patent employs light field display technology that serves multiple functions: it displays the outward facing view, renders three-dimensional reconstruction of the user's face, and presents accurate depth information. This multi-functionality improves the realism and reliability of facial features representation while addressing manufacturing challenges by consolidating multiple display functions into a single integrated system rather than requiring separate components.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution enhances in-person interaction by providing a more natural and immersive experience, allowing onlookers to better engage with AR/VR users and improving co-presence by offering a realistic and accurate representation of the user's attention and focus, thus reducing friction between VR users and non-users.
Implementation Method 1
an eye imaging system configured to collect an image of a face of the user and a light field display configured to provide an autostereoscopic image
Data Source
AI summary
A device for providing a reverse pass-through view of a user of a headset display to an onlooker includes an eyepiece comprising an optical surface configured to provide an image to a user on a first side of the optical surface. The device also includes a first camera configured to collect an image of a portion of a face of the user reflected from the optical surface in a first field of view, a display adjacent to the optical surface and configured to project forward an image of the face of the user, and a screen configured to receive light from the display and provide the image of the face of the user to an onlooker.