Mixed Reality Display with Gaze-Tracked Sensor Array
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current mixed reality (MR) displays have a narrow field of view and a depth of field that is far from exquisite, limiting the immersion and quality of the mixed reality experience.
Innovation Solution
The method involves an MR apparatus with a display screen, an eye tracker, and an image sensor array, where the image sensors are correspondingly related to display sub-regions. The apparatus determines the user's gaze point, adjusts the resolution of specific image sensors based on gaze duration, and superimposes environment images from the image sensor array with virtual images to enhance the MR experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If holographic projection is used for superimposition and combination of virtual and real worlds, then the MR display can be achieved, but the field of view becomes narrow and depth of field becomes far from exquisite
Solution Approach 1:
The patent divides the display screen into multiple display sub-regions and corresponds each sub-region with a dedicated image sensor from the array. This segmentation allows different regions to be captured with appropriate resolution and depth of field characteristics, thereby improving overall field of view and depth of field quality simultaneously
Solution Approach 2:
The patent transitions from traditional 2D holographic projection to a multi-dimensional approach by using an array of image sensors that capture depth information and spatial characteristics. This enables exquisite depth of field rendering while expanding the field of view through multi-perspective imaging
2Adaptability or versatility
If image sensor array is used to capture environment images, then the field of view and depth of field are improved, but the device complexity increases
Solution Approach 1:
The patent makes the display screen serve multiple functions: it acts as both the display surface for virtual content and the mounting surface for the image sensor array. This multi-functionality reduces device complexity by eliminating separate structural components while still achieving improved field of view and depth of field
Solution Approach 2:
The patent merges the display system and imaging system into a unified structure where the display screen and image sensor array are integrated. This combination simplifies the overall device architecture while maintaining the benefits of enhanced field of view and depth of field through the image sensor array
Data Source
AI summary
Provided are a mixed reality display method, a mixed reality device, and a storage medium. The method includes: after receiving a mixed display signal, tracking the eyeballs of a user by means of an eyeball tracker, and determining a gaze point of the user on a display screen; determining, according to a correspondence between the gaze point and a sub-display region in the display screen, the sub-display region the user gazes; determining at least one first image sensor according to a correspondence between the sub-display region and an image sensor in an image sensor array, and adjusting the at least one first image sensor; and superimposing and rendering an environment image output by the image sensor array and a virtual image, and obtaining and displaying an MR image.