HMD Gyroscope Segmentation for Motion-to-Photon Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Head-mounted display systems for virtual and augmented reality suffer from motion-to-photon latency, leading to user disorientation due to the lag between head movements and the rendering and display of 3D imagery, which is not adequately addressed by existing technologies.
Innovation Solution
The implementation of an electronic display stabilization process that decouples application processor and display hardware, using separate or differently sampled gyroscopes to determine and compensate for head rotations, allowing for independent operation and reduced latency by warping images before display, thereby aligning the displayed scene with the user's current head pose.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If the system uses a single shared gyroscope for both application processor and display hardware, then device complexity is reduced, but motion-to-photon latency increases due to shared resource contention
Solution Approach 1:
The patent divides the single gyroscope resource into two separate gyroscopes, one dedicated to the application processor and another dedicated to the display hardware. This segmentation eliminates resource contention and allows independent operation, reducing motion-to-photon latency while maintaining manageable device complexity through modular architecture.
Solution Approach 2:
The patent introduces an intermediary mechanism (separate gyroscope for display hardware) that mediates between the application processor and the display system. This intermediary allows the display hardware to independently measure head rotation without waiting for the application processor, effectively bridging the latency gap.
2Device complexity
If the system couples application processor and display hardware operations, then device complexity is reduced, but motion-to-photon latency increases due to synchronized operation requirements
Solution Approach 1:
The patent segments the coupled system into independent operational units: the application processor handles texture rendering while display hardware independently handles frame composition and stabilization. This segmentation allows parallel processing without synchronization delays, reducing motion-to-photon latency while maintaining architectural simplicity.
Solution Approach 2:
The patent implements dynamic independence where the display hardware can operate autonomously from the application processor. The display system dynamically adjusts frame composition and stabilization based on real-time gyroscope data without waiting for application processor completion, enabling flexible, latency-reduced operation.
3Use of energy by moving object
If the system renders textures at lower rates to reduce processing load, then energy consumption decreases, but motion-to-photon latency increases due to fewer update opportunities
Solution Approach 1:
The patent uses the display hardware as an intermediary that independently composes frames from rendered textures. Even though textures are rendered at lower rates, the display hardware can create multiple display frames from each texture by applying different stabilization transformations based on intermediate gyroscope measurements, maintaining high effective frame rates without proportionally increasing processing energy.
Solution Approach 2:
The patent performs preliminary texture rendering at lower rates, then uses the display hardware to generate multiple display frames from each rendered texture through intermediate stabilization steps. This preliminary action approach reduces processing energy while maintaining perceptual frame rates through intelligent frame generation.
Data Source
AI summary
A method includes determining, at a first time, a representation of a first head rotation of a head mounted display (HMD) using a first inertial sensor sample stream and rendering, at an application processor, a texture based on the first head rotation. The method further includes determining, at a second time subsequent to the first time, a representation of a second head rotation of the HMD using a second inertial sensor sample stream having a higher sampling rate than the first inertial sensor sample stream, and generating, at a compositor, a rotated representation of the texture based on a difference between the first head rotation and the second head rotation.