Vehicle HMD Layered Rendering for Stable AR During Motion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Head-mounted displays (HMDs) used in motor vehicles experience rendering artifacts such as jumpy virtual objects and motion sickness when tracking granular structures or textures during fast head movements or vehicle rotations, due to low frame rates and latency, which are exacerbated by the complexity of graphic calculations and the need to differentiate between head and vehicle movements.
Innovation Solution
A method for operating HMDs in motor vehicles that maintains a congruent coordinate system between the virtual and real environments by calculating new frames with a preset frame rate, using contextual layers for separate rendering and homographic reprojection of virtual objects, and applying a restricted pose signal to simulate inertia and limit movement dynamics, reducing rendering artifacts and motion sickness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If the frame rate is increased to reduce rendering artifacts and improve tracking of granular structures, then the quality of VR experience is improved, but the computational complexity and processing time increase
Solution Approach 1:
The system pre-calculates and stores transformation matrices for common head pose changes. When a head movement is detected, the system retrieves and applies the pre-computed matrix instead of performing full rendering calculations, significantly reducing processing time while maintaining rendering precision for granular structures
Solution Approach 2:
The system dynamically adjusts the rendering pipeline based on detected motion intensity. During fast head movements, it applies optimized transformation algorithms with reduced calculation steps, while during stable periods it uses full-resolution rendering. This dynamic adaptation maintains quality where needed while reducing overall processing time
2Speed
If asynchronous reprojection is applied to correct disparity and ensure fast response time, then the response time is improved, but rendering artifacts are induced in the form of flickering virtual objects and undefined edge areas
Solution Approach 1:
The system applies different processing strategies to different regions of the rendered image. Static virtual objects (HUD elements, text) are exempt from aggressive reprojection warping, while dynamic background elements use full asynchronous reprojection. This local differentiation maintains response time while preventing flickering artifacts in critical display areas
Solution Approach 2:
The rendering pipeline is segmented into multiple layers: a static layer for HUD elements and text that uses precise coordinate tracking without warping, and a dynamic layer for background scenery that uses asynchronous reprojection. This segmentation allows the system to maintain fast response time for overall rendering while preserving rendering quality for critical static elements
3Stability of the object's composition
If the coordinate system is continuously tracked to maintain congruency between virtual and real environments, then the stability of the virtual environment is improved, but computational complexity increases due to the need to differentiate head movements from vehicle movements
Solution Approach 1:
The system introduces an intermediary reference frame that represents the vehicle's motion. By separating head movement detection into two components (vehicle motion via sensors and relative head motion via IMU), the system can maintain coordinate stability without directly processing the complex differentiation of all motion sources. The intermediary vehicle reference frame simplifies the computational model
Data Source
AI summary
The invention relates to a method for operating a head-mounted display (17) in a motor vehicle (10) during a journey through a real external environment (11). In this method individual images or frames of a view (21) of a virtual environment (20) are successively newly rendered, so that the virtual environment (20) can be kept congruent with the real external environment (11), and by shifting and/or rotating the view (21) of the virtual environment (20) it is possible to compensate for a change in a position (19) of the head-mounted display caused by a head movement and/or a travelling movement. According to the invention pixels of the relevant frame (40) are rendered in at least two different contextual levels and thereafter the relevant frame is composed of the pixels of the contextual levels, the pixels of different virtual objects (22) being displayed in each of the contextual levels, and when the frames (40) are newly rendered (38) for the shifting and/or rotation of the view (21) in at least two of the contextual levels, different position signals (36, 37) are used as the basis.