HMD Road Surface Alignment for In-Vehicle AR Driving Assistance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing driving assistance systems, such as HUDs and HMDs, fail to effectively integrate augmented reality images with the surrounding environment, leading to misalignment and reduced driving safety due to the complexity and limited quality of image projection.
Innovation Solution
A system incorporating a Head Mounted Display (HMD) with a GNSS module, cameras, and markers to calibrate and align augmented reality objects with the real-world environment, ensuring precise superimposition of images on the windscreen and surrounding environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If HUD 1.0 projection system is used to display driving information, then information can be projected onto the windscreen, but the projection extension is insufficient to cover the driver's complete field of view including side windows
Solution Approach 1:
The patent combines multiple display technologies into a unified system: HUD 1.0 for the windscreen, HUD 2.0 for the main field of view, and HMD for the secondary field of view including side windows. This merging of technologies enables complete coverage of all driving fields of view while maintaining the advantages of each individual technology.
Solution Approach 2:
The system creates a universal display platform that can adapt to different display requirements across multiple fields of view. The multi-functional system can display different types of information (navigation, speed, warnings) across different display surfaces (windscreen, main FOV, side windows) based on the driver's needs.
2Measurement precision
If HMD with static markers and dynamic head tracker is used to determine pose of augmented reality images, then positioning can be achieved, but the system becomes complex and image placement quality is limited with offsets from intended positions
Solution Approach 1:
The system performs preliminary calibration by capturing reference images of the environment from known positions and pre-computing the relationship between camera positions and environmental features. This preliminary action eliminates the need for complex real-time marker tracking and head pose estimation, achieving precise image placement through pre-established spatial relationships.
Solution Approach 2:
Instead of using complex physical markers and trackers, the system creates a digital copy or map of the environment from reference images. This digital representation is then used to accurately place augmented reality images by matching them with corresponding features in the pre-captured environmental model, simplifying the system while maintaining precision.
3Reliability
If augmented reality images are displayed through HMD, then driving assistance information can be provided, but the images do not harmonise with the real environment and appear offset from their intended placement
Solution Approach 1:
The system uses feedback from the camera to continuously monitor the driver's field of view and adjust the placement of augmented reality images accordingly. By comparing the current camera position and orientation with the pre-established environmental model, the system dynamically corrects image placement to ensure harmonious integration with the real environment.
Solution Approach 2:
The system transitions from static marker-based positioning to a dynamic solution that adapts to the driver's movements and environmental changes. The augmented reality images are dynamically repositioned based on real-time camera data and the pre-computed environmental model, ensuring consistent alignment reliability regardless of driver position or environmental variations.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention relates to a method (200,300) of assisting driving a vehicle (5). The method (200,300) is implemented by a system (1) comprising an HMD (10) and a positioning module (20) mounted on the vehicle (5), wherein the HMD (10) comprises a screen (11) and a pair of video cameras (15) located on opposite sides of the screen (11) along a main length direction of the screen (11). The method (200,300) comprises the steps of: a. acquiring (301) a pair of images (IL,IR) by means of the HMD (10), each image being acquired by a respective camera (15) of the HMD (10) and framing a same portion of field of view (FOV), said same portion of field of view (FOV) comprising a portion of road surface (ORO); b. estimating (303) a position in a three-dimensional space (VOL) of each point comprised in the same portion of field of view based on the pair of acquired images and the characteristics of the cameras; c. estimating (305) a volume of interest (VC) of the three-dimensional space (VOL) comprising the portion of the road surface (ORO) based on the pair of images (IL,IR) acquired and a position determined by the positioning module (5) in the contiguity of the acquisition of the pair of images (IL,IR); d. selecting (307) a subset of points comprised in the same portion of the field of view (FOV) and comprised in the volume of interest (VC); e. calculating (309) a planar surface (RS) corresponding to the road surface (ORO) based on the estimated position of the points comprised in the subset of selected points, and g. reproducing (311, 313, 215) a two-dimensional image (AR) on the screen (11) of the HMD (10) so that it is arranged on the planar surface (RS) corresponding to the road surface (ORO) calculated in step e.