Multi-Plane AR Windshield Projection With Feedback Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vehicle heads-up displays (HUDs) face misalignment and low quality when projecting augmented reality (AR) images onto windshields, resulting in an undesirable aesthetic appearance and unclarity due to the challenge of aligning multiple sub-images with their corresponding objects around the vehicle.
Innovation Solution
A neural network is trained to generate a pixel-wise phase matrix, which is used by a spatial light modulator (SLM) to output AR images into multiple focal planes, with an image sensor providing feedback to update the network's parameters and improve alignment and quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If multiple sub-images are projected onto a single focal plane (windshield), then the AR image can be displayed, but misalignment and low quality occur between sub-images and corresponding objects
Solution Approach 1:
The patent segments the AR image into multiple sub-images, each corresponding to a different focal plane. Instead of projecting all sub-images onto a single plane, each sub-image is projected to its appropriate depth plane, resolving the alignment issue between sub-images and real-world objects at different distances.
Solution Approach 2:
The patent adds the depth dimension to the traditional 2D HUD display by utilizing multiple focal planes. This transforms the display from a flat single-plane projection to a multi-planar 3D-like presentation, allowing sub-images to be aligned with objects at various depths in the real world.
2Device complexity
If a single focal plane is used for AR image projection, then the display system is simpler, but the aesthetic appearance and clarity deteriorate
Solution Approach 1:
The patent employs a dynamic approach where the display system can adaptively adjust which focal plane each sub-image is projected to, based on the depth information of corresponding real-world objects. This dynamic allocation optimizes alignment and image quality while maintaining system flexibility.
Solution Approach 2:
The patent replaces complex mechanical adjustment mechanisms with a computational approach using neural networks to determine optimal focal plane assignments. This substitution maintains simplicity while achieving high-quality alignment through software-based optimization rather than mechanical adjustment.
3Ease of operation
If traditional HUD projection methods are used, then the implementation is straightforward, but alignment precision and image quality are poor
Solution Approach 1:
The patent implements a feedback mechanism where the neural network learns from training data to optimize the assignment of sub-images to focal planes. The system uses feedback from alignment quality metrics to continuously improve its projection decisions, achieving high precision without complex manual calibration.
Solution Approach 2:
The patent changes the projection parameters by varying which focal plane each sub-image is assigned to, based on the depth characteristics of corresponding real-world objects. This parameter optimization enables precise alignment while maintaining implementation simplicity through automated decision-making.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution enhances the alignment and quality of AR images displayed on the windshield, providing a desirable aesthetic appearance with high clarity by optimizing the pixel-wise phase matrix based on the vehicle's windshield and SLM, thus improving the overall HUD experience.
Implementation Method 1
a spatial light modulator (SLM) actuated to output the AR image based on the pixel-wise phase matrix
Data Source
AI summary
A set of training images are input into a neural network that outputs a pixel-wise phase matrix identifying pixels in the training images. Based on the pixel-wise phase matrix, a spatial light modulator (SLM) is actuated to output, onto a vehicle windshield, an augmented reality (AR) image including a plurality of sub-images each output in one of a plurality of focal planes. Each training image corresponds to one respective sub-image. A feedback image of the AR image is obtained via an image sensor. An offset is determined based on comparing the training images to the feedback image. Parameters of a loss function are updated based on the offset, and the updated parameters are provided to the neural network to obtain an updated offset.