Multimodal Sensor Alignment in Near-Eye Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of addressing binocular alignment in augmented reality (AR) devices with a form factor similar to glasses, which introduces structural deformations leading to tracking errors and visual discomfort due to sensor misalignments, is not adequately solved by existing systems.
Innovation Solution
A near-to-eye display device utilizing multiple sensors across different modalities, such as stereo depth and time-of-flight measurements, along with inertial measurement units (IMUs), to track device deformations and adjust rendered images in real-time to maintain alignment with the environment, compensating for structural changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional single-modality sensors are used, then device complexity is low, but measurement precision and reliability of misalignment detection deteriorate
Solution Approach 1:
The patent merges multiple sensing modalities (optical sensors, acoustic sensors, capacitive sensors, inductive sensors) into a coordinated sensor system. Each sensor type detects misalignment through different physical principles, and their data is fused to achieve superior detection precision while maintaining manageable system complexity through modular architecture.
Solution Approach 2:
The sensor system is designed with multi-functionality where various sensor modalities serve the common purpose of misalignment detection. The system can detect misalignment in different dimensions (lateral, vertical, angular) using appropriate sensor types, making the overall system universally applicable to various misalignment scenarios without requiring separate specialized systems for each type.
2Reliability
If multiple sensors are deployed to improve detection accuracy, then measurement precision improves, but device complexity and cost increase
Solution Approach 1:
The system implements feedback mechanisms where sensor data is continuously processed to determine misalignment conditions, and this information feeds back to control mechanisms that adjust positioning or alert operators. This feedback loop enhances reliability by enabling real-time detection and response, while the automated feedback processing reduces manual coordination complexity.
Solution Approach 2:
The sensor system is segmented into modular functional units (optical detection module, acoustic detection module, capacitive detection module, inductive detection module). Each module operates independently with its own signal processing chain, then results are integrated at a higher level. This segmentation improves reliability through redundancy while simplifying coordination complexity through standardized modular interfaces.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach allows for accurate tracking of device deformations, reducing binocular disparity and user discomfort by maintaining alignment and stability of rendered objects, even under flexible conditions, ensuring a visually comfortable AR experience.
Implementation Method 1
Optical sensors may include cameras, photodetectors, or other devices that detect light properties such as intensity, color, or polarization
Implementation Method 2
Optical sensors may include cameras, photodetectors, or other devices that detect light properties such as intensity, color, or polarization
Implementation Method 3
Acoustic sensors may include microphones, ultrasonic transducers, or other devices that detect sound properties such as frequency or amplitude
Implementation Method 4
Capacitive sensors detect changes in capacitance caused by proximity or position changes
Implementation Method 5
Inductive sensors detect changes in inductance or magnetic field properties caused by proximity or position changes
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The techniques disclosed herein detect sensor misalignments in a display device by the use of sensors operating under different modalities. In some configurations, a near-to-eye display device can include a number of sensors that can be used to track movement of the device relative to a surrounding environment. The device can utilize multiple sensors operating under multiple modalities. For each sensor, there is a set of intrinsic and extrinsic properties that are calibrated. The device is also configured to determine refined estimations of the intrinsic and extrinsic properties at runtime. The refined estimations of the intrinsic and extrinsic properties can then be used to derive knowledge on how the device has deformed over time. The device can then use the refined estimations of the intrinsic and extrinsic properties and/or any other resulting data that quantifies any deformations of the device to make adjustments to rendered images at runtime.