Vanishing-Point Orientation for Stable XR Downward Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing extended reality (XR) systems using inertial measurement units (IMUs) for defining a coordinate system are unstable or skewed in user perception, particularly in moving environments such as inclines or vehicles undergoing acceleration.
Innovation Solution
Determine a coordinate system based on vanishing points derived from image analysis of the physical environment, using algorithms like RANSAC and voting, to align virtual content with a user's perception of 'down' direction, which is more stable than relying solely on IMU vectors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a coordinate system is defined based on IMU gravity vector, then the system is simple to implement, but the coordinate system becomes unstable or skewed in moving environments
Solution Approach 1:
The patent replaces the mechanical/physical IMU-based coordinate system definition with an image-processing-based approach. Instead of relying on gravity vectors measured by IMU, the system uses vanishing point detection from images to define the downward direction, thereby substituting a different measurement paradigm to achieve better stability in dynamic environments
Solution Approach 2:
The patent introduces an intermediary approach by using image analysis as a mediator between the physical environment and the coordinate system definition. The image processing pipeline (detecting lines, calculating vanishing points) serves as an intermediary that translates visual information into a stable coordinate system, avoiding direct reliance on IMU data
2Ease of manufacture
If a coordinate system is defined based on IMU gravity vector, then the implementation is straightforward, but the downward direction becomes inaccurate on inclines or during acceleration
Solution Approach 1:
The patent substitutes the IMU-based measurement approach with an image-based measurement approach. By detecting vanishing points from images of the physical environment, the system accurately determines the downward direction even on inclines or during acceleration, overcoming the limitations of gravity vector measurement
Solution Approach 2:
The patent employs feedback mechanisms through iterative refinement of the coordinate system. The system continuously processes images to detect vanishing points and adjusts the downward direction definition based on the detected visual features, providing feedback that corrects inaccuracies in real-time
Data Source
AI summary
In one implementation, a method of determining a downward direction is performed by a device including one or more processors, non-transitory memory, and an image sensor. The method includes capturing, using the image sensor, an image of a physical environment. The method includes detecting a plurality of lines within the image of the physical environment. The method includes determining one or more vanishing directions based on the plurality of lines. The method includes defining a downward direction based on the one or more vanishing directions.