AR Depth Mapping Using IMU Gravity Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality devices face challenges in maintaining high depth value accuracy away from the camera due to methods like stereo vision and ToF, which require additional hardware and increase power consumption, affecting portability and usage time.
Innovation Solution
An augmented reality device that utilizes an inertial measurement unit (IMU) sensor to measure gravity direction, adjusts normal vectors, and applies AI models to modify depth values based on these vectors, reducing the need for additional hardware and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If stereo vision method or ToF method is used to obtain depth information, then depth estimation accuracy is improved, but additional hardware modules are required and power consumption increases
Solution Approach 1:
The patent combines the camera module with depth estimation functionality by integrating AI-based depth prediction algorithms directly into the image processing pipeline. Instead of using separate depth sensors like ToF or structured light modules, the system merges depth estimation with the existing camera system, using the same image data for both visual input and depth map generation through AI models.
Solution Approach 2:
The patent replaces mechanical/optical depth sensing systems (ToF sensors, structured light projectors) with an AI-based computational approach. The depth information is obtained through software-based depth prediction models that process standard camera images, substituting physical depth measurement hardware with intelligent algorithms that estimate depth from visual data.
2Measurement precision
If structured light method or ToF method is used to obtain depth information, then depth value accuracy is improved, but power consumption increases
Solution Approach 1:
The patent replaces energy-intensive optical depth sensing systems (structured light projectors, ToF illuminators) with AI-based computational depth estimation. The AI model processes standard camera images to generate depth maps without requiring additional active illumination or specialized depth-sensing hardware, significantly reducing power consumption while maintaining acceptable depth accuracy.
Solution Approach 2:
The patent creates a virtual depth map as a computational copy of the physical depth information by training AI models to predict depth values from standard images. This virtual representation replicates the depth data that would otherwise require specialized hardware to capture, enabling depth-based applications without the associated power consumption.
3Adaptability or versatility
If depth information is obtained continuously for all augmented reality applications, then depth information availability is improved, but power consumption increases
Solution Approach 1:
The patent implements periodic or on-demand depth map generation based on application requirements and scene changes rather than continuous depth estimation. The system activates depth prediction only when needed for specific augmented reality functions or when scene changes warrant updated depth information, reducing unnecessary computational overhead and power consumption while maintaining depth information availability when required.
Data Source
AI summary
Provided are an augmented reality device and a method of operating the same. The augmented reality device adjusts a depth value based on a gravity direction measured by an IMU sensor in order to obtain a depth map having a high accuracy without an additional hardware module. The augmented reality device obtains a depth map from an image obtained using the camera, obtains a normal vector of at least one pixel included in the depth map, modifies a direction of the normal vector of the at least one pixel, based on the gravity direction measured by the IMU sensor, and adjusts a depth value of the at least one pixel based on the modified direction of the normal vector.