HMD Depth Data Adjustment Using IMU Motion Compensation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing VR and AR display systems face inaccuracies in depth information due to motion artifacts, leading to detrimental visual artifacts that negatively impact the user experience.

Innovation Solution

A head-mounted display (HMD) system that adjusts depth information using inertial measurement unit (IMU) data to correct for motion-induced errors, enhancing the accuracy of depth data and reducing visual artifacts by transforming raw images captured by a depth camera to account for system movement.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If depth information is collected from a depth camera during motion, then depth data can be captured continuously, but the depth information becomes unreliable and inaccurate due to motion artifacts

Engineering Contradiction:

Improvecontinuous depth data captureVSAvoiddepth information accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The system performs preliminary actions by capturing non-visual motion data (accelerometer and gyroscope readings) during the depth image capture period. This motion data is then used to predict and compensate for motion artifacts in the depth information, allowing continuous depth capture while maintaining accuracy despite device movement.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements feedback by using non-visual sensor data (accelerometer and gyroscope) to detect motion during depth capture, then applying compensation transformations to correct the depth information. This closed-loop approach continuously monitors motion and adjusts depth data accordingly, resolving the contradiction between continuous capture and accuracy.

Inventive Principle:

Principle #23Feedback

2Measurement precision

If motion compensation is applied to depth data, then depth information accuracy is improved, but additional processing steps and computational resources are required

Engineering Contradiction:

Improvedepth information accuracyVSAvoidprocessing complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system introduces an intermediary approach by using non-visual motion sensors (accelerometer and gyroscope) as mediators to detect device motion. These sensors provide motion data that serves as an intermediate step between raw depth capture and corrected depth output, enabling compensation without requiring complex direct analysis of depth image changes.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system replaces complex mechanical or visual analysis methods with simpler sensor-based detection. Instead of analyzing depth image sequences to detect motion (which would be computationally intensive), the system substitutes this with direct readings from accelerometer and gyroscope sensors, significantly reducing processing complexity while maintaining compensation effectiveness.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentEP3533219B1Depth data adjustment based on non-visual pose data

Publication Date: 2022.02.16 GOOGLE LLC

Figure 1A~1B
Figure 2
Figure 3

AI summary

An HMD adjusts adjusting depth information based on detected motion of the system. The HMD includes a depth camera that collects depth data for objects in the local environment of the HMD. The HMD further includes an inertial measurement unit (IMU) including non-visual motion sensors such as one or more accelerometers, gyroscopes, and the like. The HMD adjusts the received depth information based on motion data provided by the IMU, thereby improving the accuracy of the depth information, and in turn reducing visual artifacts that can result from inaccuracies in the depth information.