HMD Motion Control Using Image-Based Vehicle Motion Subtraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mixed reality and virtual reality head-mounted display (HMD) systems face issues when a user is in a moving vehicle, as the information from acceleration and angular velocity sensors includes both user and vehicle motion, leading to incorrect video image display, and require an additional motion sensor on the vehicle, which is inconvenient.
Innovation Solution
A display device system that includes a controller and a head-mounted display with inertial measurement units to distinguish user operations from vehicle motion without the need for a separate motion sensor, using image capturing and estimation units to calculate and subtract vehicle motion from user input, allowing accurate display control based on user intentions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If motion sensors are used to detect user operations in an HMD, then user operations can be detected, but vehicle motion is also detected causing incorrect video image display
Solution Approach 1:
The patent segments the motion detection task by separating user motion from vehicle motion through multi-step calculation. It divides the total motion detected by sensors into components: vehicle body motion (estimated from captured images) and user operation motion (obtained from sensors), then subtracts the vehicle motion component to isolate pure user operations for accurate control.
Solution Approach 2:
The patent introduces captured images as an intermediary element to estimate vehicle motion. Instead of directly using sensor data that contains both user and vehicle motion, the system uses images captured by the HMD's imaging unit as a mediator to infer vehicle movement, which then serves as a reference to correct the sensor-based motion detection.
2Measurement precision
If a motion sensor is installed on the vehicle to separate vehicle motion from user motion, then accurate user operation detection is possible, but device complexity increases
Solution Approach 1:
The patent extracts vehicle motion information from the captured images without requiring a separate motion sensor on the vehicle. It takes out only the necessary vehicle motion component from the image data through estimation algorithms, eliminating the need to install additional sensors while still achieving accurate separation of user and vehicle motion.
Solution Approach 2:
The system uses the HMD's own captured images to estimate vehicle motion, making the device self-sufficient. The HMD utilizes its built-in imaging capabilities to generate the reference data needed for vehicle motion estimation, eliminating dependence on external vehicle-mounted sensors or additional hardware.
3Productivity
If sensor data from HMD is used directly for display control, then simple control processing is achieved, but video image display does not match user intention when vehicle is moving
Solution Approach 1:
The patent performs preliminary estimation of vehicle motion from captured images before processing the sensor data for display control. By pre-calculating the vehicle motion component and storing it as reference data, the system prepares the correction information in advance, allowing efficient real-time subtraction from sensor data during actual control operations without significant processing delay.
Data Source
AI summary
An information processing device connected to or integrated into a display device controls the display device to display a virtual object, obtains a first amount of change as an amount of change in one of a position and an orientation of a controller, obtains a second amount of change as an amount of change in one of a position and an orientation of the display device, estimates one of the position and the orientation of the display device based on a captured image, and obtains a third amount of change as an amount of change in one of the position and the orientation of the display device based on an estimation result, wherein the display device displays the virtual object based on a fourth amount of change as an amount of change in one of a position and an orientation based on the first, second and third amounts of change.