Hand Attitude Estimation Using Inertial and Imaging Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing XR systems face challenges in accurately detecting the attitude of a user's hand due to the need for multiple light-emitting diodes in hand controllers, which hinders miniaturization, and detection accuracy is affected by the user's hand direction.
Innovation Solution
A hand controller equipped with an inertial sensor and an HMD system that combines inertial information with imaging data to estimate hand attitude, using a combination of inertial and imaging units to enhance accuracy and allow for miniaturization without additional sensors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple light-emitting diodes are mounted in the hand controller to detect hand position and attitude, then detection accuracy is improved, but the controller size increases and miniaturization becomes difficult
Solution Approach 1:
The patent extracts the light-emitting function from the hand controller and relocates it to the HMD. The HMD's imaging unit emits infrared light and detects hand reflections, eliminating the need for light-emitting diodes in the controller while maintaining detection capability.
Solution Approach 2:
The patent introduces an intermediary approach where the HMD's imaging unit serves as both the light source and detector. The imaging unit emits infrared light and captures reflected light from the hand, acting as an intermediary system that enables detection without adding components to the controller.
2Ease of operation
If body portion comparison with bone model is used to detect hand position, then detection capability is provided, but detection accuracy reduces depending on hand direction
Solution Approach 1:
The patent employs feedback by continuously capturing images of the hand and comparing them with the stored bone model. The system adjusts and refines the hand attitude estimation based on the visual feedback from captured images, maintaining high accuracy across different hand directions.
Solution Approach 2:
The patent utilizes infrared light reflection properties where different body parts reflect infrared light differently. By detecting these reflection patterns and comparing them with the bone model, the system accurately determines hand attitude regardless of hand direction.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables accurate estimation of hand attitude by integrating inertial and imaging data, ensuring precise hand detection and miniaturization of the controller.
Implementation Method 1
an acquisition unit configured to acquire inertial information from an inertial sensor provided in the controller
Implementation Method 2
a determination unit configured to determine whether a specific portion of the hand of the user is detected in a captured image acquired by imaging of an imaging unit
Data Source
AI summary
An information processing device estimates an attitude of a hand of a user holding a controller with the hand. The information processing device has an acquisition unit, a determination unit, and an estimation unit. The acquisition unit acquires inertial information from an inertial sensor provided in the controller. The determination unit determines whether a specific portion of the hand of the user is detected in a captured image acquired by imaging of an imaging unit. The estimation unit estimates the attitude of the hand of the user on a basis of the captured image and the inertial information in a case where the specific portion is detected in the captured image.