Hand Position Rerendering Using Headset-Eye Offset for Realistic VR/AR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional motion capture systems are cumbersome, expensive, and limited by the need for bulky equipment and real-time data processing, leading to unrealistic interactions in virtual reality (VR) and augmented reality (AR) environments, particularly in grasping virtual objects and aligning AR with the real world.
Innovation Solution
A system that uses sensors to determine the offset between a user's eyes and a head-mounted device, re-rendering the hand's position to align with the user's perspective, allowing for realistic hand interactions in VR/AR by detecting contact points to infer actions such as grasping, rotating, or stretching virtual objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional motion capture systems are used, then hand position can be detected, but the equipment becomes bulky and expensive
Solution Approach 1:
The patent uses image processing to create a virtual copy of the hand from 2D images, replacing the need for physical motion capture equipment. The system captures images of the hand and computationally reconstructs the 3D hand model and position, eliminating bulky hardware while maintaining measurement precision.
Solution Approach 2:
The patent replaces mechanical motion capture systems with an optical imaging-based system. Instead of using sensors and markers to physically track hand movement, the system uses cameras to capture images and computationally determines hand position and geometry through image processing algorithms.
2Measurement precision
If conventional motion capture systems are used, then hand position can be detected, but real-time processing becomes difficult
Solution Approach 1:
The patent extracts only the essential information needed for hand tracking from the captured images - specifically the hand position, orientation, and geometry - while discarding unnecessary visual data. This selective extraction enables real-time processing by reducing the computational burden compared to processing complete motion capture datasets.
Solution Approach 2:
The system uses a simplified image processing approach that processes only the necessary portions of the image data required to determine hand position and pose. By using selective image analysis rather than comprehensive motion capture data processing, the system achieves real-time performance while maintaining sufficient measurement precision.
3Ease of operation
If hand position is rendered using conventional methods, then virtual interactions can be implemented, but the hand size appears incorrect and interactions look unrealistic
Solution Approach 1:
The patent creates an accurate 3D copy of the hand by analyzing the geometry and proportions from captured images. The system computes the hand model by scaling and transforming the captured hand geometry to match the user's actual hand size, ensuring realistic hand representation in virtual interactions.
Solution Approach 2:
The system dynamically adjusts hand model parameters including size, shape, and position based on the captured image data. By computing scale factors and transformation parameters from the captured hand geometry, the system accurately renders the hand at the correct size and position for realistic virtual interactions.
Data Source
AI summary
The technology disclosed relates to a method of realistic rendering of a real object as a virtual object in a virtual space using an offset in the position of the hand in a three-dimensional (3D) sensory space. An offset between expected positions of the eye(s) of a wearer of a head mounted device and a sensor attached to the head mounted device for sensing a position of at least one hand in a three-dimensional (3D) sensory space is determined. A position of the hand in the three-dimensional (3D) sensory space can be sensed using a sensor. The sensed position of the hand can be transformed by the offset into a re-rendered position of the hand as would appear to the wearer of the head mounted device if the wearer were looking at the actual hand. The re-rendered hand can be depicted to the wearer of the head mounted device.