Virtual Skeleton Remapping for NUI Tracking Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Natural User Interface (NUI) systems face challenges in distinguishing users from other objects and accurately tracking body parts, especially when users move quickly or out of the field of view, leading to unreliable image data and a less than desirable user experience.
Innovation Solution
The technology dynamically remaps components of a virtual skeleton by identifying needs for enhancement, replacement, or correction, using model data to generate a remapped skeleton that can drive user-controlled animations with enhanced movement and appearance characteristics, and resolve conflicts with virtual environment objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional image recognition systems are used to track body parts, then the system can detect user gestures, but the tracking becomes unreliable when users move quickly or out of the field of view
Solution Approach 1:
The system performs preliminary actions by predicting future positions of body parts based on current motion trends and historical data. This allows the system to proactively prepare for upcoming gestures or movements, ensuring continuous tracking even when parts move quickly or temporarily exit the field of view, thereby resolving the unreliability issue.
Solution Approach 2:
The system creates virtual copies of body parts in the tracking model. When a body part moves out of view or becomes obscured, the virtual copy maintains the expected position and trajectory based on predictive algorithms, allowing the system to continue tracking as if the actual part were visible, thus overcoming detection difficulties.
2Adaptability or versatility
If the virtual skeleton components are strictly based on captured image data, then the system maintains data accuracy, but the appearance and movement characteristics lack enhancement
Solution Approach 1:
The system merges captured image data with pre-defined 3D skeletal models and motion capture databases. This combination allows the system to maintain measurement precision from the captured data while incorporating enhanced appearance and movement characteristics from the reference models, achieving both accuracy and adaptability simultaneously.
Solution Approach 2:
The system dynamically adjusts parameters of the virtual skeleton components based on the captured data. When enhancement is desired, it modifies parameters such as joint rotation ranges, limb proportions, or movement smoothness while maintaining the core positional accuracy from the captured image data, thus achieving both precision and enhanced characteristics.
3Productivity
If the system tracks all body parts continuously, then complete gesture recognition is achieved, but computational resources and processing time increase
Solution Approach 1:
The system segments the body into multiple independent tracking components (head, torso, limbs, hands, feet) and processes them separately using specialized algorithms for each segment. This segmentation allows parallel processing, reducing overall computation time while maintaining complete gesture recognition capability, as each segment can be tracked independently and simultaneously.
Solution Approach 2:
The system applies partial tracking strategies by focusing computational resources on critical body parts most relevant to the current gesture being performed. Instead of uniformly processing all body parts with equal detail, it adapts the tracking intensity based on contextual relevance, reducing processing time while maintaining sufficient accuracy for gesture recognition.
Data Source
AI summary
Technologies are described herein for dynamically remapping components of a virtual skeleton to enhance the control and appearance of an avatar. Embodiments disclosed herein may remap components of the virtual skeleton if a need for an enhancement, replacement or correction is identified. For example, one or more components of a virtual skeleton, e.g., a joint, hand, arm or leg, may be remapped with modeled components if the component is distorted, missing or incomplete. Components of a virtual skeleton may be remapped with modeled components if enhancements, augmentations, corrections or special effects are desired. A remapped skeleton defining one or more modeled components may be generated from model data. The remapped skeleton may be used to drive user-controlled animations having enhanced movement and appearance characteristics. In addition to driving user-controlled animations, the remapped skeleton may be used to drive robotic devices.