Body Tracking Using Probabilistic Fitting and Super Points
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing body tracking systems using depth sensors and cameras face challenges in accurately determining the shape and position of moving bodies, particularly in handling shape changes and self-intersections, with limitations in precision and computational efficiency.
Innovation Solution
A system that employs a probabilistic fitting algorithm using a depth sensor and a body model with constrained constraints such as self-intersection, angle, and pose prior constraints, combined with a Gaussian mixture model, to fit data points from the depth sensor and camera, with additional weighting for 'super points' obtained from markers or body joints, enabling accurate three-dimensional tracking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If explicit point correspondences methods (mesh fitting, random walk classifier, global minima search) are used to determine shape and position, then measurement precision can be improved, but device complexity and computational requirements increase significantly
Solution Approach 1:
The patent transforms the complex 6D pose estimation problem into a simplified 3D position estimation problem by using depth sensor data to directly determine distance from the sensor. This parameter reduction changes the problem from estimating position (x,y,z) plus orientation (roll, pitch, yaw) to only estimating position, thereby reducing computational complexity while maintaining measurement precision for the critical position parameter
Solution Approach 2:
The patent extracts and separates the position determination function from the orientation determination function. By using depth sensor data to independently determine position, the system removes the computational burden of simultaneous pose estimation, allowing orientation to be derived separately if needed, thus reducing overall device complexity
2Productivity
If deformation models with Gaussian Mixture models are used for real-time pose and shape estimation, then productivity and real-time performance are improved, but measurement precision may be compromised due to model constraints
Solution Approach 1:
The patent segments the body into multiple rigid segments (limbs, torso, head) connected by joints, where each segment is modeled as a rigid body. This segmentation allows the system to process each segment independently using simple rigid body transformations rather than complex global deformation models, improving real-time processing speed while maintaining accuracy through the hierarchical structure
Solution Approach 2:
The patent implements a dynamic hierarchical model where proximal segments (closer to body center) have fixed transformations and distal segments (farther from center) have movable joints with degrees of freedom. This dynamic structure allows real-time adaptation to body movements while maintaining computational efficiency through the hierarchical constraint propagation
3Reliability
If multiple constraints (self-intersection, angle, pose prior) are applied in the fitting algorithm, then reliability of tracking is improved, but device complexity increases
Solution Approach 1:
The patent applies pose prior constraints that encode anatomically plausible pose ranges before processing begins. By pre-defining valid joint angle ranges and anatomical constraints based on human body mechanics, the system ensures tracking reliability under occlusion and shape change without requiring complex real-time constraint solving, as the search space is already limited to physically valid configurations
4Measurement precision
If super points with additional weighting are used in the fitting algorithm, then measurement precision is improved, but loss of time for processing increases
Solution Approach 1:
The patent applies different weights to different data points based on their reliability and importance. Super points (landmark points, joint points, points with high confidence) receive higher weights in the fitting algorithm, while ordinary points receive lower weights. This local quality differentiation improves measurement precision for critical features without requiring uniform high-precision processing of all points, thereby reducing overall processing time
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
Systems, methods and apparatuses for tracking at least a portion of a body by fitting data points received from a depth sensor and/or other sensors and/or "markers" as described herein to a body model. For example, in some embodiments, certain of such data points are identified as "super points," and apportioned greater weight as compared to other points. Such super points can be obtained from objects attached to the body, including, but not limited to, active markers that provide a detectable signal, or a passive object, including, without limitation, headgear or a mask (for example for VR (virtual reality)), or a smart watch. Such super points may also be obtained from specific data points that are matched to the model, such as data points that are matched to vertices that correspond to joints in the model.