Multi-View Human Motion Tracking Without Markers or Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing marker-based and marker-less motion capture systems for human 3D motion estimation are either complex and costly or inefficient in terms of real-time performance, or they fail to achieve real-time performance and require extensive training data, or they are too slow for real-time applications.
Innovation Solution
A human model represented by a set of spatial Gaussians, combined with a kinematic skeleton and statistical shape model, using Sums of 3D and 2D Gaussians for real-time pose estimation without relying on silhouettes or training data, and employing a novel model-to-image similarity function optimized with gradient ascent.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If marker-based optical capture systems are used, then accurate motion capture is achieved, but complex setup of cameras and markers is required
Solution Approach 1:
The invention extracts and removes the markers from the motion capture system, using only natural image features (edges, corners, blobs) detected by standard cameras. This eliminates the need for complex marker placement and camera calibration while maintaining accurate 3D motion estimation through feature tracking and pose detection algorithms
Solution Approach 2:
The invention creates a virtual 3D model copy of the human body with articulated joints and uses this digital twin to represent and track motion. This virtual model is fitted to detected image features and used to estimate 3D pose, replacing the need for physical markers while preserving measurement accuracy
2Ease of operation
If pose detection based on image features is used, then marker-less tracking is achieved, but real-time performance is not attained
Solution Approach 1:
The invention performs preliminary actions by pre-defining a parametric 3D human model with fixed anatomical structure and pre-computing projection matrices for multiple camera views. During tracking, only the pose parameters need to be optimized, significantly reducing computation time and enabling real-time performance while maintaining marker-less operation
Solution Approach 2:
The invention transforms the complex image processing problem into a parameter optimization problem by representing the human body with a small number of pose parameters (joint angles, positions). This parameterization allows efficient optimization using gradient-based methods, achieving real-time speeds while maintaining ease of marker-less tracking
3Measurement precision
If iterative morphological growing operations and Markov priors are used for pose estimation, then 2D pose detection is achieved, but real-time performance is not reached
Solution Approach 1:
The invention replaces the iterative morphological growing operations and Markov random field optimization with a direct parameter optimization approach using gradient descent. This substitution of the computational mechanism dramatically reduces processing time from seconds to milliseconds per frame while maintaining accurate 2D and 3D pose detection through the parametric model fitting
Data Source
Figure 1
Figure 2
Figure 3
AI summary
According to the invention, a computer-implemented method for tracking an object in a sequence of multi-view input video images comprises the steps: - acquiring a model of the object; - tracking the object in the multi-view input video image sequence, using the model.