Single Camera 3D Pose Tracking via Foot-Floor Contact Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for three-dimensional human pose tracking from a single camera face challenges due to depth ambiguities, occlusions, and significant variations in appearances and scenes, leading to inaccurate and unstable tracking results.
Innovation Solution
A real-time multi-person absolute 3D pose tracking system using a single RGB camera, which involves camera calibration, 2D pose detection, root-relative 3D pose regression, foot-floor contact detection, and absolute 3D pose optimization, ensuring accurate and robust tracking by minimizing global position tracking error and mean per joint position error.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single camera is used for 3D pose tracking, then device complexity is reduced, but measurement precision deteriorates due to depth ambiguities
Solution Approach 1:
The patent transforms 2D image coordinates into 3D pose information by detecting foot-floor contact points and using geometric relationships. The system recovers absolute 3D positions from 2D images by establishing constraints based on ground contact, effectively adding depth information through dimensional transformation rather than using multiple cameras.
Solution Approach 2:
The patent introduces foot-floor contact detection as an intermediary mechanism to resolve depth ambiguities. By detecting which feet are in contact with the ground and using this as a reference plane, the system can establish geometric constraints that enable accurate 3D pose recovery from single-camera 2D images.
2Ease of operation
If traditional 2D pose detection is used, then ease of operation is maintained, but reliability deteriorates due to occlusions and appearance variations
Solution Approach 1:
The patent employs iterative optimization that uses feedback from multiple sources including 2D pose detections, foot-floor contact constraints, and geometric relationships. The system continuously refines 3D pose estimates by comparing projected 3D positions with actual 2D detections and adjusting based on the differences, thereby improving reliability despite occlusions and appearance variations.
Solution Approach 2:
The patent transforms the pose representation from simple 2D coordinates to comprehensive 3D parameters including absolute positions, orientations, and foot-floor contact states. By changing the parameter space to include these additional constraints and relationships, the system achieves more reliable tracking that is robust to occlusions and appearance changes.
3Measurement precision
If absolute 3D pose optimization is implemented, then measurement precision is improved, but device complexity increases due to additional processing steps
Solution Approach 1:
The patent divides the complex 3D pose optimization into distinct modular steps: 2D pose detection, foot-floor contact detection, root-relative 3D pose regression, and absolute 3D pose optimization. Each module handles a specific aspect of the problem, making the overall complex system manageable and implementable through sequential processing stages.
Data Source
AI summary
This disclosure describes systems, methods, and devices related to real-time multi-person three-dimensional pose tracking using a single camera. A method may include receiving, by a device, two-dimensional image data from a camera, the two-dimensional image data representing a first person and a second person; generating, based on the two-dimensional image data, two-dimensional positions of body parts represented by the first person; generating, using a deep neural network, based on the two-dimensional positions, a three-dimensional pose regression of the body parts represented by the first person; identifying, based on the two-dimensional positions and the three-dimensional pose regression, contact between a ground plane and a foot of the first person; generating an absolute three-dimensional position of the contact between the ground plane and the foot of the first person; generating, based on the absolute three-dimensional position, a three-dimensional pose of the body parts represented by the first person.