3D Pose Estimation via Synthetic Viewpoint Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional human pose estimation systems struggle with accurately recognizing human poses in images across different datasets and environments, due to their reliance on 2D keypoint representations that fail to account for occlusion and camera viewpoint variations.
Innovation Solution
The system employs a representation using opaque 3D limbs to preserve occlusion information, and uses synthetic images and viewpoint/pose encoding to train AI models, allowing them to predict poses independent of the camera's viewpoint and requiring no dataset-dependent normalization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If 2D keypoint representations are used for pose estimation, then the system is simple to implement, but it fails to account for occlusion and camera viewpoint variations, leading to poor cross-dataset performance
Solution Approach 1:
The patent transitions from 2D keypoint representations to 3D pose estimations with viewpoint encodings. By adding the third dimension (depth) and viewpoint information, the system can better represent occlusion and camera angle variations, significantly improving cross-dataset performance from 50 cm error to around 4 cm error while maintaining implementation feasibility through synthetic image generation.
2Measurement precision
If dataset-dependent normalization is applied, then same-dataset benchmark performance is optimized, but the system cannot generalize to different datasets and environments
Solution Approach 1:
The patent creates a universal pose estimation system that works across multiple datasets and environments by using synthetic images generated from 3D human models. The system learns viewpoint-invariant features and occlusion handling that generalize to any dataset without requiring dataset-specific normalization, making the system multi-functional across different evaluation benchmarks.
3Adaptability or versatility
If synthetic images with viewpoint encoding are used for training, then the system achieves viewpoint-independent pose prediction, but the training process becomes more complex
Solution Approach 1:
The patent uses synthetic images generated from 3D human models as training data. By creating virtual copies of human poses from various viewpoints and occlusion conditions, the system learns to generalize to real-world scenarios without requiring complex real-world data collection. This copying approach simplifies the training process while achieving viewpoint independence.
Data Source
AI summary
A device receives a real image (e.g., photograph or video frame) that includes a human. The device creates a synthetic image corresponding to the real image. The synthetic image includes a synthetic environment and a humanoid shape that correspond to the human. The device predicts, using a trained viewpoint neural network and based on the synthetic image, a predicted viewpoint heatmap. The device predicts, using a trained pose neural network and based on the synthetic image, a predicted pose heatmap. The device provides, as input to a random synthetic environment, the predicted viewpoint heatmap and the predicted pose heatmap and creates a reconstructed three-dimensional pose based on the predicted viewpoint heatmap, the predicted pose heatmap, and the random synthetic environment. The device classifies the reconstructed three-dimensional pose as a particular type of pose of the human in the real image.