3D Pose Estimation via Epipolar Geometry Triangulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current human pose estimation systems face challenges in estimating three-dimensional poses from a single image without requiring 3D supervision or camera extrinsics, and they struggle with limited availability of in-the-wild 3D ground-truth data.
Innovation Solution
A system that uses a dual-branch neural network architecture, where one branch generates 2D poses and the other performs 3D reconstruction using epipolar geometry, allowing for self-supervision and eliminating the need for 3D ground-truth data, with features extracted using residual networks and deconvolution networks, and employing a soft argmax function for pose estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional 3D pose estimation methods are used, then 3D pose can be estimated with ground-truth supervision, but they require expensive 3D ground-truth data collection and camera extrinsics
Solution Approach 1:
The system uses self-supervised learning where the model learns to estimate 3D pose from 2D pose data without requiring external 3D ground-truth supervision. The network performs triangulation using epipolar geometry constraints derived from multi-view 2D poses, enabling it to self-supervise using only readily available 2D pose annotations from large datasets.
Solution Approach 2:
The system introduces an intermediary triangulation module that bridges 2D pose estimation and 3D pose reconstruction. This module uses epipolar geometry as a mathematical intermediary to convert 2D keypoint coordinates from multiple views into 3D joint positions, eliminating the need for direct 3D supervision while maintaining accuracy.
2Measurement precision
If 3D ground-truth data is collected in the wild, then accurate 3D pose estimation can be achieved, but data collection becomes costly and time-consuming
Solution Approach 1:
The system performs preliminary triangulation during the training phase using multi-view 2D poses to generate pseudo-3D pose targets. These pre-computed triangulated poses serve as training supervision signals, eliminating the need for time-consuming manual 3D annotation in the wild while providing accurate training targets for the pose estimation network.
3Measurement precision
If camera extrinsics are used for 3D reconstruction, then accurate triangulation can be performed, but the system becomes dependent on calibrated camera setups
Solution Approach 1:
The system changes the parameter representation by working directly with epipolar geometry constraints and fundamental matrices derived from multi-view geometry, rather than requiring explicit camera extrinsic parameters. This allows the triangulation to be performed using only 2D keypoint correspondences across views, making the system adaptable to uncalibrated camera setups while maintaining geometric accuracy.
Data Source
AI summary
A system for estimating a three dimensional pose of one or more persons in a scene is disclosed herein. The system includes one or more cameras and a data processor configured to execute computer executable instructions. The computer executable instructions include: (i) receiving one or more images of the scene from the one or more cameras; (ii) extracting features from the one or more images of the scene for providing inputs to a first branch pose estimation neural network and second branch pose estimation neural network; (iii) generating a first training signal from the second branch pose estimation neural network using a three dimensional reconstruction module for input into the first branch pose estimation neural network; (iv) generating one or more volumetric heatmaps; and (v) applying a maximization function to the one or more volumetric heatmaps to obtain a 3D pose of one or more persons in the scene.