3D Human Pose Estimation via Orientation Keypoints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for estimating 3D human poses from 2D images lack depth information and skeletal rotation data, making it difficult to accurately identify and separate keypoint mappings for multi-person images without mixing body parts, and fail to provide full 3-axis rotations, limiting their utility in real-world applications.
Innovation Solution
A neural network detector is used to predict the 3D location of orientation keypoints, enabling the estimation of full 3D human poses in terms of position and 3-axis rotations by employing a convolutional neural network for detection and a regression-based neural network for adding depth, which reduces computation and memory complexity by using 1D heatmaps instead of volumetric heatmaps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If 2D images are used for pose estimation, then the system is simple and computationally efficient, but depth information and 3-axis rotation data are lost
Solution Approach 1:
The patent transitions from 2D pose estimation to 3D pose estimation by adding depth information and rotational components. The system estimates three-dimensional keypoint positions and full three-axis rotations (pitch, roll, yaw) for each joint, effectively moving the solution into a higher dimensionality space while maintaining computational feasibility through neural network-based approaches
2Measurement precision
If neural network detector is used to predict 3D location of orientation keypoints, then measurement precision of 3D pose is improved, but device complexity increases
Solution Approach 1:
The patent segments the pose estimation task into multiple components: detecting 2D keypoint positions, predicting depth information, and calculating rotational angles for each joint. This segmentation allows each component to be optimized independently and simplifies the overall neural network architecture while maintaining high measurement precision for the complete 3D pose
3Reliability
If complete 3D pose with full 3-axis rotations is estimated, then accuracy and utility for real-world applications is improved, but computation and memory complexity increases
Solution Approach 1:
The patent changes the parameter representation from traditional 2D coordinates to a comprehensive set of 3D parameters including x, y, z positions, depth, pitch, roll, and yaw angles for each joint. This parameter transformation enables complete 3D pose representation while the neural network optimizes the computational efficiency of processing these parameters
4Adaptability or versatility
If multi-person images are processed, then versatility and adaptability are improved, but difficulty of detecting and measuring keypoint mappings increases
Solution Approach 1:
The patent applies segmentation to separate and identify keypoints for different persons in multi-person images. The neural network detector independently estimates pose parameters for each detected person, effectively segmenting the complex task of multi-person pose estimation into individual person processing units, thereby reducing the difficulty of keypoint mapping separation
Data Source
AI summary
Embodiments of the present invention describe a system that receives an image depicting at least one subject, predicts at least one orientation keypoint associated with a section of the body part of the at least one subject via a neural network detector and determines a three-axis joint rotation associated with the section of the body part of the at least one subject based on at least one orientation keypoint associated with the body part of the at least one subject and at least one joint keypoint associated with the body part of the at least one subject. Orientation keypoints can improve the estimation of an associated joint keypoints, dense pose correspondence and landmark.