3D Global Pose Estimation from Images with Neural Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer animation systems require high-cost hardware and extensive calculations to accurately project a human pose onto a world coordinate system, leading to inefficiencies and high processing times.
Innovation Solution
A method utilizing a computing device with a camera, neural networks, and a numerical optimization solver to generate a 3D global pose, involving detection, 2D heatmap generation, 3D pose estimation, and iterative fitting using Levenberg-Marquardt algorithm and Ceres solver to optimize the pose in a world coordinate system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional hardware equipment and calculation methods are used to obtain satisfactory 3D pose results, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent replaces conventional mechanical motion capture systems with a computational approach using neural networks and optimization algorithms. The system substitutes physical sensors and complex hardware with software-based processing of standard camera images, achieving comparable accuracy without specialized equipment
Solution Approach 2:
The patent creates a computational model (3D global pose) that copies and represents the real-world human pose from 2D images. By using neural networks to learn the mapping from 2D images to 3D poses, the system generates accurate 3D representations without requiring physical motion capture hardware
2Measurement precision
If conventional calculation methods are used to achieve accurate pose projection, then measurement precision is improved, but productivity decreases due to large amount of calculations
Solution Approach 1:
The patent pre-trains neural network models offline to learn the complex mapping from 2D images to 3D poses. This preliminary action transfers the computational burden from real-time processing to offline training, enabling fast inference during actual pose estimation without requiring extensive real-time calculations
Solution Approach 2:
The patent employs an iterative optimization process that dynamically adjusts pose parameters to minimize projection errors. The system starts with an initial pose estimate and refines it through multiple iterations, adapting the solution based on feedback from the optimization algorithm to achieve accurate results efficiently
3Measurement precision
If high-precision pose estimation is achieved through extensive calculations, then measurement precision is improved, but loss of time increases
Solution Approach 1:
The patent performs preliminary pose estimation using a neural network to generate an initial 3D pose estimate from the 2D image. This preliminary action provides a good starting point for subsequent optimization, reducing the number of iterations needed to achieve final accuracy and thereby reducing processing time
Solution Approach 2:
The patent implements a continuous iterative optimization process that progressively refines the pose estimate. Each iteration builds upon the previous result, continuously improving accuracy while converging to the final solution. This continuous refinement approach balances computational efficiency with measurement precision
Data Source
AI summary
A method for generating a three-dimensional (3D) global pose includes: receiving an image and performing a detection operation to detect a human body in the image; obtaining a two-dimensional (2D) heatmap that is related to a skeleton structure of the human body and that includes a plurality of human keypoints, and obtaining a plurality of 2D coordinate sets each indicating a position of a corresponding one of the human keypoints; performing a 3D human pose estimation operation on the plurality of 2D coordinate sets to obtain a 3D human pose that is related to the skeleton structure in a local coordinate system, and that includes a plurality of 3D keypoints corresponding to the plurality of human keypoints, respectively; and based on the 3D human pose, using a numerical optimization solver to generate a 3D global pose in a world coordinate system.