3D Human Model Reconstruction from 2D Images Using Perspective Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for reconstructing 3D human models from 2D images fail to accurately account for perspective and distortion effects, leading to inaccuracies in 3D reconstructions, especially in scenarios like 'selfie' photographs where perspective distortion is significant, affecting the size of body parts like the head and hand compared to the torso.
Innovation Solution
The system employs neural networks to determine the absolute depth of key points in the image, applying a position-dependent rotation to the 3D representation and using convolutional neural networks to estimate disparity and offset fields, allowing for accurate positioning of 3D models in a 3D scene, and incorporates a parametric model-based approach to handle joint positions and orientations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional 3D reconstruction methods are used, then the reconstruction process is simple, but the accuracy of 3D models is poor due to perspective and distortion effects
Solution Approach 1:
The patent introduces an intermediary transformation process that converts 2D image coordinates to 3D scene coordinates through a series of mathematical transformations including perspective projection and distortion correction. This intermediary step acts as a mediator between the 2D input and 3D output, resolving the contradiction by adding computational complexity to achieve higher measurement precision.
Solution Approach 2:
The patent changes multiple parameters including camera intrinsic parameters (focal length, principal point), extrinsic parameters (rotation, translation), and distortion parameters (radial and tangential coefficients) to accurately reconstruct 3D models. By systematically adjusting and optimizing these parameters, the system achieves high reconstruction accuracy despite the increased complexity.
2Manufacturing precision
If perspective distortion is not corrected, then the processing is faster, but the size of body parts in 3D models is inaccurate
Solution Approach 1:
The patent applies preliminary distortion correction and perspective normalization to the 2D input images before performing 3D reconstruction. By pre-processing the images to remove perspective distortion effects, the system ensures accurate body part sizing in the final 3D model without requiring complex real-time corrections during reconstruction, thus maintaining processing efficiency.
Solution Approach 2:
The patent replaces traditional mechanical measurement methods with computational image processing techniques. Instead of using physical measurement tools that would be slow, the system uses automated algorithms including convolutional neural networks and mathematical transformation models to rapidly correct perspective distortion and determine accurate body part dimensions.
3Measurement precision
If orthographic projection is used, then the computation is simpler, but the reconstruction is inaccurate for objects at different depths
Solution Approach 1:
The patent employs asymmetric perspective projection instead of symmetric orthographic projection to accurately represent objects at different depths. The perspective projection model inherently accounts for the asymmetry in how objects at varying distances from the camera appear in the 2D image, with closer objects appearing larger. This asymmetric approach significantly improves depth accuracy despite increasing computational complexity.
Solution Approach 2:
The patent transitions from 2D orthographic projection to 3D perspective projection by introducing depth as an explicit dimension in the transformation model. By incorporating the Z-depth coordinate and using perspective division (dividing by depth), the system accurately maps 3D points to 2D image coordinates and back, achieving high depth measurement precision through this dimensional transformation.
Data Source
AI summary
Systems and methods for reconstructing 3D models of human bodies from 2D images that counts for perspective and/or distortion effects are provided. The systems and methods include reconstructing a three-dimensional model of an object in a three-dimensional scene from a two-dimensional image comprising an image of the object. The systems and methods include determining an absolute depth of a key point of the object in the image; determining, using the absolute depth of the key point, a three-dimensional position of the key point in the three-dimensional scene; generating, using a neural network, a three-dimensional representation of the object, the three-dimensional representation comprising mesh nodes defined in a coordinate system relative to the key point; and positioning the three-dimensional representation of the object in the scene based on the position of the key point by applying a position dependent rotation to the three-dimensional object.