3D Human Reconstruction Using Absolute Depth and Scene Rotation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for reconstructing 3D human models from 2D images fail to accurately account for perspective and distortion effects, leading to inaccuracies in downstream augmented reality applications where precise positioning is required.
Innovation Solution
Systems and methods that utilize neural networks to determine absolute depth and position key points, apply position-dependent rotations, and incorporate convolutional neural networks to estimate 3D offsets and joint positions, correcting for distortion and perspective in 3D reconstructions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional 3D reconstruction methods are used from 2D images, then the reconstruction process is simple, but the accuracy of 3D models is poor due to failure to account for perspective and distortion effects
Solution Approach 1:
The patent introduces an intermediary calibration process that captures images of a calibration object with known 3D geometry to establish transformation matrices between 2D image coordinates and 3D world coordinates. This intermediary step mediates between the simple 2D-to-3D mapping and the complex perspective distortion correction, enabling accurate reconstruction without requiring direct complex geometric calculations.
Solution Approach 2:
The patent transforms the reconstruction problem by changing parameters from direct 2D-to-3D coordinate mapping to using transformation matrices derived from calibration data. The system changes the parameter representation from raw pixel coordinates to calibrated 3D coordinates that account for camera perspective, focal length, and distortion characteristics.
2Manufacturing precision
If perspective and distortion correction is applied, then the accuracy for augmented reality applications is improved, but the computational complexity increases
Solution Approach 1:
The patent performs perspective and distortion correction in advance during a calibration phase, where transformation matrices are pre-computed from calibration object images. This preliminary action prepares the correction parameters before actual 3D reconstruction, so that during runtime, the system only needs to apply pre-computed transformations rather than performing complex real-time calculations.
Solution Approach 2:
The patent creates a copied and simplified representation of the complex perspective transformation through pre-computed transformation matrices. Instead of performing complex geometric transformations during reconstruction, the system uses these copied transformation parameters to efficiently map 2D image points to corrected 3D coordinates.
3Measurement precision
If calibration with known 3D geometry is used, then the transformation between 2D and 3D coordinates is accurate, but the setup process becomes more complex
Solution Approach 1:
The patent uses a simple calibration object with known geometry (such as a planar pattern or object with predetermined 3D coordinates) that can be easily manufactured and discarded. This calibration object serves as a temporary tool during the setup phase to establish transformation parameters, eliminating the need for complex permanent calibration infrastructure.
Solution Approach 2:
The patent changes the calibration approach from requiring complex multi-point 3D measurements to using a simple object with pre-known geometry parameters. By changing from measuring unknown 3D coordinates to using objects with manufacturer-provided coordinate data, the system simplifies the calibration setup while maintaining transformation accuracy.
Data Source
AI summary
Systems and methods for reconstructing 3D models of human bodies from 2D images that counts for perspective and/or distortion effects are provided. The systems and methods include reconstructing a three-dimensional model of an object in a three-dimensional scene from a two-dimensional image comprising an image of the object. The systems and methods include determining an absolute depth of a key point of the object in the image; determining, using the absolute depth of the key point, a three-dimensional position of the key point in the three-dimensional scene; generating, using a neural network, a three-dimensional representation of the object, the three-dimensional representation comprising mesh nodes defined in a coordinate system relative to the key point; and positioning the three-dimensional representation of the object in the scene based on the position of the key point by applying a position dependent rotation to the three-dimensional object.