Monocular 3D Human Reconstruction Using Depth-Based Implicit Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D object reconstruction methods based on 2D depth map prediction struggle to accurately represent object shapes at multiple viewpoints and fail to naturally reconstruct invisible areas, relying on low-quality 3D data or template models that lack detail and realism.
Innovation Solution
A deep learning-based method that predicts orthographic normals, colors, and depth maps from monocular images, using a signed distance function to reconstruct 3D human objects by encoding pixel-aligned and ray-invariant features, enabling natural reconstruction of visible and invisible areas.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If 3D reconstruction is performed based on 2D depth map prediction, then reconstruction speed is improved, but reconstruction accuracy and completeness of invisible areas deteriorate
Solution Approach 1:
The patent transitions from 2D depth map prediction to 3D occupancy volume prediction by introducing a third dimension. The neural network directly predicts occupancy values for 3D space voxels, enabling the system to represent and reconstruct invisible areas and multiple viewpoints simultaneously, thereby improving both accuracy and completeness while maintaining reasonable speed through efficient 3D volume processing
Solution Approach 2:
The patent introduces occupancy volume as an intermediary representation between the input 2D image and the final 3D output. This occupancy volume serves as a dense 3D spatial map that captures both visible and invisible regions, acting as a bridge that enables accurate reconstruction of complete object shapes including areas not directly visible in the input image
2Productivity
If template models such as SMPL are used for shape learning, then reconstruction speed is improved, but naturalness and detail accuracy deteriorate
Solution Approach 1:
Instead of relying on pre-defined template models like SMPL, the patent learns to directly copy and reconstruct the actual object geometry from 2D input images by predicting 3D occupancy volumes. This data-driven approach captures real object variations and details without being constrained by template assumptions, producing more natural and accurate reconstructions
Solution Approach 2:
The patent changes the fundamental parameter representation from template-based joint angles and shape coefficients to direct 3D occupancy grid predictions. This parameter transformation enables the system to represent arbitrary object shapes and fine details without relying on pre-defined anatomical templates, significantly improving reconstruction naturalness
Data Source
AI summary
There are provided an apparatus and a method for reconstructing a 3D human object based on a monocular image through depth image-based implicit function learning. A 3D human object reconstruction method according to an embodiment includes: predicting a double-sided orthographic depth map from a front perspective color image of a human object; predicting a signed distance (SD) regarding points on a 3D space from the predicted double-sided orthographic depth map; and reconstructing a 3D human object by using the predicted SD. Accordingly, a human object and details can be naturally reconstructed with respect to not only an area visible through a front perspective color image of the human object but also an invisible area.