3D Human Body Model Reconstruction from Single Viewpoint Depth Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional 3D stereo capture systems for human body modeling are expensive and generate inaccurate 3D models due to occlusions and high computation costs, especially when using a single depth sensor, which leads to visual artifacts and erroneous shape representation.
Innovation Solution
A VR-based apparatus that uses a depth sensor to capture depth values from a single viewpoint, employs depth buffering to detect occluded portions, and constrains joint rotations to generate a realistic 3D human body model, accounting for self-occlusions and improving rendering speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a 3D stereo capture system with multiple cameras is used, then the 3D model can be generated from multiple viewpoints, but the system becomes expensive and complex
Solution Approach 1:
The patent segments the 3D modeling task into two phases: a coarse 3D model is generated from a single viewpoint using depth data, then refinement is applied using 2D image data and self-occlusion detection. This segmentation allows using simpler, cheaper equipment while achieving accurate 3D models through computational refinement rather than requiring complex multi-camera hardware.
Solution Approach 2:
The patent introduces an intermediary computational process that includes self-occlusion detection and refinement algorithms. These computational intermediaries process the coarse 3D model and 2D images to generate the final accurate model, compensating for the limitations of single-viewpoint capture hardware.
2Device complexity
If a single depth sensor is used, then the system cost is reduced, but the 3D model exhibits visual artifacts and has inaccurate shape due to occluded portions
Solution Approach 1:
The patent performs preliminary actions by first generating a coarse 3D model from single-viewpoint depth data, then systematically detecting self-occluded portions through rendering and depth buffer comparison. This preliminary coarse model serves as a foundation that is then refined by addressing occlusion issues, allowing low-cost hardware to produce accurate models through multi-step processing.
Solution Approach 2:
The patent implements feedback mechanisms where the rendered 3D model is used to detect self-occluded portions by comparing depth values against the depth buffer. This feedback loop identifies occlusion regions, which are then used to refine the 3D model by constraining joint rotations and filling in missing depth information, continuously improving model accuracy.
3Productivity
If self-occlusion is not considered during 3D model generation, then the processing is simplified, but the 3D model becomes erroneous and inaccurate
Solution Approach 1:
The patent applies local quality by detecting self-occluded portions through rendering and depth buffer comparison, then applying different processing strategies to different regions. Occluded regions are identified and processed separately through joint rotation constraints and depth value inference, while visible regions maintain their original processing. This localized approach maintains overall processing efficiency while improving accuracy where needed.
4Manufacturing precision
If depth values are available for all portions of the 3D model, then the model is complete, but computation cost becomes high
Solution Approach 1:
The patent applies partial action by only computing and refining depth values for portions of the 3D model where self-occlusion is detected. Instead of uniformly processing the entire model, the system identifies occluded regions and applies computational refinement only to those areas, reducing overall computation cost while maintaining completeness where needed.
Data Source
AI summary
A virtual Reality (VR)-based apparatus that includes a depth sensor that captures a plurality of depth values of a first human subject from a single viewpoint and a modeling circuitry that detects a set of visible vertices and a set of occluded vertices from a plurality of vertices of the first 3D human body model rendered on a display screen. The modeling circuitry determines a set of occluded joints and a set of visible joints from a plurality of joints of a skeleton of the first 3D human body model in the rendered state. The modeling circuitry updates a rotation angle and a rotation axis of the determined set of occluded joints to a defined default value in the skeleton and thereafter, re-renders the first 3D human body model as a reconstructed 3D human model of the first human subject on the display screen.