Parametric Deformable Mesh for Single Depth Camera 3D Avatar Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for constructing 3D mesh models of human subjects from depth sensor data require multiple sensors or video sequences and fail to accurately handle clothing variations and sensor noise, limiting their precision and applicability, especially in medical imaging scenarios.
Innovation Solution
A method and system for automatically generating a personalized 3D mesh from a single depth camera image using a trained parametric deformable model (PDM), which detects anatomical landmarks, initializes a template mesh, and optimizes pose and shape deformations to create a detailed mesh model that accounts for clothing and sensor noise, enabling precise body shape and pose estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple sensors or video sequences are used to construct 3D mesh models, then measurement completeness is improved, but device complexity and loss of time increase
Solution Approach 1:
The patent segments the complex task of 3D mesh construction into distinct functional components: a parametric deformable model handles body shape and pose variations, while a clothing deformation model separately addresses garment variations. This segmentation allows each component to be optimized independently and combined to achieve accurate mesh construction from a single sensor
Solution Approach 2:
The patent employs parameterized models that represent human body shapes, poses, and clothing variations through controlled parameters. By adjusting these parameters based on depth sensor data, the system can generate accurate 3D meshes without requiring multiple sensors or video sequences, thus resolving the contradiction between measurement precision and device complexity
2Adaptability or versatility
If SCAPE method is used for body modeling, then adaptability to pose variations is improved, but measurement precision deteriorates due to inability to handle clothing variations and sensor noise
Solution Approach 1:
The patent merges the SCAPE body model with a clothing deformation model to create a unified framework. The body model handles pose variations while the clothing model addresses garment variations and sensor noise, combining their capabilities to achieve both adaptability and measurement precision that neither model could achieve alone
3Measurement precision
If detailed mesh reconstruction from partial views is achieved, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent uses pre-trained parametric deformable models that have been learned from extensive training data. These pre-trained models encode knowledge about human body shapes, poses, and clothing variations, allowing the system to reconstruct detailed meshes from partial views without requiring complex real-time processing or additional sensors
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
A method and apparatus for generating a 3D personalized mesh of a person from a depth camera image for medical imaging scan planning is disclosed. A depth camera image of a subject is converted to a 3D point cloud. A plurality of anatomical landmarks are detected in the 3D point cloud. A 3D avatar mesh is initialized by aligning a template mesh to the 3D point cloud based on the detected anatomical landmarks. A personalized 3D avatar mesh of the subject is generated by optimizing the 3D avatar mesh using a trained parametric deformable model (PDM). The optimization is subject to constraints that take into account clothing worn by the subject and the presence of a table on which the subject in lying.