Hybrid Body Reconstruction Model for Pose Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for reconstructing three-dimensional representations of bodies from two-dimensional images often result in loss of spatial detail and fail to accurately capture variant shapes and poses, as they rely on either overly restrained parametric models or unrealistic direct mesh models.
Innovation Solution
A hybrid approach using a machine-learned reconstruction model that predicts surface marker locations and generates both relaxed-constraint and parametric representations, leveraging the regularizing capabilities of kinematically constrained models and the predictive freedom of direct models, with a consistency loss to refine parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If parametric models with kinematic constraints are used for reconstruction, then reliability of body pose representation is improved, but device complexity and computational requirements increase
Solution Approach 1:
The reconstruction model is segmented into two distinct components: a relaxed-constraint representation component that captures variant shapes without kinematic constraints, and a parametric representation component that enforces kinematic constraints. This segmentation allows each component to specialize in different aspects of body representation, improving overall reliability while managing complexity through modular design.
Solution Approach 2:
The patent merges the relaxed-constraint representation and parametric representation into a unified hybrid model. The relaxed-constraint representation provides flexibility for variant shapes, while the parametric representation ensures kinematic consistency. By combining these two approaches, the system achieves both accuracy in representing diverse body poses and reliability in maintaining anatomical correctness.
2Adaptability or versatility
If direct mesh models are used for reconstruction, then adaptability to variant shapes is improved, but manufacturing precision and realism of body poses deteriorate
Solution Approach 1:
Different parts of the reconstruction model are assigned different qualities: the relaxed-constraint representation component handles local flexibility and adaptability to capture variant body shapes, while the parametric representation component ensures local precision and realism by enforcing kinematic constraints on body joints and segments.
Solution Approach 2:
The patent creates a composite reconstruction model that combines the properties of relaxed-constraint models (adaptability to variant shapes) and parametric models (precision in pose geometry). This composite approach allows the system to benefit from both the flexibility of direct mesh models and the anatomical correctness of kinematically constrained models.
3Ease of operation
If 2D images are used for reconstruction, then ease of data capture is improved, but loss of spatial information increases
Solution Approach 1:
The patent introduces an intermediate representation layer that bridges 2D image data and 3D body reconstruction. The relaxed-constraint representation serves as an intermediary that captures spatial relationships from 2D images without requiring complex 3D sensing, while the parametric representation refines this intermediate result to recover lost spatial information through kinematic constraints.
Data Source
AI summary
The present disclosure is generally directed to reconstructing representations of bodies from images. An example method of the present disclosure includes inputting, into a machine-learned reconstruction model, input data descriptive of an image depicting a body; predicting, using a machine-learned marker prediction component of the reconstruction model, a set of surface marker locations on the body; and outputting, using a machine-learned marker poser component of the reconstruction model, an output representation of the body that corresponds to the set of surface marker locations. In the example method, one or more parameters of the reconstruction model were learned at least in part based on a consistency loss corresponding to a distance between relaxed-constraint representations generated from a prior set of surface marker locations predicted according to the one or more parameters and parametric representations generated from the prior set using kinematic constraints associated with the body.