3D Shape Model Refinement via Optical Flow and Back-Projection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional 3D modeling techniques struggle to accurately refine the shape of 3D models, particularly for objects like human faces, resulting in inaccurate representation of intricate details and features.
Innovation Solution
An electronic apparatus and method that utilize an optical flow-based approach to refine 3D shape models by generating a back-projected image, computing an optical flow map between the back-projected image and a 2D color image, and determining 3D correspondence points to optimize vertex positions, thereby improving the accuracy of shape features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional 3D modeling techniques are used, then the basic shape can be represented, but the accuracy of intricate details and features is insufficient
Solution Approach 1:
The patent introduces an intermediary 2D image plane between the 3D shape model and the target object. By projecting the 3D model to this intermediate 2D plane and comparing it with actual 2D images, the system achieves accurate shape refinement without directly manipulating complex 3D data, thus resolving the contradiction between precision and complexity.
Solution Approach 2:
The patent transforms the shape refinement problem from a complex 3D domain to a simpler 2D domain. By working with 2D projections and images instead of directly manipulating 3D mesh data, the system achieves higher accuracy in representing intricate details while reducing the computational complexity of the modeling process.
2Shape
If the 3D model uses polygonal surfaces to define shape and geometry, then the basic structure is established, but the visual quality and realism are insufficient
Solution Approach 1:
The patent implements a feedback mechanism where the generated 2D image from the 3D model is compared with actual 2D images of the target object. The differences are used to adjust and refine the 3D model's shape and texture, creating a feedback loop that continuously improves visual quality while maintaining ease of generation through automated iterative optimization.
Solution Approach 2:
The patent creates a 2D copy or projection of the 3D model that can be directly compared with actual 2D images. This 2D representation serves as an intermediate that captures visual qualities and can be easily manipulated to match target images, simplifying the overall model generation process while improving realism.
3Measurement precision
If the 3D model is rendered from a 3D mesh, then the geometry is defined, but the accuracy in matching actual object shape is insufficient
Solution Approach 1:
The patent introduces a 2D image plane as an intermediary between the 3D mesh and the target object. By projecting the 3D mesh to this intermediate plane and comparing the projection with actual 2D images, the system achieves precise shape measurement and matching without directly solving the complex 3D shape alignment problem.
Solution Approach 2:
The patent simplifies the shape matching problem by transforming it from a complex 3D domain to a 2D domain. Working with 2D projections and images allows for more accurate measurement and comparison of shape features while reducing the computational complexity inherent in direct 3D shape matching.
Data Source
AI summary
An electronic apparatus for shape refinement of a three-dimensional (3D) shape model is provided. The electronic apparatus generates a back-projected image for an object portion based on an initial 3D shape model of the object portion and a texture map of the object portion. The electronic apparatus computes an optical flow map between the back-projected image and a two-dimensional (2D) color image of the object portion. The electronic apparatus determines a plurality of 3D correspondence points for a corresponding plurality of vertices of the initial 3D shape model, based on the optical flow map and a depth image of the object portion. The electronic apparatus estimates a final 3D shape model that corresponds to a shape-refined 3D model of the object portion based on the initial 3D shape model and the plurality of 3D correspondence points for the corresponding plurality of vertices of the initial 3D shape model.