3D Mesh Generation from Single Color and Depth Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods fail to effectively generate realistic three-dimensional (3D) images when the viewpoint of an object changes, as they lack the ability to seamlessly integrate depth and color information from different viewpoints, leading to occlusion areas and inconsistent representations.
Innovation Solution
An image processing apparatus that generates a 3D mesh by combining 2D vertexes from a color image and 3D vertexes from a depth image, using feature extractors to identify gradients and curvature, and interpolates depth values to create a unified 3D model, which is then used to inpaint occlusion areas and render images from various viewpoints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single color image and a single depth image are used for object modeling, then the process is simple and efficient, but the ability to represent the object accurately across different viewpoints is insufficient
Solution Approach 1:
The patent merges 2D vertexes from color images with 3D vertexes from depth images to create a unified 3D mesh. The color feature extractor and depth feature extractor process their respective images independently, then the 3D mesh generator integrates them by matching vertexes and generating corresponding 3D coordinates, achieving both efficiency and accuracy.
Solution Approach 2:
The patent transitions from 2D images to a 3D mesh representation by adding depth dimension information. The 2D vertexes are transformed into 3D vertexes through depth value assignment and interpolation, enabling the model to represent objects in three-dimensional space while maintaining the simplicity of processing single images.
2Manufacturing precision
If depth values are interpolated for all 2D vertexes, then the 3D mesh is complete and consistent, but the processing time and computational complexity increase
Solution Approach 1:
The patent applies partial action by interpolating depth values only for 2D vertexes that do not have corresponding depth information, rather than processing all vertexes uniformly. The 3D mesh generator identifies which vertexes need depth values and applies interpolation selectively, reducing computational time while maintaining mesh completeness.
Data Source
AI summary
Disclosed is a mesh generating apparatus, method and computer-readable medium, and an image processing apparatus, method and computer-readable medium. A single 3D mesh and a single 3D object model may be generated using a single color image and a single depth image. A plurality of viewpoint images seen from a plurality of viewpoints may be generated from a single 3D object model.