3D Model Geometry Refinement via Predicted Surface Normals
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in generating realistic 3D models of objects, particularly in aligning multiple view textures and ensuring visual consistency, while also improving depth perception and geometric precision.
Innovation Solution
A semi-automatic approach that combines automated algorithms with manual interventions to align multiple view textures on a 3D item, utilizing machine learning algorithms to enhance depth perception and refine the 3D model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated algorithms are used to generate 3D models from multiple view textures, then productivity is improved, but manufacturing precision deteriorates due to misalignment and visual inconsistencies
Solution Approach 1:
The system implements feedback by using predicted surface normals to generate depth maps that are then used to refine and correct the initial 3D model geometry. This closed-loop approach allows the automated system to self-correct alignment errors and improve geometric precision without manual intervention.
Solution Approach 2:
Surface normals act as an intermediary element that bridges the gap between 2D texture images and 3D geometry. By predicting normals from images and using them to construct depth maps, the system mediates the transformation process to achieve better geometric accuracy in the final 3D model.
2Manufacturing precision
If manual interventions are added to align textures and refine geometry, then manufacturing precision is improved, but productivity deteriorates due to increased time requirements
Solution Approach 1:
The system performs self-service by automatically predicting surface normals from images and using these predictions to generate corrective depth maps. This eliminates the need for manual geometry refinement while maintaining high precision, as the system serves its own alignment and refinement needs through automated ML-based normal prediction.
Solution Approach 2:
The patent replaces manual mechanical alignment operations with an automated machine learning system that predicts surface normals and generates depth maps. This substitution of mechanical/manual processes with intelligent algorithms maintains precision while dramatically improving productivity.
3Measurement precision
If depth maps are generated from predicted surface normals, then depth perception is improved, but device complexity increases due to additional processing steps
Solution Approach 1:
The surface normal prediction model serves multiple functions: it provides geometric information for depth map generation, ensures consistency across multiple views, and enables refinement of the 3D model geometry. This multi-functionality justifies the added processing complexity by delivering comprehensive depth perception improvement through a single integrated component.
Data Source
AI summary
Methods and systems are disclosed for generating 3D assets, such as for an extended reality (XR) experience. The system accesses a three-dimensional (3D) model of an object that has been generated based on at least one texture image of the object. The system processes the at least one texture image of the object by a machine learning model to generate a depth map and selects a portion of the 3D model. The system modifies the selected portion of the 3D model based on the depth map that has been generated by the machine learning model.