Mobile 3D Model Texture Resolution via 2D Image Projection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D modeling technologies face challenges in accurately capturing and rendering complex objects and environments using mobile devices, as they struggle with imperfect reconstructions, especially for thin, reflective, transparent, or highly geometric objects, and often lack the texture resolution of raw 2D imagery due to hardware limitations.
Innovation Solution
The system generates and renders 3D models from mobile device-captured 2D and 3D image data, allowing users to navigate and view 3D scenes with high fidelity by aligning 3D meshes with original 2D images, projecting 2D data onto 3D models, and providing novel perspectives, even from positions not captured during the scan, using techniques like occlusion mapping and similarity ranking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If existing 3D modeling processes are used, then 3D reconstruction can be generated from 2D image data, but the reconstruction quality deteriorates for thin, reflective, transparent, or highly complex geometric objects
Solution Approach 1:
The system segments the 3D reconstruction task into two independent parts: a 3D mesh structure component and a 2D texture component. The 3D mesh is generated from depth data while the 2D texture is generated from corresponding 2D images, allowing each component to be optimized independently for different object types without compromising the other.
Solution Approach 2:
The system merges the 3D mesh and 2D texture components into a unified 3D model representation where the mesh provides geometric structure and the texture provides surface appearance. This combination allows the system to handle complex objects by integrating structural accuracy with textural fidelity that neither component could achieve alone.
2Manufacturing precision
If 3D models are generated from mobile device data, then texture resolution is improved, but the ability to navigate and view from novel perspectives deteriorates
Solution Approach 1:
The system separates the texture information (stored at high resolution in 2D images) from the geometric information (stored as 3D mesh). This segmentation allows the texture to be applied to multiple virtual camera positions without reprocessing, enabling novel perspective views while maintaining the original high texture resolution from the mobile device captures.
Solution Approach 2:
The system transforms the 2D texture images into a 3D context by mapping them onto the 3D mesh surface. This dimensional transformation allows the 2D texture data to be viewed from multiple angles and perspectives in 3D space without losing resolution, effectively adding viewing flexibility while preserving texture quality.
3Manufacturing precision
If sophisticated cameras with rotating tripods are used, then structured panoramic imagery can be captured, but device complexity and ease of operation deteriorate
Solution Approach 1:
The system creates a virtual copy of the physical capture process by using software algorithms to reconstruct 3D scenes from simple mobile device images. Instead of requiring complex physical camera systems, the invention uses computational methods to generate the structured panoramic imagery and 3D models from standard mobile device captures.
Solution Approach 2:
The system replaces the mechanical rotating tripod camera system with a software-based 3D reconstruction algorithm. The physical mechanical complexity of rotating cameras and tripods is substituted by computational geometry and image processing algorithms that automatically generate 3D models from simple mobile device images.
4Ease of operation
If mobile devices are used for capture, then ease of operation is improved, but measurement precision and 3D reconstruction accuracy deteriorate
Solution Approach 1:
The system enables mobile devices to perform 3D reconstruction tasks independently using their own integrated sensors and processing capabilities. The mobile device self-services by capturing both 2D images and depth information, then automatically generating 3D models without requiring external sophisticated equipment, thereby maintaining ease of operation while achieving acceptable measurement precision.
Data Source
AI summary
This application generally relates to systems and methods for generating and rendering visualizations of an object or environment using 2D and 3D image data of the object or the environment captured by a mobile device. In one embodiment, a method includes providing, by the system, a representation of a 3D model of an environment from a first perspective of the virtual camera relative to the 3D model, receiving, by the system, input requesting movement of the virtual camera relative to the 3D model, and selecting, by the system, a first 2D image from a plurality of two dimensional images associated with different capture positions and orientations relative to the 3D model based on association of a capture position and orientation of the first 2D image with a second perspective of the virtual camera relative to the 3D model determined based on the movement.