Reverse Rasterization Mesh Reconstruction for Lightweight 3D Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for simplifying complex 3D models for devices with modest processing power result in significant information loss, leading to low-fidelity representations.
Innovation Solution
The reverse rasterization process reconstructs accurate and lightweight 3D models by determining the best visibility from multiple virtual camera viewpoints and generating meshes from these renders, allowing for real-time display on devices with limited computational resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If photogrammetry techniques are used to simplify complex 3D models, then the model complexity is reduced for devices with modest processing power, but significant information loss occurs resulting in low-fidelity, low resolution, and blurry reconstructed models
Solution Approach 1:
The patent inverts the traditional rasterization process by performing reverse rasterization: instead of converting 3D mesh data to 2D pixel data for display, it converts 2D pixel data from multiple rendered views back into 3D mesh data. This inversion allows the system to generate high-fidelity 3D models by synthesizing information from multiple angle renders, thereby reducing information loss while maintaining model accuracy for streaming to devices with modest processing power
2Productivity
If high-fidelity 3D models with hundreds of millions of triangles are streamed to devices without comparable processing power, then real-time preview and interaction can be provided, but the models must be simplified which results in loss of detail and accuracy
Solution Approach 1:
The patent segments the 3D model representation into multiple 2D rendered views from different camera angles. Each render captures geometric and texture information from a specific viewpoint. By segmenting the model this way, the system can stream compressed 2D render data instead of full 3D mesh data, enabling real-time transmission to devices with modest processing power while preserving the ability to reconstruct high-fidelity models on demand
Solution Approach 2:
The patent introduces an intermediary reverse rasterization process that acts as a mediator between the high-fidelity source model and the target display device. This intermediary converts 2D pixel data from multiple renders into reconstructed 3D mesh data, allowing the system to maintain high model accuracy while adapting to the computational constraints of devices with modest processing power
Data Source
AI summary
Reverse rasterization may be used as a technique to reconstruct accurate lightweight 3D models from complex and/or high-fidelity 3D graphical models or scenes. The process of reverse rasterization may begin with two or more renders created from virtual camera viewpoints distributed around the 3D model or scene that is to be reconstructed. Using the various virtual camera viewpoints, a lightweight version of the complex 3D model/scene may be reconstructed, e.g., by determining which virtual camera viewpoint has the best visibility for each point on the surface of the 3D model/scene. Once the reversion rasterization process determines the virtual camera viewpoint with the best visibility to use for a given point on the reconstructed model, that point may be kept, while other pixels in the vicinity of that point may be filtered and/or deleted. Contiguous pixels are then converted to vertices and joined by triangles to form a reconstructed 3D mesh.