AI-Enhanced Light Texture Data for Mobile Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Physically based rendering (PBR) techniques are computationally intensive, making it challenging to render high-quality, photo-realistic 3D scenes efficiently on computationally constrained devices like mobile devices, which often lack the processing power and storage capacity to handle complex light interactions and viewing perspectives in real-time.
Innovation Solution
A server-client system offloads computationally intensive PBR tasks to a powerful server, using AI models like convolutional autoencoders, vision transformers, or recurrent neural networks to enhance light texture data, reducing noise and improving details, and then sends the enhanced data to mobile devices for efficient rendering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If PBR techniques are used to generate photo-realistic 3D scenes, then rendering quality is improved, but computational complexity increases
Solution Approach 1:
The patent pre-computes view-independent light texture data and visibility information before rendering, storing them in data structures that can be quickly referenced during real-time rendering. This preliminary computation of lighting information for multiple directions and surface regions eliminates the need for complex real-time ray tracing on mobile devices, thereby reducing computational complexity while maintaining photo-realistic rendering quality.
2Manufacturing precision
If PBR techniques with path tracing are used, then rendering quality is improved, but processing time increases
Solution Approach 1:
The system performs path tracing and light transport simulation in advance to generate pre-computed light texture data structures containing visibility probabilities and light measurements for multiple directions. During actual rendering on mobile devices, the system only needs to sample from these pre-computed data structures, dramatically reducing processing time from minutes or hours to milliseconds while preserving the high-quality rendering effects of path tracing.
Solution Approach 2:
The patent segments the lighting computation into view-independent components (light texture data, visibility information) that can be pre-computed and stored, and view-dependent components that are computed in real-time based on camera position. This segmentation allows the computationally intensive path tracing to be performed only once for each scene configuration, rather than for every frame, thereby reducing overall processing time.
3Manufacturing precision
If high-quality light texture data is computed for all directions and surface regions, then rendering quality is improved, but memory usage increases
Solution Approach 1:
The patent computes and stores light texture data and visibility information selectively for different surface regions and direction ranges, rather than uniformly for all possible directions. The system identifies which surface regions are visible from the camera position and computes light measurements only for those regions and their relevant light directions, reducing memory usage while maintaining rendering quality for visible areas.
Data Source
AI summary
System, method and media for processing a first light data structure that specifies, for each of plurality of directions for each of a plurality surface regions corresponding to a scene, respective light measurements, including: applying a trained artificial intelligence (AI) model to the first light texture data structure to generate an enhanced light data structure that specifies, for each of the plurality of directions for each of the plurality surface regions corresponding to the scene, respective enhanced light measurements; and storing an enhanced scene model that includes the enhanced light data structure together with geometric data that maps the enhanced light measurements.