Machine Learning Image Rendering with Pre-computed Light Probes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Ray tracing is a computationally expensive process with unpredictable computational burdens, making it challenging to maintain a consistent frame rate for image rendering, as the variance in computational cost impacts both image quality and frame rate consistency.
Innovation Solution
A machine learning system, specifically a neural network, is trained to learn the relationship between pixel surface properties and rendered pixels, allowing for the approximation of ray-traced images within a consistent computational budget by predicting pixel values based on bidirectional scattering distribution functions (BSDFs) and bidirectional reflectance distribution functions (BRDFs), reducing the need for costly ray tracing calculations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If ray tracing is used to achieve high image quality, then image quality is improved, but computational cost increases and frame rate consistency deteriorates
Solution Approach 1:
The patent pre-calculates and stores lighting information (light probes) at multiple positions in the scene before rendering. This preliminary action allows the rendering system to access pre-computed lighting data during actual image generation, avoiding the need for expensive real-time ray tracing calculations while maintaining high image quality and consistent frame rates
Solution Approach 2:
The patent creates copies of lighting information by generating light probes at multiple discrete positions throughout the scene. These probe copies capture lighting characteristics from different locations, which can then be interpolated to provide accurate lighting data for any position without performing full ray tracing, thus reducing computational cost while preserving image quality
2Productivity
If average image quality is set low to maintain frame rate, then frame rate consistency is improved, but image quality deteriorates
Solution Approach 1:
The patent changes the rendering approach by switching from global illumination ray tracing to a hybrid method that combines direct lighting calculations with pre-computed indirect lighting from light probes. This parameter change in the rendering pipeline maintains high image quality while reducing computational cost and ensuring consistent frame rates
3Loss of time
If ray tracing computational burden is reduced, then processing time is improved, but image quality deteriorates
Solution Approach 1:
The patent segments the lighting calculation into two parts: direct lighting (calculated in real-time) and indirect lighting (pre-calculated and stored in light probes). This segmentation allows the system to reduce real-time processing time by using pre-computed indirect lighting while maintaining image quality through the combination of both lighting components
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An image rendering method comprises the steps of: selecting at least a first trained machine learning model from among a plurality of machine learning models, the machine learning model having been trained to generate data contributing to a render of at least a part of an image, wherein the at least first trained machine learning model has an architecture based learning capability that is responsive to at least a first aspect of a virtual environment for which it is trained to generate the data, and using the at least first trained machine learning model to generate data contributing to a render of at least a part of an image.