Machine Learning Image Illumination Rendering for Consistent Frame Rates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Ray tracing for image rendering is computationally expensive and varies unpredictably, making it difficult to maintain a consistent frame rate and quality in image rendering processes.
Innovation Solution
Employ a machine learning system, such as a neural network, to learn the relationship between pixel surface properties and rendered pixels, approximating ray-traced images with a consistent computational budget by training on high-quality ray-traced images to predict pixel values based on material properties and viewing angles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If ray tracing is used for image rendering, then image quality is improved, but computational cost increases and frame rate consistency deteriorates
Solution Approach 1:
The patent pre-calculates and stores illumination data for different lighting conditions and material properties before runtime. During actual rendering, the system queries this pre-computed data instead of performing full ray tracing, thereby maintaining high image quality while achieving consistent frame rates.
Solution Approach 2:
The patent creates simplified copies of complex illumination data in the form of lookup tables and pre-computed radiance values. These copies approximate the results of full ray tracing but can be retrieved and applied much faster, resolving the contradiction between quality and speed.
2Productivity
If average image quality is set low, then frame rate consistency is improved, but image quality deteriorates
Solution Approach 1:
The patent changes the parameters used for illumination calculation by switching between different levels of pre-computed data depending on the specific rendering needs. This allows the system to maintain high quality where needed while achieving consistent performance across varying scenes.
3Manufacturing precision
If average image quality is set close to maximum, then image quality is improved, but frame rate consistency deteriorates
Solution Approach 1:
The patent implements dynamic selection of rendering techniques based on scene characteristics. The system automatically chooses between pre-computed illumination data and more detailed calculations based on the specific lighting conditions, material properties, and viewport requirements, thereby maintaining maximum quality while ensuring consistent frame rates.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An image rendering method for a virtual scene comprises, for a plurality of IDs, generating a respective mask identifying elements of the scene that are associated with a respective ID; for the resulting plurality of masks, dividing a respective mask into a plurality of tiles; and discarding tiles that do not identify any image elements; for the resulting plurality of remaining tiles, selecting a respective trained machine learning model from among a plurality of machine learning models, the respective machine learning model having been trained to generate data contributing to a render of at least a part of an image, based upon elements of the scene associated with the same respective ID as the elements identified in the mask from which the respective tile was divided; and using the respective trained machine learning model to generate data contributing to a render of at least a part of the image based upon input data at least for the identified elements in the respective tile.