Mixed-Precision CNN Supersampling for Faster, Ghosting-Resistant Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Temporal Anti-aliasing (TAA) techniques suffer from ghosting artifacts due to frame-to-frame changes in visibility and shading, resulting in lower quality anti-aliased images when rendering at native resolution.
Innovation Solution
A mixed precision convolutional neural network is used for temporally amortized supersampling, allowing rendering at lower resolution to save time while maintaining high-quality images through a mixed precision convolutional neural network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If temporal anti-aliasing is used to generate anti-aliased images, then image quality is improved, but rendering time increases and ghosting artifacts appear
Solution Approach 1:
The patent segments the anti-aliasing process into two parts: temporal anti-aliasing for high-quality images and spatial anti-aliasing for real-time rendering. The system uses a neural network to separate the computationally intensive temporal processing from the real-time spatial processing, allowing each to be optimized independently.
Solution Approach 2:
The patent introduces a neural network as an intermediary between the temporal anti-aliasing stage and the final rendering output. This neural network learns to predict and correct ghosting artifacts and optimize the combination of temporal and spatial samples, acting as a mediator that resolves the contradiction between quality and speed.
2Manufacturing precision
If temporal anti-aliasing accumulates warped samples, then anti-aliasing quality is improved, but ghosting artifacts occur due to frame-to-frame changes
Solution Approach 1:
The patent implements feedback mechanisms where the neural network continuously refines its predictions based on the actual rendered images. The system uses feedback from the temporal anti-aliasing process to adjust and correct ghosting artifacts, improving the final image quality while reducing harmful artifacts.
Solution Approach 2:
The patent changes the precision parameters used in different stages of processing. The neural network operates with different precision levels for different parts of the computation, optimizing the balance between anti-aliasing quality and computational cost while reducing ghosting artifacts through parameter optimization.
3Manufacturing precision
If rendering is performed at native resolution, then image quality is maintained, but rendering time increases significantly
Solution Approach 1:
The patent applies partial anti-aliasing by using temporal anti-aliasing to generate supersampled images at lower resolution, then using spatial anti-aliasing with neural networks to enhance only the necessary regions. This partial action approach maintains image quality while significantly reducing the computational burden of rendering at full native resolution.
Solution Approach 2:
The patent creates a learned representation or copy of the temporal anti-aliasing process through the neural network, which can then generate high-quality images faster by predicting the results of full temporal processing without actually performing all the temporal samples, thus copying the effect with reduced computation.
Data Source
AI summary
One embodiment provides a graphics processor comprising a set of processing resources configured to perform a supersampling operation via a mixed precision convolutional neural network, the set of processing resources including circuitry configured to receive, at an input block of a neural network model, history data, velocity data, and current frame data, pre-process the history data, velocity data, and current frame data to generate pre-processed data, provide the pre-processed data to a feature extraction network of the neural network model, process the pre-processed data at the feature extraction network via one or more encoder stages and one or more decoder stages, and generate an output image via an output block of the neural network model via direct reconstruction or kernel prediction.