Backward Projection Temporal Filtering for Ray-Traced Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional approaches to computing temporal gradients for real-time rendering, such as forward projection, fail to effectively handle reflections and refractions, leading to artifacts like ghosting and noise, especially when dealing with rasterized geometry-buffers and systems like Primary Surface Replacement (PSR).
Innovation Solution
The use of backward projection to locate matching surfaces for gradient estimation, followed by surface parameter carry-forward and patching, which allows for accurate rendering of reflections and refractions by correlating pixel data between frames and updating surface parameters in the G-buffer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If forward projection is used to compute temporal gradients, then the method is simple and works with rasterized geometry-buffers, but it fails to handle reflections and refractions accurately, leading to artifacts like ghosting and noise
Solution Approach 1:
The patent inverts the conventional forward projection approach by implementing backward projection. Instead of projecting surfaces from the previous frame forward to the current frame, the system traces rays backward from the current frame to locate matching surfaces in the previous frame. This inversion enables accurate handling of reflections and refractions by properly tracking the complete light path through multiple bounces, thereby eliminating ghosting artifacts while maintaining computational feasibility.
2Productivity
If forward projection is used for temporal gradients, then computation is straightforward, but it produces false positive gradients when camera or secondary surfaces move, invalidating accumulated history and increasing noise
Solution Approach 1:
The patent implements a feedback mechanism through backward projection that continuously validates surface matching across frames. By tracing rays backward and verifying surface correspondence, the system detects when camera or secondary surface movement invalidates the accumulated history. This feedback loop allows the renderer to selectively discard only the affected temporal history portions rather than invalidating entire frames, maintaining computation efficiency while preserving gradient accuracy.
Solution Approach 2:
The system dynamically adapts the temporal gradient computation based on detected motion and surface changes. When backward projection identifies that camera or secondary surfaces have moved, the system adjusts which historical data is valid and modifies the gradient computation accordingly. This dynamic approach allows the renderer to maintain high computation speed by avoiding unnecessary recalculations while ensuring measurement precision when changes are detected.
3Productivity
If lower sample counts are used for stochastic sampling, then real-time performance is achieved, but noise increases requiring sophisticated reconstruction filters
Solution Approach 1:
The patent applies preliminary action by computing and accumulating temporal gradients across multiple frames before final image reconstruction. The backward projection method pre-establishes accurate surface correspondence and temporal coherence, creating a foundation of reliable gradient data that can be reused across frames. This preliminary computation of temporal relationships allows the use of lower sample counts per frame while maintaining image quality, as the temporal reconstruction filters can leverage the pre-computed accurate gradients to reduce noise effectively.
Data Source
AI summary
Robust temporal gradients, representing differences in shading results, can be computed between current and previous frames in a temporal denoiser for ray-traced renderers. Backward projection can be used to locate matching surfaces, with the relevant parameters of those surfaces being carried forward and used for patching. Backward projection can be performed for each stratum in a current frame, a stratum representing a set of adjacent pixels. A pixel from each stratum is selected that has a matching surface in the previous frame, using motion vectors generated during the rendering process. A comparison of the depth of the normals, or the visibility buffer data, can be used to determine whether a given surface is the same in the current frame and the previous frame, and if so then parameters of the surface from the previous frame G-buffer is used to patch the G-buffer for the current frame.