Neural Denoising Filter Preserving Jitter Vectors for Image Upsampling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image rendering technologies face challenges in achieving real-time rendering of high-quality images while managing computational complexity and reducing image artifacts such as pixel noise, resolution scaling artifacts, and limited ray tracing.
Innovation Solution
The proposed solution involves a neural denoising and upsampling process that preserves pixel sampling jitter offsets. This is achieved by using a denoising filter that retains jitter information, allowing for effective denoising without averaging out jitter offsets. The denoised image is then upsampled using the preserved jitter vectors, reducing artifacts like checkerboarding and aliasing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the number of triangles rendered per image is reduced to maintain reasonable frame rates, then productivity is improved, but manufacturing precision deteriorates due to resolution scaling artifacts
Solution Approach 1:
A denoising filter is introduced as an intermediary processing step between rendering and display. The filter receives low-resolution rendered images with noise and artifacts, processes them through convolution operations with learned kernels, and outputs enhanced images with reduced noise and improved quality, effectively mediating between the conflicting requirements of fast rendering and high quality output
Solution Approach 2:
The patent changes the parameter of image resolution from the original low rendering resolution to a higher effective resolution through denoising enhancement. By adjusting the kernel size and sigma parameters in the Gaussian blur operation, the system dynamically controls the balance between noise reduction and detail preservation, transforming the image quality parameter without requiring higher rendering resolution
2Productivity
If the number of light rays traced is reduced to maintain reasonable frame rates, then productivity is improved, but object-generated harmful factors increase due to pixel noise
Solution Approach 1:
The denoising filter serves as an intermediary that processes noisy rendered images by applying convolution operations with learned kernels. The filter separates noise from actual image content through statistical analysis of pixel neighborhoods, removing harmful noise while preserving legitimate image features, thus eliminating the harmful effect without requiring more ray traces
Solution Approach 2:
The patent converts the harmful effect of reduced sampling (which causes noise) into a benefit by using the noise characteristics themselves as input to the denoising filter. The filter learns to distinguish between noise patterns and actual image content, transforming the noisy low-sample images into clean high-quality images, thereby turning the limitation of few samples into an opportunity for learned enhancement
3Object-generated harmful factors
If traditional denoising filters are used to reduce pixel noise, then object-generated harmful factors are reduced, but manufacturing precision deteriorates due to loss of jitter information
Solution Approach 1:
The patent applies local quality by processing different regions of the image with adaptive kernel operations. The denoising filter analyzes local pixel neighborhoods and applies different filtering strengths based on local variance and edge detection, preserving jitter information in regions where it contains useful sampling variation while aggressively denoising uniform regions. This localized approach maintains manufacturing precision while reducing harmful noise
Solution Approach 2:
The patent introduces dynamics by making the denoising process adaptive rather than static. The kernel selection and filtering strength dynamically adjust based on local image characteristics, motion detection, and temporal consistency analysis. This dynamic adaptation allows the filter to preserve jitter information when beneficial while removing noise when harmful, resolving the contradiction between denoising and jitter preservation
Data Source
AI summary
One or more lighting components are projected onto pixel locations of a rendered image with sampling locations set off from pixel location centers according to associated jitter vectors. The sampled image is denoised in way that preserves the associated jitter vectors, and may be performed separately for different lighting components. The denoised image is processed using upsampling and/or temporal antialiasing, using the associated jitter vectors, to an image format having a spatial resolution at least as high as the denoised image.