GPU Lens Blur Rendering via Depth Map Weighted Summation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing digital image editing systems face challenges in applying realistic and aesthetically pleasing lens blur effects due to issues like objects in focus occluding nearby objects, inefficient rendering speeds, and unnatural edges, particularly when dealing with limited data and depth discontinuities.
Innovation Solution
The system applies lens blur effects by dividing images into pixel blocks and processing them in parallel using GPUs, where each pixel's output value is a weighted sum of source pixels based on depth distance and relative depth differences, eliminating background leakage and artifacts through accurate depth mapping and iris threshold calculations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If lens blur effects are applied using existing digital image editing systems, then blur effects can be added to images, but the rendering speed is slow and the quality is reduced due to objects in focus occluding nearby objects, unnatural edges, and dull bokeh highlights
Solution Approach 1:
The patent segments the blur computation by partitioning the aperture into multiple portions and processing different pixel blocks in parallel using GPU threadgroups. Each threadgroup handles a specific pixel block and processes only the relevant aperture portions, enabling parallel computation while maintaining quality. This segmentation resolves the contradiction by distributing the computational load across multiple processing units, improving rendering speed without sacrificing blur effect quality.
Solution Approach 2:
The patent applies local quality by computing blur effects with high precision only where needed (in the accumulation region around each output pixel) while using depth maps to selectively apply different processing levels. The system calculates weighted sums of source pixels based on depth distances, applying more computational effort to regions requiring accurate occlusion handling while maintaining overall efficiency. This localized approach improves blur quality without uniformly increasing rendering time across the entire image.
2Measurement precision
If lens blur effects are applied using simplified ray casting with random aperture sampling, then rendering can be performed, but texture memory access patterns are inefficient and oversampling is required to obtain sharp bokeh, leading to slow rendering speeds
Solution Approach 1:
The patent applies preliminary action by pre-sorting source pixels in the accumulation region based on their depth distances before performing the weighted sum calculation. This pre-sorting enables the system to efficiently identify which source pixels should contribute to each output pixel without requiring extensive oversampling. By preparing the pixel data in advance and organizing it by depth, the system achieves sharp bokeh with fewer samples, resolving the contradiction between precision and speed.
Solution Approach 2:
The patent replaces the mechanical ray casting approach with a computational method using depth maps and weighted pixel summation. Instead of performing complex ray tracing operations that require oversampling, the system uses depth information to directly calculate which source pixels contribute to each output pixel and applies appropriate weights. This substitution eliminates the need for inefficient random sampling and oversampling while maintaining bokeh sharpness, significantly improving rendering speed.
3Productivity
If lens blur effects are applied with techniques that improve rendering speeds, then processing is faster, but bleed-through across depth discontinuities occurs
Solution Approach 1:
The patent introduces depth maps as an intermediary data structure that mediates between the input image and the blur computation. The depth map provides explicit depth information for each pixel, enabling the system to correctly handle occlusion relationships and depth discontinuities during the weighted sum calculation. This intermediary allows the GPU to efficiently process blur effects at high speed while maintaining accuracy at depth boundaries, as the depth map pre-resolves which pixels should be visible and which should be occluded.
4Device complexity
If blending is performed directly on gamma-encoded values, then processing is simplified, but objects in front of the focal plane have unnaturally sharp edges
Solution Approach 1:
The patent addresses the edge sharpness issue by incorporating depth information as an additional dimension in the blending process. Instead of simply blending gamma-encoded pixel values, the system weights source pixels based on their depth distances from the focal plane. This depth dimension allows the system to naturally blur edges of objects in front of the focal plane while maintaining the simplicity of working with gamma-encoded values. The depth-weighted approach preserves processing simplicity while achieving natural-looking edges.
Data Source
AI summary
Embodiments are disclosed for adding lens blur effects to input images. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving a request to apply a lens blur effect to an image an accumulation region of the input image corresponding to a first pixel of an output image, determining a weight for each of the one or more source pixels based at least in part on an iris map for the one or more source pixels and a depth distance value between each of the one or more source pixels and the focal plane, and generating an output pixel value for the first pixel by calculating a weighted sum of the one or more source pixels from the accumulation region.