Mask-Based Image Composition for Artifact-Free Remote Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing remote rendering technologies suffer from artifacts at the edges of features due to numerical errors in depth values caused by lossy video compression, particularly noticeable during motion, leading to incorrect layer composition and perceptible flicker.
Innovation Solution
Utilizing depth dilation and cutout masks to preprocess image layers, where cutout masks are generated and transmitted losslessly to indicate valid and invalid pixel locations, allowing accurate composition at the local device without artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If lossy video compression is used to transmit depth information, then transmission efficiency is improved, but numerical errors in depth values occur causing artifacts at edges
Solution Approach 1:
The patent segments the depth information transmission into two parts: compressed dilated depth information for efficiency and lossless cutout masks for boundary accuracy. The cutout masks identify valid pixels, allowing the system to use compressed depth data for most areas while preserving exact boundary information where needed, thus resolving the contradiction between transmission efficiency and depth accuracy.
Solution Approach 2:
The patent introduces dilated depth information as an intermediary representation. By dilating the depth values before compression, the system creates a smoothed depth map that reduces discontinuities and minimizes compression artifacts. The cutout masks then serve as intermediaries to restore accurate boundaries, combining the benefits of compression with boundary precision.
2Quantity of substance
If depth values are compressed to reduce data size, then bandwidth usage is reduced, but edge artifacts become more noticeable
Solution Approach 1:
The patent applies preliminary dilation to the depth information before compression. This preliminary action smooths out depth discontinuities and reduces the impact of compression artifacts on edges. By preparing the depth data in advance through dilation, the system minimizes harmful compression effects while maintaining smaller data sizes for efficient transmission.
Solution Approach 2:
The patent extracts boundary information into separate cutout masks that are transmitted losslessly. These masks contain the precise boundary information needed to avoid edge artifacts, while the main depth information can be compressed. By separating boundary data from bulk depth data, the system reduces overall data size while preserving edge accuracy.
3Speed
If layers are composited using standard depth comparison, then composition speed is maintained, but incorrect layer selection occurs at edges
Solution Approach 1:
The patent applies local quality by using different depth representation strategies in different regions. For valid pixels identified by cutout masks, the system uses dilated depth values for efficient composition. For boundary pixels, the cutout masks provide explicit validity information that overrides depth comparisons, ensuring correct layer selection at edges without slowing down the overall composition process.
Solution Approach 2:
The patent implements feedback through the cutout masks that provide validity information about pixels. This feedback mechanism allows the compositing system to adjust its behavior based on pixel validity, selecting layers correctly at boundaries while maintaining fast composition speeds for valid pixels. The mask information feeds back into the composition decision process.
Data Source
AI summary
A method, comprising: generating a first layer of an image; generating first depth information for the first layer; generating a second layer of the image; generating second depth information for the second layer; generating dilated depth information, comprising dilating the second depth information; generating a mask using the second depth information; and transmitting the first depth information of the first layer, the dilated depth information of the second layer and the mask.