Mask-Based Image Composition for Artifact-Free 3D Layers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing remote rendering technologies suffer from artifacts at the edges of features due to numerical errors in depth values, particularly around object edges, leading to incorrect layer composition and perceptible flicker during motion, which is exacerbated by lossy video compression and depth buffer dilation being incompatible with multi-layer image composition.
Innovation Solution
The use of cutout masks and depth dilation as preprocessing steps to improve efficiency, where masks are generated from depth information before compression, allowing accurate identification of valid and invalid pixel locations, and transmitted losslessly to the local device for precise layer composition, reducing artifacts and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If depth buffer dilation is applied to improve encoding efficiency, then compression performance is improved, but multi-layer image composition becomes incompatible and produces artifacts at object edges
Solution Approach 1:
The patent segments the depth information processing into two distinct components: dilated depth values for efficient encoding and binary masks for precise composition. The mask layer divides the image into valid and invalid pixel regions, allowing the dilated depth buffer to be used without causing composition artifacts at object edges.
Solution Approach 2:
The patent introduces a binary mask as an intermediary element between the dilated depth buffer and the composition process. This mask acts as a mediator that prevents the harmful effects of depth dilation from propagating to the final composited image, while still allowing the benefits of dilated encoding to be retained.
2Loss of energy
If lossy video compression is applied to reduce bandwidth, then transmission efficiency is improved, but numerical errors in depth values increase causing composition artifacts
Solution Approach 1:
The patent segments depth information transmission into two separate data streams: lossily compressed dilated depth values for efficient bandwidth usage, and losslessly transmitted binary masks for precise composition. This segmentation allows different compression strategies to be applied to different components based on their requirements.
Solution Approach 2:
The patent creates a simplified copy of depth information in the form of binary masks that indicate only valid and invalid regions. This copy is transmitted losslessly and used to override the imprecise depth values from lossy compression, ensuring accurate composition without requiring high-precision depth data transmission.
3Device complexity
If standard composition methods are used without masks, then processing simplicity is maintained, but artifacts appear at edges of features due to incorrect layer selection
Solution Approach 1:
The patent performs preliminary action by generating binary masks that pre-identify valid and invalid pixel regions before the composition process. This preliminary segmentation of the image allows the composition algorithm to selectively apply depth values only where appropriate, preventing edge artifacts without requiring complex real-time decisions during composition.
Data Source
Figure 1
Figure 1A
Figure 1B
AI summary
A method, comprising: generating a first layer of an image; generating first depth information for the first layer; generating a second layer of the image; generating second depth information for the second layer; generating dilated depth information, comprising dilating the second depth information; generating a mask using the second depth information; and transmitting the first depth information of the first layer, the dilated depth information of the second layer and the mask.