Depth-Guided Image Filtering for Video Conferencing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video conferencing technologies face challenges in optimizing video encoding and decoding under bitrate constraints, leading to suboptimal image quality and increased noise, especially in environments where depth information is not fully exploited for image segmentation and coding.
Innovation Solution
A depth-guided image filtering system that uses depth maps and texture data to filter images, preserving details at depth discontinuities and intensity variations while reducing noise, and is integrated with video encoders and decoders to prioritize details closer to the viewpoint, reducing coding artifacts and errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional video encoding is used under bitrate constraints, then transmission efficiency is improved, but image quality deteriorates with increased noise and artifacts
Solution Approach 1:
The patent segments the image into multiple depth layers using depth maps, separating foreground objects at different distances from the background. This segmentation allows differential filtering and encoding strategies to be applied to each layer, preserving important details in near-field objects while allowing more compression in distant background regions, thereby maintaining image quality under bitrate constraints.
Solution Approach 2:
The patent applies local quality enhancement by using depth-guided filtering that adapts filtering strength based on local depth information. Regions with small depth values (near the viewpoint) receive stronger filtering to preserve details, while regions with large depth values (far from viewpoint) receive lighter filtering. This local adaptation optimizes the balance between noise reduction and detail preservation in different spatial regions.
2Object-affected harmful factors
If noise reduction filtering is applied to the entire image, then noise is reduced, but important details at object boundaries are lost
Solution Approach 1:
The patent applies different filtering strengths to different regions based on depth information. The filtering kernel adapts its strength according to the local depth value, applying stronger smoothing in regions far from the viewpoint where noise is more problematic, and weaker filtering in near-field regions where detail preservation is critical. This resolves the contradiction by making filtering strength spatially variable rather than uniform.
Solution Approach 2:
The patent introduces depth maps as an intermediary information layer that guides the filtering process. The depth map provides spatial context about object distances, enabling the filtering algorithm to distinguish between noise and important details based on depth discontinuities. This intermediary depth information allows the system to intelligently preserve edges and boundaries while reducing noise in appropriate regions.
3Manufacturing precision
If full resolution depth maps are processed with joint-bilateral filters, then image alignment is improved, but computational cost increases
Solution Approach 1:
The patent segments the depth map processing into different resolution levels or processing stages. Rather than applying computationally intensive joint-bilateral filtering uniformly across the entire full-resolution depth map, the system processes different regions or scales at appropriate resolution levels, reducing overall computational burden while maintaining alignment accuracy where most needed.
Solution Approach 2:
The patent applies depth-guided filtering with adaptive strength based on local depth values. By modulating the filtering strength according to local depth information, the system achieves effective image alignment in critical near-field regions while applying lighter processing in distant background regions, thereby reducing total computational cost while maintaining alignment quality where it matters most.
Data Source
Figure 1
Figure 2A~2B
Figure 3~5
AI summary
A method is provided in one example embodiment that includes receiving a plurality of depth values corresponding to pixels of an image; and filtering the image as a function of a plurality of variations in the depth values between adjacent pixels of a window associated with the image. In more detailed embodiments, the method may include encoding the image into a bit stream for transmission over a network. The filtering can account for a bit rate associated with the encoding of the image.