Depth Map Nonlinear Filtering for 6DoF Video Down-sampling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for encoding immersive 6DoF video face challenges in reducing pixel rate while maintaining quality, as pruning views can introduce errors and reduce user experience due to the complexity of analyzing redundant texture patches.
Innovation Solution
The method involves nonlinear filtering of depth maps before down-sampling, using techniques like morphological filtering or machine learning algorithms to preserve foreground objects, and encoding these processed maps to generate a video bitstream, which is then decoded with complementary filtering to reconstruct the depth map.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If down-sampling is applied to reduce pixel rate, then data transmission efficiency is improved, but small foreground objects may disappear or be lost in the depth map
Solution Approach 1:
The patent applies preliminary action by performing nonlinear filtering on the depth map before down-sampling. This preprocessing step enlarges the area of foreground objects and smooths depth variations, ensuring that small objects are preserved during the subsequent down-sampling process. The filtering operation prepares the depth map in advance to withstand the information loss inherent in down-sampling.
Solution Approach 2:
The patent changes parameters by transforming the depth map through nonlinear filtering operations that modify pixel values based on neighborhood relationships. This parameter transformation enlarges foreground object areas and reduces depth variations, allowing the down-sampled version to maintain object integrity despite reduced resolution.
2Ease of manufacture
If linear filtering is used before down-sampling, then processing simplicity is maintained, but intermediate depth values are introduced making it difficult to distinguish object boundaries
Solution Approach 1:
The patent applies inversion by using nonlinear filtering instead of linear filtering. Rather than accepting the intermediate depth values produced by linear filtering, the invention inverts the approach by using ordinal statistics (min, max, median) to produce discrete depth values that maintain clear object boundaries while still providing smoothing effects.
Solution Approach 2:
The patent changes the filtering parameter from linear averaging to nonlinear ordinal operations. This parameter change transforms how depth values are computed, replacing continuous intermediate values with discrete values from the original depth map, thereby preserving boundary clarity while maintaining smoothing benefits.
3Productivity
If pruning views is applied to reduce redundancy, then pixel rate is reduced, but analysis complexity increases and quality may be reduced
Solution Approach 1:
The patent applies extraction by removing redundant information through down-sampling the depth map. Instead of pruning entire views through complex analysis, the invention extracts and retains only the essential depth information at a lower resolution, eliminating redundancy while preserving critical foreground object data through preliminary nonlinear filtering.
Data Source
AI summary
Methods of encoding and decoding video data are provided. In an encoding method, source video data comprising one or more source views is encoded into a video bitstream. Depth data of at least one of the source views is nonlinearly filtered and downsampled prior to encoding. After decoding, the decoded depth data is up-sampled and nonlinearly filtered.