Immersive Video Processing Valid Pixel Distinction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current immersive video technologies primarily support rotational movements, lacking efficient methods to distinguish and process valid and invalid pixels for 6 Degrees of Freedom (6DoF) motion parallax, which limits the realism and immersion in virtual reality experiences.
Innovation Solution
The method involves parsing video data and metadata from a bitstream, extracting patches from an atlas based on user movement metadata, and generating an occupancy map to differentiate between valid and invalid pixels, using threshold information and occupancy flags to determine pixel validity, thereby enhancing the processing efficiency of immersive videos.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If occupancy information is embedded into a depth atlas, then the need for separate occupancy map is reduced, but the depth information precision may be compromised
Solution Approach 1:
The patent segments the depth atlas into multiple depth maps, each representing different depth ranges. This allows embedding occupancy information in a structured way while preserving depth precision through separate processing of each depth map layer.
Solution Approach 2:
The patent introduces a new dimension by creating an occupancy map that operates independently from the depth atlas. This occupancy map uses threshold information to distinguish valid and invalid pixels, adding a binary validity dimension to the existing depth information without compromising the depth precision stored in the depth atlas.
2Reliability
If threshold information is used to distinguish valid and invalid pixels, then pixel validity can be determined, but processing complexity increases
Solution Approach 1:
The patent performs preliminary action by pre-defining threshold values and organizing depth information into structured depth maps before the actual processing. This allows efficient pixel validity distinction during synthesis without complex real-time calculations, as the threshold comparison is a simple operation on pre-organized data.
3Adaptability or versatility
If 6DoF motion parallax is implemented, then immersion and realism are improved, but the requirement for valid pixel distinction and data processing increases
Solution Approach 1:
The patent extracts and separates the validity information from the depth atlas by using threshold comparison. This creates a clear distinction between valid and invalid pixels, which is essential for 6DoF motion parallax. The extracted validity information can then be used independently to control which depth information is applied during synthesis, simplifying the overall processing requirement.
Solution Approach 2:
The patent applies local quality by treating different regions of the depth atlas differently based on their validity status. Valid pixels in certain depth ranges are processed and applied, while invalid pixels are excluded. This localized processing approach enables 6DoF motion parallax in specific regions without requiring complex processing of the entire depth atlas.
Data Source
AI summary
Disclosed herein is an immersive video processing method. The immersive video processing method includes: determining a priority order of pruning for source videos; extracting patches from the source videos based on the priority order of pruning; generating at least one atlas based on the extracted patches; and encoding metadata. Herein, the metadata may include first threshold information that becomes a criterion for distinguishing between a valid pixel and an invalid pixel in the atlas video.