Volumetric Video Patch Filtering Metadata Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current volumetric video coding techniques lack tools to indicate filtering processes, such as blurring or hiding, for specific subsets of patches during reconstruction, due to limitations in playback devices or confidentiality concerns, despite the need for such processing.
Innovation Solution
A method and apparatus that project a 3D representation onto 2D patches, generate auxiliary patch information with metadata indicating filtering processes, and encode this information along with geometry and texture images in a bitstream, allowing for controlled filtering during reconstruction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If regular 2D video compression algorithms are used to compress projected surfaces, then compression efficiency is improved, but no filtering or blurring can be applied to specific patches during reconstruction
Solution Approach 1:
The patent divides the volumetric scene into multiple patches, where each patch can be independently processed. The auxiliary patch information structure allows different filtering operations to be applied to different patches during reconstruction, enabling selective blurring or hiding of specific regions while maintaining efficient compression of the entire scene.
Solution Approach 2:
The patent introduces auxiliary patch information that includes metadata specifying filtering parameters for individual patches. This allows different quality levels and processing treatments to be applied locally to different patches during reconstruction, such as applying blurring to confidential regions while maintaining high quality for other areas.
2Adaptability or versatility
If playback device limitations are considered, then content can be adapted to device capabilities, but additional processing steps are required during reconstruction
Solution Approach 1:
The patent performs filtering decisions and auxiliary information generation during the encoding phase rather than during playback. The encoder analyzes the content and determines which patches require filtering, storing this information in the auxiliary patch information structure. During reconstruction, the playback device simply follows the pre-determined filtering instructions, reducing the complexity burden on the playback device.
3Object-affected harmful factors
If confidentiality requirements are implemented, then sensitive content can be hidden, but metadata structures must be extended to support filtering indicators
Solution Approach 1:
The patent applies confidentiality protection selectively to specific patches rather than the entire volumetric scene. The auxiliary patch information contains metadata that identifies which patches require filtering and specifies the filtering operation to be applied. This allows confidential regions to be hidden or blurred while maintaining normal quality and accessibility for non-confidential areas.
Solution Approach 2:
The auxiliary patch information structure acts as an intermediary between the encoded video data and the reconstruction process. It carries metadata that instructs the reconstruction algorithm on how to process specific patches, enabling confidentiality protection without requiring complex modifications to the core compression algorithms or playback device architecture.
Data Source
AI summary
A method including projecting a 3D representation of at least one object onto at least one 2D patch; generating a geometry image, a texture image, an occupancy map and auxiliary patch information from 5 the 2D patch, wherein the auxiliary patch information includes metadata indicating whether a filtering process shall be applied on said 2D patch upon reconstruction of the 3D representation of said at least one object; and encoding the auxiliary patch information and zero or more of the following: the geometry image, the texture image and the 10 occupancy map in or along a bitstream.