Spherical Mapping Operator for Volumetric Video Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies for encoding and decoding volumetric videos struggle to efficiently allocate pixels, leading to suboptimal image quality, especially in regions of interest, due to the limitations of current mapping operators.
Innovation Solution
A continuous spherical mapping operator is introduced, which allows for the definition of a Region Of Interest (ROI) with controlled pixel density. The mapping operator varies the angular resolution isotropically from the center of the ROI to its border, using a set of five floating parameters to characterize the projection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a conventional mapping operator (e.g., ERP, CMP) is used to project 3D points onto a 2D image plane, then the encoding process is simple and codec-friendly, but the pixel density is uniformly distributed leading to suboptimal image quality in regions of interest
Solution Approach 1:
The patent applies local quality by making the pixel density function vary locally across different regions of the image. The mapping operator adjusts the projection surface area based on the distance from a reference pixel, creating higher pixel density in regions of interest (closer to reference pixel) and lower density in peripheral regions. This local adaptation of pixel density improves image quality where needed without uniformly increasing complexity across the entire mapping process
Solution Approach 2:
The patent introduces dynamics by making the mapping operator adaptive rather than static. The surface area function dynamically adjusts based on the distance parameter, allowing the pixel density to vary continuously across the image plane. This dynamic adjustment enables the system to optimize image quality for specific regions of interest while maintaining codec compatibility, resolving the contradiction between quality and complexity
2Manufacturing precision
If pixel density is uniformly distributed across the image frame, then the encoding process is straightforward, but regions of interest do not receive sufficient encoding quality
Solution Approach 1:
The patent redistributes pixel density locally by using a distance-based surface area function. Regions closer to the reference pixel (regions of interest) receive higher pixel density while peripheral regions receive lower density. This local quality adjustment ensures that encoding quality is optimized for important regions without wasting pixels on less important areas, thus improving ROI encoding quality while managing overall pixel density allocation
Solution Approach 2:
The patent changes the parameter of pixel density distribution from uniform to non-uniform based on distance from reference pixel. By introducing the distance parameter into the surface area calculation, the system dynamically adjusts pixel allocation to favor regions of interest. This parameter change resolves the contradiction by allowing flexible control over where encoding quality is prioritized
3Quantity of substance
If the entire image frame is allocated for video content, then no additional space is available for metadata, but reducing frame space for metadata decreases the resolution or quality of the encoded video content
Solution Approach 1:
The patent creates space for additional data by reducing pixel density in peripheral regions where visual importance is lower. This local reduction in pixel density frees up space in the image frame that can be allocated to metadata or depth/color patches without significantly impacting the quality of regions of interest. The technique allows simultaneous accommodation of additional data and maintenance of video content quality where it matters most
Data Source
AI summary
Methods and devices are provided to encode points of a sphere onto an image by projecting them as a function of an isotropic region guided mapping operator and/or to decode pixels of an image onto a sphere by de-projected them according to the same parametrized mapping operator. A region of interest is determined on the sphere and the angular resolution of pixels of the image is modulated as a function of their distance to the reference pixel corresponding to the center of the region of interest. Chosen angular resolutions and size of the region of interest are associated with the image and encoded in a stream.