Mesh Geometry Coding with Depth-Scaled Rasterization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for compressing 3D meshes in video-based point cloud compression standards lack efficient mechanisms to encode connectivity and attributes of triangle faces, particularly for sparse meshes, leading to inefficiencies and potential loss of data.
Innovation Solution
A method involving depth information mapping to a luma channel using fewer bits, generating a depth scaling factor for bilinear interpolation, and performing normal filtering to adjust vertex positions based on estimated surface normals, utilizing one-ring, two-ring, or three-ring face neighborhoods for minimization of total normal angles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If depth information is mapped to luma channel using fewer bits, then encoding efficiency is improved, but depth precision deteriorates
Solution Approach 1:
The patent applies parameter changes by introducing a depth scaling factor that transforms the depth values after quantization. By scaling the depth values back up during rendering, the system recovers precision from the reduced bit depth without needing to use more bits for encoding. This resolves the contradiction by changing the parameter representation rather than increasing the encoding bit depth.
2Adaptability or versatility
If vertices are encoded using RAW patch data, then connectivity encoding flexibility is improved, but encoding efficiency deteriorates
Solution Approach 1:
The patent makes the depth image serve multiple functions: it encodes both the depth information and the normal direction information simultaneously. The normal filtering process uses the same depth data to compute surface normals, eliminating the need for separate normal maps or additional connectivity encoding. This multi-functionality improves encoding efficiency while maintaining flexibility.
3Manufacturing precision
If normal filtering is applied to adjust vertex positions, then mesh reconstruction quality is improved, but processing complexity increases
Solution Approach 1:
The patent performs normal filtering as a preliminary action during the encoding process, computing accurate surface normals before the actual mesh reconstruction. By pre-computing the normals from the depth image and applying them during rendering, the system improves mesh reconstruction quality without adding complex post-processing steps. The normal filtering is integrated into the existing rendering pipeline rather than being a separate complex operation.
Data Source
AI summary
Depth image generation is improved by more efficient encoding using video codecs. The mapping of the depth to the luma channel is performed by not using all bits available, and with the remaining bits, a depth scaling factor is generated and incorporated into the bilinear interpolation algorithm used during rasterization. A normal filtering procedure is described, where the positions of vertices are adjusted according to the normal estimated from the surface pixels. After decoding the depth image, the pixels related to the surface of a triangle are collected and used to estimate a plane and the normal of the plane. The normal is compared to the normal obtained from the plane defined by the three vertices of the triangle. If there is no match, the positions of the vertices are adjusted to match the estimated normal from the pixels' surfaces. The adjustment can follow an iterative minimization process.