3D Video Mesh Compression via Consistent Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Free-viewpoint video (FVV) is extremely data-intensive, requiring the storage and transmission of object and texture data for each frame, which impairs performance and limits its usage in immersive video experiences like virtual and augmented reality.
Innovation Solution
The technique splits 3D video data into segments with consistent mesh topology, constructs a consistent texture atlas, and compresses both using standard video compression techniques, allowing for efficient storage and transmission of compressed data files that can be decompressed and rendered independently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If Free-viewpoint video (FVV) stores and transmits object and texture data for each frame, then the quality and immersion of video experience is improved, but the data intensity increases significantly impairing performance
Solution Approach 1:
The patent divides the 3D video data into multiple segments based on temporal coherence and mesh topology consistency. Each segment contains a subset of frames that share common geometric structures, allowing independent compression and processing. This segmentation reduces the data volume that needs to be transmitted while maintaining the quality required for immersive video experiences.
Solution Approach 2:
The patent transforms the representation from traditional 2D pixel-based video to 3D mesh-based representation with associated texture maps. By organizing data in three-dimensional space with explicit geometric structures, the system enables more efficient compression through spatial redundancy exploitation while maintaining high-quality rendering for free-viewpoint video applications.
2Manufacturing precision
If FVV transmits complete object and texture data for each frame, then rendering quality is maintained, but transmission time and processing overhead increase
Solution Approach 1:
The patent performs pre-processing operations including mesh segmentation, topology analysis, and compression parameter optimization before data transmission. By preparing and organizing the 3D video data into optimized segments with consistent mesh structures in advance, the system reduces the computational burden during real-time transmission and rendering, thereby decreasing transmission time while maintaining rendering quality.
3Adaptability or versatility
If mesh topology changes are allowed between frames, then object deformation is accurately represented, but compression efficiency decreases
Solution Approach 1:
The patent implements a dynamic segmentation strategy that adapts to mesh topology changes throughout the video sequence. The system identifies regions where topology remains consistent and groups them into segments, while allowing topology changes at segment boundaries. This dynamic approach enables accurate representation of object deformation while maintaining compression efficiency by exploiting temporal coherence within each segment.
Data Source
AI summary
Mesh-based raw video data (or 3D video data) includes a sequence of frames, each of which includes geometry data (e.g., triangle meshes or other meshes) and texture map(s) defining one or more objects. The raw 3D video data is segmented based on consistent mesh topology across frames. For each segment, a consistent mesh sequence (CMS) is defined and a consistent texture atlas (CTA) is generated. The CMS and CTA for each segment are compressed and stored as compressed data files. The compressed data files can be decompressed and used to render displayable images.