3D Motion Alignment for Volumetric Scene Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D scene reconstruction methods face challenges with artifacts due to unaligned or differently framed volumetric frames, particularly when objects are encoded in separate sub-streams or rendered at varying frame rates, leading to judder artifacts and reduced immersion in 6DoF experiences.
Innovation Solution
A method and apparatus for reconstructing volumetric frames by temporally resampling and aligning them using 3D motion information, incorporating patch-atlas based representations and additional 3D motion attributes to ensure synchronized rendering across multiple objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If volumetric frames are encoded in separate sub-streams at different frame rates, then encoding flexibility and adaptability are improved, but temporal alignment and synchronization deteriorate, causing judder artifacts
Solution Approach 1:
The system performs preliminary temporal alignment by estimating 3D motion vectors between volumetric frames before rendering. This preliminary action prepares the data structure to handle different frame rates and encodings, ensuring that when frames are composed, they are already temporally aligned through motion compensation, preventing judder artifacts while maintaining encoding flexibility
Solution Approach 2:
The patent introduces an intermediary data structure that includes 3D motion vector information as a bridge between differently framed sub-streams. This intermediary layer allows the system to reconcile temporal differences between sub-streams encoded at different frame rates, enabling precise synchronization during composition without sacrificing encoding adaptability
2Reliability
If 3D motion information is added to align volumetric frames, then temporal synchronization and immersion quality are improved, but data complexity and processing requirements increase
Solution Approach 1:
The system segments the 3D motion information into discrete motion vector data associated with specific volumetric frames or groups of frames. This segmentation allows the complex synchronization task to be broken down into manageable units, where motion vectors are processed and applied individually to relevant frame segments, reducing overall processing complexity while maintaining synchronization reliability
Solution Approach 2:
The patent changes the parameter representation by encoding motion information as 3D motion vectors that can be efficiently stored and processed. These vectors represent temporal relationships between frames in a compact form, allowing the system to handle synchronization without proportionally increasing data complexity, as the motion parameters are derived from and consistent with the existing volumetric frame structure
Data Source
AI summary
A method and an apparatus for reconstructing at least one sequence of volumetric frames from a data stream are provided wherein the sequence of volumetric frames is temporally resampled and/or temporally aligned to a composition time frame using 3D motion information obtained for the volumetric frames. The method comprises decoding from the data stream the at least one sequence of volumetric frames, obtaining a 3D motion information representative of a displacement in a 3D space of points of volumetric frames of the at least one sequence, and displacing points of at least one volumetric frame of the sequence to a composition time frame using the 3D motion information.