Volumetric Media Segmentation for Angle-Aware Sub-Volume Cropping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies struggle to efficiently crop and manage volumetric video data for immersive experiences, particularly in virtual reality systems, due to the complexity of rendering and viewing from multiple angles, leading to high storage requirements and user navigation challenges.
Innovation Solution
The method employs a Bloch sphere representation of volumetric video data, using trained convolutional neural networks (CNNs) to generate correlation scores for sub-spheres, optimizing rotation matrices, and applying cylindrical weights to achieve user-defined angles of view, allowing for selective cropping and efficient storage of volumetric media.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If volumetric video data is stored in full resolution for immersive viewing from multiple angles, then user experience quality is improved, but storage capacity requirements increase significantly
Solution Approach 1:
The volumetric video data is divided into multiple sub-volumes, each representing a specific viewing angle or spatial region. The system segments the 360-degree environment into discrete angular segments, allowing selective storage and retrieval of only the portions needed for a given viewing direction, thereby reducing overall storage requirements while maintaining full immersive capability.
Solution Approach 2:
The system applies different quality levels and resolution settings to different angular segments of the volumetric video. High-resolution data is stored for regions where users are most likely to look (based on head tracking or predicted viewing patterns), while peripheral or less-important regions use lower resolution or compressed formats, optimizing the balance between quality and storage efficiency.
2Manufacturing precision
If high-resolution volumetric video data is processed to maintain quality across all viewing angles, then image quality is improved, but processing complexity increases
Solution Approach 1:
The system performs preliminary processing and pre-computation of rotation matrices and sub-volume extractions during the encoding phase. By pre-calculating the optimal sub-volumes for various viewing angles and storing them in an organized manner, the system reduces the computational burden during real-time playback, allowing high-quality rendering without excessive processing complexity during usage.
3Adaptability or versatility
If complete volumetric data is retained for future cropping and navigation, then navigation flexibility is improved, but storage requirements increase
Solution Approach 1:
The system changes the representation parameters of volumetric video data by converting it into a structured format of sub-volumes with associated metadata including rotation matrices and correlation scores. This parameter transformation allows the data to be efficiently queried and cropped based on user navigation needs without storing redundant information, achieving both flexibility and storage efficiency.
Data Source
AI summary
Cropping volumetric media is provided. Received volumetric video data is represented by a Bloch sphere. A user selected sub-volume is received to which the volumetric video data is to be cropped. CNNs having different strides are applied to sub-spheres defined within the Bloch sphere. The CNNs generate a correlation score for each sub-sphere relative to a user specified parameter. Best rotation matrices for the sub-spheres are determined to achieve a user specified angle of view, and the sub-sphere correlation scores are optimized according to the best rotation matrices. Cylindrical weights are applied to the sub-spheres according to the optimized correlation scores, and a best solution is selected for the sub-volume according to the correlation scores. The rotation matrices are reflected orthogonally onto the sub-volume, and the sub-volume is cropped from the volumetric video according to the sub-sphere with the highest correlation score to create the cropped image.