Blending View-Independent and Dependent Video Streams for 360-Degree Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Constructing a high-quality view from multiple 360-degree video streams with different properties such as frame rate and geometry is challenging due to increased storage and processing requirements, necessitating an efficient method to blend these streams without compromising quality and bandwidth.
Innovation Solution
A method and system that involve generating a view independent stream and a view dependent stream with different geometries, where the view independent stream has a higher field of view and lower quality outside the defined field of view, and the view dependent stream has higher quality within the defined field of view. These streams are blended on a client device using a GPU to create a seamless 360-degree view, optimizing bandwidth, storage, and processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If multiple video streams with different geometries are processed to construct high-quality 360-degree views, then image quality is improved, but storage and processing requirements increase
Solution Approach 1:
The video stream is divided into two separate streams: a view-independent stream containing base geometry information and a view-dependent stream containing differential geometry information. This segmentation allows the system to process and store only the necessary differences rather than complete duplicate information, reducing storage and processing requirements while maintaining high image quality for multiple viewpoints.
Solution Approach 2:
The view-dependent stream provides high-quality differential geometry information specifically for regions of interest (within the field of view), while the view-independent stream provides adequate base quality for the entire 360-degree view. This local quality approach ensures high image quality where needed without uniformly increasing quality across all regions, thereby optimizing storage and processing requirements.
2Adaptability or versatility
If view-dependent streams are generated for multiple viewpoints, then adaptability to different viewing angles is improved, but bandwidth consumption increases
Solution Approach 1:
Instead of transmitting complete video streams for each viewpoint, the system transmits a view-independent base stream and view-dependent differential streams. The client device reconstructs the full view-dependent streams locally by combining the base stream with the differential streams. This copying approach reduces bandwidth consumption by transmitting only the differences while maintaining adaptability to multiple viewpoints.
Solution Approach 2:
The system transmits only the essential differential geometry information in the view-dependent stream rather than complete video data for each viewpoint. This partial action approach provides sufficient information for high-quality view reconstruction without the excessive bandwidth consumption of transmitting full streams for all possible viewpoints, achieving adaptability with optimized bandwidth usage.
3Manufacturing precision
If high-resolution 360-degree video is encoded, then image quality is improved, but processing time increases
Solution Approach 1:
The view-independent base stream and view-dependent differential streams are pre-encoded and prepared on the server side before transmission. This preliminary action allows the client device to receive pre-processed streams that require minimal real-time processing for reconstruction, reducing client-side processing time while maintaining high image quality through the pre-computed differential information.
Solution Approach 2:
The encoding process is segmented into two passes: first encoding the view-independent base stream, then encoding the view-dependent differential stream based on the base stream. This segmentation allows parallel processing and optimization of each stream independently, reducing overall processing time while maintaining high image quality through specialized encoding approaches for each stream type.
Data Source
AI summary
A method and system for constructing view from multiple video streams is provided. The method includes receiving a view independent stream. The method further includes selecting a first view dependent stream, wherein the view independent stream and the first view dependent stream has at least one different geometry. The method also includes generating end user views corresponding to the view independent stream and the first view dependent stream. Further, the method includes blending the end user views to generate a view for display.