Multi-view Video Depth Frame Rate Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high computational and bandwidth costs associated with simultaneous depth estimation for multiple cameras in real-time pose a significant challenge for the widespread adoption of multi-view video applications, such as augmented and virtual reality, due to the extensive processing and transmission requirements.
Innovation Solution
A method that reduces the processing and transmission burden by obtaining source view images at a higher frame rate and source view depth components at a lower frame rate, allowing for the selection between source view playback mode and interpolated playback mode, where the latter generates an interpolated image based on source view images and depth components, enabling efficient data processing and transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If depth estimation is performed for multiple cameras simultaneously in real-time, then the multi-view video quality and interactivity are improved, but the computational cost and bandwidth requirements increase significantly
Solution Approach 1:
The patent segments the depth estimation process by applying it only to selected source views rather than all camera feeds. This selective approach divides the computational workload, maintaining multi-view video quality for interpolated views while reducing the overall computational cost and bandwidth requirements for real-time processing
Solution Approach 2:
The patent applies depth estimation partially - only to specific source views that serve as references for interpolation, rather than performing exhaustive depth estimation for all camera feeds. This partial action maintains sufficient video quality for the final multi-view output while significantly reducing computational power requirements
2Measurement precision
If depth components are transmitted at the same frame rate as source view images, then the temporal resolution and video quality are maintained, but the bandwidth consumption increases
Solution Approach 1:
The patent applies different quality levels to different data streams: source view images are transmitted at full frame rate with high quality, while depth components are transmitted at a reduced frame rate with lower temporal resolution. This local quality differentiation maintains overall video quality where needed while reducing bandwidth consumption for depth data transmission
3Manufacturing precision
If all source view images and depth components are processed for every frame, then the processing completeness is ensured, but the processing time and resource usage increase
Solution Approach 1:
The patent performs preliminary depth estimation only on selected source views that will serve as references for interpolation. This preliminary action on a subset of data ensures processing completeness for the reference views while reducing overall processing time and resource usage, as interpolated views can be generated from these pre-processed references without requiring separate depth estimation
Data Source
AI summary
A method for presenting multi-view video data of a scene. The method comprises obtaining one or more sequences of source view images of the scene at a source view frame rate and obtaining one or more sequences of source view depth components of the scene at a source view depth frame rate. The source view depth frame rate is lower than the source view image frame rate. A selection between a source view playback mode and an interpolated playback mode is obtained, where, if the interpolated playback mode is selected, an interpolated image of the scene is generated based on one or more source view images from one or more of the sequence of source view images and on one or more corresponding source view depth components from at least one sequence of source view depth components. The interpolated image is output to a display. If the source view playback mode is selected, a sequence of source view images is output to the display.