Multiview Video Cache for Low-Latency Timeline Scrubbing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video rendering technologies face challenges in efficiently converting 2D video streams into multiview formats for immersive displays like holographic and lightfield displays, leading to high latency during timeline scrubbing operations.
Innovation Solution
A multiview video cache is generated by preprocessing 2D frames, storing cache data entries corresponding to different timestamps, and using a pair of nearest-in-time entries to blend images in real-time for efficient multiview frame generation, reducing the need for repeated echo buffering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If 2D video streams are converted into multiview formats for immersive displays, then viewing experience quality is improved, but processing latency increases
Solution Approach 1:
The patent pre-processes 2D video frames into multiview format and stores them in a cache during an initialization phase before playback. This preliminary action converts the 2D frames into multiple view perspectives in advance, so that during actual playback, the pre-converted multiview frames can be directly retrieved from cache without real-time conversion, thereby reducing processing latency while maintaining viewing experience quality
Solution Approach 2:
The patent creates cache data entries that are copies of the original 2D video frames, storing multiple view perspectives (left eye, right eye, intermediate views) in the cache. These cached copies are then retrieved and blended during playback to generate the final multiview output, eliminating the need for repeated real-time conversion and reducing processing time
2Ease of operation
If timeline scrubbing operations are performed on multiview video, then user navigation capability is improved, but processing complexity increases
Solution Approach 1:
The patent pre-converts and caches multiple view frames at different timestamps during initialization, including frames that would be needed for timeline scrubbing operations. When users perform scrubbing to different time positions, the system can directly retrieve the pre-converted multiview frames from cache at those timestamps and blend them, avoiding complex real-time conversion operations and reducing processing complexity
Solution Approach 2:
The patent introduces a cache as an intermediary data structure that stores pre-processed multiview frame data. This cache acts as a mediator between the original 2D video source and the playback/rendering system, allowing efficient random access to frames at different timestamps for timeline scrubbing operations without requiring complex real-time processing
3Manufacturing precision
If echo buffering is used repeatedly for multiview conversion, then conversion accuracy is maintained, but resource usage increases
Solution Approach 1:
The patent creates cache data entries that are copies of the original 2D video frames and stores them in memory during initialization. These cached copies contain the necessary pixel data for multiview conversion, eliminating the need for repeated echo buffering operations during playback. The system retrieves and blends these pre-cached copies to generate multiview frames, maintaining conversion accuracy while significantly reducing CPU/GPU resource usage and energy consumption
Solution Approach 2:
The patent performs the computationally intensive 2D-to-multiview conversion process during the initialization phase, discarding the original 2D frame data after conversion and keeping only the converted multiview frames in cache. During playback, the system recycles these cached multiview frames by blending them to generate final output frames, avoiding repeated full conversion operations and reducing resource usage
Data Source
AI summary
Methods and system of multiview video rendering, preparing a multiview video cache, and real-time multiview video conversion employ video information having multiview frames corresponding to multiview video timestamps to generate views of a multiview frame using cache data entries of a multiview video cache corresponding to a target time stamp. The cache data entries may include a first cache data entry corresponding to a first multiview frame preceding the target timestamp and a second cache data entry corresponding to a second multiview frame following the target timestamp. The target timestamp for a particular multiview frame to be rendered may be based on a user input. The system may include a processor circuit and a memory that stores a plurality of instructions that, when executed by the processor circuit, cause the processor circuit to perform one or both of multiview video rendering and preparing a multiview video cache.