3D Video Overlay Buffering for Frame-Accurate Graphics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In 3D video systems, the performance requirements for drawing real-time overlay graphics are increased due to the need to composite different graphics with alternating 2D and depth video streams, which existing graphics processing units are not fast enough to handle accurately every frame.
Innovation Solution
Implementing two buffers in the graphics unit, one for each output video stream, decouples the graphics processing from the frame accurate overlaying engine, allowing for efficient generation of overlay frames by copying relevant areas from buffered frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If real-time overlay graphics are composited with alternating 2D and depth video streams, then overlay functionality is provided, but processing speed becomes insufficient
Solution Approach 1:
The patent divides the video stream processing into separate paths for 2D and depth frames. The graphics processing unit is segmented to handle overlay graphics composition independently from the main video decoding, allowing parallel processing. This segmentation enables the system to maintain overlay functionality while meeting frame-accurate timing requirements by processing graphics overlays separately for each frame type without bottlenecking the overall pipeline.
2Manufacturing precision
If frame accurate overlaying is performed for every frame, then overlay precision is improved, but processing complexity increases
Solution Approach 1:
The patent implements preliminary action by pre-processing and buffering video frames before they reach the overlay composition stage. Frames are decoded and prepared in advance, allowing the graphics processing unit to perform frame-accurate overlay composition without real-time processing pressure. This preliminary preparation reduces the complexity of real-time frame-accurate processing while maintaining precision.
Solution Approach 2:
The patent introduces an intermediary buffer memory between the video decoding stage and the overlay composition stage. This intermediary structure allows frames to be temporarily stored and managed, decoupling the timing requirements of decoding from the precision requirements of overlay composition. The buffer acts as a mediator that simplifies the overall system complexity while enabling frame-accurate precision.
3Device complexity
If single buffer is used for graphics overlay, then device complexity is reduced, but frame accurate updating becomes impossible
Solution Approach 1:
The patent segments the buffer structure into multiple dedicated buffers - one for 2D frames and another for depth frames. This segmentation allows each buffer to be optimized for its specific frame type, enabling frame-accurate updating by selecting the appropriate buffer based on the current frame type. The segmented buffer structure maintains relatively simple device complexity while achieving the precision required for alternating frame streams.
Solution Approach 2:
The patent implements dynamic buffer management where the system can switch between different buffer configurations based on the frame type being processed. The buffer structure is made dynamic rather than static, allowing it to adapt to the alternating 2D and depth frame streams. This dynamic approach enables frame-accurate updating while keeping the overall device complexity manageable through context-dependent buffer selection.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
A method of decoding and outputting video information suitable for three- dimensional [3D] display, the video information comprising encoded main video information suitable for displaying on a 2D display and encoded additional video information for enabling three-dimensional [3D] display, the method comprising: receiving or generating three-dimensional [3D] overlay information to be overlayed over the video information; buffering a first part of the overlay information to be overlayed over the main video information in a first buffer; buffering a second part of overlay information to be overlayed over the additional video information in a second buffer; decoding the main video information and the additional video information and generating as a series of time interleaved video frames, each outputted video frame being either main video frame or additional video frame; determining a type of an video frame to be outputted being either a main video frame or an additional video frame; overlaying either first or second part of the overlay information on an video frame to be outputted in agreement with the determined type of frame outputting the video frames and the overlayed information.