Differential Video Rendering via Hierarchical Region Tree
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video texturing methods require high CPU-GPU bandwidth for rendering YUV frames on RGB graphics planes, leading to increased power consumption and system performance degradation due to unnecessary rendering of unchanged pixels and high memory access, which restricts applicability to low-end systems and causes audio-video synchronization issues.
Innovation Solution
A differential video rendering system that decodes bitstreams into sets of differential blocks, normalizes and maps these blocks to tile size regions, generates a hierarchal region tree, and renders only optimal regions, minimizing GPU DDR accesses and bandwidth usage by focusing on changed pixels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full video frames are rendered on graphics display using related art video texturing methods, then complete video rendering is achieved, but system bandwidth utilization increases and power consumption rises
Solution Approach 1:
The video frame is segmented into multiple tiles, and each tile is further divided into blocks. This hierarchical segmentation allows the system to process only the necessary blocks that contain changed pixels, rather than rendering the entire frame. The patent divides the frame into tiles (e.g., 16x16 pixels) and each tile into blocks (e.g., 4x4 pixels), enabling selective rendering based on change detection.
Solution Approach 2:
The patent extracts only the changed pixels from the full video frame by comparing consecutive frames. The change detection mechanism identifies blocks that contain modified pixels, and only these extracted changed blocks are transferred to GPU memory and rendered. This extraction process eliminates the need to process unchanged portions of the frame, reducing bandwidth and power consumption.
2Reliability
If full video frames are decoded and rendered using GPU, then complete video display is achieved, but CPU-GPU bandwidth becomes a critical resource and system performance degrades
Solution Approach 1:
The patent segments the video processing pipeline into distinct stages: frame decoding, change detection, tile division, block processing, and selective rendering. By segmenting the data flow, only the necessary blocks containing changed pixels are transferred between CPU and GPU, reducing bandwidth requirements and improving system productivity.
Solution Approach 2:
Instead of rendering the complete video frame, the patent applies partial action by rendering only the portions of the frame that contain changed pixels. This partial rendering approach maintains visual quality where changes occur while avoiding unnecessary processing of unchanged areas, thereby improving system performance without compromising display completeness.
3Reliability
If high bandwidth is utilized for video rendering, then complete video frames can be rendered, but audio-video synchronization issues occur and low-end systems cannot support the feature
Solution Approach 1:
The patent segments the rendering workload into manageable tiles and blocks, allowing low-end systems to process only essential portions of the frame. This segmentation reduces the computational burden on GPUs with limited memory bandwidth, enabling video texturing features to run on low-end hardware while maintaining adaptability across different system configurations.
Solution Approach 2:
The patent changes the parameter of data transfer by transmitting only the essential changed blocks rather than complete frames. This parameter change from full-frame transmission to selective block transmission reduces bandwidth requirements, making the system compatible with low-end devices while preserving video rendering quality where changes occur.
4Reliability
If unchanged pixels are included in rendering process, then complete frame is displayed, but multiple GPU cycles are wasted and memory access increases
Solution Approach 1:
The patent extracts changed pixels from the full frame through change detection comparison between consecutive frames. By taking out only the changed blocks for processing, the system avoids wasting GPU cycles on unchanged pixels while maintaining complete frame display through subsequent blending operations that combine rendered changed blocks with previous frame data.
Solution Approach 2:
The patent applies partial action by performing rendering operations only on blocks containing changed pixels rather than the entire frame. This partial processing reduces GPU cycle consumption and memory access time while maintaining display completeness through efficient compositing of rendered blocks with unchanged portions from the previous frame.
Data Source
AI summary
A differential video rendering system, including a graphics processing unit (GPU); a graphical display coupled to the GPU; a video decoder configured to decode a bitstream of encoded data into a plurality of sets of decoded blocks; at least one processor configured to: generate, based on a first set of the plurality of sets of decoded blocks, a first differential video frame comprising a plurality of sets of differential regions, normalize each set of the plurality of sets of differential regions to a fixed size block to provide a normalized plurality of sets of differential regions, map a respective set of the normalized plurality of sets of differential regions to align with a respective tile size region of a plurality of tile size regions conforming with the GPU, generate a hierarchal region tree based on the normalized plurality of sets of differential regions mapped to the plurality of tile size regions, and generate a plurality of optimal regions based on the hierarchal region tree satisfying a predefined criteria corresponding to a pre-defined optimal number of regions and a predefined efficiency parameter; and a graphics rendering engine configured to render the first differential video frame on the graphical display based on the plurality of optimal regions and a group of differential regions.