Wavefront Scan Order Synchronization in Video Block Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video processing methods face inefficiencies in processing video frames due to dependencies between blocks, particularly in H.264 encoding, where blocks at the beginning of a row may depend on blocks at the end of a previous row, leading to broken dependencies and synchronization challenges when using wavefront ordering like knight's order.
Innovation Solution
Implementing a block processing pipeline that uses wavefront ordering, specifically knight's order, to input and process blocks, with buffering and stitching information to manage dependencies, allowing adjacent blocks to be spaced apart and using synchronization markers to maintain scan order output.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If scan order processing is used, then dependency management is simple, but processing efficiency is reduced due to sequential processing requirements
Solution Approach 1:
The patent segments the video frame processing into independent wavefront rows that can be processed in parallel. By dividing the frame into multiple wavefront rows and assigning them to different processing pipelines, the system achieves parallel processing while maintaining dependency constraints within each row.
Solution Approach 2:
The patent introduces a new dimension for processing by using wavefront row ordering instead of traditional scan order. This reorganizes the processing sequence along a different dimensional pattern (wavefront rows) that enables parallel processing of independent rows while maintaining intra-row dependency constraints.
2Productivity
If wavefront ordering is used, then processing parallelism is improved, but synchronization challenges increase due to broken dependencies
Solution Approach 1:
The patent introduces synchronization markers as intermediary elements that mediate between wavefront-ordered processing and scan-order output requirements. These markers act as synchronization points that ensure proper ordering and dependency management while enabling parallel wavefront processing.
Solution Approach 2:
The patent performs preliminary actions by pre-calculating and inserting synchronization markers at appropriate positions in the wavefront-ordered bitstream. This preliminary synchronization preparation ensures that dependency constraints are maintained before the actual parallel processing occurs.
3Productivity
If blocks are spaced apart in wavefront order, then parallel processing capability is enhanced, but output ordering complexity increases to maintain scan order
Solution Approach 1:
The patent applies periodic action by systematically inserting synchronization markers at regular intervals corresponding to wavefront row boundaries. This periodic synchronization pattern simplifies the reordering process by providing predictable synchronization points that facilitate the conversion from wavefront order to scan order.
Solution Approach 2:
The patent uses copying by creating a reordered version of the bitstream that maintains scan order while the original wavefront-ordered processing continues in parallel. The synchronization markers serve as reference copies that guide the reordering process without interfering with the parallel wavefront processing.
Data Source
AI summary
Blocks of pixels from a video frame may be encoded in a block processing pipeline using wavefront ordering, e.g. according to knight's order. Each of the encoded blocks may be written to a particular one of multiple buffers such that the blocks written to each of the buffers represent consecutive blocks of the frame in scan order. Stitching information may be written to the buffers at the end of each row. A stitcher may read the rows from the buffers in order and generate a scan order output stream for the frame. The stitcher component may read the stitching information at the end of each row and apply the stitching information to one or more blocks at the beginning of a next row to stitch the next row to the previous row. Stitching may involve modifying pixel(s) of the blocks and/or modifying metadata for the blocks.