Video Accelerator Adaptive Inter-Prediction Mode Selection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Video accelerators face challenges in processing video data due to limited internal memory and bandwidth constraints, leading to delays in the coding process when handling multiple video streams simultaneously, as they need to allocate bandwidth among streams and wait for sufficient memory bandwidth to read reference data for inter-prediction modes.

Innovation Solution

The video accelerator adaptively selects inter-prediction modes and reduces the size of reference windows based on available bandwidth, enabling modes that use less memory bandwidth and allowing it to proceed with motion compensation without waiting for sufficient bandwidth, by prioritizing modes used for neighboring blocks and combining or reusing reference windows.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If the video accelerator uses inter-prediction modes with large reference windows, then the motion compensation accuracy is improved, but the memory bandwidth consumption increases causing processing delays

Engineering Contradiction:

Improvemotion compensation accuracyVSAvoidprocessing delay

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The video accelerator dynamically adjusts the reference window size based on available memory bandwidth. When bandwidth is sufficient, larger reference windows are used for higher accuracy; when bandwidth is limited, smaller reference windows are selected to avoid processing delays. This dynamic adaptation resolves the contradiction between accuracy and speed.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes the parameter of reference window size adaptively. By modifying this physical dimension parameter based on real-time bandwidth conditions, the system optimizes the trade-off between motion compensation precision and processing throughput, preventing delays while maintaining acceptable accuracy.

Inventive Principle:

Principle #35Parameter changes

2Productivity

If the video accelerator processes multiple video streams simultaneously, then the productivity is improved, but the memory bandwidth per stream decreases causing bottlenecks

Engineering Contradiction:

ImprovethroughputVSAvoidmemory bandwidth per stream

Core Design Contradiction:

ProductivityVSQuantity of substance

Solution Approach 1:

The system segments the processing of multiple video streams by allocating different reference window sizes to different streams based on current bandwidth availability. This segmentation allows high-priority streams to receive larger windows when bandwidth permits, while lower-priority streams use smaller windows, enabling simultaneous processing without bottlenecks.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The video accelerator applies partial action by using reduced reference window sizes for some streams when total bandwidth is limited. This partial application of the full-reference-window approach allows multiple streams to be processed concurrently with acceptable rather than optimal accuracy, maintaining overall productivity.

Inventive Principle:

Principle #16Partial or excessive action

3Reliability

If the video accelerator waits for sufficient memory bandwidth before reading reference data, then the data accuracy is improved, but the processing speed decreases

Engineering Contradiction:

Improvedata accuracyVSAvoidprocessing speed

Core Design Contradiction:

ReliabilityVSSpeed

Solution Approach 1:

The system performs preliminary action by pre-loading reference data into internal memory when bandwidth is available, or by pre-selecting appropriate reference window sizes based on predicted bandwidth conditions. This preliminary preparation reduces the need to wait during actual processing, maintaining both accuracy and speed.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

When memory bandwidth is insufficient, the video accelerator skips the waiting step and proceeds with available bandwidth, using smaller reference windows that can be read immediately. This rushing through the process prevents processing stalls, accepting reduced accuracy temporarily to maintain continuous processing speed.

Inventive Principle:

Principle #21Skipping (Rushing through)

Data Source

PatentUS10798380B1Adaptive use of search modes based on neighboring blocks

Publication Date: 2020.10.06 AMAZON TECH INC

AI summary

Reference data is one type of data that the video accelerator may frequently be read from external memory. In various examples, the video accelerator can adaptively select inter-prediction modes based on the bandwidth to external memory that is available at any point in time. The video accelerator can determine the amount of bandwidth that is available, and when the bandwidth is insufficient for obtaining reference data for all possible inter-prediction modes, the video accelerator can use the encoding parameters of a neighboring block to select an inter-prediction mode to use. The video accelerator can then obtain a reference window for the selected inter-prediction mode, and perform prediction using the inter-prediction mode and the reference window.