Affine Merge Candidate Selection for Video Compression

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current video encoding and decoding technologies, particularly in the Affine Merge mode of the Joint Exploration Model (JEM), face inefficiencies due to the limited selection of motion vector predictors, which can lead to suboptimal compression performance as they rely on a single predictor from a restricted set of causal neighboring CUs, neglecting potential better predictors among surrounding CUs.

Innovation Solution

The method involves constructing a set of multiple Affine Merge predictor candidates, evaluating them based on rate distortion criteria, and selecting the best predictor to derive control point motion vectors for improved affine motion compensation, thereby enhancing the compression efficiency of video encoding and decoding.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If a single predictor from a restricted set of causal neighboring CUs is used, then device complexity is reduced, but rate distortion performance deteriorates

Engineering Contradiction:
Improvecomplexity of predictor selectionVSAvoidrate distortion performance
Core Design Contradiction:
Device complexityVSLoss of information

Solution Approach 1:

The patent segments the predictor selection process into multiple independent stages: first identifying a restricted set of causal neighboring CU candidates, then evaluating rate distortion metrics for each candidate, and finally selecting the optimal predictor. This segmentation allows the system to maintain low complexity by limiting the search space while still achieving good rate distortion performance through systematic evaluation of the segmented candidate set.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary action by pre-identifying and restricting the set of candidate predictors to only causal neighboring CUs before the actual predictor selection process. This preliminary restriction of the candidate pool reduces the computational complexity of the subsequent rate distortion evaluation, while still maintaining sufficient candidate diversity to achieve good rate distortion performance.

Inventive Principle:
Principle #10Preliminary action

2Loss of information

If a broader set of predictor candidates is considered, then rate distortion performance is improved, but device complexity increases

Engineering Contradiction:
Improverate distortion performanceVSAvoidcomplexity of predictor selection
Core Design Contradiction:
Loss of informationVSDevice complexity

Solution Approach 1:

The patent applies local quality by considering different types of candidate CUs (spatial neighbors, temporal neighbors, co-located blocks) with different weights and priorities in the predictor selection process. Rather than uniformly evaluating all possible candidates, the system focuses computational resources on locally relevant candidates that are more likely to provide accurate motion prediction, thus improving rate distortion performance without proportionally increasing complexity.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The patent introduces dynamics by adaptively adjusting the set of candidate predictors based on the specific video content and motion characteristics being encoded. The candidate pool is dynamically constructed from available causal neighboring CUs, allowing the system to expand or contract the search space depending on the local motion patterns, thereby achieving good rate distortion performance with variable complexity rather than fixed high complexity.

Inventive Principle:
Principle #15Dynamics

Data Source

PatentUS20230421801A1Multiple merge candidates for affine motion prediction
Publication Date: 2023.12.28 INTERDIGITAL VC HOLDINGS INC
  • US20230421801A1 patent drawing
  • US20230421801A1 patent drawing
  • US20230421801A1 patent drawing

AI summary

Different implementations are described, particularly implementations for selecting a predictor candidate from a set of multiple predictor candidates for motion compensation of a picture block based on a motion model. The motion model, may be, e.g., an affine model in a merge mode for a video content encoder or decoder. In an embodiment, a predictor candidate is selected from the set based on a motion model for each of the multiple predictor candidates, and may be based on a criterion such as, e.g., a rate distortion cost. The corresponding motion field is determined based on, e.g., one or more corresponding control point motion vectors for the block being encoded or decoded. The corresponding motion field of an embodiment to identifies motion vectors used for prediction of sub-blocks of the block being encoded or decoded.