Motion Compensated Temporal Filtering for Video Encoding

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current video coding methods face challenges in reducing bitstream size while maintaining video quality, particularly in efficiently processing video frames before encoding, which affects pre-encoding latency and hardware overhead.

Innovation Solution

The implementation of motion compensated temporal filtering (MCTF) as a pre-encoding processing technique, where filtered pictures are generated by applying discrete wavelet transformation and motion compensation to reference pictures, and these filtered pictures are used for encoding instead of the original target pictures, along with the reference pictures, to create a more efficient bitstream.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of substance

If motion compensated temporal filtering is applied to reduce bitstream size, then coding gain is improved, but pre-encoding latency increases

Engineering Contradiction:

Improvebitstream sizeVSAvoidpre-encoding latency

Core Design Contradiction:

Loss of substanceVSLoss of time

Solution Approach 1:

The video stream is divided into groups of pictures (GOPs), and MCTF is applied selectively to non-key frames within each GOP while key frames are encoded without filtering. This segmentation approach reduces the overall processing latency while still achieving bitstream size reduction through filtering of appropriate frames.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Instead of applying MCTF to all frames, the method applies filtering partially to only those frames that are not key frames. This partial action reduces the computational burden and latency while still achieving significant coding gain from the filtered frames.

Inventive Principle:

Principle #16Partial or excessive action

2Manufacturing precision

If motion compensated temporal filtering is applied to enhance video quality, then coding gain is improved, but hardware overhead increases

Engineering Contradiction:

Improvevideo qualityVSAvoidhardware overhead

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

The MCTF processing is extracted as a separate pre-encoding module that operates independently from the main video encoding pipeline. This allows the filtering functionality to be implemented using software or dedicated processing units without significantly increasing the complexity of the core encoding hardware.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The motion compensation module is designed to serve multiple functions: it performs motion compensation for both the MCTF filtering process and the standard inter-frame prediction in video encoding. This multi-functionality reduces hardware overhead by reusing the same motion compensation infrastructure for different purposes.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS11973985B2Video encoder with motion compensated temporal filtering

Publication Date: 2024.04.30 MEDIATEK INC

AI summary

Various schemes pertaining to pre-encoding processing of a video stream with motion compensated temporal filtering (MCTF) are described. An apparatus determines a filtering interval for a received raw video stream having pictures in a temporal sequence. The apparatus selects from the pictures a plurality of target pictures based on the filtering interval, as well as a group of reference pictures for each target picture to perform pixel-based MCTF, which generates a corresponding filtered picture for each target picture. The apparatus subsequently transmits the filtered pictures as well as non-target pictures to an encoder for encoding the video stream. Subpictures of natural images and screen content images are separately processed by the apparatus.