ML-Based Prediction Enhancement Block for Video Compression

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current video compression technologies, such as H.264 or AVC, face limitations in compression efficiency due to complex motion prediction modes, which hinder further improvements in image quality and bit rate efficiency.

Innovation Solution

An image encoder utilizing machine learning techniques to generate enhanced prediction blocks with minimal additional control data, reducing the difference between prediction and source blocks, thereby decreasing residual data size without increasing header data, and employing a rate-distortion optimization method to select between prediction and enhanced prediction blocks.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If machine learning based prediction enhancement is applied to generate enhanced prediction blocks, then compression efficiency is improved by reducing residual data size, but device complexity increases due to the need for ML models and additional processing blocks

Engineering Contradiction:
Improvecompression efficiencyVSAvoidencoder structure complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The patent introduces an intermediary ML-based prediction enhancement block that sits between the conventional predictive block and the residual calculation unit. This intermediary component uses machine learning models to generate enhanced prediction blocks from conventional prediction blocks, thereby improving compression efficiency without fundamentally redesigning the entire encoder architecture. The intermediary nature of this block allows it to leverage existing ML technologies while integrating them into the video coding pipeline.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent changes the prediction block generation process by introducing ML-based parameter transformation. Instead of using only conventional motion compensation parameters, the system transforms prediction blocks through ML models that learn optimal prediction parameters from training data. This parameter change approach allows the system to achieve better compression by adapting prediction parameters based on learned patterns from the video content.

Inventive Principle:
Principle #35Parameter changes

2Measurement precision

If multiple machine learning techniques are applied to transform prediction blocks, then prediction accuracy is improved, but processing time and computational resources increase

Engineering Contradiction:
Improveprediction accuracyVSAvoidencoding processing time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent applies partial action by selectively applying machine learning-based prediction enhancement only to certain prediction blocks or specific coding units where it provides the most benefit. Rather than uniformly applying all available ML techniques to every prediction block, the system can apply enhancement selectively based on content characteristics, thereby improving prediction accuracy where needed while minimizing computational overhead in regions where conventional prediction is sufficient.

Inventive Principle:
Principle #16Partial or excessive action

Solution Approach 2:

The system dynamically selects and switches between different machine learning techniques based on the characteristics of the input video content and prediction blocks. The prediction enhancement block can adapt its behavior by choosing different ML models or transformation methods depending on the current coding context, allowing optimal prediction accuracy to be achieved with minimal computational resources for each specific encoding scenario.

Inventive Principle:
Principle #15Dynamics

3Productivity

If rate-distortion optimization is used to select between prediction blocks and enhanced prediction blocks, then coding efficiency is improved, but complexity of mode selection increases

Engineering Contradiction:
Improvecoding efficiencyVSAvoidmode decision complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The patent implements feedback mechanisms in the rate-distortion optimization process by calculating distortion metrics and rate costs for both conventional and enhanced prediction blocks, then using this feedback information to guide the mode selection decision. The system feeds back the performance metrics of different prediction modes to the mode decision unit, which then selects the optimal mode based on the balance between compression efficiency and computational cost, thereby achieving coding efficiency through informed decision-making.

Inventive Principle:
Principle #23Feedback

Data Source

PatentUS11694125B2Image encoder using machine learning and data processing method of the image encoder
Publication Date: 2023.07.04 SAMSUNG ELECTRONICS CO LTD
  • US11694125B2 patent drawing
  • US11694125B2 patent drawing
  • US11694125B2 patent drawing

AI summary

An image encoder for outputting a bitstream by encoding an input image includes a predictive block, a machine learning based prediction enhancement (MLBE) block, and a subtractor. The predictive block is configured to generate a prediction block using data of a previous input block. The MLBE block is configured to transform the prediction block into an enhanced prediction block by applying a machine learning technique to the prediction block. The subtractor is configured to generate a residual block by subtracting pixel data of the enhanced prediction block from pixel data of a current input block.