Application-Based Rate-Distortion Optimization in Video Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video encoding standards, such as H.264/AVC, HEVC, VP9, AV1, and VVC, rely on human eye limitations for rate-distortion optimization (RDO), which may not effectively address the quality requirements of applications like AI/ML, video conferencing, and facial recognition.
Innovation Solution
The system employs application-based rate-distortion optimization (RDO) by using a weight map generated from a quality metric provided by the application, allowing different blocks within a video frame to be weighted differently based on their importance to the application.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If traditional human eye limitation-based RDO is used in video encoding, then compression efficiency is improved, but application-specific quality requirements cannot be met
Solution Approach 1:
The patent applies local quality by introducing a weight map that assigns different quality weights to different blocks within a video frame. This allows the encoder to prioritize quality in specific regions (e.g., regions of interest for AI/ML applications) while allowing lower quality in other regions, thereby meeting application-specific quality requirements without uniformly increasing compression across the entire frame.
2Loss of information
If uniform quality is maintained across all blocks, then overall image quality is improved, but bandwidth consumption increases
Solution Approach 1:
Instead of maintaining uniform quality across all blocks, the patent uses a weight map to identify and prioritize only the most important blocks for the specific application. This allows the system to concentrate bandwidth on critical regions while reducing or eliminating bandwidth for less important regions, thereby maintaining necessary image quality while reducing overall bandwidth consumption.
Solution Approach 2:
The patent applies partial action by selectively applying high-quality encoding only to specific blocks that are important for the application, rather than encoding all blocks with equal quality. The weight map enables the system to perform encoding actions partially - full quality for important blocks, reduced quality for less important blocks - thereby optimizing the balance between image quality and bandwidth usage.
3Ease of manufacture
If all blocks are encoded with equal importance, then encoding simplicity is maintained, but application-specific requirements cannot be satisfied
Solution Approach 1:
The patent introduces a weight map as an intermediary component that bridges the simple encoding process and application-specific requirements. The weight map serves as a mediator that provides application-specific quality preferences to the encoder without requiring complex changes to the encoding algorithm itself. This allows the existing simple encoding machinery to be adapted to meet diverse application requirements through the weight map interface.
Data Source
AI summary
Systems and methods herein are for a video encoder to be associated with an interface that is to receive, from an application, at least one metric that is associated with a quality preference for video compression to be performed by the video encoder and that is to provide a weight map to enable the video encoder to perform rate-distortion optimization (RDO) for received frames from the application using the weight map to weigh one or more first blocks associated with an individual one of the frames more than one or more second blocks associated with the individual one of the frames.