NN Post-Filter SEI Metadata for Video Quality and Energy Tradeoffs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern video coding systems face challenges in managing energy consumption, particularly with the introduction of high-resolution and high-dynamic range imaging, which increases energy requirements and contradicts the need for reduced energy consumption in the entertainment industry.
Innovation Solution
Incorporating syntax elements in supplemental enhancement information (SEI) metadata to define energy parameters related to neural network post-processing filters, allowing devices to make informed decisions about applying these filters based on energy criteria.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If neural network post-processing filters are applied to improve video quality, then video quality is improved, but energy consumption increases
Solution Approach 1:
The encoder performs preliminary analysis of the video content and pre-calculates the energy consumption parameters for different neural network post-processing filters. This energy information is embedded in the bitstream as SEI messages, allowing the decoder to make informed decisions about filter application without having to perform complex energy calculations in real-time, thus reducing the decoder's energy burden while maintaining video quality optimization
Solution Approach 2:
Instead of applying neural network post-processing filters to all video content uniformly, the system selectively applies filters only to specific regions or frames where they provide the most benefit. The energy parameter information allows the decoder to determine when partial application is sufficient, avoiding excessive energy consumption while maintaining acceptable video quality
2Manufacturing precision
If high-resolution and high-dynamic range imaging are used to improve video quality, then video quality is improved, but energy consumption increases
Solution Approach 1:
The system applies different processing levels to different regions of the video content based on local characteristics. High-resolution and high-dynamic range processing is applied only where necessary to maintain video quality, while other regions use standard processing. The energy parameter information in the SEI messages guides this selective application, reducing overall energy consumption while maintaining video quality in critical areas
3Manufacturing precision
If neural network post-processing filters are applied, then video quality is improved, but device complexity increases
Solution Approach 1:
The encoder provides feedback to the decoder through SEI messages containing energy parameter information and video content characteristics. This feedback mechanism allows the decoder to select appropriate neural network post-processing filters based on the encoded information, reducing the need for complex real-time analysis at the decoder while still achieving optimal video quality enhancement
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Specific syntax elements define energy characteristics of a neural network post-processing filter (NNPF) used in a video coding system. An encoded video bitstream carries such syntax elements from an encoding device to a decoding device using an extension metadata of a NNPFC-SEI message, thus allowing the encoding device to provide energy consumption information related to the NNPF and allowing the decoding device to handle its energy consumption by deciding whether or not it should apply the NNPF. An encoding method, a decoding method, an encoding apparatus and a decoding apparatus are described.