Multiple Neural Network Filtering Models Using Deblocking Strength
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies face challenges in effectively filtering distorted decoded pictures, particularly in advanced codecs like ITU-T H.266/Versatile Video Coding (VVC), where traditional filtering methods may not adequately address artifacts and inefficiencies.
Innovation Solution
Implementing neural network filtering units that utilize boundary strength data from deblocking filters to enhance the filtering process, employing multiple neural network models to refine decoded video data, incorporating additional data from various units within the video decoding device to improve filtering accuracy and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional filtering methods are used in advanced video codecs, then device complexity is reduced, but filtering effectiveness and artifact removal capability deteriorate
Solution Approach 1:
The patent replaces traditional mechanical filtering operations with neural network-based filtering. The neural network filtering unit processes decoded video data to remove artifacts more effectively than conventional filters, substituting the mechanical filtering approach with an intelligent system that adapts to different video content characteristics.
Solution Approach 2:
The patent combines multiple filtering approaches by integrating the neural network filtering unit with existing deblocking filters and other video decoding components. This composite filtering system leverages the strengths of both traditional and neural network-based methods to achieve superior artifact removal while managing device complexity.
2Measurement precision
If multiple neural network models are employed for filtering, then filtering accuracy and artifact removal improve, but device complexity and computational requirements increase
Solution Approach 1:
The patent segments the filtering process by dividing the video data into different regions or blocks that require different filtering approaches. The neural network filtering unit can apply different models or processing intensities to different segments of the video data, improving overall filtering accuracy while reducing the computational burden on any single processing unit.
Solution Approach 2:
The patent implements a selective filtering approach where the neural network filtering unit applies processing only to regions of the video data that require it, such as areas with significant artifacts or distortion. This partial action approach improves filtering accuracy where needed while avoiding unnecessary computational overhead in already clean regions.
3Adaptability or versatility
If additional data from multiple units is integrated into the filtering process, then filtering adaptability and effectiveness improve, but data processing complexity increases
Solution Approach 1:
The neural network filtering unit is designed to accept and process multiple types of input data from different video decoding units, including deblocking filter outputs, motion compensation data, and other processing stages. This multi-functional capability allows the filtering unit to adapt to various video content types and decoding scenarios while maintaining a unified processing architecture.
Solution Approach 2:
The patent implements feedback mechanisms where the neural network filtering unit receives data from multiple processing units and uses this information to adapt its filtering behavior. The boundary strength data and other inputs provide feedback about the characteristics of the video data, allowing the neural network to adjust its filtering parameters dynamically for optimal performance.
Data Source
AI summary
An example device for filtering decoded video data includes one or more processors configured to execute a neural network filtering unit to: receive data from one or more other units of the device, the data from the one or more other units of the device being different than data for a decoded picture of video data, and wherein to receive the data from the one or more other units of the device, the one or more processors are configured to execute the neural network filtering unit to receive boundary strength data from a deblocking unit of the device; determine one or more neural network models to be used to filter a portion of the decoded picture; and filter the portion of the decoded picture using the one or more neural network models and the data from the one or more other units of the device, including the boundary strength data.