Multithreaded Video Decoder Error Concealment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video decoding technologies face challenges in efficiently handling error detection, recovery, and concealment, particularly in multithreaded implementations and when using video acceleration, as they often fail to address these issues satisfactorily, leading to suboptimal performance in decoding high-quality, high-bit rate video.
Innovation Solution
A multithreaded video decoder identifies suitable neighbor pictures for error concealment, modifies syntax elements, and uses dynamic dependency edges to facilitate error concealment, employing additional GPU shaders and changing picture types to handle errors effectively, while also providing error recovery mechanisms for arbitrary bit stream positions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If error concealment is implemented in multithreaded video decoding, then error resilience is improved, but decoding complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-identifying suitable neighbor pictures and establishing task dependency data structures before error concealment is needed. The decoder proactively builds the task dependency graph and identifies candidate reference pictures in advance, so that when errors occur, the concealment process can proceed efficiently using pre-prepared data structures rather than searching for solutions during error recovery.
Solution Approach 2:
The patent introduces task dependency data structures and suitable neighbor pictures as intermediaries between the error detection mechanism and the error concealment execution. These intermediaries facilitate the transfer of control information and coordinate the actions of multiple decoding threads, enabling complex error concealment operations to be managed through structured dependencies rather than direct complex interactions.
2Reliability
If error concealment techniques are added to video decoder, then error recovery capability is improved, but decoding speed decreases
Solution Approach 1:
The patent pre-identifies suitable neighbor pictures and establishes task dependencies before errors occur, so that error concealment can be executed efficiently without adding significant processing overhead during normal decoding operations. The preliminary setup of task dependency data structures enables rapid error recovery while maintaining decoding speed.
Solution Approach 2:
The error concealment mechanism serves itself by automatically selecting suitable neighbor pictures and establishing task dependencies without requiring external intervention or complex control logic during error recovery. The system uses its own decoded pictures and internal task dependency structures to perform concealment, eliminating the need for additional external processing resources.
3Productivity
If multiple threads are used for video decoding, then decoding efficiency is improved, but error handling becomes more difficult
Solution Approach 1:
The patent segments the error handling process into distinct components: error detection in individual decoding threads, centralized task dependency data structure management, and coordinated error concealment execution. Each decoding thread operates independently on its assigned video data while sharing the task dependency structure, allowing parallel processing to continue while errors are handled through structured coordination rather than complex inter-thread communication.
Solution Approach 2:
The task dependency data structure serves as an intermediary that coordinates error handling across multiple decoding threads without requiring direct thread-to-thread communication. The structure centralizes the management of task dependencies and suitable neighbor picture identification, allowing multiple threads to access and update error recovery information in a standardized way, thereby simplifying multi-threaded error handling.
Data Source
AI summary
Error concealment techniques for video decoding are described. For example, a video decoder after finding a corrupted picture in a bit stream, finds a suitable neighbor for the corrupted picture. For example, the video decoder favors pictures with the same parity as the corrupted picture and considers picture order count and picture corruption in choosing a neighbor. The decoder then modifies syntax elements for the encoded video in the bit stream to allow the neighbor to be used in concealing the corruption in the corrupted picture. The modification of syntax elements can depend on the particular video decoder implementation. For example, in a software-only multithreaded video decoder, a task graph is modified, while in a system utilizing video acceleration, syntax elements for reference lists are modified.