Parallel In-Loop Filtering for Low Latency Video Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video encoding methods using multiple in-loop filters fail to meet the low latency requirements of applications like gaming, as they require completing the encoding of the whole frame before transmitting the encoded bitstream, leading to increased latency.
Innovation Solution
A video encoding method and apparatus that perform encoding and in-loop filtering processes in parallel for different block groups of a frame, allowing for the transmission of bitstreams before the entire frame is encoded, using multiple in-loop filters with overlapping processing segments and constrained pixel distances to ensure efficient low latency video encoding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If multiple in-loop filters are used to enhance image quality, then manufacturing precision is improved, but loss of time increases due to sequential filtering processes
Solution Approach 1:
The patent divides the frame into multiple block groups and assigns different in-loop filters to process different block groups simultaneously. This segmentation allows parallel processing of filtering operations that would otherwise be performed sequentially, reducing total encoding time while maintaining image quality through multiple filtering passes.
Solution Approach 2:
The patent introduces a spatial dimension to the filtering process by processing different spatial regions (block groups) of the image simultaneously with different filters. This transforms the traditionally sequential temporal process into a parallel spatial operation, enabling multiple filters to work concurrently without increasing per-pixel processing time.
2Reliability
If the whole frame is encoded before transmission, then reliability is improved, but loss of time increases due to complete frame encoding requirement
Solution Approach 1:
The patent segments the frame into multiple block groups that can be encoded and transmitted independently. Each block group can be processed and transmitted separately, allowing early transmission of completed blocks while the rest of the frame continues encoding. This maintains data reliability through proper reference management while significantly reducing transmission latency.
Solution Approach 2:
The patent performs preliminary encoding and transmission of certain block groups before the entire frame encoding is complete. By preparing and transmitting available block groups in advance, the system reduces latency for time-sensitive applications while ensuring complete frame encoding occurs for full reliability when needed.
3Loss of time
If parallel encoding and transmission of block groups is implemented, then loss of time is reduced, but device complexity increases due to multiple in-loop filters and overlapping processing
Solution Approach 1:
The patent divides both the frame into block groups and the filtering process into separate filter instances, allowing parallel processing units to work simultaneously on different segments. This segmentation enables pipelined processing where encoding, filtering, and transmission can occur in overlapping time periods across different block groups, reducing latency without requiring a complete redesign of the encoder architecture.
Data Source
AI summary
A video encoding method includes: during a first period, performing an encoding process upon a first block group of a current frame to generate a first block group bitstream; and during a second period, transmitting a second block group bitstream derived from encoding a second block group of the current frame, wherein the second period overlaps the first period. The encoding process includes: during a first time segment of the first period, performing a first in-loop filtering process upon a first group of pixels; and during a second time segment of the first period, performing a second in-loop filtering process upon a second group of pixels, wherein the second time segment overlaps the first time segment, and a non-zero pixel distance exists between a first edge pixel of the first group of pixels and a second edge pixel of the second group of pixels in a filter direction.