Multi-Pipe Image Decoding Scheduling for Variable Workloads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing resolution and dynamic range of images and videos lead to larger encoded bitstreams, causing inefficiencies and bottlenecks in image decoding due to variability in processing times across multiple decoder pipelines.
Innovation Solution
Implement scheduling circuitry to monitor and adjust the assignment of compressed slices to decoder pipelines based on their processing efficiency, ensuring slices are sent to the pipeline with the least amount of work remaining, thereby optimizing synchronization and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If compressed slices are assigned to multiple decoder pipelines for parallel processing, then productivity is improved, but loss of time increases due to synchronization gaps and latency
Solution Approach 1:
The patent implements dynamic workload assignment that adjusts slice distribution to multiple decoder pipelines based on real-time processing rates. The system monitors syntax element variability and processing speed of each pipeline, then dynamically reassigns slices to balance the workload and minimize synchronization gaps, resolving the contradiction between parallel processing throughput and synchronization latency
Solution Approach 2:
The system incorporates feedback mechanisms that monitor the processing rate of each decoder pipeline and use this information to adjust slice assignment. By continuously tracking syntax element variability and pipeline performance, the system feeds this information back to the scheduler to optimize workload distribution, thereby maintaining high productivity while reducing synchronization delays
2Device complexity
If slice assignment is fixed across decoder pipelines, then device complexity is reduced, but productivity decreases due to bottleneck effects from syntax element variability
Solution Approach 1:
The patent segments the bitstream into multiple compressed slices that can be independently assigned to different decoder pipelines. This segmentation allows the system to distribute variable-syntax workloads across multiple processing units, preventing bottleneck effects while maintaining manageable complexity through modular slice-level assignment rather than requiring complex frame-level coordination
3Adaptability or versatility
If decoder pipelines process variable-length syntax elements, then adaptability is improved, but loss of time increases due to processing time variability
Solution Approach 1:
The system applies local quality optimization by assigning slices with similar syntax element characteristics to the same decoder pipeline. This creates localized processing units that handle specific types of syntax elements more efficiently, reducing processing time variability while maintaining the adaptability to handle diverse compressed content through specialized pipeline assignment
Data Source
AI summary
The present disclosure relates to systems and methods of multi-pipe scheduling for image decoding. For example, a bitstream may include compressed slices that are scheduled to each of the multiple decoding pipelines present in the image processing circuitry of an electronic device. The bitstream may include image data that was encoded using variable-length coding. This results in some bits of the image data containing denser and/or sparser syntax elements and result in variable processing times. The scheduling circuitry may be able to monitor each of the multiple decoder pipelines rate of bit consumption and the number of bits initially scheduled to each of the multiple pipelines and the number of bits consumed by each of the multiple pipelines over time so that incoming compressed slices may be scheduled to the pipe that will be able to process the compressed slice the fastest.