Memory Copy Priority Assignment in Parallel Processing Subsystems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for scheduling memory copies in parallel processing subsystems often result in unacceptable latency and throughput due to strict issue-order execution and time-slice constraints, leading to delays and jitter in applications like video decoding and encoding.
Innovation Solution
A method for prioritizing memory copy operations by using semaphores to guide the execution order within copy hardware channels, allowing software applications to strategically allocate resources and override strict issue-order and time-slice constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If memory copies are executed in strict issue-order subject to time-slice constraints, then the scheduling is simple and deterministic, but latency and throughput become unacceptable due to delays in latency-sensitive operations
Solution Approach 1:
The patent changes the scheduling parameter from strict issue-order to priority-based ordering. The host scheduler assigns priorities to memory copy operations and executes them according to priority levels rather than arrival order, allowing latency-sensitive operations to be expedited while maintaining systematic control over the scheduling process
Solution Approach 2:
The scheduling system transitions from static time-slice allocation to dynamic priority-based scheduling. The host scheduler can adaptively adjust the execution order of memory copies based on their priority assignments, allowing the system to respond to varying latency requirements in real-time while maintaining overall system throughput
2Productivity
If multiple memory copies execute simultaneously to improve throughput, then bandwidth utilization increases, but resource contention and scheduling complexity increase
Solution Approach 1:
The patent segments memory copy operations into different priority levels and queues. The host scheduler manages multiple priority queues separately, allowing simultaneous execution of memory copies at different priority levels while maintaining organized control. This segmentation enables throughput improvement through parallel execution while managing complexity through structured queue management
Solution Approach 2:
The host scheduler acts as an intermediary between the copy hardware channels and the copy engines. It receives memory copy requests, assigns priorities, and mediates their execution order, enabling multiple simultaneous operations while centralizing the scheduling logic to manage complexity rather than distributing it across multiple components
3Device complexity
If copy hardware channels are allocated to streams without priority differentiation, then resource allocation is simple, but latency-sensitive applications experience unacceptable delays
Solution Approach 1:
The patent introduces a priority parameter to differentiate memory copy operations. Instead of treating all copies equally, the system assigns priority levels that reflect latency sensitivity. This parameter change enables the system to guarantee latency requirements for critical operations while maintaining simple resource allocation infrastructure
Solution Approach 2:
Different priority treatments are applied to different memory copy operations based on their specific requirements. Latency-sensitive operations receive high priority treatment while less critical operations use standard scheduling. This local differentiation of quality enables reliable latency guarantees for specific operations without complicating the overall allocation mechanism
Data Source
AI summary
One embodiment sets forth a method for guiding the order in which a parallel processing subsystem executes memory copies. A driver creates semaphores for all but the lowest priority included in a plurality of priorities and associates one priority with each copy hardware channel included in the parallel processing subsystem. The driver then aliases prioritized streams to the copy hardware channels based on the priorities. Upon receiving a request to execute a memory copy within one of the streams, the driver inserts commands into the aliased copy hardware channel. These commands use the semaphores to direct the parallel processing subsystem to execute the memory copy based on the priority of the copy hardware channel. Advantageously, by assigning priorities to streams and, subsequently, strategically requesting memory copies within the prioritized streams, an application developer may fine-tune their software application to increase the overall processing efficiency of the software application.