Total Order Queue for Streaming Multiprocessor Resource Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In parallel processing systems, the existing methods for managing common resource access lead to priority inversions and inefficiencies, where newer requests are serviced before older pending requests, causing older requests to be repeatedly blocked and unable to make forward progress.
Innovation Solution
A computer-implemented method that schedules common resource access requests by delaying newer requests when the resource is not available, ensuring that older requests are serviced first once the resource becomes available, thereby avoiding priority inversions and ensuring older requests can make progress.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If newer requests are allocated common resources before older pending requests, then resource utilization appears improved, but priority inversion occurs and older requests are repeatedly blocked
Solution Approach 1:
The system performs preliminary actions by allocating the common resource to the older pending request in advance, before the newer request can consume it. The scheduler identifies that resource A is available and proactively allocates it to the older request that is waiting, preventing the priority inversion from occurring in the first place.
Solution Approach 2:
The scheduler continuously monitors the state of common resources and the queue of pending requests, using this feedback to make intelligent allocation decisions. When resource A becomes available, the scheduler detects this change and responds by allocating it to the appropriate older request, rather than allowing newer requests to consume it.
2Speed
If common resource is allocated to available requests immediately, then processing speed increases, but older pending requests cannot make forward progress
Solution Approach 1:
The system performs preliminary allocation of common resources to older pending requests when those resources become available, before newer requests can consume them. This ensures older requests make forward progress without excessive waiting time.
Solution Approach 2:
Instead of the conventional approach where newer requests are serviced first, the system inverts the priority order by servicing older pending requests first when resources are available, thereby eliminating priority inversion and reducing waiting time for older requests.
3Productivity
If pending requests are placed in a queue awaiting resource availability, then resource allocation appears efficient, but pipeline latency causes newer requests to consume resources before pending requests can complete
Solution Approach 1:
The scheduler performs preliminary allocation of the common resource to the older pending request as soon as the resource becomes available, rather than waiting for the request to be at the front of the queue. This preliminary action prevents pipeline latency from causing newer requests to consume the resource first.
Solution Approach 2:
The system uses feedback from resource availability status to trigger immediate allocation to older pending requests, rather than following a strict FIFO queue approach that would allow pipeline latency to cause priority inversion.
Data Source
AI summary
One embodiment of the present disclosure sets forth an effective way to maintain fairness and order in the scheduling of common resource access requests related to replay operations. Specifically, a streaming multiprocessor (SM) includes a total order queue (TOQ) configured to schedule the access requests over one or more execution cycles. Access requests are allowed to make forward progress when needed common resources have been allocated to the request. Where multiple access requests require the same common resource, priority is given to the older access request. Access requests may be placed in a sleep state pending availability of certain common resources. Deadlock may be avoided by allowing an older access request to steal resources from a younger resource request. One advantage of the disclosed technique is that older common resource access requests are not repeatedly blocked from making forward progress by newer access requests.