Total Order Queue for Resource Request Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In parallel processing systems, the existing methods for managing common resource requests lead to issues like priority inversion and pipeline latency, where newer requests may preempt older ones, causing older requests to be blocked from completing their operations.
Innovation Solution
A computer-implemented method that schedules common resource access requests by delaying servicing of older requests until the resource becomes available, ensuring that older requests are prioritized over newer ones, and using a total order queue (TOQ) to manage and allocate resources fairly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If newer requests are allocated common resources before older pending requests, then resource utilization appears improved, but priority inversion occurs and older requests are blocked indefinitely
Solution Approach 1:
The patent reserves common resources for older pending requests in advance, before they are actually needed. When a request enters the pending state, the system proactively reserves the required common resources, ensuring they will be available when the request is serviced. This prevents the scenario where newer requests consume resources that older requests need, thereby avoiding priority inversion while maintaining high resource utilization through efficient reservation management.
2Speed
If common resources are allocated to pending requests immediately when available, then resource allocation speed increases, but pipeline latency causes newer requests to consume resources before older requests can complete
Solution Approach 1:
The patent introduces a reservation mechanism as an intermediary between resource availability and actual resource allocation. Instead of directly allocating resources to the first waiting request, the system uses a reservation layer that tracks which older requests are entitled to which resources. This intermediary ensures that even though allocation happens quickly, the correct (older) requests receive resources in the proper order, preventing pipeline latency from causing priority inversion.
3Productivity
If a request waiting for resource B is scheduled when resource A becomes available, then resource A utilization improves, but pipeline latency prevents the request from accessing resource B, causing it to be blocked
Solution Approach 1:
The patent applies preliminary action by reserving both resource A and resource B for the waiting request in advance. When a request is pending for resource B, the system proactively reserves resource B alongside resource A, creating a complete resource reservation package. This ensures that when the request is eventually serviced, both resources are guaranteed to be available, preventing the request from being blocked mid-execution and maintaining smooth operation flow while still utilizing resource A efficiently.
Data Source
AI summary
One embodiment of the present disclosure sets forth an effective way to maintain fairness and order in the scheduling of common resource access requests related to replay operations. Specifically, a streaming multiprocessor (SM) includes a total order queue (TOQ) configured to schedule the access requests over one or more execution cycles. Access requests are allowed to make forward progress when needed common resources have been allocated to the request. Where multiple access requests require the same common resource, priority is given to the older access request. Access requests may be placed in a sleep state pending availability of certain common resources. Deadlock may be avoided by allowing an older access request to steal resources from a younger resource request. One advantage of the disclosed technique is that older common resource access requests are not repeatedly blocked from making forward progress by newer access requests.