GPU Execution Graph Cache for Dependency Stall Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Graphics Processing Units (GPUs) experience idle time due to difficulties in determining the proper order of execution for graphics commands with dependencies, leading to wasted cycles and increased latency.
Innovation Solution
A method involving the construction of an execution graph with dependency information and a cache mechanism to prioritize and execute commands based on wait counts, utilizing a high-priority interrupt thread and low-priority background thread to asynchronously manage command execution and reduce dependency stalls.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the GPU firmware determines dependencies between commands and determines whether commands are ready for submission, then the correct execution order is ensured, but the GPU sits idle for significant time
Solution Approach 1:
The system performs preliminary actions by pre-calculating and storing the execution order of graphics commands in an execution graph before the GPU needs to execute them. The high-priority interrupt thread maintains this execution graph in advance, so when the GPU completes a command, the next command is already determined and ready to be submitted, eliminating the idle waiting time while ensuring correct execution order.
2Reliability
If multiple graphics commands depend on each other, then dependency relationships must be resolved, but it becomes difficult to select proper execution order
Solution Approach 1:
The patent introduces an execution graph as an intermediary data structure that explicitly represents dependency relationships between commands. The graph nodes represent commands and edges represent dependencies, making the complex dependency relationships visually and structurally clear. This intermediary structure simplifies the determination of execution order by providing a straightforward traversal path through the dependencies.
3Reliability
If the high-priority interrupt thread waits for the low-priority background thread to update ready commands, then command readiness is accurately determined, but valuable GPU cycles are wasted
Solution Approach 1:
The high-priority interrupt thread performs preliminary actions by maintaining the execution graph independently without waiting for the low-priority background thread. It proactively determines command readiness by traversing the execution graph and checking completion status of parent commands, thus eliminating the need to wait for background thread updates while maintaining accurate readiness determination.
Solution Approach 2:
Instead of the traditional approach where the interrupt thread waits for the background thread to update command readiness, the patent inverts the relationship by having the interrupt thread actively determine readiness itself using the pre-maintained execution graph. This role reversal allows the high-priority thread to operate independently and immediately submit ready commands to the GPU.
Data Source
AI summary
A first command is fetched for execution on a GPU. Dependency information for the first command, which indicates a number of parent commands that the first command depends on, is determined. The first command is inserted into an execution graph based on the dependency information. The execution graph defines an order of execution for plural commands including the first command. The number of parent commands are configured to be executed on the GPU before executing the first command. A wait count for the first command, which indicates the number of parent commands of the first command, is determined based on the execution graph. The first command is inserted into cache memory in response to determining that the wait count for the first command is zero or that each of the number of parent commands the first command depends on has already been inserted into the cache memory.