Multi-pass GPU Performance Profiling via Range Replay
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for obtaining performance information about code execution on graphics processing units (GPUs) are limited, particularly due to constraints on hardware counters, which restrict the collection of performance metrics and can lead to inaccurate results when handling concurrent kernel dependencies.
Innovation Solution
The implementation of a range replay technique, where a graph of nodes representing GPU kernels is executed multiple times, allowing for the collection of performance metrics by updating hardware counters in each pass, ensuring accurate reflection of concurrent execution and normal control flow.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If hardware counters are used to collect performance metrics on GPU, then performance information can be obtained, but the number of collectable metrics is limited by the number of available hardware counters
Solution Approach 1:
The patent segments the collection of performance metrics into multiple passes or iterations. Instead of attempting to collect all desired metrics simultaneously (which would require more hardware counters than available), the system divides the metric collection process into sequential phases, where different subsets of metrics are collected in different passes over the kernel execution.
Solution Approach 2:
The patent implements periodic re-execution of the kernel with different counter configurations. The system periodically resets and reconfigures the hardware counters between passes, allowing different metric subsets to be collected in each periodic cycle. This periodic action enables comprehensive metric collection over time without requiring all counters to be active simultaneously.
2Loss of information
If kernel execution is replayed multiple times for metric collection, then more performance information can be gathered, but timing consistency and dependency observability must be maintained
Solution Approach 1:
The patent performs preliminary actions to preserve the execution context before each replay. Memory states, register values, and other contextual information are saved prior to kernel re-execution. This preliminary preservation of state ensures that when the kernel is replayed, the timing relationships and dependency observations remain consistent with the original execution, allowing accurate metric collection across multiple passes.
Data Source
AI summary
Apparatuses, systems, and techniques to collect compute performance information. In at least one embodiment, an API is performed to cause two or more portions of at least one software program to be concurrently performed a plurality of times in order to generate one or more performance metrics.