Binary Instrumentation for GPU Trace Buffer Sizing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional binary instrumentation tools are inefficient for tracing GPU code due to simultaneous execution across multiple execution units, leading to high overhead and difficulty in generating dynamic traces such as control-flow and memory access traces.

Innovation Solution

A method involving three phases: pre-processing to determine the trace buffer size, processing to efficiently collect data from GPU with minimal information, and post-processing to analyze and generate the final trace, with atomic access to the trace buffer and minimal information saved on the GPU side, allowing for quick trace generation with minimal overhead.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If conventional binary instrumentation tools are used for tracing GPU code, then trace information can be collected, but the overhead is high and efficiency is low due to simultaneous execution across multiple execution units

Engineering Contradiction:

Improvetrace generation efficiencyVSAvoidoverhead time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent segments the tracing process into three distinct phases: pre-processing (determining trace buffer size and preparing instrumentation), processing (efficient data collection during GPU execution with minimal overhead), and post-processing (analyzing collected data to generate final traces). This segmentation allows each phase to be optimized independently, reducing overall overhead while maintaining trace generation efficiency.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary actions during the pre-processing phase by determining the required trace buffer size before GPU execution and instrumenting the code in advance. This preliminary preparation eliminates the need for runtime buffer allocation and complex synchronization, significantly reducing overhead during actual trace collection while enabling efficient trace generation.

Inventive Principle:

Principle #10Preliminary action

2Adaptability or versatility

If conventional binary instrumentation is used, then dynamic traces can be generated, but it is difficult to handle simultaneous execution across multiple execution units

Engineering Contradiction:

Improvedynamic tracing capabilityVSAvoidhandling complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent introduces an intermediary trace buffer that serves as a mediator between the GPU's multiple execution units and the analysis system. Each execution unit writes trace data to this shared buffer with minimal coordination, and the post-processing phase analyzes the collected data. This intermediary structure simplifies handling of simultaneous execution by decoupling the complexity of multi-unit coordination from the tracing mechanism itself.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The instrumentation code embedded in the GPU program automatically manages trace data collection during execution, with each execution unit independently writing its trace information to the buffer. This self-service approach eliminates the need for external coordination mechanisms during GPU execution, reducing handling complexity while maintaining full dynamic tracing capability across all execution units.

Inventive Principle:

Principle #25Self-service

3Productivity

If trace buffer is allocated during runtime, then dynamic tracing can be performed, but CPU-GPU synchronization is required which increases overhead

Engineering Contradiction:

Improvetrace collection speedVSAvoidsynchronization complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent performs the trace buffer allocation and size determination in advance during the pre-processing phase, before GPU execution begins. By determining the required buffer size beforehand and allocating it statically, the system eliminates the need for runtime buffer management and CPU-GPU synchronization during trace collection. This preliminary action maintains high trace collection speed while removing synchronization complexity.

Inventive Principle:

Principle #10Preliminary action

4Loss of time

If minimal information is saved on GPU side, then overhead is reduced, but processing must be shifted to CPU side

Engineering Contradiction:

ImproveGPU overhead timeVSAvoidtrace generation speed

Core Design Contradiction:

Loss of timeVSProductivity

Solution Approach 1:

The patent segments the processing workload between GPU and CPU based on optimization criteria: the GPU side performs minimal trace data collection during execution (reducing overhead), while the CPU side performs the intensive analysis and trace generation in the post-processing phase. This segmentation allows each side to operate at optimal speed for its designated tasks, with the CPU's superior processing power handling the computationally intensive analysis without impacting GPU execution performance.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS10949330B2Binary instrumentation to trace graphics processor code

Publication Date: 2021.03.16 INTEL CORP

AI summary

An embodiment of a semiconductor package apparatus may include technology to determine a size for a trace buffer based on instrumented code to be executed on a graphics processor, initialize the trace buffer in a shared memory based on the determined size, provide the instrumented code to the graphics processor to be executed, collect data in the trace buffer from the executed instrumented code, analyze the data collected in the trace buffer on a processor, and generate a trace of the instrumented code on the processor based on the analyzed data. Other embodiments are disclosed and claimed.