Method and apparatus for generating graphics processing unit long instruction traces

Abstract

To provide methods, apparatus, systems and articles of manufacture to generate a graphics processing unit (GPU) long instruction trace.SOLUTION: An example apparatus includes at least one memory and at least one processor. The at least one processor executes instructions to: at least identify a first routine based on an identifier of a second routine executed by the GPU, the first routine based on an emulation of the second routine; execute the first routine to determine a first value of a GPU state of the GPU, the first routine having (i) a first argument associated with the second routine and (ii) a second argument corresponding to a second value of the GPU state prior to executing the first routine; and control a workload of the GPU based on the first value of the GPU state.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates generally to computers, and more particularly to a method and apparatus for generating a graphics processing unit long instruction trace. [Background technology]

[0002] Software developers attempt to develop code that executes as efficiently as possible. To better understand code execution, profiling is used to measure different code execution statistics, such as execution time, memory consumption, etc. In some instances, profiling is implemented by inserting profiling instructions into the code. Such profiling instructions can be used to store and analyze information about the code execution. [Brief explanation of the drawings]

[0003] [Figure 1] FIG. 1 is a block diagram of an exemplary graphics processing unit long instruction trace (GLIT) engine inserting profiling instructions into an exemplary graphics processing unit (GPU) kernel executed by an exemplary GPU.

[0004] [Figure 2] 2 is a diagram of an example implementation of an example portion of the GPU of FIG. 1.

[0005] [Figure 3] 1 illustrates an exemplary format for an exemplary long instruction trace.

[0006] [Figure 4] FIG. 2 is a block diagram of an example implementation of the GLIT engine of FIG. 1.

[0007] [Figure 5]3 is a diagram of an example system in which the example GPU of FIG. 1 and / or the example GPU portion of FIG. 2 may be implemented to control the operation of hardware threads of example execution units.

[0008] [Figure 6] 3 is a diagram of an example GPU long instruction trace for the example GPU of FIG. 1 and / or the example GPU portion of FIG. 2.

[0009] [Figure 7] FIG. 7 is a diagram of an exemplary system for generating and analyzing the GLIT of FIG. 6.

[0010] [Figure 8] FIG. 7 is a diagram of an exemplary system for emulating and analyzing the GLIT of FIG. 6.

[0011] [Figure 9] 1 illustrates an exemplary kernel and an exemplary instrumentation kernel.

[0012] [Figure 10] FIG. 10 is a diagram of an example workflow for emulating the execution of the example instrumented kernel of FIG.

[0013] [Figure 11] 10 shows exemplary source code for emulating the execution of the exemplary instrumented kernel of FIG. 9.

[0014] [Figure 12] 1 shows exemplary source code for emulating the execution of an exemplary software thread.

[0015] [Figure 13] 1 shows exemplary source code for emulating the execution of an exemplary instrumented software thread.

[0016] [Figure 14]1 shows exemplary source code for implementing an emulation routine.

[0017] [Figure 15] 10 is a flowchart representing machine-readable instructions that may be executed to implement the GLIT engine of FIG. 1 and / or FIG. 4 to improve the operation of the example GPU of FIG. 1 and / or the example GPU portion of FIG. 2.

[0018] [Figure 16] 5 is a flowchart representing machine-readable instructions that may be executed to implement the GLIT engine of FIG. 1 and / or FIG. 4 to emulate one or more exemplary GLITs.

[0019] [Figure 17] 10 is another flowchart representing machine-readable instructions that may be executed to implement the GLIT engine of FIG. 1 and / or FIG. 4 to improve the operation of the example GPU of FIG. 1 and / or the example GPU portion of FIG. 2.

[0020] [Figure 18] FIG. 11 is a block diagram of an exemplary processing platform configured to execute the machine-readable instructions of FIGS. 11-17 to implement the exemplary GLIT engine of FIGS. 1 and / or 4.

[0021] [Figure 19] FIG. 11 is a block diagram of an exemplary software distribution platform for distributing software (e.g., software corresponding to the exemplary computer-readable instructions of FIGS. 11-17) to client devices, such as consumers (e.g., for licensing, sale, and / or use), retailers (e.g., for sale, resale, licensing, and / or sublicensing), and / or original equipment manufacturers (OEMs) (e.g., for inclusion in products distributed to retailers and / or sold directly to customers). DETAILED DESCRIPTION OF THE INVENTION

[0022] The numbers are not to scale. Generally, the same reference numbers are used throughout the drawings and accompanying written description to refer to the same or similar parts. As used herein, connection references (e.g., attached, coupled, connected, and joined) may include intermediate members between and / or relative movement between the elements referenced by the connection reference, unless otherwise indicated. As such, a connection reference does not necessarily infer that the two elements are directly connected and / or in a fixed relationship to each other.

[0023] Unless otherwise specified, descriptors such as "first," "second," "third," and the like are used solely as labels and / or arbitrary names to distinguish elements to facilitate understanding of the disclosed examples, without implying any sense of priority, physical order, placement within a list, and / or any sequence whatsoever. In some instances, the descriptor "first" may be used to refer to an element in the detailed description, while the same element may be referred to in the claims using a different descriptor, such as "second," "third," etc. In such cases, it should be understood that such descriptors are used solely to distinctively identify those elements that may otherwise share the same name, for example.

[0024] Developers want to create the most computationally efficient machine-readable code to perform desired tasks on a processor, such as a central processing unit (CPU). In some cases, developers create machine-readable code for a CPU and analyze the efficiency of the machine-readable code using a CPU simulator that runs a long instruction trace (LIT). The LIT is a snapshot of the architectural state of the CPU. The architectural state may include the state of system memory, which may include the values ​​of memory registers associated with the CPU. Some such LITs may include a list of system interrupts necessary to simulate system events, such as direct memory access (DMA) traffic. Some LITs include an entire snapshot of system memory in response to the execution of user and / or kernel instructions.

[0025] A developer may develop a CPU kernel and use a profiler and / or profiling system to collect operational statistics (e.g., operational parameters, performance statistics, etc.) of the CPU kernel to gain a better understanding of the efficiency of the CPU kernel executed by the CPU. The profiler inserts additional instructions into the CPU kernel to collect such operational statistics. Such a profiler and / or profiling system may be used to determine CPU utilization. Such a profiler and / or profiling system may be used to determine CPU utilization because an operating system running on the CPU provides visibility into CPU utilization for each of the CPU's cores and threads. A developer may not be able to utilize such LIT and / or profiling techniques for alternative types of processors, such as graphics processing units (GPUs).

[0026] A GPU is an electronic circuit that executes instructions to modify the contents of a buffer. Typically, the buffer is a frame buffer used to output information to a display device (e.g., a monitor, touchscreen, etc.). Recently, GPUs have been used for tasks that do not necessarily involve generating an output image.

[0027] GPUs execute instruction packages commonly referred to as kernels, compute kernels, or shaders. The term kernel is used for general-purpose computational tasks, such as Open Computing Language (OpenCL) tasks and C for media tasks. Typically, the term shader is used when the kernel is used for graphics-related tasks, such as DirectX, Open Graphics Library (OpenGL) tasks, pixel shader / shading tasks, and vertex shader / shading tasks. While the exemplary approach disclosed herein uses the term kernel, such approaches are equally suitable for using shaders. Such kernels roughly correspond to the inner loops of a program that are repeated multiple times. As used herein, a GPU kernel refers to a kernel in binary form. A GPU programmer develops a kernel / shader in a high-level programming language, such as High-Level Shader Language (HLSL) or OpenCL, and then compiles the code into a binary version of the kernel, which is executed by the GPU. The exemplary approach disclosed herein applies to the binary version of the kernel.

[0028] Like CPU developers, GPU developers also desire to create the most computationally efficient machine-readable code to perform desired tasks on a GPU. However, profilers and / or profiling systems may not be efficient enough for GPU developers to analyze their machine-readable code. Unlike CPUs, which have an operating system running on them, GPUs do not have an operating system running on them, and therefore do not have the ability to measure operational statistics, such as busy and idle time intervals and register values ​​in response to kernel execution, at the granularity of the GPU's execution units and hardware threads. While some GPU device vendors provide GPU profiling tools, such tools are limited and inefficient for dynamically applying complex analyses of GPU workloads at the level of each specific GPU instruction without impairing the performance of GPU execution.

[0029] Examples disclosed herein improve GPU profiling, which can be used to identify improvements to GPU operation, by generating and analyzing a GPU long instruction trace (GLIT). In some disclosed examples, the GLIT captures the state of the GPU (e.g., GPU state) in response to the GPU executing an instrumented kernel (e.g., an instrumented GPU kernel). Some examples disclosed herein improve GPU operation by measuring operational parameters of the GPU based on an analysis of the GLIT and determining whether to adjust the operation of the GPU based on the measured operational parameters. In some disclosed examples, a processor such as a CPU can determine one or more operational parameters (e.g., operational statistics, performance statistics, etc.) associated with the GPU, including at least one of a GPU state, an execution time parameter, a busy time parameter, an idle time parameter, an occupied time parameter, or a utilization parameter based on the GLIT.

[0030] As used herein, an instrumented kernel refers to a kernel that includes profiling and / or tracing instructions executed by hardware that, when executed, statistically measure and / or monitor the execution of the kernel. As used herein, a GPU state refers to one or more first values ​​stored in a general-purpose register file (GRF) and / or one or more second values ​​stored in an architectural register file (ARF) associated with a hardware thread of the GPU. For example, a GPU may have a hardware thread having a GRF that includes a plurality of first registers and an ARF that includes a plurality of second registers. In such an example, a first value in a first one of the GRF registers may be a first GPU state, a first value in a first one of the ARF registers may be a second GPU state, etc.

[0031] As used herein, execution time of a GPU refers to a time interval, time duration, etc., that a hardware thread of a GPU, and / or more generally, a GPU uses to execute a kernel (e.g., an instrumented kernel). As used herein, busy time of a GPU refers to a time interval, time duration, etc., when a hardware thread of a GPU is busy executing a computational task. Idle time of a GPU, as used herein, refers to a time interval, time duration, etc., when a hardware thread of a GPU is not executing a computational task. GPU occupancy, as used herein, refers to a set of busy and / or idle time intervals associated with an execution unit and / or hardware thread of a GPU during the execution of one or more computational tasks. Utilization of a GPU, as used herein, refers to the proportion of busy time and total time associated with the execution of one or more computational tasks.

[0032] In some disclosed examples, the CPU inserts additional instructions into the kernel to collect information corresponding to one or more operating parameters associated with the execution of the kernel. The additional instructions include profiling instructions to instruct the GPU to generate a GLIT, which may include a hardware thread identifier (TID), a GPU state of the hardware thread, an opcode for identifying the GPU instruction, a type of GPU instruction (e.g., a "read SEND" or an EOT (End-of-Thread) instruction), timestamps associated with the start and / or end of execution of the kernel, and / or combinations thereof. For example, when the GPU executes a kernel including the additional instructions, the GPU may store (i) a first value of a GRF register before executing the kernel, (ii) a second value of the GRF register after executing the kernel in the GLIT, and / or (iii) a hardware thread identifier corresponding to the hardware thread that executed the kernel. The GPU may store the GLIT in a trace buffer in memory.

[0033] In some disclosed examples, the CPU can retrieve the GLIT from the trace buffer and replay the GLIT for GPU analysis. For example, the CPU can emulate execution of a kernel based on the first and / or second values ​​of the GRF register. In some examples, the CPU can register a callback routine (e.g., with a software application, an operating system (OS), and / or a combination thereof) to provide output data from the emulated execution of the kernel to a GPU profiling tool to determine one or more operating parameters associated with the GPU. Advantageously, the GPU profiling tool may be utilized to determine the efficiency of kernels executed by the GPU. For example, the GPU profiling tool may determine that the GPU can perform additional computational tasks, fewer additional computational tasks, etc. based on the one or more operating parameters, and thus identify improvements to the kernel and / or, more generally, the scheduling of GPU operation, CPU operation, etc.

[0034] 1 is a block diagram illustrating an example system 100 including an example GPU long instruction trace (GLIT) engine 102 that inserts example profiling instructions 104A-104C into a first example kernel 106 to generate a second example kernel 108. In this example, the first kernel 106 is a GPU kernel executed by an example GPU 110. In this example, the second kernel 108 is an instrumented kernel (e.g., an instrumented GPU kernel). Alternatively, the first kernel 106 may be another type of kernel, such as a kernel executed by a neural network processor, a visual processing unit (VPU), etc.

[0035] The GPU 110 may be implemented with multiple execution units arranged into slices (e.g., GPU slices). For example, the GPU 110 may be implemented with multiple slices (e.g., 3 slices, 6 slices, 12 slices, etc.). An exemplary implementation of a GPU slice 200 is shown in the exemplary example of FIG. 2. Referring to FIG. 2, the GPU slice 200 includes three exemplary sub-slices 202 and 24 exemplary execution units 204. In this example, each of the sub-slices 202 includes eight execution units 204. The execution units 204 are independent computational units used to execute three-dimensional (3D) shader, media, and general-purpose graphics processing unit (GPGPU) kernels. For example, the execution units 204 may be implemented with multi-threaded hardware capable of performing SIMD (multi-issue single instruction, multiple data) operations. In this example, each of the execution units 204 may be implemented with seven exemplary threads (e.g., hardware threads, GPU threads, etc.) 206.

[0036] 2, GPU slice 200 includes an exemplary fixed function unit 207 that communicates with one or more of sub-slices 202. Fixed function unit 207 may be implemented by hardware that is partially and / or otherwise not fully programmable (e.g., by a user, an application, etc.). Alternatively, GPU slice 200 may not include fixed function unit 207. For example, fixed function unit 207 is emulated and / or otherwise implemented by a programmable shader.

[0037] 2, GPU slice 200 includes an exemplary cache memory 210. In this example, cache memory 210 is implemented by a level 3 (L3) data cache that includes an exemplary atomic barrier 212 and an exemplary shared local memory 214. Alternatively, cache memory 210 may be implemented with any other type of memory, data storage, etc.

[0038] In the depicted example of FIG. 2 , one(s) of the sub-slices 202 communicate with the cache memory 210 via at least one of an exemplary sampler (e.g., texture sampler) 216 or an exemplary data port 218. In some examples, the sampler 216 may be implemented as a self-contained functional block (e.g., a hardware, firmware, and / or software block) within the graphics core. In some examples, the sampler 216 may receive messages from other agents within the graphics core, fetch data from external memory sources, sometimes referred to as “surfaces,” perform operations on the data, and / or return results in a standard format to the requester (or, if requested, to an intermediate memory buffer (e.g., directly to a Render Target Texture (RTT))). In some examples, the sampler 216 may return filtered / blended pixels from a location in a texture map.

[0039] In this example, the sampler 216 and / or the data port 218 can read data from the cache memory 210 at a rate of 64 bytes per cycle. For example, the sampler 216 can sample the GPU state of one or more of the threads 208 of the execution unit 204 by reading a value from a first register of a corresponding ARF and / or a second register of a corresponding GRF implemented by one or more of the threads 208. Alternatively, the sampler 216 and / or the data port 218 can read data from the cache memory 210 at any other rate. In this example, the data port 218 can write data to the cache memory 210 at a rate of 64 bytes per cycle. Alternatively, the data port 218 can write data to the cache memory 210 at any other rate.

[0040] 2, one or more of the execution units 204 communicate with an exemplary local thread dispatcher 220. In this example, the local thread dispatcher 220 may be implemented using hardware that retrieves instructions, such as from the second kernel 108 of FIG. 1, and stores the instructions in an exemplary instruction cache 222. For example, the instruction cache 222 may be implemented using memory (e.g., non-volatile memory, volatile memory, etc.) that can store instructions.

[0041] In this example, the local thread dispatcher 220 may dispatch, distribute, and / or otherwise send instructions, such as the second kernel 108, to one or more of the execution units 204 for execution. For example, the local thread dispatcher 220 may spread instances of the second kernel 108 across available one or more of the execution units 204 for execution. In some examples, hundreds or thousands of instances of the second kernel 108 may run in parallel and / or otherwise execute on available one or more of the execution units 204, with each one(s) of the execution units 204 processing a subset or portion of the data intended by an application, such as the application 120 of FIG. 1 . As used herein, a “job” or a “software thread” may refer to an instance of dispatching the second kernel 108 to one of the threads 208 and / or more generally to one of the execution units 204.

[0042] In the illustrated example of FIG. 2, one or more of the execution units 204 receives and / or otherwise obtains an instruction to be executed (e.g., a kernel) from an exemplary instruction fetch interface 224. For example, one or more of the execution units 204 may obtain a kernel, such as the second kernel 108 of FIG. 1, for execution from the instruction fetch interface 224. The instruction fetch interface 224 may assign the kernel to one or more of the threads 208 of the execution unit 204. In this example, one or more of the threads 208 may each be implemented with 128 32-byte registers. For example, one or more of the threads 208 may each have an exemplary general-purpose register file (GRF) and an exemplary architectural register file (ARF). Data read or written by a thread 208 may be stored in the corresponding one of the threads 208. In this example, the GRF may be implemented with 128 general-purpose registers, with one or more of the general-purpose registers each storing 32 bytes. A data element address within a GRF may be indicated by a register number (eg, r0 to r127 for a 128 general-purpose register GRF) and a sub-register number.

[0043] In the illustrated example of FIG. 2, the ARF may be implemented using a register file containing registers used to implement specific Instruction Set Architecture (ISA) functions. For example, an instruction pointer and / or condition flags may be implemented using ARF registers. As used herein, "ISA function" refers to aspects of a processor that are visible to programs and programmers (e.g., developers) and independent of a specific implementation, including data types, registers, memory access, addressing modes, exceptions, instruction encodings, and the instruction set itself. In some examples, a hardware thread of a GPU, such as one or more of threads 208, can execute instructions corresponding to the ISA function. In some examples, each instruction may be a vector instruction that can operate in different SIMD modes with different floating-point and integer data types. In some examples, each of the instructions may have a corresponding opcode. For example, a GPU architecture may support a limited number of opcodes (e.g., 60 opcodes, 80 opcodes, 100 opcodes, etc.).

[0044] 2, one or more of the threads 208 may be in communication with an exemplary thread arbiter 226. In this example, the thread arbiter 226 may be implemented in hardware that obtains data output(s) from the threads 208 and determines whether the data output(s) correspond to an exemplary SEND instruction 228, a branch instruction 230, or an exemplary SIMD floating-point unit (FPU) instruction 232. In this example, the SEND instruction 228 may be generated by the thread 208 in response to the thread 208 finishing execution of a kernel. In this example, the branch instruction 230 may be generated by the thread 208 in response to executing a kernel that includes a conditional instruction, such as an “if,” “do,” or “while” instruction. In this example, the FPU instruction 232 may be generated by the thread 208 in response to the thread performing a floating-point calculation.

[0045] Returning to the illustrated example of FIG. 1 , the GPU 110 may execute profiling instructions 104A-104C to generate an exemplary GLIT 112. In this example, the GPU 110 stores the GLIT 112 in an exemplary trace buffer 114. In this example, the trace buffer 114 is stored in an exemplary memory 116. The GLIT 112 includes GLIT data generated and / or otherwise output by the GPU 110 in response to executing profiling instructions 104A-104C included in the second kernel 108 in response to being configured by the GLIT engine 102 to generate the GLIT data, etc. For example, the GLIT 112 may include GLIT data that implements and / or otherwise stores a snapshot of the architectural state of the GPU 110. In some examples, the architectural state of the GPU 110 may include a first value stored in a GRF and / or a second value stored in an ARF associated with a hardware thread, such as thread 208 of FIG. 2 , of the GPU 110. In some examples, the GLIT 112 stores data associated with one or more of the SEND instruction 228, the branch instruction 230, or the SIMD FPU instruction 232 of Figure 2, and / or corresponding timestamps. The GLIT engine 102 can obtain and analyze the GLIT 112 to better understand the execution of the second kernel 108 by the GPU 110. The GLIT engine 102 can determine to adjust the operation of the GPU 110 based on the analysis of the GLIT 112.

[0046] In some examples, the profiling instructions 104A-104C are profile routines (e.g., machine-readable code and / or software profile routines, etc.) that, when executed by the GPU 110, generate, determine, and / or store operational information, such as counters, hardware thread identifiers, register values, timestamps, etc., that can be used to better understand the execution of the second kernel 108. For example, the profiling instructions 104A-104C may profile and / or otherwise characterize the execution of the second kernel 108 by the GPU 110.

[0047] In some examples, the profiling instructions 104A-104C are inserted at a first address (e.g., a first location) of a kernel (e.g., at the beginning of the first kernel 106) to initialize variables used for profiling. In some examples, the profiling instructions 104A-104C are inserted at an intermediate location of the original instructions (e.g., between instructions of the first kernel 106). In some examples, the profiling instructions 104A-104C are inserted at a second address (e.g., a second location) of the kernel (e.g., after instructions from the first kernel 106) and, when executed, cause the GPU 110 to collect and / or store metrics accessible by the GLIT engine 102. In some examples, the profiling instructions 104A-104C are inserted at the end of a kernel (e.g., the first kernel 106) to perform cleanup (e.g., freeing memory locations, etc.). However, such profiling instructions 104A-104C may additionally or alternatively be inserted in any place or position and in any order.

[0048] In the depicted example of FIG. 1 , the exemplary CPU 118 includes and / or otherwise implements the GLIT engine 102, an exemplary application 120, an exemplary GPU driver 122, and an exemplary GPU compiler 124. The application 120 is a software application that may be used to display output from the GPU 110 on one or more display devices when the GPU 110 performs graphics-related tasks, such as, for example, DirectX tasks, OpenGL tasks, pixel shader / shading tasks, vertex shader / shading tasks, etc. In some examples, the application 120 may be implemented in one or more dynamic link libraries (DLLs). Additionally or alternatively, the application 120 may be used to display and / or otherwise process output from the GPU 110 when the GPU 110 performs non-graphics-related tasks. Additionally or alternatively, the application 120 may be used by a GPU programmer to facilitate the development of kernels / shaders in a high-level programming language, such as HLSL, OpenCL, etc. For example, the application 120 may be a profiling tool, such as a GPU profiling tool, a GPU analysis tool, or the like.

[0049] 1 , application 120 sends a task (e.g., a computational task, a graphics-related task, a non-graphics-related task, etc.) to GPU driver 122. In some examples, GPU driver 122 receives the task and instructs GPU compiler 124 to compile code associated with the task into a binary version (e.g., a binary format corresponding to the binary code, binary instructions, machine-readable instructions, etc.) to generate first kernel 106. GPU compiler 124 sends the compiled binary version of first kernel 106 to GPU driver 122.

[0050] In some examples, the GLIT engine 102 configures, programs, and / or otherwise controls the GPU 110 to output data to the trace buffer 114. For example, the GLIT engine 102 may instruct the GPU driver 122 to control the GPU 110 to dump and / or output data and / or information described below in FIG. 3 at a particular execution point of a kernel, such as the first kernel 106 or the second kernel 108. In some examples, the GLIT engine 102 may instruct the GPU driver 122 to cause the GPU 110 to output data associated with instructions executed by the GPU 110 to the trace buffer 114. For example, the GLIT engine 102 may cause the GPU 110 to output data associated with GPU instructions (e.g., instructions included in the first kernel 106, the second kernel 108, etc.), device access instructions (e.g., memory access instructions, instructions executed by the GPU 110 that cause the GPU 110 to access the sampler 216 of FIG. 2, the cache memory 210, etc.), etc.

[0051] In some examples, in response to GPU 110 executing a GPU instruction (e.g., an add instruction, a move instruction, etc.), GPU 110 may output the GPU instruction, a first value of the register before executing the GPU instruction, a second value of the register after executing the GPU instruction, etc. to trace buffer 114. In some examples, in response to GPU 110 executing a device access instruction that causes GPU 110 to send a register value to sampler 216, GPU 110 may output the device access instruction, the register value, etc. to trace buffer 114. Advantageously, in some such examples, GLIT engine 102 can control GPU 110 to output GLIT data to trace buffer 114 without instrumenting a kernel.

[0052] In some examples, the GLIT engine 102 may control the GPU 110 to output GLIT data to the trace buffer 114 via binary instrumentation. For example, the GLIT engine 102 may obtain a first kernel 106 (e.g., in binary format) from the GPU driver 122. The GLIT engine 102 may instrument the first kernel 106 by inserting additional instructions, such as profiling instructions 104A-104C, into the first kernel 106. For example, the GLIT engine 102 may modify the first kernel 106 to generate an instrumented GPU kernel, such as the second kernel 108. That is, the GLIT engine 102 generates the second kernel 108 without performing any compilation of the first kernel 106. In this manner, an already compiled GPU kernel can be instrumented and / or profiled. The second kernel 108 is passed to the GPU 110 via the memory 116. For example, the GLIT engine 102 can send the second kernel 108 to the GPU driver 122, which can store the second kernel 108 in the memory 116 for retrieval by the GPU 110.

[0053] In some examples, the GPU 110 executes profiling instructions 104A-104C to generate one or more GLITs 112. In this example, the profiling instructions 104A-104C include an exemplary first profiling instruction 104 of “TRACE(0, TID)” inserted at a first position, where the first profiling instruction 104A corresponds to generating a trace (e.g., one of the GLITs 112). For example, a trace may refer to a sequence of data records written (e.g., dynamically written) to a memory buffer such as the trace buffer 114. In some examples, the first tracing operation may be implemented using a read operation of a register (e.g., a hardware register) associated with a hardware thread and a store operation of the first value read from the register in a first variable. In such examples, the first tracing operation may be implemented by generating a first one of the GLITs 112 that includes (i) the first value and / or (ii) a thread identifier (TID) associated with the hardware thread that accessed the register.

[0054] In the illustrated example of FIG. 1 , the profiling instructions 104A-104C include an exemplary second profiling instruction 104B of “TRACE(1, TID)” inserted at a second position, where the second profiling instruction 104B corresponds to a second trace operation. In some examples, the second trace operation may be implemented using a read operation of a register associated with a hardware thread and a store operation of a second value read from the register in a second variable. For example, the second value may be generated in response to the GPU 110 executing the second kernel 108 and may therefore differ from the first value of the first trace operation. In such examples, the second trace operation may be implemented by generating a second GLIT 112 that includes (i) the second value and / or (ii) a TID associated with the hardware thread that accessed the register.

[0055] In the illustrated example of FIG. 1 , the profiling instructions 104A-104C include an exemplary third profiling instruction 104C of “TRACE(2, TID)” inserted at a third position, where the third profiling instruction 104C corresponds to a third trace operation. In some examples, the third trace operation may be implemented using a read operation of a register associated with a hardware thread and a store operation of a third value read from the register in a third variable. For example, the third value may be generated in response to the GPU 110 executing the second kernel 108 and may therefore differ from the first value of the first trace operation and / or the second value of the second trace operation. In such examples, the third trace operation may be implemented by generating a third GLIT 112 that includes (i) the third value and / or (ii) a TID associated with the hardware thread that accessed the register.

[0056] In some examples, in response to executing the profiling instructions 104A-104C, and / or more generally, the second kernel 108, the GPU 110 stores the GLIT 112 in the trace buffer 114. The trace buffer 114 includes exemplary records (e.g., data records) 126 that may implement the GLIT 112. For example, the records 126 may implement GLIT data from the GPU 110. In some examples, the records 126, and / or more generally, the GLIT 112, may be encoded in a binary format based on the exemplary GLIT format 300 shown in the illustrated example of FIG.

[0057] 3, GLIT format 300 is shown in plain text and may represent and / or otherwise correspond to an exemplary binary data format that may implement and / or otherwise correspond to one or more of GLITs 112 of FIG. 1. For example, GLIT format 300 may be used to implement an exemplary binary file (e.g., an encoded binary file) that GPU 110 may use to store GLIT 112. Alternatively, GLIT format 300 may be implemented using any other format.

[0058] 1 may retrieve records 126 from trace buffer 114. In such examples, CPU 118 may generate one of GLITs 112 to include one or more of records 126 based on GLIT format 300. In some examples, GLIT format 300 may be implemented as a buffer in an encoded binary format that includes multiple exemplary records (e.g., data records) 302. For example, record 302 may implement record 126 of FIG. 1. In such examples, a first one of records 302 may correspond to a first one of records 126 of FIG. 1.

[0059] In some examples, the GLIT format 300 may be generated in an atomic manner. For example, the GPU 110 may sequentially generate the GLIT 112 in the GLIT format 300, where a first one of the records 302 is adjacent to a second one of the records 302 and where the first one of the records 302 is generated before the second one of the records 302. Alternatively, the GLIT(s) 112 comprising the GLIT format 300 may be generated in a manner other than atomic, for example, using a round-robin technique. The GPU 110 may generate the records 302 from multiple hardware threads, such as the threads 208 of FIG. 2.

[0060] 3, the GLIT format 300 includes data records 302 of an administrative nature, such as a format version (VERSION) of the GLIT format 300, a GEN model identifier (GEN MODEL ID), etc. For example, the GEN MODEL ID may refer to a particular architecture of the GPU 110. In some examples, the CPU 118 may determine the behavior, specifications, etc. of the GPU 110 based on the GEN MODEL ID.

[0061] In the depicted example of FIG. 3, the GLIT format 300 includes decoded information for instructions of a kernel, such as the second kernel 108 of FIG. 1. For example, INST_DECODE_T INST0 may correspond to a decoded version of a first kernel instruction, such as INSTR1 DST, SRC1, SRC2 of FIG. 1 for the second kernel 108. In some examples, INST_DECODE_T INST1 may correspond to a decoded version of a second kernel instruction, such as INSTR2 DST, SRC1, SRC2 of FIG. 1 for the second kernel 108. In some examples, the decoded kernel instructions may implement decoded GLIT data that the GLIT engine 102 may use to emulate and / or otherwise simulate execution of the instructions of the second kernel 108 by the GPU 110.

[0062] In the illustrated example of FIG. 3 , the GLIT format 300 includes exemplary operational parameters such as a number of instructions (NUMBER OF INSTRUCTIONS) (e.g., the number of instructions of the second kernel 108), a number of relevant basic blocks (BBLs) (NUMBER OF RELEVANT BBLs), a number of SEND instructions (NUM OF SENDS) (e.g., the number of SEND instructions 228 of FIG. 2 ), data associated with each of the SEND instructions (e.g., SEND0 DATA, SEND1 DATA, etc.), a maximum number of hardware threads (MAX NUM OF HW THREADS) (e.g., the maximum number of threads 208 of FIG. 2 ), and a hardware thread identifier count (HW TID COUNT). For example, a BBL can refer to a contiguous set of instructions with a single entry and exit point. In such an example, a kernel such as the second kernel 108 may be logically divided into one or more BBLs. Additionally or alternatively, the GLIT 300 may include operational parameters corresponding to different types of instructions, such as load instructions. For example, NUM OF SENDS may be replaced with the number of load instructions (NUM OF LOADS), SEND0 DATA may be replaced with LOAD0 DATA, SEND0 DESTINATION VALUES may be replaced with LOAD0 DESTINATION VALUE, etc., and / or combinations thereof.

[0063] In some examples, the GLIT format 300 may be implemented to store data associated with a device access instruction, such as a SEND instruction, a READ SEND instruction, etc. For example, the GLIT format 300 may include an offset value (OFFSET), a destination register (DST), a number of registers (NUM OF REG), etc. In some examples, the GLIT format 300 may be implemented to include header data (e.g., CE, DMASK, CR0.0, etc.) associated with device access instruction data (e.g., SEND destination value data, SEND0 DESTINATION VALUES, SEND1 DESTINATION VALUES, etc.), which may include a value of a first register (e.g., a CE register) of an ARF associated with the GPU 110, a value of a second register (e.g., a dispatch mask (DMASK) register), etc. Additionally or alternatively, there may be fewer or more records than the records 302 shown in FIG. 3 . Advantageously, the GLIT engine 102 may obtain the GLIT 112 of FIG. 1 based on and / or otherwise comprising the GLIT format 300, which may be used to improve profiling of the GPU 110.

[0064] In the illustrated example of FIG. 3, a GLIT based on GLIT format 300 can store data associated with multiple hardware threads, such as threads 208 of FIG. 2. For example, one or more of GLITs 112 based on GLIT format 300 may store first data corresponding to a first one of threads 208, second data corresponding to a second one of threads 208, etc. In this example, the first data may correspond to the NUM OF BBL RECORDS, BBL ID, HEADER, SEND0 DESTINATION VALUES, SEND1 DESTINATION VALUES, etc. corresponding to a first one of threads 208 having an identifier of TID 0. In this example, the second data may correspond to the NUM OF BBL RECORDS, BBL ID, HEADER, SEND0 DESTINATION VALUES, SEND1 DESTINATION VALUES, etc. corresponding to a second one of threads 208 having an identifier of TID 1. In this example, GLIT format 300 may sequentially list the first data, the second data, etc. Alternatively, the GLIT format 300 may list the first data, the second data, etc. in any order and / or format.

[0065] 1 , the GLIT engine 102 retrieves (e.g., repeatedly retrieves, periodically retrieves, etc.) the trace buffer 114 from the memory 116. In some examples, the GLIT engine 102 determines one or more operational parameters associated with the second kernel 108 and / or, more generally, the GPU 110. For example, the GLIT engine 102 may determine a GPU state, a run-time parameter, a busy time parameter, an idle time parameter, an occupied time parameter, and / or a utilization parameter associated with the GPU 110. In some examples, the GLIT engine 102 adjusts the operation of the GPU 110 based on the one or more operational parameters. For example, the GLIT engine 102 can instruct the CPU 118 to schedule an increase in the amount of instructions executed by the GPU 110, a decrease in the amount of instructions executed by the GPU 110, etc., based on the one or more operational parameters.

[0066] 1 , memory 116 includes one or more kernels, such as second kernel 108, trace buffer 114, and exemplary GPU data 128. Alternatively, memory 116 may not store one or more kernels. In some examples, memory 116 may be implemented by volatile memory, non-volatile memory (e.g., flash memory), etc., and / or a combination thereof. In some examples, GPU data 128 corresponds to data generated by GPU 110 in response to executing at least second kernel 108. For example, GPU data 128 may include graphics-related data, output information to a display device, etc.

[0067] 4 is a block diagram of an example implementation of the GLIT engine 102 of FIG. 1 to improve the operation of the GPU 110 of FIG. 1. In some examples, the GLIT engine 102 instruments binary shaders / kernels before sending them to the GPU 110. The GLIT engine 102 can collect the GLIT 112 of FIG. 1, which may be based on the GLIT format 300 of FIG. 3, from the memory 116 of FIG. 1. The GLIT engine 102 can emulate the operation of the GPU 110 based on the records 126 stored in the GLIT 112. The GLIT engine 102 can determine operating parameters associated with the GPU 110, which can be used to determine improvements in the operation of the GPU 110, the CPU 118, etc.

[0068] 4, the GLIT engine 102 includes an example instruction generator 410, an example trace extractor 420, an example trace emulator 430, an example trace analyzer 440, an example hardware configurer 450, and an example storage 460. In this example, the storage 460 includes and / or otherwise stores an example GLIT 470. In this example, at least one of the instruction generator 410, the trace extractor 420, the trace emulator 430, the trace analyzer 440, the hardware configurer 450, and the storage 460 may be in communication with one or more of each other via an example bus 480. For example, the bus 480 may be implemented by an Inter-Integrated Circuit (I2C) bus, a Serial Peripheral Interface (SPI) bus, and / or a Peripheral Component Interconnect (PCI) bus.

[0069] In the illustrated example of FIG. 4, the GLIT engine 102 includes an instruction generator 410 that instruments a kernel, such as the first kernel 106 of FIG. 1. For example, the instruction generator 410 may access the first kernel 106 (e.g., access the first kernel 106 from a memory included in the CPU 118). The instruction generator 410 may instrument the first kernel 106 to generate the second kernel 108 of FIG. 1. For example, the instruction generator 410 may generate and insert binary code associated with the profiling instructions 104A-104C of FIG. 1 into the first kernel 106 to generate the second kernel 108. In some examples, the instruction generator 410 provides and / or otherwise transmits the second kernel 108 to the GPU driver 122 of FIG. 1. In such an example, in response to obtaining the second kernel 108 from the instruction generator 410, the GPU driver 122 may store the second kernel 108 in the memory 116 for later retrieval by the GPU 110.

[0070] In some examples, the instruction generator 410 implements means for inserting one or more profiling routines, such as one or more of the profiling instructions 104A-104C, into a kernel executed by one of the threads 208 of the GPU 110. In some examples, the means for inserting may be implemented by one or more analog or digital circuits, logic circuits, programmable processors, programmable controllers, GPUs, digital signal processors (DSPs), application specific integrated circuits (ASICs), programmable logic devices (PLDs), and / or field programmable logic devices (FPLDs) (e.g., field programmable gate arrays (FPGAs)). In some examples, the means for inserting may be implemented by at least one of block 1602 of FIG. 16 or block 1802 of FIG. 18.

[0071] In some examples, the instruction generator 410 implements means for generating binary code (e.g., binary instructions, machine-readable instructions, etc.) based on the profiling instructions 104A-104C. In some examples, the instruction generator 410 implements means for inserting the generated binary code into the first kernel 106 at one or more places or locations within the first kernel 106 to generate the second kernel 108.

[0072] In the illustrated example of FIG. 4, the GLIT engine 102 includes a trace extractor 420 that pulls and / or collects the GLIT 112 from the memory 116 of FIG. 1 and / or, more generally, the trace buffer 114. In some examples, the trace extractor 420 extracts the GLIT 112 from the trace buffer 114 and / or extracts records 126 from the GLIT 112. In some examples, the trace extractor 420 processes the GLIT 112 by traversing the GLIT 112 from a first position (e.g., a start) of the GLIT format 300 to a second position (e.g., an end) of the GLIT format 300, extracting records 126 along the way. For example, the trace extractor 420 may extract, identify, and / or otherwise determine a first one of the records 302 of FIG. 3, a second one of the records 302 of FIG. 3, etc., from the GLIT format 300 of FIG. 3.

[0073] In some examples, the trace extractor 320 extracts the records 126 from the GLIT 112 by decoding the binary kernel representation of the GLIT 112 to generate decoded binary data. In some examples, the trace extractor 320 extracts instruction identifiers and / or opcodes from the decoded binary data. For example, the trace extractor 320 may extract a SEND instruction, a READ SEND instruction, a branch instruction, and a first opcode corresponding to the SEND instruction, a second opcode corresponding to the branch instruction, etc., executed by the GPU 110. In some examples, the trace extractor 320 classifies and / or otherwise groups one or more of the records 126 based on at least one of the instruction identifiers or opcodes corresponding to the one or more of the records 126.

[0074] In some examples, trace extractor 320 stores an association between an opcode and an emulation routine (e.g., machine-readable code, firmware and / or software routine, etc.). For example, trace extractor 320 may identify that a first opcode corresponds to a first emulation routine. In such an example, the first emulation routine may represent an algorithm, machine-readable instructions, etc. that, when executed, mimics and / or otherwise performs the same or substantially similar function as a SEND instruction corresponding to the first opcode. In some examples, trace extractor 320 stores records 126, instruction identifiers, opcodes, associations, etc. in storage 460.

[0075] In some examples, the trace extractor 420 implements means for identifying a first routine based on an identifier of a second routine executed by the GPU 110, the first routine being based on an emulation of the second routine. In some examples, the trace extractor 420 implements means for extracting the GLIT 112 from the trace buffer 114 and / or extracting the records 126 from the GLIT 112. In some examples, the means for identifying and / or the means for extracting may be implemented by one or more analog or digital circuits, logic circuits, programmable processors, programmable controllers, GPUs, DSPs, ASICs, PLDs, and / or FPLDs. In some examples, the means for identifying may be implemented by at least one of blocks 1602, 1604, 1606, 1608 of FIG. 16 .

[0076] 4, the trace emulator 430 emulates and / or otherwise replays the GLIT 112 of FIG. 1 to perform analysis of the operation of the GPU 110. For example, the trace emulator 430 may replay the execution of the second kernel 108 by the GPU 110 based on data stored in the GLIT 112. In some examples, the trace emulator 430 may replay the execution of one or more of the second kernel 108 by each of the threads 208 of FIG. 2 based on data stored in the GLIT 112 corresponding to each of the threads 208. In some examples, the trace emulator 430 executes emulation routines that simulate routines executed by the GPU 110. For example, trace emulator 430 may retrieve one or more of data records 126 from GLIT 112 and input the retrieved one or more of data records 126 as an argument to a first emulation routine that may simulate execution of an instruction (e.g., an add instruction, a subtract instruction, a multiply instruction, etc.) by GPU 110. In such an example, the retrieved one or more of data records 126 may be state of GPU 110, such as values ​​of registers such as ARF, GRF, etc., associated with a thread of interest in processing by GPU 110.

[0077] In some examples, trace emulator 430 instruments the emulation routine with callback routines (e.g., callback instructions) to facilitate analysis by a developer or user associated with application 120 of FIG. 1 , CPU 118 of FIG. 1 . For example, trace emulator 430 may include high-level language (HLL) instructions, which may represent machine-readable instructions, in the emulation routine. In such examples, in response to trace emulator 430 executing the instrumented emulation routine, trace emulator 430 may call an API to provide and / or otherwise transmit output data in connection with the execution of the instrumented emulation routine to a higher-level analysis construct, such as application 120. Advantageously, trace emulator 430 may instrument and execute the emulation routine to generate and provide data to a GPU profiling tool, which may be used to identify improvements to the operation of GPU 110, CPU 118, etc., and / or combinations thereof.

[0078] In some examples, the trace emulator 430 implements means for executing a first routine to determine a first value of a GPU state of a GPU, the first routine having (i) a first argument associated with a second routine and (ii) a second argument corresponding to a second value of the GPU state before executing the first routine. In some examples, the GPU state is a state of a first register in an ARF associated with a hardware thread of the GPU or a second register in a GRF of the hardware thread. In some examples, the identifier may be a first identifier extracted from an encoded binary file, and the means for executing may determine the first value, the second value, and a hardware thread identifier from a long instruction trace generated by the hardware thread in response to execution of one or more profile routines by the hardware thread. In such examples, the first value may correspond to a GPU register value after execution of a kernel by the hardware thread, the second value may correspond to a GPU register value before execution of the kernel by the hardware thread, and the hardware thread identifier may identify the hardware thread.

[0079] In some examples, the means for performing determines one or more first register values ​​of one or more respective first registers of a GRF of the GPU, determines one or more second register values ​​of one or more respective second registers of an ARF of the GPU, and / or stores the one or more first register values, the one or more second register values, the one or more third register values, and the device access instruction (e.g., a SEND instruction, a READ SEND instruction, etc.) in a long instruction trace such as a GLIT. In some examples, the one or more third registers may correspond to one or more respective destination registers associated with the device access instruction.

[0080] In some examples, the means for executing may insert a first callback routine into the instrumentation routine before the emulation routine, the first callback routine calling a first application programming interface (API) to provide the second GPU state to the application. In some examples, the means for executing may insert a second callback routine into the instrumentation routine after the emulation routine, the second callback routine calling the first API or the second API to provide the first GPU state to the application.

[0081] In some examples, the means for performing may be implemented by one or more analog or digital circuits, logic circuits, programmable processors, programmable controllers, GPUs, DSPs, ASICs, PLDs, and / or FPLDs. In some examples, the means for performing may be implemented by at least one of block 1508 of FIG. 15, blocks 1612, 1614, 1616 of FIG. 16, or block 1714 of FIG. 17.

[0082] In the illustrated example of FIG. 4, the GLIT engine 102 includes a trace analyzer 440 that determines one or more operational parameters associated with the GPU 110 of FIG. 1. In some examples, the trace analyzer 440 implements means for determining a GPU state, an execution time parameter, a busy time parameter, an idle time parameter, an occupied time parameter, and / or a utilization parameter associated with the GPU 110. In some examples, the trace analyzer 440 determines the one or more operational parameters based on an emulation of the operation of the GPU 110 by replaying the GLIT 112. For example, the trace analyzer 440 may determine the GPU state of a first one of the threads 208 of FIG. 2 by identifying a change in a register value of a GRF corresponding to the first one of the threads 208 in response to executing the second kernel 108. In some examples, the trace analyzer 440 may calculate the execution time parameter of the first one of the threads 208 by determining an amount of time required by the first one of the threads to execute the second kernel 108. In some examples, the trace analyzer 440 may determine the utilization parameter of the first one of the threads 208 by calculating the ratio of the busy time of the first one of the threads 208 to the total time of the time period of interest.

[0083] In some examples, the trace analyzer 440 determines an aggregate operational parameter based on two or more of the threads 208. For example, the trace analyzer 440 may calculate an aggregate execution time parameter, an aggregate utilization parameter, etc. In such examples, the trace analyzer 440 may determine the aggregate utilization parameter by calculating the ratio of busy one or more of the threads 208 to the total number of threads 208 for a duration or period of interest.

[0084] In some examples, the trace analyzer 440 implements means for determining an operating parameter of the GPU based on the GPU state. For example, the means for determining may determine a utilization rate of the GPU based on a first GPU state. In some examples, the means for determining may be implemented by one or more analog or digital circuits, logic circuits, programmable processors, programmable controllers, GPUs, DSPs, ASICs, PLDs, and / or FPLDs. In some examples, the means for determining may be implemented by at least one of block 1510 of FIG. 15 or block 1716 of FIG. 17.

[0085] In the illustrated example of FIG. 4 , the GLIT engine 102 includes a hardware configurator 450 that adjusts the operation of the GPU 110 and / or the CPU 118 based on the GLIT 112, one or more operating parameters associated with the GLIT 112, etc. In some examples, the hardware configurator 450 delivers, provides, and / or otherwise communicates the one or more operating parameters to the application 120 of FIG. 1 . For example, the hardware configurator 450 can report and / or otherwise communicate GPU state, hardware thread utilization, execution unit utilization, etc. associated with the GPU 110 to a developer (e.g., a software developer, a processor designer, a GPU engineer, etc.) using a performance analysis tool (e.g., a GPU profiling tool), a graphical user interface (GUI) included in the performance analysis tool, etc. In such an example, the developer can improve the software by, for example, improving load balancing of computational tasks, providing different data distribution among the hardware threads, execution units, etc. of the GPU 110, etc.

[0086] In some examples, the hardware configurer 450 may invoke hardware, software, firmware, and / or any combination of hardware, software, and / or firmware (e.g., GPU driver 122, CPU 118, etc.) to improve the operation of the GPU 110. For example, the hardware configurer 450 may generate and send instructions (e.g., commands, one or more machine-readable instructions, etc.) to the GPU driver 122, CPU 118, etc. of FIG. 1 . In response to receiving and / or otherwise executing the instructions, the GPU driver 122, CPU 118, etc. may be invoked to determine whether to adjust the operation of the GPU 110. For example, the GPU driver 122 and / or, more generally, the CPU 118 may be called to adjust the scheduling of computational tasks, jobs, workloads, etc., executed by the GPU 110 based on one or more operating parameters.

[0087] In some examples, the hardware configurer 450 invokes and / or otherwise instructs the GPU driver 122 to analyze one or more operating parameters based on the GLIT 112. For example, the GPU driver 122, and / or more generally the CPU 118, may compare the operating parameters to operating parameter thresholds (e.g., GPU state thresholds, execution time thresholds, busy time thresholds, idle time thresholds, utilization thresholds, etc.). For example, when invoked, the GPU driver 122, and / or more generally the CPU 118, may determine that the utilization of the GPU 110 is 95%, corresponding to the GPU 110 being busy for 95% of the measured time interval. The GPU driver 122 may compare the 95% utilization with a utilization threshold of 80% and determine that the GPU 110 should not accept more computational tasks based on the utilization that meets the utilization threshold. As used herein, a job or workload may refer to a set of one or more computational tasks executed by one or more hardware threads, such as threads 208 of FIG. 2.

[0088] In some examples, when invoked by the hardware configurer 450, the GPU driver 122, and / or, more generally, the CPU 118, may determine that the GPU 110 is at 40% utilization. The GPU driver 122 may compare the 40% utilization with an 80% utilization threshold and determine that the GPU 110 has available bandwidth to perform more computational tasks. For example, the GPU driver 122 may determine that the 40% utilization does not meet the 80% utilization threshold. In response to determining that the GPU 110 utilization does not meet the utilization threshold, the GPU driver 122 may adjust or modify resource scheduling to facilitate tasks performed by the GPU 110. For example, the GPU driver 122 may increase the amount of computational tasks that the GPU 110 is currently executing and / or will perform based on a utilization parameter, which may be determined based on the GLIT 112 of FIG. 1 .

[0089] In some examples, the hardware configurator 450 implements means for improving and / or otherwise optimizing resource scheduling (e.g., hardware scheduling, memory allocation, etc.) by the CPU 118. For example, a developer may develop and / or improve a hardware scheduling function or mechanism by analyzing one or more operating parameters associated with the GPU 110.

[0090] In some examples, the hardware configurer 450 implements means for controlling the workload of the GPU based on the first value of the GPU state. In some examples, the means for controlling, in response to determining that the operating parameter (e.g., busy time, utilization, etc.) does not meet a threshold, adjusts a routine executed by the GPU 110 (e.g., one or more instructions included in the second kernel 108) or increases the number of computational tasks to control the workload of the GPU 110. In some examples, the means for controlling may be implemented by one or more analog or digital circuits, logic circuits, programmable processors, programmable controllers, GPUs, DSPs, ASICs, PLDs, and / or FPLDs. In some examples, the means for controlling may be implemented by at least one of blocks 1512, 1514 of FIG. 15 or block 1720 of FIG. 17.

[0091] In the example of FIG. 4, GLIT engine 102 includes storage 460 that records data such as GLIT 470. For example, GLIT 470 may include one or more of GLIT 112 of FIG. 1. In such an example, GLIT 470 may be stored in storage 460 in an encoded binary format, such as GLIT format 300 of FIG. 3. In some examples, storage 460 records and / or otherwise stores one or more of records 126 of FIG. 1, which may include instruction identifiers, opcodes, and / or data associated with one or more of the instruction identifiers and / or one or more of the opcodes, one or more emulation routines, one or more associations between one or more of the one or more emulation routines and one or more of the instruction identifiers and / or opcodes, etc. In some examples, storage 460 stores instrumented versions of emulation routines, such as emulation routines that may include callback routines that invoke data transfers via one or more APIs.

[0092] Storage 460 in this example may be implemented by volatile memory (e.g., Dynamic Random Access Memory (SDRAM), Dynamic Random Access Memory (DRAM), RAMBUS Dynamic Random Access Memory (RDRAM), etc.) and / or non-volatile memory (e.g., flash memory). Storage 460 may additionally or alternatively be implemented by one or more dual data rate (DDR) memories, such as DDR, DDR2, DDR3, DDR4, mDDR (mobile DDR), etc. Storage 460 may additionally or alternatively be implemented by one or more mass storage devices, such as hard disk drives (HDDs), compact disc (CD) drives, digital versatile disc (DVD) drives, solid state disk (SSD) drives, etc. Although storage 460 is shown as a single storage in the illustrated example, storage 460 may be implemented by any number (e.g., at least one storage disk or device) and / or any type of storage. Furthermore, the data stored in storage 460 may be in any data format, such as, for example, binary data, comma-separated data, tab-separated data, Structured Query Language (SQL) structures, and the like.

[0093] 1 is illustrated in Figure 4, one or more of the elements, processes, and / or devices illustrated in Figure 4 may be combined, divided, rearranged, omitted, removed, and / or implemented in any other manner. Furthermore, the example instruction generator 410, the example trace extractor 420, the example trace emulator 430, the example trace analyzer 440, the example hardware configurator 450, the example storage 460, the example GLIT 470, and / or, more generally, the example GLIT engine 102 of Figure 1 may be implemented by hardware, software, firmware, and / or any combination of hardware, software, and / or firmware. Thus, for example, any of the example instruction generator 410, the example trace extractor 420, the example trace emulator 430, the example trace analyzer 440, the example hardware configurator 450, the example storage 460, the example GLIT 470, and / or, more generally, the example GLIT engine 102 may be implemented by one or more analog or digital circuits, logic circuits, programmable processors, programmable controllers, GPUs, digital signal processors (DSPs), application specific integrated circuits (ASICs), programmable logic devices (PLDs), and / or field programmable logic devices (FPLDs) (e.g., field programmable gate arrays (FPGAs)). When reading any of the apparatus or system claims of this patent to cover purely software and / or firmware implementations, at least one of the exemplary instruction generator 410, exemplary trace extractor 420, exemplary trace emulator 430, exemplary trace analyzer 440, exemplary hardware configurator 450, exemplary storage 460, and / or exemplary GLIT 470 is hereby expressly defined to include a non-transitory computer-readable storage device or storage disk, such as memory, DVD, CD, Blu-ray disc, etc., containing software and / or firmware.1 may include one or more elements, processes, and / or devices in addition to or instead of those shown in FIG. 4, and / or may include one or more of any or all of the illustrated elements, processes, and devices. As used herein, the phrase "in communication," including variations thereof, encompasses direct and / or indirect communication through one or more intermediate components and does not require direct physical (e.g., wired) communication and / or constant communication, but rather includes selective communication at periodic intervals, scheduled intervals, aperiodic intervals, and / or one-time events.

[0094] 5 is a diagram of an example system 500 that may implement GPU 110 of FIG. 1 or a portion thereof, and / or GPU slice 200 of FIG. 2 or a portion thereof. In this example, system 500 may be utilized to control the operation of an example execution unit hardware thread 502. In this example, execution unit hardware thread 502 may implement one of threads 208 of FIG. 2.

[0095] 5, system 500 includes an execution unit hardware thread 502, an example gateway shared function 504, an example thread dispatcher 506, and an example device 508. In this example, system 500 may exhibit different mechanisms, techniques, etc. for modifying and / or otherwise controlling the operation of execution unit hardware thread 502.

[0096] In the depicted example of FIG. 5 , system 500 includes a gateway sharing function 504 that implements inter-thread communication control. In this example, gateway sharing function 504 communicates asynchronously with execution unit hardware threads 502. Alternatively, gateway sharing function 504 may interact with execution unit hardware threads 502 on a synchronous basis. In some examples, gateway sharing function 504 may be implemented as hardware that performs thread-to-thread (e.g., hardware thread-to-hardware thread) synchronization. In some examples, gateway sharing function 504 can facilitate remote register write operations. For example, gateway sharing function 504 can obtain a write request from a first register of a first one of threads 208 of FIG. 2 and forward the write request to a second register of a second one of threads 208 of FIG. 2 .

[0097] In some examples, the gateway sharing function 504 implements active thread-thread communication based on direct register access. For example, a first thread (e.g., a requesting thread) may be able to write to the GRF register space of another thread (e.g., a receiving thread). Such direct register access between two threads in a multiprocessor environment may be referred to as a remote register access. The remote register access may implement a read operation or a write operation. In some examples, the architecture of the GPU 110 may support remote register writes but not (natively) support remote register reads. For example, the gateway sharing function 504 facilitates such remote register writes via message passing. In some examples, the requesting thread may send a message to the gateway sharing function 504 requesting a write to the GRF register space of the receiving thread. The gateway sharing function 504 may send a write message to the receiving thread to complete the register write on behalf of the requester. The requesting thread and the receiving thread may be on the same execution unit or different execution units of the GPU 110.

[0098] In the illustrated example of FIG. 5, the system 500 includes a thread dispatcher 506 that provides initial register values ​​as input payloads (e.g., input data payloads) to the execution unit hardware threads 502. In some examples, the thread dispatcher 506 may be implemented as a hardware functional unit for arbitrating thread start requests from the fixed function units 207 of FIG. 2 and instantiating threads 208 of the execution units 204. For example, the thread dispatcher 506 may determine which one(s) of the execution units 204 and which threads 208 of the execution units 204 to dispatch a job or software thread to. In some examples, the thread dispatcher 506 may load an initial GPU state into an idle one of the threads 208 and begin its execution based on the determination. In this example, the thread dispatcher 506 provides initial register values ​​to register files, such as the GRF and / or ARF, of the execution unit hardware threads 502 on a synchronous basis. In some examples, the thread dispatcher 506 may implement the local thread dispatcher 220 of FIG. 2.

[0099] In the depicted example of FIG. 5, system 500 includes a device 508 that executes responses to device access instructions from execution unit hardware thread 502. In some examples, device 508 may implement sampler 216 of FIG. 2, data port 218 of FIG. 2, shared local memory 214 of FIG. 2, and / or cache memory 210 of FIG. 2. In some examples, device 508 facilitates execution of device access requests. For example, a device access request may be implemented by any instruction that causes execution unit hardware thread 502 to write data to and / or read data from device 508. In some examples, a device access request may be implemented by a SEND instruction, a READ SEND instruction, a LOAD instruction, etc. For example, execution unit hardware thread 502 may execute a device access request by generating a SEND instruction in response to completing execution of a kernel, such as second kernel 108 of FIG. 1. In this example, because the SEND instruction is generated in response to completing execution of the kernel, the SEND instruction is known at the time of consumption.

[0100] In some examples, in response to executing a SEND instruction, the execution unit hardware thread 502 may send one or more register values ​​associated with the execution unit hardware thread 502 to the device 508. In some examples, in response to executing a READ SEND instruction, the execution unit hardware thread 502 may request one or more register values ​​stored in the device 508. In such examples, the device 508 may prepare a response to the READ SEND instruction by sending data read from the requested registers stored in the device 508 to the execution unit hardware thread 502.

[0101] 1 may capture different mechanisms, techniques, etc. for modifying and / or otherwise controlling the operation of the execution unit hardware thread 502. For example, the GLIT 112 may include a first GPU state of the execution unit hardware thread 502 at a first time, which may correspond to a first value of the ARF, a first value of the GRF, etc., of the execution unit hardware thread 502 during an initialization state, such as receiving initial register values ​​from the thread dispatcher 506, in preparation for executing the second kernel 108. In some examples, the GLIT 112 may include a second GPU state of the execution unit hardware thread 502 at a second time after the first time. The second GPU state may correspond to a second value of the ARF, a second value of the GRF, etc., in response to the gateway shared function 504 executing one or more remote register write operations from a different one or more of the threads 208 of FIG. 2. In some examples, one or more remote register write operations may change one or more of the first values ​​of the GRF to one or more of the second values. In some examples, the GLIT 112 may include a third GPU state of the execution unit hardware thread 502 at a third time after the second time. The third GPU state may correspond to a third value of the ARF, a third value of the GRF, etc., in response to the execution unit hardware thread 502 generating a SEND instruction that invokes the device 508 to read data from the ARF, the GRF, etc.

[0102] Figure 6 is a diagram of an example GLIT 600 for the GPU 110 of Figure 1, the GPU slice 200 of Figure 2, and / or the execution unit hardware thread 502 of Figure 5. In some examples, the GLIT 600 of Figure 6 may implement one or more of the GLITs 112 of Figure 1. In some examples, the GLIT 600 may be encoded in a binary kernel having a format based on the GLIT format 300 of Figure 3. For example, the GLIT 600 may be implemented by an encoded binary file representing an example execution of the kernel 108 of Figure 1 by one of the threads 208 of Figure 2.

[0103] In some examples, one of the GLITs 112 of Figure 1 may include multiple binary kernels. In some examples, the GLIT 600 may implement one of the multiple binary kernels. Advantageously, the second kernel 108 of Figure 1 may be distributed for execution across multiple threads 208 of Figure 2, such that multiple binary kernels can implement multi-threaded GPU tracing.

[0104] In this example, GLIT 600 may implement a GLIT for a single software thread running on a hardware thread (e.g., thread 208 of FIG. 2, execution unit hardware thread 502 of FIG. 5, etc.) of an exemplary processor, such as GPU 110 of FIG. 1 or GPU slice 200 of FIG. 2. In some examples, a GLIT for an execution unit, such as execution unit 204 of FIG. 2, may be implemented by a group or collection of GLITs for ones of threads 208 of execution unit 204. In some examples, a GLIT for a subslice, such as subslice 202 of FIG. 2, may be implemented by a group or collection of GLITs for ones of execution units 204.

[0105] 3, GLIT 600 begins at an exemplary start point 602 with a GPU having a state (e.g., GPU state) based on the initial values ​​of the registers of the hardware thread's ARF and the initial values ​​of the registers of the hardware thread's GRF. In this example, the initial values ​​of the registers of the GRF may be for the entire GRF. Alternatively, the initial values ​​of the registers of the GRF may be for a partial number of the GRF's registers.

[0106] In this example, the initial values ​​of the registers of the ARF may be for a partial number of the registers of the ARF, such as a first register value corresponding to the dispatch mask of the ARF, a second register value corresponding to a hardware thread identifier (TID), a third register value corresponding to the execution mask, a fourth register value corresponding to a control register, etc. Alternatively, the initial values ​​of the registers of the ARF may be for all of the registers of the ARF.

[0107] After the GPU state is initialized at start point 602, GLIT 600 includes a first exemplary event (EVENT 1) 604 at a first time after start point 602, a second exemplary event (EVENT 2) 606 at a second time after the first time, a third exemplary event (EVENT 3) 608 at a third time after the second time, and a fourth exemplary event (EVENT 4) 610 at a fourth time after the third time. In this example, events 604, 606, 608, 610 are READ SEND instructions, which may represent sending a message from external hardware (e.g., device 508 of FIG. 5 ) to a hardware thread, such as sampler 216 and / or cache memory 210 of FIG. 2 . For example, first event 604 may represent a read from global memory, such as cache memory 210, using a value in a destination register of the hardware thread, represented by DST. In another example, the second event 606 may represent an access to a sampler, such as sampler 216 of Figure 2, using a value in a destination register of a hardware thread represented by a DST. Additionally or alternatively, a GLIT, such as GLIT 600 of Figure 6, may include fewer or more events than those shown in Figure 6.

[0108] Advantageously, information related to the GLIT 600 of Figure 6 is encoded in a binary format, such as the GLIT format 300 of Figure 3, and stored in binary format in memory 116 for later access and / or retrieval by a processor, such as the CPU 118 of Figure 1. For example, initial values ​​of GPU state, such as GRF register values, ARF register values, etc., at starting point 602 may be encoded using the GLIT format 300. In some examples, one or more of the events 604, 606, 608, 610 may be stored using the GLIT format 300. In such examples, register values, such as values ​​of the ARF register and / or GRF register before and / or after one or more of the events 604, 606, 608, 610, may be stored using the GLIT format 300.

[0109] FIG. 7 is a diagram of an example system 700 for generating and analyzing the GLIT 600 of FIG. 6. The system 700 of FIG. 7 includes an example GPU 702 and an example CPU 704. In some examples, the GPU 702 may implement the GPU 110 of FIG. 1 and / or the GPU slice 200 of FIG. 2. In some examples, the CPU 704 may implement the CPU 118 of FIG. 1.

[0110] In the illustrated example of FIG. 7, the GPU 702 executes an exemplary kernel 706. In this example, the kernel 706 is an instrumented kernel that may implement the second kernel 108 of FIG. 1. In this example, the kernel 706 is distributed among a plurality of exemplary hardware threads 708 of the GPU 702 and / or scheduled for execution by other methods. For example, each of the threads 708 may be implemented by one of the threads 208 of FIG. 2. In this example, a first one of the hardware threads 708 may have a hardware thread identifier of TID 0, a second one of the hardware threads 708 may have a hardware thread identifier of TID 1, and an Nth one of the hardware threads 708 may have a hardware thread identifier of TID N.

[0111] 7, hardware threads 708 may execute instances of kernel 706 and generate respective exemplary GLIT data 710. For example, a first one of the hardware threads 708 may generate and / or otherwise output GLIT DATA 0, a second one of the hardware threads 708 may generate and / or otherwise output GLIT DATA 1, and an Nth one of the hardware threads 708 may generate and / or otherwise output GLIT DATA 2. In some examples, the GPU 702 may store the GLIT data 710 as records 126 in the trace buffer 114 of memory 116 of FIG. 1.

[0112] In some examples, GLIT data 710 may include at least one of GPU state (e.g., one or more ARF register values ​​of a hardware thread, one or more GRF register values, etc.) or data associated with kernel 706. For example, data associated with kernel 706 may include GPU instructions included in kernel 706, opcodes corresponding to the instructions, instruction identifiers corresponding to the instructions, etc. In some examples, a portion of GLIT data 710 may implement one or more of records 126 of FIG. 1, one or more of records 302 of FIG. 3, etc. For example, a first portion of GLIT DATA 0 may include instructions of kernel 706, which may be stored by GPU 702 in trace buffer 114 as INST_DECODE INST0 of GLIT format 300 of FIG. 3.

[0113] In some examples, the CPU 704 obtains and / or otherwise retrieves GLIT data 710 from a buffer stored in memory, such as the trace buffer 114 stored in memory 116 of FIG. 1 , and generates an example GLIT 712 based on the GLIT data 710. In some examples, the GLIT 712 may implement the GLIT 112 of FIG. 1 . In some examples, the CPU 704 may generate a first one of the GLITs 712 corresponding to a first one of the hardware threads 708 based on GLIT DATA 0, a second one of the GLITs 712 corresponding to a second one of the hardware threads 708 based on GLIT DATA 1, a third one of the GLITs 712 corresponding to a third one of the hardware threads 708 based on GLIT DATA N, etc. In such examples, the first one of the GLITs 712, the second one of the GLITs 712, the third one of the GLITs 712, etc. may be generated by arranging and / or otherwise organizing each one of the GLIT data 710 into a file (e.g., a binary file) based on the GLIT format 300 of FIG. 3 . In some examples, the CPU 704 may generate a first one of the GLITs 712 by arranging and / or otherwise organizing GLIT DATA 0, GLIT DATA 1, GLIT DATA N, etc., in a binary file based on the GLIT format 300.

[0114] In this example, the CPU 704 implements an exemplary GLIT replay application 714 that replays the execution of the kernel 706 by the GPU 702 based on the GLIT 712 by simulating the execution of the kernel 706. In some examples, the GLIT replay application 714 may implement application 120 of Figure 1. For example, the GLIT replay application 714 may be a software application that instruments emulation routines (e.g., emulation instructions, emulation software routines, etc.) corresponding to the simulation of GPU routines (GPU instructions, GPU kernel routines, etc.) utilized to execute the kernel 706.

[0115] In some examples, the instrumented emulation routine calls the example API 716 to communicate and / or otherwise send data to the example hardware profiling and analysis tool 718. For example, the GLIT replay application 714 may instrument the first emulation routine with a first callback routine before execution of instructions included in the first emulation instructions (e.g., instructions that simulate execution of the kernel 706) and / or a second callback routine after execution of the instructions.

[0116] In some examples, in response to executing the first callback routine, the GLIT replay application 714 may call one of the APIs 716 to provide the hardware profiling analysis tool 718 with a first GPU state corresponding to a first value of the GRF register of the GPU 702 before execution of the instructions included in the first emulation routine. In some examples, in response to executing the second callback routine, the GLIT replay application 714 may call one of the APIs 716 to provide the hardware profiling analysis tool 718 with a second GPU state corresponding to a second value of the GRF register after execution of the instructions included in the first emulation routine. In some examples, the first GPU state may be the same as the second GPU state. That is, the GRF register was not changed in response to executing the first emulation routine. In some examples, the first GPU state may differ from the second GPU state. That is, the GRF register was changed in response to executing the first emulation routine to indicate that execution of the kernel 706 modified the GRF register.

[0117] 1. For example, the hardware profiling analysis tool 718 may be a software application that analyzes replays and / or other emulations of the execution of the kernel 706 to identify improvements to the operation of at least one of the GPU 702 or the CPU 704 based on the replays and / or other emulations of the execution of the kernel 706. In some examples, the hardware profiling analysis tool 718 may be implemented by one or more DLLs. Additionally or alternatively, the hardware profiling analysis tool 718 may analyze the operation of any other type of hardware processor, such as a neural network processor, a VPU, etc.

[0118] In some examples, the hardware profiling analysis tool 718 can identify improvements based on the changes in the GRF registers, as described above. In some examples, the hardware profiling analysis tool 718 can determine that the changes in the GRF registers are not a typical or expected result, which can notify the developer to modify the second kernel 108 for improved execution by the GPU 702. In some examples, the hardware profiling analysis tool 718 can determine that the lack of detected changes in the GRF registers indicates that the distribution of the kernel 706 to the hardware threads of the GPU 702 is not an efficient distribution, which can notify the developer to modify the scheduling of the second kernel 108 to improve the distribution of the kernel 706.

[0119] FIG. 8 is a diagram of another example system 800 for emulating and analyzing the GLIT 600 of FIG. 6 . For example, the system 800 of FIG. 8 may implement the system 700 of FIG. 7 , or portions thereof. In this example, the system 800 includes an example GLIT replay application 802 and multiple example tools 804, 806, and 808. For example, the GLIT replay application 802 may implement the GLIT replay application 714 of FIG. 7 . In some examples, the tools 804, 806, and 808 may implement the hardware profiling analysis tool 718 of FIG. 7 . For example, one or more of the tools 804, 806, and 808 may be implemented as software applications that analyze execution of kernels by a GPU by replaying the execution using data stored and / or otherwise included in the GLIT 810. In some examples, one or more of the tools 804, 806, and 808 may be implemented as one or more DLLs that perform different analyses of the execution of the kernels. For example, a first tool 804 of tools 804, 806, 808 may profile the execution of the kernel using a first set of analysis routines, functions, etc., a second tool 806 of tools 804, 806, 808 may profile the execution of the kernel using a second set of analysis routines, functions, etc., and / or a third tool 808 of tools 804, 806, 808 may profile the execution of the kernel using a third set of analysis routines, functions, etc., wherein one or more of the first set, second set, and / or third set may differ from one another.

[0120] In the illustrated example of FIG. 8, the GLIT replay application 802 obtains an exemplary GLIT 810. For example, the GLIT 810 may implement the GLIT 112 of FIG. 1, the GLIT 470 of FIG. 4, the GLIT 600 of FIG. 6, and / or the GLIT 712 of FIG. 7. In some examples, the GLIT 810 may be an encoded binary kernel, and the GLIT replay application 802 may decode the GLIT 810. For example, the GLIT replay application 802 may decompress and / or otherwise extract data stored in a binary format, such as the GLIT format 300 of FIG. 3. In some examples, the GLIT replay application 802 may associate a portion of the extracted data with an exemplary hardware thread identifier (TID0-TIDN) 812 of a GPU, such as the GPU 110 of FIG. 1, or a slice of the GPU, such as the GPU slice 200 of FIG. 2.

[0121] 8, the GLIT replay application 802 communicates with one or more of the tools 804, 806, 808 via one or more exemplary APIs 814. For example, the GLIT replay application 802 can instrument an emulation routine that simulates the execution of a GPU kernel by including callback routines before and / or after the execution of the instrumented emulation routine. In some examples, the callback routines can include a “CALLBACKBEFORE()” callback routine, which, when executed, can call a first one of the APIs 814 to provide data such as the GPU state to a corresponding one of the tools 804, 806, 808 before executing instructions included in the instrumented emulation routine. For example, the “CALLBACKBEFORE()” callback routine can call “GETSTATE()” to provide the GPU state. In some examples, the callback routine may include a “CALLBACKAFTER()” callback routine, which, when executed, after executing instructions included in the instrumentation emulation routine, may call a second one of the APIs 814 to provide data such as GPU state to corresponding ones of the tools 804, 806, 808. For example, the “CALLBACKAFTER()” callback routine may call “GETSTATE()” to provide GPU state. Additionally or alternatively, fewer or more APIs 814 than those shown in FIG. 8 may be used. Additionally or alternatively, one or more APIs 814 different from those shown in FIG. 8 may be used. For example, one or more APIs 814 may be implemented using a PIN API, which may be used to insert machine-readable code (e.g., C code, C++ code, etc.) into one or more locations in the kernel.

[0122] FIG. 9 illustrates a first example kernel 902 and a second example kernel 904 that may be executed by a GPU, such as GPU 110 of FIG. 1 , a slice of a GPU, such as GPU slice 200 of FIG. 2 , and / or GPU 702 of FIG. 7 . In this example, the first kernel 902 may implement a non-instrumented kernel. For example, the first kernel 902 may implement the first kernel 106 of FIG. 1 . In this example, the second kernel 904 may implement an instrumented kernel, such as the second kernel 108 of FIG. 1 and / or the kernel 706 of FIG. 7 . In this example, the second kernel 904 may correspond to an instrumented version of the first kernel 902.

[0123] In the depicted example of FIG. 9 , the first kernel 902 includes exemplary instructions such as a move instruction (MOV), an or instruction (OR), a multiply (MUL) instruction, and an and (AND) instruction. In response to executing the MOV, OR, MUL, and AND instructions, the first kernel 902 causes execution of a first SEND instruction (SEND) and a second SEND instruction (SEND). In this example, the SEND instructions are read instructions from a global memory, such as cache memory 210 of FIG. 2 or device 508 of FIG. 5. In this example, the first SEND instruction implements a first read operation of two 32-byte wide registers (e.g., register r12 of global memory is 32 bytes wide, register r13 of global memory is 32 bytes wide, etc.). In this example, the second SEND instruction implements a second read operation of two 32-byte wide registers (e.g., registers r9 and r10 of global memory).

[0124] In the depicted example of FIG. 9, the second kernel 904 includes the MOV, OR, MUL, and SEND instructions of the first kernel 902. In this example, the second kernel 904 includes exemplary instrumentation instructions (TRACE) 906, 908, 910 that generate an exemplary GLIT, such as GLIT 112 of FIG. 1 or GLIT 600 of FIG. 6. In this example, the instrumentation instructions 906, 908, 910 include a first exemplary trace instruction (TRACE(TID, R0-R15, CE, DMASK...)) 906 that traces the full input payload of GRF registers and a portion or subset of ARF registers associated with the hardware thread of the GPU executing the second kernel 904. For example, the first trace instruction 906 may read registers r0-r15 of the hardware thread's GRF and at least the CE and DMASK registers of the hardware thread's ARF. In such an example, the input payload, represented as r0 through r15, may include sixteen 32-byte registers (e.g., r0, r1, r2, . . . r15).

[0125] In the illustrated example, the instrumentation instructions 906, 908, 910 include a second exemplary trace instruction 908 that traces a resulting destination value after execution of the first SEND instruction. For example, in response to executing the first SEND instruction, the second trace instruction 908 may obtain the resulting values ​​of the destination registers (e.g., r12 and r13 of global memory). In some examples, the second trace instruction 908, when executed, may generate a trace record (e.g., one of records 126 of FIG. 1 , one of records 302 of FIG. 3 , etc.) to include the TID of the hardware thread that executed the second kernel 904 and a first offset (e.g., a first offset value) of the original instruction within the first kernel 902.

[0126] In the illustrated example, the instrumentation instructions 906, 908, 910 include a third exemplary trace instruction 910 that traces the resulting destination values ​​after execution of the second SEND instruction. For example, in response to executing the second SEND instruction, the third trace instruction 910 can obtain the resulting values ​​of the destination registers (e.g., r9 and r10 of global memory). In some examples, the third trace instruction 910, when executed, can generate a trace record (e.g., one of records 126 of FIG. 1 , one of records 302 of FIG. 3 , etc.) to include the TID of the hardware thread that executed the second kernel 904 and a second offset (e.g., second offset value) of the original instruction within the first kernel 902. Advantageously, the order of the trace records corresponds to offset zero, which can provide an order of software thread dispatch. For example, the dispatch order of the second kernel 904 may be determined based on the first offset value and the second offset value relative to offset zero.

[0127] FIG. 10 is a diagram of an example workflow 1000 for emulating the execution of an instrumented GPU kernel, such as the second kernel 108 of FIG. 1 , the kernel 706 of FIG. 7 , and / or the second kernel 904 of FIG. 9 . For example, the workflow 1000 may be implemented by the GLIT engine 102 of FIG. 1 and / or FIG. 4 . In this example, the workflow 1000 is implemented by example kernel instruction static data 1002, an example opcode emulation table 1004, an example emulation routine 1006, and example GPU states 1008, 1010. Alternatively, any other example workflow may be utilized to emulate the execution of an instrumented GPU kernel.

[0128] In the illustrated example of FIG. 10 , kernel instruction static data 1002 may correspond to instructions decoded from a binary kernel. For example, the second kernel 108 may include multiple example encoded instructions in binary format. In some examples, the trace extractor 420 of FIG. 4 may extract and / or otherwise decode the encoded instructions from the second kernel 108 to generate the kernel instruction static data 1002. In this example, the kernel instruction static data 1002 includes a first example instruction having a first instruction identifier (INST 0) as an index, which may correspond to a first decoded instruction from the second kernel 108 of FIG. 1 . For example, INST 0 may correspond to a first SEND instruction (e.g., SEND(16)R12 R6 0XC 0X4205E00) of the second kernel 904 of FIG. 9 .

[0129] In the depicted example of Figure 10, opcode emulation table 1004 may correspond to opcodes supported by a particular GPU architecture, such as the architecture of GPU 110 of Figure 1. In this example, opcode emulation table 1004 includes a first example opcode (OPCODE 0), which may correspond to a first type of instruction that GPU 110 supports and / or is otherwise configured to execute upon invocation.

[0130] 10 , emulation routine 1006 may correspond to a first instruction (e.g., a machine-readable instruction) that, when executed, may simulate the execution of a second instruction that GPU 110 is configured to execute. In this example, emulation routine 1006 includes a first exemplary emulation routine (ADD_EMUL), which may correspond to the emulation of an additional operation that GPU 110 supports and / or is otherwise configured to execute upon invocation. In this example, the opcodes in opcode emulation table 1004 correspond to respective ones of emulation routine 1006. For example, OPCODE 0 corresponds to ADD_EMUL, OPCODE 1 corresponds to SUB_EMUL, etc.

[0131] In the depicted example of FIG. 10 , GPU state 1008, 1010 includes an exemplary GRF state 1008 and an exemplary ARF state 1010. In this example, GRF state 1008 is the value of a register stored in a GRF implemented by a hardware thread of a GPU, such as one of threads 208 of FIG. 2 . In this example, GRF state 1008 is implemented using 128 registers (r0-r127). In this example, ARF state 1010 is the value of a register stored in an ARF implemented by a hardware thread of a GPU, such as one of threads 208 of FIG. 2 .

[0132] In this example, the ARF state 1010 includes a portion of the ARF, such as a first register value (F0.0) storing a value at a first end of a first floating-point saturation range, a second register value (F0.1) storing a value at a second end of the first floating-point saturation range, a third register value (F1.0) storing a value at a first end of a second floating-point saturation range, a fourth register value (F1.1) storing a value at a second end of the second floating-point saturation range, a fifth register value (IP) storing a value of an instruction pointer, a sixth register value (DMASK register), a seventh register value (CE register), an eighth register value (ACC0) storing a value of an accumulation register, a ninth register value (A0) storing an address register, a notification register (N0), and a tenth register value (execution mask). As an example, the IP register may implement a pointer that references the current instruction in the GPU's memory. In some examples, each of the threads 208 may have their own IP. Additionally or alternatively, the portion of the ARF may include fewer or more ARF states than shown in the example of FIG. 10.

[0133] In exemplary operation, the trace extractor 420 of FIG. 4 may decode a GLIT, such as the GLIT 600 of FIG. 6, to generate and / or otherwise output decoded binary data and GPU state. In some examples, the trace extractor 420 stores portions of the decoded binary data as kernel instruction static data 1002 that utilizes the instruction identifier as an index. In some examples, the trace extractor 420 stores portions of the decoded binary data as GPU state 1008, 1010. In some examples, the trace extractor 420 associates one or more of the kernel instruction static data 1002, an opcode in the opcode emulation table 1004, one or more of the emulation routines 1006, and / or one or more of the GPU state 1008, 1010. For example, the trace extractor 420 may determine that INST 0 corresponds to OPCODE 0, and that OPCODE 0 corresponds to ADD_EMUL. In such an example, the trace extractor 420 may store an association of at least one of INST 0, OPCODE 0, ADD_EMUL, or a corresponding one of the GPU states 1008, 1010. For example, the trace extractor 420 may store the association in the storage 460 of FIG.

[0134] 4 may emulate the execution of a GPU kernel for which kernel instruction static data 1002 and / or GPU state 1008, 1010 are generated. In some examples, trace emulator 430 may replay a GLIT, such as GLIT 600, by selecting INST 0 for execution. In some examples, in response to selecting INST 0 for execution, trace emulator 430 calls a first emulation routine of ADD_EMUL and inputs at least one of INST 0, OPCODE 0, or a corresponding GPU state 1008, 1010 as arguments to the first emulation routine. Advantageously, trace emulator 430 may replay the execution of a GPU kernel, such as second kernel 108 of FIG. 1, by executing (e.g., repeatedly executing) one or more emulation routines in emulation routine 1006 that correspond to instructions of the GPU kernel, which may be represented by information included in kernel instruction static data 1002.

[0135] Source code and / or flowcharts representing exemplary hardware logic, machine-readable instructions, hardware-implemented state machines, and / or any combination thereof for implementing the exemplary GLIT engine 102 of FIGS. 1 and / or 4 are illustrated in FIGS. 11-17. The machine-readable instructions may be one or more executable programs or portions thereof for execution by a computer processor and / or processor circuitry, such as the processor 1812 shown in the exemplary processor platform 1800 described below in connection with FIG. 18. The programs may be implemented in software stored on a non-transitory computer-readable storage medium, such as a CD-ROM, floppy disk, hard drive, DVD, Blu-ray disk, or memory associated with the processor 1812, although alternatively, the entire program and / or portions thereof may be executed by a device other than the processor 1812 and / or implemented in firmware or dedicated hardware. Furthermore, although the exemplary programs are described with reference to the source code and / or flowcharts illustrated in FIGS. 11-17, many other methods of implementing the exemplary GLIT engine 102 may alternatively be used. For example, the order of execution of the blocks may be changed, and / or some of the described blocks may be modified, eliminated, or combined. Additionally or alternatively, any or all of the blocks may be implemented by one or more hardware circuits (e.g., discrete and / or integrated analog and / or digital circuits, FPGAs, ASICs, comparators, operational amplifiers (op-amps), logic circuits, etc.) structured to perform the corresponding operations without executing software or firmware. Processor circuits may be distributed across different network locations and / or locally distributed across one or more devices (e.g., multi-core processors in a single machine, multiple processors distributed across a server rack, etc.).

[0136] The machine-readable instructions described herein may be stored in one or more of a compressed format, an encrypted format, a fragmented format, a compiled format, an executable format, a packaged format, etc. The machine-readable instructions described herein may be stored as data or a data structure (e.g., portions of instructions, code, representations of code, etc.) that can be utilized to create, manufacture, and / or generate machine-executable instructions. For example, the machine-readable instructions may be fragmented and stored in one or more storage devices and / or computing devices (e.g., servers) located in the same or different locations of a network or collection of networks (e.g., a cloud, an edge device, etc.). The machine-readable instructions may require one or more of installing, modifying, adapting, updating, combining, supplementing, configuring, decrypting, decompressing, unpacking, distributing, reallocating, compiling, etc. to become directly readable, interpretable, and / or executable by computing devices and / or other machines. For example, the machine-readable instructions may be individually compressed, encrypted, and stored in multiple portions stored on separate computing devices, which portions, when decoded, decompressed, and combined, form a set of executable instructions that implement one or more functions that can together form a program as described herein.

[0137] In another example, machine-readable instructions may be stored in a state where they can be read by a processor circuit, but may require the addition of a library (e.g., a DLL), a software development kit (SDK), an API, etc., to execute the instructions on a particular computing device or other device. In another example, the machine-readable instructions (e.g., stored settings, data entries, recorded network addresses, etc.) may need to be configured before the machine-readable instructions and / or corresponding program can be executed, in whole or in part. Thus, as used herein, a machine-readable medium can include machine-readable instructions and / or programs, whether stored at rest or during transport, or otherwise, regardless of the particular format or state of the machine-readable instructions and / or program.

[0138] The machine-readable instructions described herein may be expressed by any past, present, or future command language, scripting language, programming language, etc. For example, the machine-readable instructions may be expressed using any of the following languages: C, C++, Java, C#, Perl, Python, JavaScript, HyperText Markup Language (HTML), Structured Query Language (SQL), Swift, etc.

[0139] 11-17 may be implemented using executable instructions (e.g., computer- and / or machine-readable instructions) stored on a non-transitory computer- and / or machine-readable medium, such as a HDD, flash memory, read-only memory, CD, DVD, cache, random access memory, and / or any other storage device or storage disk that stores information for any duration (e.g., long-term, permanent, short-term, brief, temporarily buffers, and / or caches information). As used herein, the term non-transitory computer-readable medium is expressly defined to include any type of computer-readable storage device and / or storage disk and to exclude propagating signals and to exclude transmission media.

[0140] " The terms "including" and "comprising" (and all forms and tenses thereof) are used herein as open-ended terms. Thus, whenever a claim uses any form of "include" or "comprise" (e.g., comprises, includes, comprising, including, having, etc.) as a preamble or within any kind of claim recitation, it is to be understood that additional elements, terms, etc. may be present without exceeding the scope of the corresponding claim or recitation. As used herein, when the phrase "at least" is used as a transitional phrase, for example, in a claim preamble, it is open-ended in the same way that the terms "comprising" and "including" are open-ended. The term "and / or," when used in the form of, for example, A, B, and / or C, refers to any combination or subset of A, B, and C, such as (1) A only, (2) B only, (3) C only, (4) A and B, (5) A and C, (6) B and C, or (7) A, B, and C. When used herein in the context of describing structures, components, items, objects, and / or things, the phrase "at least one of A and B" is intended to refer to an implementation that includes either (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. Similarly, when used herein in the context of describing structures, components, items, objects, and / or things, the phrase "at least one of A or B" is intended to refer to an implementation that includes either (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. As used herein in the context of describing the performance or execution of a process, instruction, action, activity, and / or step, the phrase "at least one of A and B" is intended to refer to an implementation that includes either: (1) at least one A; (2) at least one B; or (3) at least one A and at least one B.Similarly, when used herein in the context of describing the execution of a process, instruction, action, activity, and / or step, the phrase "at least one of A or B" is intended to refer to an implementation that includes either (1) at least one A, (2) at least one B, or (3) at least one A and at least one B.

[0141] As used herein, singular references (e.g., "a," "an," "first," "second," etc.) do not exclude a plurality. The term "a" or "an" entity, when used herein, refers to one or more of that entity. The terms "a" (or "an"), "one or more," and "at least one" may be used interchangeably herein. Furthermore, although individually listed, a plurality of means, elements, or method actions may be implemented by, for example, a single unit or processor. Furthermore, although individual features may be included in different examples or claims, these may be combined, and inclusion in different examples or claims does not imply that a combination of features is infeasible and / or advantageous.

[0142] FIG. 11 shows example source code 1100 for emulating the execution of an example instrumented kernel, such as second kernel 108 of FIG. 1 , kernel 706 of FIG. 7 , and / or second kernel 904 of FIG. 9 . Alternatively, any other source code may be executed to emulate the execution of an instrumented kernel. In some examples, source code 1100 of FIG. 11 may represent machine-readable instructions that may be executed by trace emulator 430 of FIG. 4 and / or, more generally, GLIT engine 102 of FIG. 1 and / or FIG. 4 . For example, trace emulator 430 may execute source code 1100 to emulate (e.g., iteratively emulate) instructions included in an instrumented kernel executed by a GPU, such as GPU 110 of FIG. 1 .

[0143] In some examples, in response to executing the source code 1100, the trace emulator 430 may select one of the instructions included in the kernel instruction static data 1002 of FIG. 10. For example, the trace emulator 430 may select the instruction corresponding to INST 0. In some examples, in response to executing the source code 1100, the trace emulator 430 may determine whether the instruction corresponding to INST 0 is a SEND instruction to global memory (e.g., the cache memory 210) or a sampler (e.g., the sampler 216 of FIG. 2). If the trace emulator 430 determines that the instruction is a SEND instruction to global memory or a sampler, the trace emulator 430 may update register values ​​from the trace. For example, the trace emulator 430 may update the register values ​​based on the GPU state 1008, 1010 before and / or after executing the instruction.

[0144] In some examples, if the trace emulator 430 determines that the instruction is not a SEND instruction to global memory or a sampler, the trace emulator 430 may emulate the instruction. For example, the trace emulator 430 may emulate the instruction by calling one of the emulation routines in the emulation routine table 1006 to emulate the instruction. In this example, the trace emulator 430 may execute (e.g., repeatedly execute) the source code 1100 for one or more of the instructions included in the kernel instruction static data 1002 of FIG. 10 .

[0145] FIG. 12 shows exemplary source code 1200 for emulating the execution of an exemplary software thread. Alternatively, any other source code may be executed to emulate the execution of a software thread. In some examples, the source code 1200 of FIG. 12 may represent machine-readable instructions that may be executed by the trace emulator 430 of FIG. 4 and / or, more generally, the GLIT engine 102 of FIGS. 1 and / or 4. For example, the trace emulator 430 may execute the source code 1200 to emulate an instance of dispatching a kernel, such as the second kernel 108, to a hardware thread, such as one of the threads 208, by emulating instructions included in the kernel.

[0146] In some examples, in response to executing source code 1200, trace emulator 430 emulates instructions included in the kernel by determining an offset identifier (e.g., an offset value) (OffsetToID) corresponding to one of the instructions. For example, trace emulator 430 may determine the offset identifier based on the GPU state (State.IP) of the IP register value of the ARF. In some examples, in response to executing source code 1200, trace emulator 430 returns an instruction (ins), such as INST 0 in FIG. 10 , based on the instruction identifier. In some examples, in response to executing source code 1200, trace emulator 430 identifies an opcode based on the instruction. In some examples, in response to executing source code 1200, trace emulator 430 identifies one of the emulation routines in emulation routine table 1006 based on the opcode. In some examples, in response to executing source code 1200, trace emulator 430 executes one of the identified emulation routines using the instruction and one or more GPU states (State) as exemplary arguments. In this example, trace emulator 430 may execute (e.g., repeatedly execute) source code 1200 until an end-of-time (EOT) instruction is generated, which may be generated in response to the last instruction of the kernel being executed. For example, the EOT instruction may be generated in response to INST N being emulated.

[0147] FIG. 13 shows exemplary source code 1300 for emulating the execution of an exemplary instrumented software thread. Alternatively, any other source code may be executed to emulate the execution of an instrumented software thread. In some examples, the source code 1300 of FIG. 13 may represent machine-readable instructions that may be executed by the trace emulator 430 of FIG. 4 and / or, more generally, the GLIT engine 102 of FIGS. 1 and / or 4. For example, the trace emulator 430 may execute the source code 1300 to emulate an instance of dispatching a kernel, such as the second kernel 108, to a hardware thread, such as one of the threads 208, by emulating instructions included in the kernel.

[0148] 12 with a first example instrumentation routine (e.g., an instrumentation routine, an instrumentation instruction, etc.) 1302 and a second example instrumentation routine 1304. For example, trace emulator 430 may execute first instrumentation routine 1302 before executing emulation routines (EmulRoutines) and execute second instrumentation routine 1304 after executing the emulation routines.

[0149] In some examples, in response to executing the first instrumentation routine 1302, the trace emulator 430 may call a callback routine (e.g., “CallbackBefore();”) to invoke an API to provide the GPU state of the hardware thread that executed the software thread to a higher-level component, such as the application 120 of FIG. 1 or the hardware profiling analysis tool 718 of FIG. 7.

[0150] In some examples, in response to executing the second instrumentation routine 1304, the trace emulator 430 may call a callback routine (e.g., “CallbackAfter();”) to invoke an API to provide the GPU state of the hardware thread that executed the software thread to a higher-level component, such as the application 120 of FIG. 1 or the hardware profiling analysis tool 718 of FIG. 7. Advantageously, by registering the callback routine with the higher-level component, the trace emulator 430 may provide the GPU state of the hardware thread before and / or after executing the emulation routine to determine changes in the GPU state in response to executing the emulation routine.

[0151] FIG. 14 shows exemplary source code 1400 for implementing an emulation routine. Alternatively, other source code may be executed to implement the emulation routine. In some examples, source code 1400 may implement one or more of the emulation routines in emulation routine table 1006 of FIG. 10 and / or emulation routines (EmulRoutines) of FIG. 13. In some examples, source code 1400 of FIG. 14 may represent machine-readable instructions that may be executed by trace emulator 430 of FIG. 4 and / or, more generally, GLIT engine 102 of FIG. 1 and / or FIG. 4 to simulate execution of GPU kernel instructions, such as instructions included in second kernel 108 of FIG. 1.

[0152] In some examples, in response to executing source code 1400, trace emulator 430 may prepare data for emulation of the instruction by determining a first source operand (src0) and a second source operand (src1). For example, trace emulator 430 may determine the first source operand based on a first GPU state, such as a GRF state, associated with a hardware thread that executed the GPU kernel instruction. In some examples, in response to executing source code 1400, trace emulator 430 may determine the second source operand based on a second GPU state, such as an ARF state, associated with a hardware thread that executed the GPU kernel instruction.

[0153] In some examples, in response to executing source code 1400, trace emulator 430 may emulate the instruction by determining an execution mask (exec_mask), a destination register (dst), and a next IP register (next_ip). In some examples, in response to executing source code 1400, trace emulator 430 may commit a new GPU state based on the GPU state, the destination register, and the next IP register to process. For example, trace emulator 430 may store the new GPU state for subsequent processing and / or analysis.

[0154] Figure 15 is a flowchart representing machine-readable instructions 1500 that may be executed to implement the GLIT engine 102 of Figure 1 and / or Figure 4 to improve GPU operation. The machine-readable instructions 1500 of Figure 15 begin at block 1502, where the GLIT engine 102 instruments a kernel executed by a graphics processing unit (GPU). For example, the instruction generator 410 (Figure 4) may instrument the first kernel 106 of Figure 1 by inserting profiling instructions 104A-104C of Figure 1 to generate the second kernel 108 of Figure 1.

[0155] At block 1504, the GLIT engine 102 sends the instrumented kernel to the GPU for execution. For example, the instruction generator 410 may provide the second kernel 108 for storage in the memory 116 of FIG. 1. In some examples, the GPU 110 of FIG. 1 may retrieve the second kernel 108 from the instruction generator 410, the GPU driver 122 of FIG. 1, and / or the memory 116.

[0156] At block 1506, the GLIT engine 102 obtains a GPU long instruction trace (GLIT) from the GPU in response to the GPU executing the instrumented kernel. For example, in response to obtaining the second kernel 108, the GPU 110 may execute the second kernel 108. In some examples, in response to executing the second kernel 108, the GPU 110 may generate the GLIT 112 of FIG. 1, the GLIT 600 of FIG. 6, etc. For example, the GPU 110 may encode the GLIT 112 of FIG. 6, the GLIT 600 of FIG. 6, etc. in a binary format such as the GLIT format 300 of FIG. 3.

[0157] At block 1508, the GLIT engine 102 emulates GLIT. For example, the trace extractor 420 (FIG. 4) can decode an encoded binary kernel that can implement GLIT 112, GLIT 600 of FIG. 6, etc. In some examples, the trace emulator 430 (FIG. 4) can instrument the emulation routines to provide GPU state before and / or after executing the emulation routines to the application 120 of FIG. 1 via one or more APIs. An exemplary process that may be performed to implement block 1508 is described below in connection with FIG. 16.

[0158] In block 1510, the GLIT engine 102 determines operational parameters of the GPU based on the emulated GLIT. For example, the trace analyzer 440 (FIG. 4) can determine a GPU state, a run-time parameter, a busy-time parameter, an idle-time parameter, an occupied-time parameter, or a utilization parameter based on the emulation of GLIT 112, GLIT 600, etc.

[0159] At block 1512, the GLIT engine 102 determines whether to adjust the GPU workload based on the operating parameters. For example, the hardware configurer 450 (FIG. 4) may determine to increase the number of instructions executed by the GPU 110 in response to determining that the utilization of one or more of the threads 208 of FIG. 2 and / or, more generally, the GPU slice 200 of FIG. 2 is below a utilization threshold. In some examples, the hardware configurer 450 may determine that one or more of the threads 208 are underutilized based on the corresponding GPU state not changing value in response to the allocation of the second kernel 108 to the GPU 110. In some examples, in response to determining that one or more of the threads 208 are underutilized and / or underutilized based on the utilization threshold not being met, the hardware configurer 450 may increase the number of instructions executed by one or more of the threads 208.

[0160] If, at block 1512, the GLIT engine 102 determines not to adjust the GPU workload based on the operating parameters, control proceeds to block 1516 to determine whether to generate another instrumented kernel. If, at block 1512, the GLIT engine 102 determines to adjust the GPU workload based on the operating parameters, at block 1514, the GLIT engine 102 invokes the GPU driver to adjust the GPU workload. For example, the hardware configurer 450 may instruct the GPU driver 122 to increase the number of instructions executed by the GPU 110, decrease the number of instructions executed by the GPU 110, adjust the scheduling of the second kernel 108 among one or more of the threads 208 of one or more of the execution units 204 of FIG. 2, and / or a combination thereof.

[0161] In response to invoking the GPU driver to adjust the GPU workload at block 1514, the GLIT engine 102 determines whether to generate another instrumented kernel at block 1516. For example, the instruction generator 410 may determine to instrument a different kernel than the first kernel 106 of FIG. 1. In some examples, the instruction generator 410 determines to re-instrument the first kernel 106 by adding, subtracting, and / or modifying one or more of the profiling instructions 104A-104C, adding, subtracting, and / or modifying one or more of the kernel instructions (e.g., INSTR1, INSTR2, etc. of FIG. 1), and / or by a combination thereof.

[0162] At block 1518, the GLIT engine 102 determines whether to continue analyzing the GPU. For example, the trace emulator 430 may determine to continue analyzing the GPU 110 to determine operating parameters associated with the GPU 110. In some examples, the trace emulator 430 may determine to continue the analysis by re-initiating and / or re-emulating the GLIT 112 of FIG. 1, the GLIT 600 of FIG. 6, etc. If the GLIT engine 102 determines at block 1518 to continue analyzing the GPU, control returns to block 1506 to obtain another GLIT from the GPU in response to the GPU executing the instrumented kernel, or otherwise, the example machine-readable instructions 1500 of FIG. 15 end.

[0163] FIG. 16 is a flowchart representing machine-readable instructions 1600 that may be executed to implement the GLIT engine 102 of FIG. 1 and / or FIG. 4 to emulate one or more exemplary GLITs. In some examples, the machine-readable instructions 1600 may implement block 1508 of FIG. 15. The machine-readable instructions 1600 of FIG. 16 begin at block 1602, where the GLIT engine 102 selects a graphics processing unit (GPU) long instruction trace (GLIT) to emulate. For example, the trace extractor 420 (FIG. 4) may select a first one of the GLITs 112 of FIG. 1, a first one of the GLITs 470 of FIG. 4, etc. to emulate. In some examples, the first one of the GLITs 112 may include one or more binary kernels, including a first binary kernel. In some examples, the first binary kernel may include data corresponding to and / or otherwise associated with the GLIT 600 of FIG. 6. In such an example, the first binary kernel may have a binary format such as GLIT format 300 of FIG.

[0164] At block 1604, the GLIT engine 102 decodes the GLIT and generates decoded GLIT data including routines to be executed by the GPU. For example, the trace extractor 420 may decode the first binary kernel to generate and / or otherwise output records 126 of FIG. 1, records 302 of FIG. 3, etc. In some examples, the trace extractor 420 may identify kernel instruction static data 1002 of FIG. 10 based on records 126 of FIG. 1, records 302 of FIG. 3, etc. For example, the trace extractor 420 may identify routines to be executed by the GPU 110, such as an add instruction, a multiply instruction, a send instruction, a read send instruction, etc., and / or combinations thereof.

[0165] At block 1606, the GLIT engine 102 stores the decoded GLIT data based on the instruction identifier. For example, the trace extractor 420 may use the instruction identifier decoded from the first binary kernel as an index to store the kernel instruction static data 1002. In some examples, the trace extractor 420 may store the decoded GLIT data in storage 460 (FIG. 4).

[0166] In block 1608, the GLIT engine 102 identifies an emulation routine based on an identifier of the routine executed by the GPU. For example, the trace extractor 420 may identify a first one of the emulation routines in the emulation routine table 1006 based on an opcode corresponding to a first one of the routines in the kernel instruction static data 1002.

[0167] In block 1610, the GLIT engine 102 stores an association of at least one of an instruction identifier or an emulation routine. For example, the trace extractor 420 may associate one or more of the instruction identifiers (e.g., INST 0, INST 1, INST 2, etc., of FIG. 10 ) of the kernel instruction static data 1002 of FIG. 10 , one or more of the opcodes (e.g., OPCODE 0, OPCODE 1, OPCODE 2, etc.) of the opcode emulation table 1004 of FIG. 10 , or one or more of the emulation routines (e.g., ADD_EMUL, SUB_EMUL, MUL_EMUL, etc.) of the emulation routine table 1006 of FIG. 10 . In some examples, the trace extractor 420 may store the association in the storage 460.

[0168] In block 1612, the GLIT engine 102 instruments the emulation routines with the callback routines. For example, the trace emulator 430 (FIG. 4) can instrument one or more of the emulation routines included in the emulation routine table 1006 of FIG. 10 by inserting example instrumentation instructions, such as the first instrumentation routine 1302 of FIG. 13 and / or the second instrumentation routine 1304 of FIG. 13, into the source code 1300 of FIG. 13.

[0169] In block 1614, the GLIT engine 102 registers callback routines to call application programming interfaces (APIs). For example, the trace emulator 430 can register instrumented ones of the emulation routines in the emulation routine table 1006 with the application 120 of FIG. 1, the OS running on the CPU 118 of FIG. 1, and / or a combination thereof.

[0170] At block 1616, the GLIT engine 102 executes an instrumentation emulation routine to call an API to observe the GPU state. For example, in response to executing a registered callback routine included in the instrumentation emulation routine, the trace emulator 430 may execute the registered callback routine to call one or more APIs to observe the GPU state. In some examples, the GPU state may correspond to the GPU states 1008, 1010 of FIG. 10. For example, in response to calling the one or more APIs, the trace emulator 430 may observe a first value of the GPU state of one of the threads 208 before executing the second kernel 108 and / or a second value of the GPU state of that one of the threads 208 after executing the second kernel 108.

[0171] At block 1618, the GLIT engine 102 determines whether to select another GLIT to emulate. For example, the trace emulator 430 and / or the trace analyzer 440 may determine to select another one of the GLITs 112 to simulate. If at block 1618 the GLIT engine 102 determines to select another GLIT to process, control may return to block 1602 to select another GLIT to emulate. If at block 1618 the GLIT engine 102 determines not to select another GLIT to emulate, control may return to block 1510 of the example machine-readable instructions 1500 of FIG. 15 to determine operating parameters of the GPU based on the emulated GLIT. Additionally or alternatively, the example machine-readable instructions 1600 of FIG. 16 may end.

[0172] FIG. 17 is a flowchart representing machine-readable instructions 1700 that may be executed to implement the GLIT engine 102 of FIG. 1 and / or FIG. 4 to improve GPU operation. The machine-readable instructions 1700 of FIG. 17 begin at block 1702, where the GLIT engine 102, and / or more generally, the CPU 118 of FIG. 1, inserts profiling routines into kernels that include graphics processing unit (GPU) instructions to be executed by the GPU. For example, the instruction generator 410 (FIG. 4) may insert profiling instructions 104A-104C into the first kernel 106 to generate the second kernel 108 of FIG. 1, which is executed by the GPU 110 of FIG. 1. In some examples, the first kernel 106 and the second kernel 108 include GPU instructions such as an add instruction, a multiply instruction, a send instruction, a read send instruction, and / or combinations thereof.

[0173] At block 1704, the GPU 110 distributes the kernel for execution by the hardware threads (HWTs) of the GPU. For example, the instruction generator 410 may provide the second kernel 108 for storage in the memory 116 of FIG. 1. In some examples, the GPU 110 of FIG. 1 may retrieve the second kernel 108 from the instruction generator 410, the GPU driver 122 of FIG. 1, and / or the memory 116. For example, the local thread dispatcher 220 of FIG. 2 may obtain the second kernel 108 and distribute the second kernel to one or more of the threads 208 for execution.

[0174] At block 1706, GPU 110 determines a first register value of each first register of a general-purpose register file (GRF) of the HWT. For example, a first thread of threads 208 may determine one or more first register values ​​of one or more first registers of a first GRF implemented by the first thread. In some examples, a second thread of threads 208 may determine one or more second register values ​​of one or more second registers of a second GRF implemented by the second thread.

[0175] At block 1708, the GPU 110 determines second register values ​​of respective second registers of an architectural register file (ARF) of the HWT. For example, a first thread of the threads 208 may determine one or more first register values ​​of one or more first registers of a first ARF implemented by the first thread. In some examples, a second thread of the threads 208 may determine one or more second register values ​​of one or more second registers of a second ARF implemented by the second thread.

[0176] At block 1710, GPU 110 determines a third register value in response to the HWT executing the GPU instruction. For example, a first thread of threads 208 may determine one or more fifth register values ​​of one or more respective first destination registers in response to the first thread executing a SEND instruction to sampler 216 of FIG. 2, cache memory 210 of FIG. 2, etc. In some examples, a second thread of threads 208 may determine one or more sixth register values ​​of one or more respective second destination registers in response to the second thread executing a SEND instruction to sampler 216 of FIG. 2, cache memory 210 of FIG. 2, etc.

[0177] At block 1712, GPU 110 stores the first register value, the second register value, the third register value, and the GPU instruction in a GPU long instruction trace (GLIT). For example, a first thread of threads 208 may store at least one of one or more first register values, one or more third register values, one or more fifth register values, or one or more GPU instructions in an encoded binary file that can implement a GLIT, such as one of GLITs 112 of FIG. 1, one of GLITs 470 of FIG. 4, or GLIT 600 of FIG. 6. In some examples, a second thread of threads 208 may store at least one of one or more second register values, one or more fourth register values, one or more sixth register values, or one or more GPU instructions in an encoded binary file.

[0178] In block 1714, the GLIT engine 102, and / or more generally the CPU 118, inserts callback routines into routines and calls APIs to provide information from GLIT to the application. For example, the trace emulator 430 (FIG. 4) can insert the first instrumentation routine 1302 and / or the second instrumentation routine 1304 of FIG. 13 into the source code 1300 of FIG. 13 to provide data from GLIT 112, such as GPU state, to the application 120 of FIG. 1 via one or more APIs.

[0179] In block 1716, the GLIT engine 102, and / or more generally the CPU 118, determines operating parameters of the GPU based on the GLIT, including the GLIT utilization. For example, the trace analyzer 440 (FIG. 4) may determine one or more operating parameters of the GPU 110, including the utilization of the GPU 110, based on data from the GLIT 112.

[0180] At block 1718, the GLIT engine 102, and / or more generally the CPU 118, compares the operating parameter to a threshold. For example, the trace analyzer 440 may compare the utilization to a threshold, such as a utilization threshold. In some examples, the trace analyzer 440 may compare the busy time, occupancy, etc. of the GPU 110 to a busy time threshold, occupancy threshold, etc.

[0181] At block 1720, the GLIT engine 102, and / or more generally, the CPU 118, adjusts the number of computational tasks executed by the GPU based on the comparison. For example, the hardware configurer 450 (FIG. 4) may determine to increase the number of computational tasks executed by the GPU 110 or a different GPU based on the comparison of the utilization to a utilization threshold. In some examples, in response to determining that 70% utilization of the GPU 110 is below a 90% utilization threshold of the GPU 110, thereby not meeting the utilization threshold, the hardware configurer 450 may instruct and / or cause the GPU driver 110 of FIG. 1 to increase the number of computational tasks, kernels, etc., executed by the GPU 110. In response to adjusting the number of computational tasks executed by the GPU based on the comparison at block 1720, the example machine-readable instructions 1700 of FIG. 17 end.

[0182] Figure 18 is a block diagram of an exemplary processor platform 1800 configured to execute the instructions of Figures 11-17 to implement the GLIT engine 102 of Figure 1 and / or Figure 4. The processor platform 1800 can be, for example, a server, a personal computer, a workstation, a self-learning machine (e.g., neural network), or any other type of computing device.

[0183] The processor platform 1800 of the illustrated example includes a processor 1812. The processor 1812 of the illustrated example is hardware. For example, the processor 1812 can be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers of any desired family or manufacturer. The hardware processor may be a semiconductor-based (e.g., silicon-based) device. In this example, the processor 1812 implements the example instruction generator 410, example trace extractor 420, example trace emulator 430, example trace analyzer 440, and example hardware configurer 450 of FIG. 4 .

[0184] The processor 1812 of the illustrated example includes a local memory 1813 (e.g., a cache). The processor 1812 of the illustrated example communicates with a main memory, including a volatile memory 1814 and a non-volatile memory 1816, via a bus 1818. The volatile memory 1814 may be implemented by Synchronous Dynamic Random Access Memory (SDRAM), Dynamic Random Access Memory (DRAM), RAMBUS® Dynamic Random Access Memory (RDRAM), and / or any other type of random access memory device. The non-volatile memory 1816 may be implemented by flash memory and / or any other desired type of memory device. Access to the main memory 1814, 1816 is controlled by a memory controller.

[0185] The processor platform 1800 of the depicted example also includes an interface circuit 1820. The interface circuit 1820 may be implemented with any type of interface standard, such as an Ethernet interface, a Universal Serial Bus (USB), a Bluetooth interface, a Near Field Communication (NFC) interface, and / or a PCI Express interface.

[0186] In the illustrated example, one or more input devices 1822 are connected to the interface circuit 1820. The input devices 1822 allow a user to input data and / or commands to the processor 1812. The input devices may be implemented by, for example, audio sensors, microphones, cameras (still or video), keyboards, buttons, mice, touchscreens, trackpads, trackballs, isopoint devices, and / or voice recognition systems.

[0187] One or more output devices 1824 are also connected to the interface circuit 1820 of the illustrated example. The output device 1824 may be implemented, for example, by a display device (e.g., a light emitting diode (LED), an organic light emitting diode (OLED), a liquid crystal display (LCD), a cathode ray tube (CRT) display, an in-field switching (IPS) display, a touch screen, etc.), a tactile output device, a printer, and / or a speaker. Accordingly, the interface circuit 1820 of the illustrated example typically includes a graphics driver card, a graphics driver chip, and / or a graphics driver processor.

[0188] The interface circuitry 1820 of the illustrated example also includes communications devices such as transmitters, receivers, transceivers, modems, residential gateways, wireless access points, and / or network interfaces to facilitate the exchange of data with external machines (e.g., any type of computing device) over the network 1826. Communications may be via, for example, an Ethernet connection, a digital subscriber line (DSL) connection, a telephone line connection, a coaxial cable system, a satellite system, a line-of-sight wireless system, a cellular telephone system, etc.

[0189] The processor platform 1800 of the illustrated example also includes one or more mass storage devices 1828 for storing software and / or data. Examples of such mass storage devices 1828 include floppy disk drives, hard drive disks, compact disk drives, Blu-ray disk drives, redundant array of independent disks (RAID), and digital versatile disk (DVD) drives. In this example, the one or more mass storage devices 1828 implement the storage 460 of FIG. 4, which stores the example GLIT 470 of FIG. 4.

[0190] The machine-executable instructions 1832 of Figures 11-17 may be stored in the mass storage device 1828, in the volatile memory 1814, in the non-volatile memory 1816, and / or on a removable non-transitory computer-readable storage medium such as a CD or DVD.

[0191] A block diagram illustrating an exemplary software distribution platform 1905 for distributing software, such as the example computer-readable instructions 1832 of FIG. 18, to third parties is shown in FIG. 19. The example software distribution platform 1905 may be implemented by any computer server, data facility, cloud service, etc., capable of storing software and transmitting it to other computing devices. The third party may be a customer entity that owns and / or operates the software distribution platform. For example, the entity that owns and / or operates the software distribution platform may be a developer, seller, and / or licensor of software, such as the example computer-readable instructions 1832 of FIG. 18. The third party may be a consumer, user, retailer, OEM, etc., that purchases and / or licenses the software for use and / or resale and / or sublicensing. In the illustrated example, the software distribution platform 1905 includes one or more servers and one or more storage devices. The storage device stores computer-readable instructions 1832, which may correspond to the example computer-readable instructions 1100, 1200, 1300, 1400, 1500, 1600, and 1700 of Figures 11-17, as described above. One or more servers of the example software distribution platform 1905 are in communication with the network 1910, which may correspond to any one or more of the Internet and / or any one or more of the example networks 1826 described above. In some examples, the one or more servers respond to requests to send software to requesters as part of a commercial transaction. Payment for delivery, sale, and / or licensing of the software may be processed by one or more servers of the software distribution platform 1905 and / or via a third-party payment entity. The servers enable purchasers and / or licensors to download the computer-readable instructions 1832 from the software distribution platform 1905.For example, software that may correspond to the example computer-readable instructions 1832 of Figure 18 may be downloaded to the example processor platform 1800, which executes the computer-readable instructions 1832 to implement the example GLIT engine 102 of Figure 1 and / or Figure 4. In some examples, one or more servers of the software distribution platform 1905 periodically provide, transmit, and / or force updates to the software (e.g., the example computer-readable instructions 1832 of Figure 18) to ensure that improvements, patches, updates, etc. are distributed and applied to the software at end-user devices.

[0192] By now it should be appreciated that exemplary systems, methods, apparatus, and articles of manufacture are disclosed that can be used to improve the operation of a hardware processor, such as a GPU. The disclosed systems, methods, apparatus, and articles of manufacture define LITs for different hardware processors, such as GPUs, and facilitate the development of flexible analysis tools that can be developed in high-level languages ​​such as C, C++, and the like. Advantageously, such analysis tools can analyze the operation of the hardware processor to generate profiling data at the granularity level of a single hardware thread of the hardware processor. Advantageously, the disclosed systems, methods, apparatus, and articles of manufacture can generate multi-threaded traces, since the same kernel can be distributed across multiple threads of the hardware processor.

[0193] The disclosed systems, methods, apparatus, and articles of manufacture can improve kernel debugging, development of models of memory, cache, samplers, etc., which can be utilized to improve GPU operation. For example, the disclosed systems, methods, apparatus, and articles of manufacture improve the efficiency of using a computing device by improving the operation of the computing device's hardware processor by increasing the amount of computational tasks performed by the hardware processor. Thus, the disclosed methods, apparatus, and articles of manufacture are directed to one or more improvements in computer functionality.

[0194] Disclosed herein are exemplary methods, apparatus, systems, and articles of manufacture for generating long instruction traces of a graphics processing unit. Further examples and combinations thereof include:

[0195] Example 1 includes an apparatus including at least one memory and at least one processor that executes instructions to at least: identify a first routine based on an identifier of a second routine executed by a graphics processing unit (GPU), the first routine being based on emulation of the second routine; execute a first routine that determines a first value of a GPU state of the GPU, the first routine having (i) a first argument associated with the second routine and (ii) a second argument that corresponds to a second value of the GPU state before executing the first routine; and control a workload of the GPU based on the first value of the GPU state.

[0196] Example 2 includes the apparatus of example 1, wherein the GPU state is a state of a first register in an architectural register file associated with a hardware thread of the GPU or a second register in a general purpose register file of the hardware thread.

[0197] Example 3 includes the apparatus of any one of Examples 1-2, wherein the identifier is a first identifier extracted from the encoded binary file, and wherein the at least one processor inserts one or more profile routines into a kernel executed by a hardware thread of the GPU; and determines a value of 1, a second value, and a hardware thread identifier from a long instruction trace generated by the hardware thread in response to execution of the one or more profile routines by the hardware thread, the long instruction trace corresponding to a GPU register value after execution of the kernel by the hardware thread and the second value corresponding to a GPU register value before execution of the kernel by the hardware thread, and the hardware thread identifier identifies the hardware thread.

[0198] Example 4 includes the apparatus of any one of Examples 1-3, wherein the hardware thread is a first hardware thread, the long instruction trace is a first long instruction trace associated with the first hardware thread, the encoded binary file includes the first long instruction trace and one or more second long instruction traces associated with one or more second hardware threads, and the encoded binary file represents a multi-threaded GPU trace.

[0199] Example 5 includes the apparatus of any one of Examples 1-4, wherein the kernel includes a device access instruction executed by the hardware thread, and wherein at least one processor is configured to: determine one or more first register values ​​for one or more respective first registers of a general purpose register file of the GPU; determine one or more second register values ​​for one or more respective second registers of an architectural register file of the GPU; and store the one or more first register values, the one or more second register values, the one or more third register values, and the device access instruction in a long instruction trace, wherein the one or more third register values ​​correspond to one or more respective destination registers associated with the device access instruction.

[0200] Example 6 includes the apparatus of any one of Examples 1-5, wherein the at least one processor determines a utilization of the GPU based on the first GPU state, compares the utilization to a threshold, and, in response to determining that the threshold is not met based on the comparison, performs at least one of adjusting the second routine or increasing the number of computational tasks performed by the GPU to control a workload of the GPU.

[0201] Example 7 includes the apparatus of any one of Examples 1-6, wherein the first routine is an instrumentation routine including an emulation routine, and the at least one processor inserts a first callback routine into the instrumentation routine before the emulation routine, the first callback routine calling a first application programming interface (API) to provide the second GPU state to the application, and inserts a second callback routine into the instrumentation routine after the emulation routine, the second callback routine calling the first API or the second API to provide the first GPU state to the application.

[0202] Example 8 includes at least one storage device containing instructions that, when executed, cause at least one processor to: identify a first routine based on an identifier of a second routine executed by a graphics processing unit (GPU), the first routine being based on emulation of the second routine; execute a first routine that determines a first value of a GPU state of the GPU, the first routine having (i) a first argument associated with the second routine and (ii) a second argument that corresponds to a second value of the GPU state before executing the first routine; and control a workload of the GPU based on the first value of the GPU state.

[0203] Example 9 includes the at least one storage device of Example 8, wherein the GPU state is the state of a first register in an architectural register file associated with a hardware thread of the GPU or a second register in a general-purpose register file of the hardware thread.

[0204] Example 10 includes the storage device of any one of Examples 8-9, wherein the identifier is a first identifier extracted from the encoded binary file, and the instructions, when executed, cause at least one processor to: insert one or more profile routines into a kernel executed by a hardware thread of a GPU; and determine a value of 1, a second value, and a hardware thread identifier from a long instruction trace generated by the hardware thread in response to execution of the one or more profile routines by the hardware thread, the first value corresponding to a GPU register value after execution of the kernel by the hardware thread and the second value corresponding to a GPU register value before execution of the kernel by the hardware thread, and the hardware thread identifier identifying the hardware thread.

[0205] Example 11 includes at least one storage device described in any one of Examples 8-10, wherein the hardware thread is a first hardware thread, the long instruction trace is a first long instruction trace associated with the first hardware thread, and the encoded binary file includes the first long instruction trace and one or more second long instruction traces associated with one or more second hardware threads, and the encoded binary file represents a multi-threaded GPU trace.

[0206] Example 12 includes at least one storage device described in any one of Examples 8-10, wherein the kernel includes device access instructions executed by the hardware threads that, when executed, cause the at least one processor to: determine one or more first register values ​​for one or more respective first registers of a general purpose register file of the GPU; determine one or more second register values ​​for one or more respective second registers of the GPU's architectural register file; and store the one or more first register values, the one or more second register values, the one or more third register values, and the device access instruction in a long instruction trace, wherein the one or more third register values ​​correspond to one or more respective destination registers associated with the device access instruction.

[0207] Example 13 includes at least one storage device according to any one of Examples 8-12, wherein the instructions, when executed, cause the at least one processor to: determine a utilization of the GPU based on the first GPU state; compare the utilization to a threshold; and, in response to determining that the threshold is not met based on the comparison, perform at least one of adjusting the second routine or increasing the number of computational tasks performed by the GPU to control the workload of the GPU.

[0208] Example 14 includes the at least one storage device of any one of Examples 8-13, wherein the first routine is an instrumentation routine including an emulation routine, and the instructions, when executed, cause the at least one processor to: insert a first callback routine into the instrumentation routine before the emulation routine, where the first callback routine calls a first application programming interface (API) to provide a second GPU state to the application; and insert a second callback routine into the instrumentation routine after the emulation routine, where the second callback routine calls the first API or the second API to provide the first GPU state to the application.

[0209] Example 15 includes an apparatus including: means for identifying a first routine based on an identifier of a second routine executed by a graphics processing unit (GPU), the first routine being based on emulation of the second routine; means for executing the first routine to determine a first value of a GPU state of the GPU, the first routine having (i) a first argument associated with the second routine and (ii) a second argument corresponding to a second value of the GPU state prior to executing the first routine; and means for controlling a workload of the GPU based on the first value of the GPU state.

[0210] Example 16 includes the apparatus of example 15, wherein the GPU state is a state of a first register in an architectural register file associated with a hardware thread of the GPU or a second register in a general purpose register file of the hardware thread.

[0211] Example 17 includes the apparatus of any one of Examples 15-16, further including: means for inserting one or more profile routines into a kernel executed by a hardware thread of the GPU, where the identifier is a first identifier extracted from the encoded binary file; and means for determining, in response to execution of the one or more profile routines by the hardware thread, a first value, a second value, and a hardware thread identifier from a long instruction trace generated by the hardware thread, where the first value corresponds to GPU register values ​​before execution of the kernel by the hardware thread, the second value corresponds to GPU register values ​​after execution of the kernel by the hardware thread, and the hardware thread identifier identifies the hardware thread.

[0212] Example 18 includes the apparatus of any one of Examples 15-17, wherein the hardware thread is a first hardware thread, the long instruction trace is a first long instruction trace associated with the first hardware thread, the encoded binary file includes the first long instruction trace and one or more second long instruction traces associated with one or more second hardware threads, and the encoded binary file represents a multi-threaded GPU trace.

[0213] Example 19 includes the apparatus of any one of Examples 15-18, wherein the kernel includes a device access instruction executed by the hardware thread, and the means for executing includes determining one or more first register values ​​for one or more respective first registers of a general purpose register file of the GPU; determining one or more second register values ​​for one or more respective second registers of an architectural register file of the GPU; and storing the one or more first register values, the one or more second register values, the one or more third register values, and the device access instruction in a long instruction trace, wherein the one or more third register values ​​correspond to one or more respective destination registers associated with the device access instruction.

[0214] Example 20 includes the apparatus of any one of Examples 15-19, further including means for determining a utilization rate of the GPU based on the first GPU state, and wherein the means for controlling, in response to determining that the utilization rate does not meet a threshold, performs at least one of adjusting the second routine or increasing the number of increased computational tasks performed by the GPU to control the workload of the GPU.

[0215] Example 21 includes at least one storage device according to any one of Examples 15-20, wherein the first routine is an instrumentation routine including an emulation routine, and the means for executing includes inserting a first callback routine into the instrumentation routine before the emulation routine, the first callback routine calling a first application programming interface (API) to provide the second GPU state to the application, and inserting a second callback routine into the instrumentation routine after the emulation routine, the second callback routine calling the first API or the second API to provide the first GPU state to the application.

[0216] Example 22 includes a system including: a graphics processing unit (GPU) having a hardware thread that determines a first value of a GPU state; executes a GPU routine included in a kernel to determine a second value of the GPU state; and generates a long instruction trace that includes the GPU routine, the first value, and the second value; and a central processing unit (CPU) that inserts one or more profile routines into the kernel; identifies the first routine based on an identifier of the GPU routine, the first routine being based on emulation of the GPU routine; executes the first routine to replay execution of the GPU routine that determines the second value of the GPU state, the first routine having (i) a first argument associated with the GPU routine and (ii) a second argument that corresponds to the first value of the GPU state; and controls the workload of the GPU based on the execution of the first routine.

[0217] Example 23 includes the system of example 22, wherein the GPU state is a state of a second register in an architectural register file associated with a hardware thread of the GPU or a second register of a general purpose register file of the hardware thread.

[0218] Example 24 includes the system of any one of Examples 22-23, wherein the identifier is a first identifier extracted from an encoded binary file, the encoded binary file including a long instruction trace, and the CPU determines the first value, the second value, and a hardware thread identifier from the encoded binary file, the hardware thread identifier identifying a hardware thread.

[0219] Example 25 includes the system of any one of Examples 22-24, wherein the hardware thread is a first hardware thread, the long instruction trace is a first long instruction trace associated with the first hardware thread, the encoded binary file includes the first long instruction trace and one or more second long instruction traces associated with one or more second hardware threads, the one or more second hardware threads generate the one or more second long instruction traces in response to one or more executions of the kernel, and the encoded binary file represents a multi-threaded GPU trace.

[0220] Example 26 includes the system of any one of Examples 22-25, wherein the kernel includes a device access instruction executed by the hardware thread, and the GPU determines one or more first register values ​​for one or more respective first registers of a general purpose register file of the GPU; determines one or more second register values ​​for one or more respective second registers of one or more architectural register files of the GPU; and stores the one or more first register values, the one or more second register values, the one or more third register values, and the device access instruction in a long instruction trace, wherein the one or more third register values ​​correspond to one or more respective destination registers associated with the device access instruction.

[0221] Example 27 includes the system of any one of Examples 22-26, wherein the CPU determines a utilization of the GPU based on the first GPU state, compares the utilization to a threshold, and, in response to determining that the threshold is not met based on the comparison, performs at least one of adjusting a GPU routine or increasing the number of computational tasks performed by the GPU to control the workload of the GPU.

[0222] Example 28 includes the system of any one of Examples 22-27, wherein the first routine is an instrumentation routine that includes an emulation routine and further includes an application, and the CPU inserts a first callback routine into the instrumentation routine before the emulation routine, the first callback routine calling a first application programming interface (API) to provide a second GPU state to the application, and inserts a second callback routine into the instrumentation routine after the emulation routine, the second callback routine calling the first API or the second API to provide the first GPU state to the application.

[0223] Example 29 includes a method including: identifying a first routine based on an identifier of a second routine to be executed by a graphics processing unit (GPU), the first routine being based on emulation of the second routine; executing a first routine to determine a first value of a GPU state of the GPU, the first routine having (i) a first argument associated with the second routine and (ii) a second argument corresponding to a second value of the GPU state before executing the first routine; and controlling a workload of the GPU based on the first value of the GPU state.

[0224] Example 30 includes the method of example 29, wherein the GPU state is a state of a first register in an architectural register file associated with a hardware thread of the GPU or a second register in a general purpose register file of the hardware thread.

[0225] Example 31 includes the method of any one of Examples 29-30, wherein the identifier is a first identifier extracted from the encoded binary file, further including: inserting one or more profile routines into a kernel executed by a hardware thread of the GPU; determining a second value of the GPU state before the hardware thread executes the kernel; and generating a long instruction trace in response to the hardware thread executing the kernel, the long instruction trace including the first value of the GPU state and the second identifier corresponding to the hardware thread.

[0226] Example 32 includes the method of any one of Examples 29-31, wherein the hardware thread is a first hardware thread, the long instruction trace is a first long instruction trace associated with the first hardware thread, the encoded binary file includes the first long instruction trace and one or more second long instruction traces associated with one or more second hardware threads, and the encoded binary file represents a multi-threaded GPU trace.

[0227] Example 33 includes the method of Examples 29-32, wherein the kernel includes a device access instruction executed by the hardware thread, and further includes determining one or more first register values ​​for one or more respective first registers of a general purpose register file of the GPU; determining one or more second register values ​​for one or more respective second registers of an architectural register file of the GPU; and storing the one or more first register values, the one or more second register values, the one or more third register values, and the device access instruction in a long instruction trace, wherein the one or more third register values ​​correspond to one or more respective destination registers associated with the device access instruction.

[0228] Example 34 includes the method of any one of Examples 29-33, further including determining a utilization rate of the GPU based on the first GPU state, comparing the utilization rate to a threshold value, and, in response to determining that the threshold value is not met based on the comparison, at least one of adjusting a GPU routine or increasing the number of computational tasks performed by the GPU to control the workload of the GPU.

[0229] Example 35 includes the method of any one of Examples 29-34, wherein the first routine is an instrumentation routine that includes an emulation routine, and further includes inserting a first callback routine into the instrumentation routine before the emulation routine, the first callback routine calling a first application programming interface (API) to provide a second GPU state to the application, and inserting a second callback routine into the instrumentation routine after the emulation routine, the second callback routine calling the first API or the second API to provide the first GPU state to the application.

[0230] Although certain exemplary systems, methods, apparatus, and articles of manufacture are disclosed herein, the scope of this patent is not limited thereto. On the contrary, this patent covers all systems, methods, apparatus, and articles of manufacture that fall within the scope of the claims of this patent.

[0231] The following claims are hereby incorporated by reference into the Detailed Description, with each claim standing on its own as a separate embodiment of this disclosure.

Claims

1. 1. An apparatus for profiling hardware, comprising: at least one memory; At least one processor, comprising at least: identifying a first routine based on an identifier of a second routine executed by a graphics processing unit (GPU), the first routine being based on an emulation of the second routine; executing the first routine to determine a first value of a GPU state of the GPU, the first routine having (i) a first argument associated with the second routine, and (ii) a second argument corresponding to a second value of the GPU state prior to executing the first routine, the first value and the second value being based on a long instruction trace generated by the GPU; and controlling a workload of the GPU based on the first value of the GPU state.

2. The apparatus of claim 1 , wherein the GPU state is a state of a first register in an architectural register file associated with a hardware thread of the GPU or a second register in a general purpose register file of the hardware thread.

3. The identifier is a first identifier extracted from an encoded binary file, and the at least one processor: inserting one or more profile routines into a kernel executed by a hardware thread of the GPU; and determining the first value, the second value, and a hardware thread identifier from the long instruction trace, the long instruction trace being generated by the hardware thread in response to execution of the one or more profile routines by the hardware thread, the first value corresponding to a GPU register value after execution of the kernel by the hardware thread, the second value corresponding to the GPU register value before execution of the kernel by the hardware thread, and the hardware thread identifier identifying the hardware thread.

4. 4. The apparatus of claim 3, wherein the hardware thread is a first hardware thread, the long instruction trace is a first long instruction trace associated with the first hardware thread, the encoded binary file includes the first long instruction trace and one or more second long instruction traces associated with one or more second hardware threads, and the encoded binary file represents a multi-threaded GPU trace.

5. The kernel includes device access instructions executed by the hardware threads, and the at least one processor: determining one or more first register values ​​of one or more respective first registers of a general purpose register file of the GPU; determining one or more second register values ​​for one or more respective second registers of an architectural register file of the GPU; and storing the one or more first register values, the one or more second register values, the one or more third register values, and the device access instruction in the long instruction trace, wherein the one or more third register values ​​correspond to one or more respective destination registers associated with the device access instruction.

6. The at least one processor determining a utilization of the GPU based on a first GPU state; comparing the utilization rate to a threshold; and in response to determining based on the comparison that the threshold is not met, at least one of adjusting the second routine or increasing a number of computational tasks performed by the GPU to control a workload of the GPU.

7. the first routine is an instrumentation routine including an emulation routine, and the at least one processor: inserting a first callback routine into the instrumentation routine before the emulation routine, the first callback routine calling a first application programming interface (API) to provide a second GPU state to an application; and inserting a second callback routine into the instrumentation routine after the emulation routine, the second callback routine calling the first API or a second API to provide a first GPU state to the application.

8. at least one storage device containing instructions that, when executed, cause at least one processor to: identifying a first routine based on an identifier of a second routine executed by a graphics processing unit (GPU), the first routine being based on an emulation of the second routine; executing the first routine to determine a first value of a GPU state of the GPU, the first routine having (i) a first argument associated with the second routine, and (ii) a second argument corresponding to a second value of the GPU state prior to executing the first routine, the first value and the second value being based on a long instruction trace generated by the GPU; and controlling a workload of the GPU based on the first value of the GPU state.

9. 9. The at least one storage device of claim 8, wherein the GPU state is the state of a first register in an architectural register file associated with a hardware thread of the GPU or a second register of a general purpose register file of the hardware thread.

10. The identifier is a first identifier extracted from an encoded binary file, and the instructions, when executed, cause the at least one processor to: inserting one or more profile routines into a kernel executed by a hardware thread of the GPU; and determining the first value, the second value, and a hardware thread identifier from the long instruction trace, the long instruction trace being generated by the hardware thread in response to execution of the one or more profile routines by the hardware thread, the first value corresponding to a GPU register value after execution of the kernel by the hardware thread, the second value corresponding to the GPU register value before execution of the kernel by the hardware thread, and the hardware thread identifier identifying the hardware thread.

11. 11. The at least one storage device of claim 10, wherein the hardware thread is a first hardware thread, the long instruction trace is a first long instruction trace associated with the first hardware thread, the encoded binary file includes the first long instruction trace and one or more second long instruction traces associated with one or more second hardware threads, and the encoded binary file represents a multi-threaded GPU trace.

12. The kernel includes device access instructions that are executed by the hardware threads, the instructions, when executed, causing the at least one processor to: determining one or more first register values ​​of one or more respective first registers of a general purpose register file of the GPU; determining one or more second register values ​​for one or more respective second registers of an architectural register file of the GPU; and storing the one or more first register values, the one or more second register values, the one or more third register values, and the device access instruction in the long instruction trace, wherein the one or more third register values ​​correspond to one or more respective destination registers associated with the device access instruction.

13. The instructions, when executed, cause the at least one processor to: determining a utilization of the GPU based on a first GPU state; comparing the utilization rate to a threshold; and, in response to determining based on the comparison that the threshold is not met, at least one of adjusting the second routine or increasing a number of computational tasks performed by the GPU to control a workload of the GPU.

14. The first routine is an instrumentation routine that includes an emulation routine, and the instructions, when executed, cause the at least one processor to: inserting a first callback routine into the instrumentation routine before the emulation routine, the first callback routine calling a first application programming interface (API) to provide a second GPU state to an application; and inserting a second callback routine into the instrumentation routine after the emulation routine, the second callback routine calling the first API or a second API to provide a first GPU state to the application.

15. 1. An apparatus for profiling hardware, comprising: means for identifying a first routine based on an identifier of a second routine executed by a graphics processing unit (GPU), the first routine being based on an emulation of the second routine; means for executing a first routine to determine a first value of a GPU state of the GPU, the first routine having (i) a first argument associated with the second routine, and (ii) a second argument corresponding to a second value of the GPU state prior to executing the first routine, the first value and the second value being based on a long instruction trace generated by the GPU; means for controlling a workload of the GPU based on the first value of the GPU state.

16. 16. The apparatus of claim 15, wherein the GPU state is a state of a first register in an architectural register file associated with a hardware thread of the GPU or a second register in a general purpose register file of the hardware thread.

17. means for inserting one or more profile routines into a kernel executed by a hardware thread of the GPU, the identifier being a first identifier extracted from an encoded binary file; and means for determining, in response to execution of one or more profile routines by the hardware thread, from the long instruction trace produced by the hardware thread, the first value corresponding to a GPU register value before execution of the kernel by the hardware thread, the second value corresponding to the GPU register value after execution of the kernel by the hardware thread, and the hardware thread identifier identifying the hardware thread.

18. 18. The apparatus of claim 17, wherein the hardware thread is a first hardware thread, the long instruction trace is a first long instruction trace associated with the first hardware thread, the encoded binary file includes the first long instruction trace and one or more second long instruction traces associated with one or more second hardware threads, and the encoded binary file represents a multi-threaded GPU trace.

19. The kernel includes a device access instruction to be executed by the hardware thread, and the means for executing includes: determining one or more first register values ​​of one or more respective first registers of a general purpose register file of the GPU; determining one or more second register values ​​for one or more respective second registers of an architectural register file of the GPU; and storing the one or more first register values, the one or more second register values, the one or more third register values, and the device access instruction in the long instruction trace, wherein the one or more third register values ​​correspond to one or more respective destination registers associated with the device access instruction.

20. An apparatus as described in any one of claims 15 to 16, further comprising means for determining a utilization rate of the GPU based on a first GPU state, wherein the means for controlling, in response to a determination that the utilization rate does not meet a threshold, performs at least one of adjusting the second routine or increasing the number of computational tasks performed by the GPU to control the workload of the GPU.

21. the first routine is an instrumentation routine including an emulation routine, and the means for executing inserting a first callback routine into the instrumentation routine before the emulation routine, the first callback routine calling a first application programming interface (API) to provide a second GPU state to an application; and inserting a second callback routine into the instrumentation routine after the emulation routine, the second callback routine calling the first API or a second API to provide a first GPU state to the application.

22. 1. A system for profiling hardware, comprising: A graphics processing unit (GPU) having a hardware thread, the hardware thread comprising: determining a first value of a GPU state; executing a GPU routine included in a kernel to determine a second value of the GPU state; generating a long instruction trace including the GPU routine, the first value, and the second value; a central processing unit (CPU), inserting one or more profile routines into the kernel; identifying a first routine based on an identifier of the GPU routine, the first routine being based on an emulation of the GPU routine; executing the first routine to replay execution of the GPU routine that determines the second value of the GPU state, the first routine having (i) a first argument associated with the GPU routine, and (ii) a second argument corresponding to the first value of the GPU state; and a CPU configured to control a workload of the GPU based on execution of the first routine.

23. 23. The system of claim 22, wherein the GPU state is the state of a second register in an architectural register file associated with a hardware thread of the GPU or a second register of a general purpose register file of the hardware thread.

24. 24. The system of claim 22, wherein the identifier is a first identifier extracted from an encoded binary file, the encoded binary file including the long instruction trace, and the CPU determines the first value, the second value, and a hardware thread identifier from the encoded binary file, the hardware thread identifier identifying the hardware thread.

25. 25. The system of claim 24, wherein the hardware thread is a first hardware thread, the long instruction trace is a first long instruction trace associated with the first hardware thread, the encoded binary file includes the first long instruction trace and one or more second long instruction traces associated with one or more second hardware threads, the one or more second hardware threads generating the one or more second long instruction traces in response to one or more executions of the kernel, and the encoded binary file represents a multithreaded GPU trace.

Images