Task Graph Scheduling for Workload Processing

Abstract

Techniques are provided for scheduling the operation of a task graph on a processing device, including receiving a task graph specifying one or more paths, one or more resources, and one or more directed edges between the paths and the resources, identifying independent and dependent paths in the task graph, scheduling commands to execute the paths based on performance criteria of the processing device, and sending the scheduled commands to the processing device for scheduled execution.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of U.S. Patent Application No. 16 / 888,540, filed May 29, 2020, and is incorporated by reference as if fully set forth herein. [Background technology]

[0002] A three-dimensional ("3D") graphics processing pipeline performs a series of steps to convert input geometry into a two-dimensional ("2D") image for display on a screen. In some instances, these steps are specified as commands by a host, such as a central processing unit that runs an application. Many rendering workloads require large sequences of commands, and therefore generating and processing these sequences is a complex task.

[0003] A more detailed understanding may be had from the following description, given by way of example in conjunction with the accompanying drawings, in which: [Brief explanation of the drawings]

[0004] [Figure 1] FIG. 1 is a block diagram of an example device in which one or more features of the present disclosure may be implemented. [Figure 2] FIG. 2 illustrates details of the device of FIG. 1, according to an example. [Figure 3] FIG. 3 is a block diagram illustrating additional details of the graphics processing pipeline shown in FIG. [Figure 4A] 4 illustrates an arrangement of a task graph specification processor 402 and a command scheduler 404 for generating a task graph and scheduling workload on a processing device such as the accelerated processing device of FIG. 2. [Figure 4B]FIG. showing the configuration of a task graph specifying processor 402 and a command scheduler 404 for generating a task graph and a scheduling workload on a processing device such as the acceleration processing device of FIG. 2. [Figure 4C] FIG. showing the configuration of a task graph specifying processor 402 and a command scheduler 404 for generating a task graph and a scheduling workload on a processing device such as the acceleration processing device of FIG. 2. [Figure 5] FIG. showing an exemplary task graph. [Figure 6A] FIG. showing generation of a task graph from a task graph specifying program by an exemplary technique. [Figure 6B] FIG. showing generation of a task graph from a task graph specifying program by an exemplary technique. [Figure 7A] FIG. showing generation of a task graph from a task graph specifying program by another exemplary technique. [Figure 7B] FIG. showing generation of a task graph from a task graph specifying program by another exemplary technique. [Figure 8A] FIG. showing scheduling of tasks on a processing device by a task graph. [Figure 8B] FIG. showing scheduling of tasks on a processing device by a task graph. [Figure 9] FIG. showing a flowchart of a method for generating a task graph from a task graph specifying program. [Figure 10] FIG. showing a flowchart of a method for scheduling tasks on a processing device by a task graph. DETAILED DESCRIPTION OF THE INVENTION

[0005] Techniques are provided for scheduling the operation of a task graph on a processing device. The techniques include receiving a task graph that specifies one or more paths, one or more resources, and one or more directed edges between the paths and the resources; identifying independent and dependent paths of the task graph; scheduling commands for executing the paths based on performance criteria of the processing device; and transmitting the scheduled commands to the processing device for execution as scheduled.

[0006] FIG. 1 is a block diagram of an exemplary device 100 that can implement one or more features of the present disclosure. Device 100 can be, for example, but not limited to, a computer, a gaming device, a handheld device, a set-top box, a television, a cellular phone, a tablet computer, or any other computing device. Device 100 includes a processor 102, a memory 104, a storage device 106, one or more input devices 108, and one or more output devices 110. Device 100 also includes one or more input drivers 112 and one or more output drivers 114. Each of the input drivers 112 is embodied as hardware, a combination of hardware and software, or software, and serves to control the input device 108 (e.g., control the operation, receive input from the input driver 112, and provide data to the input driver 112). Similarly, each of the output drivers 114 is embodied as hardware, a combination of hardware and software, or software, and serves to control the output device 110 (e.g., control the operation, receive input from the output driver 114, and provide data to the output driver 114). It should be understood that device 100 can include additional components not shown in FIG. 1.

[0007] In various alternatives, processor 102 may include a central processing unit (CPU), a graphics processing unit (GPU), a CPU and a GPU located on the same die, or one or more processor cores, each of which may be a CPU or a GPU. In various alternatives, memory 104 may be located on the same die as processor 102 or may be located separately from processor 102. Memory 104 may include volatile or non-volatile memory (e.g., random access memory (RAM), dynamic RAM, cache).

[0008] Storage devices 106 include fixed or removable storage devices (e.g., but not limited to, hard disk drives, solid state drives, optical disks, flash drives). Input devices 108 include, but are not limited to, keyboards, keypads, touchscreens, touchpads, detectors, microphones, accelerometers, gyroscopes, biometric scanners, or network connections (e.g., wireless local area network cards for transmitting and / or receiving wireless IEEE 802 signals). Output devices 110 include, but are not limited to, displays, speakers, printers, haptic feedback devices, one or more optics, antennas, or network connections (e.g., wireless local area network cards for transmitting and / or receiving wireless IEEE 802 signals).

[0009] The input driver 112 and the output driver 114 each include one or more hardware, software, and / or firmware components configured to interface with and drive an input device 108 and an output device 110, respectively. The input driver 112 communicates with the processor 102 and the input device 108, enabling the processor 102 to receive inputs from the input device 108. The output driver 114 communicates with the processor 102 and the output device 110, enabling the processor 102 to send outputs to the output device 110. The output driver 114 includes an accelerated processing device (“APD”) 116 coupled to a display device 118, which in some examples is a physical display device or a simulated device that uses a remote display protocol to represent outputs. The APD 116 is configured to receive compute commands and graphics rendering commands from the processor 102, process those compute and graphics rendering commands, and provide pixel outputs to the display device 118 for display. As will be described in more detail below, the APD 116 includes one or more parallel processing units configured to perform computations according to a single-instruction-multiple-data (“SIMD”) paradigm. Thus, various functions are described herein as being performed by, or in conjunction with, the APD 116, but in various alternative examples, the functions described as being performed by the APD 116 are not driven by the host processor (e.g., the processor 102), but are additionally or alternatively performed by other computing devices having similar capabilities configured to provide graphics output to the display device 118. For example, it is contemplated that any processing system configured to perform processing tasks according to the SIMD paradigm may be configured to perform the functions described herein. Alternatively, it is contemplated that a computing system that does not perform processing tasks according to the SIMD paradigm may perform the functions described herein.

[0010] Figure 2 shows details of the device 100 and the APD 116 according to an example. The processor 102 (FIG. 1) executes the operating system 120, the driver 122, and the application 126, and may alternatively or additionally execute other software. The operating system 120 controls various aspects of the device 100, such as managing hardware resources, processing service requests, scheduling and controlling process execution, and performing other operations. The APD driver 122 controls the operation of the APD 116 and sends tasks such as graphics rendering tasks or other work to the APD 116 for processing. The APD driver 122 also includes a just-in-time compiler that compiles programs for execution by processing components of the APD 116 (such as the SIMD unit 138 described in more detail below).

[0011] The APD 116 executes commands and programs for selected functions such as graphic operations and non-graphic operations suitable for parallel processing. The APD 116 can be used to execute graphics pipeline operations such as pixel operations, geometric calculations, and rendering of images to the display device 118 based on commands received from the processor 102. The APD 116 also executes computational processing operations not directly related to graphic operations, such as operations related to video, physical simulation, computational fluid dynamics, or other tasks, based on commands received from the processor 102.

[0012] APD116 includes a computing unit 132 that includes one or more SIMD units 138 configured to perform operations in parallel according to the SIMD paradigm at the request of the processor 102 (or another unit). The SIMD paradigm is one in which multiple processing elements share a single program control flow unit and a program counter and thus execute the same program but can execute that program with different data. In one example, each SIMD unit 138 includes 16 lanes, and each lane can execute the same instruction simultaneously with other lanes within the SIMD unit 138 but can execute that instruction with different data. The lanes can be switched off predictively if not all lanes need to execute a given instruction. Also, prediction can be used to execute programs having a branch control flow. More specifically, for a program having conditional branches or other instructions where the control flow is based on calculations performed by individual lanes, prediction of lanes corresponding to currently unexecuted control flow paths and serial execution of different control flow paths enables any control flow.

[0013] The basic unit of execution within the compute unit 132 is the work item. Each work item represents a single instantiation of a program executing in parallel on a particular lane. Work items can execute simultaneously (or partially concurrently and partially sequentially) as a “wavefront” on a single SIMD unit 138. One or more wavefronts are included in a “workgroup,” which contains a collection of work items designated to execute the same program. A workgroup can be executed by executing each of the wavefronts that make up the workgroup. In alternative examples, a wavefront executes on a single SIMD unit 138 or on different SIMD units 138. A wavefront can be thought of as the largest collection of work items that can execute simultaneously (or pseudo-simultaneously) on a single SIMD unit 138. “Pseudo-simultaneous” execution occurs when there are more wavefronts than the number of lanes in the SIMD unit 138. In such a situation, the wavefront executes over multiple cycles, with different collections of work items executing in different cycles. APD scheduler 136 is configured to perform operations related to scheduling various workgroups and wavefronts on compute units 132 and SIMD units 138 .

[0014] The parallel processing provided by the compute units 132 is well suited to graphics-related operations such as pixel value calculations, vertex transformations, and other graphics operations. Thus, in some cases, the graphics processing pipeline 134, which accepts graphics processing commands from the processor 102, provides computational tasks to the compute units 132 for execution in parallel.

[0015] Computation unit 132 is also used to perform computational tasks that are not related to graphics or that are not performed as part of the "normal" operation of graphics processing pipeline 134 (e.g., custom operations performed to supplement the operations performed on graphics processing pipeline 134). Applications 126 or other software executing on processor 102 send programs defining such computational tasks to APD 116 for execution.

[0016] Figure 3 is a block diagram illustrating additional details of the graphics processing pipeline 134 shown in Figure 2. The graphics processing pipeline 134 includes stages, each of which performs a specific function of the graphics processing pipeline 134. Each stage is implemented partially or fully as a shader program executing within the programmable compute unit 132, or partially or fully as fixed-function, non-programmable hardware external to the compute unit 132.

[0017] The input assembler stage 302 reads user-filled buffers (e.g., buffers filled with requests from software executed by the processor 102, such as applications 126) and assembles the data into primitives for use by the rest of the pipeline. The input assembler stage 302 can generate different types of primitives based on the primitive data contained in the user-filled buffers. The input assembler stage 302 formats the assembled primitives for use by the rest of the pipeline.

[0018] The vertex shader stage 304 processes the vertices of the primitives assembled by the input assembler stage 302. The vertex shader stage 304 performs various vertex-by-vertex operations such as transformation, skinning, morphing, and per-vertex lighting. The transformation operations include various operations for transforming the coordinates of the vertices. These operations include one or more of modeling transformation, display transformation, projection transformation, viewport division, viewport transformation for modifying vertex coordinates, and other operations for modifying non-coordinate attributes.

[0019] The vertex shader stage 304 is implemented partially or fully as a vertex shader program executed on one or more compute units 132. The vertex shader program is provided by the processor 102 and is based on a program pre-written by a computer programmer. The driver 122 compiles such a computer program to generate a vertex shader program having a form suitable for execution within the compute unit 132.

[0020] The hull shader stage 306, the tessellator stage 308, and the domain shader stage 310 work together to implement tessellation, which transforms simple primitives into more complex primitives by subdividing the primitives. The hull shader stage 306 generates patches for tessellation based on the input primitives. The tessellator stage 308 generates a set of samples for the patches. The domain shader stage 310 calculates the vertex positions of the vertices corresponding to the samples of the patches. The hull shader stage 306 and the domain shader stage 310 can be implemented as shader programs executed on the compute unit 132 compiled by the driver 122 together with the vertex shader stage 304.

[0021] The geometry shader stage 312 performs vertex operations based on primitives. Various different types of operations can be performed by the geometry shader stage 312, including operations such as point sprite expansion, dynamic particle system operations, fur-fin generation, shadow volume generation, single pass render-to-cubemap, per-primitive material swapping, and per-primitive material setup. In some cases, the geometry shader program compiled by the driver 122 and executed on the compute unit 132 performs the operations of the geometry shader stage 312.

[0022] The rasterizer stage 314 accepts and rasterizes simple primitives (triangles) generated upstream from the rasterizer stage 314. Rasterizing consists of determining which screen pixels (or sub-pixel samples) are covered by a particular primitive. Rasterizing is performed by fixed-function hardware.

[0023] The pixel shader stage 316 calculates the output values of screen pixels based on the primitives generated upstream and the results of rasterization. The pixel shader stage 316 can apply textures from texture memory. The operations of the pixel shader stage 316 are executed by a pixel shader program compiled by the driver 122 and executed on the compute unit 132.

[0024] The output merge stage 318 accepts the outputs from the pixel shader stage 316, merges them into the frame buffer, and performs operations such as z-testing and alpha blending to determine the final color of the screen pixel.

[0025] A processing workload on APD 116 often involves executing multiple "passes." A "pass" is a unit of processing that accepts several inputs (e.g., from another pass or as inputs to the overall processing workload), processes the inputs, and produces outputs. Some outputs include outputs to other passes, and other outputs include the final output of the processing workload, such as an image for display on a screen. Thus, the particular order of passes, the inputs they use, and the outputs they produce specify the overall processing workload.

[0026] An example of a path is the processing of data through a graphics processing pipeline 134 configured in a particular way (e.g., certain stages are enabled or disabled, and certain programmable stages have particular selected shader programs for execution). The input or output for any particular path through the graphics processing pipeline 134 is configurable based on the specified operation of the graphics processing pipeline 134. Exemplary graphics processing paths include a geometry path that renders meshes to a set of render targets and a depth buffer, and a post-processing path that renders screen space quads with vertex and pixel shader programs. Another example of a path is a compute workload that executes a particular compute shader program. A compute shader program is a shader program that executes on a compute unit 132 but does not execute as part of the graphics processing pipeline 134. A compute workload is the execution of a compute shader program for a particular set of input data to produce a particular set of output data.

[0027] It is possible to "manually" specify each of the individual commands in the processing workload path. More specifically, the application 126 (or other entity such as the driver 122) can issue commands to the driver 122 and / or the acceleration processing device 116 to execute the path in an ad-hoc manner. However, the present disclosure presents techniques for a more automated method for specifying multi-path processing workloads.

[0028] Specifically, the present disclosure presents techniques for programmably specifying a processing workload as a task group. FIG. 4A shows the APD scheduler 136 of FIG. 2 configured to execute a task graph specification program to generate a task graph and schedule operations for execution on the APD 116 based on the generated task graph.

[0029] The APD scheduler 136 includes a task graph specification processor 402 and a command scheduler 404. The task graph specification processor 402 accepts a task graph specification program as input. The task graph specification program programmably specifies a method for generating a task graph that describes the processing workload. The processing workload includes a graph of tasks and resources, further described herein (e.g., with respect to FIG. 5). The task graph specification processor 402 executes the task graph specification program, which, when executed, generates a task graph that specifies the processing load as a task graph.

[0030] The command scheduler 404 accepts a task graph as input, performs scheduling of the operations to execute the processing workload specified by the task graph, and instructs the APD 116 to execute those operations. In some implementations, the command scheduler 404 accepts APD feedback information and uses that feedback information to schedule operations on the APD 116. The APD feedback information includes various information regarding the state of the APD 116, such as performance metrics, occupancy metrics, resource utilization, and other types of information. Some exemplary items of information regarding the state of the APD 116 include the work input to the graphics processing pipeline 134, the hardware performance counters of the SIMD unit 138, the percentage of time such various hardware units are busy, and the memory bandwidth utilization, and the timestamps when they are completed by them. The command scheduler 404 utilizes the timestamps to evaluate the overall effectiveness of the task scheduling. Additionally, the timestamps indicate the overall duration 502 of the path. Thus, in some implementations, the command scheduler 404 prioritizes longer paths during scheduling. The hardware block utilization counters are used to assist group paths bottlenecked at different hardware blocks to achieve better overall utilization. In some examples, the command scheduler 404 is part of the APD 116 and can directly obtain such performance metrics from other components of the APD without intervention by the processor 102, application, driver, or operating system.

[0031] In the example of FIG. 4A , either or both of the task graph specification processor 402 and the command scheduler 404 are implemented as software executing on a processor, hardware circuitry configured to perform the operations described herein, or a combination thereof. Although depicted as two separate units, the task graph specification processor 402 and the command scheduler 404 are the same unit in some implementations. FIG. 4A illustrates an example in which the task graph specification processor 402 and the command scheduler 404 are within the APD 116. FIG. 4B illustrates an example in which the task graph specification processor 402 is software executing on the processor 102 and the command scheduler 404 is implemented within the APD 116 as described with respect to FIG. 4A . FIG. 4C illustrates an example in which the task graph specification processor 402 and the command scheduler 404 are programs executing on the processor 102.

[0032] Although the command scheduler 404 is described as outputting commands for execution on the APD 116, it should be understood that the techniques described herein may be applied to generating commands for execution on a processor other than the APD 116, such as a central processing unit (e.g., the processor 102) or another unit.

[0033] 5 illustrates an exemplary task graph 500. The task graph 500 specifies a processing workload as a directed acyclic graph of paths 502 and buffers 504. The paths 502 include particular units of work, such as the processing of particular data through the graphics processing pipeline 134 configured in a particular manner, or compute shader programs, as described above. The buffers 504 include data used as inputs to the paths 502 and / or as outputs to the paths 502. The buffers 504 may be inputs to one path 502 and outputs to another path, or may be inputs or outputs to the entire processing workload (corresponding to a single task graph). Some examples of buffers include images produced by particular paths 502 through the graphics processing pipeline 134, data produced by compute shader programs, or non-image data produced by the paths 502 through the graphics processing pipeline 134.

[0034] The task graph 500 includes arrows between paths 502 and buffers 504. An arrow from a buffer 504 to a path 502 indicates that the path 502 accepts the data in that buffer 504 as input. An arrow from a path 502 to a buffer 504 indicates that the path 502 produces the data in the buffer 504 as output. Thus, the task graph 500 specifies dependencies between paths 502. More specifically, a path 502 that accepts a buffer 504 produced by another path 502 as input must wait to execute until the buffer 504 is actually produced. Thus, paths 502 that accept buffers 504 as input depend on other paths 502 that produce those buffers 504 as output. If a third path 502 depends on a second path 502, and the first path 502 depends on the third path 502, the first path 502 is also considered dependent on the second path 502. In other words, dependencies propagate through the arrows in task graph 500. Two paths 502 are independent if one does not depend on the other.

[0035] As described above, a task graph specification program is a program that specifies how to construct the task graph 500. Thus, the task graph specification program indicates which paths 502 occur, and the buffers 504 are the inputs and outputs of the paths 502. The task graph specification program may consider runtime data, such as user-defined runtime data, data generated by the APD 116, or other data, when making decisions regarding whether to include particular paths 502 and / or buffers 504 in the task graph 500 and how the paths 502 consume and / or produce buffers 504. Thus, the task graph specification program is not just a static description of the task graph 500 that is evaluated by the task graph specification processor 402. Instead, the task graph specification program may variably construct the task graph 500 based on runtime parameters.

[0036] 6A and 6B illustrate an exemplary task graph specification program format, and FIGS. 7A and 7B illustrate alternative task graph specification program formats. In FIG. 6A, the task graph specification program includes path definitions 605, path invocations 606, and program control flow instructions 604. The path definitions 605 define the operations and methods involved in executing paths on the APD 116. The path invocations 606 represent instructions to the task graph specification processor 402 to generate paths 502 of the output task graph 500. The control flow instructions 604 are instructions for controlling when and how the path invocations 606 are generated. As described above, the task graph specification processor 402 executes the control flow instructions 604 and path invocations 606 to schedule by the command scheduler 404 and build the task graph 500 for execution by the APD 116.

[0037] In some implementations, the path definition 605 includes a resource import / export instruction 608, a state instruction 610, and a path command 612. In some implementations, the path definition 605 omits the state instruction 610. The resource import / export instruction 608, when executed on the APD 116, specifies one or more resources that are imported (read) and / or exported (written) by the path definition 605. The term "resource" has the same meaning as the term "buffer" (i.e., buffer 504 of FIG. 5). In some implementations, the state instruction 610, when executed on the APD 116, indicates whether one or both shaders are used for graphics, or calculates the operation of the path, or, when executed on the APD 116, which pipeline states are used for the path and what other states are similarly considered. The path command 612 is the actual work performed on the path 606. The path command 612 input from one or more imported resources processes those one or more imported resources and outputs them to one or more exported resources. In some implementations, the path command 612 includes a draw command (e.g., for drawing geometry by sending geometry through a graphics processing pipeline 134 of a particular configuration), a dispatch command (for dispatching a general-purpose computing workload for execution), a copy command (for copying one or more portions of memory to one or more other locations), a clear command (for clearing one or more portions of memory), a resource layout transition (for modifying the layout of one or more resources), a cache action (for flushing or prefetching cache data), a synchronization operation (e.g., for synchronization between APD 116 hardware blocks or events, for example, draw commands and dispatch commands can compete for memory bandwidth, so although these items can be logically executed in parallel, the programmer sometimes inserts explicit synchronization commands to force the start of one only after the completion of the other), or other commands executable by the APD 116.In the case of cache and sync actions, the command scheduler 404 can, and sometimes does, generate such actions / behaviors automatically, however, the programmer may also specify these actions explicitly.

[0038] The control flow instructions 604 control the program flow of the task graph specification program 600 during execution by the task graph specification processor 402. The control flow instructions 604 include control flow instructions such as conditional statements (e.g., if statements), loops, etc. The control flow instructions 604 allow the task graph specification program 600 to construct the task graph 500 at runtime based on runtime parameters. Loops allow the generation of multiple passes 502 from a single pass definition 605, each with different parameters (import / export data and shader / pipeline configuration). Conditionals can specify whether a particular node definition 605 is actually instantiated as a node.

[0039] In some situations, the path invocation 606 specifies arguments for one or more of the resource import / export instructions 608 or the state instructions 610. In such situations, the arguments specify one or more of any state (e.g., shader programs, graphics processing pipeline 134 configurations) to use for the particular path generated as a result of the path invocation 606, one or more resources to import into the path generated as a result of the path invocation 606, or one or more resources to export from the path generated as a result of the path invocation 606. Furthermore, in some situations, the arguments are specified as variables, meaning that the values of the arguments are determined at runtime (e.g., based on data available to the task graph specification program 600 and / or as a particular result from execution of the control flow instructions 604). In short, the arguments, and therefore the state, import data, and export data, are runtime-determinable elements (i.e., the time the task graph specification program 600 is executed means that the programmer writing the task graph specification program 600 does not need to hard-code such information).

[0040] Figure 6B shows an example task graph 650 generated based on a task graph specification program 600 of the type described in Figure 6A. An example task graph specification program illustrated in code is now provided.

[0041] [Table 1] The above task graph specification program specifies two path definitions: one for Path A that calls Draw1(), and one for Path B that calls Draw2() and Draw3(). The Path call instantiates Path A on the graph where it inputs resource r1 and outputs resource r2. Based on certain conditions, the Path call instantiates zero, one, or two instances of Path B, inputting resource r2 and outputting r3. Note that Draw3() internally uses data produced by Draw2(), but this data is not specified as a resource because it is produced and consumed entirely within Path B.

[0042] FIG. 6B shows two different task graphs 650 generated from two different executions of the task graph specification in Table 1. In the first task graph 650(1), both condition A and condition B are true. Thus, the generated graph includes two instances of path A 652(1), which reads from resource 1 654(1) and outputs to resource 2 654(2), and path B 652(2), which both reads from resource 2 654(2) and outputs to resource 3 654(3). For task graph 650(2), only condition A is true. Thus, only one instance of path B 652(2) is generated.

[0043] 7A and 7B show another form of task graph specification program 700 and a task graph 750 generated as a result of execution of task graph specification program 700 by task graph specification processor 402. FIG.

[0044] The task graph specification program 700 includes control flow instructions 702, resource import / export instructions 704, state instructions 706, and path commands 708. The path commands 708 are individual commands that can be executed by the APD 116. Some example path commands 708 include resource import / export instructions 704 drawing commands (e.g., to draw geometry by sending it through a particular configuration of the graphics processing pipeline 134), dispatch commands (to dispatch general-purpose computational workloads for execution), copy commands (to copy one or more portions of memory to one or more other locations), clear commands (to clear one or more portions of memory), resource layout transitions (to modify the layout of one or more resources), cache actions (to flush or prefetch cached data), synchronization operations (such as synchronization between APD 116 hardware blocks or events; e.g., drawing commands and dispatch commands can compete for memory bandwidth; therefore, despite the fact that these items can logically execute in parallel, programmers sometimes insert explicit synchronization commands to force one to start only after the other has completed), or other commands executable by the APD 116. In the case of cache actions and synchronization operations, the command scheduler 404 can, and sometimes does, generate such actions / operations automatically. However, programmers may also specify these actions explicitly.

[0045] Resource import / export commands 704 specify resources to be read and / or written by path commands 708. State instructions 706 are instructions that define and / or modify the state of APD 116 (such as the state of graphics processing pipeline 134, which shader programs are used for graphics processing pipeline 134 or compute shader dispatch, or other state). Control flow instructions 702 control program flow when task graph specification processor 402 executes task graph specification program 700.

[0046] During execution, the task graph specification processor 402 traverses the task graph specification program 700 to identify path blocks 710 from which the task graph specification program 700 generates paths 502 for the corresponding task graph. In some examples, the path blocks 710 are portions of the task graph specification program 700 that contain one (or a specified number) of path commands, terminate with a path command, and contain no control flow instructions for execution by the task graph specification processor 402. In other examples, the path blocks 710 contain a different fixed number of path commands 708 or a variable number of path commands 708. In some examples, the path blocks 710 need not terminate with a path command 708. In some examples, the task graph specification processor 402 identifies portions of the task graph specification program 700 that contain at least one path command 708 and terminate with a path command 708, and divides each such portion based on the location of the path command 708 into the path blocks 710. More specifically, from the portions, the task graph specification program 700 generates one path block 710 for each specified number of path commands 708. In some examples, each such path block 710 includes all instructions from the previous path block 710 to the last path command 708 in the path block 710 , but does not include control flow instructions 702 .

[0047] With the path block 710 identified, the task graph specifying processor 702 generates the task graph 500 as follows. Each path block 710 includes definitions of one or more import resources and one or more output resources. These resources constitute the buffers 504 of the task graph. Resource reads and writes constitute the directed edges of the task graph. The edge directed from the buffer 504 to the path 502 is a resource read, and the edge directed from the path 502 to the buffer 504 is a resource write.

[0048] The following table includes an exemplary code listing of a task graph specifying program in the format described in FIG. 7A. FIG. 7B shows an exemplary task graph 750 generated from the program.

[0049] [Table 2] In this example, the path block includes only one path command. The path command is identified as a command that is one of a set of commands designated as path commands, such as those described elsewhere in this specification. In other words, the task graph specifying processor 402 has access to a list of command types that are regarded as path commands and that identify all commands of the program found in the list as path commands. The exemplary task graph specifying program in Table 2 includes three different path blocks. The first path block is the portion from the SetState(State1) description to the first path command Draw1(). The second path block is the portion in the if statement that begins with the SetState(State2) description and ends with the Draw2() path command. The third path block is the portion of the if statement that begins with SetResource(read:r5, write:r3) and ends with Draw3().

[0050] In an example of the execution of the task graph specification program in Table 2, shown as task graph 750(1), condition B is true. Therefore, the task graph specification processor 402 generates paths in task graph 750(1) including path 1 that inputs resource 1 754(1) and outputs resource 2 754(2), path 2 that inputs resource 2 754(2) and outputs resource 5 754(5), and path 3 that inputs resource 5 754(5) and outputs resource 3 754(3).

[0051] In another example of the execution of the task graph specification program in Table 2, shown as task graph 750(2), condition B is false. Therefore, the task graph specification processor 402 generates path 1 that inputs resource 1 754(1) and outputs resource 2 754(2)754(2), but does not generate path 2 or 3.

[0052] As described above, the command scheduler 404 schedules tasks for execution on the APD 116 based on the task graph output by the task graph specification processor 402. The command scheduler 404 determines the execution order of the paths based on certain considerations. One consideration includes the dependencies of the paths 502. More specifically, paths 502 that have one or more dependencies cannot be reordered with respect to each other. For example, if a first path 502 depends on a second path 502, the command scheduler 404 then causes the first path 502 to execute after the second path 502 has completed. In some examples, the command scheduler 404 inserts appropriate instructions or markers, such as barrier instructions or primitives, to enforce ordering among the paths 502. If the first pass 502 is independent of the second pass 502 and the second pass is independent of the first pass 502, the command scheduler 404 is then free to order the execution of the first pass 502 and the second pass 502 relative to each other in any technically feasible manner, and in some circumstances does so. For example, in some cases, the command scheduler 404 causes the independent passes to execute one after the other, and in other cases, the command scheduler 404 causes the independent passes to execute during at least partially overlapping time periods. In many cases, the command scheduler 404 selects passes 502 to execute together to optimize certain desired resources, such as memory usage, processing unit resources, register usage, etc., or to minimize execution time.

[0053] 8A shows a task graph 800, according to one example. A first path 802(1) reads resource 1 804(1) and outputs resource 2 804(2). Path 2 802(2) inputs resource 2 804(2) and outputs resource 3 804(3). Path 3 802(3) inputs resource 2 804(2) and outputs resource 4 804(4). Path 4 802(4) inputs resource 2 804(2) and outputs resource 5 804(5). Path 5 inputs resource 6 804(6) and outputs resource 6 804(6).

[0054] Path 2 802(2), path 3 802(3), and path 4 802(4) are independent and dependent on path 2 802(1). Path 5 802(5) is dependent on path 1 802(1), path 2 802(2), path 3 802(3), and path 4 802(4). Because path 2 802(2), path 3 802(3), and path 4 802(4) are independent, command scheduler 404 can reorder the execution of these paths 802 relative to each other. However, command scheduler 404 imposes an ordering among the dependent paths, meaning that path 1 402(1) executes before path 2 802(2), path 3 802(3), and path 4 802(4), which execute before path 5 802(5).

[0055] FIG. 8B illustrates an exemplary execution order 850 of paths 802, as scheduled by command scheduler 404. As illustrated, time progresses to the right. In example 1 850(1), path 1 802(1) executes before path 2 802(2), path 3 802(3), and path 4 802(4), and path 2 802(2), path 3 802(3), and path 4 802(4) execute concurrently. In example 2 850(2), path 1 802(1) executes before path 2 802(2) and path 3 802(3). Path 4 802(4) executes after path 2 802(2) and path 3 802(3), and executes before path 5 802(5). In example 3 850(3), paths 802 execute sequentially.

[0056] In some implementations, a particular programming language is used for the techniques of Figures 7A and 7B, as well as the entire task graph specification program for the control flow instructions 604 and path invocations 606 of Figures 6A and 6B (but not for the path definitions 605). In such implementations, in the example of Figures 6A and 6B, the path definitions 605 are written in a different programming language that is compiled. These path definitions 605 are called by the APD 116 when the corresponding paths are executed.

[0057] To determine how to schedule 802 independent paths, the command scheduler 404 considers the state of the APD 116 and performance metrics of the paths 802. There are several different items of information that constitute the “state” of the APD 116. Some examples of performance metrics include resource usage and execution speed. Resources include memory, registers, and processing units (such as the SIMD unit 138). In some examples, the command scheduler 404 records performance metrics of paths 802 during execution for use during future scheduling of the same paths 802. More specifically, the command scheduler 404 updates estimated resource usage data for particular paths based on a priori known resource usage for those paths and utilizes such updated estimated resource usage data to make future scheduling decisions.

[0058] As described above, the command scheduler 404 sometimes schedules independent paths 502 for simultaneous execution on one or more compute units 132. The command scheduler 404 determines whether to schedule different paths 502 simultaneously based on the nature of the paths 502 and the capabilities and status of the APD 116. More specifically, in some situations, it is advantageous to schedule independent paths 502 simultaneously, and in other situations, it is advantageous to schedule independent paths 502 simultaneously. The term "simultaneously" means that two (or more) paths 502 execute in at least partially overlapping time periods.

[0059] Paths 502 consume resources such as memory, processing units (e.g., SIMD units 138), registers, etc. Therefore, in situations where the APD 116 is limited by certain resources and the number of concurrently executing paths 502 exceeds the number of resources available, the command scheduler 404 does not schedule such paths 502 concurrently. Conversely, in situations where resources are available for concurrent execution, the command scheduler 404 sometimes schedules such paths 502 concurrently. Additionally, during operation, the command scheduler 404 determines whether to schedule independent paths 502, emphasizing aspects such as speed or resource usage based on such determination. In one example, executing multiple paths 502 concurrently reduces the total execution time of those paths 502 but increases total resource usage at any given time. Executing multiple paths 502 successively increases the total execution time but reduces total resource usage at a given time. Conversely, executing multiple paths 502 concurrently reduces the total execution time but increases total resource usage at a given time.

[0060] Another consideration for the command scheduler 404 is whether different paths 502 use different resources. More specifically, if different paths 502 use different types of resources, or if one path 502 uses a small amount of a particular resource and another path 502 uses a large amount of the resource, then it would be beneficial to schedule them together. In one example, one path 502 has very heavy usage of the SIMD unit 138, while another path 502 has light usage of the SIMD unit 138. In this situation, the command scheduler 404 may schedule these two paths for execution simultaneously.

[0061] Additionally, the command scheduler 404 can reduce memory usage at a given time by reducing the duration for which a particular buffer 504 is used. More specifically, by analyzing the task graph 500, the command scheduler 404 can determine the lifetime of any particular buffer 504. The lifetime of a buffer 504 is the period between the path 502 that produces the buffer 504 and the last path 502 that consumes that buffer 504. In some implementations, in some situations, such as when memory usage is high, the command scheduler 404 minimizes the lifetime of a buffer 504 by simultaneously scheduling paths 502 whose completion would tend to end its lifetime. In other words, to reduce the lifetime of a buffer 504, the command scheduler 404 identifies paths 502 that are the last possible paths 502 to use the buffer 504, and the command scheduler 404 identifies paths 502 on which the last possible path 502 depends. The command scheduler 404 then attempts to schedule such identified paths 502 for concurrent execution when possible (e.g., independent paths 502 may be scheduled for concurrent execution). A "last possible path" is either a path that is the last path in topological sorting order to use a buffer 504, or an independent path 502 that could be the last path in topological sorting order to use a buffer 504. This technique can also be used to keep a resource working set from remaining in a particular portion of the memory hierarchy, such as a cache or other memory. In other words, the technique can be used to limit the memory usage of a path 502 so that it fits within a particular portion of the memory hierarchy (e.g., all of the level 0 cache, all of the level 1 cache, etc.).

[0062] Topological sorting is a technique in which the nodes of a directed acyclic graph are ordered based on the graph's directed edges. In the sort, nodes at the start of directed edges always precede nodes at the end of those directed edges. Any order for nodes that do not have directed edges between them produces a valid topological sort. Thus, if multiple paths 502 use a buffer 504 but are independent, such a path 502 may be the last to use the buffer 504 in a valid topological sort order, and each such path 502 is then the last possible path 502.

[0063] In some implementations, the paths 502 that exist during their lifetime include multiple independent paths 502 that encompass all of the used buffers 504. By aggressively scheduling consumers of the buffers 504 to execute early, the lifetime of the buffers 504 is reduced.

[0064] In an additional scheduling technique, the command scheduler 404 automatically decomposes a large path into several smaller paths and schedules them independently.

[0065] 9 is a flow diagram of a method 900 for constructing a task graph for scheduling, according to an example. Although described with respect to the systems of FIGS. 1-8B, one skilled in the art will recognize that any system configured to perform the steps of method 900 in any technically feasible order is within the scope of this disclosure.

[0066] Method 900 begins at step 902, when task graph specification processor 402 receives a task graph specification program 902 for execution. The task graph specification program programmatically specifies how to generate a task graph for scheduling and execution on APD 116. At step 904, task graph specification program 902 executes control flow instructions to traverse task graph specification program 902 as specified in task graph specification program 902. These control flow instructions include conditional instructions, loops, and the like. As before steps 906-910, the order of step 904 in FIG. 9 does not necessarily imply that step 904 is executed only once in that order. Instead, it should be understood that step 904 indicates that control flow instructions are executed as specified by the task graph specification program in any order found in the program.

[0067] In step 906, the task graph specification processor 402 generates nodes (corresponding to paths 502) based on the path instructions of the task graph specification program. In the format of FIGS. 6A-6B, the path instructions are path calls 606. Each execution of a path call generates nodes of the task graph in the order specified by the control flow instructions. For the format of FIGS. 7A-7B, the path instructions are path blocks 710. Specifically, the task graph specification processor 402 identifies these path blocks 710 in the manner specified elsewhere herein and generates nodes from these path blocks 710.

[0068] In step 908, the task graph specification processor 402 generates resources and directed edges between the generated nodes based on the resource usage of each node as the task graph is generated. For resources that are read by a node, the task graph specification processor 402 generates a directed edge from the resource to the node. For resources that are written by a node, the task graph specification processor 402 generates a directed edge from the node to the resource. In step 910, the task graph specification processor 402 outputs the task graph to the command scheduler 404 for scheduling on the APD 116 or other device.

[0069] 10 is a flow diagram of a method 1000 for scheduling operations on a processing device based on a task graph, according to an example. Although described with respect to the systems of FIGS. 1-8B, one skilled in the art will recognize that any system configured to perform the steps of method 1000 in any technically feasible order is within the scope of this disclosure.

[0070] Method 1000 begins at step 1002, where command scheduler 404 receives a task graph. In some examples, the task graph is generated by task graph specification processor 402. In some examples, command scheduler 404 is on the same processing device as task graph specification processor 402 or on a different processing device than task graph specification processor 402. Various exemplary configurations are shown in Figures 4A-4C.

[0071] In step 1004, the command scheduler 404 identifies independent and dependent paths of the task graph. Techniques for identifying such paths are described elsewhere in this specification. In step 1006, the command scheduler 404 determines how to schedule for execution on the processing device based on performance criteria. Some techniques and considerations for scheduling paths are described elsewhere in this specification, such as before the description of FIG. 9, including the descriptions related to FIGS. 8A and 8B. The scheduling step 1006 involves executing any one or more of these techniques. Generally, the command scheduler 404 schedules independent paths to provide what is considered to be the best utilization of certain resources (e.g., memory, registers, processing time, etc.) and / or other factors of the processing device (e.g., hardware units). Further, for paths that must occur continuously due to dependencies, the command scheduler 404 causes such paths to occur continuously. In some examples, the command scheduler 404 causes such paths to occur continuously via a barrier command that completes all operations of one path before the dependent path begins to complete. In step 1008, the command scheduler 404 sends the scheduled commands to the processing device for execution as scheduled.

[0072] In the present disclosure provided herein, the device that ultimately executes the commands scheduled for execution of the task graph is the APD 116. However, it should be understood that implementations are contemplated by this specification in which processing devices other than those specifically described execute these commands. In some examples, devices that are similar to the APD 116 but do not have graphics capabilities (e.g., do not include the graphics processing pipeline 134 and can execute a compute workload) are processing devices. In other examples, the central processing unit is a processing device. In other examples, any other processing unit is a processing device.

[0073] Any of the units in FIG. 3 , including processor 102, storage 106, memory 104, input driver 112, input device 108, output driver 114, APD 116, display device 118, output device 110, driver 122, operating system 120, application 126, graphics processing pipeline 134, compute unit 132, SIMD unit 138, APD scheduler 136, and stages of graphics processing pipeline 134, may, where appropriate, be implemented as hardware configured to perform the described operations, as software running on a processor, as software configured to perform the described operations, or as a combination of hardware and software.

[0074] It should be understood that many variations are possible based on the disclosure herein, and although features and elements are described above in particular combinations, each feature or element can be used alone without other features and elements, or in various combinations with or without other features and elements.

[0075] The provided methods can be implemented in a general-purpose computer, processor, or processor core. Suitable processors include, by way of example, a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), and / or a state machine. Such processors can be fabricated by configuring a manufacturing process using the results of processed hardware description language (HDL) instructions and other intermediate data, such as a netlist (such instructions can be stored on a computer-readable medium). The result of such processing can be a mask work, which is then used in a semiconductor manufacturing process to produce a processor implementing features of the present disclosure.

[0076] The methods or flow diagrams provided herein may be implemented in a computer program, software, or firmware embodied in a non-transitory computer-readable storage medium for execution by a general-purpose computer or processor. Examples of non-transitory computer-readable storage media include read-only memory (ROM), random-access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media (e.g., internal hard disks and removable disks), magneto-optical media, and optical media (e.g., CD-ROM disks and digital versatile disks (DVDs)).

Claims

1. A method for scheduling the operation of a task graph on a processing device, comprising: receiving a task graph specifying one or more paths including a first path and a second path, one or more resources, and one or more directed edges between the paths and the resources, wherein the first path and the second path are independent paths, the first path includes execution via a graphics processing pipeline including a pixel shader, and any resource includes a buffer that is output by any one of the one or more paths and input by another one of the one or more paths; identifying independent paths and dependent paths of the task graph; scheduling commands for executing the paths according to the task graph based on performance criteria of the processing device; transmitting the scheduled commands to processing elements of the processing device for execution according to the schedule. A method executed by a computer.

2. Identifying independent paths includes identifying a set of paths that are not dependent on each other. The method of Claim 1.

3. Scheduling commands for executing the paths includes: scheduling at least two paths of the set of paths to be executed at least partially simultaneously. The method of Claim 2.

4. Scheduling the two paths to be at least partially simultaneous is performed in response to determining that one of the two paths utilizes a small number of resources of the processing device and the other of the two paths utilizes a large number of resources of the processing device. The method of Claim 3.

5. The performance criteria include requirements for reducing the utilization amount of resources of the processing device from a first utilization amount that would occur if the at least two paths were not executed at least partially simultaneously to a second utilization amount. The method of Claim 3.

6. The resources include either memory utilization or register utilization. The method of Claim 5.

7. The resources include memory utilization of a buffer read by the two paths, and scheduling the two paths to be at least partially simultaneous reduces the lifetime of the buffer. The method of Claim 5.

8. Further comprising identifying a first path among the one or more paths that depends on a second path among the one or more paths The method of claim 1

9. Further comprising sending a barrier command to the processing device to cause the first path to be executed after the second path has been completed The method of claim 8

10. Said scheduling is performed by a scheduler within the processing device that obtains processing metrics from the processing device without communicating with an operating system executed by a processor external to the processing device The method of claim 1

11. A system for scheduling the operation of a task graph on a processing device, comprising A command scheduler circuit Said command scheduler circuit Receiving a task graph that specifies one or more paths including a first path and a second path, one or more resources, and one or more directed edges between the paths and resources, wherein the first path and the second path are independent paths, the first path includes execution via a graphics processing pipeline including a pixel shader, and any resource includes a buffer that is output by any one of the one or more paths and input by another one of the one or more paths Identifying independent and dependent paths of the task graph Scheduling commands for executing the paths according to the task graph based on performance criteria of the processing device Sending the scheduled commands to processing elements of the processing device for execution as scheduled Configured to perform System

12. Identifying independent paths includes identifying a set of paths that are not dependent on each other The system of claim 11

13. Scheduling commands for executing the paths Includes scheduling at least two paths of the set of paths to be executed at least partially simultaneously The system of claim 12

14. Scheduling the at least partially simultaneous execution of the two paths is performed in response to determining that one of the two paths utilizes a small number of resources of the processing device and the other of the two paths utilizes a large number of resources of the processing device. The system of claim 13.

15. The performance criteria includes requirements for reducing the utilization of resources of the processing device from a first utilization amount that would occur if the at least two paths were not executed at least partially simultaneously to a second utilization amount. The system of claim 13.

16. The resources include either memory utilization or register utilization. The system of claim 15.

17. The resources include the memory utilization of a buffer read by the two paths, and scheduling the two paths at least partially simultaneously reduces the lifetime of the buffer. The system of claim 15.

18. The command scheduler is further configured to identify a first path among the one or more paths that depends on a second path among the one or more paths. The system of claim 11.

19. The command scheduler is further configured to send a barrier command to the processing device to cause the first path to be executed after the second path has completed. The system of claim 18.

20. A computer-readable storage medium storing instructions that, when executed by a processor, receive a task graph specifying one or more paths including a first path and a second path, one or more resources, and one or more directed edges between the paths and the resources, wherein the first path and the second path are independent paths, the first path includes execution via a graphics processing pipeline including a pixel shader, and any resource includes a buffer that is output by one of the one or more paths and input by another of the one or more paths; identify independent paths and dependent paths of the task graph; and schedule commands for executing the paths according to the task graph based on performance criteria of a processing device. To execute according to the scheduling, sending the scheduled command to the processing elements of the processing device; Causing the processor to schedule the operation of the task graph on the processing device; A computer-readable storage medium.

Images