Task graph generation for workload processing

Abstract

To provide a method for generating a task graph for workload scheduling based on a task graph specification program.SOLUTION: The method includes a task graph specification processor receiving 902 a task graph specification program for execution, executing 904 control flow instructions specified in the task graph specification program to traverse the task graph specification program, generating 906 nodes based on path instructions of the task graph specification program, generating 908, in a task graph, resources and directed edges between the generated nodes based on resource utilization of each node, and outputting 910 the task graph to a command scheduler for scheduling on an acceleratedprocessingdevice (APD) or other device.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority to U.S. Patent Application No. 16 / 888,521, filed May 29, 2020, the entire contents of which are incorporated herein by reference. [Background technology]

[0002] A three-dimensional (3D) graphics processing pipeline performs a series of steps to convert input geometry into a two-dimensional (2D) image for display on a screen. In some instances, these steps are specified as commands by a host, such as a central processing unit that runs an application. Many rendering workloads require large sequences of commands, and therefore generating and processing these sequences is a complex task.

[0003] A more detailed understanding may be had from the following description, given by way of example in conjunction with the accompanying drawings, in which: [Brief explanation of the drawings]

[0004] [Figure 1] FIG. 1 is a block diagram of an example device in which one or more features of the present disclosure may be implemented. [Figure 2] FIG. 2 illustrates details of the device of FIG. 1, according to an example. [Figure 3] FIG. 3 is a block diagram illustrating additional details of the graphics processing pipeline shown in FIG. [Figure 4A] 4 illustrates an arrangement of a task graph specification processor 402 and a command scheduler 404 for generating a task graph and scheduling workload on a processing device such as the accelerated processing device of FIG. 2. [Figure 4B] 4 illustrates an arrangement of a task graph specification processor 402 and a command scheduler 404 for generating a task graph and scheduling workload on a processing device such as the accelerated processing device of FIG. 2. [Figure 4C] 4 illustrates an arrangement of a task graph specification processor 402 and a command scheduler 404 for generating a task graph and scheduling workload on a processing device such as the accelerated processing device of FIG. 2. [Figure 5] FIG. 2 illustrates an exemplary task graph. [Figure 6A] FIG. 1 illustrates the generation of a task graph from a task graph specification program according to an exemplary technique. [Figure 6B] FIG. 1 illustrates the generation of a task graph from a task graph specification program according to an exemplary technique. [Figure 7A] FIG. 1 illustrates the generation of a task graph from a task graph specification program according to another exemplary technique. [Figure 7B] FIG. 1 illustrates the generation of a task graph from a task graph specification program according to another exemplary technique. [Figure 8A] FIG. 1 illustrates scheduling tasks on a processing device according to a task graph. [Figure 8B] FIG. 1 illustrates scheduling tasks on a processing device according to a task graph. [Figure 9] FIG. 1 is a flow diagram of a method for generating a task graph from a task graph specification program. [Figure 10] 1 is a flow diagram of a method for scheduling tasks on a processing device according to a task graph. DETAILED DESCRIPTION OF THE INVENTION

[0005] A technique is provided for generating a task graph for workload scheduling based on a task graph specification program, the technique including executing control flow instructions of the task graph specification program to traverse the task graph specification program, generating path nodes of the task graph based on path instructions of the task graph specification program, generating resource nodes and directed edges based on resource declarations of the task graph specification program, and outputting the task graph specification program to a command scheduler for scheduling.

[0006] 1 is a block diagram of an example device 100 capable of implementing one or more features of the present disclosure. Device 100 may be, for example, but not limited to, a computer, a gaming device, a handheld device, a set-top box, a television, a mobile phone, a tablet computer, or any other computing device. Device 100 includes a processor 102, a memory 104, a storage device 106, one or more input devices 108, and one or more output devices 110. Device 100 also includes one or more input drivers 112 and one or more output drivers 114. Any of the input drivers 112 may be embodied as hardware, a combination of hardware and software, or software, and serve the purpose of controlling (e.g., controlling the operation of, receiving input from, and providing data to) input driver 112. Similarly, any of the output drivers 114 may be embodied as hardware, a combination of hardware and software, or software, and serve the purpose of controlling (e.g., controlling the operation of, receiving input from, and providing data to) the output device 110. It should be understood that device 100 may include additional components not shown in FIG.

[0007] In various alternatives, processor 102 may include a central processing unit (CPU), a graphics processing unit (GPU), a CPU and a GPU located on the same die, or one or more processor cores, each of which may be a CPU or a GPU. In various alternatives, memory 104 may be located on the same die as processor 102 or may be located separately from processor 102. Memory 104 may include volatile or non-volatile memory (e.g., random access memory (RAM), dynamic RAM, cache).

[0008] Storage devices 106 include fixed or removable storage devices (e.g., but not limited to, hard disk drives, solid state drives, optical disks, flash drives). Input devices 108 include, but are not limited to, keyboards, keypads, touchscreens, touchpads, detectors, microphones, accelerometers, gyroscopes, biometric scanners, or network connections (e.g., wireless local area network cards for transmitting and / or receiving wireless IEEE 802 signals). Output devices 110 include, but are not limited to, displays, speakers, printers, haptic feedback devices, one or more optics, antennas, or network connections (e.g., wireless local area network cards for transmitting and / or receiving wireless IEEE 802 signals).

[0009] The input driver 112 and the output driver 114 include one or more hardware, software, and / or firmware components configured to interface with and drive the input device 108 and the output device 110, respectively. The input driver 112 communicates with the processor 102 and the input device 108, allowing the processor 102 to receive input from the input device 108. The output driver 114 communicates with the processor 102 and the output device 110, allowing the processor 102 to send output to the output device 110. The output driver 114 includes an accelerated processing device (APD) 116 coupled to a display device 118, which in some examples is a physical display device or a simulated device that presents output using a remote display protocol. The APD 116 is configured to accept computational commands and graphics rendering commands from the processor 102, process the computational and graphics rendering commands, and provide pixel output to the display device 118 for display. As described in further detail below, APD 116 includes one or more parallel processing units configured to perform calculations according to the single-instruction-multiple-data (SIMD) paradigm. Thus, although various functions are described herein as being performed by or in conjunction with APD 116, in various alternatives, functions described as being performed by APD 116 are additionally or alternatively performed by other computing devices having similar capabilities that are not driven by a host processor (e.g., processor 102) and that are configured to provide graphics output to display device 118. For example, it is contemplated that any processing system that performs processing tasks according to the SIMD paradigm may be configured to perform the functions described herein. Alternatively, it is contemplated that computing systems that do not perform processing tasks according to the SIMD paradigm perform the functions described herein.

[0010] FIG. 2 illustrates details of device 100 and APD 116, according to one example. Processor 102 (FIG. 1) executes operating system 120, drivers 122, and applications 126, and may alternatively or additionally execute other software. Operating system 120 controls various aspects of device 100, such as managing hardware resources, handling service requests, scheduling and controlling process execution, and performing other operations. APD driver 122 controls the operation of APD 116 and sends tasks, such as graphics rendering tasks or other work, to APD 116 for processing. APD driver 122 also includes a just-in-time compiler that compiles programs for execution by processing components of APD 116 (such as SIMD unit 138, described in more detail below).

[0011] APD 116 executes commands and programs for selected functions, such as graphics and non-graphics operations that may be suitable for parallel processing. APD 116 may be used to perform graphics pipeline operations, such as pixel operations, geometric calculations, and rendering of images to display device 118, based on commands received from processor 102. APD 116 also performs computational operations not directly related to graphics operations, such as operations related to video, physics simulation, computational fluid dynamics, or other tasks, based on commands received from processor 102.

[0012] The APD 116 includes a computation unit 132 that includes one or more SIMD units 138 configured to perform operations in a parallel manner according to the SIMD paradigm at the request of the processor 102 (or another unit). The SIMD paradigm allows multiple processing elements to share a single program control flow unit and program counter, thus executing the same program but with different data. In one example, each SIMD unit 138 includes 16 lanes, each of which executes the same instruction simultaneously with other lanes in the SIMD unit 138 but can execute the instruction with different data. Lanes can be predictively switched off when not all lanes need to execute a given instruction. Prediction can also be used to execute programs with branching control flow. More specifically, for programs with conditional branches or other instructions where control flow is based on calculations performed by individual lanes, prediction of lanes corresponding to currently unexecuted control flow paths and serial execution of different control flow paths enables arbitrary control flow.

[0013] The basic unit of execution within the compute unit 132 is the work item. Each work item represents a single instantiation of a program executing in parallel on a particular lane. Work items can execute simultaneously (or partially concurrently, partially sequentially) as a “wavefront” on a single SIMD unit 138. One or more wavefronts are included in a “workgroup,” which contains a collection of work items designated to execute the same program. A workgroup can be executed by executing each of the wavefronts that make up the workgroup. In alternative examples, a wavefront executes on a single SIMD unit 138 or on different SIMD units 138. A wavefront can be thought of as the largest collection of work items that can execute simultaneously (or pseudo-simultaneously) on a single SIMD unit 138. “Pseudo-simultaneous” execution occurs when there are more wavefronts than the number of lanes in the SIMD unit 138. In such a situation, the wavefront executes over multiple cycles, with different collections of work items executing in different cycles. APD scheduler 136 is configured to perform operations related to scheduling various workgroups and wavefronts on compute units 132 and SIMD units 138 .

[0014] The parallel processing provided by the compute units 132 is well suited to graphics-related operations such as pixel value calculations, vertex transformations, and other graphics operations. Thus, in some cases, the graphics processing pipeline 134, which accepts graphics processing commands from the processor 102, provides computational tasks to the compute units 132 for execution in parallel.

[0015] Computation unit 132 is also used to perform computational tasks that are not related to graphics or that are not performed as part of the "normal" operation of graphics processing pipeline 134 (e.g., custom operations performed to supplement the operations performed on graphics processing pipeline 134). Applications 126 or other software executing on processor 102 send programs defining such computational tasks to APD 116 for execution.

[0016] Figure 3 is a block diagram illustrating additional details of the graphics processing pipeline 134 shown in Figure 2. The graphics processing pipeline 134 includes stages, each of which performs a particular function of the graphics processing pipeline 134. Each stage is implemented partially or fully as a shader program executing within the programmable compute unit 132, or partially or fully as fixed-function, non-programmable hardware external to the programmable compute unit 132.

[0017] The input assembler stage 302 reads user-filled buffers (e.g., buffers filled with requests from software executed by the processor 102, such as applications 126) and assembles the data into primitives for use by the rest of the pipeline. The input assembler stage 302 can generate different types of primitives based on the primitive data contained in the user-filled buffers. The input assembler stage 302 formats the assembled primitives for use by the rest of the pipeline.

[0018] The vertex shader stage 304 processes the vertices of the primitives assembled by the input assembler stage 302. The vertex shader stage 304 performs various per-vertex operations such as transformation, skinning, morphing, and per-vertex lighting. Transformation operations include various operations for transforming the coordinates of vertices. These operations include one or more of the following: modeling transformations, display transformations, projection transformations, viewpoint divisions, viewport transformations that modify vertex coordinates, and other operations that modify non-coordinate attributes.

[0019] The vertex shader stage 304 is implemented partially or completely as a vertex shader program that runs on one or more compute units 132. The vertex shader program is provided by the processor 102 and is based on a program pre-written by a computer programmer. The driver 122 compiles such a computer program to generate a vertex shader program having a form suitable for execution within the compute units 132.

[0020] The hull shader stage 306, the mosaicizer stage 308, and the domain shader stage 310 work together to implement tessellation, which converts simple primitives into more complex primitives by subdividing the primitives. The hull shader stage 306 generates patches for tessellation based on the input primitives. The mosaicizer stage 308 generates a set of samples for the patches. The domain shader stage 310 calculates vertex positions for vertices corresponding to the samples in the patches. The hull shader stage 306 and the domain shader stage 310 can be implemented as shader programs executed on the compute unit 132 that are compiled by the driver 122 along with the vertex shader stage 304.

[0021] The geometry shader stage 312 performs vertex operations on a primitive basis. A variety of different types of operations can be performed by the geometry shader stage 312, including operations such as point sprite expansion, dynamic particle system operations, fur-fin generation, shadow volume generation, single pass render-to-cubemap, per-primitive material swapping, and per-primitive material setup. In some cases, a geometry shader program compiled by the driver 122 and executed on the compute unit 132 performs the operations of the geometry shader stage 312.

[0022] The rasterizer stage 314 accepts and rasterizes simple primitives (triangles) generated upstream from the rasterizer stage 314. Rasterization involves determining which screen pixels (or sub-pixel samples) are covered by a particular primitive. Rasterization is performed by fixed function hardware.

[0023] The pixel shader stage 316 calculates the output values ​​of screen pixels based on the primitives generated upstream and the results of rasterization. The pixel shader stage 316 can apply textures from texture memory. The operations of the pixel shader stage 316 are performed by pixel shader programs compiled by the driver 122 and executed on the compute unit 132.

[0024] The output merge stage 318 accepts the outputs from the pixel shader stage 316, merges them into the frame buffer, and performs operations such as z-testing and alpha blending to determine the final color of the screen pixel.

[0025] A processing workload on APD 116 often involves the execution of multiple "passes." A "pass" is a unit of processing that accepts some input (e.g., input from another pass or input to the entire processing workload) and produces output. Some outputs include outputs to other passes, and other outputs include the final output of the processing workload, such as an image for display on a screen. Thus, the particular order of passes, the inputs they use, and the outputs they produce specify the entire processing workload.

[0026] An example of a path is the processing of data through a graphics processing pipeline 134 configured in a particular way (e.g., certain stages are enabled or disabled, certain programmable stages have particular selected shader programs for execution). The input or output for any particular path through the graphics processing pipeline 134 is configurable based on the specified operation of the graphics processing pipeline 134. Exemplary graphics processing paths include a geometry path that renders meshes to a set of render targets and a depth buffer, and a post-processing path that renders screen space quads with vertex and pixel shader programs. Another example of a path is a compute workload that executes a particular compute shader program. A compute shader program is a shader program that executes on a compute unit 132 but does not execute as part of the graphics processing pipeline 134. A compute workload is the execution of a compute shader program for a particular input data set to generate a particular set of output data.

[0027] It is possible to "manually" specify each individual command of a processing workload's paths. More specifically, an application 126 (or other entity, such as driver 122) can issue commands to driver 122 and / or acceleration processing device 116 to execute paths ad-hoc. However, this disclosure presents techniques for a more automated method for specifying a multi-path processing workload.

[0028] Specifically, this disclosure presents techniques for programmably specifying processing workloads as task groups. Figure 4A illustrates the APD scheduler 136 of Figure 2 configured to execute a task graph specification program to generate a task graph and to schedule operations for execution on the APD 116 based on the generated task graph.

[0029] The APD scheduler 136 includes a task graph specification processor 402 and a command scheduler 404. The task graph specification processor 402 accepts as input a task graph specification program. The task graph specification program programmatically specifies how to generate a task graph that describes a processing workload. The processing workload includes a graph of tasks and resources, as described further herein (e.g., with reference to FIG. 5). The task graph specification processor 402 executes the task graph specification program, which, when executed, generates a task graph that specifies the processing workload as a task graph.

[0030] The command scheduler 404 accepts a task graph as input, performs scheduling of operations to execute the processing workload specified by the task graph, and instructs the APD 116 to execute those operations. In some embodiments, the command scheduler 404 accepts APD feedback information and uses the feedback information to schedule operations on the APD 116. The APD feedback information includes various information about the state of the APD 116, such as performance metrics, occupancy metrics, resource utilization, and other types of information. Some example items of information about the state of the APD 116 include timestamps when work is initiated and completed by the graphics processing pipeline 134, hardware performance counters of the SIMD unit 138, the percentage of time that various such hardware units are busy, and memory bandwidth utilization. The command scheduler 404 uses the timestamps to evaluate the overall effectiveness of task scheduling. Additionally, the timestamps indicate the overall duration of passes 502. Thus, in some embodiments, the command scheduler 404 prioritizes longer passes when scheduling. Hardware block utilization counters are used to help group paths that are bottlenecked together in various hardware blocks to achieve better overall utilization. In some examples, the command scheduler 404 is part of the APD 116 and can obtain such performance metrics directly from other components of the APD without intervention by the processor 102, applications, drivers, or operating system.

[0031] In the example of FIG. 4A , either or both of the task graph specification processor 402 and the command scheduler 404 are implemented as software executing on a processor, hardware circuitry configured to perform the operations described herein, or a combination thereof. Although depicted as two separate units, the task graph specification processor 402 and the command scheduler 404 are the same unit in some embodiments. FIG. 4A illustrates an example in which the task graph specification processor 402 and the command scheduler 404 are within the APD 116. FIG. 4B illustrates an example in which the task graph specification processor 402 is software executing on the processor 102 and the command scheduler 404 is implemented within the APD 116 as described with respect to FIG. 4A . FIG. 4C illustrates an example in which the task graph specification processor 402 and the command scheduler 404 are programs executing on the processor 102.

[0032] Although the command scheduler 404 is described as outputting commands for execution on the APD 116, it should be understood that the techniques described herein may be applied to generating commands for execution on a processor other than the APD 116, such as a central processing unit (e.g., the processor 102) or another unit.

[0033] 5 illustrates an example task graph 500. Task graph 500 specifies a processing workload as a directed acyclic graph of paths 502 and buffers 504. Paths 502 include particular units of work, such as the processing of particular data through graphics processing pipelines 134 configured in a particular manner, or compute shader programs, as described above. Buffers 504 include data used as inputs to paths 502 and / or as outputs to paths 502. Buffers 504 may be inputs to one path 502 and outputs to another path, or may be inputs or outputs to the entire processing workload (corresponding to a single task graph). Some examples of buffers include images produced by particular paths 502 through graphics processing pipelines 134, data produced by compute shader programs, or non-image data produced by paths 502 through graphics processing pipelines 134.

[0034] The task graph 500 includes arrows between paths 502 and buffers 504. An arrow from a buffer 504 to a path 502 indicates that the path 502 accepts the data in that buffer 504 as input. An arrow from a path 502 to a buffer 504 indicates that the path 502 produces the data in the buffer 504 as output. Thus, the task graph 500 specifies dependencies between paths 502. More specifically, a path 502 that accepts a buffer 504 produced by another path 502 as input must wait to execute until the buffer 504 is actually produced. Thus, paths 502 that accept buffers 504 as input depend on other paths 502 that produce those buffers 504 as output. If a third path 502 depends on a second path 502, and the first path 502 depends on the third path 502, then the first path 502 is also considered to depend on the second path 502. In other words, dependencies propagate through the arrows of task graph 500. Two paths 502 are independent if neither depends on the other.

[0035] As described above, a task graph specification program is a program that specifies how to construct a task graph 500. Thus, the task graph specification program indicates which paths 502 occur and which buffers 504 are inputs and outputs of the paths 502. The task graph specification program can consider runtime data, such as user-defined runtime data, data generated by the APD 116, or other data, when making decisions regarding whether to include particular paths 502 and / or buffers 504 in the task graph 500 and how the paths 502 consume and / or produce buffers 504. Thus, the task graph specification program is not simply a static description of the task graph 500 that is evaluated by the task graph specification processor 402. Instead, the task graph specification program can variably construct the task graph 500 based on runtime parameters.

[0036] 6A and 6B illustrate an exemplary task graph specification program format, and FIGS. 7A and 7B illustrate alternative task graph specification program formats. In FIG. 6A, the task graph specification program includes path definitions 605, path invocations 606, and program control flow instructions 604. The path definitions 605 define the operations and methods involved in executing paths on the APD 116. The path invocations 606 represent instructions to the task graph specification processor 402 to generate paths 502 of the output task graph 500. The control flow instructions 604 are instructions for controlling how the path invocations 606 are generated. As described above, the task graph specification processor 402 executes the control flow instructions 604 and path invocations 606 to construct the task graph 500 for scheduling by the command scheduler 404 and execution by the APD 116.

[0037] In some embodiments, a pass definition 605 includes resource import / export instructions 608, state instructions 610, and path commands 612; in some embodiments, the pass definition 605 omits the state instructions 610. The resource import / export instructions 608 specify one or more resources to be imported (read) and / or exported (written) by the pass definition 605 when executed on the APD 116. The term "resource" has the same meaning as the term "buffer" (i.e., buffer 504 in FIG. 5). In some embodiments, the state instructions 610 indicate either or both which shaders are used for graphics or compute operations for a pass when executed on the APD 116 or which graphics pipeline state is used for a pass when executed on the APD 116, although other states are also contemplated. The path commands 612 are the actual work performed for the pass 606. Path commands 612 receive input from one or more imported resources, process those one or more imported resources, and output to one or more exported resources. In some embodiments, path commands 612 include commands such as drawing commands (e.g., drawing geometry by sending the geometry through a particular configuration of the graphics processing pipeline 134), dispatch commands (dispatching a general-purpose computational workload for execution), copy commands (copying one or more portions of memory to one or more other locations), clear commands (clearing one or more portions of memory), resource layout transitions (changing the layout of one or more resources), cache actions (flushing or prefetching cached data), synchronization operations (synchronization between APD 116 hardware blocks or events, etc. For example, because drawing commands and dispatch commands may compete for memory bandwidth, despite the fact that these items can logically execute in parallel, a programmer may insert explicit synchronization commands that cause one to start only after the other has completed, or other commands executable by the APD 116.In the case of cache and sync actions, the command scheduler 404 can, and sometimes does, generate such actions / behaviors automatically, however, the programmer can also explicitly specify these actions.

[0038] The control flow instructions 604 control the program flow of the task graph specification program 600 during execution by the task graph specification processor 402. The control flow instructions 604 include control flow instructions such as conditional statements (e.g., if statements), loops, etc. The control flow instructions 604 allow the task graph specification program 600 to construct the task graph 500 at runtime based on runtime parameters. Loops allow the generation of multiple passes 502 from a single pass definition 605, each with different parameters (import / export data and shader / pipeline configuration). Conditions can specify whether a particular node definition 605 is actually instantiated as a node.

[0039] In some situations, the path invocation 606 specifies arguments for one or more of the resource import / export instructions 608 or the state instructions 610. In such situations, the arguments specify one or more of the state (e.g., shader program, graphics processing pipeline 134 configuration) to use for the particular path generated as a result of the path invocation 606, one or more resources to import into the path generated as a result of the path invocation 606, or one or more resources to export from the path generated as a result of the path invocation 606. Furthermore, in some situations, the arguments are specified as variables, meaning that the values ​​of the arguments are determined at runtime (e.g., based on data available to the task graph specification program 600 and / or as a particular result from the execution of the control flow instructions 604). In summary, the arguments, and therefore the state, import data, and export data, are elements that can be specified at runtime (when the task graph specification program 600 is executed), meaning that the programmer writing the task graph specification program 600 does not need to hard-code such information.

[0040] Figure 6B illustrates an example task graph 650 generated based on the task graph specification program 600 in the format described in Figure 6A. An example task graph specification program shown in code is now provided.

[0041] [Table 1]

[0042] The above task graph specification program specifies two pass definitions: one for PassA that calls Draw1(), and one for PassB that calls Draw2() and Draw3(). The pass call instantiates passA on the graph as input resource r1 and output resource r2. Based on certain conditions, the pass call instantiates zero, one, or two instances of PassB, input resource r2, and output resource r3. Note that Draw3() internally uses data produced by Draw2(), but this data is not specified as a resource because the data is produced and consumed entirely within PassB.

[0043] FIG. 6B illustrates two different task graphs 650 generated from two different executions of the task graph specification in Table 1. In the first task graph 650(1), both condition A and condition B are true. Thus, the generated graph includes two instances: path A 652(1), which reads from resource 1 654(1) and outputs to resource 2 654(2), and path B 652(2), which reads from resource 2 654(2) and outputs to resource 3 654(3). For task graph 650(2), only condition A is true. Thus, only one instance of path B 652(2) is generated.

[0044] 7A and 7B illustrate a task graph specification program 700 in another format and a task graph 750 generated as a result of execution of the task graph specification program 700 by the task graph specification processor 402. FIG.

[0045] The task graph specification program 700 includes control flow instructions 702, resource import / export instructions 704, state instructions 706, and path commands 708. Path commands 708 are individual commands executable by the APD 116. Some example path commands 708 include resource import / export instructions 704, drawing commands (e.g., drawing geometry by sending the geometry through the graphics processing pipeline 134 in a particular configuration), dispatch commands (dispatching general-purpose computational workloads for execution), copy commands (copying one or more portions of memory to one or more other locations), clear commands (clearing one or more portions of memory), resource layout transitions (changing the layout of one or more resources), cache actions (flushing or prefetching cached data), synchronization operations (synchronization between APD 116 hardware blocks or events, etc.), etc. For example, because draw commands and dispatch commands may compete for memory bandwidth, despite the fact that these items can logically execute in parallel, a programmer may insert explicit synchronization commands that cause one to start only after the other has completed, or other commands that can be executed by APD 116. In the case of cache actions and synchronization operations, command scheduler 404 can, and sometimes does, generate such actions / operations automatically; however, a programmer can also specify these actions explicitly.

[0046] Resource import / export commands 704 specify resources to be read and / or written by path commands 708. State instructions 706 are instructions that define and / or modify the state of APD 116 (such as the state of graphics processing pipeline 134) that a shader program uses for graphics processing pipeline 134 or compute shader dispatch or other state. Control flow instructions 702 control program flow when task graph specification processor 402 executes task graph specification program 700.

[0047] During execution, the task graph specification processor 402 traverses the task graph specification program 700 to identify path blocks 710, from which the task graph specification program 700 generates paths 502 for the corresponding task graph. In some examples, a path block 710 is a portion of the task graph specification program 700 that contains one (or a specified number) path commands, terminates in a path command, and contains no control flow instructions for execution by the task graph specification processor 402. In other examples, the path block 710 contains a different fixed number of path commands 708 or a variable number of path commands 708. In some examples, the path block 710 need not terminate in a path command 708. In some examples, the task graph specification processor 402 identifies portions of the task graph specification program 700 that contain at least one path command 708 and terminate in a path command 708, and partitions each such portion based on the location of the path command 708 into path blocks 710. More specifically, from those portions, the task graph specification program 700 generates one path block 710 for each specified number of path commands 708. In some examples, each such path block 710 includes all instructions from the previous path block 710 to the last path command 708 in the path block 710, but does not include any control flow instructions 702.

[0048] With the path blocks 710 identified, the task graph specification processor 702 generates the task graph 500 as follows: Each path block 710 contains the definition of one or more import resources and one or more output resources. These resources make up the buffers 504 of the task graph. Resource reads and writes make up the directed edges of the task graph. An edge directed from a buffer 504 to a path 502 is a resource read, and an edge directed from a path 502 to a buffer 504 is a resource write.

[0049] The following table contains an example code listing for a task graph specification program of the form set forth in Figure 7A. Figure 7B shows an example task graph 750 generated from the program.

[0050] [Table 2]

[0051] In this example, the path block contains only one path command. A path command is identified as a command that is any of a set of commands designated as path commands, such as those described elsewhere herein. In other words, the task graph specification processor 402 has access to a list of command types that are considered path commands and identifies all commands of the program found in the list as path commands. The example task graph specification program of Table 2 contains three different path blocks. The first path block is the section from the SetState(State1) statement to the first path command Draw1(). The second path block is the section of the if statement that begins with the SetState(State2) statement and ends with the Draw2() path command. The third path block is the section of the if statement that begins with SetResource(read:r5, write:r3) and ends with Draw3().

[0052] In one example of the execution of the task graph specification program of Table 2, shown as task graph 750(1), condition B is true. Thus, task graph specification processor 402 generates paths in task graph 750(1), including path 1, which inputs resource 1 754(1) and outputs resource 2 754(2), path 2, which inputs resource 2 754(2) and outputs resource 5 754(5), and path 3, which inputs resource 5 754(5) and outputs resource 3 754(3).

[0053] In another example of the execution of the task graph specification program of Table 2, shown as task graph 750(2), condition B is false. Therefore, task graph specification processor 402 generates path 1, which inputs resource 1 754(1) and outputs resource 2 754(2), but does not generate path 2 or path 3.

[0054] As described above, the command scheduler 404 schedules tasks for execution on the APD 116 based on the task graph output by the task graph specification processor 402. The command scheduler 404 determines the execution order of the paths based on certain considerations. One set of considerations includes the dependencies of the paths 502. More specifically, paths 502 with one or more dependencies cannot be reordered with respect to each other. For example, if a first path 502 depends on a second path 502, the command scheduler 404 executes the first path 502 after the second path 502 is completed. In some examples, the command scheduler 404 inserts appropriate instructions or markers, such as barrier instructions or primitives, to enforce ordering among the paths 502. If the first pass 502 is not dependent on the second pass 502, or if the second pass is not dependent on the first pass 502, the command scheduler 404 is free to order the execution of the first pass 502 and the second pass 502 relative to each other, possibly in any technically feasible manner. For example, in some instances, the command scheduler 404 causes the independent passes to execute sequentially, while in other instances, the command scheduler 404 causes the independent passes to execute during at least partially overlapping time periods. Often, the command scheduler 404 selects passes 502 to execute together to optimize certain desired resources, such as memory usage, processing unit resources, register usage, etc., or to minimize execution time.

[0055] 8A illustrates a task graph 800, according to an example. A first path 802(1) reads resource 1 804(1) and outputs resource 2 804(2). Path 2 802(2) inputs resource 2 804(2) and outputs resource 3 804(3). Path 3 802(3) inputs input resource 2 804(2) and outputs resource 4 804(4). Path 4 802(4) inputs resource 2 804(2) and outputs resource 5 804(5). Path 5 inputs input 802(5) and outputs resource 6 804(6).

[0056] Path 2 802(2), path 3 802(3), and path 4 802(4) are independent and dependent on path 2 802(1). Path 5 802(5) is dependent on path 1 802(1), path 2 802(2), path 3 802(3), and path 4 802(4). Because path 2 802(2), path 3 802(3), and path 4 802(4) are independent, command scheduler 404 can reorder the execution of these paths 802 relative to one another. However, command scheduler 404 enforces ordering among dependent paths, meaning that path 1 402(1) executes before path 2 802(2), path 3 802(3), and path 4 802(4), which execute before path 5 802(5).

[0057] FIG. 8B illustrates an exemplary execution order 850 of paths 802 scheduled by command scheduler 404. As illustrated, time progresses to the right. In example 1 850(1), path 1 802(1) executes before path 2 802(2) and path 3 802(3), and path 4 802(4), path 2 802(2), path 3 802(3), and path 4 802(4) execute concurrently. In example 2 850(2), path 1 802(1) executes before path 2 802(2) and path 3 802(3). Path 4 802(4) executes after path 2 802(2) and path 3 802(3), and executes before path 5 802(5). In example 3 850(3), paths 802 execute sequentially.

[0058] In some embodiments, a particular programming language is used for the techniques of Figures 7A and 7B, as well as the entire task graph specification program for the control flow instructions 604 and path invocations 606 (but not the path definitions 605) of Figures 6A and 6B. In such embodiments, in the example of Figures 6A and 6B, the path definitions 605 are written in a different programming language that is compiled. These path definitions 605 are called by the APD 116 when the corresponding paths are executed.

[0059] To determine how to schedule an independent path 802, the command scheduler 404 considers the state of the APD 116 and performance metrics of the path 802. There are several different items of information that constitute the “state” of the APD 116. Some examples of performance metrics include resource usage and execution rate. Resources include memory, registers, and processing units (such as the SIMD unit 138). In some examples, the command scheduler 404 records performance metrics of an executing path 802 for use during future scheduling of the same path 802. More specifically, the command scheduler 404 updates estimated resource usage data for particular paths based on previously observed resource usage for those paths and utilizes such updated estimated resource usage data to make future scheduling decisions.

[0060] As described above, the command scheduler 404 may schedule independent paths 502 for execution simultaneously on one or more compute units 132. The command scheduler 404 determines whether to schedule different paths 502 simultaneously based on the nature of the paths 502 and the capabilities and status of the APD 116. More specifically, in some situations, it is advantageous to schedule independent paths 502 simultaneously, while in other situations, it is advantageous to schedule independent paths 502 non-concurrently. The term "concurrently" means that two (or more) paths 502 execute within at least partially overlapping time periods.

[0061] Paths 502 consume resources such as memory, processing units (e.g., SIMD units 138), registers, etc. Therefore, in a situation where the APD 116 is limited by certain resources and concurrent execution of paths 502 would exceed the number of available resources, the command scheduler 404 does not schedule such paths 502 concurrently. Conversely, in a situation where resources are available for concurrent execution, the command scheduler 404 may schedule such paths 502 concurrently. Additionally, during operation, the command scheduler 404 determines whether to emphasize aspects such as speed or resource usage based on such determination and schedules independent paths 502. In one example, executing multiple paths 502 concurrently reduces the total execution time of those paths 502 but increases the total resource usage at any given time. Executing multiple paths 502 sequentially increases the total execution time but reduces the total resource usage at a given time. Conversely, executing multiple paths 502 concurrently reduces the total execution time but increases the total resource usage at a given time.

[0062] Another consideration for the command scheduler 404 is whether different paths 502 use different resources. More specifically, if different paths 502 use different types of resources, or if one path 502 uses a small amount of a particular resource and another path 502 uses a large amount of the resource, it may be beneficial to schedule them together. In one example, one path 502 makes very heavy use of the SIMD unit 138, while another path 502 makes very little use of the SIMD unit 138. In this situation, the command scheduler 404 may schedule these two paths simultaneously.

[0063] Additionally, the command scheduler 404 can reduce memory usage at a given time by reducing the duration for which a particular buffer 504 is used. More specifically, by analyzing the task graph 500, the command scheduler 404 can determine the lifetime of any particular buffer 504. The lifetime of a buffer 504 is the period between the path 502 that produces the buffer 504 and the last path 502 that consumes that buffer 504. In some embodiments, in some situations, such as when memory usage is high, the command scheduler 404 minimizes the lifetime of a buffer 504 by simultaneously scheduling paths 502 whose completions tend to end their lifetimes. In other words, to reduce the lifetime of a buffer 504, the command scheduler 404 identifies paths 502 that are the last path candidates 502 that use the buffer 504, and the command scheduler 404 identifies paths 502 on which the last path candidate 502 depends. The command scheduler 404 then attempts to schedule such identified paths 502 for concurrent execution when possible (e.g., independent paths 502 may be scheduled for concurrent execution). A "last path candidate" is either the last path in topological sort order using a buffer 504, or an independent path 502 that could be the last path in topological sort order using a buffer 504. This technique can also be used to keep a resource working set to stay in a particular portion of the memory hierarchy, such as a cache or other memory. In other words, the above technique can be used to limit the memory usage of a path 502 so that it fits within a particular portion of the memory hierarchy (e.g., all in level 0 cache, all in level 1 cache, etc.).

[0064] A topological sort is a technique in which the nodes of a directed acyclic graph are ordered based on the graph's directed edges. In the sort, nodes at the start of directed edges always precede nodes at the end of those directed edges. Any order for nodes that do not have directed edges between them produces a valid topological sort. Thus, if multiple paths 502 use a buffer 504 but are independent of each other, and such paths 502 are the last paths to use a buffer 504 in the topological sort order, then each such path 502 is a candidate final path 502.

[0065] In some embodiments, the paths 502 that exist during their lifetime include all multiple independent paths 502 that use the buffer 504. By aggressively scheduling consumers of the buffer 504 to execute early, the lifetime of the buffer 504 is reduced.

[0066] In an additional scheduling technique, the command scheduler 404 automatically splits a large path into several smaller paths and schedules those paths independently.

[0067] 9 is a flow diagram of a method 900 for constructing a task graph for scheduling, according to an example. Although described with respect to the systems of FIGS. 1-8B, one skilled in the art will recognize that any system configured to perform the steps of method 900 in any technically feasible order is within the scope of this disclosure.

[0068] Method 900 begins at step 902, when task graph specification processor 402 receives a task graph specification program 902 for execution. The task graph specification program programmatically specifies how to generate a task graph for scheduling and execution on APD 116. At step 904, task graph specification program 902 traverses task graph specification program 902, executing control flow instructions specified within task graph specification program 902. These control flow instructions include conditional instructions, loops, and other instructions. The ordering of step 904 before steps 906-910 in FIG. 9 does not necessarily mean that step 904 is executed only once in that order. Instead, it should be understood that step 904 indicates that the control flow instructions are executed as specified by the task graph specification program, regardless of the order in which they appear within that program.

[0069] In step 906, the task graph specification processor 402 generates nodes (corresponding to paths 502) based on the path instructions of the task graph specification program. For the format of FIGS. 6A-6B, the path instructions are path calls 606. Each execution of a path call generates nodes of the task graph in the order specified by the control flow instructions. For the format of FIGS. 7A-7B, the path instructions are path blocks 710. Specifically, the task graph specification processor 402 identifies these path blocks 710 as specified elsewhere herein and generates nodes from these path blocks 710.

[0070] In step 908, the task graph specification processor 402 generates resources and directed edges between the generated nodes in the task graph based on the resource usage of each node. For resources that are read by a node, the task graph specification processor 402 generates a directed edge from the resource to the node. For resources that are written by a node, the task graph specification processor 402 generates a directed edge from the node to the resource. In step 910, the task graph specification processor 402 outputs the task graph to the command scheduler 404 for scheduling on the APD 116 or other device.

[0071] 10 is a flow diagram of a method 1000 for scheduling operations on a processing device based on a task graph, according to an example. Although described with respect to the systems of FIGS. 1-8B, one skilled in the art will recognize that any system configured to perform the steps of method 1000 in any technically feasible order is within the scope of this disclosure.

[0072] Method 1000 begins at step 1002, where command scheduler 404 receives a task graph. In some examples, the task graph is generated by task graph specification processor 402. In some examples, command scheduler 404 is on the same processing device as task graph specification processor 402 or on a different processing device than task graph specification processor 402. Various exemplary configurations are shown in Figures 4A-4C.

[0073] In step 1004, the command scheduler 404 identifies independent and dependent paths in the task graph. Techniques for identifying such paths are described elsewhere herein. In step 1006, the command scheduler 404 determines how to schedule them for execution on the processing device based on performance criteria. Many techniques and considerations for scheduling paths are described elsewhere herein, such as before the description of FIG. 9, including the description in conjunction with FIGS. 8A and 8B. The scheduling step 1006 includes performing any one or more of these methods. Generally, the command scheduler 404 schedules independent paths to provide what is considered to be the best utilization of particular resources (e.g., memory, registers, processing time, etc.) and / or other factors (e.g., hardware units) of the processing device. Furthermore, for paths that must occur sequentially due to dependencies, the command scheduler 404 sequentially generates such paths. In some examples, the command scheduler 404 sequentially generates such paths via a barrier command that allows all operations of one path to complete before a dependent path. In step 1008, the command scheduler 404 sends the scheduled commands to the processing device for execution as scheduled.

[0074] In the disclosure provided herein, the device that ultimately executes the commands scheduled for execution of a task graph is the APD 116. However, it should be understood that embodiments are contemplated hereby in which processing devices other than those specifically described execute these commands. In some examples, a device similar to the APD 116 but lacking graphics capabilities (e.g., a device that does not include a graphics processing pipeline 134 and is capable of executing computational workloads) is a processing device. In other examples, a central processing unit is a processing device. In other examples, any other processing unit is a processing device.

[0075] Any of the units in the figure, including processor 102, storage 106, memory 104, input driver 112, input device 108, output driver 114, APD 116, display device 118, output device 110, driver 122, operating system 120, application 126, graphics processing pipeline 134, compute unit 132, SIMD unit 138, APD scheduler 136, and each stage of graphics processing pipeline 134 in FIG. 3 may be implemented as hardware circuitry configured to perform the described operations, software configured to perform the described operations, or a combination of hardware and software, as appropriate.

[0076] It should be understood that many variations are possible based on the disclosure herein, and although features and elements are described above in particular combinations, each feature or element can be used alone without other features and elements, or in various combinations with or without other features and elements.

[0077] The provided methods can be implemented in a general-purpose computer, processor, or processor core. Suitable processors include, by way of example, a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), and / or a state machine. Such processors can be fabricated by configuring a manufacturing process using the results of processed hardware description language (HDL) instructions and other intermediate data, such as a netlist (such instructions can be stored on a computer-readable medium). The result of such processing can be a mask work, which is then used in a semiconductor manufacturing process to produce a processor implementing features of the present disclosure.

[0078] The methods or flow diagrams provided herein may be implemented in a computer program, software, or firmware embodied in a non-transitory computer-readable storage medium for execution by a general-purpose computer or processor. Examples of non-transitory computer-readable storage media include read-only memory (ROM), random-access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media (e.g., internal hard disks and removable disks), magneto-optical media, and optical media (e.g., CD-ROM disks and digital versatile disks (DVDs)).

Claims

[Claim 1] 1. A method for a computer to generate a task graph for workload scheduling based on a task graph specification program, comprising: executing control flow instructions of the task graph specification program to traverse the task graph specification program; generating a path node of the task graph based on a path instruction of the task graph specification program; generating resource nodes and directed edges based on the resource declarations of the task graph specification program; and outputting the task graph to a command scheduler for scheduling. method.

Images