Including dedicated accelerators in graph nodes

Abstract

Systems, apparatus, and methods are disclosed for implementing hierarchical scheduling in a fixed-function graphics pipeline. In various embodiments, a processor includes a pipeline comprising a plurality of fixed-function units and a scheduler. The scheduler is configured to schedule a first operation for execution by one or more fixed-function units of the pipeline by scheduling the first operation with the first unit of the pipeline in response to a first mode of operation, and to schedule a second operation for execution by a selected fixed-function unit of the pipeline by scheduling the second operation directly to the selected fixed-function unit, regardless of the sequential placement of the one or more fixed-function units in the pipeline in response to a second mode of operation.

Description

[Background technology]

[0001] Description of Related Art A graphics processing pipeline performs a series of steps to transform input geometry into an image that is displayed on a screen. In some instances, these steps are specified as commands by a host, such as a central processing unit that runs an application. Many rendering workloads require large arrays of commands, and therefore generating and processing these arrays is a complex task.

[0002] Generally, a graphics processing pipeline consists of fixed-function units configured to perform a set of dedicated functions in a graphics processing unit designed to support fixed-function units. Each fixed-function unit typically has a programming abstraction with a series of well-defined and specifically named graphics pipeline stages. However, in a graphical pipeline containing multiple fixed-function units integrated into a pipeline, each unit performs its assigned tasks in a sequential manner, and there is no way to know the status of a particular intermediate task until the entire pipeline has been traversed by an operation.

[0003] In view of the above, there is a need for an improved system and method for controlling a graphics processing pipeline having fixed function units.

[0004] Advantages of the methods and mechanisms described herein may be better understood by referring to the following description taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0005] [Figure 1] FIG. 1 is a block diagram of one embodiment of a computing system. [Figure 2]FIG. 1 is a block diagram illustrating a parallel processor implementing a hierarchical scheduler. [Figure 3] FIG. 2 is a block diagram illustrating details of an exemplary graphics processing pipeline. [Figure 4A] FIG. 2 illustrates an exemplary scheduler graph. [Figure 4B] FIG. 2 illustrates an exemplary scheduler graph. [Figure 5] FIG. 2 is a block diagram illustrating control logic for implementing hierarchical scheduling for pipelines. [Figure 6A] FIG. 1 illustrates an exemplary graphics processing pipeline performing in different modes of operation. [Figure 6B] FIG. 1 illustrates an exemplary graphics processing pipeline performing in different modes of operation. [Figure 7] FIG. 1 is a block diagram illustrating an example data flow configuration for a graphics processing pipeline implementing a shader. [Figure 8] FIG. 1 illustrates a method for scheduling operations to one or more fixed function units in a pipeline. DETAILED DESCRIPTION OF THE INVENTION

[0006] In the following description, numerous specific details are set forth to provide a thorough understanding of the methods and mechanisms presented herein. However, those skilled in the art should recognize that various embodiments can be practiced without these specific details. In some instances, well-known structures, components, signals, computer program instructions, and techniques have not been shown in detail to avoid obscuring the approaches described herein. It should be understood that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some elements may be exaggerated relative to other elements.

[0007] Systems, apparatus, and methods are disclosed for implementing scheduling of fixed function units in a pipeline. In various embodiments, a processor includes a pipeline comprising one or more fixed function units, control logic, and a scheduler. The scheduler is configured to schedule one or more operations into the pipeline using a pipeline mode (first mode) or a direct access mode (second mode). In the pipeline mode, operations scheduled for execution by fixed function units of the pipeline enter the head of the pipeline and have results exit the end of the pipeline, regardless of where the fixed function unit is located in the pipeline. In the direct access mode, the control logic may schedule operations for execution by selected fixed function units of the pipeline by using data from the scheduler to schedule the operations directly to the selected fixed function units without having to enter the head of the pipeline or otherwise traverse other fixed function units in the pipeline.

[0008] 1, a block diagram of one embodiment of computing system 100 is shown. In one embodiment, computing system 100 includes at least processors 105A-105N, control unit 110, input / output (I / O) interface 120, bus 125, memory controller 130, network interface 135, memory device 140, power supply 145, power management unit 150, display controller 160, and display 165. In other embodiments, computing system 100 includes other components and / or is arranged differently. Processors 105A-105N represent any number of processors included in system 100, with the number of processors varying from embodiment to embodiment.

[0009] In one embodiment, processor 105A is a general-purpose processor such as a central processing unit (CPU). In one embodiment, processor 105N is a data-parallel processor with a highly parallel architecture. Data-parallel processors include graphics processing units (GPUs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc. In one embodiment, processor 105N is a GPU that provides pixels to display controller 160 to be driven to display 165. In some embodiments, processors 105A-105N include multiple data-parallel processors. In one embodiment, control unit 110 is a software driver running on processor 105A. In other embodiments, control unit 110 includes control logic separate from processors 105A-105N and / or embedded within processors 105A-105N. Generally speaking, control unit 110 is any suitable combination of software and / or hardware.

[0010] Memory controller 130 represents any number and type of memory controller accessible by processors 105A-105N. Memory controller 130 is coupled to any number and type of memory devices 140. Memory devices 140 represent any number and type of memory devices. For example, types of memory in memory devices 140 may include dynamic random access memory (DRAM), static random access memory (SRAM), NAND flash memory, NOR flash memory, ferroelectric random access memory (FeRAM), etc.

[0011] I / O interface 120 represents any number and type of I / O interface (e.g., a peripheral component interconnect (PCI) bus, a PCI-Extended (PCI-X), a PCI Express (PCI Express) bus, a gigabit Ethernet (GBE) bus, or a universal serial bus (USB)). Various types of peripheral devices (not shown) are coupled to I / O interface 120. Such peripheral devices include (but are not limited to) displays, keyboards, mice, printers, scanners, media recording devices, external storage devices, network interface cards, and the like. Network interface 135 is used to receive and transmit network messages across a network. Bus 125 represents any type of bus or fabric having any number of links for connecting different components of system 100 to one another.

[0012] In one embodiment, queue 142 is located in memory device 140. In other embodiments, queue 142 is stored elsewhere in system 100. Queue 142 represents any number and type of queue allocated in system 100. In one embodiment, queue 142 stores rendering tasks to be performed for a frame being rendered. In one embodiment, rendering tasks are added to queue 142 based on input received over network interface 135. For example, in one scenario, input is generated by a user of a video game application and sent to system 100 over a network (not shown). In another embodiment, input is generated by a peripheral device connected to I / O interface 120.

[0013] In one embodiment, power management unit 150 manages the supply of power from power source 145 to the components of system 100, and power management unit 150 controls various power performance states of the components in system 100. In response to receiving updates from control unit 110, power management unit 150 causes other components in system 100 to increase or decrease their current power performance state. In various embodiments, changing the power performance state includes changing the current operating frequency of the device and / or changing the current voltage level of the device. A reduced power performance state of processor 105A-105N generally results in a longer time to complete computing tasks being performed by processor 105A-105N.

[0014] In one embodiment, control unit 110 sends commands to power management unit 150 to operate one or more processors 105 in a relatively high power performance state in response to determining that the number of tasks for a processor exceeds a threshold, that certain quality of service requirements need to be met, or otherwise.

[0015] In various embodiments, computing system 100 is a computer, a laptop, a mobile device, a server, or any of a variety of other types of computing systems or devices. It should be noted that the number of components in computing system 100 may vary from embodiment to embodiment. For example, in other embodiments, more or fewer components may be present than those shown in FIG. 1 . It should also be noted that in other embodiments, computing system 100 may include other components not shown in FIG. 1 and / or omit one or more of the components shown in computing system 100. Additionally, in other embodiments, computing system 100 may be structured in other ways than those shown in FIG. 1 .

[0016] Referring now to FIG. 2 , a parallel processor 200 implementing hierarchical scheduling domains is shown. In one embodiment, the parallel processor 200 includes multiple scheduling domains 204. Each scheduling domain 204 corresponds to a shader engine. As shown, each shader engine 204 includes multiple workgroup processors (WGPs) 208, each including one or more compute units (not shown). Each of the shader engines 208 is configured to execute multiple work items received from a command processor (also referred to as a “global scheduler”) 214 external to the scheduling domain 204. In one embodiment, each scheduling domain further includes a local work graph scheduler (WGS) 206 (or “scheduler”) and a local cache 210. In various embodiments, each of the shader engines 204 includes a local cache 210, while the cache 212 is shared by the shader engines 204. In this manner, data can be communicated between the shader engines 204. In an exemplary embodiment, each of the WGSs and the global scheduler can have access to an individual mailbox that can be used by a given entity to communicate with another entity in the system without using the main memory subsystem of parallel processor 200. In one example, a dedicated mailbox for the global scheduler can be located in cache 212. Additionally, each WGS 206 can also have a dedicated mailbox, which in one embodiment can be located in cache 210 associated with WGS 206. Other possible locations for the dedicated mailbox are also contemplated and are within the scope of this disclosure.

[0017] In one embodiment, the WGS 206 is configured to directly access the local cache 210, thereby avoiding the need to communicate through a higher level of the scheduling hierarchy. In this manner, scheduling latency is reduced and finer-grained scheduling can be achieved. That is, the WGS 206 can schedule work items to one or more WGPs 208 faster and more locally. Furthermore, the structure of the shader engines 204 is such that a single WGS 206 is available per shader 204, thereby making the shader engines 204 more easily scalable. For example, because each of the shader engines 204 is configured to perform local scheduling, additional shader engines can be easily added to the processor.

[0018] In one embodiment, a given shader 204 may be used to control one or more fixed-function units, which may be part of, for example, a graphics processing pipeline including multiple fixed-function units (e.g., graphics pipeline 300 of FIG. 3 ) or other processing pipeline composed of one or more fixed-function units, by exposing the given fixed-function unit to a graph scheduling mechanism via scheduler 206. In the following description, for purposes of explanation, reference may be made generally to a graphics pipeline. However, the methods and mechanisms described herein are applicable to pipelines other than graphics pipelines. For example, a given fixed-function unit may be externally controlled by scheduler 206, such that scheduler 206 notifies the fixed-function unit of one or more operations to be performed and may instruct the fixed-function unit to write data to a particular memory location (e.g., a dedicated mailbox) or system memory, the data relating to the results of execution of the one or more operations. Furthermore, in another embodiment, scheduler 206 may obtain information associated with the status of the fixed-function unit, whether the fixed-function unit is overloaded or underloaded, a signal error condition associated with the fixed-function unit, etc.

[0019] In operation, a given shader 204 may execute one or more pipelines of fixed-function units (e.g., a graphics pipeline or another pipeline) in different modes of operation to control the fixed-function units using scheduler 206. In one embodiment, in a first mode of operation, scheduler 206 may schedule operations for execution by the pipeline by scheduling the operation of a first fixed-function unit of the pipeline, such that the output of the operation's execution may be used as an input by a second fixed-function unit of the pipeline, which is connected to the first fixed-function unit in a sequential manner. That is, in the first mode of operation, each operation that is executed must traverse all fixed-function units of the pipeline before scheduler 206 can determine the output of the operation's execution.

[0020] However, in the second mode of operation, scheduler 206 can schedule operations for selected fixed function units in the pipeline regardless of the sequence of the fixed function units in the pipeline. That is, in the second mode of operation, the scheduler can directly assign operations for execution to selected fixed function units without having to sequentially traverse all units in the pipeline. To do so, scheduler 206 notifies the selected fixed function unit to perform the operation using the input data buffer of the selected fixed function unit. Furthermore, once the operation is performed by the selected fixed function unit, scheduler 206 can be notified using the memory subsystem or by writing to a dedicated mailbox accessible by scheduler 206. In one embodiment, scheduler 206 notifies the selected fixed function unit where to send data associated with the results of the execution of the operation, for example, to the input data buffer of another fixed function unit or to a memory location accessible by another scheduler.

[0021] In one embodiment, fixed function units may also send push messages to scheduler 206, and messages may be pulled from fixed function units by scheduler 206. Whether a message is pushed or pulled may depend on the type of message. For example, a message indicating completion of execution of operation 206 may be a push message sent from the fixed function unit to scheduler 206. Additionally, messages indicating overload or underutilization of the fixed function unit may be pulled from the fixed function unit by scheduler 206.

[0022] In one embodiment, the flow of data to and from the fixed function units may be established by scheduler 206 using a messaging protocol that may comprise messaging formats for different messages, such as messages for scheduling operations to be performed, notifications that operations have been performed by the fixed function units, messages comprising status reports, etc.

[0023] Advantageously, implementing a pipeline such that one or more fixed function units are controlled using nodes in a scheduling graph may provide better control of the pipeline while reducing latency in executing one or more operations, since traversing all units of the pipeline may not be required for a given operation. Furthermore, exposing one or more fixed function units to memory locations associated with shaders may facilitate accurate knowledge of overload or underutilization of fixed function units without having to wait for the entire pipeline to execute, thereby increasing efficiency in scheduling operations.

[0024] 3 is a block diagram illustrating an exemplary graphics processing pipeline. Graphics processing pipeline 300 includes one or more fixed function units, each performing a specific function of graphics processing pipeline 300. Each fixed function unit is implemented partially or fully as fixed-function, non-programmable hardware external to the compute units.

[0025] Input assembler 302 reads primitive data from user-filled buffers (e.g., buffers filled with requests from software executed by processor 200, such as applications) and assembles the data into primitives for use by the rest of the pipeline. Input assembler 302 can generate different types of primitives based on the primitive data contained in the user-filled buffers. Input assembler 302 formats the assembled primitives for use by the rest of the pipeline.

[0026] The vertex shader 304 processes the vertices of the primitives assembled by the input assembler 302. The vertex shader 304 performs various per-vertex operations, such as transformations, skinning, morphing, and per-vertex lighting. Transformation operations include various operations for transforming vertex coordinates. These operations include one or more of modeling transformations, viewing transformations, projection transformations, perspective division, viewport transformations (which modify vertex coordinates), and other operations that modify non-coordinate attributes.

[0027] The vertex shader 304 is implemented partially or completely as a vertex shader program that runs on one or more compute units. The vertex shader program is provided by the processor 200 and is based on a program prewritten by a computer programmer. A driver (not shown) compiles such a computer program to generate the vertex shader program in a format suitable for execution within the compute units.

[0028] The hull shader 306, tessellator 308, and domain shader 310 work together to implement tessellation, which converts simple primitives into more complex primitives by subdividing the primitives. The hull shader 306 generates patches for tessellation based on the input primitives. The tessellator 308 generates a set of samples for the patch. The domain shader 310 calculates vertex positions for vertices corresponding to the samples in the patch. The hull shader 306 and domain shader 310 can be implemented as shader programs that run on a compute unit and are compiled by a driver along with the vertex shader 304.

[0029] The geometry shader 312 performs vertex operations on a primitive basis. The geometry shader 312 can perform a variety of different types of operations, including operations such as point sprite expansion, dynamic particle system operations, fur-fin generation, shadow volume generation, single pass render-to-cubemap, per-primitive material swapping, and per-primitive material setup. In some cases, the geometry shader program, which is compiled by the driver and executed on the compute unit, performs the operations of the geometry shader 312.

[0030] The rasterizer 314 accepts and rasterizes simple primitives (triangles) generated upstream from the rasterizer 314. Rasterization consists of determining which screen pixels (or subpixel samples) are covered by a particular primitive. The pixel shader 316 calculates the output value of the screen pixel based on the upstream generated primitives and the results of rasterization. The pixel shader 316 can apply textures from texture memory. The operations of the pixel shader 316 are performed by pixel shader programs that are compiled by the driver and executed on the compute unit.

[0031] The output merger 318 accepts the outputs from the pixel shaders 316, merges them into the frame buffer, and performs operations such as z-testing and alpha blending to determine the final color of the screen pixels.

[0032] In many cases, processing of a workload by scheduler 206 involves generating multiple “nodes” for scheduling using a scheduling graph. A “node” may be a unit of processing that accepts some input (e.g., from another node or as input to the overall processing workload), processes the input, and generates an output. Some outputs may include outputs to other nodes, and other outputs may include the final output of the processing workload, such as an image displayed on a screen. Thus, the particular node order, the inputs those nodes use, and the outputs those nodes produce specify the overall processing workload. In one embodiment, for execution of operations by fixed-function units of pipeline 300, scheduler 206 may generate one or more nodes having operations as inputs and schedule each of these nodes to a predetermined fixed-function unit. The nodes may be executed in a sequential manner in a first operating mode or in a selected non-sequential order in a second operating mode (as illustrated in FIGS. 4A-4B ). That is, based on the operating mode, the output of a node result resulting from the execution of an operation by a first fixed function unit may be used as an input for a second fixed function unit in the pipeline in the first mode, or as an input for a selected fixed function unit in the second mode.

[0033] Referring now to FIG. 4A , an exemplary task graph 400 for scheduling the operation of the graphics pipeline 300 is shown. The task graph 400 specifies a processing workload as a directed acyclic graph of data buffers 402 (or “buffers 402”) and nodes 404. The nodes 404 contain particular units of work, such as the processing of particular data through the fixedly configured graphics pipeline 300, as described above. The data buffers 402 contain data used as input to the nodes 404 and / or as output to the nodes 404. The data buffers 402 may be input to one node 404 and output to another node 404, or may be input or output for the entire processing workload (corresponding to a single task graph). In one embodiment, the buffers may contain images produced by particular nodes 404 through the graphics pipeline 300 or non-image data produced by nodes 404 through the graphics pipeline 300.

[0034] The task graph 400 includes arrows between buffers 402 and nodes 404. An arrow from a buffer 402 to a node 404 indicates that the node 404 accepts the data in that buffer 402 as input. An arrow from a node 404 to a buffer 402 indicates that the node 404 produces the data in the buffer 402 as output. Thus, the task graph 400 specifies dependencies between nodes 404. More specifically, a node 404 that accepts a buffer 402 produced by another node 404 as input must wait to execute until the buffer 402 is actually produced. Thus, nodes 404 that accept buffers 402 as input depend on other nodes 404 that produce those buffers 402 as output. If a third node 404 depends on a second node 404, and a first node 404 depends on a third node 404, the first node 404 is also considered to depend on the second node 404. In other words, dependencies propagate through the arrows in the task graph 400. Two nodes 404 are independent if they do not depend on the other.

[0035] As described above, a task graph specification program is a program that specifies how to construct the task graph 400. Thus, the task graph specification program indicates which nodes 404 occur, and which buffers 402 are the inputs and outputs of the nodes 404. The task graph specification program can consider runtime data, such as user-defined runtime data, when making decisions regarding whether to include particular nodes 404 and / or buffers 402 in the task graph 400 and how the nodes 404 consume and / or produce buffers 402. Thus, the task graph specification program is more than just a static description of the task graph 400 that is evaluated by the processor 200. Instead, the task graph specification program can variably construct the task graph 400 based on runtime parameters.

[0036] Referring now to FIG. 4B, an exemplary task graph 410 for scheduling operations of graphics pipeline 300 using hierarchical scheduling is shown. In one embodiment, one or more shaders 418 in a hierarchical scheduling structure such as that described in FIG. 2 may use data buffers 412 and nodes 414 to schedule operations directly to fixed function units 416, regardless of the sequential order of the units in pipeline 300. For example, shader 412A may schedule one or more operations for fixed function unit 416A by writing data to data buffer 412A as input to node 414A, which is then provided to fixed function unit 416A for execution. In one embodiment, scheduler 206 may write data to buffer 412A by mapping some special memory or by using direct memory access (DMA) to write the data to buffer 412A. In another embodiment, scheduler 206 may also send a pointer to data in a specific memory location that can be read by fixed function unit 412A.

[0037] 4B, in an exemplary embodiment, the output generated by fixed function unit 416A may be accepted as input in buffer 412B. In one embodiment, the data in buffer 412B may be utilized based on the operating mode of processor 200. For example, in a first mode, the data in buffer 412B is used as input to node 414B, which may be used to schedule the operation of fixed function unit 416B, which is next in the sequential order of pipeline 300 after fixed function unit A. That is, in the first operating mode, i.e., pipeline mode, the output generated from the fixed function unit is used to generate a node that serves as input to the next fixed function unit in the sequence of pipeline 300. Once the entire pipeline 300 has been traversed, the final output is read by a given shader engine of processor 200. 4B, pipeline 300 may include two fixed-function units 416A and 416B, and once both fixed-function units have completed execution of their operations (or once data has traversed through the complete pipeline 300), the output is stored as data in buffer 412C, from which it may be accessed by shader 418B. Although Figure 4B describes pipeline 300 as including two fixed-function units, other configurations of pipeline 300 having more fixed-function units are contemplated and within the scope of the present disclosure.

[0038] In one embodiment, in a second mode of operation, i.e., direct access mode, the output from a given fixed function unit may be accessed by a shader without having to traverse all other fixed function units of pipeline 300. For example, as shown in FIG. 4B , the output from fixed function unit 416A (stored in buffer 412B) may be directly accessed by shader 418A without traversing fixed function unit 412B and associated nodes and buffers. Furthermore, in the second mode, shader 418A may notify fixed function unit 416A (i.e., using the memory subsystem or a dedicated mailbox) of a given memory location to which fixed function unit 416A can write data associated with the output. In such a scenario, fixed function unit 416A is configured to send an indication to shader 418A regarding the completion of execution of one or more operations scheduled for fixed function unit 416A by shader 418A. Fixed function unit 416A also reports its workload status, any error conditions, and other information to shader 418A.

[0039] A given scheduler may use the second mode of operation, as described above, to control the flow of data through task graph 410B to ensure mitigation of problems associated with overloading of a particular fixed function unit. Further, to communicate with each fixed function unit that the scheduler controls, the scheduler utilizes a messaging protocol with each fixed function unit, which may include messages to initiate execution of work, notifications that work has been executed, and one or more status messages, as described above.

[0040] Referring now to FIG. 5, an exemplary graphics processing pipeline is shown executing in different operational modes. FIG. 5 illustrates a pipeline 500, circuitry 503 associated with direct access mode, and an expanded view of one embodiment of direct access circuitry 503. In one embodiment, a task graph (e.g., task graph 410) may identify a first operation that generates output (and is stored in a buffer, e.g., buffer 412) that can be consumed by a second operation. Based on the task graph, a scheduler (e.g., scheduler 206) may generate one or more commands to schedule the first operation in pipeline 500, commands to store the result of the first operation in a buffer (or memory location), and then commands to execute a second operation that consumes data from the buffer and generates new output. Scheduling of pipeline 500 may be performed in “pipeline mode” 501 or “direct access mode” 503, as described below. Generally speaking, in pipeline mode, data to be processed by any of the fixed function units (502A-502N) enters the pipeline via input 504 at the head (or beginning) of the pipeline 502, and the results of the operation exit via output 506 at the tail (or end) of the pipeline 500. As used herein, a single reference number (e.g., 502) followed by a distinguishing letter (e.g., A, B, C) may be collectively referred to by the numeral alone (e.g., fixed function units 502).

[0041] In one embodiment, graphics pipeline 500 includes one or more fixed function units 502 (or "FFUs" 502), as shown. In one embodiment, in pipeline mode, input operations may be scheduled to pipeline 500 (e.g., by a scheduler using a scheduling graph, as described above) by queuing one or more operations in input queue 504 or other locations. Further, in pipeline mode, output of the operations is generated at the end of the pipeline and stored in output buffer 506. While operating in pipeline mode 501, one or more of FFUs 502 may operate on data. In various scenarios, not all FFUs operate on the received data. Rather, a signal or other indication may be provided that indicates whether a given FFU 502 operates on the received data. If a given FFU 502 does not operate on the received data, the FFU 502 may simply pass the received data to another FFU 502 (e.g., the next FFU 502 in pipeline 500). Nevertheless, when operating in pipeline mode, data traverses the pipeline 500 before the output of a given operation is accessible.

[0042] As an example, when operating in pipeline mode, the minimum latency for execution of an operation includes at least the cumulative number of clock cycles for moving received data through the pipeline (e.g., from input buffer 504 and between each FFU 502), processing the data by the FFU 502 designated to process the data, and moving the processed data to output queue 506 of pipeline 500. Furthermore, in various embodiments, while operating in pipeline mode 501, the status of a given operation may not be known to the scheduler (or other control circuitry), making it difficult to see when dependent operations may be scheduled or otherwise scheduled.

[0043] As an alternative to pipeline mode 501, various embodiments may utilize direct access mode 503. In direct access mode 503, additional circuitry is implemented that allows direct access to one or more of FFUs 502. The number of FFUs 502 to provide direct access to is a matter of design choice. FIG. 5 illustrates one such example of additional circuitry 503. In the illustrated example, additional direct access circuitry 503 is shown, including a control unit 508 and storage for input data to be processed (In) and output data (Out) (e.g., input buffers and output buffers) for storing the processed data. Utilizing this additional circuitry, scheduler 206 can schedule operations directly to selected FFUs 502 without having to traverse other FFUs 502 in pipeline 500. 5, control unit 508 may receive data from scheduler 206 for scheduling one or more operations, receive the data directly from a cache or memory associated with scheduler 206, store the data in an input queue (In), and select an operation from the one or more operations to be processed. Once the operation is selected, control unit 508 provides the received data directly to FFU-3 502C without having to first traverse FFUs 502A and 502B. That is, each time a new operation is scheduled for FFU 502, the operation may be scheduled directly to FFU 502, regardless of the execution of other FFUs 502 in pipeline 500.

[0044] Furthermore, for each scheduled operation for FFU 502C, data associated with the output of that operation may be accessed directly by control unit 508 without waiting for the entire pipeline 500 to finish executing. In the example shown, control unit 508 may access data associated with the output of the operation scheduled for FFU 502C directly from the unit (e.g., from output buffer Out) and store the data in its own output queue. This data may then be accessed from the output queue of control unit 508 by scheduler 206.

[0045] In one or more embodiments, the control unit 508 may further monitor one or more conditions associated with the operation of a given FFU 502. These conditions may include detecting the presence and / or amount of data received in an input buffer (In), detecting the presence and / or amount of data in an output buffer (Out), the status of the corresponding FFU (e.g., whether the FFU is currently busy, whether the FFU will be unavailable for a predetermined period of time), the status of the execution of one or more operations (e.g., not started, running, completed, etc.), overload or under-utilization of the FFU 502, an error state of the FFU 502, control signals received from the scheduler 206, data associated with an operational mode, etc.

[0046] As shown in FIG. 5 (a close-up of direct access circuit 503 in the lower right corner), in some embodiments, control unit 508 may be further connected to multiplexer 510 such that multiplexer 510 can provide input to FFU 502 from a previous FFU in pipeline 500 or directly from the input queue of direct access circuit 503. In various embodiments, control unit 508 provides a signal to multiplexer 510 to indicate which data to select based at least in part on the operating mode (pipeline mode or direct access mode). For example, the multiplexer can provide data from a previous FFU in pipeline mode or from the input queue of control unit 508 in direct access mode. In one embodiment, control unit 508 may determine the operating mode based on metadata (or a signal) received from scheduler 206. For example, control unit 508 can detect metadata associated with data stored in an input buffer (In) or detect a signal generated by scheduler 206.

[0047] In some scenarios, direct access mode 503 allows scheduler 206 to directly access a single FFU. In other scenarios, direct access mode 503 allows scheduler 206 to operate pipeline 500 in a hybrid mode in which both direct access mode and pipeline mode are used. For example, in such a scenario, the scheduler may schedule operations to be processed by two or more FFUs 502, including at least one direct access mode operation. In such a scenario, the scheduler may use direct access mode 503 to directly access a first FFU 502 of the two or more FFUs. When performing direct access, the scheduler may provide metadata (or a signal) indicating which FFU is used in the operation. As an example, scheduler 206 may provide a bit vector including a bit for each FFU 502 in the pipeline. Such a bit vector may then be provided to a given FFU 502 as metadata along with the input data. By setting the corresponding bit, scheduler 206 can indicate which FFU 502 should process the data. In various scenarios, the indicated FFUs 502 may or may not be directly coupled sequentially within pipeline 500. For example, if three FFUs 502 process the data, the indicated FFUs 502 may be 502B, 502C, and 502D (i.e., direct sequence). Alternatively, the indicated FFUs 502 may be 502B, 502D, and 502F (i.e., some FFUs 502 in pipeline 500 do not process). In these scenarios, the first access is a direct access to the first FFU, and the remaining accesses are in pipeline mode, performed by moving the data along the pipeline to the next FFU 502. Note also that direct access mode 503 can be used to retrieve the final data instead of waiting for the processed data to traverse the pipeline to the output buffer 506.For example, in a given scenario, if the scheduler is scheduled to operate with FFU-2 and FFU-3, it may schedule the operation directly to FFU-2, but then exit the pipeline to a direct access mode buffer after FFU-3. These and other embodiments are possible and contemplated.

[0048] 6A , an exemplary operation of a pipeline 600 in pipeline mode is shown. As described above, the minimum latency for execution of an operation in pipeline mode may be determined as the cumulative amount of clock cycles it takes to clock data from the input queue 504 in a first FFU 502A, process the data by the FFU 502A, and clock data associated with the operation's output in the output queue 506 at the end of the pipeline 600. Traditionally, in pipeline mode, data produced as a result of execution of an operation by a given FFU 602 must traverse the entire pipeline 600 before it can be accessed as input to one or more pending scheduled operations, regardless of which FFUs 602 within the pipeline 600 are enabled or disabled (e.g., bypassed or otherwise pass the data to another FFU without processing the data). In one embodiment, a scheduler may schedule operations of a pipeline in pipeline mode using one or more control signals that include, at least in part, data associated with the operation and metadata indicating which FFUs 602 need to process the data. For example, each FFU 602 that does not need to process data may simply forward the data to the next FFU 602 in the pipeline 600. However, the data is still required to traverse the entire pipeline 600 before it can be accessed in the output queue by the scheduler.

[0049] In one example, the minimum latency of pipeline 600 is assumed to be equal to N clock cycles, where N is a positive integer. While reference to clock cycles is used herein for purposes of explanation, it should be noted that other units of measurement are possible and contemplated. In the example shown in the figure, the minimum latency of the pipeline is 100 clock cycles. It should be noted that other values ​​of clock cycles, e.g., non-integer values, are contemplated and are within the scope of the present disclosure. In an exemplary embodiment, an operation performed by any FFU 602 in pipeline 600 may consume a predetermined number of data clock cycles, which may be fixed or variable. In the example shown in Table 1, the fixed number of clock cycles for FFU 602A may be 50 clock cycles, the number of clock cycles for FFU 602B may be 40 clock cycles, and the number of clock cycles to process data by FFU 602C may be 55 clock cycles. In the following example, assume there are three operations scheduled by a scheduler for execution by at least one FFU 602 of pipeline 600. When pipeline 600 operates in pipeline mode, the first operation scheduled for FFU 602A at N=0 may produce an output that may be accessible at the end of execution of the complete pipeline 600, i.e., with a minimum latency of at least N=100. That is, pipeline 600 is fully traversed before any output from a given FFU 602 is accessible. This involves scheduling the first operation in FFU 602A at N=0, processing the first operation by FFU 602A (e.g., using 50 clock cycles), and finally accessing the output at the end of execution of pipeline 600 at N=100.

[0050] In one example, the output of a first operation performed by FFU 602A is used as input to a pending scheduled operation corresponding to FFU 602B. In such a situation, while pipeline 600 is operating in pipeline mode, the earliest that scheduling of a second operation for FFU 602B can occur is at N=100. As further shown in Table 1, the second operation is scheduled at N=100. For simplicity, the time that may be required to move data output from pipeline 600 to a memory location before being used by the second operation is ignored. Furthermore, because FFU 602B uses 40 clock cycles to process the data and must traverse the entire pipeline 600 before accessing the output of the second operation, the output of the second operation is only available at N=200. This is because the output of the first operation retrieved at N=100 is fed back into the input queue of FFU 602A, and time is consumed to move the data through or past FFU 602A, and the data is then processed by FFU 602B, which takes 40 cycles. This example assumes a minimum latency of 100 cycles, so the earliest access to the output of this second operation is N=200 cycles. As shown in Table 1, the second operation is scheduled at N=100, and the output is accessed only at N=200. Continuing with this example, a third operation scheduled later consumes at least 100 cycles. Therefore, the shortest possible latency for this scenario is 300 clock cycles.

[0051] In summary, when executed in pipeline mode, the pipeline 600 must execute completely (i.e., sequentially traverse data through all FFUs 502) before any outputs can be accessed. In some scenarios, a particular sequence of operations may be scheduled in the pipeline such that data output from one FFU is used as input to another FFU. However, when scheduling operations that have dependencies at different times, as described above, the scheduler must wait for the pipeline 600 to complete its execution, access the data, and use it as input to schedule another operation. Furthermore, if a given FFU 602 is not the first FFU 602 in the pipeline 600, the scheduler must wait for all preceding FFUs 602 to process data before an operation can be scheduled for the given FFU 602, thereby resulting in increased latency in the execution of the operations.

[0052] 6B, an exemplary operation of pipeline 600 in direct access mode is shown. Using the same example described above, pipeline 600 has a minimum latency of 100 clock cycles for a full traversal, with FFU 602A having 50 clock cycles of processing time to process data, FFU 602B having 40 clock cycles of processing time, and FFU 602C having 55 cycles of processing time. In direct access mode, the first operation is scheduled for FFU 602A with N=0, which takes 50 clock cycles to process data and produce output. This is in contrast to pipeline mode, where output has the earliest availability of N=100 cycles.

[0053] The control unit (not shown in FIG. 6B) can store data associated with the output of the first operation in its output buffer, where it can be accessed by the scheduler. In the example shown, this output data can be used to schedule a second operation for FFU 602B. In this case, the scheduler schedules the second operation for FFU 602B at N=50. As noted above, for ease of explanation, the time required to move data to and from memory is ignored. As shown in Table 2, the first operation is scheduled at N=0, the output of the first operation is accessed at N=50, and the second operation is scheduled for FFU 602B (using data from the output of the first operation) at N=50. Similarly, to schedule a third operation for FFU 602C using the output from the second operation, the scheduler can directly access the data associated with the output of the second operation at N=90 and schedule the third operation at the same time, producing the final output at time N=145, as opposed to N=300 in FIG. 6A.

[0054] Such a system that directly accesses individual FFUs 602 may therefore reduce the overall latency of the pipeline 600. Furthermore, the control unit may continuously monitor the status of each FFU 602 during each execution cycle. Furthermore, using direct access mode, the scheduler may not need to wait for an indication of execution completion before beginning the next execution.

[0055] 6A and 6B show execution of FFUs 502 in a sequential manner, in one or more embodiments, two or more FFUs 502 may be scheduled by the scheduler simultaneously (i.e., for concurrent execution) in direct access mode if there is no dependency between the scheduled operations. For example, at the beginning of execution, i.e., at N=0, the scheduler may schedule both a first operation to FFU 602A and a second operation to FFU 602B such that execution of both operations is complete by N=50. This may further reduce the overall latency of pipeline 600.

[0056] 7, a block diagram illustrating an example data flow of a graphics processing pipeline implementing shaders is shown. In one embodiment, the pipeline may include fixed function units, such as a NIC 702, a video decoder 706, a machine learning unit (or “ML unit”) 708, and a video encoder unit 710. Each of the fixed function units is controlled by a shader 704, such as shader 704A connected to the NIC 702 and the video decoder 706, shader 704B connected to the ML unit 708 and the video encoder 710, and shader 704C connected to the video decoder 706 and the video encoder 710.

[0057] In one embodiment, the NIC 702 can generate one or more work items at a fixed rate. For example, network packets containing data to be processed can be received over a network and queued by the NIC 702 for execution. The NIC 702 can notify a scheduler (not shown) that a work item is ready to execute, for example, as a push notification. In response, the scheduler accesses the incoming network packets, and once a predetermined number of packets have accumulated, the scheduler can notify the shader 704A to begin processing. In one embodiment, the shader 704A generates output based on the notification from the scheduler. For example, the shader 704A can generate a second number of predetermined packets. The shader 704A can then notify the scheduler that the output has been generated.

[0058] The generated output from shader 704A may be input to video decoder 706. In one embodiment, the scheduler may monitor the amount of work items input to each fixed function block, and if there is an overload on a given fixed function block, the next execution is not launched (as opposed to a pipeline mode, where insight into the status of the fixed function units may not be available). The video decoder 706 may generate output based on the input it receives, and notification that the output has been generated and saved to a particular memory location may be received by the scheduler from the video decoder 706. Based on the notification, the scheduler may read the particular memory location and store the generated output in a cache (not shown). Based on the output, the video decoder 706 may forward some of the work items to the ML unit 708 for execution, and the remaining work items may be forwarded by the scheduler to shader 704C.

[0059] In one embodiment, based on the portion of the work items forwarded to the ML unit 708, the ML unit can notify a scheduler, which in turn notifies shader 704B when to begin execution. Similarly, the remaining work items are scheduled by the scheduler to the video encoder 710 by notifying shader 704C. The video encoder 710 executes the received work items, generates the final output (e.g., an image or video presented on a user device), and notifies the scheduler that execution is complete. The scheduler can notify the video encoder 710 of a memory location to send the final output to or, in certain circumstances, discard the output. Furthermore, once execution is completed or the execution results are discarded, the scheduler can recycle each memory location accessed during execution.

[0060] In one embodiment, a fixed function unit (e.g., video decoder 706) may receive one or more work items from a given processor unit (not shown) on behalf of a given scheduler, thereby enabling the processor unit to drive the fixed function unit. For example, when the processor unit drives the fixed function unit, a given scheduler may only be able to throttle or configure one or more work items. In the context of video decoder 706, for example, if the work items are related to generating video, the scheduler may be able to perform resolution changes to adjust the data rate. However, the initiation of execution of the work items may still be controlled by an external processor unit.

[0061] 8, one embodiment of a method 800 for controlling one or more fixed function units in a graphics pipeline is shown. As described above, a control unit (e.g., control unit 508) can select operations to be performed by the graphics pipeline (block 802). The graphics pipeline, in one embodiment, may comprise one or more fixed function units such as a geometry shader, an input assembler, a mosaicizer, a rasterizer, a video encoder, a video decoder, etc. For example, if the fixed function unit is a geometry shader, the operation may be any of point sprite expansion, dynamic particle system operations, farfin generation, shadow volume generation, single pass render to cube map, per-primitive material swapping, per-primitive material setting, etc.

[0062] In one embodiment, the control unit can select an operation to be performed based at least in part on one or more control signals received from the scheduler. In one example, the control signal can include data associated with the operation and metadata indicating which fixed function unit is required to process the data (i.e., whether pipeline mode or direct access mode is used). The control unit can store the data and associated metadata in its input queue and, based on the operating mode, write the data and metadata to an input buffer of the fixed function unit. Once an operation is selected, the control unit can determine whether the fixed function unit is to operate in a first operating mode (pipeline mode) or a second operating mode (direct access mode) (conditional block 804).

[0063] In a first operating mode (conditional block 804: "first operating mode"), the scheduler may notify a first fixed function unit from the pipeline about the selected operation (block 806). In one embodiment, in the first operating mode, the control unit determines, based on data received from the scheduler, that the selected operation should be executed by a first fixed function unit of a sequence of fixed function units included in the pipeline. Data related to the execution of the selected operation is written to an input queue of the pipeline for execution by the first fixed function unit (block 808).

[0064] The selected operation may then be executed by the first fixed function unit, and data associated with the output of the execution may traverse to each subsequent fixed function unit in the pipeline. That is, in the first mode of operation, the data associated with the output of the selected operation is accessible only after the entire pipeline has completed execution. The control unit may read the data associated with the output from an output queue associated with the pipeline (block 810). In one embodiment, this data from the pipeline's output queue may be read by the control unit and stored in its own output queue, from which it may be accessed by a scheduler.

[0065] However, in a second mode of operation (conditional block 804: "Second Mode of Operation"), the control unit may notify a selected fixed function unit in the sequence of units included in the pipeline (block 812). That is, in the second mode of operation, the control unit may select a given fixed function unit (based on data received from the scheduler) to perform a given operation, regardless of where that fixed function unit is located in the sequence of units in the pipeline. The selection of the fixed function unit, in one embodiment, may be based on the application using the pipeline.

[0066] Once the selected fixed function unit is notified, the control unit may write data for execution of the operation to an input buffer of the selected fixed function unit (block 814). The control unit may then determine whether another operation is due for scheduling (conditional block 816). In one embodiment, one or more other operations may be scheduled by the scheduler such that each operation may be scheduled to a different fixed function unit (sequential or non-sequential) in the pipeline.

[0067] If it is determined that no other operations should be scheduled (conditional block 816: "NO"), the control unit may identify a memory location to which data associated with the output of the execution needs to be sent (block 818). Based on such identification, the control unit may read data from the output buffer of the selected fixed function unit and send the data to the identified memory location once the selected fixed function unit notifies the control unit that the data is ready for consumption. The control unit may then monitor the status of the fixed function unit and send this information to the scheduler (block 820).

[0068] However, if the control unit determines, based on instructions received from the scheduler, that another operation should be scheduled (conditional block 816: "YES"), the control unit may select another fixed function unit and send data resulting from the execution of the operation to the input buffer of the other selected fixed function unit (block 822). The control may then again monitor the status of the fixed function unit (block 820). In one embodiment, the scheduler may access the workload status, error conditions, data flow, etc. of the fixed function unit via the control unit and make scheduling decisions based on this accessed information.

[0069] It should be emphasized that the above-described embodiments are merely non-limiting examples of embodiments. Numerous variations and modifications will become apparent to those skilled in the art once the above disclosure is fully appreciated. It is intended that the following claims be interpreted to embrace all such variations and modifications.

Claims

1. 1. A processor, comprising: a pipeline including one or more fixed function units; a scheduler; The scheduler scheduling operations for execution by any of the one or more fixed function units of the pipeline by scheduling each of the operations with a first fixed function unit of the pipeline in accordance with a first mode of operation; scheduling a predetermined operation for execution by a selected fixed function unit of the pipeline by directly scheduling the predetermined operation to a selected fixed function unit other than the first fixed function unit in response to a second mode of operation; configured to: Processor.

2. the scheduler is configured to generate a scheduling graph comprising one or more nodes and to schedule each of the one or more nodes of the scheduling graph for execution by one or more of the fixed function units. The processor of claim 1 .

3. In response to the second mode of operation, data generated by the selected fixed function unit is not communicated to a next fixed function unit in the pipeline. The processor of claim 1 .

4. the scheduler is configured to directly schedule two nodes of the scheduling graph onto two different fixed function units for concurrent execution. The processor of claim 2 .

5. In response to the second mode of operation, the scheduler is configured to read data associated with an output of the second operation directly from an output buffer of the selected fixed function unit. The processor of claim 1 .

6. the scheduler is configured to send the data to a memory location, the memory location being either a memory subsystem associated with the processor or a mailbox accessible by the scheduler; The processor of claim 5.

7. the selected fixed function unit is configured to send a notification to the scheduler, the notification indicating completion of execution of the second operation; The processor of claim 1 .

8. 1. A method comprising: scheduling operations for execution by any of one or more fixed function units of the pipeline by scheduling each of the operations with a first fixed function unit of the pipeline in accordance with a first mode of operation; and scheduling, in response to a second mode of operation, the predetermined operation for execution by a selected fixed function unit of the pipeline by directly scheduling the predetermined operation to a selected fixed function unit other than the first fixed function unit. method.

9. a scheduler generating a scheduling graph including one or more nodes; the scheduler scheduling each of the one or more nodes of the scheduling graph for execution by one or more of the fixed function units.

9. The method of claim 8.

10. In response to the second mode of operation, data generated by the selected fixed function unit is not communicated to a next fixed function unit in the pipeline.

10. The method of claim 9.

11. the scheduler directly scheduling two nodes of the scheduling graph onto two different fixed function units for concurrent execution; 10. The method of claim 9.

12. In response to the second mode of operation, the scheduler reads data associated with an output of the second operation directly from an output buffer of the selected fixed function unit.

10. The method of claim 9.

13. the scheduler transmitting the data to a memory location; the memory location is any of the memory subsystems associated with mailboxes accessible by the scheduler; 13. The method of claim 12.

14. the scheduler receiving a notification from the selected fixed function unit; the notification indicating completion of execution of the second operation.

10. The method of claim 9.

15. 1. A computing system comprising: a central processing unit; a graphics processing unit, The graphics processing unit: a pipeline including one or more fixed function units; a scheduler; The scheduler scheduling operations for execution by any of the one or more fixed function units of the pipeline by scheduling each of the operations with a first fixed function unit of the pipeline in accordance with a first mode of operation; scheduling a predetermined operation for execution by a selected fixed function unit of the pipeline by directly scheduling the predetermined operation to a selected fixed function unit other than the first fixed function unit in response to a second mode of operation; configured to: Computing system.

16. the scheduler is configured to generate a scheduling graph comprising one or more nodes and to schedule each of the one or more nodes of the scheduling graph for execution by one or more of the fixed function units.

16. The computing system of claim 15.

17. In response to the second mode of operation, data generated by the selected fixed function unit is not communicated to a next fixed function unit in the pipeline.

16. The computing system of claim 15.

18. the scheduler is configured to directly schedule two nodes of a scheduling graph onto two different fixed function units for concurrent execution; 16. The computing system of claim 15.

19. In response to the second mode of operation, the scheduler is configured to read data associated with an output of the second operation directly from an output buffer of the selected fixed function unit.

16. The computing system of claim 15.

20. the scheduler is configured to transmit the data to a memory location, the memory location being any of a memory subsystem associated with a mailbox accessible by the scheduler; 20. The computing system of claim 19.

Images