Hardware queue priority mechanism

Abstract

A processing system includes scheduling circuitry that sends elements to one or more processing circuitry, such as shader circuitry, for execution. The scheduling circuitry includes a scheduling queue and arbitration circuitry. The scheduling queue stores elements to be sent to one or more processing circuitry. Arbitration circuitry schedules elements of the scheduling queue for execution based on priority indicators corresponding to the elements. Thus, prioritization of the elements is implemented at scheduling circuitry in hardware without changing the design of the scheduling queue to store priority information.

Description

Background Technology

[0001] Parallel processors, such as accelerator processors and graphics processing units (GPUs), implement graphics processing pipelines that concurrently process commands retrieved from a scheduling queue of command buffers. In some cases, parallel processors comprise multiple processing elements (also known as processor cores, compute units, or workgroup processors) that execute different programs, or multiple instances of a single program that execute concurrently on multiple datasets as a single “wave” (i.e., a group of threads running concurrently on the GPU). The graphics pipeline in a GPU includes one or more shader engines that execute using the pipeline’s resources, such as compute units, memory, and cache. Shader engines have traditionally been used for graphics lighting computations. However, in modern computing, shader engines are often used as general-purpose processors that perform work independently of the graphics processing pipeline. Summary of the Invention

[0002] A system of one or more processing units may be configured to perform specific operations or actions by installing software, firmware, hardware, or combinations thereof on the system, which in operation cause the system to perform actions. One or more computer programs may be configured to perform specific operations or actions by including instructions that, when executed by a data processing device, cause that device to perform actions. A general aspect includes: the system further including: a scheduling queue configured to store elements to be executed by a shader circuitry system; and an arbitration circuit configured to store a plurality of priority indicators corresponding to respective elements in the scheduling queue, wherein the arbitration circuit is configured to schedule these elements in the scheduling queue based on the plurality of priority indicators for execution at the shader circuitry system. Other embodiments of this aspect include a corresponding computer system, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of these methods.

[0003] Specific implementations may include one or more of the following features. Scheduling elements of a scheduling queue for execution in the system may include: identifying the next element to be sent to the shader circuitry system in response to receiving a specific priority corresponding to a particular element. Before receiving a corresponding request for the next element to be sent to the shader circuitry system, the execution identifies the next element to be sent to the shader circuitry system. The scheduling queue may include a plurality of slots configured to store corresponding elements, and wherein a plurality of priority indicators correspond to corresponding slots among the plurality of slots. Scheduling elements of the scheduling queue for execution may include scheduling at least one element with a second priority in response to a scheduling threshold number of elements with a first priority, wherein the second priority is a priority lower than the first priority. The plurality of priority indicators are identifiers of corresponding procedures corresponding to the elements. The plurality of priority indicators are identifiers of corresponding virtual machines corresponding to the elements. The arbitration circuit is also configured to store a second plurality of priority indicators, and wherein scheduling elements of the scheduling queue for execution may include selecting between elements indicated by the plurality of priority indicators and elements indicated by the second plurality of priority indicators based on a priority selection signal. The priority selection signal indicates the availability of hardware circuitry. The hardware circuitry is memory circuitry or a shader circuitry system. Specific implementations of the described technology may include hardware, methods or processes, or computer software on a computer-accessible medium.

[0004] One general aspect includes a method comprising: receiving at an arbitration circuit an indication of priority for an element to be executed by a shader circuitry system, wherein the element is stored at a scheduling queue; ordering the priorities stored at the arbitration circuitry based on the indication of priority; and, in response to a request for an element, providing to the shader circuitry system the next element indicated by the ordered priorities. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of these methods.

[0005] Specific implementations may include one or more of the following features. In the method, a priority indicator element has a first priority, and a second element currently stored in the scheduling queue has a second priority. Receiving a priority indicator may include extracting the priority from a received opcode corresponding to the element. The priority indicator for an element is specified by a user. The priority indicator for an element is specified by a driver based on the state of a user-visible process to be used using data generated from the element. Specific implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.

[0006] One general aspect includes a processing system. The processing system further includes: a bus; a first processing circuit configured to issue a plurality of commands via the bus; and a second processing circuit configured to receive the plurality of commands from the first processing circuit, the second processing circuit including: a scheduling queue configured to store a plurality of elements from the plurality of commands; a shader circuit system configured to process the plurality of elements; and an arbitration circuit configured to store a plurality of priority indicators corresponding to corresponding elements among the plurality of elements, and to schedule the corresponding elements for processing by the shader circuit system based on the plurality of priority indicators. Other embodiments of this aspect include a corresponding computer system, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of these methods.

[0007] Specific implementations may include one or more of the following features. In a processing system, scheduling elements of a scheduling queue for processing may include sending a previously identified element to a shader circuit system in response to receiving a request for an element. An arbitration circuit is also configured to prefetch previously identified elements. Scheduling elements of a scheduling queue for processing may include identifying the next element to be sent to the shader circuit system in response to receiving a request for an element. Scheduling elements of a scheduling queue for processing may include scheduling at least one element with a second priority in response to a scheduling threshold number of elements with a first priority, wherein the second priority is a priority lower than the first priority. Specific implementations of the described techniques may include hardware, methods or processes, or computer software on a computer-accessible medium. Attached Figure Description

[0008] This disclosure is better understood by referring to the accompanying drawings, and many of its features and advantages will be apparent to those skilled in the art. The same reference numerals are used in different drawings to denote similar or identical items.

[0009] Figure 1 is a block diagram of a processing system that includes a hardware queue priority mechanism according to some specific implementations.

[0010] Figure 2 is a block diagram of an example processing circuit system that implements a hardware queue priority mechanism according to some specific implementations.

[0011] Figure 3 is a block diagram illustrating an example scheduling circuit system based on some specific implementations of a hardware queue priority mechanism.

[0012] Figure 4 is a flowchart of a method for prioritizing elements in a hardware queue according to some specific implementations. Detailed Implementation

[0013] In some implementations, parallel processors, such as accelerated processing devices or GPUs, include shader circuitry systems, such as one or more shader engines, each comprising a corresponding number of computational units. Based on one or more commands to be executed, a command processor generates multiple workgroups (collections of processing threads) to be assigned to the shader circuitry systems for processing. Assigning workgroups to the shader circuitry systems involves sending elements from the commands to a scheduling circuitry system. The scheduling circuitry system schedules the elements for processing at the shader circuitry system. In some processing systems, the scheduling circuitry system includes a scheduling queue that outputs received elements in a fixed manner based on insertion order. For example, in some cases, the scheduling queue stores elements in a first-in, first-out (FIFO) or last-in, first-out (LIFO) (stack) manner. However, in some cases (e.g., quality of service or work graph processes), it is desirable to execute certain elements with priority over others. However, because the scheduling queue is implemented in a fixed manner, execution priority is difficult to achieve.

[0014] In some processing systems that include scheduling queues, priority is implemented using multiple hardware queues, where some hardware queues are prioritized over others to schedule instructions in a fixed manner. However, such systems inherently have a fixed number of priority levels corresponding to the number of hardware queues, and inefficiently use power and area for queue entries where elements are not frequently stored. In other processing systems that include scheduling queues, software implemented using synchronization primitives (e.g., locks) is used to prioritize certain elements. However, in some cases, the software introduces an undesirable delay between the time an element is received and the time the element to be scheduled is identified.

[0015] Using the techniques disclosed herein, in some implementations, the scheduling circuitry includes an arbitration circuit that stores priority indicators corresponding to the respective elements of the scheduling queue. The arbitration circuit schedules the elements of the scheduling queue based on these priority indicators for execution at the shader circuitry. Therefore, priority ordering is implemented at the scheduling circuitry in hardware without altering the design of the scheduling queue to store priority information. Furthermore, in some implementations, compared to a scheduling circuitry where each priority level corresponds to a different scheduling queue, the arbitration circuitry allows the scheduling circuitry to utilize more priority levels with less wasted power and area.

[0016] In various specific implementations, the techniques described herein may employ any of a variety of parallel processors, such as vector processors, graphics processing units (GPUs), general-purpose GPUs (GPGPUs), nonscalar processors, highly parallel processors, artificial intelligence (AI) processors, inference engines, machine learning processors, and other multi-threaded processing units. For ease of illustration, reference is made to example systems and methods employing processing circuitry therein. However, it should be understood that, unless otherwise indicated, the systems and techniques described herein are equally applicable to the use of other types of parallel processors.

[0017] Referring now to Figure 1, a processing system 100 including a hardware queue priority mechanism is presented according to some embodiments. The processing system 100 includes or has access to memory 110 or other storage components implemented using non-transitory computer-readable media such as dynamic random access memory (DRAM). However, in some specific embodiments, memory 110 is implemented using other types of memory, including, for example, static random access memory (SRAM), non-volatile RAM, etc. According to some specific embodiments, memory 110 includes external memory implemented outside the processing units implemented in the processing system 100. The processing system 100 also includes a bus 120 to support communication between entities implemented in the processing system 100, such as memory 110. Some specific embodiments of the processing system 100 include other buses, bridges, switches, routers, etc., which are not shown in Figure 1 for clarity.

[0018] In various implementations, the techniques described herein are employed at a graphics processing unit (GPU) 130. GPU 130 includes, for example, a vector processor, a coprocessor, a graphics processing unit (GPU), a general-purpose GPU (GPGPU), a nonscalar processor, a highly parallel processor, an artificial intelligence (AI) processor, an inference engine, a machine learning processor, other multi-threaded processing units, a scalar processor, a serial processor, a programmable logic device (simple programmable logic device, complex programmable logic device, field-programmable gate array (FPGA)), or any combination thereof. Figure 1 illustrates examples of parallel processors, particularly GPU 130, according to some implementations. In some implementations, GPU 130 renders images for presentation on display 142. For example, GPU 130 renders objects to produce pixel values ​​provided to display 142, which uses these pixel values ​​to display an image representing the rendered objects. In some implementations, instead of sending values ​​to display 142 via input / output (I / O) engine 140 as illustrated in Figure 1, GPU 130 sends values ​​directly to display 142.

[0019] GPU 130 includes a shader circuitry system 132 that implements multiple execution units (e.g., shader engines), each capable of executing one or more instructions individually or in parallel. In some embodiments, the shader circuitry system 132 includes execution units implemented as one or more single-instruction multiple-data (SIMD) units, compute units, processor cores, etc. For example, in some embodiments, the shader circuitry system 132 includes one or more execution units to perform operations on one or more instructions received from the graphics pipeline in parallel. To facilitate the compute units' execution of operations, GPU 130 includes one or more command processors (not shown for clarity). Such command processors include, for example, circuitry configured to execute one or more instructions from the graphics pipeline by providing one or more compute units with data indicating one or more operations, operands, instructions, variables, register files, or any combination thereof, which is necessary, helpful, or auxiliary for the execution of one or more operations of the instructions. In the illustrated specific implementation, the command processor includes a scheduling circuitry 134 for providing execution units of the shader circuitry 132 with data values, referred to herein as "elements." As discussed further below with reference to Figures 2 through 4, in various specific implementations, the scheduling circuitry 134 manages the priorities associated with elements, such that some elements are processed faster than others. Some specific implementations of the GPU 130 are used for general-purpose computing. For example, in one implementation, the GPU 130 receives one or more instructions, such as program code 112, from one or more applications 114, which indicate operations associated with, for example, one or more video tasks, physics simulation tasks, computational tasks, hydrodynamic tasks, or any combination thereof. In response to receiving program code 112, the GPU 130 executes the instructions for the video task, physics simulation task, computational task, and hydrodynamic task. The GPU 130 then stores information in memory 110, such as the results of the executed instructions.

[0020] In some implementations, the processing system 100 includes an input / output (I / O) engine 140, which includes circuitry for processing input or output operations associated with the display 142 and other components of the processing system 100, such as a keyboard, mouse, printer, external disk, etc. The I / O engine 140 is coupled to a bus 120, enabling it to communicate with the memory 110, the GPU 130, the central processing unit (CPU) 102, or any combination thereof.

[0021] In some embodiments, the processing system 100 also includes a CPU 102 that communicates with the GPU 130 and memory 110 via a bus 120. The CPU 102 implements multiple processor cores 104, 105, and 106 that execute instructions concurrently or in parallel. In various embodiments, one or more processor cores in processor core 104 operate as SIMD units performing the same operation on different datasets. Although three processors are presented in the example embodiment illustrated in Figure 1, the number of processor cores implemented in the CPU 102 is a matter of design choice. Therefore, in other embodiments, the CPU 102 may include any number of processor cores. In some embodiments, the CPU 102 has the same number of processor cores as the GPU 130 has in its shader circuitry 132. In other embodiments, the number of processor cores in the CPU 102 differs from the number of execution units in the GPU 130's shader circuitry 132. The processor cores 104-106 of CPU 102 execute instructions, such as program code 112, of one or more applications 114 (e.g., graphics applications, computing applications, machine learning applications) stored in memory 110, and CPU 102 stores information in memory 110, such as the results of the executed instructions. In some embodiments, CPU 102 initiates processing at GPU 130, for example, by issuing a draw call to GPU 130. As discussed further below with reference to Figures 2 and 3, in some cases, the instructions sent from CPU 102 to GPU 130 include elements and priority indicators that are used to prioritize processing that utilizes the elements. In some embodiments, CPU 102 runs user software or includes an application programming interface (API) that allows a user to specify the priority of elements. In some embodiments, CPU 102 runs a driver that specifies the priority of elements based on the state of a user-visible process that uses data generated from the elements.

[0022] Figure 2 is a block diagram illustrating a processing circuitry system 200 including an example hardware queue priority mechanism. In some embodiments, the processing circuitry system 200 depicts a subset of the processing system 100 of Figure 1. For example, in the illustrated embodiment, the processing circuitry system 200 includes the shader circuitry system 132 and the scheduling circuitry system 136 of Figure 1. However, in other embodiments, the processing circuitry system 200 corresponds to different systems including more or fewer components than the processing system 100. In the illustrated embodiment, the scheduling circuitry system 136 includes an arbitration circuit 202 and a scheduling queue 206. The arbitration circuit 202 includes a bitmask 204. Although the illustrated embodiment shows a specific configuration of components, other combinations or arrangements of components are contemplated in various embodiments. For example, in some embodiments, the processing circuitry system 200 includes a memory circuitry system from which elements stored at the scheduling queue 206 are retrieved. Also, in some embodiments, the shader circuitry system 132 is separate from the processing circuitry system 200.

[0023] As discussed above with reference to Figure 1, shader circuitry 132 includes one or more execution units, each capable of executing one or more instructions individually or in parallel. Instruction execution is performed using elements, which are received at shader circuitry 132 via element responses 222. Therefore, shader circuitry 132 uses elements received from scheduling circuitry 136 to execute instructions.

[0024] Scheduling circuitry 136 organizes and schedules elements for execution by shader circuitry 132. Elements such as element 212 are received at the scheduling circuitry (e.g., instructions received from another circuitry such as kernels 104-106 or from the memory circuitry of GPU 130). Elements are stored at scheduling queue 206, which includes multiple slots configured to store elements. In response to element request 220, scheduling circuitry 136 retrieves an element from scheduling queue 206 and sends an element response 222 to shader circuitry 132, thereby indicating the retrieved element as the next element to be processed. In some implementations, element request 220 is received directly from shader circuitry 132. In other implementations, element request 220 is received from another circuitry (such as part of the command processor of GPU 130). In other implementations, element request 220 is not received at all, and instead, scheduling circuitry 136 periodically sends elements to shader circuitry 132 (e.g., every clock cycle).

[0025] As discussed above, in some cases, it is desirable to prioritize certain elements over others. However, scheduling queue 206 itself is not configured to dequeue high-priority elements first. Arbitration circuit 202 is used to implement priority at scheduling queue 206. Specifically, as further discussed below with reference to FIG3, arbitration circuit 202 stores multiple priority indicators corresponding to the respective elements of scheduling queue 206 at bitmask 204. Arbitration circuit 202 schedules elements for execution at shader circuitry 132 based on the stored priority indicators. Priority indicators such as priority indicator 210 are received at the scheduling circuit (e.g., instructions received from another circuit such as kernels 104-106 or from the memory circuitry of GPU 130). In some implementations, the priority indicator is received along with the corresponding element. In other implementations, the priority indicator is received independently of the corresponding element. In some implementations, the priority indicator is extracted from the received opcode corresponding to the element. In some implementations, the priority indicator is associated with the process of generating the element. For example, in some cases, the priority indicator is the identifier of the corresponding procedure for an element. Similarly, in some cases, the priority indicator is the identifier of the corresponding virtual machine for an element.

[0026] Therefore, a priority mechanism is implemented for the elements stored in the scheduling queue 206, thereby allowing for more flexible dequeueing and supporting high-priority elements.

[0027] Figure 3 is a block diagram illustrating a scheduling circuit system 300 that implements an example hardware queue priority mechanism. In some implementations, the scheduling circuit system 300 corresponds to the scheduling circuit system 136 of Figures 1 and 2. For example, in the illustrated implementation, the scheduling circuit system 300 includes the arbitration circuit 202, bitmask 204, and scheduling queue 206 of Figure 2. However, in other implementations, the scheduling circuit system 300 corresponds to different systems that include more or fewer components than the scheduling circuit system 136. In the illustrated implementation, the bitmask 204 includes a next element identifier 302, a priority 1 list 304, a priority 2 list 305, and a priority 3 list 306. The scheduling queue 206 includes slots 310, 311, 312, 313, 314, 315, 316, and 317. Although the illustrated implementation shows a specific configuration of components, other combinations or arrangements of components are contemplated in various implementations. For example, in some implementations, scheduling queue 206 may include more than eight slots or fewer than eight slots. Also, in some implementations, bitmask 204 may include only two priority lists or may not include the next element identifier 302.

[0028] At scheduling circuit system 300, an element and its corresponding priority indicator are received. For example, element 212 and priority indicator 210 are received together as part of an opcode and its corresponding operand. Scheduling queue 206 stores element 212 in one of slots 310-317 according to the configuration of scheduling queue 206 (e.g., stacked configuration or FIFO configuration). In response to receiving a slot request (e.g., slot request 320) from arbitration circuit 202, scheduling queue 206 provides the element stored in the slot identified by the request in a slot response (e.g., slot response 322).

[0029] Arbitration circuit 202 identifies the priority of element 212, identified by priority indicator 210, and sorts one or more priorities at arbitration circuit 202 based on the received priorities. For example, in response to priority indicator 210 indicating that element 212 has a first priority level, arbitration circuit 202 identifies element 212 in priority list 304. In some cases, identifying element 212 in priority list 304 causes arbitration circuit 202 to prioritize element 212 over other elements received before element 212 at scheduling queue 206 (e.g., elements identified by priority list 305). Thus, in some cases, in response to element request 220, arbitration circuit 202 identifies element 212 instead of a previously stored element, even if scheduling queue 206 is configured as a FIFO queue. In response to identifying the element, arbitration circuit 202 requests the identified element from scheduling queue 206 via slot request 320 and sends the element received from slot response 322 as part of element response 222. In some implementations, the priority of arbitration circuit 202 corresponds to the corresponding slot in scheduling queue 206. In some implementations, different priority lists correspond to different priority levels. For example, an element identified by priority list 305 has a lower priority level than an element identified by priority list 304. In other implementations, as discussed further below, different priority lists correspond to other factors, such as the presence of priority selection signal 330.

[0030] In some implementations, the arbitration circuit 202 includes additional priority ordering features. For example, in the illustrated implementation, the arbitration circuit 202 includes a next element identifier 302, which indicates the next element to be scheduled. In response to receiving a priority indicator (e.g., priority indicator 210), the element identified by the next element identifier 302 is identified. Therefore, instead of searching one or more priority lists in response to element request 220, the arbitration circuit 202 references the next element identifier 302 in the element response 222, thus sending the element response 222 faster than implementations where the next element identifier 302 is not stored at bitmask 204. In some implementations, the element identified by the next element identifier 302 is prefetched, further reducing the response time to element requests. In other implementations, the next element identifier 302 is not stored at bitmask 204, thus saving power and area compared to systems that store the next element identifier.

[0031] For example, in some implementations, the arbitration circuit 202 includes priority inversion protection. More specifically, as a means of ensuring that an element will eventually be scheduled, priority list 305 has a threshold number of schedulings, after which the element identified by priority list 305 is scheduled. Similarly, priority list 306 has a second threshold number of schedulings, after which the element identified by priority list 306 is scheduled.

[0032] For example, in some implementations, some priority lists depend on additional signals, such as priority selection signal 330. To illustrate, in some implementations, processing elements depend on the availability of various types of hardware circuitry (e.g., processing resources), such as shader circuitry system availability or memory circuitry availability. Therefore, in those implementations, without a priority selection signal 330 indicating memory availability, arbitration circuitry 202 uses a first priority list (e.g., priority 1 list 304) to schedule elements. However, in response to the priority selection signal 330 indicating memory availability, arbitration circuitry 202 uses a different priority list (e.g., priority 2 list 305) to schedule elements.

[0033] Figure 4 is a flowchart illustrating a method 400 for prioritizing elements in a hardware queue according to some specific implementations. In some implementations, the various parts are executed in a different order. For example, in some implementations, a determination of whether the processing circuitry is executing a critical or non-critical process is performed before determining whether a memory request addresses critical data. In some implementations, method 400 is initiated by one or more processors in response to one or more instructions stored in a computer-readable storage medium.

[0034] At block 402, an indication of the priority of an element is received. For example, a priority indicator 210 is received at the arbitration circuit 202 in FIG3. At block 404, the priorities at the arbitration circuit are sorted. For example, one or more of the priority lists 304 to 306 are modified to include an indication corresponding to priority indicator 210.

[0035] At box 406, a request for an element is received. For example, arbitration circuit 202 receives element request 220. At box 408, the next element indicated by a sorted priority is provided. For example, arbitration circuit 202 provides element response 222 indicating the next element to be scheduled based on one or more of priority lists 304 to 306. Thus, a method for prioritizing elements in a hardware queue is depicted.

[0036] In some embodiments, computer-readable storage media include any non-transitory storage media or combinations of non-transitory storage media that can be accessed by a computer system during use to provide instructions and / or data to the computer system. Such storage media may include, but are not limited to, optical media (e.g., optical discs (CDs), digital versatile optical discs (DVDs), or Blu-ray discs), magnetic media (e.g., floppy disks, magnetic tapes, or magnetic hard disks), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or microelectromechanical systems (MEMS) based storage media. In some embodiments, computer-readable storage media are embedded in a computing system (e.g., system RAM or ROM), fixedly attached to a computing system (e.g., a magnetic hard disk drive), removably attached to a computing system (e.g., an optical disc or a USB-based flash memory), or coupled to a computer system via a wired or wireless network (e.g., a network-accessible storage device (NAS)).

[0037] In some implementations, certain aspects of the techniques described above are implemented by one or more processors of a processing system executing the software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied on a non-transitory computer-readable storage medium. The software may include instructions and certain data that, when executed by one or more processors, manipulate one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer-readable storage medium may include, for example, disk or optical disc storage devices, solid-state storage devices such as flash memory, cache, random access memory (RAM), or one or more other non-volatile memory devices. In some implementations, the executable instructions stored on the non-transitory computer-readable storage medium are source code, assembly language code, object code, or other instruction formats that are interpreted or otherwise executable by one or more processors.

[0038] It should be noted that not all activities or elements described above in the general description are essential, a particular activity or part of the apparatus is not essential, and in some cases, one or more additional activities or elements are performed in addition to those described. Furthermore, the order in which the activities are listed is not necessarily the order in which they are performed. Additionally, these concepts have been described with reference to specific embodiments. However, those skilled in the art will understand that various modifications and changes can be made without departing from the scope of this disclosure as set forth in the following claims. Therefore, the specification and drawings are to be considered illustrative rather than restrictive, and all such modifications are intended to be included within the scope of this disclosure.

[0039] The benefits, other advantages, and solutions to problems have been described above with respect to specific embodiments. However, the benefits, advantages, solutions to problems, and any features that lead to or make more significant any benefit, advantage, or solution should not be construed as key, essential, or fundamental features of any or all claims. Furthermore, the specific embodiments disclosed above are merely illustrative, as the disclosed subject matter can be modified and practiced in different but equivalent ways that will be apparent to those skilled in the art who have benefited from the teachings herein. No limitation is intended on the details of the constructions or designs shown herein, except as described in the following claims. Therefore, it will be apparent that the specific embodiments disclosed above can be altered or modified, and all such variations are considered to be within the scope of the disclosed subject matter. Therefore, the protection sought herein is as set forth in the following claims.

[0040] One or more of the aforementioned elements are circuits designed and configured to perform the corresponding operations described above. In at least some embodiments, such circuitry is any one or a combination of the following: hard-coded circuitry (e.g., a corresponding portion of an Application-Specific Integrated Circuit (ASIC) or a set of logic gates, memory elements, and other components selected and arranged to perform the operations), programmable circuitry (e.g., a corresponding portion of a Field-Programmable Gate Array (FPGA) or Programmable Logic Device (PLD), or one or more processors executing software instructions that cause one or more processors to perform the actions. In some embodiments, the circuitry of a particular element is selected, arranged, and configured by one or more computer-implemented design tools. For example, in some embodiments, the sequence of operations for a particular element is defined in a specified computer language (such as register-transfer language), and the computer-implemented design tools select, configure, and arrange the circuitry based on the defined sequence of operations. The terms "circuit system" and "circuit" are used interchangeably herein.

[0041] Within this disclosure, in some instances, different entities (which are referred to differently as “components,” “units,” “devices,” “circuit systems,” etc.) are described or claimed to be “configured” to perform one or more tasks or operations. This expression (the [entity] configured to [perform one or more tasks]) is used herein to refer to a structure (i.e., a physical structure, such as an electronic circuit system). More specifically, this expression is used to indicate that the physical structure is arranged to perform the one or more tasks during operation. A structure may be referred to as being “configured” to perform a task even if the structure is not currently operating. “Memory device configured to store data” is intended to cover, for example, an integrated circuit having circuitry for storing data during operation, even if the integrated circuit in question is not currently in use (e.g., power is not connected to the integrated circuit). Therefore, an entity described or stated as being “configured” to perform a task refers to a physical structure, such as a device, circuit system, memory storing program instructions that can be executed to perform the task, etc. This phrase is not used herein to refer to an intangible structure. Furthermore, the term “configured to” is not intended to mean “configurable as.” For example, an unprogrammed field-programmable gate array will not be considered "configured" to perform a particular function, but it may be "configurable" to perform that function after programming. Additionally, the statement in the appended claims that a structure is "configured" to perform one or more tasks is not expressly intended to be interpreted as having components plus functional elements.

Claims

1. A system comprising: A scheduling queue, configured to store elements to be executed by the shader circuitry system; and An arbitration circuit is configured to store a plurality of priority indicators corresponding to respective elements of the scheduling queue, wherein the arbitration circuit is configured to schedule the elements of the scheduling queue based on the plurality of priority indicators for execution at the shader circuitry system.

2. The system of claim 1, wherein scheduling the elements of the scheduling queue for execution comprises: In response to receiving a specific priority corresponding to a particular element, the next element to be sent to the shader circuitry is identified.

3. The system of claim 2, wherein, prior to receiving a corresponding request for the next element to be sent to the shader circuitry, the process of identifying the next element to be sent to the shader circuitry is performed.

4. The system according to any one of claims 1 to 3, wherein the scheduling queue comprises a plurality of slots configured to store corresponding elements, and wherein the plurality of priority indicators correspond to a corresponding slot among the plurality of slots.

5. The system according to any one of claims 1 to 4, wherein scheduling the elements of the scheduling queue for execution comprises: In response to a scheduling threshold number of elements with a first priority, at least one element with a second priority is scheduled, wherein the second priority is a priority lower than the first priority.

6. The system according to any one of claims 1 to 5, wherein the plurality of priority indicators are identifiers corresponding to the respective processes of the elements.

7. The system according to any one of claims 1 to 6, wherein the plurality of priority indicators are identifiers of the corresponding virtual machines corresponding to the elements.

8. The system according to any one of claims 1 to 7, wherein the arbitration circuit is further configured to store a second plurality of priority indicators, and wherein scheduling the elements of the scheduling queue for execution comprises: The selection is based on a priority selection signal, which selects between elements indicated by the plurality of priority indicators and elements indicated by a second plurality of priority indicators.

9. The system of claim 8, wherein the priority selection signal indicates the availability of hardware circuitry.

10. The system of claim 9, wherein the hardware circuit is a memory circuit or the shader circuit system.

11. A method, the method comprising: The arbitration circuit receives an indication of the priority of an element to be executed by the shader circuitry system, wherein the element is stored in the scheduling queue. The priorities stored in the arbitration circuit are sorted based on the indication of the priorities; as well as In response to a request for an element, the shader circuitry is provided with the next element indicated by a sorted priority.

12. The method of claim 11, wherein the priority indicates that the element has a first priority, and wherein the second element currently stored in the scheduling queue has a second priority.

13. The method of claim 11 or 12, wherein receiving the indication of the priority comprises extracting the priority from the received opcode corresponding to the element.

14. The method according to any one of claims 11 to 13, wherein the indication of the priority of the element is specified by the user.

15. The method according to any one of claims 11 to 13, wherein the indication of the priority of the element is specified by the driver based on the state of a user-visible process to be used using data generated using the element.

Images