Advanced hardware scheduling using unmapped queuing doors

Abstract

The processing device [100] includes a hardware scheduler [232], an unmapped queue unit [236], a command processor [234], and several compute units [246]. Depending on whether the queue doorbell [401] is an unmapped queue doorbell, the unmapped queue unit is configured to transmit a signal to the hardware scheduler indicating that work has been placed in a queue that is not currently mapped to a hardware queue of the processing device. The hardware scheduler is configured to map the queue to one of several hardware queues of the processing device [240] in response to the signal. The command processor is configured to dispatch work associated with the mapped queue to one or more compute units of several compute units.

Description

Background Art

[0001] In an accelerated processing device (APD) such as a graphics processing unit (GPU), hardware scheduling involves using queues to manage and schedule the execution of tasks or work items on the processing resources (e.g., cores) of the APD. Queues are an essential data structure that facilitates the organization, prioritization, and dispatch of tasks to the available hardware resources of the APD. The use of queues in hardware scheduling ensures efficient utilization of APD resources, maximizes parallelism, and optimizes overall performance. [[ID=~7]]

[0002] This disclosure may be better understood by reference to the accompanying drawings, in which various features and advantages will become apparent to those skilled in the art. The use of the same reference numerals in different drawings indicates similar or identical items.

Brief Description of the Drawings

[0003] [Figure 1] FIG. 1 is a block diagram of an exemplary processing system according to some embodiments. [Figure 2] FIG. 2 is a block diagram of an exemplary processing system for implementing hardware scheduling using an unmapped software queue doorbell according to some embodiments. [Figure 3] FIG. 3 is a block diagram showing a detailed view of an unmapped queue unit of the processing system of FIG. 2 according to some embodiments. [Figure 4] FIG. 4 is a flowchart showing an exemplary method for detecting work in an unmapped software queue and implementing hardware scheduling using an unmapped software queue doorbell according to some embodiments. [Figure 5]This flowchart illustrates exemplary methods for detecting work in unmapped software queues and performing hardware scheduling using unmapped software queue doorbells, according to several embodiments. [Figure 6] This figure shows an example of multiple register arrays for recording unmapped software cued abells, according to several embodiments. [Modes for carrying out the invention]

[0004] Traditionally, processing devices such as the central processing unit (CPU) directly wrote to the memory-mapped input / output (MMIO) registers of the APD (Application Processing Unit) to configure it. For example, the CPU scheduled tasks (kernels) or commands for the APD, such as vertex streams, texture information, and instructions for processing such information, by directly writing these tasks to the MMIO registers. However, modern APDs are typically driven by firmware-controlled microcontrollers such as hardware schedulers and command processors at the "front-end" of the graphics and computation pipeline, shifting the responsibility for register writing to these front-end microcontrollers. Therefore, distant CPUs only need to write the work packets to memory.

[0005] The firmware-controlled microcontroller of the APD reads and processes work submitted by external devices of the APD, such as a kernel-mode driver (KMD) or a user-mode driver (UMD), which run on a host processing device such as a central processing unit (CPU). For example, in at least some configurations, such as those implementing a heterogeneous system architecture (HSA), an application submits work (e.g., command packets) to a software queue managed by the CPU or APD driver and located in system memory. The CPU or APD driver copies the work from the software queue to a hardware queue associated with the APD and mapped to the software queue by a hardware scheduler. A notification mechanism (referred to herein as a "doorbell") is typically used to inform the APD, particularly the APD's command processor, that a new task is available for processing in the hardware queue. For example, an application or APD driver writes to a memory-mapped doorbell register associated with the hardware queue. The doorbell is a special register that can be accessed by both the CPU and the APD. When the doorbell register is written to, an interrupt or signal is sent to the command processor indicating that new work is available in the hardware queue. The command processor then schedules the task to be executed on the APD's processing units, such as the compute unit.

[0006] In at least some configurations, an APD is associated with multiple software queues to allow more work to be sent to the APD in parallel. For example, multiple users of an APD (such as web browsers and games) can simultaneously send work to the APD by placing command packets (including tasks, commands, or operations) to be executed on the APD into different software queues. The command packets are processed by the APD by mapping the software queues to hardware queues and scheduling work items associated with the hardware queues for processing by hardware resources such as the APD's compute units. However, the APD hardware has a limited number of pipes, each of which has a fixed number of hardware queues. Therefore, the APD hardware, such as the command processor, can usually only see a finite number of hardware queues simultaneously. Thus, if there are too many software queues, the APD's hardware scheduler typically implements one or more time-multiplexing techniques when determining which subset of the software queues to map to the hardware queues. These time-multiplexing techniques map the software queues onto the hardware queues by using time slicing to multiplex the software queues onto the hardware queues.

[0007] One technique for time-multiplexing software queues is to use a round-robin method to map software queues to available hardware queues. In this example, the hardware scheduler spends some time searching for work in software queue XY and then moves to software queue AB. If there is no work in software queue AB, this still causes a delay in processing work submitted to software queue XY. In some hardware configurations, it may even be necessary to remove work items in software queue XY from the hardware in order to check for work items in software queue AB, which results in reduced throughput in the hardware. Another technique for time-multiplexing software queues is for the user (e.g., an application) to explicitly inform the hardware scheduler when submitting work items to the software queue so that the hardware scheduler can prioritize mapping that software queue to the hardware queue whenever there is work, and unlink the software queue from the hardware queue if there is no work.

[0008] One problem with the first multiplexing technique is that it can lead to underutilization of hardware. Work items can be placed in unmapped software queues, but an "oblivious" hardware scheduler spends time searching for work items in empty software queues. Without being pushed information about when software queues have work items, the "oblivious" hardware scheduler typically has to determine which queue to attach and when by one of two options: (1) constantly rolling an execution list, which is a list of all hardware queues and locations in memory, every N milliseconds, assuming that all software queues have work in them, or (2) constantly rolling the execution list every N milliseconds, but checking whether software queues have work in them before attaching them. Both options add substantial runtime overhead, which has performance implications for both hardware and software. For example, if oversubscribed, a processor may incur runtime overhead proportional to the number of processes being used. For example, a 33% slowdown is possible with three processes, while a 50% slowdown is possible for all processes with four processes. Even when the second option is used, relatively high startup latency can exist if a software queue with work items is not attached. For example, it may take several milliseconds for the hardware schedule to arrive at the designated queue and check the work. This delay in checking the software queue for work items results in startup latency, which is a significant performance factor for most computing software. For example, a startup latency of 5 microseconds to 5 milliseconds has a significant performance impact.

[0009] Regarding the second technique, one problem is that it requires explicit software involvement in hardware scheduling / multiplexing decisions. For example, an application submitting a work item needs to determine whether the software queue associated with the work item is mapped to a hardware queue in the APD, and then ring a specific doorbell (e.g., write to a specific hardware location) to request that the hardware scheduler map this particular software queue to the hardware queue. This process breaks interface abstractions by making software directly part of the scheduling decision, which increases software complexity. For example, HSA kernel dispatch is handled by user-space code through a fixed application binary interface (ABI) (e.g., moving hardware queue write pointers, writing dispatch packets, ringing hardware queue doorbells). Modifying the ABI to ring different doorbells based on hardware scheduling logic breaks the layer of abstraction provided by the ABI, making it difficult to use such "explicit messaging" solutions.

[0010] Therefore, this disclosure describes embodiments of a system and method for performing hardware scheduling in an APD using an unmapped queue doorbell. As will be described in more detail below, the hardware scheduler of the APD is notified via a notification mechanism, such as a doorbell, when work (e.g., command packets) is placed in a software queue that is not mapped to a hardware queue of the APD. In other words, the notification mechanism signals the hardware scheduler to detect an unmapped software queue having a work item and requests the hardware scheduler to map the software queue to a hardware queue. In at least some embodiments, the notification hardware “piggybacks” onto the same doorbell, which is part of submitting command packets to the hardware queue of the command processor. However, before passing the doorbell to the command processor, the doorbell is checked by the scheduling / queue multiplexing mechanism to determine whether the queue targeted by the doorbell is currently mapped to a hardware queue of the APD. If so, the doorbell is forwarded to the command processor so that the command processor is notified that a new command packet has arrived in any hardware queue. However, if the doorbell is associated with a software queue that is not mapped to a hardware queue on the APD, the hardware scheduler is notified via an interrupt (i.e., another signaling mechanism) that the unmapped software queue currently has work. The hardware scheduler uses this knowledge of the unmapped software queues that have work when determining which software queues to map to the hardware queues on the APD, or which software queues to disconnect from the hardware queues.In other words, instead of the hardware scheduler needing to analyze each software queue to determine if it has work, the hardware scheduler is proactively notified if an unmapped software queue has work, which improves the efficiency and performance of the hardware scheduler. Also, applications that submit work items to software queues do not need to be coded with complex mechanisms to notify the hardware scheduler when work is submitted to an unmapped software queue, thereby removing software from the scheduling process and maintaining the layer of abstraction provided by the ABI.

[0011] Figure 1 illustrates an exemplary processing system 100 (also referred to herein as “computing system 100”) which may implement one or more of the techniques described herein for performing hardware scheduling in an accelerated processor using unmapped queued abels. Note that the number of components of the processing device 100 will vary from embodiment to embodiment. For example, in at least some embodiments, there will be more or fewer of each component / subcomponent than shown in Figure 1. In at least some embodiments, the processing system 100 will include other components not shown in Figure 1, or will be structured in a manner other than that shown in Figure 1. Furthermore, the components of the processing system 100 may be implemented as hardware, circuitry, firmware, software, or any combination thereof.

[0012] In at least some embodiments, the processing system 100 includes one or more processors 102, such as a central processing unit (CPU), and one or more accelerated processing devices (APDs) 104, such as a graphics processing unit (GPU) (also referred to herein as "accelerated processor 104," "processor 104," or "accelerator unit 104"). Other embodiments of the APD 104 include any of the following: various parallel processors, vector processors, coprocessors, general-purpose GPUs (GPGPUs), non-scalar processors, highly parallel processors, artificial intelligence (AI) processors, inference engines, machine learning processors, other multithreaded processing units, scalar processors, serial processors, or any combination thereof. In at least some embodiments, the processor 102 includes one or more single-core or multi-core CPUs. In at least some embodiments, the APD104 includes any collaborative set of hardware and / or software that perform functions and computations associated with accelerating graphics processing tasks, data-parallel tasks, nested data-parallel tasks, etc., for resources such as conventional CPUs, conventional graphics processing units (GPUs), and combinations thereof.

[0013] In the embodiment shown in Figure 1, the processor 102 and APD 104 are formed and combined on a single silicon die or package to provide an integrated programming and execution environment. This environment allows the APD 104 to be used as flexibly as the processor 102 for several programming tasks. In other embodiments, the processor 102 and APD 104 are formed separately and mounted on the same or different substrates. It should be understood that the processing system 100 includes more or fewer components than those shown in Figure 1, at least in some embodiments. For example, the processing system 100 additionally includes one or more input interfaces, non-volatile storage, one or more output interfaces, a network interface, and one or more displays or display interfaces.

[0014] As illustrated in Figure 1, the processing system 100 includes a system memory 106, an operating system (OS) 108, a communication infrastructure 110, one or more software applications 112, an input-output memory management unit (IOMMU) 114, an input / output (I / O) interface 116, and other devices 118. Access to the system memory 106 is managed by a memory controller (not shown) coupled to the system memory 106. For example, requests from the processor 102 or other devices to read from or write to the system memory 106 are managed by the memory controller. In some embodiments, one or more applications 112 include various programs or commands for performing calculations that are also performed by the processor 102. The processor 102 sends selected commands for processing in the APD 104. The operating system 108 and the communication infrastructure 110 are described in more detail below.

[0015] Within the processing system 100, system memory 106 includes non-persistent memory such as dynamic random access memory (not shown). In at least some embodiments, system memory 106 stores processing logic instructions, constant values, variable values ​​during the execution of an application or other part of processing logic, or other desired information. For example, in at least some embodiments, a part of the control logic for performing one or more operations on the processor 102 resides in system memory 106 during the execution of each part of the operation by the processor 102. During execution, each application, operating system function, processing logic command, and system software resides in system memory 106. Control logic commands essential to the operating system 108 generally reside in system memory 106 during execution. In some embodiments, other software commands (e.g., a set of instructions or commands used to implement a device driver 120) reside in system memory 106 during the execution of the processing system 100.

[0016] The Input / Output Memory Management Unit (IOMMU) 114 is a multi-context memory management unit. In use in this invention, the context takes into account the environment in which the kernel is executed and the domains in which synchronization and memory management are defined. The context includes a set of devices, the memory accessible to those devices, the corresponding memory properties, and one or more command queues used to schedule the execution of kernels or operations on memory objects. The IOMMU 114 includes logic for performing virtual-to-physical address translation for memory page access of devices such as the APD 104. In some embodiments, the IOMMU 114 includes or has access to a translation lookaside buffer (TLB) (not shown). The TLB is implemented in content addressable memory (CAM) to accelerate the translation of logical (i.e., virtual) memory addresses to physical memory addresses for requests made by the APD 104 for data in system memory 106.

[0017] I / O interface 116 represents any number and type of I / O interface (e.g., peripheral component interconnect (PCI) bus, PCI-Extended (PCI-X), PCI Express (PCIE) bus, Gigabit Ethernet (GBE) bus, Universal Serial Bus (USB)). Various types of peripheral devices are coupled to I / O interface 106. Such peripheral devices include (but are not limited to) displays, keyboards, mice, printers, scanners, joysticks, other types of game controllers, media recording devices, external storage devices, network interface cards, etc. Other devices 118 represent any number and type of devices (e.g., multimedia devices, video codecs).

[0018] In at least some embodiments, the communication infrastructure 110 interconnects the components of the processing system 100. The communication infrastructure 110 includes one or more of the following: a Peripheral Component Interconnection (PCI) bus, an Extended PCI (PCI-E) bus, an Advanced Microcontroller Bus Architecture (AMBA) bus, an Advanced Graphics Port (AGP), or other such communication infrastructure and interconnections (not shown). In some embodiments, the communication infrastructure 110 includes an Ethernet® network or any other suitable physical communication infrastructure that satisfies the data transfer rate requirements of the application. The communication infrastructure 110 also includes functions for interconnecting components, including the components of the processing system 100.

[0019] A driver, such as device driver 120, communicates with a device (e.g., APD 104) through an interconnection or communication infrastructure 110. When a calling program invokes a routine within device driver 120, device driver 120 issues a command to the device. When the device sends data back to device driver 120, device driver 120 invokes the routine of the original calling program. Generally, device drivers are hardware-dependent and operating system-specific in order to provide essential interrupt handling to any necessary asynchronously dependent hardware interfaces. In some embodiments, a compiler 122 is embedded within device driver 120. The compiler 122 compiles the source code into program instructions necessary for execution by the processing system 100. During such compilation, the compiler 122 applies transformations to program the instructions at various stages of compilation. In other embodiments, the compiler 122 is a standalone application. In at least some embodiments, the device driver 120 controls the operation of the APD104 by, for example, providing an application programming interface (API) to software (e.g., application 112) running on the processor 102 to access various functions of the APD104.

[0020] The processor 102 includes one or more of the following (not shown): a control processor, a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or a digital signal processor (DSP). The processor 102 executes at least a portion of the control logic that controls the operation of the processing system 100. For example, in at least some embodiments, the processor 102 runs the operating system 108 and one or more applications 112. In some embodiments, the processor 102 initiates and controls the execution of one or more applications 112 by distributing the processing associated with one or more applications 112 across the processor 102 and other processing resources such as the APD 104.

[0021] The APD104 executes commands and programs for selected functions, such as graphics operations and other operations, which are particularly well-suited for parallel processing. The APD104 is frequently used to perform graphics pipeline operations, such as pixel manipulation, geometric calculations, and rendering images to a display. In some embodiments, the APD104 performs computational operations (e.g., non-graphics-related operations such as video manipulation, physical simulations, and computational fluid dynamics) based on commands or instructions received from the processor 102. For example, such commands include special instructions not typically defined in the instruction set architecture (ISA) of the APD104. In some embodiments, the APD104 receives image geometry representing a graphics image, along with one or more commands or instructions for rendering and displaying the image. In at least some embodiments, the image geometry corresponds to a representation of a two-dimensional (2D) or three-dimensional (3D) computer graphics image.

[0022] As will be further explained with reference to Figure 2, the APD104 includes one or more parallel processing units that perform calculations according to the single-instruction-multiple-data (SIMD) paradigm. In one or more embodiments, the APD104 is used to implement a GPU, in which case the parallel processing units are referred to as shader cores or streaming multi-processors (SMX). Each parallel processing unit includes one or more processing elements, such as scalar and / or vector floating-point units and arithmetic logic units (ALUs). In at least some embodiments, the parallel processing unit includes dedicated processing units (not shown), such as inverse square root units and sine / cosine units.

[0023] Figure 2 shows a more detailed embodiment of the processing system 200 (also referred to herein as the “computing system 200”), which can implement one or more of the techniques described herein for performing hardware scheduling in the accelerated processor using an unmapped queu-a-bell. As illustrated, the processing system 200 includes at least a processor 202, an APD 204 (also referred herein as the “accelerated processor 204” or “processor 204”), and system memory 206. In at least some embodiments, the processor 202, APD 204, and system memory 206 are implemented as described above with respect to Figure 1. It should be understood that the processing system 200 also includes other components that are not illustrated for brevity. For example, in at least some embodiments, the processing system 200 includes additional components such as software, hardware, and firmware components, in addition to or different from those shown in Figure 2. In at least some embodiments, the APD204 includes other components, omits one or more of the exemplary components, has multiple instances of the components even if only one instance is shown in Figure 2, is organized in other preferred ways, or is a combination thereof.

[0024] In at least some embodiments, processing system 200 executes any of a variety of types of software applications 212. In some embodiments, as part of executing software application 212, processor 202 of processing system 200 launches tasks to be executed in APD 204. For example, if software application 212 executed in processor 202 requires graphics (or compute processing), processor 202 provides graphics commands and graphics data (or compute commands and compute data) to command buffer 224 in system memory 206 (or APD memory 230) for subsequent retrieval and processing by APD 204. In at least some embodiments, one or more device drivers 220 convert high-level commands from software application 212 into low-level command packets 226 that APD 204 can understand. Device driver 220 writes command packets 226 having commands corresponding to one or more tasks. The commands include, for example, draw commands, compute commands, global state updates, block state updates, combinations thereof, and the like. Device driver 220 assembles command packets 226 within a particular command buffer 224, which, in at least some instances, contains multiple command packets 226 for a specified task or set of tasks. In at least some embodiments, device driver 220 implements one or more software queues 228 for assembling and preparing command buffer 224 for submission to hardware queue 240 of APD 204.

[0025] In at least some embodiments, device driver 220 includes software, firmware, hardware, or any combination thereof. In at least some embodiments, device driver 220 is implemented entirely in software. Device driver 220 provides an interface, application programming interface (API), or a combination thereof for communication access to APD 204 and access to the hardware resources of APD 204. Examples of device driver 220 include a kernel-mode driver, a user-mode driver, and the like.

[0026] As described above, system memory 206 includes one or more memory buffers (including command buffer 224), through which processor 202 communicates commands to APD 204 (e.g., provided via device driver 220). In at least some embodiments, such memory buffers are implemented as a queue, a ring buffer, or other data structure suitable for efficient queuing of work or command packets 226. In the case of a queue, command packets are placed in the queue and retrieved from the queue. In at least some embodiments, system memory 206 includes an indirect buffer that holds the actual commands (e.g., instructions, data, pointers, non-pointers, etc.). For example, in some embodiments, when processor 202 communicates command packet 226 to APD 204, command packet 226 is stored in the indirect buffer, and a pointer to the indirect buffer is inserted into one or more entries (storing commands, data, or related context) of command buffer 224.

[0027] In at least some embodiments, the APD 204 includes memory 230, one or more hardware schedulers (HWS) 232, one or more processors such as one or more command processors (CP) 234, one or more unmapped queue units 236, and one or more APD subsystems 238, including, for example, compute resources and graphics / compute pipelines. Although the hardware schedulers 232, command processors 234, and unmapped queue units 236 are shown as separate components in Figure 2, it should be understood that in other embodiments, two or more of these components are part of the same component. For example, in at least some embodiments, the hardware scheduler 232 is part of the command processor 234, the unmapped queue unit 236 is part of the hardware scheduler 232, the unmapped queue unit 236 is part of the command processor 234, and so on.

[0028] In at least some embodiments, the APD memory 230 is on-chip memory accessible to both the APD 204 and the processor 202. The APD memory 230 includes, for example, a hardware queue 240 and a doorbell register 242. In at least some embodiments, the hardware queue 240 is a data structure or buffer implemented in the APD memory 230 accessible to both the CPU 202 and the APD 204. The hardware queue 240 receives command buffers 224 from the device driver 220 and holds the buffers 224 until the command processor 234 selects them for execution. In at least some embodiments, there are multiple hardware queues dedicated to different workloads, such as graphics, computation, or copy operations. In at least some embodiments, the hardware queue 240 is an HSA queue.

[0029] The doorbell register 242 is a register implemented in the APD memory 230 that facilitates communication between the device driver 220 running on the processor 202 and the hardware scheduler 232 of the APD 204. The doorbell register 242 functions as a signaling mechanism to inform the APD 204 when a new command buffer 224 has been submitted to the hardware queue 240 and is ready for execution. In at least some embodiments, when the device driver 220 submits the command buffer 224 (work) to the hardware queue 240, the device driver 220 writes a specific value to the corresponding doorbell register 242. In at least some embodiments, as will be described in more detail below, the device driver 220 is configured to write to a doorbell register 242 mapped to a software queue 228. In these embodiments, when the device driver 220 prepares the command buffer 224 in the software queue 228, the device driver 220 writes to the doorbell register 242 mapped to that software queue 228. In at least some embodiments, the doorbell register 242 is implemented using memory-mapped input / output (MMIO) such that the doorbell register 242 is mapped to the address space of the processor 202. This allows the device driver 220 running on the processor 202 to access and operate the doorbell register 242 using normal memory read and write operations. In at least some embodiments, the doorbell register 242 is maintained within a doorbell bar 244, which is an area of ​​APD memory 230 designated for the doorbell register 242. As will be described in more detail below, in some embodiments, each software queue 228 generated by the device driver 220 is associated with a separate doorbell register 242. However, in other embodiments, two or more software queues 228 are associated with the same doorbell register 242.

[0030] In at least some embodiments, the hardware scheduler 232 maps software queues 228 in system memory 206 to hardware queues 240 in APD memory 230. In at least some embodiments, the hardware scheduler 232 tracks command buffers 224 submitted to hardware queues 240 by device driver 220, or command buffers 224 written to software queues 228 mapped to hardware queues 240 by device driver 220 (also referred to as "mapped software queues 228"). The hardware scheduler 232 determines the priority of command buffers 224 in hardware queues 240 based on factors such as task type, resource availability, and scheduling policy. In at least some embodiments, the hardware scheduler 232 is implemented as hardware, circuitry, firmware, a firmware-controlled microcontroller, software, or any combination thereof.

[0031] In at least some embodiments, the command processor 234 detects when a command buffer 224 is submitted to a hardware queue 240. For example, the command processor 234 detects a doorbell associated with the hardware queue 240. In other words, the command processor 234 detects when a device driver 220 writes to a doorbell register 242 associated with the hardware queue 240 and changes the value of the doorbell register 242. The command processor 234 reads the command packet 226 in the command buffer 224, decodes the packet 226, and dispatches the appropriate command to the corresponding execution unit in the APD 204, such as a shader core, fixed-function unit, or memory controller, in order to understand which action needs to be taken. In at least some embodiments, the command processor 234 is implemented as hardware, circuitry, firmware, a firmware-controlled microcontroller, software, or any combination thereof.

[0032] The unmapped queue unit 236 includes one or more components to facilitate hardware scheduling in the APD204 using unmapped queue doorbells. For example, as shown in Figure 3, the unmapped queue unit 236 includes, in at least some embodiments, a doorbell monitor 348, a mapped queue doorbell filter 350, an interrupt unit 352, an interrupt holding mechanism 354, and an unmapped doorbell list 356. Each of these components is described in more detail below. In at least some embodiments, the unmapped queue unit 236 monitors doorbells (e.g., writes to the doorbell register 242) associated with a software queue 228 that is not mapped to the hardware queue 240 of the APD204. Such a software queue 228 is referred to herein as the “unmapped software queue 228”. When the unmapped queue unit 236 detects a doorbell associated with the unmapped software queue 228, it notifies the hardware scheduler 232 that the unmapped software queue 228 has a command buffer 224 ready for processing. The hardware scheduler 232 then uses this knowledge to determine when and how to map the unmapped software queue 228 to the hardware queue 240 so that the command processor 234 can process the command buffer 224. If a doorbell is associated with the mapped software queue 228, the unmapped queue unit 236 passes the doorbell to the command processor 234, or the command processor 234 detects the doorbell. For example, the unmapped queue unit 236 sends a doorbell signal to the command processor 234 to inform it that the hardware queue 240 has work to do (e.g., the command buffer 224).In at least some embodiments, the unmapped queue unit 236 is implemented as hardware, circuitry, firmware, a firmware-controlled microcontroller, software, or any combination thereof.

[0033] Referring again to Figure 2, the APD subsystem 238 includes, in at least some embodiments, various processing blocks, APD computing resources, etc. As used herein, the term “block” refers to a module (e.g., circuit) included in an ASIC, CPU execution pipeline, GPU graphics pipeline, or a combination thereof. Such modules include, but are not limited to, cache memory, arithmetic logic units, multiplication / division units, floating-point units, geometry shaders, vertex shaders, pixel shaders, various other shaders, clipping units, z-buffers (e.g., depth buffers), color buffers, or any other processing modules. The APD subsystem 238 includes any cooperating ensemble of hardware and / or software that performs functions and computations associated with accelerating graphics processing tasks, data-parallel tasks, nested data-parallel tasks, etc., in relation to resources such as conventional CPUs, conventional graphics processing units (GPUs), and combinations thereof.

[0034] In various embodiments, the APD subsystem 238 includes one or more compute units (CUs) 246 (exemplified as CU246-1 and CU246-2), such as one or more processing cores, each containing one or more single-instruction multiple-data (SIMD) units (for example, each configured and illustrated to execute a thread concurrently with the execution of other threads in a wavefront by other SIMD units according to a SIMD execution model). The SIMD execution model allows multiple processing elements to share a single program control flow unit and program counter, and thus execute the same program, but with different data. Processing cores are also referred to as shader cores or SMXs. The number of compute units 246 implemented in the APD 204 is configurable. Each compute unit 246 includes one or more processing elements, such as scalar and / or vector floating-point units, ALUs, etc. In various embodiments, the compute units 246 include dedicated processing units, such as inverse square root units and sine / cosine units.

[0035] Each of the one or more computing units 246 performs an instantiation of a specific work item to process incoming data, and the basic unit of execution in one or more computing units 246 is a work item (e.g., a thread). Each work item represents, for example, the creation of a single instance of a collection of parallel executions of kernels launched on the device by parallel-executing commands. A work item is executed in one or more processing elements as part of a work group executed in the computing unit 246.

[0036] The APD204 issues and executes work items, such as groups of threads, that run concurrently as a "wavefront" on a single SIMD unit. In at least some embodiments, a wavefront is interchangeably referred to as a warp, vector, or thread. In some embodiments, a wavefront includes an instance of parallel execution of a shader program, and each wavefront includes multiple work items that run concurrently on a single SIMD unit according to a SIMD paradigm (e.g., one instruction control unit that executes multiple data and the same instruction streams).

[0037] The parallelism provided by one or more compute units 246 is suitable for graphics-related operations such as pixel value calculation, vertex transformation, tessellation, geometry shading operations, and other graphics operations. The graphics processing pipeline accepts graphics processing commands from the CPU 202 and provides compute tasks to the compute units 246 for parallel execution. Some graphics pipeline operations, such as pixel processing and other parallel compute operations, require that the same command stream or compute kernel be performed on a stream or set of input data elements. Each instantiation of the same compute kernel is executed simultaneously on multiple SIMD units within one or more compute units 246 to process such data elements in parallel. As referred to herein, for example, a compute kernel is declared in a program and executed on an Accelerated Processing Device (APD) compute unit 246. This function is also referred to as a kernel, shader, shader program, or program.

[0038] As described above, the processing system 200 implements multiple software queues 228 for managing the command buffer 224 generated by the device driver 220. Implementing multiple software queues 228 makes it possible to send more tasks to the APD 204 in parallel. For example, multiple users of the APD 204 (such as web browsers and games) can send tasks to the APD 204 simultaneously by placing their command packets into different software queues 228. The hardware scheduler 232 maps the software queues 228 to the hardware queues 240 of the APD 204, and the command processor monitors a doorbell register 242 mapped to the hardware queues 240 to determine when the command buffer 224 has been submitted to the hardware queues 240 by the device driver 220.

[0039] In at least some embodiments, the APD204 hardware has a limited number of pipes, each pipe having a fixed number of hardware queues 240. Therefore, the command processor 234 can only see a finite number of hardware queues 240 at a time. For example, in one configuration, the APD204 implements 32 (or some other number) hardware queues 240. In this configuration, if 1000 software queues 228 are generated, the hardware scheduler 232 maps only 32 of these 1000 software queues 228 to the hardware queues 240 at a time. Thus, in a conventional configuration, the hardware scheduler 232 or the command processor 234 is only notified when a command buffer 224 is placed in one of the 32 mapped software queues 228, and the command processor 234 processes only the command buffer 224 in the 32 hardware queues 240, regardless of whether the command buffer 224 is placed in one of the remaining unmapped software queues 228. In at least some embodiments, the hardware scheduler 232 implements one or more time-multiplexing techniques when determining which subset of software queues 228 to map to the hardware queue 240. However, in conventional configurations, the hardware scheduler 232 performs time-multiplexing without knowledge of which software queues 228 have command buffers 224 (e.g., work). Therefore, in conventional configurations, the hardware scheduler 232 selects a software queue 228 that does not have a command buffer 224 to map to the hardware queue 240 while waiting for another software queue 228 with a command buffer 224 to be mapped.

[0040] Thus, in at least some embodiments, the APD204 is configured to detect when the device driver 220 writes to the command buffer 224, i.e., submits work to an unmapped software queue 228, so that when the hardware scheduler 232 determines which software queue 228 to map to the hardware queue 228 of the APD204, it prioritizes an unmapped software queue 240 over other software queues 228 that have no work. An example of this process is illustrated in Figures 4 and 5. For example, Figures 4 and 5 together illustrate an exemplary method 400 performed by the APD204 for detecting work in an unmapped software queue 228 and performing hardware scheduling using an unmapped software queue doorbell. For illustrative purposes, method 400 is described in relation to an exemplary embodiment in the processing system 200 of Figure 2, but it should be understood that in other embodiments, method 400 may be performed in processing devices with different configurations. Furthermore, Method 400 is not limited to the sequence of operations shown in Figures 4 and 5, and at least some of the operations can be performed in parallel or in different sequences. Moreover, in at least some embodiments, Method 400 may include one or more operations different from those shown in Figures 4 and 5.

[0041] In block 402, the hardware scheduler 232 maps a subset of the software queues 228 generated by the device driver 220 to the hardware queues 240 of the APD 204. In at least some embodiments, the hardware scheduler 232 maps the software queues 228 to the hardware queues 240 using an identifier, an index value, or other identifying information that uniquely identifies the software queues 228. It should be understood that the hardware scheduler 232 can map the software queues 228 to the hardware queues 240 and de-map them from the hardware queues 240 at different points throughout Method 400. In block 404, the device driver 220 puts work into the software queues 228. For example, the device driver 220 writes one or more command packets 226 to the command buffer 224 maintained by the software queues 228. In block 406, the device driver 220 “rings” the software queue doorbell 401 associated with the software queue 228 in which the device driver 220 has placed work. For example, the device driver 220 “rings” the doorbell 401 (also referred to herein as “generating a doorbell notification 401”) by writing to the doorbell register 242 associated with the software queue 228. In at least some embodiments, the device driver 220 writes a specific value to the doorbell register 242, such as an identifier or value representing the location of the associated command buffer 224.

[0042] In block 408, the unmapped queue unit 236 detects that a doorbell 401 has been generated for a software queue 228. For example, the doorbell monitor 348 of the unmapped queue unit 236 monitors the doorbell bar 244 for changes to any of the doorbell registers 242. In this embodiment, the doorbell monitor 348 detects when the device driver 220 writes a value to the doorbell register. In block 410, the unmapped queue unit 236 determines whether the doorbell 401 is associated with a mapped software queue 228 or an unmapped software queue 228. For example, the mapped queue doorbell filter 350 of the unmapped queue unit 236 determines the identifier of the software queue 228 associated with the doorbell register 242 that generated the doorbell 401. The mapped doorbell queue filter 350 compares the software queue identifier with the software queue identifier in the hardware queue mapping list 403. In at least some embodiments, the mapping list 403 identifies a hardware queue 240 and a software queue 228 that is currently mapped to each of the hardware queues 240.

[0043] If the identifier of the software queue 228 associated with the doorbell 401 matches the software queue identifier in the mapping list 403, the unmapped queue unit 236 determines that the software queue 228 is currently mapped to the hardware queue 240 and determines that the doorbell 401 is a mapped queue doorbell. In block 412, the unmapped queue unit 236 forwards the doorbell 401 to the command processor 234. For example, the interrupt unit 352 of the unmapped queue unit 236 generates an interrupt 405 or another type of signal and sends it to the command processor 234. In at least some embodiments, the interrupt 405 includes information such as an identifier or memory address that identifies the doorbell register 242 that generated the doorbell 401. In other embodiments, the interrupt 405 does not include this information and acts as a trigger for the command processor 234 to proceed to each of the hardware queues 240 for new work (e.g., command buffer 224) and check it. In some embodiments, the command processor 234 monitors the hardware queues 240 for new work. In at least some of these embodiments, if an unmapped queue unit 236 determines that a software queue 228 associated with the doorbell 401 is currently mapped to a hardware queue 240, the unmapped queue unit 236 does not send an interrupt 405 to the command processor 234, so that the command processor 234 can automatically detect new work in the hardware queue 240.

[0044] In block 414, the command processor 234 retrieves work (e.g., command buffer 224) from the hardware queue 240 which is mapped to the software queue 228. In at least some embodiments, if the software queue 228 is mapped to the hardware queue 240, the device driver 220 moves or copies the work (e.g., command buffer 224) from the software queue 228 to the hardware queue 240. In block 416, the command buffer 224 dispatches one or more work items 407 for the work retrieved from the hardware queue 240 to one or more compute units 246 of the APD 204. Method 400 then returns to block 404.

[0045] Referring again to block 410, if the identifier of the software queue 228 associated with the doorbell 401 does not match the software queue identifier in the mapping list 403, the unmapped queue unit 236 determines that the doorbell 401 is an unmapped queue doorbell and that the associated software queue 228 is not currently mapped to the hardware queue 240. In other words, the unmapped queue unit 236 determines that the detected software doorbell queue 401 is an unmapped queue doorbell. Method 400 then proceeds to block 418 in Figure 4. In block 418, the unmapped queue unit 236 notifies the hardware scheduler 232 that the unmapped software queue 228 has work to do for processing (e.g., command buffer 224). For example, the interrupt unit 352 of the unmapped queue unit 236 generates an interrupt 409 or another type of signal or message and sends it to the hardware scheduler 232.

[0046] In block 420, upon receiving interrupt 409, the hardware scheduler 232 identifies an unmapped software queue 228 associated with the doorbell 401. For example, in at least some embodiments, interrupt 409 signals or triggers the hardware scheduler 232 to search through each of the unmapped software queues 228, either directly or indirectly by searching through each of the doorbell registers 242, in order to identify an unmapped software queue 228 that has work. In other embodiments, as will be described in more detail below, interrupt 409 signals the hardware scheduler 232 to process an unmapped doorbell list 356 maintained by an unmapped queue unit 236 in order to identify an unmapped software queue 228 that has work.

[0047] In block 422, after identifying an unmapped software queue 228 with work, the hardware scheduler 232 prioritizes the identified unmapped software queue 228 over an unmapped software queue 228 with no work, and maps the identified unmapped software queue 228 to the hardware queue 240 of the APD204. In block 424, the command processor 234 retrieves work (e.g., command buffer 224) from the hardware queue 240 mapped to the software queue 228. In block 426, the command buffer 224 dispatches one or more work items 411 for the work retrieved from the hardware queue 240 to one or more compute units 246 of the APD204. Method 400 then returns to block 404.

[0048] In some examples, in block 418 of Figure 5, after an unmapped queue unit 236 sends an initial interrupt 409 associated with an unmapped software queue 228 to the hardware scheduler 232, additional work may be placed in the same unmapped software queue 228. If multiple instances of work are placed in the unmapped software queue 228 after the hardware scheduler 232 receives the initial interrupt 409 associated with the unmapped software queue 228, the hardware scheduler 232 will discover each instance of work when it processes the initial interrupt 409 to map the software queue 228. Therefore, the unmapped queue unit 236 only needs to send one interrupt 409 to notify the hardware scheduler 232 that the unmapped software queue 228 has work until the hardware scheduler 232 indicates that one interrupt 409 has been processed for the unmapped queue unit 236. In other words, even if an unmapped queue unit 236 can detect multiple doorbells 401 for the same unmapped software queue 228 after an initial interrupt 409 has been sent to the hardware scheduler 232, the unmapped queue unit 236 will not send another interrupt 409 for the same queue 228 until the hardware scheduler 232 indicates to the unmapped queue unit 236 that the initial (pending) interrupt 409 has been processed.

[0049] Thus, in at least some embodiments, the unmapped queue unit 236 implements an interrupt holding mechanism 354 to pause the transmission of interrupt 409 (or other signals or messages) to the hardware scheduler 232 after an initial interrupt 409 has been sent to the unmapped software queue 228. In other words, the interrupt holding mechanism 354 pauses or prevents additional signals from being transmitted to the hardware scheduler 232 when additional unmapped queue doorbells are detected for the software queue 228, until the hardware scheduler has processed the pending interrupt 409. The interrupt holding mechanism 354 is implemented as a flag in at least some embodiments, but other embodiments are also applicable. In at least some embodiments, the flag is implemented for each unmapped software queue 228 detected by the unmapped queue unit 236. In block 418 of Figure 5, if the unmapped queue unit 236 sends an initial interrupt 409 to the hardware scheduler 232, the unmapped queue unit 236 sets a flag (for example, changes the bit value from zero to a non-zero value) to indicate that the interrupt 409 has been sent to the hardware scheduler 232 but has not been processed by the hardware scheduler 232. In block 410 of Figure 4, if the unmapped queue unit 236 detects another doorbell associated with the same unmapped software queue 228, the unmapped queue unit 236 checks the queue 228's flag to determine if there are any pending interrupts 409 for the unmapped software queue 228 waiting to be processed by the hardware scheduler 232. If the flag does not indicate that there are any pending interrupts 409 (for example, the flag has a bit value of 0), the unmapped queue unit 236 sends another interrupt 409 to the hardware scheduler 232 for the unmapped software queue 228.If the flag indicates that there is a pending interrupt 409 (for example, the flag has a bit value of 1), the unmapped queue unit 236 does not send another interrupt 409 to the hardware scheduler 232 for the unmapped software queue 228. However, in at least some embodiments, the unmapped queue unit 236 continues to update the unmapped doorbell list 356 (for example, by changing a bit value) to indicate that additional work has been placed in the unmapped software queue 228, as described below. The unmapped queue unit 236 then continues to monitor for the additional doorbell (for example, block 408 in Figure 4).

[0050] In at least some embodiments, if the hardware scheduler 232 processes the pending interrupt 409 in any of blocks 418 to 422 in Figure 5, or maps an unmapped queue 228, the hardware scheduler 232 notifies the unmapped queue unit 236 that the pending interrupt 409 has been processed. When the unmapped queue unit 236 receives or detects this notification from the hardware scheduler 232, it clears the flag of the associated software queue 228 (for example, by using a bitmask to change the bit value from a non-zero value to a zero value). The notification from the hardware scheduler 232 can be carried out in various ways. For example, in at least some embodiments, the hardware scheduler 232 sends an interrupt, signal, or message to the unmapped queue unit 236 to indicate that the hardware scheduler 232 has processed the pending interrupt 409. In other embodiments, the hardware scheduler 232 indicates that the software queue 228 has work to do by setting or clearing specific bits in the unmapped doorbell list 356. For example, the hardware scheduler 232 uses a bitmask to clear one or more bits in the unmapped doorbell list 356. The unmapped queue unit 236 detects the change in the unmapped doorbell list 356 and clears the flags in the associated software queue 228, for example, using a bitmask.

[0051] In at least some embodiments, the APD204 is configured to implement various mechanisms that enable the hardware scheduler 232 to identify unmapped software queues 228 associated with the doorbell 401, as shown in block 420 of Figure 5. As described below, these mechanisms configure the APD204 to perform interrupt-based polling of the software queues 228, for example, compared to time-based polling. The interrupt-based polling described herein has the advantage that the hardware scheduler 232 does not need to sequentially search for read and write pointers for all software queues 228. This allows the hardware scheduler 232 to avoid checking for work in unmapped software queues when 228 is empty, resulting in reduced power utilization. Another advantage is that the unmapped doorbell list 356 becomes more accurate, and when an unmapped queue doorbell 401 is detected, there are fewer unmapped software queues 228 that the hardware scheduler 232 needs to search, which results in faster scheduling of the software queues 228 when new work is submitted to them.

[0052] For example, in at least some embodiments, each software queue 228 is mapped to an individual bit in an unmapped doorbell list 356. In other words, if there are N software queues 228, each of the N software queues 228 is mapped to N different bits. The unmapped doorbell list 356 is, in at least some embodiments, a mapping data structure such as a bitmap, bit array, or first-in, first-out buffer. In at least some embodiments, the unmapped doorbell list 356 is stored in APD memory 230. In at least some embodiments, the software queues 228 are mapped directly to individual bits in the unmapped doorbell list 356. In other embodiments, the software queues 228 are mapped to individual bits in the unmapped doorbell list 356 through doorbell registers 242. For example, each software queue 228 is mapped to a separate doorbell register 242, and each doorbell register 242 is mapped to an individual bit in the unmapped doorbell list 356. Each bit in the unmapped doorbell list 356 is associated with (or represents) an identifier or index corresponding to a specified software queue 228 or a doorbell register 242 mapped to the software queue 228, in at least some embodiments. In block 410 of Figure 4, if the unmapped queue unit 236 determines that the detected doorbell 401 is associated with an unmapped software queue 228, the unmapped queue unit 236 sets a bit in the unmapped doorbell list 356 of the software queue 228 (or doorbell register 242) associated with the doorbell 401 (for example, by changing a zero value to a non-zero value).When the hardware scheduler 232 receives the interrupt 409 generated in block 418 of Figure 5, the hardware scheduler 232 reads the unmapped doorbell list 356 to identify the bits that have non-zero values, indicating to the hardware scheduler 232 (either directly or through the doorbell register 242) that the software queue 228 has work to do.

[0053] In at least some embodiments, interrupt 409 includes a bit sequence 413 that directly represents all of the software queue 228. Alternatively, bit sequence 413 indirectly represents the software queue 228 through the doorbell register 242. For example, each bit in bit sequence 413 represents a doorbell register 242 that maps to the software queue 228. In block 410 of Figure 4, if the unmapped queue unit 236 determines that the doorbell 401 is associated with an unmapped software queue 228, the unmapped queue unit 236 sets a bit in bit sequence 413 for the software queue 228 (or doorbell register 242) associated with the doorbell 401 (for example, changing a zero value to a non-zero value). In block 420 of Figure 5, if the hardware scheduler 232 receives the interrupt 409 generated in block 418 of Figure 5, the hardware scheduler 232 identifies any bits in bit sequence 413 that have a non-zero value and their positions in bit sequence 413. The hardware scheduler 232 compares this information with information in the unmapped doorbell list 356 to determine the software queue 228 (or doorbell register 242) associated with the identified non-zero value in the bit sequence 413. If the bit sequence 413 is mapped to the doorbell register 242, the hardware scheduler 232 identifies the software queue 228 associated with the doorbell 501 based on information such as the software queue identifier, index value, and write pointer in the doorbell register 242 identified from the unmapped doorbell list 356 based on the bit sequence 413.

[0054] In other embodiments, multiple software queues 228 are aliased directly or indirectly through the doorbell register 242 to the same bits in the unmapped doorbell list 356. In other words, a subset of software queues 228 or doorbell registers 242 are mapped to the same bits in the unmapped doorbell list 356. For example, if the unmapped doorbell list 356 has 2 bits and there are 1000 software queues 228 (or doorbell registers 242), the first bit of the unmapped doorbell list 356 is mapped to software queues (or doorbell registers) 0-499, and the second bit of the unmapped doorbell list 356 is mapped to software queues (or doorbell registers) 500-999. In block 410 of Figure 4, if the unmapped queue unit 236 determines that the detected doorbell 401 is associated with an unmapped software queue 228, the unmapped queue unit 236 sets a bit in the unmapped doorbell list 356 of the software queue 228 (or doorbell register 242) associated with the doorbell 401 (for example, changing a zero value to a non-zero value). When the hardware scheduler 232 receives the interrupt 409 generated in block 418 of Figure 5, the hardware scheduler 232 reads the unmapped doorbell list 356 to identify the bits that have non-zero values. The bits that have non-zero values ​​indicate to the hardware scheduler 232 which subset of software queue 228 contains the software queue 228 associated with the doorbell 401, or which subset of doorbell register 242 contains the doorbell register 242 that generated the doorbell 401. In block 420 of Figure 5, the hardware scheduler 232 searches a subset of software queues 228 determined from the unmapped doorbell list 356 to identify any unmapped software queues 228 (e.g., command buffers 224) that have work to do.In embodiments where doorbell register 242 is mapped to an unmapped doorbell list 356, the hardware scheduler 232 thoroughly searches a subset of doorbell registers 242 determined from the unmapped doorbell list 356 to identify the doorbell register 242 that generated the doorbell 401. The hardware scheduler 232 then identifies the unmapped software queue 228 associated with doorbell register 242, as described above.

[0055] In other embodiments, instead of interrupt 409 functioning solely as a signal to trigger the hardware scheduler 232 to look at the unmapped doorbell list 356, interrupt 409 includes a bit sequence 413 that performs the aliasing described above. In these embodiments, when the hardware scheduler 232 receives the interrupt 409 generated in block 418 of Figure 5, the hardware scheduler 232 compares or indexes the bits with non-zero values ​​in the bit sequence 413 against the unmapped doorbell list 356 to identify a subset of the software queues 228 (or doorbell registers 242) mapped to bits.

[0056] In at least some embodiments, the unmapped doorbell list 356 includes multiple register arrays for recording doorbells 401 associated with an unmapped software queue 228. In these embodiments, in block 410 of Figure 4, when the unmapped queue unit 236 detects a doorbell 401 associated with an unmapped software queue 228, the unmapped software queue 228 records the doorbell 401 in one of the register arrays, for example, by changing a zero value to a non-zero value (e.g., 1). In block 418 of Figure 5, an interrupt 409 sent by the unmapped queue unit 236 informs the hardware scheduler 232 to look at multiple register arrays to identify at least one register array having a doorbell register 242 with a non-zero value. The hardware scheduler 232 determines the unmapped software queue 228 associated with the doorbell register 242 having a non-zero value in the register array and prioritizes this unmapped software queue 228 over other unmapped software queues 228 that have no work to do to map to the hardware queue 240. If multiple software queues 228 are associated with doorbell registers 242 having non-zero values ​​in a register array, the hardware scheduler 232 searches all of these unmapped software queues 228 to determine which of them has work to do. In at least some embodiments, an interrupt 409 sent by an unmapped queue unit 236 in block 418 of Figure 5 notifies the hardware scheduler 232 of a specific register array to look at that has a doorbell register 242 having a non-zero value.

[0057] Figure 6 shows one exemplary configuration 600 of multiple register arrays for recording or mapping doorbells 401 associated with an unmapped software queue 228. However, it should be understood that other configurations of the register arrays are also applicable. In this embodiment, the doorbell bar 244 has the memory address range [27:2], but other memory address ranges are also applicable. The memory address range [11:0]658 divides the doorbell bar 244 into 4K memory pages 660. The doorbell bar 244 is divided into four 512-bit register arrays 662 (indicated as doorbell array 662-1 to doorbell array 662-4). Each of the 512 bits in doorbell array 662 represents one or more doorbells. Thus, in this embodiment, all processes writing work to the APD 204 are assigned 512 doorbells. In at least some embodiments, each of the doorbell arrays 662 is indexed by doorbell address bits [13:3]. The two upper bits 664 (e.g., [13:12]) are used to index one of the doorbell arrays 662, and the remaining address bits (e.g., [11:3]) are used to index the 512 doorbells of the doorbell array 662. In at least some embodiments, each of the doorbell arrays 662 supports a set mask and a clear mask for updating each doorbell array 662 simultaneously, with set masks taking precedence over clear masks.

[0058] In one example, if an unmapped queue unit 236 in block 410 of Figure 4 detects a doorbell 401 associated with an unmapped software queue 228, the unmapped queue unit 236 uses the address bits [13:3] to decode the doorbell 401 and record the doorbell in one of the doorbell arrays 658. In at least some embodiments, the unmapped queue unit 236 uses the address bits [13:12] of the doorbell 401 to identify the doorbell array 658 mapped to that doorbell 401. The unmapped queue unit 236 then uses the address bits [11:3] of the doorbell 401 to index into the identified doorbell array 658 to identify one (or more) of the 512 bits to set for the doorbell 401. For example, an unmapped queue unit 236 records the doorbell 401 of an unmapped software queue 228 by changing the zero value of an identified bit to a non-zero value to indicate that the software queue 228 associated with the doorbell 401 has work (e.g., command buffer 224). In block 418 of Figure 5, the unmapped queue unit 236 sends an interrupt 409 to notify the hardware scheduler 232 that an unmapped software queue 228 has work. In block 420 of Figure 5, the hardware scheduler 232 scans multiple doorbell arrays 658 to determine which unmapped software queues 228 have work by identifying doorbell arrays 658 that contain non-zero values ​​for one or more of the 512 bits representing the doorbell 401 generated for the unmapped software queue 228.

[0059] In at least some embodiments, the APD204 implements a register that integrates the doorbell bits in the doorbell array 662 to enable the hardware scheduler 232 to quickly identify doorbell arrays 662 and segments within the array 662 that have non-zero values. In the embodiment described above with respect to Figure 6, a 32-bit register is implemented that integrates 2048 bits across four doorbell arrays 662, with each bit in the register corresponding to 64 bits in the entire array in an increasing manner. In at least some embodiments, an interrupt 409 generated by an unmapped queue unit 236 in block 418 of Figure 5 triggers the processor to read the integration register to identify doorbell arrays 662 and segments within the array 662 that have non-zero values ​​(e.g., unmapped queue doorbells). Also in at least some embodiments, the APD204 implements an index register provided to the hardware scheduler 232 for private read and write access. In the embodiments described above with respect to Figure 6, the hardware scheduler 232 can select one of 32 64-bit values ​​from the 2048 bits provided across the multiple doorbell arrays 662. In at least some embodiments, the hardware scheduler 232 clears one of the bits in the multiple doorbell arrays 662, for example, using a data mask (e.g., a 64-bit data mask) and an appropriate index.

[0060] It should be understood that separating the doorbells 401 of the unmapped software queue 228 into four sets of 512 doorbells, and setting bits for any ring on the unmapped doorbells, can be done in other ways. For example, in at least some embodiments, doorbells 401 arriving at the unmapped software queue 228 are inserted into a hardware FIFO and placed in backing memory to be read later by the hardware scheduler 232, and placed in a bitmap much larger than the 4x512 bitmap described above with respect to Figure 6 (up to and including generating one or more bits for each possible doorbell location in the doorbell bar 244). In at least some embodiments, the bitmap can be as small as a single bit for all software queues 228 in the processing system 200.

[0061] As disclosed herein, in some embodiments, the method is implemented in a processing device and includes signaling the hardware scheduler of the processing device that work has been placed in a queue that is not currently mapped to a hardware queue of the processing device, depending on whether the queue doorbell is an unmapped queue doorbell. In one embodiment, the method includes the hardware scheduler mapping the queue to one of a plurality of hardware queues in the processing device in response to the signaling. In another embodiment, the method includes determining that the queue doorbell is an unmapped queue doorbell in response to a hardware queue mapping list indicating that the queue is an unmapped queue doorbell, and passing the queue doorbell to the command processor of the processing device depending on whether the queue doorbell is a mapped queue doorbell. In yet another embodiment, the method includes updating a mapping data structure to indicate that the queue has work in response to an unmapped queue doorbell, the mapping data structure mapping each of the plurality of queues to one of a plurality of doorbell registers in the processing device.

[0062] In one embodiment, updating a mapping data structure involves changing the value of at least one bit in the mapping data structure that is mapped to any of the doorbell registers among a plurality of doorbell registers associated with an unmapped queue doorbell. In another embodiment, the method includes, in response to signaling, the hardware scheduler determining that a queue has work based on identifying at least one bit in the mapping data structure that has a non-zero value and determining that at least one bit is mapped to a queue. In yet another embodiment, the method includes, in response to signaling, the hardware scheduler determining that a queue has work based on processing a subset of queues in response to the fact that a subset of queues from a plurality of queues is mapped to at least one bit in the mapping data structure that has a non-zero value and determining that the queue has work based on having processed the subset of queues.

[0063] In some embodiments, the processing device includes a hardware scheduler and an unmapped queue unit, the unmapped queue unit being configured to transmit a signal to the hardware scheduler indicating that work has been placed in a queue that is not currently mapped to a hardware queue in the processing device, depending on whether the queue doorbell is an unmapped queue doorbell. In one embodiment, the hardware scheduler is configured to map the queue to one of a plurality of hardware queues in the processing device in response to the signal. In another embodiment, the processing device includes a plurality of compute units and a command processor, the command processor being configured to dispatch work to one or more of the plurality of compute units in response to the hardware scheduler mapping the queue to a hardware queue. In yet another embodiment, the unmapped queue doorbell is generated in response to a device driver placing work in the queue. In yet another embodiment, the unmapped queue doorbell is generated by writing to a doorbell register mapped to the queue.

[0064] In one embodiment, an unmapped queue unit is further configured to determine that a queue doorbell is an unmapped queue doorbell in response to a hardware queue mapping list indicating that the queue is an unmapped queue. In another embodiment, an unmapped queue unit is further configured to pass a queue doorbell to a command processor of a processing device in response to a mapped queue doorbell. In yet another embodiment, an unmapped queue unit is further configured to update a mapping data structure in response to an unmapped queue doorbell to indicate that the queue has work, the mapping data structure mapping each of a plurality of queues to one of a plurality of doorbell registers in a processing device.

[0065] In one embodiment, an unmapped queue unit is configured to update a mapping data structure by changing the value of at least one bit in the mapping data structure that is mapped to one of a plurality of doorbell registers associated with an unmapped queue doorbell. In another embodiment, a hardware scheduler is configured to determine if a queue has work based on, in response to a signal, identifying at least one bit in the mapping data structure that has a non-zero value, and determining that at least one bit is mapped to a queue. In yet another embodiment, a hardware scheduler is configured to determine if a queue has work based on, in response to a signal, processing a subset of queues in a plurality of queues in response to a signal that the subset of queues is mapped to at least one bit in the mapping data structure that has a non-zero value, and determining that the queue has work based on processing the subset of queues. In yet another embodiment, an unmapped queue unit is further configured to, in response to transmitting a signal to a hardware scheduler, pause the transmission of additional signals to the hardware scheduler until the hardware scheduler processes the transmitted signals, if an additional unmapped queue doorbell for the queue is detected.

[0066] In some embodiments, the processing device comprises a hardware scheduler, a mapping data structure comprising a plurality of register arrays, each of which register arrays includes a plurality of bits mapped to a different set of queue doors, and an unmapped queue unit, the unmapped queue unit is configured to detect an unmapped queue door associated with a queue that is not currently mapped to the processing device's hardware queue, and in response to the detected unmapped queue door, to update at least one bit mapped to the unmapped queue door in any of the plurality of register arrays, and to send a signal to the hardware scheduler indicating that the mapping data structure should be processed, the hardware scheduler is configured to determine, in response to the signal, that the queue has work based on the updated at least one bit, and to map the queue to the processing device's hardware queue.

[0067] In some embodiments, certain aspects of the technology described above are implemented by one or more processors of a processing system that executes the software. The software includes one or more sets of executable instructions, which are stored in a non-temporary computer-readable storage medium or otherwise clearly embodied. The software may also include instructions and specific data, which, when executed by one or more processors, operate the one or more processors to execute one or more aspects of the technology described above. Non-temporary computer-readable storage mediums may include, for example, magnetic or optical disk storage devices, solid-state storage devices such as flash memory, caches, random-access memory (RAM), or other non-volatile memory devices (one or more). Executable instructions stored in a non-temporary computer-readable storage medium can be implemented as source code, assembly language code, object code, or other instruction forms that can be interpreted or otherwise executed by one or more processors.

[0068] In addition to the foregoing, it should be noted that not all activities or elements described in the summary are required, and certain activities or parts of devices may not be required, and one or more additional activities may be performed, and one or more additional elements may be included. Furthermore, the order in which the activities are listed does not necessarily indicate the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, those skilled in the art will understand that various modifications and variations can be made without departing from the scope of the invention as described in the claims. Therefore, the specification and drawings should be considered illustrative rather than restrictive, and all of these variations are intended to fall within the scope of the invention.

[0069] Benefits, other advantages, and solutions to problems have been described above with respect to specific embodiments. However, benefits, advantages, solutions to problems, and features that may give rise to or manifest any benefits, advantages, or solutions are not to be construed as essential, necessary, or indispensable features to any or all of the claims. Furthermore, the disclosed invention can be modified and implemented in different but similar ways, in ways that are obvious to those skilled in the art who are interested in the teachings of this specification; therefore, the specific embodiments described above are merely illustrative. There are no limitations to the details of the configuration or design shown herein beyond those described in the appended claims. Accordingly, the specific embodiments described above may be modified or altered, and it is clear that all such modifications are within the scope of the disclosed invention. Accordingly, the protection sought herein is described in the appended claims.

Claims

1. A method performed in a processing device, Depending on whether the queue doorbell is an unmapped queue doorbell, the processing device's hardware scheduler signals that work has been placed in a queue that is not currently mapped to the processing device's hardware queue, method.

2. In response to the signaling, the hardware scheduler includes mapping the queue to one of a plurality of hardware queues of the processing device. The method according to claim 1.

3. In response to the hardware queue mapping list indicating that the queue is an unmapped queue, the queue doorbell is determined to be an unmapped queue doorbell. The process includes passing the queued abelt to the command processor of the processing device, depending on whether the queued abelt is a mapped queued abelt. The method according to claim 1.

4. The process includes updating the mapping data structure to indicate that the queue has work, depending on whether the queue doorbell is an unmapped queue doorbell. The mapping data structure maps each of the multiple queues to one of the multiple doorbell registers of the processing device. The method according to claim 1.

5. Updating the mapping data structure includes changing the value of at least one bit in the mapping data structure that is mapped to one of the doorbell registers among the multiple doorbell registers associated with the unmapped queue doorbell. The method according to claim 4.

6. In response to the signaling, the hardware scheduler, Identifying at least one bit having a non-zero value in the mapping data structure, Determining that at least one of the bits is mapped to the queue, Based on this, including determining that the queue has work, The method according to claim 4.

7. In response to the signaling, the hardware scheduler, Processing the subset of the plurality of queues in such a way that each subset of the plurality of queues is mapped to at least one bit having a non-zero value in the mapping data structure, In order to process a subset of the aforementioned queues, it is determined that the queue has work to do, Based on this, including determining that the queue has work, The method according to claim 4.

8. A processing device, Hardware scheduler and It includes an unmapped queue unit, The aforementioned unmapped queue unit is Depending on whether the queue doorbell is an unmapped queue doorbell, the system is configured to transmit a signal to the hardware scheduler indicating that work has been placed in a queue that is not currently mapped to the hardware queue of the processing device. Processing device.

9. The aforementioned hardware scheduler is: The queue is configured to map to one of the hardware queues among the processing device in response to the aforementioned signal. The processing device of claim 8.

10. Multiple computing units, A command processor is provided, The command processor is configured to dispatch the work to one or more of the plurality of computing units in response to the hardware scheduler mapping the queue to the hardware queue. The processing device according to claim 9.

11. The unmapped queue doorbell is generated in response to the device driver putting work into the queue. The processing device of claim 8.

12. The unmapped queue doorbell is generated by writing to the doorbell register mapped to the queue. The processing device according to claim 11.

13. The aforementioned unmapped queue unit is The queue doorbell is configured to determine that it is an unmapped queue doorbell in response to the hardware queue mapping list indicating that the queue is an unmapped queue. The processing device of claim 8.

14. The aforementioned unmapped queue unit is Depending on whether the queued abel is a mapped queued abel, the system is configured to pass the queued abel to the command processor of the processing device. The processing device of claim 8.

15. The aforementioned unmapped queue unit is Depending on whether the queue doorbell is an unmapped queue doorbell, the mapping data structure is configured to update to indicate that the queue has work. The mapping data structure maps each of the multiple queues to one of the multiple doorbell registers of the processing device. The processing device of claim 8.

16. The unmapped queue unit is configured to update the mapping data structure by changing the value of at least one bit in the mapping data structure that is mapped to one of the multiple doorbell registers associated with the unmapped queue doorbell. The processing device of claim 15.

17. The aforementioned hardware scheduler is: In accordance with the aforementioned signal, Identifying at least one bit having a non-zero value in the mapping data structure, Determining that at least one of the bits is mapped to the queue, The system is configured to determine that the queue has work to do based on the following: The processing device of claim 15.

18. The aforementioned hardware scheduler is: In accordance with the aforementioned signal, Processing a subset of the plurality of queues in such a way that each subset of the plurality of queues is mapped to at least one bit having a non-zero value in the mapping data structure, In order to process a subset of the aforementioned queues, it is determined that the queue has work to do, The system is configured to determine that the queue has work to do based on the following: The processing device of claim 15.

19. The aforementioned unmapped queue unit is In response to the transmission of the signal to the hardware scheduler, if an additional unmapped queue doorbell is detected in the queue, the transmission of the additional signal to the hardware scheduler is suspended until the hardware scheduler processes the transmitted signal. The processing device of claim 15.

20. A processing device, Hardware scheduler and A mapping data structure comprising multiple register arrays, each of which includes multiple bits mapped to different sets of cued abels, It includes an unmapped queue unit, The aforementioned unmapped queue unit is To detect unmapped queue doorbells associated with queues that are not currently mapped to the hardware queues of the processing device, In response to detecting the unmapped cued abel, update at least one bit mapped to the unmapped cued abel in one of the register arrays of the plurality of register arrays, Sending a signal to the hardware scheduler indicating that the mapping data structure should be processed, It is configured to do the following: The aforementioned hardware scheduler is: In response to the aforementioned signal, it is determined that the queue has work to do based on at least one updated bit, Mapping the aforementioned queue to the hardware queue of the processing device, It is configured to do, Processing device.

Images