Processing system with selective priority-based two-level binning

Abstract

A system and method are provided for priority-based and performance-based selection of rendering modes, such as a two-level binning mode, for executing workloads with a graphics processing unit (GPU) [102] of a system [100]. A user mode driver (UMD) [110] or kernel mode driver (KMD) [112] executing on a central processing unit (CPU) [104] configures low and medium priority workloads to be executed in the two-level binning mode and selects the binning mode for a high priority workload based on whether performance heuristics indicate that one or more binning or override conditions are met. Execution of the low and medium priority workloads on the GPU is maintained in a low / medium priority queue while the high priority workload is maintained in a high priority queue so that execution of the high priority workload may be preempted in favor of the low priority workload.

Description

[Background technology]

[0001] Computer processing systems generally use graphics processing units (GPUs) to perform graphics operations such as texture mapping, rendering, and vertex transformations. Performance requirements or specifications for a GPU can vary depending on the type of associated electronic device. For example, GPUs used in mobile devices or other battery-powered devices have characteristics and requirements that can differ significantly from other non-battery-powered platforms. Performance, battery life, and heat are generally important criteria for battery-powered device platforms, with sustained performance and low idle power consumption and temperature being desirable. However, there is generally a trade-off between GPU performance and battery life / heat in battery-powered devices.

[0002] The present disclosure may be better understood, and its numerous features and advantages made apparent to those skilled in the art by referencing the accompanying drawings, in which: The use of the same reference numbers in different drawings indicates similar or identical items. [Brief explanation of the drawings]

[0003] [Figure 1] 1 is a block diagram of an example device for setting a binning mode for a GPU workload based on GPU performance data, according to some embodiments. [Figure 2] FIG. 1 is a flow diagram illustrating a method for setting a binning mode for a GPU workload based on performance heuristics, according to some embodiments. [Figure 3] FIG. 10 is a flow diagram illustrating a method for selectively patching pending workloads before submitting commands to a GPU based on a determined binning mode, according to some embodiments. [Figure 4] FIG. 1 is a flow diagram illustrating a method for selectively executing a workload in a two-level binning mode or a non-two-level binning mode, according to some embodiments. [Figure 5] FIG. 1 is a block diagram of high-priority and low / medium-priority queues for priority-based organization and selection of workloads to be executed by a GPU, according to some embodiments. [Figure 6] FIG. 1 is a flow diagram illustrating a method for selectively executing workloads in a two-level binning mode or a non-two-level binning mode based on performance heuristics and workload priority, according to some embodiments. [Figure 7] FIG. 10 is a flow diagram illustrating a method for preempting the execution of a lower priority workload in a two-level binning mode in order to execute a higher priority workload, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0004] Using the techniques described herein, a GPU of a processing system selects a primitive binning mode (sometimes referred to herein as a “binning mode”) for executing one or more workloads based on the performance characteristics of the processing system and based on the respective priorities of those workloads. When rendering an image frame, the binning mode defines the technique by which the image frame is divided into regions and primitives are assigned to bins corresponding to each region. By selecting a binning mode based on performance characteristics, the processing system adapts the rendering process in response to system operating conditions. According to various examples, the processing system improves user experience by selecting a less hardware-intensive binning mode, such as a two-level binning mode, to render an image frame in response to detecting conditions such as low remaining battery life or high processing system temperature to reduce processing system power consumption or to lower processing system temperature.

[0005] To illustrate, to render a frame (i.e., to prepare the frame for transfer to and display on a display device such as a display panel), a GPU converts information about three-dimensional (3D) objects into a displayable two-dimensional (2D) image frame. This process, at least in some cases, requires significant processing power and memory resources. During the rendering process, pixels in the image frame are generated by rendering graphical objects to determine color values ​​for each pixel. Exemplary graphical objects include points, lines, polygons, and higher-order surfaces in three dimensions (3D). Points, lines, and polygons represent rendering primitives that are the basis of most 3D rendering instructions. More complex structures, such as 3D objects, are formed from combinations or meshes of such primitives. To display a particular scene using conventional rendering techniques, a GPU renders primitives with potential contributing pixels associated with the scene individually for each primitive by determining which pixels fall within the edge of each primitive and obtaining the primitive's attributes corresponding to each of those pixels.

[0006] In other cases, the GPU renders primitives using a binning process, in which the GPU divides an image frame into regions, identifies primitives that intersect with the predetermined region, and places the identified primitives into bins that correspond to the predetermined region. Thus, each region of the frame is associated with a corresponding bin, and the bin contains primitives or portions of primitives that intersect with the associated bin. The GPU renders the frame on a bin-by-bin basis by rendering pixels of primitives that intersect with the region of the frame that corresponds to the bin. This allows the GPU to render frames more efficiently, at least in some cases, by requiring fewer memory accesses, increasing cache usage, etc.

[0007] One example of a binning process is primitive batch binning (PBB), in which a GPU receives a sequence of primitives and advantageously segments the primitives into temporally related primitive batches. Sequential primitives are captured until a predetermined condition, such as a batch full condition, a state storage full condition, or a dependency on a previously rendered primitive is determined. When performing PBB, an image frame displaying rendered primitives is divided into several blocks. Each block of the image frame is associated with a respective bin. Each primitive in the received sequence of primitives in the batch intersects one or more bins. For each received primitive in the batch, an initial bin intercept is calculated, which is the top-leftmost bin of the image frame that the primitive intersects. After the batch is closed, a first bin for processing is identified. Primitives that intercept the identified bin are processed. For each identified primitive that intercepts a bin, the next bin intercept is identified and pixels contained in the primitives enclosed by the identified bin are sent for detailed rasterization. The next bin intercept is the next top-left most bin in raster order that is intersected by the processed primitive.

[0008] In some embodiments, the GPU implements different binning techniques, referred to herein as binning modes or primitive binning modes, where different binning modes use different binning characteristics, such as different region sizes for each bin, different numbers of binning levels, etc., or any combination thereof. For example, in some embodiments, the GPU includes both a single-level binning mode and a two-level binning mode. In single-level binning mode, also referred to as primitive batch binning (PBB) mode, the GPU divides an image frame into a specified number of regions and renders each region as described above.

[0009] In the two-level binning mode, two types of binning are performed: coarse level binning and fine level binning. In some embodiments, coarse level binning uses large bins (e.g., a total of 32 bins to cover the entire display area), which reduces binning overhead. Visibility information for each coarse bin is generated during the rendering of the first coarse bin (i.e., coarse bin 0) and used to render the other coarse bins. After coarse level binning, fine level binning is performed sequentially for each coarse bin. In some embodiments, fine level binning involves dividing each coarse bin into smaller "fine" bins, such as by performing primitive batch binning (PBB) to further bin each coarse bin into a 64x64 array of fine bins during PBB-based fine level binning. Each fine bin is then rendered using rendering information, such as primitive visibility information, generated for the corresponding coarse bin. In some embodiments, two-level binning occurs at the top of the graphics processing pipeline (e.g., before vertex processing and rasterization), as opposed to a single-level PBB-only binning mode that occurs in the middle of the graphics processing pipeline (e.g., after vertex processing and before pixel shading).

[0010] In some cases, different binning modes are suitable for different device conditions. For example, under some conditions, a single-level or PBB binning mode (PBB alone is used without the combination of coarse and fine level binning described above) provides better performance than two-level binning, but at the expense of increased device power consumption and higher operating temperature. In contrast, in some cases, two-level binning supports reduced power consumption at the expense of some performance.

[0011] To adapt the binning mode according to device conditions, in some embodiments, the GPU employs a selective two-level binning architecture that supports runtime selection of rendering modes. For example, in some embodiments, a device implementing the selective two-level binning architecture implements runtime selection between a two-level binning mode and a default rendering mode, such as a PBB rendering mode in which only PBB is executed. The selection of the binning mode is based on any of several performance criteria, such as thermal characteristics, power characteristics (e.g., battery life), etc. For example, in some embodiments, a driver, such as a user-mode driver (UMD) or a kernel-mode driver (KMD), receives performance data, such as sensor data and performance counter data, and selects the binning mode based on the performance data.

[0012] Additionally, in some embodiments, the GPU is configured to select a binning mode for a given workload based on the priority of that workload. Thus, the GPU can execute all applicable low- or medium-priority workloads in two-level binning mode while executing high-priority workloads in either two-level binning mode or non-two-level binning mode based on the performance characteristics of the GPU or the system. By executing all applicable low- and medium-priority workloads in two-level binning mode (i.e., not based on performance heuristics), the granularity of GPU preemption is improved. For example, application programming interfaces (APIs) typically do not provide a means to divide a workload into smaller units than full draw calls. In contrast, as described herein, two-level binning divides a frame being rendered into multiple coarse bins, where each coarse bin is rendered separately, thereby decomposing draw calls that span multiple coarse bins of a frame into smaller chunks. It is sometimes necessary or desirable for a GPU to preempt a low or medium priority workload being executed by the GPU so that a high priority workload can be executed instead. This preemption of execution of a lower priority workload in favor of a higher priority workload, sometimes referred to herein as "draw call preemption," helps to ensure that the execution of a more time-critical workload is not unnecessarily delayed by the execution of a less time-critical workload.

[0013] The techniques described herein involve selectively executing workloads according to a current two-level binning mode or a non-two-level binning mode, such as a PBB rendering mode, based on one or both of performance heuristics and workload priority.

[0014] 1 illustrates an example device 100 that includes parallel processors, specifically a GPU 102, and that implements a two-level binning architecture that enables runtime selection of a rendering mode for rendering image data. In addition to the GPU 102, the device 100 includes a CPU 104, a memory controller 105, a system memory 106, sensors 108, and a battery 111. In some embodiments, the GPU 102, the CPU 104, the memory controller 105, and the sensors 108 are communicatively coupled to each other via a bus 126. The memory controller 105 manages memory access requests provided by the GPU 102, the CPU 104, and the sensors 108 to access the system memory 106.

[0015] During graphics processing operations, applications in system memory 106 generate commands to instruct GPU 102 to render image data at defined locations within system memory 106 for subsequent display in image frames on device 100's electronic display (not shown). Commands output by the applications are recorded in one or more command buffers 114 by UMD 110 executing on CPU 104. A given command buffer 114 contains commands corresponding to one or more workloads, each workload configured to execute in two-level binning mode, non-two-level binning mode, or capable of executing in either mode. Once UMD 110 has completed recording the commands in command buffer 114, KMD 112 submits command buffer 114 to GPU 102, where the commands in command buffer 114 are loaded into one or more ring buffers 120 of GPU 102. Here, ring buffers such as ring buffer 120 are sometimes referred to as queues. In one example, a first one of ring buffers 120 corresponds to a high-priority queue and receives high-priority workloads for execution by GPU 102, while a second one of ring buffers 120 corresponds to a low / medium-priority queue and receives low-priority and medium-priority workloads for execution by GPU 102. One or more command processors 122 of GPU 102 retrieve commands corresponding to particular command buffers 114 from ring buffer 120 and execute those commands, for example, by retrieving image data from system memory 106 and instructing shaders, compute units, and other graphics processing circuitry (not shown) to render the retrieved image data. As used herein, a "workload" refers to a command or group of commands processed by a GPU, such as GPU 102.According to various embodiments, a workload typically includes a set of work (e.g., graphics drawing) for a given set of render targets, depth stencil targets, or buffers (e.g., unordered access views (UAVs)). In the example of FIG. 1, GPU 102 selects between a two-level binning mode and a single-level binning mode, such as PBB mode (sometimes referred to herein as a “single-level PBB mode” or a “single-level PBB only binning mode”), when executing the workload for command buffer 114. In some embodiments, GPU 102 selects which binning mode to use to execute the workload for command buffer 114 based on one or more status bits stored in system memory 106 or based on one or more patch valid bits (described below) stored in GPU memory 124. In some embodiments, GPU 102 selects the binning mode based on the status bits or patch valid bits when executing corresponding logic in command buffer 114 that causes GPU 102 to check the status bits or patch valid bits to determine which binning mode to use for execution of the corresponding workload in command buffer 114. In some embodiments, CPU 104 selectively sets the values ​​of the status bits and patch valid bits based on identified performance characteristics (sometimes referred to herein as “performance data” or “performance heuristics”) of either or both of device 100 and GPU 102. In some embodiments, each workload stored in command buffer 114 is indicated as being high priority, medium priority, or low priority (e.g., by a priority indicator such as one included in the workload, stored with the workload, or generated by CPU 104 based on a function of the workload).In some embodiments, CPU 104 selectively sets the values ​​of the status bits and patch valid bits for high-priority workloads executing in either two-level binning mode or non-two-level binning mode based on the identified performance characteristics, and CPU 104 configures all applicable low-priority and medium-priority workloads to execute in two-level binning mode (e.g., to improve preemption granularity for low-priority and medium-priority workloads). Note that some workloads, even if identified as low- or medium-priority workloads, may not be applicable for execution in two-level binning mode. For example, if a given workload uses API features that are incompatible with two-level binning, such as the use of tessellated draw calls, atomics in shaders, buffer read-modify-write in shaders, stream-out, etc., the workload is not applicable for execution in two-level binning mode. In some embodiments, high-priority workloads are stored in a different queue than low-priority and medium-priority workloads.

[0016] In some embodiments, a given command buffer 114 is configured so that some workloads stored in the command buffer 114 are executed in one-level binning mode, and other workloads in the same command buffer 114 are executed in two-level binning mode. For example, some workloads are only executable using one-level binning and are therefore configured to always execute in one-level binning mode, even if they are low-priority or medium-priority workloads, or even if they are high-priority workloads for which two-level binning conditions are met and two-level binning mode is enabled. In some embodiments, workloads (e.g., high-priority workloads) are recorded in the command buffer 114 by the UMD 110 so as to be executable in either one-level binning mode or two-level binning mode, and the binning mode in which these workloads are executed is then selected by the GPU when the workloads are executed.

[0017] Generally, drivers within an operating system run in user mode, kernel mode, or both. UMDs, such as UMD 110, run in non-privileged processor mode where other application code, including protected subsystem code, executes. UMDs cannot gain access to system data or hardware except by calling APIs that invoke system services. KMDs, such as KMD 112, run as part of the operating system and support one or more protected subsystems. UMDs and KMDs have different structures, entry points, and system interfaces. KMDs can perform certain protected operations and access system structures that UMDs cannot access. In one example, draw call commands corresponding to primitives generated by an application are recorded by UMD 110 in one or more command buffers 114, and KMD 112 submits the draw call commands to GPU 102 for subsequent rendering of the primitives. Command processor 122 causes image data to be rendered according to a particular rendering mode, such as a two-level binning mode or a non-two-level binning mode, such as a PBB rendering mode. In some embodiments, command processor 122 executes low-priority and medium-priority workloads in two-level binning mode and executes high-priority workloads based on performance heuristics. In some embodiments, command processor 122 selects which rendering mode to use to render image data associated with a particular high-priority workload by determining whether two-level binning mode is enabled or disabled. In some embodiments, command processor 122 determines whether two-level binning mode is enabled or disabled by checking one or more status bits stored in GPU memory 124 or system memory 106.

[0018] In some embodiments, CPU 104 enables or disables the two-level binning mode (e.g., for high priority workloads) based on performance data including performance counter data received from performance counters 116 stored in system memory 106, sensor data 118 stored in system memory 106 by sensors 108, or both. In some embodiments, UMD 110 or KMD 112 of CPU 104 receives the performance data and processes the performance data to determine whether to enable or disable the two-level binning mode (e.g., for high priority workloads).

[0019] In some embodiments, the sensor data 118 generated by the sensor 108 includes one or more temperature measurements, voltage measurements, current measurements, instantaneous power measurements, peak power measurements, or other applicable sensor data. In some embodiments, the sensor 108 includes one or more temperature sensors, current sensors, voltage sensors, or power sensors.

[0020] In some embodiments, performance counters 116 track activity in various modules of the device, such as battery 111, CPU 104, ring buffer 120, level 1 (L1) cache, level 2 (L2) cache, or shaders of GPU 102. In some embodiments, performance counter data includes respective quantities of one or more of cache accesses, cache hit rates, cache miss rates, memory accesses, GPU 102 utilization, CPU 104 utilization, current supplied to GPU 102, current supplied to CPU 104, voltage at GPU 102, voltage at CPU 104, frequency of GPU 102, and / or frequency of CPU 104.

[0021] In some embodiments, the performance data includes one or more parameters derived from sensor data 118 or performance counter data generated by performance counter 116, such as the average temperature of device 100, the rate of change (RoC) of the average temperature of device 100, the peak instantaneous power consumption of device 100 over a predetermined period of time, the average power consumption of device 100 over a predetermined period of time, the RoC of the average power consumption of device 100, or the state of charge (SoC) of battery 111 (i.e., the remaining charge of battery 111 expressed as a percentage of the charge capacity of battery 111). As used herein, according to various embodiments, the “average temperature” of device 100 refers to the mean, median, or mode of instantaneous temperatures measured at various locations on the device (e.g., CPU 104, GPU 102, battery 111, or a combination thereof), the mean, median, or mode of temperatures measured at various locations on the device over a defined period of time, or the mean, median, or mode of an estimated temperature of device 100 derived from estimated power consumption based on performance counter data generated by performance counters 116 over a defined period of time. As used herein, the “average power consumption” of device 100 refers, according to various embodiments, to the mean, median, or mode of instantaneous power consumption measured at battery 111 over a defined period of time, or the mean, median, or mode of estimated instantaneous power consumption based on performance counter data generated by performance counters 116 over a defined period of time.

[0022] The UMD 110 or KMD 112 monitors the performance data to determine whether one or more predefined conditions for enabling two-level binning mode (e.g., for high-priority workloads) have occurred, sometimes referred to herein as “two-level binning conditions.” In some embodiments, enabling or disabling two-level binning mode involves the UMD 110 or KMD 112 setting the value of one or more status bits in the system memory 106 or GPU 102 that indicate whether two-level binning mode is enabled (e.g., for high-priority workloads). In some embodiments, the two-level binning conditions include one or more of: an average temperature of a device exceeding a predefined temperature threshold; an RoC of an average temperature of a device exceeding a predefined RoC threshold; a local temperature at a defined location of a device exceeding a predefined temperature threshold; an RoC of such local temperature exceeding a predefined RoC threshold; a peak instantaneous power consumption of a device exceeding a predefined threshold; an average power consumption of a device exceeding a predefined threshold; an RoC of an average power consumption of a device exceeding a predefined threshold; a battery SoC below a predefined SoC threshold; or a combination of these conditions. It should be understood that in some embodiments, if a two-level binning condition is met and two-level binning mode is enabled by UMD 110 or KMD 112, and then it is subsequently determined that the two-level binning condition is no longer met based on changes in performance data, the device disables the two-level binning mode (e.g., for high-priority workloads). However, in some embodiments, other detectable conditions, sometimes referred to herein as “override conditions,” override the detection of the two-level binning condition. For example, if device 100 is determined by UMD 110 or KMD 112 to meet the two-level binning condition but is determined to meet the override condition of being plugged in (e.g., if the battery is determined to be in a “charging” state), the two-level binning mode is disabled (e.g., for high-priority workloads).In some embodiments, alternative or additional override conditions are set, such as determining that the average power consumption of the device 100 has fallen below a threshold, or determining that the GPU 102 or CPU 104 is no longer thermally throttled (e.g., this can be determined based on the clock frequency of the GPU 102 or CPU 104 exceeding a threshold).

[0023] In some embodiments, when recording workloads in the command buffer 114, the UMD 110 records a given workload (e.g., a high priority workload) differently in the command buffer 114 depending on whether the two-level binning mode is enabled or disabled based on a corresponding status bit stored in the system memory 106. In some embodiments, upon enabling the two-level binning mode, the UMD 110 records all subsequent high priority workloads in the command buffer 114 so that they can be executed according to the two-level binning mode, at least until the two-level binning mode is disabled again. In some embodiments, upon disabling the two-level binning mode, the UMD 110 records all subsequent high priority workloads in the command buffer 114 so that they can be executed according to a single-level binning mode, such as a non-two-level or single-level PBB mode. In some embodiments, the UMD 110 or the KMD 112 individually determines the binning mode for each high priority workload stored in a given command buffer 114 based on whether two-level binning mode is enabled when the high priority workload is recorded by the UMD 110, and in some cases, based on whether the given high priority workload can be executed in two-level binning mode.

[0024] For example, in some embodiments, UMD 110 is configured to store high priority workloads in command buffer 114 in one-level binning mode by default, and if two-level binning conditions are met, modify one or more pending high priority workloads in command buffer 114 to run in two-level binning mode before submission to GPU 102. In other embodiments, UMD 110 is configured to store all applicable workloads in command buffer 114 in two-level binning mode by default, and if two-level binning conditions are not met, modify one or more pending high priority workloads in command buffer 114 to run in one-level binning mode before submission to GPU 102.

[0025] In some cases, the state of the two-level binning mode (i.e., enabled / disabled) changes after the UMD 110 has already recorded or has begun recording one or more high-priority workloads in the command buffer 114, referred to in such cases as “pending workloads,” but before the pending workloads are executed by the GPU 102. In some embodiments, such pending high-priority workloads are modified via command buffer patching to be executed according to a two-level binning mode or a non-two-level binning mode, depending on whether the two-level binning mode is enabled. As used herein, command buffer patching refers to modification of data (e.g., workloads, commands) in the command buffer by a driver or other module executed by the CPU 104 or the GPU 102, and is performed in either the CPU 104 or the GPU 102.

[0026] In one example, a pending high priority workload in command buffer determination 114 that was recorded while two-level binning mode was enabled is modified by CPU 104 or GPU 102 via command buffer patching to execute according to a non-two-level binning mode in response to a determination by CPU 104 or GPU 102 that two-level binning mode has been disabled since the start of recording of the pending high priority workload. As another example, a pending high priority workload in command buffer 114 that was recorded while two-level binning mode was disabled is modified by CPU 104 or GPU 102 via command buffer patching to execute according to two-level binning mode in response to a determination by CPU 104 or GPU 102 that two-level binning mode has been enabled since the start of recording of the pending high priority workload. In general, once GPU 102 has begun executing at least a portion of a given workload, the workload cannot be patched to change its binning mode.

[0027] In some embodiments where command buffer patching is performed on CPU 104, UMD 110 or KMD 112 performs command buffer patching near the end of the command buffer recording process. In some embodiments, when command buffer patching is performed on CPU 104, UMD performs command buffer patching immediately before submitting the patched high priority workload from command buffer 114 to GPU 102, unless the pending high priority workload in command buffer 114 is configured to run more than once simultaneously (a predefined condition that is known at the time of recording the workload to command buffer 114).

[0028] In some embodiments involving CPU-side command buffer patching, the UMD 110 stores metadata for each high-priority workload. In some embodiments, the metadata stored for each high-priority workload includes one or more tokens and one or more offsets. Each offset defines a location in the command buffer 114 that needs to be modified when two-level binning mode is enabled. Each token defines how code stored in the command buffer 114 at the location defined in the corresponding offset should be modified when two-level binning mode is enabled. In one example, the metadata tokens cause the UMD 110 to modify code in the command buffer 114 that describes the visibility of primitives. In some embodiments, command buffer patching is required only when two-level binning mode is enabled, and UMD 110 initially (i.e., by default) records each high-priority workload in command buffer 114 for execution in non-two-level binning mode in such embodiments before making a determination whether to patch command buffer 114 to enable execution in two-level binning mode at the end of the recording process or immediately prior to submitting the workload stored in command buffer 114 to GPU 102. As described above, in some embodiments, UMD 110 alternatively records each workload in command buffer determination 114 to execute in two-level binning mode, for at least those workloads that can execute in two-level binning mode, and then determines whether to modify one or more of those high-priority workloads to instead execute in non-two-level binning mode based on whether two-level binning mode is enabled and, in some cases, whether predefined override conditions are met.

[0029] In some embodiments, command buffer patching is performed in GPU 102 based on a value or group of values, referred to herein as a “patch effect value,” stored in a unified memory shared between GPU 102 and CPU 104, such as GPU memory 124 or system memory 106. In some embodiments, each patch effect value is a single Boolean value corresponding to a high-priority workload in a respective pending command buffer 114. In some embodiments, KMD 112 determines whether two-level binning mode is enabled based on a corresponding status bit stored in system memory 106 or based on analysis of performance data, and then KMD 112 causes the patch effect value to be set according to whether two-level binning mode is enabled prior to execution of command buffer 114 by GPU 102. In some embodiments where GPU-side patching is performed, the UMD 110 must record the high-priority workload in the command buffer 114 so that it can be executed in both two-level binning mode and non-two-level binning mode, and the command processor 122 determines in which mode to execute the high-priority workload based on the corresponding patch validity value.

[0030] In some embodiments, the patch effect values ​​are instead command buffer-based patch effect values ​​stored in the command buffer 114 by the UMD 110 during recording. In one example, when the GPU 102 executes a given high priority workload, it checks one or more patch effect values ​​for that workload, which patch effect values ​​are stored in the command buffer 114 along with the high priority workload. In such an embodiment, the GPU 102 modifies one or more high priority workloads stored in the command buffer 114 to be executable in a two-level binning mode or a non-two-level binning mode when executed by the GPU 102 based on the command buffer-based patch effect values. In one example, a command processor of the GPU 102 or a shader core of the GPU 102 modifies the high priority workload to execute in either a two-level binning mode or a non-two-level binning mode based on a determination of whether GPU-side patching is required. In some embodiments, the need for GPU-side patching is determined based on a patch effect bit stored in the GPU memory 124 or a status bit stored in the system memory 106, as described above.

[0031] 2 illustrates an example process flow for a method 200 for selectively executing workloads (e.g., high priority workloads) in one or more command buffers in a first binning mode or a second binning mode based on performance data generated by performance counters or sensors. In some embodiments, the first binning mode is a two-level binning mode and the second binning mode is a single-level binning mode, such as a PBB mode. Method 200 will be described with reference to an example implementation in device 100 of FIG. 1 and its constituent components and modules.

[0032] In block 202, the UMD 110 or the KMD 112 (e.g., being executed by the CPU 104) retrieves or otherwise receives performance data. In some embodiments, the performance data includes sensor data 118 generated by the sensors 108. In some embodiments, the data includes performance counter data generated by the performance counters 116. In some embodiments, the performance data includes both performance counter data and sensor data 118. In some embodiments, the sensor data 118 generated by the sensors 108 includes one or more temperature measurements, voltage measurements, current measurements, instantaneous power measurements, peak power measurements, or other applicable sensor data. In some embodiments, the performance counter data includes one or more respective quantities of cache accesses, cache hit rates, cache miss rates, memory accesses, GPU 102 utilization, CPU 104 utilization, current supplied to GPU 102, current supplied to CPU 104, voltage at GPU 102, voltage at CPU 104, frequency of GPU 102, and / or frequency of CPU 104, each corresponding to activity occurring in one or more modules of device 100, such as battery 111, CPU 104, ring buffer 120, level 1 (L1) cache, level 2 (L2) cache, or shaders of GPU 102. In some embodiments, the performance data includes one or more parameters derived from sensor data 118 or performance counter data generated by performance counters 116, such as an average temperature of the device, a rate of change (RoC) of the average temperature of the device, a peak instantaneous power consumption of the device during a predetermined period of time, an average power consumption of the device over a predetermined period of time, an RoC of the average power consumption of the device, or a state of charge (SoC) of the battery (i.e., the remaining charge of the battery, in some embodiments expressed as a percentage of the battery's charge capacity), etc. In some embodiments, the performance characteristics for a given workload include a workload characteristic for that workload that indicates whether the workload is applicable to execution in two-level binning mode.For example, such workload characteristics include tessellated draw calls, atomics in shaders, read-modify-write buffers in shaders, or use of stream-out in the workload. In some embodiments, the derived parameters are calculated by the UMD 110 or the KMD 112.

[0033] At block 204, the UMD 110 or KMD 112 determines whether a binning condition (e.g., a two-level binning condition) is met based on the performance data. For example, in some embodiments, the binning condition includes one or more two-level binning conditions including one or more of: an average temperature of the device above a predefined temperature threshold; an RoC of the average temperature of the device above a predefined RoC threshold; a local temperature at a defined location of the device above a predefined temperature threshold; an RoC of such local temperature above a predefined RoC threshold; a peak instantaneous power consumption of the device above a predefined threshold; an average power consumption of the device above a predefined threshold; an RoC of the average power consumption of the device above a predefined threshold; a battery SoC below a predefined SoC threshold; or a combination of these conditions. If the UMD 110 or KMD 112 determines that the binning condition is met, method 200 proceeds to block 206. Otherwise, if the UMD 110 or KMD 112 determines that the binning condition is not met, method 200 proceeds to block 214.

[0034] At block 206, the UMD 110 or KMD 112 determines whether an override condition is satisfied based on the performance data. In some embodiments, the override condition includes one or more of: the device 100 entering a charging condition in which the battery 111 is charging; the average temperature of the device 100 falling below a predetermined threshold; the RoC of the average temperature of the device 100 falling below a predetermined threshold; or a combination thereof. In some embodiments, the one or more override conditions are determined based on workload characteristics for the given workload that indicate whether the workload is applicable for execution in two-level binning mode. In one example, the override condition is triggered in response to determining, based on the workload characteristics, that the given workload includes the use of one or more of tessellated draw calls, atomics in shaders, read-modify-write buffers in shaders, or stream-out in the workload. If the UMD 110 or KMD 112 determines that the override condition is not satisfied, the method 200 proceeds to block 208. Otherwise, if the UMD 110 or KMD 112 determines that the override condition is met, the method 200 proceeds to block 214 .

[0035] At block 208, the UMD 110 or KMD 112 enables a first binning mode for the newly created high-priority workload stored in the command buffer 114. In some embodiments, the first binning mode is a two-level binning mode. For example, to enable the first binning mode, the UMD 110 or KMD 112 sets a status bit value in the system memory 106 to indicate that the first binning mode is enabled for the high-priority workload. In some embodiments, when recording a subsequent high-priority workload to the command buffer 114, the UMD 110 checks the status bit value and determines that the high-priority workload should be configured to run in the first binning mode. In some embodiments, the first binning mode is enabled for low-priority or medium-priority workloads by default (e.g., automatically, but not in response to a binning condition being met and an override condition not being met).

[0036] In block 210, CPU 104 or GPU 102 patches pending high priority workloads in command buffer 114 to enable execution of those high priority workloads in the first binning mode. In some embodiments, UMD 110 is configured to patch such high priority workloads via CPU 104 when recorded in command buffer 114, such that the patched workloads execute in the first binning mode when recorded. In some embodiments, UMD 110 patches the pending high priority workloads via CPU 104 for execution in the first binning mode after the pending high priority workloads have already been recorded in command buffer 114 and before (e.g., immediately before) the pending high priority workloads are provided from command buffer 114 to GPU 102. In some embodiments, GPU 102 patches the pending high priority workloads for execution in the first binning mode before (e.g., immediately before) execution of the pending high priority workloads based on one or more patch enable values ​​stored in GPU memory 124.

[0037] At block 212, the high priority workload is executed in the first binning mode on the GPU 102 (e.g., unless one or more binning conditions are met and an override condition is met). In some embodiments, execution of such a high priority workload preempts execution of a lower priority workload.

[0038] In block 214, the UMD 110 or KMD 112 disables the first binning mode for the newly created high-priority workload stored in the command buffer 114. For example, to disable the first binning mode, the UMD 110 or KMD 112 sets a status bit value in the system memory 106 to indicate that the first binning mode is disabled for the high-priority workload. In some embodiments, when recording the high-priority workload in the command buffer 114 in response to detecting that the binning condition is not met in block 204, the UMD 110 checks the status bit value and, if applicable, determines that the high-priority workload should be configured to run in a second binning mode. In some embodiments, the second binning mode is a single-level binning mode, such as a PBB mode. In some embodiments, the system leaves the first binning mode enabled for low-priority and medium-priority workloads.

[0039] In block 216, the UMD 110 or KMD 112 disables the first binning mode for pending high priority workloads already stored in the command buffer 114. In embodiments in which the UMD 110 records the high priority workloads in the command buffer 114 to run in the second binning mode by default, block 216 is skipped because no further action is required to disable the first binning mode for the high priority workloads other than changing the status bit in block 214. In some embodiments in which GPU-side patching is performed, the KMD 112 disables the first binning mode for pending high priority workloads in the command buffer 114 by setting one or more patch enable values ​​in the GPU memory 124 to indicate that the first binning mode has been disabled.

[0040] At block 218, the high priority workload is executed in the second binning mode on the GPU 102 (e.g., because the binning condition is not met or because at least one override condition is met). In some embodiments, execution of such a high priority workload preempts execution of a lower priority workload.

[0041] 3 illustrates an example process flow for a method 300 for selectively patching a command buffer in a CPU to enable execution in a two-level binning mode or a non-two-level binning mode. Method 300 is described with reference to an example implementation in device 100 of FIG. 1 and its constituent components and modules. In some embodiments, one or more steps of method 300 are performed in conjunction with block 210 of FIG. 2.

[0042] In block 302, the UMD 110 collects metadata for each high-priority workload for a given command buffer 114 (i.e., "per-workload metadata") when recording the high-priority workload in the command buffer 114. In some embodiments, the metadata stored for each high-priority workload includes one or more tokens and one or more offsets. Each offset defines a location in the command buffer 114 that needs to be modified when two-level binning mode is enabled to execute the corresponding high-priority workload of the command buffer 114. Each token defines how the code of the command buffer 114 at the location defined in the corresponding offset should be modified when two-level binning mode is enabled. In one example, the metadata tokens cause the UMD 110 to modify code describing the visibility of primitives of the given high-priority workload when two-level binning mode is enabled, the code being included in the high-priority workload itself or stored in the command buffer 114 associated with the high-priority workload.

[0043] At block 304, the UMD 110 determines whether two-level binning mode is enabled at or near the end of the recording process for a given high priority workload being recorded in the command buffer 114. In some embodiments, the UMD 110 checks the values ​​of one or more status bits stored in the system memory 106 to determine whether two-level binning mode is enabled. If it is determined that two-level binning mode is enabled, the method proceeds to block 310. If it is determined that two-level binning mode is disabled, the method 300 proceeds to block 306.

[0044] At block 306, the UMD 110 determines whether two-level binning mode is enabled after recording the high-priority workload to the command buffer 114 and before (e.g., immediately before) submitting the high-priority workload from the command buffer 114 to the GPU 102. In some embodiments, the UMD 110 checks the values ​​of one or more status bits stored in the system memory 106 to determine whether two-level binning mode is enabled. If it is determined that two-level binning mode is enabled, the method proceeds to block 312. If it is determined that two-level binning mode is disabled, the method 300 proceeds to block 308.

[0045] In block 308, a scheduler (e.g., implemented via the CPU 104 via the KMD 112 or via the GPU 102 via firmware) submits high priority workloads from the command buffer 114 to the GPU 102.

[0046] In block 310, UMD 110 patches command buffer 114 to configure the high priority workloads to be executable in two-level binning mode based on per-workload metadata while the high priority workloads are being recorded to command buffer 114 (e.g., near the end of the recording process). Generally, the manner in which command buffer 114 is patched by UMD 110 depends on the hardware implementation of device 100.

[0047] In one example, two-level binning essentially uses visibility information in a buffer (i.e., a “visibility information buffer”) as a basis for determining which primitives are visible in which bins. In this example, if the UMD 110 records a high-priority workload in the command buffer 114 to be executed using one-level binning, and the UMD does not include a command for the GPU to bind such a visibility information buffer, but if the high-priority workload was recorded by the UMD 110 to be executed using two-level binning, the high-priority workload would need to include such a command. Thus, the UMD 110 generates metadata including a token and offset indicating a location within the command buffer 114, and a command to bind the visibility information buffer would need to be included for the high-priority workload when executed in two-level binning mode. In this way, if two-level binning mode is enabled before submitting the high-priority workload from the command buffer 114 to the GPU 102, the UMD 110 or KMD 112 patches the high-priority workload to include a command to bind the visibility information buffer at the location indicated in the metadata.

[0048] In another example, GPU 102 generally needs to receive bin information indicating how many bins there are, the size of those bins, and / or the order in which the bins should be processed when executing in two-level binning mode. In this example, UMD 110 generates metadata for each high-priority workload recorded in command buffer 114, including binning information indicating the number of bins, the size of each bin, and the order in which the bins should be processed, the binning information needed to execute that high-priority workload in two-level binning mode. In this manner, if two-level binning mode is enabled before submitting the high-priority workload to command buffer 114 to GPU 102, UMD 110 or KMD 112 patches the high-priority workload in command buffer 114 to include the binning information indicated in the metadata.

[0049] In block 312, after recording the high priority workload in the command buffer 114 and before submitting the high priority workload from the command buffer 114 to the GPU 102, the UMD patches the command buffer 114 and configures the high priority workload stored in the command buffer 114 so that it can be executed in two-level binning mode based on metadata for each workload.

[0050] 4 illustrates an exemplary process flow for a method 400 for selectively executing a workload of a command buffer on a GPU in a two-level binning mode or a non-two-level binning mode. In some embodiments, the GPU performs command buffer patching of the command buffer to enable execution of the workload in either a two-level binning mode or a non-two-level binning mode based on corresponding metadata generated by the UMD. Method 400 is described with respect to an exemplary implementation in device 100 of FIG. 1 and its constituent components and modules. In some embodiments, one or more steps of method 400 are performed in conjunction with method 200 of FIG. 2.

[0051] In block 402, the UMD 110 records a workload to the command buffer 114. In some embodiments, the UMD 110 configures the recorded workload to be executable in either two-level binning mode or non-two-level binning mode without patching. In some other embodiments, the UMD 110 configures the recorded high-priority workload to run in non-two-level binning mode by default, and generates metadata that enables the GPU 102 to change the recorded workload to run in two-level binning mode if necessary (i.e., if two-level binning mode is enabled after the high-priority workloads are recorded to the command buffer 114 and before their execution by the GPU 102). In some embodiments, the UMD 110 configures the recorded low-priority and medium-priority workloads to run in two-level binning mode by default.

[0052] In one example, the UMD 110 records a conditional statement for one or more high priority workloads in the command buffer 114, and the conditional statement causes the GPU 102 to check a patch valid value stored in a register in the GPU memory 124 and execute each high priority workload in the command buffer 114 in a two-level binning mode or a non-two-level binning mode depending on the value of the patch valid value. In some embodiments, the patch valid value is a Boolean value stored in a single bit of a register in the GPU memory 124. In some embodiments, the patch valid value is set by the UMD 110 or the KMD 112.

[0053] In block 404, the KMD 112 submits the workload from the command buffer 114 to the GPU 102. In some embodiments, once submitted to the GPU 102, the workload is added to the ring buffer 120.

[0054] At block 406, the GPU 102 determines whether two-level binning mode is enabled for the submitted workload. In some embodiments, the GPU 102 checks one or more patch enable values ​​stored in the GPU memory 124 to determine whether two-level binning mode is enabled for the submitted high-priority workload. In some embodiments, the KMD 112 determines whether two-level binning mode is enabled for the submitted high-priority workload based on corresponding performance data and sets the patch enable value in the GPU memory 124 accordingly. If the GPU 102 determines that two-level binning mode is enabled for the submitted workload, the method 400 proceeds to block 408. Otherwise, if the GPU 102 determines that two-level binning mode is not enabled for the submitted workload, the method 400 proceeds to block 410. In some embodiments, the two-level binning mode is enabled by default for low-priority and medium-priority workloads, and such workloads are run in two-level binning mode based on their priority (e.g., rather than performance heuristics).

[0055] At block 408, the GPU 102 executes the submitted workload in two-level binning mode. In some embodiments, the GPU 102 utilizes metadata generated by the UMD 110 during recording of the workload to the command buffer 114, as described above, to patch one or more high-priority workloads in the command buffer 114 to execute in two-level binning mode in response to determining that the patch effect value indicates that the high-priority workloads are to be executed in two-level binning mode. In some other embodiments, the UMD 110 records each high-priority workload that may be executed in two-level binning mode as being executable in either two-level binning mode or a non-two-level binning mode, and the GPU 102 is configured to execute the high-priority workloads in a selected one of the two-level binning mode or the non-two-level binning mode based on the patch effect value.

[0056] In block 410, the GPU 102 executes the submitted workload in the command buffer 114 in a non-two-level binning mode. In some embodiments, the non-two-level binning mode is a PBB rendering mode.

[0057] 5 illustrates an example in which a GPU (an embodiment of GPU 102 of FIG. 1) is configured to preempt low and medium priority workloads in order to execute a high priority workload. This example will be described with reference to an exemplary implementation in device 100 of FIG. 1 and its constituent components and modules.

[0058] In this example, command processor 122 of GPU 102 is configured to receive and execute workloads from a high priority queue 502 (e.g., one of ring buffers 120) and a low / medium priority queue 504 (e.g., another of ring buffers 120). For example, each workload stored in command buffer 114 includes a priority indicator that specifies the priority level of that workload (e.g., high priority, medium priority, low priority, etc.). In some embodiments, the priority indicator for a given workload is determined based on a function of that workload (e.g., all display configuration workloads are designated high priority). In this example, high priority workloads submitted for execution by GPU 102 are stored in high priority queue 502, and low priority and medium priority workloads submitted for execution by GPU 102 are stored in low / medium priority queue 504.

[0059] In some embodiments, GPU 102 is configured to preempt the execution of a low or medium priority workload being executed by command processor 122 upon receiving a high priority workload in high priority queue 502. For example, command processor preempts the execution of a workload from low / medium priority queue 504 in response to receiving preemption indicator 506 indicating that high priority workload 502 is ready to execute in high priority queue 502.

[0060] Conventionally, because APIs typically do not provide a means to divide workloads into smaller pieces than full draw calls, preemption of a given low- or medium-priority workload is delayed by the amount of time it takes command processor 122 to finish executing the full draw call of the workload being preempted. As used herein, a draw call refers to a call to a graphics API that causes one or more objects (e.g., one or more primitives) to be drawn during rendering. For example, the primitive count of a given draw call is typically specified as part of the draw call command. Such delays are undesirable or unacceptable because they potentially result in GPU 102 failing to timely execute a high-priority workload that initiated preemption of a lower-priority workload before its specified deadline. Therefore, it is typically desirable to reduce such delays to avoid missing the execution deadline of such high-priority workloads.

[0061] For example, in extended reality (XR) applications, such as virtual reality (VR), augmented reality (AR), or mixed reality (MR) applications, workloads for rendering virtual objects in an XR scene are typically assigned low or medium priority, while periodic reprojection workloads are assigned high priority. In XR applications, reprojection, sometimes referred to as asynchronous reprojection, is a process by which a system (e.g., a headset, wearable display, etc.) running an XR application ensures adequate display responsiveness to system movement, especially in situations where a GPU bottleneck causes reduced frame rates and therefore perceived input lag or latency. For example, reprojection generates a prediction of what an image frame will look like based on one or more previously rendered image frames and motion data generated by one or more motion sensors in the system; typically, the motion data is generated after the previous image frame has been rendered. Reprojection workloads are typically generated in parallel with regular image rendering workloads. In a typical XR application, at least one reprojection workload must be executed for each vertical sync (Vsync) of the GPU to reduce the user's perception of input latency. To ensure that a reprojection workload is executed for each Vsync, if the system is executing a lower priority image rendering workload when a reprojection workload is ready to run, the system uses draw call preemption to preempt (e.g., temporarily stop) the execution of the lower priority image rendering workload, and executes the reprojection workload once the lower priority workload has been successfully stopped.

[0062] As another example, in an automotive application, such as an automotive system having a human machine interface (HMI) display, a navigation display, and an instrument cluster, workloads for rendering HMI and navigation images are typically assigned a low or medium priority, and workloads for rendering instrument cluster images are assigned a high priority. In automotive applications, the vehicle's instrument cluster display typically needs to be updated periodically so that the vehicle operator has an accurate and up-to-date understanding of various vehicle parameters (e.g., speed, temperature, mileage, rpm, fuel level, etc.) indicated via the instrument cluster. In some instances, a minimum refresh rate for the instrument cluster in an automotive system is set according to industry standards, and it is beneficial to ensure that this minimum refresh rate is consistently met by the automotive system. In some embodiments, the instrument cluster display is configured to be updated (e.g., via execution of a corresponding instrument cluster workload) once per Vsync. To ensure that each instrument cluster workload is executed for each Vsync, if the automotive system is executing a lower priority HMI or navigation workload when the instrument cluster workload is ready to run, the automotive system uses draw call preemption to temporarily halt execution of the lower priority workload and executes the instrument cluster workload once the lower priority workload has been successfully halted.

[0063] In both the XR and automotive examples provided above, when draw call preemption occurs, higher priority workloads cannot execute until after the originally executing lower priority workload has been successfully stopped. As shown above, APIs typically do not provide a means for workloads to be decomposed into smaller pieces than full draw calls, and a given full draw call must finish executing before the corresponding workload can be successfully stopped. However, the more complex the full draw call of a low- or medium-priority workload, the more time it takes to execute, leading to unacceptable delays in draw call preemption when attempting to preempt such a complex full draw call. Such delays can cause the execution of a high-priority workload, such as a reprojection workload or instrument cluster workload, to miss a deadline (e.g., the next Vsync) for the high-priority workload to execute. In the XR example, this can result in reprojected / predicted image frames being rendered and displayed slower than expected, potentially causing motion sickness or disorientation in the user. In the automotive example, this could result in unacceptable delays in refreshing the instrument cluster display, causing the automotive system to become non-compliant with relevant standards. Therefore, it is desirable for low- and medium-priority workloads to be decomposed into smaller divisions than full draw calls so that draw call preemptions can be performed more quickly. The size of each preemptible division of a given workload is referred to herein as the “preemption granularity.” Workloads configured for execution in two-level binning mode have better preemption granularity (e.g., smaller divisions) than workloads running in non-two-level binning modes, such as PBB or immediate mode rendering; two-level binning mode workloads are divided into individually executable coarse bins, and stalling of workload execution due to draw call preemption can occur during the execution of each coarse bin.Because batches of PBB mode workloads are not independently executable and immediate mode rendering workloads are not divided into batches or bins, non-two-level binned workloads, such as those configured for PBB or immediate mode rendering, cannot be divided into individually executable portions in this manner. Accordingly, in some embodiments described herein, the system configures low- and medium-priority workloads to run in two-level binning mode without considering performance heuristics to improve preemption granularity for such workloads, while selecting a binning mode for high-priority workloads based on performance heuristics.

[0064] In this example, GPU 102 is configured to reduce draw call preemption latency by configuring all applicable low-priority and medium-priority workloads to run in two-level binning mode. As previously indicated, a workload applicable for execution in two-level binning mode is a workload that is functionally executable in two-level binning mode. A workload configured to run in two-level binning mode is divided into coarse bins, each representing a different region of the image frame to be displayed; for example, each coarse bin is independently executable by command processor 122. In contrast to workloads not configured to run in two-level binning mode, which typically include full draw calls that require the entire image frame to be rendered when executed, the draw calls of a workload configured to run in two-level binning mode are decomposed into smaller, independently executable coarse bins such that the entire image frame does not need to be rendered all at once when executing the workload's draw calls in two-level binning mode. When execution of a low- or medium-priority workload in two-level binning mode is preempted by a higher-priority workload, command processor 122 does not need to complete rendering of the entire draw call that was executing when preemption indicator 506 was received. Rather, command processor 122 only completes rendering of the coarse bin that was executing when preemption indicator 506 was received, which desirably improves preemption granularity, reduces preemption latency, and improves the likelihood that the preempting high-priority workload will successfully execute by its specified deadline.

[0065] In this example, the high-priority queue 502 contains high-priority workloads consisting of a single full draw call command (herein denoted "DC") not configured to execute in two-level binning mode, while the low / medium-priority queue contains low-priority workloads consisting of a single draw call command that included multiple coarse bins CB0-CB5 for execution in two-level binning mode. For example, if command processor 122 receives preemption indicator 506 while executing coarse bin CB3, command processor 122 terminates execution of coarse bin CB3 and then begins executing the high-priority workload in high-priority queue 502. Once the high-priority workload is executed, the command processor resumes execution of the low-priority workload by beginning rendering of the next coarse bin CB4. In some embodiments, the amount of coarse bins into which a given draw call is divided and the size of each coarse bin are adjustable, either automatically or manually; by adjusting the amount and size of the coarse bins, the preemption granularity is also adjusted.

[0066] It should be understood that this example is intended to be illustrative and not limiting. According to various embodiments, high priority queue 502 and low / medium priority queue 504 may each include multiple workloads, each having one or more draw calls, and the draw calls included in high priority queue 502 are optionally divided into coarse bins for execution according to a two-level binning mode based on performance-based heuristics (e.g., according to one or more of methods 200, 300, 400 of FIGS. 2, 3, 4).

[0067] 6 illustrates an example process flow for a method 600 for assigning workloads to queues based on their respective priorities and selectively executing the workloads in a two-level binning mode or a non-two-level binning mode based on their respective priorities and, for high-priority workloads, based on performance data generated by performance counters or sensors. Method 600 will be described with reference to an example implementation in device 100 of FIG. 1 and its constituent components and modules, and with reference to the example of FIG. 5.

[0068] In block 602 , the UMD 110 (eg, being executed by the CPU 104 ) identifies a new workload that is ready to be recorded in the command buffer 114 .

[0069] At block 604, the UMD 110 determines the priority of the workload. In some embodiments, each workload includes a priority indicator that specifies the priority level of the workload (e.g., in this example, high priority, medium priority, or low priority). In response to determining that the workload is a high priority workload, the method 600 proceeds to block 606. In response to determining that the workload is a low priority workload or a medium priority workload, the method 600 proceeds to block 614.

[0070] At block 606, the UMD 110 determines the binning mode for the high priority workload. In some embodiments, the UMD 110 determines the binning mode for the high priority workload based on whether one or more binning conditions (e.g., two-level binning conditions) are met and whether override conditions are met (e.g., as provided in connection with blocks 204 and 206 of method 200 of FIG. 2). In some embodiments, the UMD 110 determines whether the binning conditions or override conditions are met based on performance data, such as sensor data 118 generated by sensors 108 or performance counter data generated by performance counters 116. In some embodiments, the derived parameters are calculated by the UMD 110 or the KMD 112.

[0071] For example, in response to determining that the binning condition is met and the override condition is not met, UMD 110 determines that the high priority workload should be configured for execution in two-level binning mode, and method 600 proceeds to block 614.

[0072] For example, in response to determining that the binning condition is not met, or that the binning condition is met and the override condition is met, UMD 110 determines that the high priority workload should be configured for execution in a non-two-level binning mode, and method 600 proceeds to block 608.

[0073] At block 608, the UMD 110 enables the non-two-level binning mode for the high priority workload. In some embodiments, the UMD 110 configures the high priority workload to run in the non-two-level binning mode when it records the high priority workload in one of the command buffers 114. In some embodiments, all high priority workloads are configured to run in the non-two-level binning mode by default. In some embodiments, the UMD 110 sets one or more patch valid values, status bits, or both to indicate that the high priority workload (and in some cases, all high priority workloads) should run in the non-two-level binning mode.

[0074] At block 614, UMD 110 enables two-level binning mode for high priority workloads. In some embodiments, UMD 110 configures high priority workloads to run in two-level binning mode when recording the high priority workload to one of command buffers 114. In some embodiments, UMD 110 sets one or more patch valid values, status bits, or both to indicate that the high priority workload (and in some cases, all high priority workloads) should run in two-level binning mode.

[0075] In block 610, after UMD 110 enables the determined binning mode for the high priority workload, UMD 110 submits the high priority workload to GPU 102, and the high priority workload is added to a high priority queue (e.g., an exemplary embodiment of high priority queue 502 in FIG. 5).

[0076] At block 612, the high priority workload is executed on the GPU 102 in the binning mode (e.g., two-level binning mode or non-two-level binning mode) determined at block 606. In some embodiments, execution of the high priority workload preempts execution of a lower priority workload.

[0077] At block 616, UMD 110 enables two-level binning mode for low or medium priority workloads. In some embodiments, UMD 110 configures low or medium priority workloads to run in two-level binning mode when UMD 110 records the low or medium priority workload in one of command buffers 114. In some embodiments, all low or medium priority workloads are automatically configured to run in two-level binning mode by default, if applicable (i.e., functionally possible).

[0078] In block 618, after UMD 110 enables the two-level binning mode for low or medium priority workloads, UMD 110 submits the low or medium priority workloads to GPU 102, where the low or medium priority workloads are added to a low / medium priority queue (e.g., an exemplary embodiment of high priority queue 502 in FIG. 5).

[0079] At block 620, low or medium priority workloads are executed in two-level binning mode on GPU 102. In some embodiments, execution of low or medium priority workloads is temporarily halted (e.g., via draw call preemption) to allow one or more high priority workloads to execute on GPU 102.

[0080] 7 illustrates an example process flow for a method 700 for preempting execution of a lower priority workload (e.g., a low priority or medium priority workload) from a low / medium priority queue in a two-level binning mode in order to execute a high priority workload from a high priority queue in a GPU. Method 700 is described with reference to an example implementation in device 100 of FIG. 1 and its constituent components and modules, and with reference to the example of FIG. 5.

[0081] In block 702, the command processor 122 of the GPU 102 begins executing a first workload (i.e., a low-priority or medium-priority workload) from the low / medium priority queue 504 in a two-level binning mode. For example, each full draw call of the first workload is divided into multiple coarse bins, each of the coarse bins of a given draw call representing a different region of the image frame being rendered via the draw call, and each of the coarse bins is independently executable.

[0082] At block 704, while executing the first workload, the GPU 102 continuously or periodically determines whether a preemption notification, such as preemption notification 506, is received by the GPU 102. In some embodiments, a scheduler (e.g., implemented via a driver, such as UMD 110 or KMD 112, in the CPU 104 or via firmware in the GPU 102) sends a preemption notification to the GPU 102 in response to determining that a high priority workload has been submitted to the high priority queue 502. If a preemption notification is not received by the GPU 102 while continuing to execute the first workload, the method 700 proceeds to block 706, where the GPU 102 terminates execution of the first workload in two-level binning mode. If a preemption notification is received by the GPU 102 while executing the first workload, the method 700 proceeds to block 708.

[0083] At block 708, the GPU 102 determines whether a coarse bin of the first workload was only partially rendered at the time the preemption notification was received while executing the first workload (e.g., the preemption notification was received while the GPU 102 was rendering the coarse bin). In response to the GPU 102 identifying a partially rendered coarse bin, the method 700 proceeds to block 708. Otherwise, the method 700 proceeds to block 712.

[0084] In block 710, the GPU 102 (e.g., the command processor 122 of the GPU 102) finishes rendering the coarse bin.

[0085] If, in block 712, it is determined that there are either no partially rendered coarse bins or that all identified partially rendered coarse bins have been rendered, the GPU 102 executes (e.g., using the command processor 122) a second workload (e.g., a high priority workload) from the high priority queue 504, the second workload corresponding to the preemption notification received by the GPU 102 in the most recent iteration of block 704.

[0086] Upon executing the second workload, GPU 102 resumes execution of the first workload in two-level binning mode at block 714. Upon resuming execution of the first workload, method 700 returns to block 704 to determine whether any additional preemption notifications are received by GPU 102 before terminating execution of the first workload.

[0087] As described above, in some embodiments, a method includes executing, by at least one processing unit of a system, a first workload in a first binning mode; and executing, by the at least one processing unit, a second workload in a selected binning mode selected from one of the first binning mode and a second binning mode based on performance heuristics of the system. In one aspect, the method includes determining, by the at least one processing unit, based on a first priority indicator of the first workload, that the first workload is a low priority workload, wherein the first workload is executed in the first binning mode in response to determining that the first workload is a low priority workload, and determining, by the at least one processing unit, based on a second priority indicator of the second workload, that the second workload is a high priority workload, wherein the second workload is executed in the second binning mode in response to determining that the second workload is a high priority workload. In another aspect, the first binning mode is a two-level binning mode and the second binning mode is a non-two-level binning mode.

[0088] In one aspect, the first binning mode is a two-level binning mode, and the method further includes configuring the first workload to run in the first binning mode, where configuring the first workload is performed by dividing image frames associated with the first workload into a plurality of coarse bins, and for each coarse bin of the plurality of coarse bins, dividing the coarse bin into a plurality of fine bins, and segmenting a plurality of primitives associated with the image frames of the first workload into temporally related primitive batches. In another aspect, the method includes receiving, by at least one processing unit, a preemption notification while executing a first coarse bin of the plurality of coarse bins of the first workload, stopping execution of the first workload by the at least one processing unit in response to receiving the preemption notification, wherein executing a second workload occurs while execution of the first workload is stopped, and resuming execution of the first workload upon executing the second workload by the at least one processing unit. In yet another aspect, the method includes completing execution of a first coarse bin of the first workload before stopping execution of the first workload, and resuming execution of the first workload includes commencing execution of a second coarse bin of the plurality of coarse bins of the first workload.

[0089] In one aspect, the method includes selecting the selected binning mode for the second workload by determining, based on system performance heuristics, whether a two-level binning condition corresponding to the two-level binning mode is met and determining, based on the system performance heuristics, whether an override condition is met. In another aspect, the two-level binning condition includes at least one of an average temperature of a system including at least one processing unit exceeding a first predefined temperature threshold, a first rate of change of the average temperature of the system exceeding a second predefined threshold, a local temperature at a defined location of the system exceeding a third predefined threshold, a second rate of change of the local temperature exceeding a fourth predefined threshold, an average power consumption of the system exceeding a fifth predefined threshold, a peak instantaneous power consumption of the system exceeding a sixth predefined threshold, or a battery state of charge below a seventh predefined threshold. In yet another aspect, the performance heuristics include at least one of an average temperature of a system including the at least one processing unit, a first rate of change of the average temperature of the system, an average power consumption of the system, a second rate of change of the average power consumption, a peak instantaneous power consumption of the system, a state of charge of a battery of the system, or an applicability of the second workload to run in a two-level binning mode.

[0090] In some embodiments, the device includes at least one processing unit configured to execute a first workload in a first binning mode and execute a second workload in a selected binning mode selected from one of the first binning mode and the second binning mode based on performance heuristics of the device. In one aspect, the at least one processing unit is further configured to: determine, based on a first priority indicator of the first workload, that the first workload is a low-priority workload, wherein the first workload is executed in the first binning mode in response to determining that the first workload is a low-priority workload; and determine, based on a second priority indicator of the second workload, that the second workload is a high-priority workload, wherein the second workload is executed in the selected binning mode in response to determining that the second workload is a high-priority workload. In another aspect, the first binning mode is a two-level binning mode and the second binning mode is a non-two-level binning mode.

[0091] In one aspect, the at least one processing unit is configured to configure a first workload to execute in a two-level binning mode, where configuring the first workload is performed by dividing image frames associated with the first workload into a plurality of coarse bins, and for each coarse bin of the plurality of coarse bins, dividing the coarse bin into a plurality of fine bins, and segmenting a plurality of primitives associated with the image frames of the first workload into temporally related primitive batches. In another aspect, the at least one processing unit is further configured to receive a preemption notification while executing a first coarse bin of the plurality of coarse bins of the first workload, and in response to receiving the preemption notification, stop execution of the first workload, execute a second workload while the first workload is stopped, and resume execution of the first workload upon executing the second workload. In yet another aspect, the at least one processing unit is further configured to complete execution of a first coarse bin before ceasing execution of the first workload, and wherein resuming execution of the first workload includes resuming execution of a second coarse bin of the plurality of coarse bins of the first workload.

[0092] In some embodiments, the device includes at least one processing unit configured to: receive a preemption notification while executing a first coarse bin of image frames of a first workload in a two-level binning mode; in response to receiving the preemption notification, stop execution of the first workload after completing execution of the first coarse bin; execute a second workload in the selected binning mode while the first workload is suspended; and resume execution of the first workload upon execution of the second workload. In one aspect, the at least one processing unit is further configured to select the selected binning mode from one of the two-level binning mode and the non-two-level binning mode based on performance heuristics of the device. In another aspect, the at least one processing unit is further configured to configure the first workload to run in a two-level binning mode, where configuring the first workload is performed by dividing image frames associated with the first workload into a plurality of coarse bins including a first coarse bin; for each coarse bin of the plurality of coarse bins, dividing the coarse bin into a plurality of fine bins; and segmenting a plurality of primitives associated with the image frames of the first workload into temporally related primitive batches.

[0093] In one aspect, the at least one processing unit is configured to execute a second coarse bin of the plurality of coarse bins of the image frame upon resuming execution of the first workload. In another aspect, the at least one processing unit is further configured to determine, based on a first priority indicator of the first workload, that the first workload is a low priority workload, determine, based on a second priority indicator of the second workload, that the second workload is a high priority workload, submit the first workload to a first queue, and submit the second workload to a second queue different from the first queue.

[0094] In some embodiments, the above-described apparatus and techniques are implemented in a system that includes one or more integrated circuit (IC) devices (also called integrated circuit packages or microchips), such as device 100 including GPU 102, CPU 104, and system memory 106 described with reference to FIG. 1 . Electronic design automation (EDA) and computer-aided design (CAD) software tools can be used in the design and manufacture of these IC devices. These design tools are typically represented as one or more software programs. The one or more software programs include code executable by a computer system for operating the computer system to operate on code representing circuits of one or more IC devices to perform at least a portion of a process for designing or adapting a manufacturing system for producing the circuits. This code may include instructions, data, or a combination of instructions and data. The software instructions representing the design or manufacturing tools are typically stored on a computer-readable storage medium accessible to the computing system. Similarly, code representing one or more stages of the design or manufacture of the IC devices is stored on and accessed from the same or a different computer-readable storage medium.

[0095] A computer-readable storage medium includes any non-transitory storage medium or combination of non-transitory storage media that can be accessed by a computer system during use to provide instructions and / or data to the computer system. Such storage media may include, but are not limited to, optical media (e.g., compact discs (CDs), digital versatile discs (DVDs), Blu-ray discs), magnetic media (e.g., floppy disks, magnetic tape, magnetic hard drives), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or micro-electromechanical systems (MEMS)-based storage media. The computer-readable storage medium (e.g., system RAM or ROM) may be internal to the computing system, the computer-readable storage medium (e.g., a magnetic hard drive) may be permanently attached to the computing system, the computer-readable storage medium (e.g., an optical disk or Universal Serial Bus (USB)-based flash memory) may be removably attached to the computing system, or the computer-readable storage medium (e.g., network-accessible storage (NAS)) may be coupled to the computer system via a wired or wireless network.

[0096] In some embodiments, certain aspects of the techniques described above are implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied in a non-transitory computer-readable storage medium. The software may include instructions and specific data that, when executed by one or more processors, operate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer-readable storage medium may include, for example, a magnetic or optical disk storage device, a solid-state storage device such as flash memory, a cache, a random access memory (RAM), or other non-volatile memory device(s). The executable instructions stored on the non-transitory computer-readable storage medium may be implemented as source code, assembly language code, object code, or other form of instructions that can be interpreted or otherwise executed by one or more processors.

[0097] In addition to the above, it should be noted that not all activities or elements described in the summary description are required, that some of the particular activities or devices may not be required, that one or more additional activities may be performed, and that one or more additional elements may be included. Furthermore, the order in which the activities are listed is not necessarily the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, those skilled in the art will recognize that various modifications and variations can be made without departing from the scope of the invention as set forth in the claims. Accordingly, the specification and drawings should be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present invention.

[0098] Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and features from which any benefit, advantage, or solution may arise or be manifested are not construed as critical, essential, or essential features of any or all claims. Moreover, the specific embodiments described above are illustrative only, since the disclosed invention may be modified and practiced in different, but similar manners apparent to those skilled in the art having the benefit of the teachings herein. No limitations are intended to the details of construction or design herein shown, other than as described in the appended claims. It is therefore apparent that the specific embodiments described above may be altered or modified, and that all such variations are considered within the scope of the disclosed invention. Accordingly, the protection sought herein is set forth in the appended claims.

Claims

1. 1. A method comprising: Executing, on at least one processing unit of the system, a first workload in a two-level binning mode including coarse level binning and fine level binning; and executing, by the at least one processing unit, a second workload in a selected binning mode selected from the two-level binning mode and the single-level binning mode based on performance heuristics of the system. method.

2. the at least one processing unit determining, based on a first priority indicator of the first workload, that the first workload is a low priority workload, and the first workload is executed in the two-level binning mode in response to determining that the first workload is a low priority workload; the at least one processing unit determining, based on a second priority indicator of the second workload, that the second workload is a high priority workload, and the second workload is executed in the single level binning mode in response to determining that the second workload is a high priority workload.

10. The method of claim 1.

3. The method comprises: configuring the first workload to run in the two-level binning mode; Configuring the first workload comprises: Dividing image frames associated with the first workload into a plurality of coarse bins; for each of the plurality of coarse bins, dividing the coarse bin into a plurality of fine bins; segmenting a plurality of primitives associated with the image frames of the first workload into temporally related batches of primitives; carried out by The method of claim 1 or 2.

4. receiving a preemption notification while the at least one processing unit is executing a first coarse bin of the plurality of coarse bins of the first workload; the at least one processing unit suspending execution of the first workload in response to receiving the preemption notification, wherein executing the second workload occurs while execution of the first workload is suspended; and and wherein the at least one processing unit resumes execution of the first workload upon executing the second workload. The method of claim 3.

5. completing execution of the first coarse bin of the first workload before ceasing execution of the first workload; Resuming execution of the first workload includes: commencing execution of a second coarse bin of the plurality of coarse bins of the first workload. The method of claim 4.

6. selecting the selected binning mode for the second workload; The selecting comprises: by determining whether a two-level binning condition corresponding to the two-level binning mode is satisfied based on the performance heuristics of the system. The method of any one of claims 1 to 5.

7. the two-level binning conditions include at least one of: an average temperature of the system including the at least one processing unit exceeds a first predefined temperature threshold; a first rate of change of the average temperature of the system exceeds a second predefined threshold; a local temperature at a defined location of the system exceeds a third predefined threshold; a second rate of change of the local temperature exceeds a fourth predefined threshold; an average power consumption of the system exceeds a fifth predefined threshold; a peak instantaneous power consumption of the system exceeds a sixth predefined threshold; or a battery state of charge is below a seventh predefined threshold. The method of claim 6.

8. the performance heuristics include at least one of an average temperature of the system including the at least one processing unit, a first rate of change of the average temperature of the system, an average power consumption of the system, a second rate of change of the average power consumption, a peak instantaneous power consumption of the system, a state of charge of a battery of the system, or an applicability of the second workload to run in the two-level binning mode.

8. The method of claim 6 or 7.

9. A device, at least one processing unit; The at least one processing unit running a first workload in a two-level binning mode including coarse level binning and fine level binning; running a second workload in a selected binning mode selected from the two-level binning mode and the single-level binning mode based on performance heuristics of the device; configured to: device.

10. The at least one processing unit determining, based on a first priority indicator of the first workload, that the first workload is a low priority workload, wherein the first workload is executed in the two-level binning mode in response to determining that the first workload is a low priority workload; determining, based on a second priority indicator of the second workload, that the second workload is a high priority workload, wherein the second workload is executed in the selected binning mode in response to determining that the second workload is a high priority workload; and further configured to: The device of claim 9.

11. The at least one processing unit configured to configure the first workload to run in the two-level binning mode; Configuring the first workload comprises: Dividing image frames associated with the first workload into a plurality of coarse bins; for each of the plurality of coarse bins, dividing the coarse bin into a plurality of fine bins; segmenting a plurality of primitives associated with the image frames of the first workload into temporally related batches of primitives; carried out by The device of claim 9.

12. The at least one processing unit receiving a preemption notification while executing a first coarse bin of the plurality of coarse bins of the first workload; ceasing execution of the first workload in response to receiving the preemption notification; running the second workload while the first workload is stopped; resuming execution of the first workload upon execution of the second workload; and further configured to: The device of claim 11.

13. The at least one processing unit further configured to complete execution of the first coarse bin before ceasing execution of the first workload; Resuming execution of the first workload includes: resuming execution of a second coarse bin of the plurality of coarse bins of the first workload. The device of claim 12.

Images