Priority inversion mitigation

Abstract

The parallel processor allocates resources to workloads based on workload priority. Priority inversion of resource allocations among workloads of different priority may reduce the operating efficiency of the parallel processor. The parallel processor mitigates priority inversion by soft-locking resources to prevent resource allocation for processing of low priority workloads. Soft-locking is enabled in response to a soft-lock condition, such as one or more priority inversion heuristics exceeding a corresponding threshold or multiple failed allocations of high priority workloads within a period of time. Priority inversion heuristics include the amount of high priority and low priority workloads being processed or received, the ratio of such amounts, the amount of render targets, or combinations thereof. The soft-lock is released, for example, in response to expiration of a soft-lock timer or the amount of received or processed high priority workloads dropping below a threshold.

Description

[Background technology]

[0001] Processing systems often include parallel processors that handle graphics (i.e., by processing graphics workloads), perform video processing operations, machine learning operations, etc. (i.e., by processing asynchronous computational workloads). To perform such operations efficiently, parallel processors divide operations into threads and divide groups of similar threads, such as similar operations on vectors or arrays of data, into sets of threads called wavefronts. The parallel processor executes the threads of one or more wavefronts in parallel on different computational units of the parallel processor.

[0002] A graphics processing unit (GPU) is an example of a parallel processor that typically processes three-dimensional (3D) graphics using a graphics pipeline formed of a sequence of programmable shaders and fixed-function hardware blocks. Shaders are categorized into various shader types, such as geometry shaders, vertex shaders, and pixel shaders. Different graphics workloads typically require different shader types for processing, and computational resources are allocated to implement each shader type based on the priority of that shader type. The processing efficiency of a parallel processor is increased by increasing the number of wavefronts that are executing or ready to be executed in a computation unit at a given time. Typically, asynchronous computation workloads are given a higher priority than graphics workloads. Priority inversion occurs when computational resources of a parallel processor are allocated to process low priority workloads, such as graphics workloads, instead of being allocated to process high priority workloads, such as asynchronous computation workloads, that are ready to be processed.

[0003] The present disclosure may be better understood, and its numerous features and advantages made apparent to those skilled in the art by referencing the accompanying drawings, in which: The use of the same reference numbers in different drawings indicates similar or identical items. [Brief description of the drawings]

[0004] [Figure 1] FIG. 1 is a block diagram of a parallel processor configured to perform priority inversion mitigation, according to some embodiments. [Diagram 2] FIG. 2 is a block diagram of a graphics pipeline that utilizes resources of a unified shader pool to implement various types of shaders, where resource allocation is managed by a resource allocator, according to some embodiments. [Diagram 3] FIG. 1 is a block diagram of a portion of a parallel processor illustrating priority inversion heuristics computed by a resource allocator based on which soft-lock signals are selectively enabled to mitigate priority inversions, according to some embodiments. [Figure 4] FIG. 2 is a flow diagram illustrating a method for priority inversion mitigation in a parallel processor using priority inversion heuristics, according to some embodiments. [Diagram 5] FIG. 1 is a flow diagram illustrating a method for selectively enabling a soft lock on computing resources to prevent allocation to low priority workloads based on priority inversion heuristics, according to some embodiments. [Figure 6] FIG. 1 is a flow diagram illustrating a method for selectively enabling a soft lock on computing resources to prevent allocation of resources to graphics workloads in response to detecting priority inversion, according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0005] In the context of computer processing, priority inversion is considered to have occurred whenever a low priority workload is scheduled for execution and / or resources are allocated for the execution of such low priority workload, and this scheduling / allocation delays the execution of a high priority workload that is otherwise ready to run. For example, in a parallel processor such as a graphics processing unit (GPU), priority inversion occurs between high priority graphics workloads (e.g., geometry shaders, vertex shaders) and low priority graphics workloads (e.g., pixel shaders). This type of priority inversion may occur due to resource allocation in a parallel processor that allows smaller sized resource allocation requests, such as those typically associated with pixel workloads, to be successfully allocated before larger sized resource allocation requests, such as those typically associated with geometry workloads, because it is easier to find a "fit" for the smaller resource allocation requests. This behavior has the potential to continuously block larger sized resource allocation requests, leading to the low priority pixel workload continuously winning resource allocation over the high priority geometry workload, resulting in, for example, a long-term priority inversion. As another example, in a GPU that processes an asynchronous compute workload in parallel with a graphics workload, priority inversion may occur between the asynchronous compute workload and the graphics workload (the asynchronous compute workload is typically, but not in all cases, assigned high priority in a GPU). In either example, long-term priority inversion may result in an undesirable backlog of high priority work, which may reduce the efficiency with which the parallel processor operates. For example, by allowing a backlog of geometry workloads to accumulate, generation of the corresponding pixel workload may be delayed, resulting in periods when the pixel workload is not available to be processed, which prevents the parallel processor from operating at its full pixel rate.

[0006] Embodiments of the parallel processor and corresponding techniques described herein detect instances of priority inversion between high and low priority workloads and, in response, mitigate such priority inversions by temporarily preventing computational resources from being allocated to the low priority workload (sometimes referred to herein as enabling or activating a “soft-lock” of computational resources for the low priority workload). In some embodiments, a resource allocator of the parallel processor calculates one or more priority inversion heuristics (e.g., data indicative of whether a priority inversion is likely) and selectively soft-locks allocation of computational resources to the low priority workload based on the priority inversion heuristics (e.g., based on a comparison to a threshold associated with the priority inversion heuristic). In some embodiments, in response to a priority inversion event in which the resource allocator fails to allocate resources for processing a high priority workload (e.g., due to the unavailability of required computational resources) and the resource allocator successfully allocates computational resources to a low priority workload, the resource allocator initiates an allocation failure timer configured to expire at the end of a predetermined period. If the high priority workload fails to allocate (i.e., the computational resources are not allocated to the high priority workload) before the allocation failure timer expires, the resource allocator enables a soft-lock to prevent all or a portion of the computational resources of the parallel processor from being allocated to the low priority workload. In some embodiments, the soft-lock is disabled if one or more soft-lock release conditions are met. Such soft-lock release conditions include, for example, expiration of a soft-lock period, a system reset, or one or more priority inversion heuristics falling below corresponding thresholds.By selectively soft-locking computational resources from being allocated to low priority workloads in response to an indication of priority inversion, the resource allocator prevents the accumulation of a backlog of high priority workloads, thereby mitigating the impact of priority inversion on the operating efficiency of the parallel processor.

[0007] 1 illustrates a parallel processor 100 configured to detect and mitigate occurrences of priority inversion between low and high priority workloads. In some embodiments, the high priority workload corresponds to a graphics workload that requires allocation of a high priority shader, such as a geometry shader or a vertex shader (such workloads may be referred to herein as "geometry workloads"), and the low priority workload corresponds to a graphics workload that requires allocation of a low priority shader, such as a pixel shader (such workloads may be referred to herein as "pixel workloads"). In some embodiments, the high priority workload is an asynchronous computation workload and the low priority workload is a graphics workload.

[0008] As shown, parallel processor 100 includes a graphics engine 102, an asynchronous compute engine (ACE) 104, a pipeline scheduler 106, a resource allocator 108 configured to compute a priority inversion heuristic 110, a memory controller 112, a system memory 114, one or more caches 116, and a shader engine 118 including compute units 120. Commands received by parallel processor 100 (e.g., received from a host processor coupled to parallel processor 100) are stored in queues 122 and 124 to await processing by graphics engine 102 (in the case of queue 122) and ACE 104 (in the case of queue 124). In this specification, queue 122 may be referred to as a "graphics queue" and queue 124 may be referred to as an "asynchronous compute queue." The commands stored in queues 122 and 124 are typically commands related to rendering images, such as three-dimensional (3D) images of a scene. In some embodiments, each of the queues 122 and 124 includes one or more first-in, first-out buffers.

[0009] The shader engine 118 is implemented using shared hardware resources of the parallel processor 100, such as the compute units 120. In some embodiments, the shader engine 118 is used to implement shaders, such as geometry shaders, pixel shaders, etc. The resource allocator 108 is configured to allocate shared resources, including computational resources, such as the compute units 120 of the shader engine 118, to process workloads (e.g., to implement various types of shaders in connection with processing the workloads). In some embodiments, the resource allocator 108 periodically determines how the available shared resources should be allocated for execution of one or more workloads during an allocation period and allocates resources to process those workloads. The graphics engine 102 is configured to execute graphics workloads, which may involve, for example, implementing geometry shaders or pixel shaders. The ACE 104 is configured to execute, for example, a computational workload (sometimes referred to herein as an "asynchronous computational workload"), which may involve the implementation of a computational shader. The ACE 104 is a separate functional hardware block that is each capable of executing a computational workload simultaneously with the execution of other computational workloads by other ACEs in the ACE 104 and the processing of graphics workloads by the graphics engine 102. That is, the graphics engine 102 is capable of processing graphics workloads in parallel with the processing of asynchronous computation workloads by the ACE 104, and the ACE 104 is capable of collectively processing multiple asynchronous computation workloads in parallel.

[0010] Pipeline scheduler 106 is configured to schedule commands for execution by graphics engine 102, ACE 104, and computational resources (e.g., compute units 120 of shader engine 118) allocated by resource allocator 108. For example, pipeline scheduler 106 is used to schedule execution of commands when implementing a graphics pipeline, as described below.

[0011] Cache 116 includes one or more caches and, in some examples, is organized into a cache hierarchy. In some embodiments, cache 116 is configured to store data (e.g., commands, context information, vertex data, texture data, etc.) prefetched from system memory 114 for later use, for example, when processing a graphics workload or an asynchronous computation workload. Memory controller 112 is configured to manage the fulfillment of memory access requests (e.g., issued by graphics engine 102, ACE 104, or shader engine 118 during processing of a corresponding workload) via communication with cache 116, system memory 114, or both, as identified in the memory access requests.

[0012] The resource allocator 108 is configured to compute one or more priority inversion heuristics 110 indicative of a priority inversion between a high priority workload and a low priority workload. As described above, priority inversion occurs when a parallel processor's computational resources are allocated to execute a low priority workload rather than allocated to process a high priority workload, and execution of a low priority workload typically blocks allocation of computational resources to process a high priority workload. According to various embodiments, the priority inversion heuristics 110 include a respective amount of one or more of a received high priority workload, a processing high priority workload, a received low priority workload, a render target (RT) of an active workload of the processing workload, or one or more ratios of a low priority workload to a high priority workload. For example, an "active workload" of a processing workload refers to a workload that is being actively executed by the parallel processor 100. In some embodiments, the resource allocator 108 uses the priority inversion heuristics 110 to detect priority inversion between a high priority geometry workload and a low priority pixel workload. In some embodiments, resource allocator 108 uses priority inversion heuristics 110 to detect priority inversions between high-priority asynchronous compute workloads and low-priority graphics workloads. In response to detecting a priority inversion based on priority inversion heuristics 110, resource allocator 108 selectively enables a soft lock on one or more compute resources (e.g., compute units 120 of shader engine 118), which prevents allocation of the compute resources for processing of the low-priority workloads.

[0013] In some embodiments, resource allocator 108 manages one or more allocation failure timers. For example, a given allocation failure timer is activated in response to a failure to allocate a computational resource for a high priority workload and expires after a given amount of time (measured in some examples by counting clock cycles on the allocation failure timer). As used herein, a "processing" workload is one that is being actively processed (e.g., as part of a graphics pipeline) as opposed to a "receiving" workload in queues 122 and 124 that is awaiting scheduling and allocation.

[0014] For example, if the priority inversion heuristics 110 indicate a priority inversion between a high priority geometry workload and a low priority pixel workload, the resource allocator 108 soft-locks one or more computing resources from being allocated to process the low priority pixel workload until a soft-lock release condition is met. For example, if the priority inversion heuristics 110 indicate a priority inversion between a high priority asynchronous computation workload and a low priority graphics workload, the resource allocator 108 soft-locks one or more computing resources from being allocated to process the low priority graphics workload until a soft-lock release condition is met. Such soft-lock release conditions include, for example, expiration of a soft-lock period, a system reset, or one or more priority inversion heuristics falling below a corresponding threshold. By selectively soft-locking computing resources from being allocated to low priority workloads in response to an indication of priority inversion, the resource allocator 108 prevents a backlog of high priority workloads from building up, thereby mitigating the impact of priority inversion on the operating efficiency of the parallel processor 100. In some embodiments, the soft-lock is maintained as long as one or more of the priority inversion heuristics 110 continue to indicate a priority inversion, even if the soft-lock release conditions would otherwise be met.

[0015] Figure 2 shows a block diagram of a portion 200 of a parallel processor configured to implement a graphics pipeline 202 capable of processing high-order geometric primitives to generate rasterized images, according to some embodiments. The graphics pipeline 202 is implemented in some embodiments of the parallel processor 100 shown in Figure 1, and like elements are referenced using like numerals in this example. The graphics pipeline 202 is subdivided into a geometry processing portion 201 that comprises a portion of the graphics pipeline 202 before rasterization, and a pixel processing portion 203 that comprises a portion of the graphics pipeline 202 after rasterization.

[0016] Graphics pipeline 202 has access to storage resources 205 (also referred to herein as storage components), such as one or more memories or hierarchies of caches used to implement buffers and store vertex data, texture data, and the like. Storage resources 205 are implemented using some embodiments of system memory 114 shown in FIG. 1. Some embodiments of storage resources 205 include (or have access to) one or more caches or random access memories (RAMs). Portions of graphics pipeline 202 utilize texture data stored in storage resources 205 to generate rasterized images, such as rasterized images of a 3D scene.

[0017] The input assembler 210 accesses information from storage resources 205, for example, used to define objects that represent portions of a model of a scene. The vertex shader 215 logically receives as input a single vertex of a primitive (e.g., a basic shape such as a triangle) and outputs a single shaded vertex. Some embodiments of shaders, such as the vertex shader 215, perform single-instruction-multiple-data (SIMD) processing so that multiple vertices are processed simultaneously. In some embodiments, the graphics pipeline 202 implements a unified shader model such that all of the shaders included in the graphics pipeline 202 have the same execution platform on a shared large-scale SIMD compute unit. Thus, the shaders, including the vertex shader 215, are implemented using a common set of resources (e.g., compute resources), represented here as a unified shader pool 216, which includes, for example, an embodiment of the compute unit 120 of the shader engine 118. In some embodiments, the resource allocator 108 allocates computational resources of the unified shader pool 216 for implementation of shaders in the graphics pipeline 202 to execute corresponding workloads (a "geometry workload" corresponds to the geometry processing portion 201 and a "pixel workload" corresponds to the pixel processing portion 203) at times determined by the pipeline scheduler 106.

[0018] The hull shader 218 operates on input high-order patches or control points that are used to define the input patches. The hull shader 218 outputs tessellation coefficients and other patch data. In some embodiments, the primitives generated by the hull shader 218 are provided to the tessellator 220. The tessellator 220 receives objects (such as patches) from the hull shader 218 and generates information identifying primitives that correspond to the input objects, e.g., by tessellating the input objects based on tessellation coefficients provided to the tessellator 220 by the hull shader 218. The tessellation subdivides the input high-order primitives, such as patches, into a set of lower-order output primitives that represent finer levels of detail, e.g., as indicated by tessellation coefficients that specify the granularity of the primitives generated by the tessellation process. In this way, a model of the scene can be represented by a smaller number of high-order primitives (to save memory or bandwidth), and additional detail is added by tessellating the high-order primitives.

[0019] The domain shader 224 inputs the domain location and (optionally) other patch data. The domain shader 224 operates on the information provided and generates a single vertex for output based on the input domain location and other information. The geometry shader 226 receives an input primitive and outputs up to four primitives that are generated by the geometry shader 226 based on the input primitive. In the illustrated embodiment, the geometry shader 226 generates an output primitive 228 based on the tessellated primitive 222.

[0020] The stream of primitives is provided to a rasterizer 230, and in some embodiments, multiple streams of primitives are concatenated into buffers in storage resources 205. Rasterizer 230 performs shading operations and other operations such as clipping, perspective division, cutting, and viewport selection. Rasterizer 230 produces a set of pixels that are subsequently processed (as a "pixel flow") in pixel processing portion 203 of graphics pipeline 202.

[0021] In the illustrated embodiment, pixel shader 234 inputs a pixel flow (e.g., a set of pixels) and outputs zero or another pixel flow depending on the input pixel flow. Output merger block 236 performs blending, depth, stencil, or other operations on the pixels received from pixel shader 234.

[0022] Some or all of the shaders in the graphics pipeline 202 perform texture mapping using texture data stored in the storage resources 205. For example, the pixel shader 234 may read the texture data from the storage resources 205 and shade one or more pixels using the texture data. The shaded pixels are then provided to a display for presentation to a user. In some embodiments, the resource allocator 108 prevents computational resources of the unified shader pool 216 from being allocated for the implementation of a pixel shader, such as the pixel shader 234, based on whether one or more of the priority inversion heuristics 110 indicate a priority inversion between high priority geometry workloads and low priority pixel workloads.

[0023] 3 illustrates a portion 300 of a parallel processor having a resource allocator 108 configured to compute priority inversion heuristics to mitigate priority inversion of workloads in the parallel processor and selectively soft-lock computing resources based on the priority inversion heuristics 110. This example is implemented in some embodiments of the parallel processor 100 shown in FIG. 1, where like elements are referenced using like numerals.

[0024] As illustrated, the priority inversion heuristics 110 include one or more of an amount of received high priority workload 312, an amount of processing high priority workload 314, an amount of RT in the processing active workload 316, one or more ratios of low priority workload 318 to high priority workload, and an amount of received low priority workload 320. In some embodiments, the resource allocator 108 calculates or otherwise tracks the priority inversion heuristics 110 at least in part by monitoring the graphics queue 122 and one or more graphics pipelines 306. The resource allocator 108 further includes one or more allocation failure timers 322. In some cases, the resource allocator 108 determines whether priority inversion mitigation should be enabled (e.g., whether a soft lock of a computing resource should be enabled) based on whether one or more of the priority inversion heuristics 110 exceed or are otherwise outside of a range defined by one or more corresponding thresholds 324.

[0025] In some embodiments, resource allocator 108 monitors the received high priority workload 302 (in some embodiments, the geometry workload) in graphics queue 122 to calculate the amount of received high priority workload 312. A large amount of received high priority work indicates a backlog of high priority work that may be exacerbated by priority inversion. As such, in response to determining that the amount of received high priority workload 312 exceeds a predetermined one of thresholds 324, resource allocator 108 activates one or more soft lock signals 326, thereby enabling soft locking one or more computing resources to prevent allocation of those computing resources to process lower priority workloads.

[0026] In some embodiments, the resource allocator 108 monitors the in-flight high priority workload 308 being processed in the graphics pipeline 306 to determine the amount of in-flight high priority workload 314. A large amount of in-flight high priority work indicates a backlog of high priority work that may be exacerbated by priority inversion. As such, in response to determining that the amount of in-flight high priority workload 314 exceeds a predetermined one of the thresholds 324, the resource allocator 108 activates one or more soft lock signals 326, thereby enabling soft locking one or more computing resources to prevent allocation of those computing resources to process lower priority workloads.

[0027] The resource allocator 108 monitors the context state of each processing workload to determine the number of RTs in the active ones of the processing workloads 316. Some graphics workloads include multiple render targets, which causes an image to be rendered to multiple RT textures at once. When multiple RTs are being processed within an active workload, the likelihood of priority inversion between high priority geometry workloads and low priority pixel workloads increases. Therefore, in response to determining that the amount of RTs of an active one of the processing workloads 316 exceeds a predetermined one of the thresholds 324, the resource allocator 108 activates one or more soft lock signals 326, which allows soft locking one or more computing resources to prevent allocation of those computing resources to process low priority workloads.

[0028] In some embodiments, resource allocator 108 calculates one or more ratios 318 of low priority workload to high priority workload by monitoring some or all of the received high priority workload 302 and received low priority workload 304 of graphics queue 122 and the processing high priority workload 308 and processing low priority workload 310 of graphics pipeline 306. According to various embodiments, such ratios include one or more of a ratio of processing low priority workload 310 to processing high priority workload 308, a ratio of receiving low priority workload 304 to receiving high priority workload 302, or a ratio of both processing and receiving low priority workload to both processing and receiving high priority workload. Priority inversion may occur when the amount of received high priority workload and / or processing high priority workload is much greater than the amount of received low priority workload and / or processing low priority workload. As such, in response to determining that any of the ratios 318 of low priority workloads to high priority workloads is lower than a corresponding one of the thresholds 324, the resource allocator 108 activates one or more soft-lock signals 326, thereby enabling one or more computing resources to be soft-locked to prevent allocation of those computing resources to process low priority workloads.

[0029] In some embodiments, the resource allocator 108 starts the allocation failure timer 322 in response to a high priority workload failing to allocate. In some embodiments, the allocation failure timer 322 is activated only if a low priority workload is successfully allocated during the same allocation period during which the high priority workload failed to allocate, for example, because it indicates that the allocation failure of the high priority workload was potentially contributed to by a successful allocation of the low priority workload. The resource allocator 108 monitors the high priority workload that triggered the activation of the allocation failure timer 322 over the period between the start of the allocation failure timer 322 and its expiration. If, upon expiration of the allocation failure timer, the computational resource has not yet been successfully allocated to the corresponding high priority workload, the resource allocator 108 activates one or more soft lock signals 326, which allows soft locking one or more computational resources to prevent allocation of those computational resources to process low priority workloads. In some embodiments, an indication of whether a high priority workload has failed to be allocated during the allocation failure timer period is considered a priority inversion heuristic, and this indication is included in the priority inversion heuristic 110 .

[0030] In some embodiments, the resource allocator 108 monitors the received low priority workload 310 being processed in the graphics pipeline 306 to determine an amount of received low priority workload 320. A large amount of received low priority work indicates that there is no immediate need for priority inversion mitigation. For example, if a priority inversion occurs while there is a large amount of received low priority work, the priority inversion is unlikely to adversely affect the performance of the parallel processor 100 because, in this case, the priority inversion serves to clear a backlog of received low priority work. Thus, in response to determining that the amount of received low priority workload 320 exceeds a corresponding one of the thresholds 324, the resource allocator 108 is configured to prevent activation of the soft lock signal 326 even if one or more other priority inversion heuristics 110 are outside their respective threshold ranges (e.g., defined by one or more of the thresholds 324), thereby selectively preventing priority inversion mitigation via soft locking computational resources.

[0031] 4 illustrates an example process flow for a method 400 for selectively controlling allocation of computing resources to low priority workloads based on one or more priority inversion heuristics. In some embodiments, the method 400 is performed by executing computer readable instructions in a parallel processor. In this example, the method 400 is described in the context of an embodiment of the parallel processor 100 of FIG. 1, with like elements referenced using like numerals.

[0032] At block 402, the resource allocator 108 calculates or otherwise tracks one or more priority inversion heuristics 110. According to various embodiments, the priority inversion heuristics 110 include one or more ratios of the received high priority workload, the in-flight high priority workload, the received low priority workload, the in-flight workload, or the low priority workload to the high priority workload, respectively. In some embodiments, the resource allocator 108 manages one or more allocation failure timers, such as some embodiments of the allocation failure timer 322 of FIG. 3, as part of the priority inversion heuristics 110. For example, a predetermined allocation failure timer is activated in response to a failure to allocate a computing resource for a high priority workload and expires after a predetermined amount of time (measured in some examples by counting clock cycles on the allocation failure timer).

[0033] In some embodiments, the high priority workload is a geometry workload and the low priority workload is a pixel workload, hi some embodiments, the high priority workload is an asynchronous compute workload and the low priority workload is a graphics workload.

[0034] At block 404, the resource allocator 108 analyzes the priority inversion heuristics 110 to determine whether a soft-lock condition has been met. For example, each of the amounts and ratios of the priority inversion heuristics 110 is compared by the resource allocator 108 to a respective threshold 324, and any amount or ratio that is outside its corresponding threshold range indicates a priority inversion and therefore triggers a soft-lock condition.

[0035] In embodiments in which the resource allocator 108 includes an allocation failure timer 322, if a high priority workload is not successfully allocated at any time during the period defined by the allocation failure timer 322 (i.e., before the expiration of the allocation failure timer 322), this indicates a priority inversion and therefore triggers a soft-lock condition.

[0036] At block 406, if any soft-lock conditions are triggered at block 404, the method 400 proceeds to block 410. Otherwise, if no soft-lock conditions are triggered at block 404, the method 400 proceeds to block 408.

[0037] In some embodiments, method 400 automatically proceeds to block 408 if there is no low priority workload, regardless of whether the soft-lock condition is met. For example, in such embodiments, no priority inversion mitigation is performed if there is no low priority workload queued in the parallel processor, even if the analysis of the priority inversion heuristic 110 indicates a priority inversion.

[0038] At block 408, the resource allocator 108 allows applicable computing resources to be allocated to the low priority workload. The method 400 then returns to block 402 to continue periodically computing or otherwise tracking the priority inversion heuristics 110.

[0039] At block 410, resource allocator 108 soft-locks all or a subset of the computing resources (e.g., compute units 120 of shader engine 118) of parallel processor 100 from being allocated to low priority workloads. In some embodiments, resource allocator 108 activates one or more soft-lock signals (such as an embodiment of soft-lock signal 326) to initiate the soft-lock. Soft-locking computing resources in this manner mitigates priority inversion by increasing the availability of computing resources for processing high priority workloads while the computing resources are soft-locked against allocation to low priority workloads.

[0040] At block 412, the resource allocator 108 determines whether a softlock release condition has been met. According to various embodiments, such a softlock release condition includes any of the following: expiration of a corresponding softlock timer (such that the softlock is only enabled for a predetermined softlock period), a determination (e.g., by the resource allocator 108) that more than a predetermined threshold (e.g., of the threshold 324) of high priority workload is being processed, or the parallel processor 100 has been reset since activation of the softlock. In some embodiments, the softlock is maintained even if the softlock release condition would otherwise have been met, so long as one or more of the priority inversion heuristics 110 continue to indicate a priority inversion. If the softlock release condition has not been met, the method 400 returns to block 410 and the softlock is maintained. Otherwise, if the softlock release condition has been met, the method 400 proceeds to block 408, where the softlock is released and the associated computing resources are again allowed to be allocated for processing of lower priority workloads.

[0041] 5 illustrates an example process flow of a method 500 for selectively blocking allocation of computational resources to a low priority workload (e.g., a pixel workload) based on one or more priority reversal heuristics. For example, such blocking is triggered if a high priority workload (e.g., a geometry workload) fails to be allocated for a predefined period (defined via an allocation failure timer, such as some embodiments of the allocation failure timer 322 of FIG. 3) following an initial failure to allocate computational resources to the high priority workload, if the amount of received high priority workload exceeds a threshold (e.g., of threshold 324), if the amount of in-flight high priority workload exceeds a threshold (e.g., of threshold 324), if the amount of RT of an active workload of in-flight workload exceeds a threshold (e.g., of threshold 324), or if one or more ratios of high priority workload to low priority workload are lower than their corresponding thresholds (e.g., of threshold 324). In some embodiments, the method 500 is performed by executing computer readable instructions in a parallel processor. In this example, the method 500 is described in the context of an embodiment of the parallel processor 100 of Figure 1, and like elements are referenced using like numerals. In some embodiments, the method 500 corresponds to an embodiment of blocks 402, 404, 406, 410 of the method 400 of Figure 4.

[0042] In block 502, the resource allocator 108 detects an initial allocation failure for a high priority workload. In some embodiments, the high priority workload is a geometry workload, such as a geometry shader or a vertex shader.

[0043] At block 504, the resource allocator 108 starts an allocation failure timer, such as an embodiment of the allocation failure timer 322 of FIG. 3. The resource allocator 108 activates the allocation failure timer 322 in response to detecting (at block 502) a failure to allocate a computational resource to a high priority workload. In some embodiments, the initiation of the allocation failure timer 322 by the resource allocator 108 is further in response to the resource allocator 108 detecting that a computational resource was successfully allocated to a low priority workload during the same allocation period during which the high priority workload failed to be allocated. Once activated, the allocation failure timer 322 is configured to expire when a predetermined amount of time has passed, in some examples measured by counting clock cycles on the allocation failure timer (the period during which the allocation failure timer 322 is active (before expiration) may be referred to herein as the "allocation failure timer period"). The resource allocator 108 monitors the high priority workload during the allocation failure timer period to determine whether resources are successfully allocated to process the high priority workload during the allocation failure timer period (successful allocation) or whether resources are not successfully allocated to process the high priority workload during the allocation failure timer period (allocation failure).

[0044] At block 506, if the high priority workload fails to be allocated during the allocation failure timer period, method 500 proceeds to block 514. Otherwise, if the high priority workload is successfully allocated during the allocation failure timer period, method 500 returns to block 502 to monitor for initial allocation failures of other high priority workloads.

[0045] In some embodiments, multiple instances of blocks 502, 504, 506 are performed in parallel by the resource allocator 108 for multiple high priority workloads. In some embodiments, blocks 508, 510, 512 are performed in parallel with blocks 502, 504, 506.

[0046] At block 508, resource allocator 108 calculates or otherwise tracks values ​​for one or more priority inversion heuristics 110 based on the determined state of one or more graphics queues, such as graphics queue 122, one or more graphics pipelines, such as some embodiments of graphics pipeline 306 of Figure 3, or both the graphics queues and the graphics pipelines. In some embodiments, priority inversion heuristics 110 include one or more of the following (e.g., as described in the example of Figure 3): amount of received high priority workload 312, amount of processing high priority workload 314, amount of received low priority workload 320, amount of RT among active ones of processing workload 316, and one or more ratios 318 of low priority workload to high priority workload.

[0047] At block 510, resource allocator 108 compares the computed priority inversion heuristics 110 to each of thresholds 324. If at block 512 any priority inversion heuristics exceed or are otherwise outside the range defined by its corresponding one of thresholds 324, method 500 proceeds to block 514. Otherwise, the method returns to block 508 to recalculate priority inversion heuristics 110 (such that the computation of priority inversion heuristics 110 is repeated over time, and in some cases periodically).

[0048] In response to determining, at block 514, that either the high priority workload has failed to allocate during a corresponding allocation failure timer period or that the priority inversion heuristic 110 has exceeded or otherwise fallen outside a range defined by a corresponding one of the thresholds 324, the resource allocator 108 soft-locks all or a subset of the computing resources of the parallel processor (e.g., the compute units 120 of the shader engine 118) from being allocated to the low priority workload. For example, the resource allocator 108 activates one or more soft-lock signals (such as an embodiment of the soft-lock signal 326) to initiate a soft-lock of one or more computing resources of the system, preventing such computing resources from being allocated to process the low priority workload.

[0049] FIG. 6 illustrates an example process flow for a method 600 of selectively blocking allocation of computational resources to a low priority graphics workload based on a determination that a high priority asynchronous computation workload continues to fail allocation for a predetermined period of time immediately following an initial instance of the asynchronous computation workload failing allocation, which indicates a priority inversion between the high priority asynchronous computation workload and the low priority graphics workload. In some embodiments, the method 600 is performed by executing computer readable instructions in a parallel processor. In this example, the method 600 is described in the context of an embodiment of the parallel processor 100 of FIG. 1, with like elements being referenced using like numerals. In some embodiments, the method 600 corresponds to an embodiment of blocks 402, 404, 406, 410 of the method 400 of FIG. 4.

[0050] At block 602, the resource allocator 108 detects an initial allocation failure of the high priority asynchronous computation workload. At block 604, the resource allocator 108 starts an allocation failure timer, such as an embodiment of the allocation failure timer 322 of FIG. 3. The resource allocator 108 activates the allocation failure timer 322 in response to detecting (at block 602) the failure to allocate a computation resource to the high priority asynchronous computation workload. In some embodiments, the initiation of the allocation failure timer 322 by the resource allocator 108 is further in response to the resource allocator 108 detecting that a computation resource was successfully allocated to a low priority graphics workload during the same allocation period during which the high priority asynchronous computation workload failed to allocate. Once activated, the allocation failure timer 322 is configured to expire after a predetermined amount of time has passed (measured, in some examples, by counting clock cycles on the allocation failure timer). The period during which the allocation failure timer 322 is active (before expiration) is sometimes referred to herein as the “allocation failure timer period.” The resource allocator 108 monitors the high priority asynchronous computing workload during the allocation failure timer period to determine whether resources are successfully allocated to process the high priority asynchronous computing workload during the allocation failure timer period (successful allocation) or whether resources are not successfully allocated to process the high priority asynchronous computing workload during the allocation failure timer period (allocation failure).

[0051] At block 606, if the high priority asynchronous computation workload fails to be allocated during the allocation failure timer period, method 600 proceeds to block 608. Otherwise, if the high priority asynchronous computation workload is successfully allocated during the allocation failure timer period, method 600 returns to block 602 to monitor for initial allocation failures of other high priority asynchronous computation workloads.

[0052] At block 608, resource allocator 108 soft-locks all or a subset of the computing resources of the parallel processor (e.g., compute units 120 of shader engine 118) from being allocated to the low priority graphics workload. For example, resource allocator 108 activates one or more soft-lock signals (such as an embodiment of soft-lock signal 326) to initiate a soft-lock of one or more computing resources of the system, preventing such computing resources from being allocated to process the low priority graphics workload.

[0053] While this example is provided in the context of a scenario in which the graphics workload is a lower priority than the asynchronous computation workload, it is possible that the graphics workload may instead be designated as a higher priority than the asynchronous computation workload. For example, alternative embodiments of method 600 may be implemented where the graphics workload is designated as a higher priority than the asynchronous computation workload, in which case the roles of the asynchronous computation workload and the graphics workload in method 600 are switched. For example, in some such alternative embodiments of method 600, a determination of an initial allocation failure of the graphics workload starts an allocation failure timer at block 602, a failure of the graphics workload to be allocated during the allocation failure timer period is determined at block 606, and allocation of computing resources to the asynchronous computation workload is prevented at block 608.

[0054] In some embodiments, the above apparatus and techniques are implemented in a system that includes one or more integrated circuit (IC) devices (also referred to as integrated circuit packages or microchips), such as the parallel processors described above with reference to FIGS. 1-6. Electronic design automation (EDA) and computer aided design (CAD) software tools can be used in the design and manufacture of these IC devices. These design tools are typically represented as one or more software programs. The one or more software programs include code executable by a computer system for operating the computer system to operate on code representing the circuits of one or more IC devices to perform at least a portion of a process for designing or adapting a manufacturing system for manufacturing the circuits. This code may include instructions, data, or a combination of instructions and data. The software instructions representing the design or manufacturing tools are typically stored in a computer readable storage medium accessible to the computing system. Similarly, code representing one or more stages of the design or manufacture of the IC devices is stored in and accessed from the same computer readable storage medium or a different computer readable storage medium.

[0055] A computer-readable storage medium includes any non-transitory storage medium or combination of non-transitory storage media that can be accessed by a computer system during use to provide instructions and / or data to the computer system. Such storage media may include, but are not limited to, optical media (e.g., compact discs (CDs), digital versatile discs (DVDs), Blu-ray discs), magnetic media (e.g., floppy disks, magnetic tape, magnetic hard drives), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or micro-electromechanical systems (MEMS) based storage media. The computer-readable storage medium (e.g., system RAM or ROM) may be internal to the computing system, the computer-readable storage medium (e.g., a magnetic hard drive) may be permanently attached to the computing system, the computer-readable storage medium (e.g., an optical disk or Universal Serial Bus (USB)-based flash memory) may be removably attached to the computing system, or the computer-readable storage medium (e.g., network-accessible storage (NAS)) may be coupled to the computer system via a wired or wireless network.

[0056] In some embodiments, certain aspects of the techniques described above are implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied in a non-transitory computer-readable storage medium. The software may include instructions and specific data that, when executed by the one or more processors, operate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer-readable storage medium may include, for example, a magnetic or optical disk storage device, a solid-state storage device such as a flash memory, a cache, a random access memory (RAM), or other non-volatile memory device(s), etc. The executable instructions stored in the non-transitory computer-readable storage medium may be implemented as source code, assembly language code, object code, or other form of instructions that can be interpreted or otherwise executed by one or more processors.

[0057] In addition to the above, it should be noted that not all activities or elements described in the summary description are required, some of the specific activities or devices may not be required, one or more additional activities may be performed, and one or more additional elements may be included. Furthermore, the order in which the activities are listed is not necessarily the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, those skilled in the art will appreciate that various changes and modifications can be made without departing from the scope of the invention as set forth in the claims. Thus, the specification and drawings should be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the invention.

[0058] Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and features from which any benefit, advantage, or solution may arise or be manifested are not to be construed as critical, essential, or essential features of any or all of the claims. Moreover, the specific embodiments described above are illustrative only, as the disclosed invention may be modified and practiced in different but similar manners apparent to those skilled in the art having the benefit of the teachings herein. No limitations are intended to the details of construction or design herein shown, other than as described in the appended claims. It is therefore apparent that the specific embodiments described above may be altered or modified, and all such variations are considered to be within the scope of the disclosed invention. Accordingly, the protection sought herein is set forth in the appended claims.

Claims

1. 1. A method comprising: Computing at least one priority inversion heuristic indicative of priority inversions between high and low priority workloads of the parallel processor; selectively preventing allocation of at least one computing resource of the parallel processor to process the low priority workload based on the at least one priority reversal heuristic. method.

2. enabling allocation of the at least one computing resource to process the low priority workload in response to a release condition, the release condition including at least one of expiration of a timer, resetting the parallel processor, or an amount of the high priority workload being less than a threshold.

2. The method of claim 1.

3. the priority reversal heuristic includes at least one of: an amount of received high priority workload in a queue of the parallel processor; an amount of in-flight high priority workload in at least one pipeline of the parallel processor; an amount of received low priority workload in the at least one pipeline; an amount of render targets for the in-flight high priority workload and the received low priority workload; and one or more ratios of at least a subset of the low priority workload to at least a subset of the high priority workload; The method of claim 1 or 2.

4. initiating a timer in response to determining that an allocation for a high priority workload has failed; Selectively blocking allocation of the at least one computing resource includes: selectively preventing allocation of the at least one computing resource of the parallel processor for processing the low priority workload in response to determining that allocation for the high priority workload has failed for an entire timer period between initiation of the timer and expiration of the timer. The method of claim 1 or 2.

5. selectively blocking allocation of the at least one computing resource in response to successful allocation of at least one low priority workload. The method of claim 4.

6. the high priority workload is a geometry workload and the low priority workload is a pixel workload; 2. The method of claim 1.

7. the high priority workload is an asynchronous computation workload and the low priority workload is a graphics workload; 2. The method of claim 1.

8. 1. A parallel processor comprising: a graphics engine configured to receive a graphics workload via a graphics queue and to implement at least one graphics pipeline for processing the graphics workload; a resource allocator; The resource allocator: computing at least one priority reversal heuristic based on a graphics workload in at least one of the graphics queue or the at least one graphics pipeline, the at least one priority reversal heuristic indicating a priority inversion between a high priority workload and a low priority workload in the graphics workload; selectively blocking allocation of at least one computational resource to the lower priority workloads of the graphics workloads based on the at least one priority reversal heuristic; [0023] Parallel processors.

9. the high priority workload is a geometry workload and the low priority workload is a pixel workload; The parallel processor of claim 8.

10. a shader engine comprising at least one compute unit, the at least one compute resource comprising the shader engine comprising the at least one compute unit; The parallel processor of claim 8.

11. the priority reversal heuristics include at least one of: an amount of received high priority workload in the graphics queue; an amount of in-progress high priority workload in the at least one graphics pipeline; an amount of received low priority workload in the at least one graphics pipeline; an amount of render targets for the in-progress high priority workload and the received low priority workload; and one or more ratios of at least a subset of the low priority workloads to at least a subset of the high priority workloads. The parallel processor according to any one of claims 8 to 10.

12. the resource allocator is configured to enable allocation of the at least one computing resource to process the low priority workload in response to a release condition; the termination condition includes at least one of: expiration of a timer, resetting the parallel processor, or an amount of the high priority workload being less than a threshold. The parallel processor according to any one of claims 8 to 10.

13. The resource allocator: starting a timer in response to determining that an allocation for a high priority workload has failed, wherein the resource allocator is configured to selectively prevent allocation of the at least one computing resource in response to determining that an allocation for the high priority workload has failed for an entire timer period between starting the timer and expiry of the timer. The parallel processor according to any one of claims 8 to 10.

14. the resource allocator is configured to prevent allocation of the at least one computing resource in response to determining that the at least one priority inversion heuristic has exceeded a corresponding threshold. The parallel processor according to any one of claims 8 to 10.

15. The resource allocator: and configured to enable allocation of the at least one computing resource in response to determining that the graphics queue has less than a predetermined amount of low priority workload, regardless of whether the at least one priority reversal heuristic has exceeded the corresponding threshold.

15. The parallel processor of claim 14.

16. 1. A resource allocator for a parallel processor, comprising: The resource allocator: computing at least one priority inversion heuristic based on workloads in the queue, at least one pipeline, or both, the at least one priority inversion heuristic indicating priority inversions between high and low priority workloads among the workloads; selectively soft-locking at least one computing resource to temporarily prevent the at least one computing resource from being allocated to process the lower priority workloads of the workloads based on the at least one priority inversion heuristic; [0023] Resource allocator.

17. the priority reversal heuristics include at least one of: an amount of received high priority workload in a queue; an amount of in-progress high priority workload in the at least one pipeline; an amount of received low priority workload in the at least one pipeline; an amount of render targets for the in-progress high priority workload and the received low priority workload; and one or more ratios of at least a subset of the low priority workload to at least a subset of the high priority workload; 17. The resource allocator of claim 16.

18. The resource allocator: starting a timer in response to determining that an allocation for a high priority workload has failed, wherein the resource allocator is configured to selectively and temporarily soft-lock the at least one computing resource in response to determining that an allocation for the high priority workload has failed for an entire timer period between starting the timer and expiry of the timer.

18. A resource allocator according to claim 16 or 17.

19. the high priority workload is a geometry workload and the low priority workload is a pixel workload; 20. The resource allocator of claim 18.

20. the high priority workload is an asynchronous computation workload and the low priority workload is a graphics workload; 20. The resource allocator of claim 18.

Images