Throttling while managing upstream resources

Abstract

Various systems, apparatuses, and methods are disclosed for arbitrating threads in a computing system. The computing system includes a processor with multiple cores, each capable of simultaneously processing instructions for multiple threads. When a thread throttling unit receives an indication that a shared cache has resource contention, the throttling unit sets a threshold number of cache misses for the cache. If the number of cache misses exceeds this threshold, the throttling unit notifies a particular upstream computing unit to throttle processing of instructions for the thread. If, after a period of time has elapsed, the cache continues to exceed the threshold, the throttling unit notifies the upstream computing unit to throttle the thread more restrictively by performing one or more of decreasing the selection ratio and increasing the period of time. Otherwise, the unit notifies the upstream computing unit to throttle the thread less restrictively.

Description

[Background technology]

[0001] Processors, such as general-purpose central processing units, process software application instructions through any number of pipeline stages. These processors include hardware for issuing, dispatching, executing, and retiring multiple instructions, additional routing and logic to determine simultaneous data transfers for multiple instructions per clock cycle, and complex branch prediction schemes. Many processors also replicate processor cores multiple times to increase the parallel execution of software applications. To further exploit instruction-level parallelism, processor cores perform simultaneous multithreading.

[0002] Depending on the workload, two or more threads in a processor core may compete for shared resources, such as an instruction fetch unit (IFU), a branch prediction unit, reservation stations that issue instructions to an integer execution unit or floating-point unit (FPU), load and store queues in a load-store unit (LSU), a common data bus that transmits the results of executed instructions, or other processor cores. Threads across two or more processor cores may also compete for shared resources, such as level 2 (L2) or level 3 (L3) caches, memory buses, input / output (I / O) devices, etc.

[0003] Resource contention occurs when the number of computing units requesting access to a shared resource exceeds the number of units the shared resource can support for simultaneous access. Examples of computing units are hardware threads, processor cores, microprocessors, or others. Resource contention typically results in a multi-cycle stall. The duration of a resource contention stall depends on the amount of time granted to one or more other computing units currently accessing the shared resource. Stalls due to resource contention reduce the benefits of replicating cores or other computing units capable of multithreaded execution.

[0004] In view of the above, an efficient method for arbitrating threads in a computing system is desirable.

[0005] The advantages of the methods and mechanisms described herein may be better understood by referring to the following description in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 2 is a block diagram of one embodiment of a processor core. [Figure 2] FIG. 1 is a block diagram of one embodiment of control logic for thread arbitration in a computing system. [Figure 3] 1 is a flowchart of one embodiment of a method for thread arbitration in a computing system. [Figure 4] 1 is a flowchart of one embodiment of a method for thread arbitration in a computing system. [Figure 5] 1 is a flowchart of one embodiment of a method for thread arbitration in a computing system. [Figure 6] FIG. 1 is a block diagram of one embodiment of a microprocessor. DETAILED DESCRIPTION OF THE INVENTION

[0007] While the invention is susceptible to various modifications and alternative forms, specific embodiments have been shown by way of example in the drawings and are herein described in detail. It should be understood, however, that the drawings and detailed description thereto are not intended to limit the invention to the particular forms disclosed, but on the contrary, the invention is intended to cover all modifications, equivalents, and alternatives falling within the scope of the invention as defined by the appended claims.

[0008] In the following description, numerous specific details are set forth to provide a thorough understanding of the methods and mechanisms presented herein. However, those skilled in the art will recognize that various embodiments can be practiced without the use of such specific details. In some instances, well-known structures, components, signals, computer program instructions, and techniques have not been shown in detail to avoid obscuring the approaches described herein. It should be understood that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some elements may be exaggerated relative to other elements.

[0009] Various systems, apparatuses, methods, and computer-readable media for arbitrating threads in a computing system are disclosed. In various embodiments, the computing system includes multiple shared resources. Examples of resources are various types of computational units used in a processor core at different stages of instruction processing. The stages include an instruction fetch stage, a decode stage, a dispatch stage, and an issue stage. A multithreaded operating system or hypervisor divides a software application into multiple processes, each of which has multiple threads. A scheduler in the operating system or hypervisor manages the assignment of these different software threads to hardware, such as specific cores of multiple processor cores in a computing system. Multiple threads in a specific core share computational units and data storage queues in the core. Another example of a resource is a memory controller and data storage array in a hierarchical memory subsystem shared by multiple cores. A core includes one or more local cache levels of a cache hierarchy shared by multiple threads in the core, and the computing system includes one or more global cache levels shared by multiple cores.

[0010] Resource contention occurs when the number of sources requesting access to a shared resource exceeds the number of sources the shared resource supports for simultaneous access. Examples of sources are threads within a core, cores, etc. A source attempting to utilize a shared resource but not being granted access stalls. To reduce stall penalties in a computing system, one or more software and hardware monitors determine whether a particular source (e.g., thread, core, etc.) is eligible for throttling or reducing instruction processing in the shared resource. However, throttling the shared resource may stall other resources upstream from the shared resource or reduce performance because the upstream resource is already being utilized by the stalled resource. As used herein, "software monitor" includes any of a variety of software used to monitor resource utilization by software threads, and identifies specific threads for throttling, such as operating systems, hypervisors, basic input / output software (BIOS) control functions, user-level software utilizing application programmer interfaces (APIs), and any other type of software capable of manipulating the configuration of a computing system.

[0011] While moving the throttling point upstream from the shared resource frees some of the upstream resources from resource contention, if the point is moved too far upstream, the thread's performance may fall below a threshold and decrease. Also, the original shared resource may continue to experience resource contention. In an embodiment, the original contended shared resource is the core level 2 (L2) cache. Other levels of the cache hierarchy are possible and considered. Throttling arbitration for a thread upstream in instruction processing from the L2 cache does not guarantee a reduction in the number of memory requests conveyed by the L2 cache. The L2 cache receives memory requests from the L1 data cache and the L1 instruction cache. The L2 cache controller also has the ability to generate prefetch requests for one or more miss requests sent to lower-level memory, such as a level 3 (L3) cache or system memory. To balance supporting throttling of the local L2 cache and maintaining a specific memory bandwidth in the L2 cache, the core's control logic monitors the number of cache misses in the L2 cache and sets a threshold number of cache misses for a thread in the L2 cache at a specific cache level.

[0012] When the control logic of the core determines and receives an indication specifying that the number of cache misses for a particular thread in the L2 cache is greater than a threshold number of cache misses, the control logic notifies a particular upstream compute unit to throttle processing of instructions for the thread. In one embodiment, the upstream compute unit is a dispatch unit. In another embodiment, the upstream compute unit is an instruction fetch unit, a decode unit, a load / store unit (LSU), or the like.

[0013] In various embodiments, the control logic of the core sets a throttling severity level for an upstream compute unit, which specifies a selection rate for selecting instructions of a thread for processing in the compute unit. The severity level also specifies a time period for maintaining the selection rate. When the time period has elapsed, the control logic updates the severity level to a more restrictive level of throttling if the number of cache misses in the L2 cache is still greater than a threshold number of cache misses. However, if the number of cache misses at a particular cache level is less than or equal to the threshold number of cache misses, the control logic updates the severity level to a less restrictive level of throttling.

[0014] Referring to FIG. 1, one embodiment of a general-purpose processor core 100 that performs out-of-order execution is shown. In one embodiment, processor core 100 processes two or more threads simultaneously. An instruction cache (i-cache) stores instructions for software applications, and a corresponding translation lookaside buffer (TLB) stores addresses for accessing the instructions. An instruction fetch unit (IFU) 104 fetches multiple instructions from i-cache 102 per clock cycle when there is no i-cache miss. IFU 104 includes a program counter that is compared with an address in the i-TLB and holds a pointer to the address of the next instruction to fetch in i-cache 102. IFU 104 also includes a branch prediction unit (not shown) that predicts the outcome of conditional instructions before the execution unit determines the actual outcome in later pipeline stages.

[0015] Decoder unit 106 decodes the opcodes of multiple fetched instructions and allocates entries to an in-order retirement queue, such as reorder buffer 118, reservation stations 108, and load / store unit 114. In some embodiments, decode unit 106 performs register renaming of instruction operands to eliminate write-after-read (WAR) and write-after-write (WAW) hazards. In other embodiments, reorder buffer 118 performs register renaming. An instruction is allocated to a reservation station 108 if a read-after-write (RAW) hazard is eliminated for the instruction. The allocation of entries to reservation stations 108 is considered dispatch. Reservation stations 108 act as an instruction queue where instructions wait until their operands are available. If operands are available and hardware resources are also available, logic in reservation station 108 issues instructions out of order to integer and floating point functional units 110 or load / store unit 114 .

[0016] Functional units 110 include an arithmetic logic unit (ALU) for computer calculations such as addition, subtraction, multiplication, division, and square root. Logic may be included to determine the results of conditional instructions. Load / store unit (LSU) 114 receives issued memory accesses, such as load and store operations. Load / store unit 114 includes queues and logic for executing memory access instructions. In an embodiment, load / store unit 114 includes verification logic to ensure that load instructions receive data forwarded from the correct youngest store instruction.

[0017] Functional units 110 and load / store units 114 present results on common data bus 112. Reorder buffer 118 receives the results from common data bus 112. In one embodiment, reorder buffer 118 is a first-in-first-out (FIFO) queue that ensures in-order retirement of instructions according to program order, where an instruction that receives its result is marked for retirement. If the instruction is at the head of the queue, logic in reorder buffer 118 sends the result to register file 120, which maintains the architectural state of the general-purpose registers of processor core 100. The instruction in reorder buffer 118 is then retired in order, and logic updates its head of queue pointer to point to the subsequent instruction in program order.

[0018] The results on common data bus 112 are also sent to reservation stations 108 to forward values to operands of instructions awaiting the results. When those awaiting instructions have values for their operands and hardware resources are available to execute the instructions, the instructions can be issued out of order from reservation stations 108 to the appropriate resources in functional units 110 or load / store unit 114. Uncommitted or unretired memory access instructions have entries in the load / store unit. Forwarded data values for in-flight or uncommitted load instructions from the youngest uncommitted older store instruction are placed on common data bus 112 or simply routed to the appropriate entry of a load buffer in load / store unit 114.

[0019] The load / store unit 114 sends memory access requests to an on-chip level 1 (L1) data cache (d-cache) 116. In some embodiments, the core 100 includes a level 2 (L2) cache 130 for servicing memory requests from the L1 data cache 116 and the instruction cache 102, which is also the L1 cache. Each of the L1 data cache 116 and the L2 cache 130 has its own TLB for address comparison with received memory requests. The cache controllers of the L1 data cache 116 and the L2 cache 130 search the data arrays in a serial or parallel manner. If the requested memory line is not found in the L1 data cache 116 or the instruction cache 102, the corresponding cache controller sends a miss request to the L2 cache 130.

[0020] If the requested memory line is not found in the L2 cache 130, the L2 cache controller sends a miss request to access memory in a lower-level memory, such as a level 3 (L3) cache or system memory. In various embodiments, the core 100 and other external cores share the lower-level memory. The wait time for the requested memory line to arrive from the lower-level memory typically requires a significant number of clock cycles. To reduce performance degradation due to this wait time, in some embodiments, the L2 cache controller generates one or more prefetch requests in response to one or more miss requests. The L2 cache controller updates the number of prefetch requests based on factors such as the number of accesses in the address range, the amount of time since the address range was accessed, the process identifier (ID) or thread ID, and the priority level. This information is unknown to the cache controllers of the L1 data cache 116, the instruction cache 102, and the load / store unit 114.

[0021] Multiple threads share multiple resources within core 100. For example, these multiple threads share each of blocks 102-130 shown in FIG. 1 . In an embodiment, hardware monitor 140 collects data to support tuning and understanding the behavior and hardware resource utilization of software applications on core 100. Additionally, some events, such as peak memory contention or response time to invoke an interrupt handler, are inconvenient to measure in software. Hardware, such as circuitry in hardware monitor 140, extends the variety and detail of measurements with little or no impact on application execution. Based on information provided by hardware monitor 140 and one or more of software monitor 142, thread throttling unit 150 manages instruction throttling of one or more of blocks 102-130.

[0022] In one embodiment, hardware monitor 140 includes counters for counting the number of processor cycles required to perform pipeline flushes, data cache snoops and snoop hits, cache and TLB misses and miss ratios, read and write operations, data cache lines written back, branch operations, branch operations taken, the number of instructions in the integer or floating-point pipeline, bus utilization, and utilization of various queues. As described above, software monitor 142 includes one of a variety of software used to monitor resource utilization by software threads and identify specific threads to be throttled. Examples of software monitor 142 are operating systems, hypervisors, basic input / output software (BIOS) control functions, user-level software utilizing an application programmer interface (API), and any type of software capable of adjusting the configuration of a computing system. Each of hardware monitor 140, software monitor 142, and thread throttling unit 150 is shown for illustrative purposes as being located outside of processing blocks 102-130 of processor core 100. In other embodiments, the functionality of one or more of hardware monitor 140, software monitor 142, and thread throttling unit 150 is integrated throughout the floorplan of core 100. Alternatively, portions of hardware monitor 140 and thread throttling unit 150 reside both within core 100 and outside of core 100. All such combinations are contemplated.

[0023] The functionality of thread throttling unit 150 may be implemented in hardware, such as circuitry, one of various types of software, or both. In various embodiments, thread throttling unit 150 receives an indication specifying that one or more shared resources corresponding to any of blocks 102-130 are utilized by a particular thread above a threshold utilization. One or more of hardware monitor 140 and software monitor 142 transmit the indication to thread throttling unit 150. In one embodiment, the identified shared resource is L2 cache 130, which has one or more queues shared by multiple threads. In an embodiment, thread throttling unit 150 sets a threshold number of cache misses for a particular thread in L2 cache 130. Thereafter, when thread throttling unit 150 determines and receives an indication specifying that the number of cache misses for a particular thread in L2 cache is greater than the threshold number of cache misses, thread throttling unit 150 notifies a particular upstream compute unit to throttle processing of instructions for the particular thread. In one embodiment, the upstream compute unit is a reorder buffer (dispatch unit) 118. In another embodiment, the upstream compute unit is an instruction fetch unit 104, a decode unit 106, a load / store unit (LSU) 114, or the like.

[0024] Although the reorder buffer 118 is used in the following description, in other embodiments, a different upstream compute unit is used. In various embodiments, the thread throttling unit 150 sets a throttling severity level for the reorder buffer 118, which specifies a selection ratio for selecting instructions of a thread for processing in the reorder buffer 118. The severity level also specifies a time period for maintaining the selection ratio. The severity level is also referred to as a level of throttling. After the time period has elapsed, the thread throttling unit 150 updates the severity level to a more restrictive level of throttling if the number of cache misses in the L2 cache 130 is still greater than a threshold number of cache misses. However, if the number of cache misses in the L2 cache 130 is equal to or less than the threshold number of cache misses, the thread throttling unit 150 updates the severity level to a less restrictive level of throttling. A more restrictive level of throttling has a selection ratio that is less than the current selection ratio. In some embodiments, the time period corresponding to the more restrictive level of throttling is longer than the current time period. In contrast, a less restrictive level of throttling has a selection ratio that is greater than the current selection ratio. In some embodiments, the time period corresponding to the less restrictive level of throttling is less than the current time period.

[0025] In one embodiment, one or more of hardware monitor 140 and thread throttling unit 150 include monitoring output pins so that the data can be accessed by the operating system, another software monitor 142, a software programmer, or the like. The output pins toggle after a predetermined event, counter overflow, status information, or the like. Wiring one of these pins to an interrupt pin allows software to react to the reported data. In another embodiment, specific instructions are included in the instruction set architecture (ISA) to respectively disable and enable data collection and to read one or more specific registers in one or more of hardware monitor 140 and thread throttling unit 150.

[0026] In some embodiments, kernel-level support is required to access these registers. For example, a program must be in supervisor mode and execute a system call to access these registers. In yet another embodiment, the operating system provides one or more application programming interfaces (APIs) that correspond to one or more of hardware monitor 140, software monitor 142, and thread throttling unit 150. The APIs allow for programming configurable thresholds, thread arbitration ratios, and time periods that correspond to severity levels of resource contention.

[0027] Referring to FIG. 2, a generalized block diagram of one embodiment of a thread throttling unit 200 is shown. Thread throttling unit 200 (or unit 200) includes control and status registers 210, control logic 220, and tables 230. Control logic 220 may be implemented in hardware, such as circuitry, software, such as firmware, or both. The control and status registers receive information from one or more software and hardware monitors. Examples of information include an indicator designating a particular thread requiring throttling, an identification of a shared resource experiencing resource contention, a selection ratio and time period to use during thread arbitration for the identified shared resource, another indicator of the measured utilization of the resource contention or the severity level of the resource contention, and other thresholds.

[0028] Additionally, control and status register 210 receives information from a particular shared resource. One example of a shared resource is a particular cache level in a multi-level cache hierarchy. Table 230 includes multiple entries 240A-240D. One of a variety of data storage structures may be used to implement table 230. Example structures include flip-flop registers, various random access memories (RAMs), and content addressable memories (CAMs). As shown, each of entries 240A-240D includes multiple fields. While fields 250-264 are shown contiguous, other embodiments use a different number of fields and storage arrangements that differ from those shown, possibly using non-contiguous storage between some fields.

[0029] Control logic 220 or control logic 220 in table 230 allocates entries 240A-240D when a thread requiring throttling is identified. Field 250 stores a thread identifier identifying the thread, and field 252 stores the process identifier of the process from which the thread originated. If the particular shared resource experiencing resource contention is a cache level, field 254 stores the threshold number of cache misses for the cache level. Field 256 stores an identifier identifying the throttled compute unit. This compute unit processes instructions upstream from the shared resource experiencing resource contention. In one embodiment, the shared resource experiencing resource contention is an in-core L2 cache, and the upstream compute unit being throttled is an in-core dispatch unit. Field 258 stores an age indicator for the table entry.

[0030] Field 260 stores a severity level that indicates how strongly the thread needs to be throttled. In one embodiment, the severity level is based on a utilization measurement of the shared resource, the throughput or bandwidth of the shared resource, and the duration of the resource contention. One or more of an external software monitor, an external hardware monitor, and control logic 220 set the severity level. Field 262 stores a selection ratio for storing instructions of the thread for processing on a particular upstream compute unit. Field 264 stores an arbitration period for maintaining the selection ratio. Fields 262 and 264 are set by the severity level stored in field 260. In various embodiments, the values stored in fields 262 and 264 are read from registers within control and status registers 210.

[0031] Referring to Figure 3, one embodiment of a method 300 for arbitrating threads in a computing system is shown. For purposes of illustration, the steps in this embodiment (and in Figures 4-5) are shown sequentially. However, it should be noted that in various embodiments of the described method, one or more of the described elements may be performed simultaneously, in a different order than shown, or omitted entirely. Other additional elements may also be performed as needed. Any of the various systems or devices described herein may be configured to perform method 300.

[0032] Multithreaded software applications create multiple software processes within the same application. A multithreaded operating system divides a software application into multiple processes, each of which has multiple threads. A scheduler within the operating system manages the allocation of these different software threads to hardware, such as specific cores of a computing system's multiple processor cores. One or more of the hardware and software monitors the multiple threads being processed (block 302). For example, the operating system's scheduler may do the monitoring. Other types of software monitors include at least a hypervisor, basic input / output software (BIOS) control functions, and user-level software that utilizes an application programmer interface (API). Similarly, a dedicated hardware monitor in a processor core may measure contention for resources shared by multiple threads on the core.

[0033] One or more of a software monitor and a hardware monitor can determine whether a thread qualifies as a candidate for throttling. In embodiments, an operating system or other software manages a structurally defined quality of service (QoS) target for memory bandwidth for a particular thread. The particular thread is generated from a software application process that must meet the QoS memory bandwidth requirement. When another thread consumes a resource, such as queue occupancy or bus utilization, preventing the other particular thread from meeting the QoS memory bandwidth requirement, the operating system or other software identifies the resource-consuming thread as a candidate for throttling. Similarly, in some embodiments, a hardware monitor compares resource consumption, such as queue occupancy, bus utilization, or the like, with a threshold. When the particular thread exceeds one or more thresholds, the hardware monitor identifies the resource-consuming thread as a candidate for throttling. When one or more of the operating system and hardware determine that the particular thread is a candidate for throttling, an indication is sent to control logic within the core.

[0034] If a thread in the core is not a candidate for throttling (conditional block 304: "NO"), control flow of method 300 returns to block 302. However, if any of the multiple threads in the core is a candidate for throttling (conditional block 304: "YES"), the control logic of the core sets a throttling severity level for the thread (block 306). The control logic sets a threshold number of misses for the thread at a particular cache level based on the severity level (block 308). In an embodiment, the particular cache level is one in which the number of memory access requests generated by the particular cache level (particularly those due to speculative prefetch requests) is unknown to higher levels of the cache hierarchy. For example, the cache controller generates one or more prefetch requests in response to one or more miss requests, so the actual number of memory requests issued from the particular cache level is unknown to higher levels of the cache hierarchy.

[0035] In one embodiment, either the software monitor or the hardware monitor determines a threshold number of misses for the thread at a particular cache level. If, during processing of the thread's instructions, the number of cache misses at a particular cache level is less than or equal to the threshold number (conditional block 310: "No"), the control logic continues processing the thread (block 312) while one or more of the control logic and the cache controller monitor the number of cache misses at the particular cache level. If the number of cache misses at a particular cache level is greater than the threshold number (conditional block 310: "Yes"), the control logic throttles the processing of the thread at a stage of processing upstream from the particular cache level (block 314).

[0036] Referring to FIG. 4, one embodiment of a method 400 for arbitrating threads in a computing system is shown. A processor core processes instructions for multiple threads. The core's control logic identifies threads that qualify as candidates for throttling and that have not exceeded their miss threshold at a particular cache level (block 402). This identification at block 402 corresponds to the control flow of the "No" branch of block 310 to block 312 of method 300 (of FIG. 3). If the control logic determines that the thread does not remain a candidate for throttling (conditional block 404: "No"), the control logic removes the threshold number of misses for the thread (block 406). For example, the control logic and / or one or more of the cache controller at the particular cache level reset the threshold number to a default maximum or disable the threshold number for the thread. In an embodiment, one or more of the software monitor and / or hardware monitor determine that the thread is no longer a candidate and notify the control logic. In another embodiment, the control logic determines that the threshold number of cache misses has not been exceeded for a duration that exceeds the limit. After eliminating the threshold number of cache misses for the thread, the core continues to process the thread (block 408).

[0037] If the control logic determines that the thread remains a candidate for throttling (conditional block 404: "yes") but is no longer active because it has completed (conditional block 410: "no"), then method 400 is complete (block 412). If the control logic determines that the thread remains a candidate for throttling (conditional block 404: "yes") and that the thread is still active (conditional block 410: "yes"), then the logic determines whether the thread has caused a number of cache misses at a particular cache level that exceeds its threshold number of cache misses.

[0038] If the number of cache misses for the thread at a particular cache level is less than or equal to the threshold number of misses (conditional block 414: "No"), control flow of method 400 returns to block 402, where instructions for the thread continue to be processed while the thread does not exceed its threshold number of cache misses at a particular cache level. If the number of cache misses for the thread at a particular cache level is greater than the threshold number of misses (conditional block 414: "Yes"), the core processes the thread by throttling it at a stage of processing upstream from the particular cache level (block 416). Further details on throttling are provided in method 500.

[0039] Referring to FIG. 5, one embodiment of a method 500 for arbitrating threads in a computing system is shown. A processor core processes instructions for multiple threads. The core's control logic identifies threads that qualify as candidates for throttling and that have not exceeded their miss threshold at a particular cache level (block 502). The core's control logic selects a stage of processing upstream from the particular cache level (block 504). A compute unit is referred to as an "upstream" compute unit when it processes instructions at a stage of instruction processing before data is accessed at the particular cache level. In one embodiment, the control logic notifies a dispatch unit that is upstream in the processing of instructions from the particular cache level. In another embodiment, the control logic notifies another upstream compute unit, such as an instruction fetch unit, a decode unit, or a load / store unit (LSU).

[0040] The core's control logic selects an initial severity level of throttling for threads in a selected upstream stage of processing (block 506). The control logic selects a selection ratio (or arbitration ratio) to use during arbitration of multiple threads for access to a shared resource. The control logic also selects a period based on the severity level (block 508). In an embodiment, the control logic sets the selection ratio to be a maximum of a second number of clock cycles to a first number of instructions. Each of the first and second numbers is stored in a programmable configuration register, such as a control and status register. In one embodiment, the second number of clock cycles is an exponential value based on the severity level. An example of the second number of clock cycles is the square of the severity level indication. Thus, a severity level of 1 results in a second number of clock cycles of 21 or 2 cycles. A more restrictive severity level of 2 results in a second number of clock cycles of 22 or 4 cycles. A more restrictive severity level of 3 results in a second number of clock cycles of 23 or 8 cycles, and so on. In an embodiment, the selected period for using the selected ratio is an integer multiple of the second number: for a severity level of 3, the second number of clock cycles is 8 cycles, an integer multiple of 5, and the period is 8 x 5 or 40 cycles.

[0041] The core throttles the thread in the selected upstream stage of processing based on the selected arbitration ratio (block 510). If the time period has not elapsed (conditional block 512: "No"), control flow of method 500 returns to block 510, where the core continues throttled processing of the thread. If the time period has elapsed (conditional block 512: "Yes") but the thread has not caused more than a threshold number of cache misses at a particular cache level (conditional block 514: "No"), the control logic selects a less restrictive severity level of throttling for the thread in the selected upstream stage of processing (block 518). Control flow of method 500 then returns to block 508, where the control logic selects an arbitration ratio and a time period based on the severity level.

[0042] If the time period has elapsed (conditional block 512: "YES") and the thread has caused more than a threshold number of cache misses at a particular cache level (conditional block 514: "YES"), the control logic selects a more restrictive severity level of throttling for the thread at the selected upstream stage of processing (block 518). Control flow of method 500 then returns to block 508, where the control logic selects an arbitration rate and time period based on the severity level.

[0043] Referring to FIG. 6, one embodiment of an exemplary microprocessor 600 is shown. Microprocessor 600 includes a memory controller 620 coupled to memory 630, interface logic 640, one or more processing units 615A-615B including one or more processor cores 612A-612B and corresponding cache memory subsystems 614A-614B, crossbar interconnect logic 616, a shared cache memory subsystem 618, and a shared graphics processing unit (GPU) 650. Memory 630 is shown to include operating system code 632. Operating system 632 includes a scheduler 634 for assigning software threads to hardware in microprocessor 600. Memory 620 also includes a software monitor 636. Examples of software monitor 636 are a hypervisor, basic input / output software (BIOS) control functions, user-level software utilizing an application programmer interface (API), and any other type of software capable of adjusting the configuration of microprocessor 600. It should be noted that various portions of the operating system code 632 and software monitor 636 may reside in memory 630, may reside in one or more caches (614A-614B, 618), may be stored on non-volatile storage such as a hard disk (not shown), etc. In one embodiment, the functionality of the illustrated microprocessor 600 is incorporated onto a single integrated circuit.

[0044] GPU 650 may include one or more graphics processor cores and data storage buffers dedicated to the graphics rendering device for a personal computer, workstation, or video game console. Interface 640 generally provides an interface for input / output (I / O) devices to the shared cache memory subsystem 618 and processing units 615A-615B away from microprocessor 600. I / O devices include peripheral network devices such as printers, keyboards, monitors, cameras, card readers, hard or floppy disk drives or drive controllers, network interface cards, video accelerators, sound cards, modems, and various data acquisition cards.

[0045] The I / O devices are shared by each of the processing units 615A-615B of the microprocessor 600 and the GPU 650. These I / O devices are also shared by the processing units 615A-615B in the other microprocessors. An interface 640 is used to communicate with these other microprocessors and / or other processing nodes. Generally, the interface logic 640 includes buffers for receiving packets from the corresponding link and buffering packets to be transmitted on the corresponding link. Any suitable flow control mechanism for transmitting packets to and from the microprocessor 600 may be used, and the interconnect logic 616 supports the mechanism.

[0046] Microprocessors 600 are connected to their respective memories through respective memory controllers 620. The memories may use any of a variety of types of memory devices. The address space of microprocessor 600 is divided among multiple memories. Each microprocessor 600 or each processing node that includes microprocessor 600 uses a memory map that is used to determine which addresses map to which memories, and thus to which microprocessor 600 or processing node a memory request for a particular address is routed. In one embodiment, the coherency point for an address is memory controller 620 connected to the memory that stores the bytes corresponding to the address. Memory controller 620 includes control circuitry for interfacing with the memories and request queues to query memory requests.

[0047] The cache memory subsystems 614A-614B, 618 use high-speed cache memory for storing blocks of data. In some embodiments, the cache memory subsystems 614A-614B are integrated within each processor core 612A-612B. Alternatively, the cache memory subsystems 614A-614B are connected to the processor cores 612A-612B in a backside cache configuration or an inline configuration, as desired. In various embodiments, the cache memory subsystems 614A-614B are implemented as a hierarchy of caches. The caches of the closer processor cores 612A-612B (in the hierarchy) are integrated within the processor cores 612A-612B, as desired. In one embodiment, each of the cache memory subsystems 614A-614B represents an L2 cache structure, and the shared cache subsystem 618 represents an L3 cache structure.

[0048] Processor cores 612A-612B include circuitry for executing instructions according to a predetermined general-purpose instruction set, and processor cores 612A-612B support simultaneous multithreading. In various embodiments, processor cores 612A-612B have functionality equivalent to processor core 100 (of FIG. 1). Similarly, thread throttling units 613A-613B (or TTUs 613A-613B) have functionality identical to thread throttling unit 150 (of FIG. 1) and thread throttling unit 200 (of FIG. 2). Multiple threads executed by processor cores 612A-612B share at least a shared cache memory subsystem 618, a graphics processing unit (GPU) 650, and associated I / O devices.

[0049] In various embodiments, program instructions of a software application are used to implement the methods and / or mechanisms described above. The program instructions describe the operation of hardware in a high-level programming language such as C. Alternatively, a hardware design language (HDL) such as Verilog is used. The program instructions are stored on a non-transitory computer-readable storage medium. Many types of storage media are available. The storage medium is accessible by a computing system during use and provides the program instructions and associated data to the computing system for program execution. The computing system includes at least one or more memories and one or more processors that execute the program instructions.

[0050] It should be emphasized that the above-described embodiments are merely non-limiting examples of embodiments. Many variations and modifications will become apparent to those skilled in the art once the above disclosure is fully appreciated. It is intended that the following claims be interpreted to encompass all such variations and modifications.

Claims

1. a plurality of computational units each configured to process instructions; a cache configured to store data; a control circuit; The control circuit receiving an indication that a given thread is a candidate for throttling; In response to identifying the predetermined thread from a plurality of threads as a candidate for throttling, setting a threshold number of cache misses for the predetermined thread; throttling a number of instructions processed by the predetermined thread in a predetermined computing unit among the plurality of computing units in response to determining that the number of cache misses for the predetermined thread is greater than the threshold number of cache misses; configured to: Device.

2. the given computing unit processes instructions at a stage of instruction processing before data is accessed in the cache; 10. The apparatus of claim 1.

3. The control circuit a selection ratio for selecting instructions of the given thread for processing in the given computational unit; a period of time for maintaining the selection ratio; configured to set a level of throttling, including 10. The apparatus of claim 1.

4. The control circuit The period has elapsed; and after the period of time has elapsed, the number of cache misses in the cache is greater than the threshold number of cache misses; and updating the level of throttling to a more restrictive level of throttling by performing one or more of: decreasing the selection ratio; and increasing the time period in response to determining 4. The apparatus of claim 3.

5. The control circuit The period has elapsed; and after the period of time has elapsed, the number of cache misses in the cache is less than or equal to the threshold number of cache misses; and updating the level of throttling to a less restrictive level of throttling by performing one or more of increasing the selection ratio and decreasing the time period in response to determining 4. The apparatus of claim 3.

6. the control circuitry is configured to allow the cache to handle a number of cache misses greater than the threshold number of cache misses.

10. The apparatus of claim 1.

7. The indicator is generated based at least in part on a quality of service of any one of the plurality of threads.

10. The apparatus of claim 1.

8. the instruction processing stage includes any one of an instruction fetch stage, a decode stage, a dispatch stage, and an issue stage; 3. The apparatus of claim 2.

9. a plurality of computational units processing instructions; the cache storing the data; receiving an indication that a given thread is a candidate for throttling; In response to control logic identifying the given thread from a plurality of threads as a candidate for throttling, setting a threshold number of cache misses for the given thread; throttling a number of instructions processed by the given thread on a given one of the plurality of compute units in response to the control logic determining that a number of cache misses for the given thread is greater than the threshold number of cache misses. method.

10. the given computing unit further processing instructions at a stage of instruction processing before data is accessed in the cache.

10. The method of claim 9.

11. a selection ratio for selecting instructions of the given thread for processing in the given computational unit; a period of time for maintaining the selection ratio; and further comprising setting a level of throttling including 10. The method of claim 9.

12. The period has elapsed; and after the period of time has elapsed, the number of cache misses in the cache is greater than the threshold number of cache misses; and updating the level of throttling to a more restrictive level of throttling by performing one or more of: decreasing the selection ratio; and increasing the time period in response to determining The method of claim 11.

13. The period has elapsed; and after the period of time has elapsed, the number of cache misses in the cache is less than or equal to the threshold number of cache misses; and updating the level of throttling to a less restrictive level of throttling by performing one or more of increasing the selection ratio and decreasing the time period in response to determining The method of claim 11.

14. The indicator is generated based at least in part on a quality of service of any one of the plurality of threads.

10. The method of claim 9.

15. the instruction processing stage includes any one of an instruction fetch stage, a decode stage, a dispatch stage, and an issue stage; The method of claim 10.

16. 1. A computer-readable storage medium storing program instructions configured to perform thread arbitration, comprising: The program instructions include: a plurality of computational units processing instructions; the cache storing the data; Detecting a received indication that a given thread is a candidate for throttling; In response to identifying the predetermined thread from a plurality of threads as a candidate for throttling, setting a threshold number of cache misses for the predetermined thread; throttling a number of instructions processed by the predetermined thread in a predetermined computing unit among the plurality of computing units in response to determining that the number of cache misses for the predetermined thread is greater than the threshold number of cache misses; It is possible to execute A computer-readable storage medium.

17. the given computing unit processes instructions at a stage of instruction processing before data is accessed in the cache; 17. The computer-readable storage medium of claim 16.

18. The program instructions include: a selection ratio for selecting instructions of the given thread for processing in the given computational unit; a period of time for maintaining the selection ratio; It is possible to set the severity level of throttling, including 17. The computer-readable storage medium of claim 16.

19. The program instructions include: The period has elapsed; and after the period of time has elapsed, the number of cache misses in the cache is greater than the threshold number of cache misses; and updating the level of throttling to a more restrictive level of throttling by decreasing the selection ratio and / or increasing the time period in response to determining 20. The computer-readable storage medium of claim 18.

20. The program instructions include: The period has elapsed; and after the period of time has elapsed, the number of cache misses in the cache is less than or equal to the threshold number of cache misses; and updating the level of throttling to a less restrictive level of throttling by performing one or more of increasing the selection ratio and decreasing the time period in response to determining 20. The computer-readable storage medium of claim 18.

Images