A distributed scheduler that provides execution pipe balancing.

Abstract

The processor includes a plurality of execution pipes and a distributed scheduler coupled to the plurality of execution pipes. The distributed scheduler includes a first queue that buffers instruction operations from a front end of an instruction pipeline of the processor and a plurality of second queues, each second queue buffering instruction operations allocated from the first queue to a different subset of the plurality of execution pipes. The distributed scheduler further includes a queue controller that selects an allocation mode from among a plurality of allocation modes based on whether at least one indicator of imbalance in the distributed scheduler is detected, and controls the distributed scheduler to allocate instruction operations from the first queue among the plurality of second queues in accordance with the selected allocation mode.

Description

[Background technology]

[0001] The distributed scheduler employs at least two levels of scheduler queues: the first level has non-pickable scheduler queues, and the second level has two or more pickable scheduler queues, each of which operates to store instruction operations for a corresponding subset of multiple execution pipes. The pickable scheduler queues serve to temporarily buffer instruction operations from the instruction pipeline frontend before they are assigned to the pickable scheduler queues.

[0002] Because each pickable scheduler queue exclusively supplies its own subset of execution pipes, there is often a possibility of performance-hindering imbalances between different pickable scheduler queues, between different execution pipes, or in combination thereof. For example, as a result of the specific way a program is built or compiled, or despite concurrent queue allocations, one pickable scheduler queue may load a large number of instruction operations, each requiring a significant number of cycles, such as load-dependent operations, while another pickable scheduler queue may have a smaller proportion of such delay-causing operations. Another example is that one pickable scheduler queue may be allocated more operations that can only be executed by the specific execution pipe associated with that queue than another pickable scheduler queue with similar execution pipes. As a result, one execution pipe associated with one pickable scheduler queue may be over-occupied, while other similar execution pipes remain unnecessarily idle. Such skewed queue / pipe occupancy typically leads to fewer operations being picked per cycle and underutilization of the execution pipe, either of which negatively impacts the execution performance of processor cores implementing instruction pipelines with distributed schedulers.

Summary of the Invention

Problems to be Solved by the Invention

[0003] In one embodiment, the method includes buffering instruction operations in a first queue of a processor's distributed scheduler; selecting an allocation mode from among several allocation modes of the distributed scheduler based on whether at least one indicator of imbalance is detected in the distributed scheduler; allocating instruction operations from the first queue among several second queues according to the selected allocation mode; and in each second queue, selecting instruction operations buffered in the second queue for execution by execution pipes associated with the second queue. The multiple allocation modes may include a queue-first allocation mode in which instruction operations are allocated substantially equally among the several second queues, and a pipe-first allocation mode in which instruction operations are allocated among the several second queues based on the pipe occupancy rate of execution pipes associated with the second queues. In some embodiments, at least one indicator of imbalance includes at least one of a first indicator indicating that at least one of the second queues has a current occupancy rate exceeding a first threshold, and a second indicator indicating that the difference in current occupancy rates between any two second queues exceeds a second threshold. Thus, selecting an allocation mode from among multiple allocation modes may include selecting a queue-priority allocation mode depending on whether the first and second indicators are detected, and selecting a pipe-priority allocation mode depending on whether at least one of the first and second indicators is detected. Alternatively, selecting an allocation mode from among multiple allocation modes may include selecting a queue-priority allocation mode depending on whether either the first or second indicator is detected, and selecting a pipe-priority allocation mode depending on whether both the first and second indicators are detected.

[0004] In some embodiments, the multiple allocation modes further include a speculative allocation mode in which, based on an analysis of the pipe occupancy of the execution pipe capable of executing the instruction operations, each copy of an instruction operation in a set of instruction operations is allocated to each of a plurality of second queues, and then each copy of an instruction operation in the set is invalidated in all but one of the second queues. Thus, selecting an allocation mode from among the multiple allocation modes may include selecting a speculative allocation mode in response to the current pick rate at which instruction operations are selected from the second queues for execution by the corresponding execution pipe falling below a threshold. At least one indicator of imbalance may include a first indicator indicating that at least one of the second queues has a current occupancy rate above a first threshold, a second indicator indicating that the difference between the current occupancy rates of any two second queues is above a second threshold, and a third indicator indicating that the current pick rate at which instruction operations are selected from the second queues for assignment to the corresponding execution pipe is below a threshold. Selecting an assignment mode from among several assignment modes includes selecting a queue-priority assignment mode depending on the presence of the first or second indicator, selecting a pipe-priority assignment mode depending on the presence of both the first and second indicators, and selecting a speculative assignment mode depending on the presence of the third indicator. During the speculative assignment mode, each instruction operation in a set may have a pre-allocated execution pipe, and a second queue associated with a pre-allocated execution pipe holds a valid copy of the instruction operation as long as the difference between the pipe occupancy rate of the pre-allocated execution pipe and the pipe occupancy rate of another execution pipe of the same type associated with another second queue does not exceed a threshold.

[0005] In some embodiments, the assignment modes include a queue-priority assignment mode in which, for each set of instruction actions, the instruction actions of the set are assigned among a plurality of second queues using an interleaving pattern, the second queue that receives the first instruction action in the interleaving pattern for a given set is selected according to either a fixed selection sequence or pseudo-randomness.

[0006] In another embodiment, the processor is configured to perform embodiments of the method described above. In some embodiments, the processor includes a plurality of execution pipes and a distributed scheduler coupled to the plurality of execution pipes, wherein the distributed scheduler includes a first queue configured to buffer instruction operations from the front end of the processor's instruction pipeline and a plurality of second queues, each second queue configured to buffer instruction operations allocated from the first queue to a corresponding subset of the plurality of execution pipes and a queue controller, wherein the queue controller is configured to select an allocation mode from a plurality of allocation modes based on whether at least one indicator of imbalance is detected in the distributed scheduler and to control the distributed scheduler to allocate instruction operations from the first queue among the plurality of second queues according to the selected allocation mode.

[0007] In another embodiment, the processor includes a plurality of execution pipes and a distributed scheduler coupled to the plurality of execution pipes, the distributed scheduler includes a first queue configured to buffer instruction operations from the front end of the processor's instruction pipeline, a plurality of second queues, each second queue configured to buffer instruction operations allocated from the first queue to a corresponding subset of the plurality of execution pipes, and a queue controller, the queue controller is configured to speculatively allocate copies of instruction operations from the first queue to each second queue for each instruction operation in the subset of instruction operations in the first queue, and to invalidate copies of instruction operations in all but one second queue based on the current pipe occupancy of execution pipes capable of executing the instruction operations. In some embodiments, an instruction operation is pre-assigned to a predetermined execution pipe among a plurality of execution pipes, and the queue controller is further configured to maintain a copy of the instruction operation in a second queue associated with the predetermined execution pipe as a valid copy of the instruction operation, depending on whether the current pipe occupancy of the predetermined execution pipe is within a threshold of the current pipe occupancy of other execution pipes capable of executing the instruction operation, and to invalidate the copy of the instruction operation in the second queue associated with the predetermined execution pipe and maintain the copy of the instruction operation in the different second queue, depending on whether the current pipe occupancy of the predetermined execution pipe is at least above a threshold of the current pipe occupancy of an execution pipe associated with a different second queue.

[0008] In a further different embodiment, the processor includes a plurality of execution pipes and a distributed scheduler coupled to the plurality of execution pipes, the distributed scheduler includes a first queue configured to buffer instruction operations from the front end of the processor's instruction pipeline, a plurality of second queues, each second queue configured to buffer instruction operations allocated from the first queue to a corresponding subset of the plurality of execution pipes, and a queue controller, the queue controller configured to allocate sets of instruction operations from the first queue among the plurality of second queues in an interleaved pattern, and for each set, to select a second queue to which the first instruction operation of the set is allocated in an interleaved pattern, based on either a predetermined sequence or pseudo-randomness. In some embodiments, the queue controller is further configured to select a second queue to which the first instruction operation of the set is allocated in an interleaved pattern based on a predetermined sequence, where the predetermined sequence is an alternating sequence. The processor may further include a pseudorandom number generator, and the queue controller is further configured to pseudorandomly select a second queue in an interleaved pattern to which the first instruction operation of the set is assigned, based on the output of the pseudorandom number generator.

[0009] By referring to the attached drawings, this disclosure can be better understood, and many of its features and advantages will become apparent to those skilled in the art. Where the same reference numerals are used in different drawings, they indicate similar or identical items. [Brief explanation of the drawing]

[0010] [Figure 1] This block diagram shows a processor implementing a multimodal distributed scheduler according to several embodiments. [Figure 2] This block diagram shows in more detail the multimodal distributed scheduler of Figure 1 according to several embodiments. [Figure 3]This flowchart shows queue priority allocation modes for a multimodal distributed scheduler according to several embodiments. [Figure 4] This figure shows an exemplary embodiment of the method of Figure 3, according to several embodiments. [Figure 5] This flowchart shows pipe-priority allocation modes for a multimodal distributed scheduler according to several embodiments. [Figure 6] This flowchart shows speculative allocation modes of a multimodal distributed scheduler according to several embodiments. [Figure 7] This figure shows an exemplary embodiment of the method of Figure 6, according to several embodiments. [Figure 8] This flowchart shows a method for selectively activating different allocation modes of a multimodal distributed scheduler based on various imbalance indicators, according to several embodiments. [Modes for carrying out the invention]

[0011] Figures 1 to 8 illustrate techniques for mitigating queue and pipe imbalances in a processing system employing a multimodal distributed scheduler for instruction operation execution scheduling. In at least one embodiment, the distributed scheduler includes a non-pickable scheduler queue for temporarily buffering instruction operations from the processor's instruction front-end, and two or more pickable scheduler queues for receiving instruction operations from the non-pickable scheduler queue, each of which operates to buffer instruction operations for a relevant, distinct subset of the set of execution pipes supplied by the distributed scheduler. The distributed scheduler further includes a queue controller for configuring the distributed scheduler to allocate instruction operations between the non-pickable scheduler queue and the pickable scheduler queue according to a selection allocation mode of multiple allocation modes supported by the distributed scheduler. In at least one embodiment, the queue controller selects an allocation mode which will be implemented in response to the presence or absence of one or more indicators of imbalance between pickable scheduler queues or between execution pipes of identical or similar capabilities. Such indicators include, for example, queue occupancy above a threshold, pipe occupancy above a threshold, the difference between the occupancy rates of two or more queues above a threshold, the difference between the pipe occupancy rates of two or more execution pipes of the same or similar type above a threshold, and the rate at which instruction operations are picked for execution to fall below a threshold.

[0012] In some embodiments, the multiple allocation modes supported by the multimodal distributed scheduler include a queue-first allocation mode in which a subset of instruction operations to be allocated to pickable scheduler queues within a given allocation cycle are allocated substantially equally among the pickable scheduler queues, such as via an interleaving pattern. To further ensure equitable distribution of instruction operations, in some embodiments, the interleaving pattern starts with a different pickable scheduler queue for each allocation cycle, and the pickable scheduler queues that receive initial instruction operations in the interleaving pattern during a given allocation cycle are selected, for example, according to a predetermined, fixed selection sequence (e.g., an alternating sequence or a "ping-pong" sequence), or pseudo-randomly, such as by using the output values ​​of a linear feedback shift register (LFSR) or other pseudo-random number generator. Furthermore, in some embodiments, the multiple allocation modes also include a pipe-priority allocation mode in which the pipe occupancy of each execution pipe adapted to perform the instruction operation to be allocated is taken into consideration, and the execution pipe and corresponding associated pickable queue selected to perform the instruction operation are based on a comparison and evaluation of their pipe occupancy.

[0013] In some embodiments, the multiple allocation modes supported by the multimodal distributed scheduler include a speculative allocation mode in which, for a given allocation cycle, copies of the same instruction operation are speculatively allocated from a non-pickable scheduler queue to each of several pickable scheduler queues. The queue controller then selects an appropriate execution pipe to execute the instruction operation, thereby maintaining a copy of the instruction operation as valid in the pickable scheduler queue associated with the selected execution pipe, while invalidating copies of the instruction operation in other pickable scheduler queues.

[0014] Figure 1 shows a processor 100 implementing a multimodal distributed scheduler 102 for queue and execution pipe balancing, according to several embodiments. The processor 100 includes an instruction front-end 104, instruction execution units such as an integer unit 106 and a floating-point / single-instruction multiplexed data (SIMD) unit 108, and a cache / memory interface subsystem 110. The instruction front-end 104 operates to fetch instructions as part of an instruction stream, decode the instructions into one or more instruction operations (e.g., microoperations or uops), and then dispatch each instruction operation to either the execution units 106, 108 for execution. When executing an instruction operation, the execution unit frequently utilizes one or more caches implemented in the cache / memory interface subsystem 110, or accesses data from external memory (e.g., random access memory, RAM) or external input / output (I / O) devices, or provides data to external memory or external I / O devices for storage, via the load / store unit (LSU), memory controller, or I / O controller (not shown) of the cache / memory interface subsystem 110.

[0015] The processor 100 implements a multimodal distributed scheduler 102 in each of one or more execution units of the processor 100. For illustrative purposes, an embodiment in which the multimodal distributed scheduler 102 is implemented in a floating-point / SIMD unit 108 is described herein. However, in other embodiments, an integer unit 106 or other execution unit of the processor 100 implements the multimodal distributed scheduler in addition to or instead of the one implemented by the floating-point / SIMD unit 108, using the guidelines provided herein.

[0016] The floating-point / SIMD unit 108 includes a rename module 112, a physical register file 114, a plurality of execution pipes 116 such as six execution pipes 116-1 to 116-6 in the illustrated embodiment, and a multimodal distributed scheduler 102. The rename module 112 performs renaming of operations for instruction operations received from the instruction front-end 104, including renaming them from architected registers to physical registers in the physical register file 114, and outputs the renamed instruction operations to the multimodal distributed scheduler 102 for buffering and allocation among the execution pipes 116. In at least one embodiment, the multimodal distributed scheduler 102 includes a first scheduler queue 118, a multiplexer (mux) network 120, a plurality of second scheduler queues 122 and pickers 124 for each second scheduler queue 122, such as two second scheduler queues 122-1, 122-2 and corresponding pickers 124-1, 124-2 in the illustrated example, and a queue controller 126. Each of the second scheduler queues 122 is responsible for buffering instruction operations for a corresponding subset of a plurality of execution pipes 116. For example, in the illustrated embodiment, the second scheduler queue 122-1 buffers instruction operations for a subset consisting of execution pipes 116-1, 116-2, and 116-3, while the second scheduler queue 122-2 buffers instruction operations for another subset consisting of execution pipes 116-4, 116-5, and 116-6.

[0017] The multimodal distributed scheduler 102 performs two levels of queuing, thereby a first scheduler queue 118 temporarily buffers instruction operations, which are then allocated among multiple second scheduler queues 122 via the mux network 120. A picker 124 for each second scheduler queue 122 then picks the instruction operations buffered in the corresponding second scheduler queue 122 for allocation to a subset of execution pipes associated with that second scheduler queue 122 or for other allocation. Since instruction operations are not pickable from the first scheduler queue 118 for direct execution, the first scheduler queue 118 is referred to herein as the “non-pickable scheduler queue 118” or “NSQ118”. Conversely, since instruction operations can be picked from the second scheduler queue 122 for execution, each of the second scheduler queues 122 is called a "pickable scheduler queue 122" or "SQ122".

[0018] In at least one embodiment, the multimodal distributed scheduler 102 supports multiple allocation modes, each allocation mode representing a specific process for allocating instruction operations from NSQ 118 to SQ 122 during a given allocation cycle. Each allocation mode typically addresses current or anticipated execution inefficiencies, and thus, in one embodiment, the queue control 126 selects a specific allocation mode to be implemented during the next one or more allocation cycles based on the presence or absence of one or more indicators of imbalance or execution inefficiency that are potentially addressed by the selected allocation mode, and then configures the mux network 120 and picker 124 to implement the selected allocation mode. Examples of allocation modes supported by the multimodal distributed scheduler 102, and exemplary processes for selecting a specific allocation mode for implementation, are described in detail below.

[0019] Figure 2 is a diagram showing in more detail embodiments of the multimodal distributed scheduler 102 according to several embodiments. In this example, NSQ118 includes a plurality of queue entries 202, each queue entry 202 capable of storing instruction operations 204 received from the instruction front end 104. NSQ118 operates as a first-in, first-out (FIFO) queue and includes a plurality of output ports 206, each output port capable of outputting the corresponding buffered instruction operation 204 from NSQ118 during the allocation cycle. Thus, NSQ118 has the capability to output N instruction operations per allocation cycle, where N represents the number of output ports 206. Each of SQ122-1 and 122-2 includes a plurality of queue entries 208, each queue entry 208 capable of buffering the instruction operation 204 allocated from NSQ118 to the corresponding SQ122. Therefore, each SQ122 includes multiple input ports 210, each capable of writing received command operations to the corresponding queue entry 208 of the receiving SQ122. In at least one embodiment, the total number of input ports 210 for all SQ122 is equal to the number of output ports 206 of the NSQ118, N. Thus, if each SQ122 includes K input ports 210, then N = K * J, where J is the number of SQ122s. For illustrative purposes, in the embodiment described, the NSQ118 includes 6 output ports 206 (N=6), and each of SQ122-1 and SQ122-2 (J=2) includes 3 input ports (K=3). Therefore, in this embodiment, it is possible to assign up to six (6) instruction operations from NSQ118 to SQ122-1,122-2 during a predetermined assignment cycle, and each of SQ122-1,122-2 can receive up to three (3) instruction operations during the assignment cycle.

[0020] The mux network 120 operates to allocate instruction operations 204 from the NSQ 118 between SQ 122-1 and 122-2 based on a specific allocation mode. For this purpose, the mux network 120 includes one or more levels of multiplexers connecting the output ports of the NSQ 118 to the input ports 210 of SQ 122-1 and 122-2, so that each output port 206 can be connected to either input port 210. The queue controller 126 operates to control the mux network 120 via control signaling 212 so that each instruction operation 204 allocated from the NSQ 118 is directed between the associated output ports 206 to the corresponding input port 210 of the SQ 122 to which the instruction operation 204 is allocated.

[0021] Each queue entry 208 of SQ122-1 and 122-2 may buffer the instruction operation 204 assigned from NSQ118 before the instruction operation 204 is picked by the associated picker 124 for execution. In the illustrated embodiment, the queue entry 208 includes an opcode field 214 for storing an opcode or other identifier of the type of instruction operation to be executed, and one or more operand fields 216 for storing a register identifier for a source or destination register used during execution of the instruction operation 204, or an immediate value or displacement value used during this execution. Further, as described below, in some embodiments, the instruction operation is pre-assigned to a specific execution pipe, and thus the queue entry 208 includes an assignment field 218 for storing an identifier of the execution pipe pre-assigned to the corresponding instruction operation. Moreover, to facilitate invalidation and flushing of instruction operations, in at least one embodiment, the queue entry 208 includes a valid field 220 for storing a valid (v) bit indicating whether the corresponding instruction operation is valid or invalid for its SQ122. The queue entry 208 includes additional fields (not shown), such as a status field indicating, for example, whether the instruction operation is ready to be picked for execution, and a dependency field for identifying dependencies between the instruction operation and other instruction operations, in some embodiments.

[0022] Pickers 124-1 and 124-2 operate to select or "pick" instruction operations from each SQ122 for execution by the selected execution pipe associated with SQ122. This selection process is typically based on the current capabilities of one or more execution pipes 116 associated with the SQ122 having the ability to execute the instruction operation, along with the availability of the given instruction operation to be executed at that point. Thus, the picker 124 tracks the instruction operations assigned to the corresponding subset of the execution pipes 116 or otherwise assigned, and tracks the state of the subset of the execution pipes 116 as part of the selection process.

[0023] Each of execution pipelines 116-1 to 116-6 includes logic for executing one or more instruction operation types and other circuits. In some embodiments, each execution pipeline has a specialization function such as an execution pipeline for executing a floating-point multiplication (FMUL) operation, an execution pipeline for executing a floating-point addition (FADD) operation, and an execution pipeline for executing a fused multiply-add (FMU) operation. Further, in some embodiments, the SQ122 of the execution pipeline and their associated subsets are "symmetric" in that each SQ122 has the same configuration of the execution pipeline associated with them. That is, the execution pipeline configuration for SQ122-1 is the same as the execution pipeline configuration for SQ122-2, and as a result, execution pipeline 116-1 (EX0) and execution pipeline 116-4 (EX1) are the same type of execution pipeline, execution pipeline 116-2 (EX2) and execution pipeline 116-5 (EX3) are the same type of execution pipeline, and execution pipeline 116-3 (EX4) and execution pipeline 116-6 (EX5) are the same type of execution pipeline. Therefore, the instruction operation 204 appropriate for execution in execution pipeline EX0 is also appropriate for execution in pipeline EX1 due to the same capability relationship for execution pipeline pairs EX2 / EX3 and EX4 / EX5, and vice versa.

[0024] The queue controller 126 includes various components, including an assignment component 222, a mode control component 224, a queue update component 226, a queue status component 228, a pipe status component 230, a pick rate component 232, and an interleave selection component 236. Each of these components is implemented as hardware logic and associated circuitry, as programmable logic and associated circuitry, as a processor core that executes software instructions representing the corresponding function, or as a combination thereof. The pipe status component 230 monitors the execution pipes 116 to determine the current pipe occupancy rate for each execution pipe 116. Similarly, the queue status component 228 monitors NSQ118 and SQ122 to determine their current queue occupancy rate and the rate of change in occupancy rate, etc. The pick rate component 232 monitors instruction operation picks made by pickers 124-1 and 124-2 to determine the current pick rate statistics for pickers 124-1 and 124-2. The queue update component 226 operates to update various fields of queue entries 208 for SQ122-1,122-2 and queue entry 202 for NSQ118, based on the operation of the distributed scheduler 102, as described in detail below. The interleaved selection component 236 operates to select the initial SQ (first SQ) 122 that will receive the instruction operation in the interleaved assignment pattern, as described below.

[0025] As described herein, in at least one embodiment, the multimodal distributed scheduler 102 supports multiple allocation modes to facilitate the dynamic adaptation of instruction operation allocation processing between NSQ118 and SQ122-1,122-2 to mitigate performance degradation caused by imbalances between identical types of SQ122-1,122-2 or execution pipes 116. As described below, these multiple allocation modes include, in one embodiment, a queue-first allocation mode 240, in which instruction operations are first allocated from NSQ118 to SQ122, and then, once the instruction operations are in each of those SQ122, pipe allocation for the instruction operations follows. Another allocation mode includes a pipe-first allocation mode 242, in which instruction operations are first allocated to the corresponding execution pipe 116 while buffered in NSQ118, and then the instruction operations are allocated to the SQ122 associated with the execution pipe allocated during the NSQ-SQ allocation cycle. As yet another example, the multiple allocation modes supported by the multimodal distributed scheduler 102 include a speculative allocation mode 244, in which each instruction operation copy of a subset of instruction operations is speculatively allocated from NSQ118 to SQ122-1, 122-2 respectively within the same allocation cycle, and then, as the pipe allocation is subsequently completed, only the copy of the instruction cycle in SQ122 associated with the allocated execution pipe is retained, while other copies in other SQ122 are invalidated and then flushed. These various allocation modes are described in more detail below with reference to Figures 3 to 7.

[0026] The mode control component 224 operates to select a specific allocation mode to be implemented during one or more allocation cycles, depending on whether it has detected the presence or absence of one or more indicators of imbalance between SQ122 or between execution pipes, as determined from information provided by other components of the queue controller 126, such as pick rate statistics from the pick rate component 232, current pipe occupancy from the pipe status component 230, and current queue occupancy from the queue status component 228. An exemplary process for selecting an allocation mode is described below with reference to Figure 8. Based on the selected allocation mode and various queue and execution pipe status information, the allocation component 222 operates to control the allocation of instruction operations from NSQ118 to SQ122 (e.g., via the configuration of the control signaling 212) and the allocation of instruction operations to the execution pipe 116.

[0027] Figure 3 shows exemplary methods of instruction operation assignment for the queue-priority assignment mode 240 of the multimodal distributed scheduler 102 according to several embodiments. As described above, the queue-priority assignment mode 240 results in the assignment of instruction operations 204 from NSQ118 to SQ122-1,122-2 for the purpose of queue capacity balancing, and the pipe assignment of instruction operations occurs after the instruction operations have been assigned to SQ122-1,122-2. Each SQ122 receives up to K instruction operations within a given assignment cycle, such that each SQ122 has K input ports 210 (K=3 in the embodiment of Figure 2). Thus, one possible approach is to assign the first K instruction operations in NSQ118 to SQ122-1 and the next K instruction operations in NSQ118 to SQ122-2 during the assignment cycle. However, this block-style assignment approach can lead to imbalance if, for example, one block of K instruction operations contains instruction operations that depend on load operations, memory operations, or other operations that require a reasonable number of clock cycles to complete, while the other blocks of K instruction operations have few or no such instruction operations. One approach to mitigate this is to employ an interleaving pattern for instruction assignment between SQ122s during the assignment cycle, such that one SQ122 acquires one instruction operation, another SQ122 acquires the next instruction operation, and so on. In this way, each SQ122 acquires one instruction operation in sequence, and then the sequence is repeated again through all SQ122s until all instruction operations have been assigned for the current assignment cycle. However, if the same SQ122 acts as the first SQ to receive instruction operations in the interleaving pattern for each assignment cycle, the occupancy rate among SQ122s may be unbalanced in some situations.For example, if the program being executed has operations that alternately perform addition and multiplication, such as [ADD, MUL, ADD, MUL, ADD, MUL, ADD, MUL, ADD, MUL, ADD, MUL, ADD, MUL], then the interleaving pattern of assignments between two SQ122s that provide the same SQ122 with the first instruction operation within each assignment cycle would result in six ADD operations being assigned to one SQ122 and six MUL operations (which take significantly longer to execute than ADD operations) being assigned to the other SQ122. Therefore, to avoid such potential queue imbalances, in at least one embodiment, in the queue priority assignment mode 240 using the interleaving pattern, the queue controller 126 performs a selection process to select an SQ122 that will receive the first instruction operation or the first instruction operation in a subset of instruction operations to be assigned within a given cycle, and this selection process is based, for example, on alternating selection or ping-pong selection, or on pseudo-random selection. By sequentially or pseudo-randomly changing the first received SQ122 between allocation cycles, the probability that the instruction operation stream will result in an unbalanced operation assignment between the SQ122 and their corresponding execution pipe 116 is typically reduced compared to embodiments in which the same SQ122 is selected as the first received SQ in each allocation cycle.

[0028] Therefore, in response to the start of the assignment cycle, in block 302, the interleaving selection component 236 selects either SQ122-1 or 122-2 to receive the first instruction operation assigned from NSQ118 during the assignment cycle. That is, the interleaving selection component 236 selects either SQ122-1 or 122-2 as the "first" SQ122 in the interleaving pattern. In one embodiment, the interleaving selection component 236 uses a predetermined fixed selection sequence, such as a 1-2-1-2-1-2-1... sequence (i.e., a ping-pong sequence) or a 1-1-2-2-1-1-2-2 sequence for two SQ122s, or, in an embodiment using three SQ122s, a 1-2-3-1-2-3-1-2-3-1... sequence or a 1-2-3-2-3-1... sequence, etc. In other embodiments, the interleave selection component 236 implements a pseudorandom number generator, such as a linear feedback shift register (LFSR), which generates a pseudorandom number for each allocation cycle, and then uses one or more bits output by the pseudorandom number generator to select the first received SQ122 for the interleave pattern to be adopted during the current allocation cycle. With the selected first received SQ122, in block 304, the allocation component 222 controls the mux network 120 via output port 206 and input port 210 to distribute a subset of instruction operations from NSQ118 to SQ122-1,122-2 according to the interleave pattern, starting with the selected first received SQ122.

[0029] Figure 4 shows an example of processing in an embodiment in which six instruction operations from NSQ118 are assigned between two SQ122-1 and 122-2 in an interleaved pattern using a ping-pong type initial received SQ selection sequence. Initially, NSQ118 buffers 12 instruction operations (OP1 to OP12), with OP1 being the oldest instruction operation (and therefore assigned first). During the assignment cycle 401, a subset of six instruction operations (OP1 to OP6) are assigned. During this assignment cycle, SQ122-1 is selected as the first received SQ, and therefore the interleaved assignment pattern results in the assignment of operations OP1, OP3, and OP5 to SQ122-1, and operations OP2, OP4, and OP6 to SQ122-2. During the next assignment cycle 402, a subset of six instruction operations OP7-OP12 will be assigned, and during this assignment cycle, SQ122-2 will be selected as the first received SQ using a ping-pong selection pattern. Thus, during assignment cycle 402, the interleaved assignment pattern results in instruction operations OP7, OP9, and OP11 being assigned to SQ121-2, and OP8, OP10, and OP12 being assigned to SQ121-1.

[0030] Figure 5 shows exemplary methods of instruction operation assignment for the pipe-priority assignment mode 242 of the multimodal distributed scheduler 102 according to several embodiments. The queue-priority assignment mode 240 described above has the advantage of allowing all output ports 206 to be utilized within an assignment cycle, and thus maximizing the number of instruction operations assigned from NSQ118 to SQ122 within an assignment cycle. However, a disadvantage is that the situation in the instruction operation stream may result in an imbalance in pipe utilization. For example, queue-priority assignment may result in an unbalanced number of ADD operations in one SQ122 compared to other SQ122s, and an unbalanced number of MUL operations in other SQ122s. In such a scenario, the execution pipe 116 of the first SQ122 used for ADD operations may be backed up while the execution pipe 116 of the first SQ122 used for MUL operations becomes idle, and the opposite may occur for the second SQ122, where its ADD-type execution pipe 116 becomes idle while its MUL-type execution pipe 116 becomes overloaded. The pipe priority allocation mode 242 mitigates this imbalance by evaluating the pipe occupancy of all execution pipes 116 in the distributed scheduler 102, and then assigning each instruction operation of the subset of instruction operations to be allocated in the current allocation cycle to the execution pipe with the lowest pipe occupancy and capable of executing the type of operation represented by the instruction operation.

[0031] Therefore, during a predetermined allocation cycle, in block 502, the mode control component 224 evaluates the current pipe occupancy of the set of execution pipes 116 of the distributed scheduler 102 by accessing, for example, pipe occupancy data in memory, register files, or other storage components maintained by the pipe status component 230. Generally, the pipe occupancy data indicates the number of instruction operations currently allocated to each execution pipe 116, and thus represents the total number of operable "works" already committed to each execution pipe 116. With this information, in block 504, the allocation component 222 selects the next instruction operation in NSQ 118 to be allocated within the current allocation cycle, identifies the execution pipes 116 capable of executing the selected instruction operation, and then allocates the selected instruction operation to the execution pipe 116 having the lowest pipe occupancy within the identified subset. This allocation is performed, for example, by writing a value identifying the selected execution pipe to the allocation field of the queue entry 202 of NSQ 118 that stores the selected instruction operation. If an instruction operation is available for assignment to SQ122 (for example, if the instruction operation is included in a subset of instruction operations to be assigned within the current assignment cycle), in block 506, the assignment component 222 accesses the assignment field of the queue entry 202 that stores the instruction operation to determine the pre-assigned execution pipe 116, and controls the mux network 120 to route the instruction operation from NSQ118 to the entry 208 of SQ122 associated with the assigned execution pipe 116. The processing in blocks 502, 504, and 506 is performed for each instruction operation in the subset of instruction operations to be assigned within a given assignment cycle.

[0032] Prioritizing pipe occupancy in pipe-priority allocation mode 242 is typically effective in maintaining a balance of pipe occupancy within the distributed scheduler 102. However, unlike queue-priority allocation mode 240, where NSQ-SQ allocation throughput is maximized, a situation may arise where a sequence of instruction operations in NSQ118 is allocated to a specific SQ122, resulting in a limited number of instruction operations that can be allocated within a corresponding allocation cycle because the number of input ports 210 on a given SQ122 is small compared to the total number of output ports 206 on NSQ118. For example, if the following six instruction operations in NSQ118 in Figure 2 are pre-allocated to the execution pipe 116 associated with SQ122-2, then since SQ122-2 in Figure 2 has only three input ports 210, only three of the possible six instruction operations can be allocated during the next allocation cycle. In contrast, queue-priority allocation mode 240 allows all six output ports 206 to be utilized to allocate six instruction operations in the same situation (assuming sufficient capacity in SQ122). As will be further described below with reference to Figure 8, the multimodal distributed scheduler 102 leverages the advantages of queue-priority allocation mode 240 and pipe-priority allocation mode 242 by switching their allocation modes in response to detecting various indicators of imbalance between SQ122 or between execution pipes 116.

[0033] Figure 6 shows exemplary methods for instruction assignment operations for speculative assignment mode 244 provided by a multimodal distributed scheduler 102, according to several embodiments. Typically, the distributed scheduler 102 is designed and manufactured such that once an instruction operation is assigned to one SQ122, that instruction operation cannot be moved to another SQ122. Thus, the distributed scheduler 102 has little resource to correct imbalances caused by instruction operations already committed to SQ122 using either queue-priority assignment mode 240 or pipe-priority assignment mode 242. However, if the current pick rate of the execution pipe 116 is sufficiently low and an underflow of SQ122 is not a significant risk, the distributed scheduler 102 will employ speculative assignment mode 244 to assign instruction operations to SQ122 in a manner that allows for balance between instruction operations in SQ122.

[0034] For processing speculative allocation in speculative allocation mode 244, in block 602, the mode control component 224 evaluates the current pipe occupancy of the set of execution pipes 116 of the distributed scheduler 102, for example, by accessing the pipe occupancy data as described above. In block 604, the allocation component 222 identifies unallocated instruction operations in NSQ 118 and pre-allocates the identified instruction operations to specific execution pipes 116 based on the current pipe occupancy of the execution pipes 116 capable of executing instruction operations of that type. This pre-allocation is reflected, for example, by storing the identifier of the pre-allocated execution pipe in the allocation field of the queue entry 202 that stores the instruction operations of NSQ 118. This pre-allocation process is repeated for any number of unallocated instruction operations in NSQ 118.

[0035] If a pre-allocated instruction action is available for assignment to an SQ122 (for example, if the instruction action is included in a subset of the instruction actions to be assigned within the current assignment cycle), instead of assigning the instruction action to a single SQ122, in block 606, the assignment component 222 routes a copy of the instruction action to each of the SQ122s, regardless of which SQ122 is associated with the instruction pipe 116 pre-allocated for the instruction action (for example, routing one copy to SQ122-1 and another copy to SQ122-2). This process is repeated for each pre-allocated instruction action in the subset to be assigned during the current assignment cycle.

[0036] Subsequently, in block 608, the assignment component 222 sequentially selects the next speculatively assigned instruction operation in SQ122 and determines whether there is an imbalance in the current pipe occupancy of the pre-assigned execution pipe 116 and the same or similar type of execution pipe 116 associated with other SQ122s. For example, if an instruction operation is pre-assigned to execution pipe 116-2(EX2) associated with SQ122-1, the pipe occupancy of execution pipe 116-2(EX2) is compared with the current pipe occupancy of execution pipe 116-5(EX3) associated with SQ122-2. For this process, an imbalance in pipe occupancy between the pre-assigned execution pipe 116 and other similar execution pipes in other SQ122s may be indicated, for example, by the pipe occupancy of the pre-assigned execution pipe 116 exceeding the pipe occupancy of the same type of execution pipe 116 in the other SQ122 by at least a threshold. For example, assuming the threshold is set to 3 operations, if a pre-allocated execution pipe 116 has 8 instruction operations in flight and another execution pipe 116 of the same type has 6 instruction operations, or if another execution pipe 116 of the same type has more instruction operations in flight than the pre-allocated execution pipe 116, then no imbalance is expected. On the other hand, if a pre-allocated execution pipe 116 has 12 instruction operations in flight and another execution pipe 116 of the same type has 5 instruction operations in flight, then an imbalance is detected.

[0037] If no sufficient imbalance in pipe occupancy is detected in block 608, in block 610, the allocation component 222 instructs the queue update component 226 to maintain the valid status of the copy of the instruction operation in SQ122 associated with the pre-allocated execution pipe 116 and invalidate the copies of the instruction operation in other SQ122. That is, if no sufficient imbalance is detected, the queue controller 126 maintains the pre-allocation of the instruction operation by maintaining the copy of the instruction operation in SQ122 associated with the pre-allocated execution pipe 116, and invalidates all other copies of the instruction operation in other SQ122. In at least one embodiment, invalidation is achieved by clearing the valid bit of the valid field 220 of the queue entry 208 that stores the copy of the instruction operation to be invalidated. In doing so, entry 208 becomes available to flush the invalid copy of the instruction operation so that entry 208 can be used to store a different instruction operation.

[0038] Instead, if a sufficient imbalance in pipe occupancy is detected in block 608, in block 612, the allocation component 222 evaluates various parameters, such as the current pipe occupancy and the number of subsequent instruction operations pre-assigned to each of the execution pipes 116 capable of executing the instruction operation, in order to dynamically select the appropriate execution pipe 116 to which the instruction operation will ultimately be assigned. In some situations, the selected execution pipe 116 is the same as the execution pipe to which the instruction operation has been pre-assigned. In other situations, another execution pipe 116 with the same or similar capabilities is well-positioned to ensure timely execution of the instruction operation, and therefore the assignment of the instruction operation is switched to this other execution pipe. In either case, the allocation component 222 instructs the queue update component 226 to maintain the valid status of the copy of the instruction operation in the SQ 122 associated with the selected execution pipe 116 and to invalidate the copy of the instruction operation in other SQ 122s. For blocks 610 and 612, if more than two SQ122s are implemented, only a single copy of the instruction operation remains valid (i.e., the copy in the SQ associated with the execution pipe ultimately assigned to the instruction operation), and all other copies in the other SQ122s are invalidated.

[0039] Speculative allocation mode 244 allows the queue controller 126 to fine-tune pipe occupancy more precisely by delaying the final allocation of each instruction operation until after the instruction operation has been allocated to a pickable scheduler queue level, rather than requiring execution pipe allocation at the non-pickable scheduler queue level, thereby achieving improved pipe balance and pipe throughput. This delay allows the queue controller 126 additional time to evaluate the pipe occupancy of compatible execution pipes 116 closer to the time the instruction operation is picked for execution, thus enabling the queue controller 126 to make more accurate allocation decisions. However, speculative allocation mode 244 requires the use of two output ports 206 for each instruction operation allocated from NSQ 118 to SQ 122 (one output port 206 for each copy of the allocated instruction operation), and therefore the effective allocation rate of instruction operations per allocation cycle is only half of the maximum NSQ vs SQ allocation rate. Therefore, as will be explained in more detail below with reference to Figure 8, the speculative allocation mode 244 is implemented more effectively when the current pick rate of the execution pipe 116 is sufficiently low, in order to reduce or eliminate the possibility of queue underflow caused by the relatively slow input rate brought about by the speculative allocation mode 244.

[0040] Figure 7 shows an example of speculative allocation processing in an embodiment in which copies of three instruction operations (OP1, OP2, OP3) are speculatively allocated from NSQ118 to each of two SQ122-1 and 122-2. During the pre-allocation processing (block 604), OP1 is pre-allocated to execution pipe 116-1 (EX0), OP2 is pre-allocated to execution pipe 116-5 (EX3), and OP3 is pre-allocated to execution pipe 116-3 (EX4). As shown by stage 701, during the speculative allocation processing (block 606), each copy of instruction operations OP1, OP2, and OP3 is allocated to both SQ122-1 and SQ122-2. At this point, the validity status of those operations is marked with an "X" in the valid field (Figure 2) of the corresponding queue entry 208 to indicate that the validity of those copies of the instruction operations is not appropriate at this point. Therefore, in some embodiments, the copy is initially set to the enabled state, and in other embodiments, the copy is initially set to the disabled state.

[0041] As represented by Stage 702, after the initial assignment, the assignment component 222 determines the current pipe occupancy of the execution pipes 116 capable of executing instruction operation OP1. In this example, it reveals that execution pipe 116-1 (EX0) has 10 pending instruction operations, while execution pipe 116-4 (EX1) has only 1 pending instruction operation. Thus, the difference in pipe occupancy is 9 operations. In this particular example, the threshold for indicating pipe imbalance is 6 operations. Therefore, since the difference in pipe occupancy exceeds this threshold, a pipe imbalance is signaled. Thus, although OP1 is pre-assigned to execution pipe 116-1 (EX0) associated with SQ122-1, the assignment component 222 evaluates this pipe occupancy imbalance by considering other conditions, such as the difference in queue occupancy between SQ122-1 and SQ122-2, in order to dynamically select the SQ122 and corresponding execution pipe best suited to executing instruction operation OP1 due to the identified pipe imbalance. In this example, the assignment component 222 determines that instruction operation OP1 should be assigned to execution pipe 116-4 (EX1), and therefore reassigns instruction operation OP1 to execution pipe 116-4 associated with SQ122-2. Accordingly, as shown in stage 703, the queue update component 226 signals this final assignment by clearing the valid field of entry 208 of SQ122-1, which stores the corresponding copy of instruction operation OP1 for SQ122-1 (e.g., setting it to "0"), and by asserting the valid field of entry 208 of SQ122-2, which stores the corresponding copy of instruction operation OP1 for SQ122-2 (e.g., setting it to "1").

[0042] Next, as represented by stage 704, the allocation component 222 determines the current pipe occupancy of the execution pipes 116 capable of executing instruction operation OP2, and in this example, it reveals that execution pipe 116-2 (EX2) has 8 pending instruction operations and execution pipe 116-5 (EX3) has 3 pending instruction operations. Since the current pipe occupancy of the pre-allocated execution pipe (EX3) is lower than the current pipe occupancy of the alternative execution pipe (EX2) in the other SQ122, no pipe imbalance is signaled. Therefore, since OP2 is pre-allocated to execution pipe 116-5 (EX3) associated with SQ122-2, the allocation component 222 maintains this allocation as the final allocation. Therefore, as represented by stage 705, the queue update component 226 signals this final assignment by clearing the valid field of entry 208 of SQ122-1, which stores the corresponding copy of instruction operation OP2 for SQ122-1, and by asserting the valid field of entry 208 of SQ122-2, which stores the corresponding copy of instruction operation OP2 for SQ122-2.

[0043] Subsequently, as represented by stage 706, the allocation component 222 determines the current pipe occupancy of the execution pipe 116 capable of executing instruction operation OP3, and in this example, it reveals that execution pipe 116-3 (EX4) has 9 pending instruction operations and execution pipe 116-5 (EX3) has 6 pending instruction operations. Since the current pipe occupancy of the pre-allocated execution pipe (EX4) does not exceed the current pipe occupancy of the alternative execution pipe (EX5) in the other SQ122 by more than a threshold of 6 operations, no pipe imbalance is signaled. Therefore, since OP3 is pre-allocated to execution pipe 116-3 (EX4) associated with SQ122-1, the allocation component 222 maintains this allocation as the final allocation. Therefore, as represented by stage 707, the queue update component 226 signals this final assignment by clearing the valid field of entry 208 of SQ122-2, which stores the corresponding copy of instruction operation OP3 for SQ122-2, and by asserting the valid field of entry 208 of SQ122-1, which stores the corresponding copy of instruction operation OP3 for SQ122-1.

[0044] Figure 8 shows exemplary methods 800 for selecting an allocation mode for a distributed scheduler 102, according to several embodiments. As described above, each of the queue-priority allocation mode 240, pipe-priority allocation mode 242, and speculative allocation mode 244 may effectively mitigate imbalances in some situations, but may exacerbate (or at least not substantially correct) imbalances in other situations. Therefore, as shown by method 800, in at least one embodiment, the distributed scheduler 102 operates to switch between two or more allocation modes depending on specific indicators of actual or expected imbalances. In this way, the distributed scheduler 102 selects an allocation mode that is particularly suitable to address detected imbalances or to prevent expected imbalances from occurring.

[0045] In one embodiment, an iteration of method 800 begins in block 802 by the mode control component 224 evaluating queue status information and pipe status information obtained by the queue status component 228 and the pipe status component 230, respectively, to determine whether there is a pipe imbalance, a queue imbalance, or both. For example, in some embodiments, an actionable imbalance is signaled if there is an indicator that the difference in pipe occupancy between two running pipes of the same or similar type exceeds a predetermined threshold. In other embodiments, an actionable imbalance is signaled if there is an indicator that the difference in queue occupancy between SQ122 exceeds a predetermined threshold. In yet another embodiment, an actionable imbalance is signaled only if both a pipe imbalance indicator and a queue imbalance indicator are present. For example, in some embodiments, an imbalance in the distributed scheduler 102 is signaled only if the queue occupancy rate of one SQ122 exceeds the queue occupancy rate of other SQ122 by at least a first threshold amount, and the pipe occupancy rate of one execution pipe 116 exceeds the pipe occupancy rate of other execution pipes 116 of the same type by at least a second threshold amount. Otherwise, in this embodiment, an imbalance is not signaled even if either of the two thresholds is exceeded.

[0046] If a queue / pipe imbalance is detected in block 802, in block 804, the mode control component 224 configures the queue controller 126 to implement the pipe-priority allocation mode 242 during subsequent allocation cycles until the allocation mode is changed. Otherwise, if no queue / pipe imbalance is found in block 802, in block 806, the mode control component 224 queries the pick rate component 232 to determine the current pick rate of the execution pipes 116 and compares this average pick rate to a predetermined minimum pick rate threshold. In one embodiment, the current pick rate represents the average pick rate over a sliding time window, where the average is, for example, the average pick rate of all execution pipes 116 within the sliding window, the average pick rate of a particular type of execution pipe 116 within the sliding window, and the average pick rate of individual execution pipes 116. If the average pick rate is insufficient (i.e., does not exceed this minimum pick rate threshold), in block 808, the mode control component 224 selects speculative allocation mode 244 for implementation by the distributed scheduler 102 during subsequent allocation cycles until the average pick rate rises above a predetermined threshold, the predetermined threshold being the same or a different threshold used to place the distributed scheduler 102 in speculative allocation mode 244. For example, a threshold pick rate of 2.5 operations per cycle is used to trigger a transition to speculative allocation mode, while a threshold pick rate of 2.65 operations per cycle is used to trigger an exit from speculative allocation mode 244. Thus, if the average pick rate is sufficient (i.e., exceeds the minimum pick rate threshold), in block 810, the mode control component 224 selects queue-priority allocation mode 240 for implementation by the distributed scheduler 102 during subsequent allocation cycles.Next, method 800 returns to block 802, in which the mode control component 224 monitors or polls the queue status component 228 and the pipe status component 230 in the corresponding monitoring / polling cycle to detect any pipe / queue imbalance or pick rate decrease that has occurred in between.

[0047] Method 800 represents a flow in which, during a given analysis cycle, pipe / queue imbalance is checked first, and then the pick rate is analyzed; however, in other embodiments, the pick rate is analyzed first, and if it is found to be sufficient, the pipe / queue imbalance is evaluated. Furthermore, in at least some embodiments, the allocation mode selection process represented by Method 800 is implemented as a hardware state machine, in which the presence of pipe / queue imbalance triggers a mode change from queue-priority allocation mode 240 to pipe-priority allocation mode 242, and the detection of an insufficient pick rate triggers a change from any currently implemented allocation mode to speculative allocation mode 244.

[0048] In some embodiments, the above apparatus and technology are implemented in a system that includes one or more integrated circuit (IC) devices (also called integrated circuit packages or microchips), such as a processor 100 implementing the multimodal distributed scheduler 102 described above with reference to Figures 1 to 8. Electronic design automation (EDA) and computer-aided design (CAD) software tools are used to design and manufacture these IC devices. These design tools are typically represented as one or more software programs. One or more software programs include code executable by a computer system that operates a computer system to operate with code representing the circuit of one or more IC devices in order to perform at least a portion of the process of designing or adapting a manufacturing system for manufacturing the circuit. This code may include instructions, data, or combinations of instructions and data. Software instructions representing the design or manufacturing tools are typically stored in a computer-readable storage medium accessible by the computing system. Similarly, code representing one or more phases of designing or manufacturing an IC device may be stored in the same or different computer-readable storage medium, and may be accessed from the same or different computer-readable storage medium.

[0049] Computer-readable storage media include any non-temporary storage media or combination of non-temporary storage media that are accessible by a computer system during use to provide instructions and / or data to the computer system. Such storage media may include, but are not limited to, optical media (e.g., compact discs (CDs), digital versatile discs (DVDs), Blu-ray® discs), magnetic media (e.g., floppy disks, magnetic tapes, magnetic hard drives), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or microelectromechanical system (MEMS) based storage media. Computer-readable storage media (e.g., system RAM or ROM) may be built into the computing system, computer-readable storage media (e.g., magnetic hard drives) may be permanently mounted to the computing system, computer-readable storage media (e.g., optical disks or Universal Serial Bus (USB) based flash memory) may be detachably mounted to the computing system, and computer-readable storage media (e.g., network-accessible storage (NAS)) may be connected to the computer system via a wired or wireless network.

[0050] In some embodiments, some aspects of the above-described technology may be implemented by one or more processors of a processing system that executes the software. The software includes one or more sets of executable instructions stored in or tangibly embodied on a non-temporary computer-readable storage medium. When executed by one or more processors, the software may include instructions and specific data that operate one or more processors to execute one or more aspects of the above-described technology. The non-temporary computer-readable storage medium may include, for example, magnetic or optical disk storage devices, solid-state storage devices such as flash memory, cache, random access memory (RAM), or one or more other non-volatile memory devices. The executable instructions stored in the non-temporary computer-readable storage medium may be source code, assembly language code, object code, or other instruction formats that can be interpreted or executed by one or more processors.

[0051] In addition to the foregoing, it should be noted that not all activities or elements described in the summary are required, and certain activities or parts of devices may not be required, and one or more additional activities may be performed, and one or more additional elements may be included. Furthermore, the order in which the activities are listed does not necessarily indicate the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, those skilled in the art will understand that various modifications and variations can be made without departing from the scope of the invention as described in the claims. Therefore, the specification and drawings should be considered illustrative rather than restrictive, and all of these variations are intended to fall within the scope of the invention.

[0052] Benefits, other advantages, and solutions to problems have been described above with respect to specific embodiments. However, benefits, advantages, solutions to problems, and features that may give rise to or manifest any benefits, advantages, or solutions are not to be construed as essential, necessary, or indispensable features to any or all of the claims. Furthermore, the disclosed invention can be modified and implemented in different but similar ways, in ways that are obvious to those skilled in the art who are interested in the teachings of this specification; therefore, the specific embodiments described above are merely illustrative. There are no limitations to the details of the configuration or design shown herein beyond those described in the appended claims. Accordingly, the specific embodiments described above may be modified or altered, and it is clear that all such modifications are within the scope of the disclosed invention. Accordingly, the protection sought herein is described in the appended claims.

Claims

1. buffering instruction operations in a first queue of a distributed scheduler of the processor; selecting an allocation mode from among a plurality of allocation modes of the distributed scheduler based on whether at least one indicator of imbalance in the distributed scheduler is detected; allocating instruction operations of the first queue among a plurality of second queues according to a selected allocation mode; and selecting, for each second queue, an instruction operation buffered in the second queue for execution by an execution pipe associated with the second queue. method.

2. The plurality of allocation modes include: a queue priority allocation mode in which instruction operations are allocated substantially equally among the plurality of second queues; a pipe priority allocation mode in which instruction operations are allocated among the plurality of second queues based on pipe occupancy of the execution pipe associated with the second queue.

10. The method of claim 1.

3. The at least one indicator of the imbalance is: a first indicator that at least one of the second queues has a current occupancy above a first threshold; a second indicator that the difference in current occupancy between the two second queues exceeds a second threshold; at least one of The method of claim 2.

4. Selecting an allocation mode from among the plurality of allocation modes includes: selecting the queue priority allocation mode in response to neither the first indicator nor the second indicator being detected; selecting the pipe priority allocation mode in response to at least one of the first indicator and the second indicator being detected. The method of claim 3.

5. Selecting an allocation mode from among the plurality of allocation modes includes: selecting the queue priority allocation mode in response to detection of either the first indicator or the second indicator; selecting the pipe priority allocation mode in response to both the first indicator and the second indicator being detected. The method of claim 3.

6. the plurality of allocation modes further include a speculative allocation mode in which a copy of each instruction operation of a set of instruction operations is allocated to each of the plurality of second queues, and copies of each instruction operation of the set are disabled in all but any second queue based on an analysis of pipe occupancy of the execution pipes in which the instruction operations can be executed. The method of claim 2.

7. Selecting an allocation mode from among the plurality of allocation modes includes: selecting the speculative allocation mode in response to a current pick rate at which instruction operations are selected from the second queue for execution by a corresponding execution pipe being below a threshold. The method of claim 6.

8. The at least one indicator of the imbalance is: a first indicator that at least one of the second queues has a current occupancy above a first threshold; a second indicator that the difference in current occupancy between the two second queues exceeds a second threshold; a third indicator that indicates that a current pick rate at which instruction operations are selected from the second queue for allocation to a corresponding execution pipe is below a threshold; Selecting an allocation mode from among the plurality of allocation modes includes: selecting the queue priority allocation mode in response to the presence of either the first indicator or the second indicator; selecting the pipe priority allocation mode in response to the presence of both the first indicator and the second indicator; selecting the speculative allocation mode in response to the third indicator being present. The method of claim 6.

9. During the speculative allocation mode, each instruction operation of the set has a pre-assigned execution pipe, and the second queue associated with the pre-assigned execution pipe holds a valid copy of the instruction operation unless a difference between a pipe occupancy of the pre-assigned execution pipe and a pipe occupancy of another execution pipe of the same type associated with another second queue exceeds a threshold. The method of claim 6.

10. the plurality of allocation modes include, for each of a plurality of sets of instruction operations, a queue priority allocation mode in which the instruction operations of the set are allocated among the plurality of second queues using an interleaved pattern; the second queue to receive the first instruction operation in the interleaved pattern for a given set is selected according to either a fixed selection sequence or pseudo-randomly; The method of any one of claims 1 to 9.

11. 1. A processor, comprising: Multiple execution pipes; a distributed scheduler coupled to the plurality of execution pipes; The distributed scheduler a first queue configured to buffer instruction operations from a front end of an instruction pipeline of the processor; a plurality of second queues, each second queue configured to buffer instruction operations assigned from the first queue to a different subset of a corresponding one of the plurality of execution pipes; a queue controller; The queue controller selecting an allocation mode from among a plurality of allocation modes based on whether at least one indicator of imbalance in the distributed scheduler is detected; controlling the distributed scheduler to allocate instruction operations of the first queue among a plurality of second queues according to a selected allocation mode; configured to: Processor.

12. The plurality of allocation modes include: a queue priority allocation mode in which instruction operations are allocated substantially equally among the plurality of second queues; a pipe priority allocation mode in which instruction operations are allocated among the plurality of second queues based on pipe occupancy of the execution pipe associated with the second queue. The processor of claim 11.

13. The at least one indicator of the imbalance is: a first indicator that at least one of the second queues has a current occupancy above a first threshold; a second indicator that the difference in current occupancy between the two second queues exceeds a second threshold; at least one of The processor of claim 12.

14. The queue controller selecting the queue priority allocation mode in response to neither the first indicator nor the second indicator being detected; selecting the pipe priority allocation mode in response to at least one of the first indicator and the second indicator being detected; and selecting an allocation mode from the plurality of allocation modes by The processor of claim 13.

15. The queue controller selecting the queue priority allocation mode in response to detection of either the first indicator or the second indicator; selecting the pipe priority allocation mode in response to both the first indicator and the second indicator being detected; and selecting an allocation mode from the plurality of allocation modes by The processor of claim 13.

16. the plurality of allocation modes further include a speculative allocation mode in which a copy of each instruction operation of a set of instruction operations is allocated to each of the plurality of second queues, and copies of each instruction operation of the set are disabled in all but any second queue based on an analysis of pipe occupancy of the execution pipes in which the instruction operations can be executed. The processor of any one of claims 12 to 15.

17. The queue controller and selecting an allocation mode from among the plurality of allocation modes by selecting the speculative allocation mode in response to a current pick rate at which instruction operations are selected from the second queue for execution by a corresponding execution pipe being below a threshold.

17. The processor of claim 16.

18. The at least one indicator of the imbalance is: a first indicator that at least one of the second queues has a current occupancy above a first threshold; a second indicator that the difference in current occupancy between the two second queues exceeds a second threshold; a third indicator that indicates that a current pick rate at which instruction operations are selected from the second queue for allocation to a corresponding execution pipe is below a threshold; The queue controller selecting the queue priority allocation mode in response to the presence of either the first indicator or the second indicator; selecting the pipe priority allocation mode in response to the presence of both the first indicator and the second indicator; selecting the speculative allocation mode in response to the presence of the third indicator; and selecting an allocation mode from the plurality of allocation modes by 17. The processor of claim 16.

19. During the speculative allocation mode, each instruction operation of the set has a pre-assigned execution pipe, and the second queue associated with the pre-assigned execution pipe holds a valid copy of the instruction operation unless a difference between a pipe occupancy of the pre-assigned execution pipe and a pipe occupancy of another execution pipe of the same type associated with another second queue exceeds a threshold.

17. The processor of claim 16.

20. the plurality of allocation modes include, for each of a plurality of sets of instruction operations, a queue priority allocation mode in which the instruction operations of the set are allocated among the plurality of second queues using an interleaved pattern; the queue controller selects the second queue to receive a first instruction operation in the interleaved pattern for a given set according to either a fixed selection sequence or pseudo-randomly. The processor of claim 11.

21. 1. A processor, comprising: Multiple execution pipes; a distributed scheduler coupled to the plurality of execution pipes; The distributed scheduler a first queue configured to buffer instruction operations from a front end of an instruction pipeline of the processor; a plurality of second queues, each second queue configured to buffer instruction operations assigned from the first queue to a different subset of a corresponding one of the plurality of execution pipes; a queue controller; The queue controller For each instruction operation of the subset of instruction operations in the first queue: speculatively allocating copies of the instruction operations from the first queue to each second queue; invalidating copies of the instruction operation in all but any second queues based on current pipe occupancy of execution pipes in which the instruction operation can be executed; configured to: Processor.

22. the instruction operation is pre-assigned to a predetermined execution pipe among the plurality of execution pipes; The queue controller maintaining a copy of the instruction operation in the second queue associated with the given execution pipe as a valid copy of the instruction operation in response to the current pipe occupancy of the given execution pipe being within a threshold of the current pipe occupancy of another execution pipe that can execute the instruction operation; In response to a current pipe occupancy of the given execution pipe exceeding a current pipe occupancy of the execution pipe associated with a different second queue by at least a threshold, invalidating a copy of the instruction operation in the second queue associated with the given execution pipe and maintaining a copy of the instruction operation in the different second queue; configured to:

22. The processor of claim 21.

23. 1. A processor, comprising: Multiple execution pipes; a distributed scheduler coupled to the plurality of execution pipes; The distributed scheduler a first queue configured to buffer instruction operations from a front end of an instruction pipeline of the processor; a plurality of second queues, each second queue configured to buffer instruction operations assigned from the first queue to a different subset of a corresponding one of the plurality of execution pipes; a queue controller; The queue controller allocating sets of instruction operations from the first queue among the plurality of second queues in an interleaved pattern; selecting, for each set, the second queue to which a first instruction operation of the set will be assigned in the interleaved pattern based on either a predetermined sequence or pseudo-randomly; configured to: Processor.

24. the queue controller is configured to select the second queue to which a first instruction action of the set is assigned in the interleaving pattern based on a predetermined sequence, the predetermined sequence being an alternating sequence.

24. The processor of claim 23.

25. A pseudo-random number generator is provided, the queue controller is configured to pseudo-randomly select the second queue to which a first instruction operation of the set is assigned in the interleaved pattern based on an output of the pseudo-random number generator.

24. The processor of claim 23.

Images