Distributed scheduler providing execution pipeline balancing

Abstract

A processor includes a plurality of execution pipelines and a distributed scheduler coupled to the plurality of execution pipelines. The distributed scheduler includes a first queue to buffer instruction operations from a front end of an instruction pipeline of the processor and a plurality of second queues, where each second queue is to buffer, for a corresponding separate subset of execution pipelines of the plurality of execution pipelines, instruction operations allocated from the first queue. The distributed scheduler further includes a queue controller to select an allocation mode from a plurality of allocation modes based on at least one indicator of whether an imbalance at the distributed scheduler is detected and further to control the distributed scheduler to allocate instruction operations from the first queue among the plurality of second queues in accordance with the selected allocation mode.

Description

BACKGROUND

[0001] The distributed scheduler employs at least two levels of scheduler queues, with a first level having a non-selectable scheduler queue and a second level having two or more selectable scheduler queues, and each of the selectable scheduler queues operates to store instruction operations for a corresponding subset of the plurality of execution pipelines. The non-selectable scheduler queue is used to temporarily buffer instruction operations from the instruction pipeline front end prior to their assignment to the selectable scheduler queues.

[0002] Because each selectable scheduler queue is dedicated to servicing its own subset of execution pipelines, there is often the potential for performance inhibiting imbalances between different selectable scheduler queues, between different execution pipelines, or a combination thereof. By way of illustration, due to the particular manner in which a program is constructed or compiled, or due to a coincidence of queue assignments, one selectable scheduler queue can be burdened with many instruction operations that each require a large number of cycles, such as operations that depend on load operations, while another selectable scheduler queue can have a smaller proportion of operations that incur such delays. As another example, another selectable scheduler queue with similar execution pipelines can be assigned a higher number of operations than one selectable scheduler queue, the operations of which can only be executed by a particular execution pipeline associated with the queue. As a result, the execution pipelines associated with one selectable scheduler queue experience excessive occupancy, while another similar execution pipeline is unnecessarily idle. Such skewed queue / pipeline occupancy often results in a lower number of operations being selected per cycle, also yielding underutilized execution pipelines, either of which negatively impacts execution performance at a processor core implementing an instruction pipeline using a distributed scheduler. SUMMARY

[0003] According to one aspect, a method includes buffering instruction operations at a first queue of a distributed scheduler of a processor; selecting a distribution mode of a plurality of distribution modes of the distributed scheduler based on at least one indicator of whether an imbalance at the distributed scheduler is detected; distributing instruction operations from the first queue among a plurality of second queues in accordance with the selected distribution mode; and at each second queue, selecting instruction operations buffered at the second queue for execution by an execution pipe associated with the second queue. The plurality of distribution modes can include a queue- first distribution mode in which instruction operations are substantially equally distributed among the plurality of second queues, and a pipe-first distribution mode in which instruction operations are distributed among the plurality of second queues based on a pipe occupancy of the execution pipe associated with the second queue. In some embodiments, the at least one indicator of imbalance includes at least one of a first indicator representing that at least one of the second queues has a current occupancy that exceeds a first threshold, and a second indicator representing that a difference between current occupancies of any two second queues exceeds a second threshold. Accordingly, selecting one of the plurality of distribution modes can include selecting the queue-first distribution mode in response to neither the first indicator nor the second indicator being detected, and selecting the pipe-first distribution mode in response to at least one of the first indicator and the second indicator being detected. Alternatively, selecting one of the plurality of distribution modes can include selecting the queue-first distribution mode in response to no more than one of the first indicator and the second indicator being detected, and selecting the pipe-first distribution mode in response to both the first indicator and the second indicator being detected.

[0004] In some embodiments, the plurality of allocation modes further includes: a speculative allocation mode in which a copy of each instruction operation in a group of instruction operations is allocated to each of the plurality of second queues, and based on an analysis of pipe occupancy of the execution pipes capable of executing the instruction operations, the copy of each instruction operation of the group is subsequently invalidated in all but one second queue. Thus, selecting one of the plurality of allocation modes can include selecting the speculative allocation mode in response to a current pick rate of selecting instruction operations from the second queues for execution by corresponding execution pipes falling below a threshold. The at least one indicator of imbalance can include at least one of: a first indicator that at least one of the second queues has a current occupancy that exceeds a first threshold; and a second indicator that a difference between current occupancies of any two second queues exceeds a second threshold; and a third indicator that a current pick rate of selecting instruction operations from the second queues for allocation to corresponding execution pipes falls below a threshold; and selecting one of the plurality of allocation modes includes: selecting the queue-first allocation mode in response to there being no more than one of the first indicator or the second indicator; selecting the pipe-first allocation mode in response to there being both the first indicator and the second indicator; and selecting the speculative allocation mode in response to there being the third indicator. During the speculative allocation mode, each instruction operation of the group can have a pre-assigned execution pipe, and the second queue associated with the pre-assigned execution pipe retains a valid copy of the instruction operation unless a difference between pipe occupancy of the pre-assigned execution pipe and pipe occupancy of another execution pipe of the same type associated with another second queue exceeds a threshold.

[0005] In some embodiments, the plurality of allocation modes includes: a queue-first allocation mode in which for each group of instruction operations in a plurality of groups, the instruction operations of the group are allocated among the plurality of second queues using an interleaving pattern; and in which the second queue used to receive an initial instruction operation in the interleaving pattern for a given group is selected according to one of a fixed selection sequence or a pseudo-random.

[0006] According to another aspect, a processor is configured to perform an embodiment of the above-described method. In some embodiments, the processor comprises: a plurality of execution pipelines; and a distributed scheduler coupled to the plurality of execution pipelines and comprising: a first queue configured to buffer instruction operations from a front end of an instruction pipeline of the processor; a plurality of second queues, each second queue configured to buffer, for a corresponding separate subset of execution pipelines of the plurality of execution pipelines, instruction operations allocated from the first queue; and a queue controller configured to: select, based on at least one indicator of whether an imbalance at the distributed scheduler is detected, one allocation mode from a plurality of allocation modes; and control the distributed scheduler to allocate instruction operations from the first queue among the plurality of second queues in accordance with the selected allocation mode.

[0007] According to another aspect, a processor comprises: a plurality of execution pipelines; and a distributed scheduler coupled to the plurality of execution pipelines and comprising: a first queue configured to buffer instruction operations from a front end of an instruction pipeline of the processor; a plurality of second queues, each second queue configured to buffer, for a corresponding separate subset of execution pipelines of the plurality of execution pipelines, instruction operations allocated from the first queue; and a queue controller configured to: for each instruction operation of a subset of instruction operations in the first queue: speculatively allocate a copy of the instruction operation from the first queue to each second queue; and based on a current pipeline occupancy of an execution pipeline capable of executing the instruction operation, invalidate the copy of the instruction operation in all second queues except one second queue. In some embodiments, the instruction operation is pre-issued to a designated execution pipeline of the plurality of execution pipelines; and the queue controller is further configured to: in response to a current pipeline occupancy of the designated execution pipeline being within a threshold of current pipeline occupancies of other execution pipelines capable of executing the instruction operation, maintain the copy of the instruction operation in the second queue associated with the designated execution pipeline as a valid copy of the instruction operation; and in response to the current pipeline occupancy of the designated execution pipeline exceeding the current pipeline occupancy of the execution pipeline associated with the second queue by at least the threshold, invalidate the copy of the instruction operation in a different second queue associated with the designated execution pipeline and maintain the copy of the instruction operation in the different second queue.

[0008] According to yet another aspect, a processor includes: a plurality of execution pipelines; and a distributed scheduler coupled to the plurality of execution pipelines and including: a first queue configured to buffer instruction operations from a front end of an instruction pipeline of the processor; a plurality of second queues, each second queue configured to buffer, for a corresponding separate subset of execution pipelines of the plurality of execution pipelines, instruction operations allocated from the first queue; and a queue controller configured to: allocate groups of instruction operations from the first queue among the plurality of second queues in an interleaved pattern; and for each group, select a second queue to which an initial instruction operation of the group is to be allocated in the interleaved pattern based on one of: a predefined sequence or a pseudorandom manner. In some embodiments, the queue controller is further configured to select the second queue to which the initial instruction operation of the group is to be allocated in the interleaved pattern based on a predefined sequence, wherein the predefined sequence is an alternating sequence. The processor can also include a pseudorandom number generator; and wherein the queue controller is further configured to pseudorandomly select the second queue to which the initial instruction operation of the group is to be allocated in the interleaved pattern based on an output of the pseudorandom number generator. BRIEF DESCRIPTION OF DRAWINGS

[0009] The present disclosure can be better understood, and its numerous features and advantages can become more apparent to one skilled in the art by reference to the following drawing, in which:

[0010] Figure 1 is a block diagram illustrating a processor implementing a multi-mode distributed scheduler according to some embodiments.

[0011] Figure 2 is a block diagram illustrating a multi-mode distributed scheduler of Figure 1 in more detail according to some embodiments.

[0012] Figure 3 is a flow diagram illustrating a queue-first allocation mode of a multi-mode distributed scheduler according to some embodiments.

[0013] Figure 4 is a flow diagram illustrating a pipeline-first allocation mode of a multi-mode distributed scheduler according to some embodiments. Figure 3

[0014] Figure 5 is a flow diagram illustrating a pipeline-first allocation mode of a multi-mode distributed scheduler according to some embodiments.

[0015] Figure 6 is a flow diagram illustrating a speculative allocation mode of a multi-mode distributed scheduler according to some embodiments. ​

[0016] Figure 7 is a diagram illustrating an example implementation of a method according to some embodiments of Figure 6

[0017] Figure 8 is a flow diagram illustrating a method for selectively activating different allocation modes of a multi-mode distributed scheduler based on various imbalance indicia according to some embodiments. DETAILED DESCRIPTION

[0018] Figures 1 to 8 Example techniques are shown for mitigating queue imbalance and pipe imbalance in a processing system employing a multi-mode distributed scheduler for scheduling of instruction operation execution. In at least one embodiment, the distributed scheduler includes an unpickable scheduler queue for temporarily buffering instruction operations from an instruction front end of a processor and two or more pickable scheduler queues for receiving instruction operations from the unpickable scheduler queue, where each of the pickable scheduler queues operates to buffer instruction operations for an associated separate subset of a set of execution pipes serviced by the distributed scheduler. The distributed scheduler further includes a queue controller for configuring the distributed scheduler to allocate instruction operations from the unpickable scheduler queue among the pickable scheduler queues according to a selected allocation mode of a plurality of allocation modes supported by the distributed scheduler. In at least one embodiment, the queue controller selects the allocation mode to implement in response to the presence or absence of one or more indicia of imbalance between the pickable scheduler queues or between execution pipes of the same or similar capabilities. Such indicia include, for example, a queue occupancy above a threshold, a pipe occupancy above a threshold, a difference between queue occupancies of two or more queues above a threshold, a difference between pipe occupancies of two or more execution pipes of the same or similar type above a threshold, a ratio of pickable instruction operations executed falling below a threshold, etc.

[0019] ​In some embodiments, the multiple allocation modes supported by the multi-mode distributed scheduler include a queue-priority allocation mode in which a subset of instruction operations to be allocated to the selectable scheduler queues in a given allocation cycle is allocated substantially equally among the selectable scheduler queues, such as by an interlaced pattern. To further ensure fair distribution of instruction operations, in some embodiments, the interlaced pattern starts with a different selectable scheduler queue for each allocation cycle, with the selectable scheduler queue to receive the initial instruction operation in the interlaced pattern for a given allocation cycle being selected, such as by using an output value of a linear feedback shift register (LFSR) or other pseudo-random number generator, for example, according to a specified fixed selection sequence (e.g., an alternating or "ping-pong" sequence) or pseudo-randomly. Further, in some embodiments, the multiple allocation modes also include a pipe-priority allocation mode in which the pipe occupancy of each of the execution pipes suitable for executing the instruction operations to be allocated is considered, and the execution pipe and corresponding associated selectable queue selected for execution of the instruction operation is based on a comparison and evaluation of these pipe occupancies.

[0020] In some embodiments, the multiple allocation modes supported by the multi-mode distributed scheduler include a speculative allocation mode in which, for a given allocation cycle, a copy of the same instruction operation is allocated speculatively from a non-selectable scheduler queue to each of a plurality of selectable scheduler queues. Thereafter, the queue controller selects an appropriate execution pipe to execute the instruction operation, thus keeping the copy of the instruction operation valid in the selectable scheduler queue associated with the selected execution pipe while invalidating the copy of the instruction operation in the other selectable scheduler queues.

[0021] Figure 1A processor 100 implementing a multi-mode distributed scheduler 102 for queue and execution pipe balancing is shown in accordance with some embodiments. The processor 100 includes an instruction front end 104, one or more instruction execution units such as an integer unit 106 and a floating point / single instruction multiple data (SIMD) unit 108, and a cache / memory interface subsystem 110. The instruction front end 104 operates to fetch instructions as part of an instruction stream, decode the instructions into one or more instruction operations (e.g., micro-operations or uops), and then dispatch each instruction operation to one of the execution units 106, 108 for execution. In executing the instruction operations, the execution units often utilize one or more caches implemented at the cache / memory interface subsystem 110, or access data from or provide data for storage to external memory (e.g., random access memory or RAM) or external I / O devices through a load / store unit (LSU), memory controller, or input / output (I / O) controller (not shown) of the cache / memory interface subsystem 110.

[0022] The processor 100 implements the multi-mode distributed scheduler 102 at each of the one or more execution units of the processor 100. For illustrative purposes, implementations of the multi-mode distributed scheduler 102 at the floating point / SIMD unit 108 are described herein. However, in other embodiments, the integer unit 106 or other execution units of the processor 100 also implement the multi-mode distributed scheduler using the guidance provided herein, in addition to or instead of the scheduler implemented by the floating point / SIMD unit 108.

[0023] The floating-point / SIMD unit 108 includes a rename module 112, a physical register file 114, and a plurality of execution pipelines 116, such as six execution pipelines 116-1 through 116-6 in the depicted embodiment, as well as a multi-mode distributed scheduler 102. The rename module 112 performs renaming operations for instruction operations received from the instruction front end 104, including renaming of architectural registers to physical registers in the physical register file 114, and outputs the renamed instruction operations to the multi-mode distributed scheduler 102 for buffering and dispatch among the execution pipelines 116. In at least one embodiment, the multi-mode distributed scheduler 102 includes first scheduler queues 118, a multiplexer (mux) network 120, a plurality of second scheduler queues 122, and a picker 124 for each second scheduler queue 122, such as the two second scheduler queues 122-1 and 122-2 and corresponding pickers 124-1 and 124-2 shown, as well as a queue controller 126. Each second scheduler queue 122 is used to buffer instruction operations for a corresponding separate subset of the plurality of execution pipelines 116. For example, in the depicted embodiment, the second scheduler queue 122-1 buffers instruction operations for a subset consisting of execution pipelines 116-1, 116-2, and 116-3, while the second scheduler queue 122-2 buffers instruction operations for a separate subset consisting of execution pipelines 116-4, 116-5, and 116-6.

[0024] The multi-mode distributed scheduler 102 implements a two-level queuing process, whereby the first scheduler queues 118 temporarily buffer instruction operations, which are then distributed among the plurality of second scheduler queues 122 through the mux network 120. The picker 124 of each second scheduler queue 122 then picks instruction operations buffered in the corresponding second scheduler queue 122 for dispatch or other allocation to the execution pipelines of the subset associated with the corresponding second scheduler queue 122. Since instruction operations are not directly pickable from the first scheduler queues 118 for execution, the first scheduler queues 118 are referred to herein as “non-pickable scheduler queues 118” or “NSQs 118.” Conversely, since instruction operations are pickable from the second scheduler queues 122 for execution, the second scheduler queues 122 are each referred to herein as “pickable scheduler queues 122” or “SQs 122.”

[0025] In at least one embodiment, multi-mode distributed scheduler 102 supports multiple allocation modes, where each allocation mode represents a particular procedure for allocating instruction operations from NSQ 118 to SQs 122 during a given allocation period. Each allocation mode generally addresses a current or predicted inefficiency in execution, and thus in one embodiment, queue control 126 selects a particular allocation mode to implement during a next or multiple allocation periods based on the presence or absence of one or more imbalance or inefficiency markers that the selected allocation mode can address, and then configures mux network 120 and pickers 124 to implement the selected allocation mode. Examples of allocation modes supported by multi-mode distributed scheduler 102, and an exemplary procedure for selecting a particular allocation mode for implementation, are described in detail below.

[0026] Figure 2 An implementation of multi-mode distributed scheduler 102 is shown in greater detail in accordance with some embodiments. In this example, NSQ 118 includes a plurality of queue entries 202, where each queue entry 202 is capable of storing an instruction operation 204 received from instruction front end 104. NSQ 118 operates as a first-in-first-out (FIFO) queue and includes a plurality of output ports 206, each of which is capable of outputting a corresponding buffered instruction operation 204 from NSQ 118 during an allocation period. Thus, NSQ 118 has the capability of outputting N instruction operations during each allocation period, where N represents the number of output ports 206. Each of SQs 122-1 and 122-2 includes a plurality of queue entries 208, where each queue entry 208 is capable of buffering an instruction operation 204 allocated from NSQ 118 to the corresponding SQ 122. To this end, each SQ 122 includes a plurality of input ports 210, each of which is capable of writing a received instruction operation to a corresponding queue entry 208 of the receiving SQ 122. In at least one embodiment, the total number of input ports 210 of all SQs 122 is equal to the number of output ports 206 of NSQ 118, N. Thus, if each SQ 122 includes K input ports 210, then N = K*J, where J is the number of SQs 122. To illustrate, in the illustrated implementation, NSQ 118 includes six output ports 206 (N = 6), and each of SQ 122-1 and SQ 122-2 (J = 2) includes 3 input ports (K = 3). Thus, in this implementation, up to six (6) instruction operations can be allocated from NSQ 118 to SQs 122-1 and 122-2 during any given allocation period, where SQs 122-1 and 122-2 are each capable of receiving up to three (3) instruction operations during an allocation period.

[0027] The mux network 120 operates to allocate instruction operations 204 from the NSQ 118 among the SQs 122-1 and 122-2 based on a designated allocation pattern. To this end, the mux network 120 includes one or more levels of multiplexers that connect the output ports of the NSQ 118 to the input ports 210 of the SQs 122-1, 122-2 such that each output port 206 can be connected to any of the input ports 210. The queue controller 126 operates to control the mux network 120 through control signaling 212 in order to direct each instruction operation 204 allocated from the NSQ 118 among the associated output ports 206 to the corresponding input port 210 of the SQ 122 to which the instruction operation 204 is being allocated.

[0028] Each queue entry 208 of the SQs 122-1, 122-2 can buffer an instruction operation 204 prior to the instruction operation 204 being selected by the associated picker 124 for execution from the NSQ 118. In the illustrated embodiment, the queue entry 208 includes a number of fields pertaining to the buffered instruction operation 204, including: an opcode field 214 for storing an opcode or other identifier of the type of instruction operation to be executed; and one or more operand fields 216 for storing register identifiers of any source or destination registers to be used during execution of the instruction operation 204 or any immediate or displacement values to be used during such execution. In addition, as described below, in some embodiments the instruction operations are pre-assigned to particular execution pipelines, and thus the queue entry 208 also includes: an assignment field 218 for storing an identifier of any execution pipeline to which the corresponding instruction operation is pre-assigned. Furthermore, to facilitate invalidation and flushing of instruction operations, in at least one embodiment the queue entry 208 includes: a valid field 220 for storing a valid (v) bit indicating whether the corresponding instruction operation is valid or invalid for the SQ 122. In some embodiments, the queue entry 208 includes additional fields (not shown) such as a status field indicating, for example, whether the instruction operation is ready to be selected for execution, a dependency field identifying dependencies between the instruction operation and other instruction operations, etc.

[0029] The pickers 124-1, 124-2 operate to select or "pick" instruction operations from their respective SQs 122 for execution by the selected execution pipelines 116 associated with the SQs 122. This selection process is generally based on the availability of a given instruction operation to be executed at the time, as well as the current capacity of the one or more execution pipelines 116 associated with the SQ 122 that are capable of executing the instruction operation. Thus, the pickers 124 track instruction operations assigned or otherwise allocated to a subset of the corresponding execution pipelines 116, and track the status of the subset of execution pipelines 116 as part of the selection process.

[0030] Each of the execution pipelines 116-1 through 116-6 includes logic and other circuitry to perform one or more instruction operation types. In some embodiments, each execution pipeline has a specialized function, such as an execution pipeline for performing floating point multiplication (FMUL) operations, an execution pipeline for performing floating point addition (FADD) operations, an execution pipeline for performing fused multiply-add (FMU) operations, and so on. Moreover, in some embodiments, the SQs 122 and their associated subset of execution pipelines are “symmetric” in that each SQ 122 has the same configuration of execution pipelines associated therewith. That is, the execution pipeline configuration of the SQ 122-1 is the same as the execution pipeline configuration of the SQ 122-2, such that the execution pipeline 116-1 (EX0) and the execution pipeline 116-4 (EX1) are the same type of execution pipeline, the execution pipeline 116-2 (EX2) and the execution pipeline 116-5 (EX3) are the same type of execution pipeline, and the execution pipeline 116-3 (EX4) and the execution pipeline 116-6 (EX5) are the same type of execution pipeline. Thus, an instruction operation 204 that is suitable for execution at the execution pipeline EX0 will also be suitable for execution at the execution pipeline EX1, and vice versa, with the same relationship of capabilities for the execution pipeline pairs EX2 / EX3 and EX4 / EX5.

[0031] The queue controller 126 includes various components, including a dispatch component 222, a mode control component 224, a queue update component 226, a queue state component 228, a pipeline state component 230, a pick rate component 232, and an interleave selection component 236. Each of these components is implemented as hardware logic and associated circuitry, programmable logic and associated circuitry, a processor core executing software instructions representative of the corresponding functionality, or a combination thereof. The pipeline state component 230 operates to monitor the execution pipelines 116, determining the current pipeline occupancy of each execution pipeline 116. Similarly, the queue state component 228 operates to monitor the NSQs 118 and the SQs 122, determining their current queue occupancy, rate of occupancy change, and so on. The pick rate component 232 operates to monitor instruction operation picking by the pickers 124-1, 124-2 and to determine current pick rate statistics for the pickers 124-1, 124-2. The queue update component 226 operates to update the various fields of the queue entries 208 of the SQs 122-1, 122-2 and the queue entries 202 of the NSQs 118 based on operation of the distributed scheduler 102, as described in detail below. As described below, the interleave selection component 236 operates to select an initial SQ 122 to receive instruction operations in an interleave dispatch mode.

[0032] As described herein, in at least one embodiment, the multi-mode distributed scheduler 102 supports multiple allocation modes in order to facilitate dynamic adaptation of the instruction operation allocation process between the NSQ 118 and the SQs 122-1, 122-2, thereby mitigating performance degradation due to imbalances between the SQs 122-1, 122-2 or between execution pipelines 116 of the same type. As described below, in one embodiment, this multiple allocation modes includes a queue-first allocation mode 240 in which instruction operations are first allocated from the NSQ 118 to the SQs 122 and then, once the instruction operations are in their respective SQs 122, pipeline allocation of the instruction operations is subsequently performed. Another allocation mode includes a pipeline-first allocation mode 242 in which instruction operations are first allocated to the corresponding execution pipeline 116 while buffered in the NSQ 118 and then the instruction operations are allocated to the SQ 122 associated with the dispatched execution pipeline during the NSQ-SQ allocation cycle. As yet another example, the multiple allocation modes supported by the multi-mode distributed scheduler 102 includes a speculative allocation mode 244 in which, in the same allocation cycle, a copy of each instruction operation in a subset of instruction operations is speculatively allocated from the NSQ 118 to each of the SQs 122-1, 122-2 and then pipeline dispatch is subsequently completed with only the copy of the instruction cycle in the SQ 122 associated with the dispatched execution pipeline being maintained while the other copy in the other SQ 122 is invalidated and subsequently flushed. Reference is made below to Figures 3 to 7 These various allocation modes are described in more detail.

[0033] The mode control component 224 operates to select the particular allocation mode implemented in one or more allocation cycles in response to a determination of the presence or absence of one or more imbalance indicia detected between the SQs 122 or between the execution pipelines as determined from information provided by other components of the queue controller 126, such as the selection rate statistics from the selection rate component 232, the current pipeline occupancy from the pipeline state component 230, the current queue occupancy from the queue state component 228, etc. Reference is made below to Figure 8 An exemplary process of selecting the allocation mode is described. The dispatch component 222 operates to control (e.g., through configuration of the control signaling 212) the allocation of instruction operations from the NSQ 118 to the SQs 122 and to the execution pipelines 116 based on the selected allocation mode and based on various queue and execution pipeline state information.

[0034] Figure 3An exemplary method of instruction operation allocation is shown for a queue- first allocation mode 240 for the multi-mode distributed scheduler 102 in accordance with some embodiments. As noted, the queue-first allocation mode 240 provides allocation of instruction operations 204 from the NSQ 118 to the SQs 122-1, 122-2 for queue capacity balancing purposes, with pipeline dispatch of instruction operations occurring after allocation of instruction operations to the SQs 122-1, 122-2. Since each SQ 122 has K input ports 210 (in this example, K = 4), the queue-first allocation mode 240 allocates instruction operations 204 from the NSQ 118 to the SQs 122-1, 122-2 in a round-robin manner, with each SQ 122-1, 122-2 receiving an equal number of instruction operations 204 from the NSQ 118. In this example, each SQ 122-1, 122-2 receives 4 instruction operations 204 from the NSQ 118, for a total of 8 instruction operations 204 allocated from the NSQ 118 to the SQs 122-1, 122-2. Figure 2In the example where K = 3, each SQ 122 thus receives up to K instruction operations in any given allocation period. One possible approach is thus to allocate the first K instruction operations in NSQ 118 to SQ 122-1 and the next K instruction operations in NSQ 118 to SQ 122-2 in an allocation period. However, such a block allocation approach can create imbalances, for example when one block of K instruction operations includes instruction operations that depend on load operations, store operations, or other operations that require a large number of clock cycles to complete, while another block of K instruction operations has few or no such instruction operations. One approach to mitigating this is to employ an interleaved pattern for instruction allocation among SQs 122 during an allocation period, such that one SQ 122 gets one instruction operation, another SQ 122 gets the next instruction operation, and so on. In this way, each SQ 122 gets one instruction operation in sequence, and then the sequence is repeated again across all SQs 122 until all instruction operations for the current allocation period have been allocated. However, when the same SQ 122 acts as the first SQ to receive instruction operations in an interleaved pattern for each allocation period, the situation can result in imbalanced occupancy among SQs 122. To illustrate, if a program being executed has a sequence of operations that alternate between add and multiply such as [ADD, MUL, ADD, MUL, ADD, MUL, ADD, MUL, ADD, MUL, ADD, MUL], an interleaved allocation pattern that provides the first or initial instruction operation in each allocation period to the same SQ 122 between two SQs 122 will result in six ADD operations being allocated to that SQ 122, while the six MUL operations (which take significantly longer to execute compared to ADD operations) end up in the other SQ 122. Thus, to avoid such possible queue imbalances, in at least one embodiment, when in a queue-first allocation mode 240 that uses an interleaved pattern, queue controller 126 implements a selection process for selecting the SQ 122 that receives the first or initial instruction operation in a subset of instruction operations to be allocated in a given period, where this selection process is based on, for example, an alternating or ping-pong selection or a pseudo-random selection. By sequentially or pseudo-randomly changing the initial receiving SQ 122 between allocation periods, a given stream of instruction operations will generally result in a reduced probability of imbalanced operation allocation among SQs 122 and their corresponding execution pipelines 116 compared to implementations in which the same SQ 122 is selected as the initial receiving SQ for each allocation period.

[0035] Accordingly, in response to initiation of an allocation cycle, at block 302, the interleave selection component 236 selects one of the SQs 122-1, 122-2 to receive the initial instruction operations allocated from the NSQ 118 for the allocation cycle. That is, the interleave selection component 236 selects one of the SQs 122-1, 122-2 as the “first” SQ 122 in an interleave pattern. In one embodiment, the interleave selection component 236 utilizes a predefined fixed selection sequence, such as a 1-2-1-2-1-2-1... sequence (i.e., a ping-pong sequence) or a 1-1-2-2-1-1-2-2 sequence for two SQs 122, or a 1-2-3-1-2-3-1-2-3-1... sequence or a 1-2-3-2-3-1... sequence for three SQs 122 implementations, etc. In other embodiments, the interleave selection component 236 implements a pseudo-random number generator, such as a linear feedback shift register (LFSR), to generate a pseudo-random number for each allocation cycle, and then uses one or more bits of the pseudo-random number generator output to select the initial receiving SQ 122 for the interleave pattern to be employed for the current allocation cycle. With the initial receiving SQ 122 selected, at block 304, the dispatch component 222 controls the mux network 120 to distribute the subset of instruction operations from the NSQ 118 to the SQs 122-1, 122-2 210 according to the interleave pattern, starting with the selected initial receiving SQ 122 through the output port 206 and the input ports.

[0036] Figure 4 An example of this process is shown in an implementation in which six instruction operations are allocated from the NSQ 118 in an interleave pattern using a ping-pong initial receiving SQ selection sequence among two SQs 122-1, 122-2. Initially, the NSQ 118 buffers twelve instruction operations OP 1 through OP 12, with OP 1 being the oldest instruction operation (and thus the first to be allocated). During an allocation cycle 401, a subset of six instruction operations (OP 1 through OP 6) is allocated. For this allocation cycle, SQ 122-1 is selected as the initial receiving SQ, so the interleave allocation pattern results in operations OP 1, OP 3, and OP 5 being allocated to SQ 122-1 and operations OP 2, OP 4, and OP 6 being allocated to SQ 122-2. For the next allocation cycle 402, a subset of six instruction operations OP 7 through OP 12 is allocated, and for this allocation cycle, SQ 122-2 is selected as the initial receiving SQ using the ping-pong selection pattern. Accordingly, during the allocation cycle 402, the interleave allocation pattern results in instruction operations OP 7, OP 9, and OP 11 being allocated to SQ 121-2 and instruction operations OP 8, OP 10, and OP 12 being allocated to SQ 121-1.

[0037] Figure 5 An exemplary method of instruction operation assignment is shown for the pipe-first assignment mode 242 of the multi-mode distributed scheduler 102 according to some embodiments. While the queue-first assignment mode 240 described above has the advantage of allowing all of the output ports 206 to be utilized in an assignment cycle, thus maximizing the number of instruction operations assigned from the NSQ 118 to the SQs 122 in an assignment cycle, it has the disadvantage that the circumstances in the stream of instruction operations can cause pipe utilization imbalance. To illustrate, queue-first assignment can result in an imbalance of ADD operations in one SQ 122 compared to another SQ 122, while the other SQ 122 ends up with an imbalance of MUL operations. In this case, the execution pipe 116 of the first SQ 122 for ADD operations can be congested, while the execution pipe 116 of the first SQ 122 for MUL operations is idle, and the opposite is true for the second SQ 122, as its ADD type execution pipe 116 is idle, while its MUL type execution pipe 116 is overloaded. The pipe-first assignment mode 242 seeks to mitigate such imbalances by evaluating the pipe occupancy of all execution pipes 116 in the distributed scheduler 102, and then assigning each instruction operation of a subset of instruction operations to be assigned in the current assignment cycle to the execution pipe with the lowest pipe occupancy that is capable of executing the operation type represented by the instruction operation.

[0038] Therefore, for a given allocation period, at block 502, mode control unit 224 evaluates the current pipeline occupancy of a set of execution pipelines 116 of the distributed scheduler 102, for example, by accessing pipeline occupancy data from memory, register files, or other storage units maintained by pipeline status unit 230. Typically, pipeline occupancy data indicates the number of instruction operations currently assigned to each of the execution pipelines 116, thus representing the total amount of operational "work" submitted to each execution pipeline 116. Using this information, at block 504, dispatch unit 222 selects the next instruction operation in NSQ 118 to be allocated in the current allocation period, identifies the execution pipeline 116 capable of executing the selected instruction operation, and then dispatches the selected instruction operation to the execution pipeline 116 with the lowest pipeline occupancy within the identified subset. This dispatch is performed, for example, by writing the value of the identified selected execution pipeline to the dispatch field of the queue entry 202 of NSQ 118 storing the selected instruction operation. When an instruction operation is available for allocation to SQ 122 (e.g., when the instruction operation is included in a subset of instruction operations to be allocated in the current allocation period), at block 506, dispatching component 222 accesses the dispatch field of queue entry 202 storing instruction operations to determine the pre-assigned execution pipeline 116, and controls mux network 120 to route instruction operations from NSQ 118 to entry 208 of SQ 122 associated with the assigned execution pipeline 116. The processes of blocks 502, 504, and 506 are executed for each instruction operation in the subset of instruction operations to be allocated in a given allocation period.

[0039] The pipe utilization focus of pipe priority allocation mode 242 is generally effective in maintaining pipe utilization balance within the distributed scheduler 102. However, unlike queue priority allocation mode 240, which maximizes NSQ-to-SQ allocation throughput, situations may arise where a sequence of instruction operations in NSQ 118 is assigned to a specific SQ 122. This could, in turn, limit the number of instruction operations allocated in a corresponding allocation cycle, due to the smaller number of input ports 210 on a given SQ 122 compared to the total number of output ports 206 of NSQ 118. For example, if Figure 2 The next six instruction operations in NSQ 118 are pre-assigned to execution pipeline 116 associated with SQ 122-2. Therefore, during the next allocation cycle, only three of the total six possible instruction operations can be allocated because... Figure 2 The SQ 122-2 has only three input ports 210. In contrast, queue-first allocation mode 240 would allow all six output ports 206 to be used for allocating six instruction operations under the same conditions (assuming sufficient capacity in the SQ 122). See the following reference. Figure 8In more detail, the multi-mode distributed scheduler 102 leverages the advantages of each of the queue-first allocation mode 240 and the pipe-first allocation mode 242 by switching between these allocation modes in response to detecting imbalances between the SQs 122 or imbalances between the execution pipes 116.

[0040] Figure 6 An exemplary method for the speculative allocation mode 244 provided by the multi-mode distributed scheduler 102 is shown in accordance with some embodiments. Generally, the distributed scheduler 102 is designed and manufactured such that once an instruction operation is allocated to one of the SQs 122, it cannot be moved to another SQ 122. Thus, the distributed scheduler 102 has little way to correct imbalances caused by instruction operations committed to the SQs 122 using the queue-first allocation mode 240 or the pipe-first allocation mode 242. However, if the current pick rate of the execution pipes 116 is low enough that underflow of the SQs 122 is not a significant risk, the distributed scheduler 102 employs the speculative allocation mode 244 to allocate instruction operations to the SQs 122 in a manner that allows the distributed scheduler 102 to provide balance between instruction operations once they are in the SQs 122.

[0041] For the process of speculative allocation in the speculative allocation mode 244, at block 602, the mode control component 224 assesses the current pipe occupancy of the set of execution pipes 116 of the distributed scheduler 102 by, for example, accessing pipe occupancy data as described above. At block 604, the dispatch component 222 identifies instruction operations that are not dispatched in the NSQ 118 and pre-dispatches the identified instruction operations to particular execution pipes 116 based on the current pipe occupancy of those execution pipes 116 that are capable of executing the instruction operation type. This pre-dispatch is reflected by, for example, storing an identifier of the pre-dispatched execution pipe to the dispatch field of the queue entry 202 storing the instruction operation in the NSQ 118. This pre-dispatch process is repeated for any number of instruction operations that are not dispatched in the NSQ 118.

[0042] When a pre-dispatched instruction operation is available for allocation to the SQs 122 (e.g., when the instruction operation is included in the subset of instruction operations to be allocated in the current allocation cycle), rather than allocating the instruction operation to a single SQ 122, at block 606, the dispatch component 222 routes a copy of the instruction operation to each of the SQs 122 (e.g., one copy of the instruction operation to SQ 122-1 and another copy to SQ 122-2) regardless of which SQ 122 is associated with the execution pipe 116 to which the instruction operation was pre-dispatched. This process is repeated for each pre-dispatched instruction operation of the subset to be allocated during the current allocation cycle.

[0043] Thereafter, at block 608, the dispatch component 222 selects the next speculatively dispatched instruction operation in the SQ 122 in order and determines whether there is an imbalance in the current pipe occupancy of the pre-dispatched execution pipe 116 and the same or similar type of execution pipe 116 associated with the other SQ 122. By way of illustration, if the instruction operation is pre-dispatched to the execution pipe 116-2 (EX2) associated with the SQ 122-1, the pipe occupancy of the execution pipe 116-2 (EX2) will be compared to the current execution pipe occupancy of the execution pipe 116-5 (EX3) associated with the SQ 122-2. For this process, an imbalance between the pipe occupancy of the pre-dispatched execution pipe 116 and other similar execution pipes in another SQ 122 can be indicated by, for example, the pipe occupancy of the pre-dispatched execution pipe 116 exceeding the pipe occupancy of the same type of execution pipe 116 in another SQ 122 by at least a threshold amount. For example, assuming a threshold is set to three operations, when the pre-dispatched execution pipe 116 has 8 running instruction operations and the other same type of execution pipe 116 has 6 instruction operations, or when the other same type of execution pipe 116 has more running instruction operations than the pre-dispatched execution pipe 116, there is no imbalance, whereas when the pre-dispatched execution pipe 116 has 12 running instruction operations and the other same type of execution pipe 116 has 5 running instruction operations, an imbalance is detected.

[0044] When no sufficient imbalance in pipe occupancy is detected at block 608, then at block 610, the dispatch component 222 directs the queue update component 226 to maintain the valid state of the copy of the instruction operation in the SQ 122 associated with the pre-dispatched execution pipe 116 and invalidate the copy of the instruction operation in the other SQ 122. That is, when no sufficient imbalance is detected, the queue controller 126 maintains the pre-dispatch of the instruction operation by maintaining the copy of the instruction operation in the SQ 122 associated with the pre-dispatched execution pipe 116 while invalidating all other copies of the instruction operation in the other SQ 122. In at least one embodiment, this invalidation is accomplished by clearing the valid bit in the valid field 220 of the queue entry 208 storing the copy of the instruction operation to be invalidated. In doing so, the entry 208 becomes available to flush the invalid copy of the instruction operation so as to make the entry 208 available to store a different instruction operation.

[0045] Otherwise, when sufficient imbalance in pipe occupancy is detected at block 608, then at block 612, dispatch component 222 evaluates various parameters (such as current pipe occupancy, subsequent instruction operands pre-dispatched to each of the execution pipes 116 capable of performing the type of instruction operation, etc.) to dynamically select the appropriate execution pipe 116 to which the instruction operation is ultimately dispatched. In some cases, the selected execution pipe 116 is the same execution pipe to which the instruction operation was pre-dispatched. In other cases, another execution pipe 116 having the same or similar capabilities is better positioned to provide timely execution of the instruction operation, so the dispatch of the instruction operation switches to this other execution pipe. In either case, dispatch component 222 directs queue update component 226 to maintain the valid state of the instruction operation copy in the SQ 122 associated with the selected execution pipe 116 and to invalidate the copy of the instruction operation in the other SQ 122. It should be appreciated that for blocks 610 and 612, when more than two SQs 122 are implemented, only a single copy of the instruction operation is maintained as valid (that is, the copy in the SQ associated with the execution pipe to which the instruction operation is ultimately dispatched) and all other copies in the other SQs 122 are invalidated.

[0046] The speculative dispatch mode 244 allows the queue controller 126 to more finely tune pipe occupancy, thereby enabling improved pipe balancing and pipe throughput by delaying the ultimate dispatch of each instruction operation until the instruction operation is assigned to a pickable scheduler queue level rather than requiring the execution pipe to be assigned after the unpickable scheduler queue level. This delay allows the queue controller 126 additional time to evaluate the pipe occupancy of compatible execution pipes 116 closer to the point at which the instruction operation will be picked for execution, so the queue controller 126 is able to make more accurate dispatch decisions. However, the speculative dispatch mode 244 requires the use of two output ports 206 for each instruction operation assigned from the NSQ 118 to the SQ 122 (one output port 206 for each copy of the assigned instruction operation), so the effective dispatch rate of instruction operations per dispatch cycle is only one-half of the maximum NSQ-to-SQ assignment rate. Thus, as described below with reference to FIG. 6B, the speculative dispatch mode 244 is more effective when the current pick rate of the execution pipes 116 is sufficiently low so as to reduce or eliminate the likelihood of queue underflow due to the relatively slow input rate provided by the speculative dispatch mode 244. Figure 8 More particularly, the speculative dispatch mode 244 is more effective when the current pick rate of the execution pipes 116 is sufficiently low so as to reduce or eliminate the likelihood of queue underflow due to the relatively slow input rate provided by the speculative dispatch mode 244.

[0047] Figure 7An instance of the speculative allocation process in implementations that show three copies of instruction operations OP 1, OP 2, and OP 3 being speculatively allocated from the NSQ 118 to each of the two SQs 122-1, 122-2. During the pre-dispatch process (block 604), OP 1 is pre-dispatched to execution pipe 116-1 (EX0), OP 2 is pre-dispatched to execution pipe 116-5 (EX3), and OP 3 is pre-dispatched to execution pipe 116-3 (EX4). As shown in stage 701, during the speculative allocation process (block 606), a copy of each of the instruction operations OP 1, OP 2, and OP 3 is allocated to both SQ 122-1 and SQ 122-2. At this time, the validity status of these operations is marked with an "X" in the valid field 220 of the corresponding queue entry 208 to indicate that the validity of these copies of the instruction operations is not relevant at this time. Thus, in some implementations, the copies are initially set to a valid state, while in other implementations, the copies are initially set to an invalid state. Figure 2 ) in the valid field 220 of the corresponding queue entry 208 to indicate that the validity of these copies of the instruction operations is not relevant at this time. Thus, in some implementations, the copies are initially set to a valid state, while in other implementations, the copies are initially set to an invalid state.

[0048] As represented by stage 702, after the initial allocation, the dispatch component 222 determines the current pipe occupancy of the execution pipes 116 that are capable of executing the instruction operation OP 1, which in this instance indicates that execution pipe 116-1 (EX0) has 10 pending instruction operations, while execution pipe 116-4 (EX1) has only one pending instruction operation. Thus, the difference in pipe occupancy is 9 operations. In this particular instance, the threshold that indicates pipe imbalance is six (6) operations. Thus, the difference in pipe occupancy exceeds this threshold, signaling a pipe imbalance. Accordingly, although OP 1 was pre-dispatched to execution pipe 116-1 (EX0) associated with SQ 122-1 due to the identified pipe imbalance, the dispatch component 222 evaluates this pipe occupancy imbalance in view of other conditions such as any difference in queue occupancy between Q 122-1 and 122-2 to dynamically select the SQ 122 and the corresponding execution pipe that is best positioned to execute the instruction operation OP 1. In this instance, the dispatch component 222 determines that the instruction operation OP 1 should be dispatched to execution pipe 116-4 (EX1), thus re-dispatching the instruction operation OP 1 to the execution pipe 116-4 associated with SQ 122-2. Thus, as represented by stage 703, the queue update component 226 signals this final dispatch by clearing the valid field (e.g., setting to "0") of the entry 208 in SQ 122-1 that stores the corresponding copy of the instruction operation OP 1 for SQ 122-1, and asserting the valid field (e.g., setting to "1") of the entry 208 in SQ 122-2 that stores the corresponding copy of the instruction operation OP 1 for SQ 122-2.

[0049] Next, as represented by stage 704, dispatch component 222 determines the current pipe occupancy of execution pipes 116 capable of executing instruction operation OP 2, which in this example indicates that execution pipe 116-2 (EX2) has 8 pending instruction operations, while execution pipe 116-5 (EX3) has 3 pending instruction operations. Since the current pipe occupancy of the pre-dispatched execution pipe (EX3) is less than the current pipe occupancy of the alternative execution pipe (EX2) in the other SQ 122 by more than the threshold amount of 6 operations, no pipe imbalance is signaled. Thus, since OP 2 was pre-dispatched to execution pipe 116-5 (EX3) associated with SQ 122-2, dispatch component 222 maintains this dispatch as the final dispatch. Thus, as represented by stage 705, queue update component 226 signals this final dispatch by clearing the valid field of entry 208 in SQ 122-1 storing the corresponding copy of instruction operation OP 2 for SQ 122-1, and asserting the valid field of entry 208 in SQ 122-2 storing the corresponding copy of instruction operation OP 2 for SQ 122-2.

[0050] Thereafter, as represented by stage 706, dispatch component 222 determines the current pipe occupancy of execution pipes 116 capable of executing instruction operation OP 3, which in this example indicates that execution pipe 116-3 (EX4) has 9 pending instruction operations, while execution pipe 116-5 (EX3) has 6 pending instruction operations. Since the current pipe occupancy of the pre-dispatched execution pipe (EX4) does not exceed the current pipe occupancy of the alternative execution pipe (EX5) in the other SQ 122 by at least the threshold amount of 6 operations, no pipe imbalance is signaled. Thus, since OP 3 was pre-dispatched to execution pipe 116-3 (EX4) associated with SQ 122-1, dispatch component 222 maintains this dispatch as the final dispatch. Thus, as represented by stage 707, queue update component 226 signals this final dispatch by clearing the valid field of entry 208 in SQ 122-2 storing the corresponding copy of instruction operation OP 3 for SQ 122-2, and asserting the valid field of entry 208 in SQ 122-1 storing the corresponding copy of instruction operation OP 3 for SQ 122-1.

[0051] Figure 8An exemplary method 800 for allocation mode selection by a multi-mode distributed scheduler 102 is shown, in accordance with some embodiments. As explained above, each of the queue-first allocation mode 240, the pipe-first allocation mode 242, and the speculative allocation mode 244 is effective in mitigating imbalances in some cases, but can contribute to (or at least not significantly correct) imbalances in other cases. Accordingly, as shown by the method 800, in at least one embodiment, the distributed scheduler 102 operates to switch between two or more allocation modes in response to certain actual or anticipated imbalances. In this way, the distributed scheduler 102 selects an allocation mode that is particularly well suited to address a detected imbalance or to avoid the formation of an anticipated imbalance.

[0052] In one embodiment, the iteration of the method 800 is initiated at block 802, where the mode control component 224 evaluates the queue state information and the pipe state information obtained by the queue state component 228 and the pipe state component 230, respectively, to determine whether there is a pipe imbalance, a queue imbalance, or both. To illustrate, in some embodiments, an actionable imbalance is signaled if there is an indicator that the difference in pipe occupancy between two execution pipes of the same or similar type exceeds a specified threshold. In other embodiments, an actionable imbalance is signaled if there is an indicator that the difference in queue occupancy between the SQs 122 exceeds a specified threshold. In yet other embodiments, an actionable imbalance is signaled only if there are both a pipe imbalance indicator and a queue imbalance indicator. To illustrate, in some embodiments, an imbalance is signaled in the distributed scheduler 102 only if the queue occupancy of one of the SQs 122 exceeds the queue occupancy of the other SQ 122 by at least a first threshold amount and the pipe occupancy of one execution pipe 116 exceeds the pipe occupancy of another execution pipe 116 of the same type by at least a second threshold amount. Otherwise, in this embodiment, an imbalance is not signaled even if one of the two thresholds is exceeded.

[0053] If the presence of queue / pipeline imbalance is detected at block 802, then at block 804 the mode control component 224 configures the queue controller 126 to implement the pipeline-first allocation mode 242 for the subsequent allocation period until the allocation mode changes. Otherwise, if the absence of queue / pipeline imbalance is deemed at block 802, then at block 806 the mode control component 224 queries the pick rate component 232 to determine the current pick rate of the execution pipelines 116 and compares this average pick rate to a defined minimum pick rate threshold. In one embodiment, the current pick rate represents an average pick rate over a sliding time window, where the average is, for example, an averaging of the pick rates of all execution pipelines 116 over the sliding window, an averaging of the pick rates of a particular type of execution pipeline 116 over the sliding window, an averaging of the pick rate of a single execution pipeline 116, etc. If the average pick rate is not sufficient (that is, does not exceed this minimum pick rate threshold), then at block 808 the mode control component 224 selects the speculative allocation mode 244 to implement at the distributed scheduler 102 for the subsequent allocation period until the average pick rate rises above a specified threshold, which is the same or different threshold used to place the distributed scheduler 102 into the speculative allocation mode 244. As one example, a threshold pick rate of 2.5 operations per period is used to trigger entry into the speculative allocation mode, while a threshold pick rate of 2.65 operations per period is used to trigger exit from the speculative allocation mode 244. Thus, if the average pick rate is sufficient (that is, exceeds the minimum pick rate threshold), then at block 810 the mode control component 224 selects the queue-first allocation mode 240 for the distributed scheduler 102 to implement for the subsequent allocation period. The method 800 then returns to block 802, where the mode control component 224 monitors or polls the queue status component 228 and the pipeline status component 230 with the corresponding monitor / polling period to detect any pipeline / queue imbalances or pick rate drops that have occurred since.

[0054] While the method 800 represents a flow that first checks for pipeline / queue imbalances and then analyzes the pick rate for a given analysis period, in other embodiments the pick rate is first analyzed, and if found to be sufficient, the pipeline / queue imbalances are evaluated. Further, in at least some embodiments the allocation mode selection process represented by the method 800 is implemented as a hardware state machine, where the presence of pipeline / queue imbalances triggers a mode change from the queue-first allocation mode 240 to the pipeline-first allocation mode 242, and where detection of a pick rate deficiency triggers a change from whatever allocation mode is currently implemented to the speculative allocation mode 244.

[0055] In some embodiments, the above-described devices and techniques are implemented in systems that include one or more integrated circuit (IC) devices (also referred to as integrated circuit packages, or microchips). Such IC devices are typically included in a machine, such as a computer, a cell phone, a gaming device, a television, a set-top box, a server, a router, a network switch, a network bridge, and so on. In some embodiments, the above-described devices and techniques are implemented in systems that include one or more IC devices and one or more other components, such as components that are not IC devices (e.g., discrete electronic components, one or more memories and / or storage devices, one or more communication interfaces, one or more user interfaces, and so on). Figures 1 to 8The described implementations are implemented in a processor 100 implementing a multi-mode distributed scheduler 102. Electronic design automation (EDA) and computer-aided design (CAD) software tools are often used in the design and fabrication of these IC devices. These design tools are typically represented as one or more software programs. The one or more software programs include code that is executable by a computer system to manipulate the computer system to operate on code represented in the one or more IC devices, so as to perform at least a portion of a process for designing or adapting a manufacturing system to manufacture the circuit. Such code includes instructions, data, or combinations of code and data. The software instructions representing a design tool or a fabrication tool are typically stored in a computer readable storage medium accessible to the computing system. Likewise, the code representing one or more phases of the design or fabrication of an IC device is stored in the same or a different computer readable storage medium and accessed from it.

[0056] A computer readable storage medium includes any non-transitory storage medium, or combination of non-transitory storage mediums, accessible by a computer system during use to provide instructions and / or data to the computer system. Such storage media include, but are not limited to, optical media (e.g., compact disc (CD), digital versatile disc (DVD), Blu-Ray disc), magnetic media (e.g., floppy disc, magnetic tape, or magnetic hard drive), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or Flash memory), or microelectromechanical systems (MEMS)-based storage media. The computer readable storage medium can be embedded in the computing system (e.g., system RAM or ROM), fixedly attached to the computing system (e.g., a magnetic hard drive), removably attached to the computing system (e.g., an optical disc or Universal Serial Bus (USB)-based Flash memory), or coupled to the computer system via a wired or wireless network (e.g., network accessible storage (NAS)).

[0057] In some embodiments, certain aspects of the techniques described above are implemented by one or more processors of a processing system executing software. The software includes one or more sets of instructions stored or otherwise tangibly embodied on a non-transitory computer- readable storage medium. The software can include the instructions and certain data that, when executed by the one or more processors, manipulate the one or more processors to perform one or more aspects of the described techniques. The non-transitory computer-readable storage medium can include, for example, a magnetic or optical disk storage device, solid state storage devices such as Flash memory, cache memory, random access memory (RAM) or other non-volatile memory device or devices, etc. The executable instructions stored on the non-transitory computer-readable storage medium can be in source code, assembly language code, object code, or other instruction format that is interpreted or otherwise executable by one or more processors.

[0058] It should be noted that not all of the activities or elements described above in the general description are required, that a portion of a specific activity or device can not be required, and that one or more further activities can be performed, or elements included, in addition to those described. Still further, the order in which activities are listed are not necessarily the order in which activities are performed. Also, the concepts have been described with reference to particular embodiments. Various modifications and changes can be made thereto without departing from the scope of the disclosure as set forth in the following claims. The disclosure and the descnption of the application are not intended to be limited to the subject matter recited in the closing clauses of the claims. Accordingly, although specific embodiments have been illustrated and described herein, it should be appreciated that the application is not limited to the exact construction described above. Only the claims issued by the patent office are restrictive. Changes can be made to the embodiments above without departing from the broad concepts disclosed above, and replacing like components with other like components will also contribute to obtaining the benefits of the concepts disclosed above.

[0059] Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems and any one or more of the features can be implemented or accomplished by other features as can be gained from the teachings of the present disclosure. The implementation of the foregoing is not intended to suggest that architectural standards are in any way necessary or necessary to implementation of the application. Furthermore, to those skilled in the art of the art to which the application relates, the consequences will be apparent, the disclosure is merely illustrative of specific embodiments thereof and is presented for purposes of example and not limitation. Therefore, obvious changes are contemplated as falling within the scope of the disclosed subject matter. Accordingly, the particular embodiments disclosed are illustrative only and, as such, not limiting of the disclosure. Rather, the scope of the disclosure is to be determined with reference to the following claims.

Claims

1. A method comprising: buffering instruction operations at a first queue of a multi-mode distributed scheduler of a processor; selecting, based on at least one indicator of whether an imbalance at the multi-mode distributed scheduler is detected, a distribution mode of a plurality of distribution modes of the multi-mode distributed scheduler, wherein each distribution mode of the plurality of distribution modes corresponds to a respective configuration of at least a portion of the multi-mode distributed scheduler; allocating instruction operations from the first queue among a plurality of second queues in accordance with the selected distribution mode; and at each second queue, selecting instruction operations buffered at the second queue for execution by an execution pipe associated with the second queue.

2. The method of claim 1, wherein the plurality of distribution modes comprises: a queue-priority distribution mode in which instruction operations are allocated substantially equally among the plurality of second queues; and a pipe-priority distribution mode in which instruction operations are allocated among the plurality of second queues based on pipe occupancy of the execution pipes associated with the second queues.

3. The method of claim 2, wherein the at least one indicator of imbalance comprises at least one of: a first indicator that at least one of the second queues has a current occupancy that exceeds a first threshold; and a second indicator that a difference between current occupancies of any two second queues exceeds a second threshold.

4. The method of claim 3, wherein selecting a distribution mode of the plurality of distribution modes comprises: selecting the queue-priority distribution mode in response to neither the first indicator nor the second indicator being detected; and selecting the pipe-priority distribution mode in response to at least one of the first indicator and the second indicator being detected.

5. The method of claim 3, wherein selecting a distribution mode of the plurality of distribution modes comprises: selecting the queue-priority distribution mode in response to no more than one of the first indicator and the second indicator being detected; and selecting the pipe-priority distribution mode in response to both the first indicator and the second indicator being detected.

6. The method of claim 2, wherein the plurality of distribution modes further comprises: a speculative distribution mode in which a copy of each instruction operation of a group of instruction operations is allocated to each of the plurality of second queues, and subsequently invalidated in all but one second queue based on an analysis of pipe occupancy of execution pipes capable of executing the respective instruction operation.

7. The method of claim 6, wherein selecting a distribution mode of the plurality of distribution modes comprises: selecting the speculative distribution mode in response to a current pick rate of selecting instruction operations from the plurality of second queues for execution by corresponding execution pipes falling below a threshold.

8. The method of claim 6, wherein: the at least one indicator of imbalance comprises at least one of: ​ ​ ​ ​ a first indicator that at least one of the second queues of the plurality of second queues has a current occupancy that exceeds a first threshold; and a second indicator that a difference between current occupancies of any two of the second queues of the plurality of second queues exceeds a second threshold; and a third indicator that a current rate of selection of instruction operations from the second queues for allocation to corresponding execution pipelines falls below a threshold; and selecting an allocation mode of the plurality of allocation modes comprises: selecting the queue-priority allocation mode in response to there being no more than one of the first indicator or the second indicator; selecting the pipeline-priority allocation mode in response to there being both the first indicator and the second indicator; and selecting the speculative allocation mode in response to there being the third indicator.

9. The method of claim 6, wherein: during the speculative allocation mode, each instruction operation of the group has a pre-assigned execution pipeline, and the second queue associated with the pre-assigned execution pipeline retains a valid copy of the instruction operation unless a difference between a pipeline occupancy of the pre-assigned execution pipeline and a pipeline occupancy of another execution pipeline of the same type associated with another second queue exceeds a threshold.

10. The method of any one of claims 1 to 9, wherein the plurality of allocation modes comprises: a queue-priority allocation mode in which, for each group of instruction operations of a plurality of groups, the instruction operations of the group are allocated among the plurality of second queues using an interleaved pattern; and wherein a second queue that receives an initial instruction operation of a given group in the interleaved pattern is selected according to one of: a fixed selection sequence or a pseudo-random manner.

11. A processor comprising: a plurality of execution pipelines; and a multi-mode distributed scheduler coupled to the plurality of execution pipelines and comprising: a first queue configured to buffer instruction operations from a front-end of an instruction pipeline of the processor; a plurality of second queues each configured to buffer instruction operations allocated from the first queue for a corresponding separate subset of execution pipelines of the plurality of execution pipelines; and a queue controller configured to: select an allocation mode of a plurality of allocation modes based on whether at least one indicator of imbalance at the multi-mode distributed scheduler is detected, wherein each allocation mode of the plurality of allocation modes corresponds to a respective configuration of at least a portion of the multi-mode distributed scheduler; and configure the multi-mode distributed scheduler to allocate instruction operations from the first queue among the plurality of second queues according to the selected allocation mode.

12. The processor of claim 11, wherein the plurality of allocation modes comprises: a queue-priority allocation mode in which instruction operations are allocated substantially equally among the plurality of second queues; and a pipeline-priority allocation mode in which instruction operations are allocated to execution pipelines of the same type substantially equally among the plurality of second queues. ​ ​ a pipeline preferential allocation mode in which instruction operations are allocated among the plurality of second queues based on a pipe occupancy of the execution pipelines associated with the second queues.

13. The processor of claim 12, wherein the at least one indicator of imbalance comprises at least one of: a first indicator that at least one of the second queues has a current occupancy that exceeds a first threshold; and a second indicator that a difference between current occupancies of any two second queues exceeds a second threshold.

14. The processor of claim 13, wherein the queue controller is configured to select an allocation mode of the plurality of allocation modes by: selecting the queue preferential allocation mode in response to neither the first indicator nor the second indicator being detected; and selecting the pipeline preferential allocation mode in response to at least one of the first indicator and the second indicator being detected.

15. The processor of claim 13, wherein the queue controller is configured to select an allocation mode of the plurality of allocation modes by: selecting the queue preferential allocation mode in response to no more than one of the first indicator and the second indicator being detected; and selecting the pipeline preferential allocation mode in response to both the first indicator and the second indicator being detected.

16. The processor of any one of claims 12 to 15, wherein the plurality of allocation modes further comprises: a speculative allocation mode in which a copy of each instruction operation of a group of instruction operations is allocated to each of the plurality of second queues and subsequently invalidated in all but one second queue based on an analysis of a pipe occupancy of an execution pipeline capable of executing the respective instruction operation.

17. The processor of claim 16, the queue controller configured to select an allocation mode of the plurality of allocation modes by: selecting the speculative allocation mode in response to a current pick rate of selecting instruction operations from the plurality of second queues for execution by a corresponding execution pipeline falling below a threshold.

18. The processor of claim 16, wherein: the at least one indicator of imbalance comprises at least one of: a first indicator that at least one of the plurality of second queues has a current occupancy that exceeds a first threshold; and a second indicator that a difference between current occupancies of any two of the plurality of second queues exceeds a second threshold; and a third indicator that a current pick rate of selecting instruction operations from the second queues for allocation to a corresponding execution pipeline falls below a threshold; and the queue controller is configured to select an allocation mode of the plurality of allocation modes by: selecting the queue preferential allocation mode in response to no more than one of the first indicator or the second indicator being present; and selecting the pipeline preferential allocation mode in response to both the first indicator and the second indicator being present. selecting the speculative allocation mode in response to the presence of the third indicator.

19. The processor of claim 16, wherein: during the speculative allocation mode, each instruction operation of the group has a pre-assigned execution lane, and the second queue associated with the pre-assigned execution lane is to retain a valid copy of the instruction operation unless a difference between a lane occupancy of the pre-assigned execution lane and a lane occupancy of another execution lane of the same type associated with another second queue exceeds a threshold.

20. The processor of claim 11, wherein the plurality of allocation modes includes: a queue-first allocation mode in which, for each group of instruction operations of a plurality of groups, the instruction operations of the group are allocated among the plurality of second queues using an interleaving pattern; and wherein the queue controller is to select a second queue for receiving an initial instruction operation in the interleaving pattern for a given group according to one of: a fixed selection sequence or a pseudo-random manner.

21. A processor comprising: a plurality of execution lanes; and a multi-mode distributed scheduler coupled to the plurality of execution lanes and comprising: a first queue configured to buffer instruction operations from a front-end of an instruction pipeline of the processor; a plurality of second queues each configured to buffer instruction operations allocated from the first queue for a corresponding separate subset of execution lanes of the plurality of execution lanes; and a queue controller configured to: for each instruction operation of a subset of instruction operations in the first queue, configure at least a portion of the multi-mode distributed scheduler to: allocate a copy of the instruction operation from the first queue to each second queue speculatively; and based on a current lane occupancy of execution lanes capable of executing the instruction operation, invalidate the copy of the instruction operation in all second queues except one.

22. The processor of claim 21, wherein: the instruction operation is pre-assigned to a designated execution lane of the plurality of execution lanes; and the queue controller is further configured to: in response to a current lane occupancy of the designated execution lane being within a threshold of current lane occupancies of other execution lanes capable of executing the instruction operation, maintain the copy of the instruction operation in a second queue associated with the designated execution lane as a valid copy of the instruction operation; and in response to the current lane occupancy of the designated execution lane exceeding a current lane occupancy of an execution lane associated with a different second queue by at least the threshold, invalidate the copy of the instruction operation in the second queue associated with the designated execution lane and maintain the copy of the instruction operation in the different second queue.

23. A processor comprising: a plurality of execution lanes; and a multi-mode distributed scheduler coupled to the plurality of execution lanes and comprising: a first queue configured to buffer instruction operations from a front-end of an instruction pipeline of the processor; a plurality of second queues each configured to buffer instruction operations allocated from the first queue for a corresponding separate subset of execution lanes of the plurality of execution lanes; and a queue controller configured to: for each instruction operation of a subset of instruction operations in the first queue, configure at least a portion of the multi-mode distributed scheduler to: allocate a copy of the instruction operation from the first queue to each second queue speculatively; and based on a current lane occupancy of execution lanes capable of executing the instruction operation, invalidate the copy of the instruction operation in all second queues except one. A multi-mode distributed scheduler coupled to the plurality of execution pipelines and comprising: a first queue configured to buffer instruction operations from a front end of an instruction pipeline of the processor; a plurality of second queues, each second queue configured to buffer, for a corresponding separate subset of execution pipelines of the plurality of execution pipelines, instruction operations allocated from the first queue; and a queue controller configured to configure at least a portion of the multi-mode distributed scheduler to: allocate groups of instruction operations from the first queue among the plurality of second queues in an interleaved mode; and for each group, select a second queue to which an initial instruction operation of the group is to be allocated in the interleaved mode based on one of: a predefined sequence or a pseudorandom manner.

24. The processor of claim 23, wherein the queue controller is further configured to select the second queue to which the initial instruction operation of the group is to be allocated in the interleaved mode based on a predefined sequence, wherein the predefined sequence is an alternating sequence.

25. The processor of claim 23, further comprising: a pseudorandom number generator; and wherein the queue controller is further configured to pseudorandomly select the second queue to which the initial instruction operation of the group is to be allocated in the interleaved mode based on an output of the pseudorandom number generator.

Images