Processor with multiple OP cache pipelines

Abstract

The processor employs multiple op cache pipelines to simultaneously provide previously decoded operations to a dispatch stage of the instruction pipeline. In response to receiving a first branch prediction at the processor, the processor selects a first op cache pipeline from among the processor's multiple op cache pipelines based on the first branch prediction, and provides a first set of operations associated with the first branch prediction to the dispatch queue via the selected first op cache pipeline.

Description

[Background technology]

[0001] To improve efficiency and instruction throughput, modern processors typically support instruction processing parallelism, in which the processor simultaneously performs designated processing tasks for different instructions in a received instruction stream during a given processor cycle. For example, many processors employ multiple execution units that simultaneously perform operations for different instructions, thereby reducing the overall time required to execute different instructions. However, implementing parallelism before the processor's execution stage presents additional challenges. Some processors employ decode units that simultaneously decode multiple instructions, but circuit area and power considerations limit the size and efficiency of these decode units.

[0002] The present disclosure may be better understood, and its numerous features and advantages made apparent to those skilled in the art by referencing the accompanying drawings, in which: The use of the same reference numbers in different drawings indicates similar or identical items. [Brief explanation of the drawings]

[0003] [Figure 1] FIG. 1 is a block diagram of a processor employing multiple op cache pipelines that simultaneously serve operations from the cache based on branch prediction, according to some embodiments. [Figure 2] 2 is a block diagram illustrating an example of the processor of FIG. 1 providing different operations to a dispatch queue through different ones of multiple op cache pipelines, according to some embodiments. [Figure 3] 2 is a block diagram illustrating an example of the processor of FIG. 1 that selects one of multiple op cache pipelines to service operations based on the relative fullness of corresponding read ports, according to some embodiments. [Figure 4]FIG. 2 is a block diagram illustrating an example of the processor of FIG. 1 that selects one of multiple op cache pipelines to service an operation based on fullness associated with a read port being below a threshold, according to some embodiments. [Figure 5] 2 is a block diagram of the processor of FIG. 1 including a reordering control module for reordering operations received from multiple op cache pipelines, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0004] To improve instruction throughput, some processors employ an operation cache, or op-cache, that stores previously decoded operations from currently executing instructions. For example, in some embodiments, the op-cache stores previously decoded operations associated with a set of instructions called a prediction window, where each prediction window is associated with either 1) a branch target representing the first instruction of the prediction window, or 2) an instruction following a previous prediction window because the previous prediction window ended before reaching the branch instruction. Each prediction window is defined by a first instruction of the window, such as a branch target, and a defined last instruction of the window, such as another branch instruction, an instruction at a specified memory address (e.g., an instruction at the end of the instruction cache line associated with the first instruction, etc.). The op-cache includes entries that store decoded instructions for at least a subset of prediction windows. Thus, the op-cache provides a way to quickly generate decoded operations for prediction windows that are frequently executed by the processor within a relatively short period of time.

[0005] However, conventional op caches typically cannot support the parallel provision of operations for different prediction windows, thereby limiting processor efficiency. In particular, op cache entries do not always coincide with prediction windows. For example, in some cases, a prediction window spans multiple op cache entries, while in other cases, a single op cache entry contains decoded instructions for multiple prediction windows and multiple instruction cache lines. In some cases, op cache entries contain an integer number of instructions and therefore have instruction starts and ends as their boundaries, but for at least some types of prediction windows, instructions span instruction cache line boundaries and prediction window boundaries. Due to the differences in prediction window, instruction cache line, and op cache entry boundaries, conventional op caches include only a single op cache pipeline to provide decoded instructions.

[0006] 1-5 illustrate a technique for a processor employing multiple op cache pipelines by identifying prediction windows (e.g., branch prediction windows) corresponding to known or well-defined op cache entries. For example, in some embodiments, a processor includes a branch predictor that generates branch prediction windows, each of which includes a start address and an end address. For some prediction windows, called non-sequential exit windows, the end address is an address corresponding to a branch instruction. In other words, a non-sequential exit window ends at an address corresponding to the end of the branch instruction. For non-sequential exit windows, the start of the next branch prediction window corresponds to the branch target address of the branch instruction. Other prediction windows, called sequential exit windows, end at an address corresponding to a specified boundary, such as the end of the instruction cache line corresponding to the branch prediction window, and may not coincide with the end of an instruction. For sequential exit windows, the start of the next branch prediction window corresponds to a sequential address following the specified boundary.

[0007] In some embodiments, the control module directs the prediction window to flow down the op cache pipeline or fetch-decode pipeline. The control module stores state information describing op cache entries, such as start and end pointers, and uses the state information to generate the sequence of op cache entry accesses needed to retrieve and provide decoded instructions for the incoming prediction window from the op cache. In some embodiments, the control module allocates this ordered list of op cache accesses in parallel to multiple op cache pipelines, which then simultaneously provide the decoded instructions from the accesses to their op cache entries. By allocating the ordered list of accesses across an ordered set of op cache pipelines (e.g., one access per pipeline per clock), the control module generates an ordered, parallel output flow that is divided among different op cache pipelines, improving operation throughput and overall processing efficiency while maintaining the order of retrieved operations.

[0008] In some embodiments, if the op cache does not contain a portion of the prediction stream, the control module controls the fetch-decode pipeline to provide decoded instructions from the missing portion to the dispatch queue. This fetch-decode pipeline proceeds independently of op cache pipeline fetches. The processor includes an ordering mechanism that ensures that the dispatch queue provides the decoded instruction stream in program order. The control module continues processing the incoming set of prediction windows and building queues of work (accesses) for both the op cache pipeline and the fetch-decode pipeline, e.g., based on op cache residency and entry contents. The op cache pipeline and fetch-decode pipeline simultaneously provide decoded instructions from access allocations flowing from these work queues.

[0009] In some embodiments, the control module ensures that prediction windows following a sequentially-ended prediction window are always assigned to the same op cache pipeline, so that all such streams of sequentially-ended prediction windows flow down the same op cache pipeline. This allows an op cache entry accessed in one clock to provide the address of the last instruction byte it contains, thereby identifying the starting address of the next sequential op cache entry accessed in the next clock. In such embodiments, only prediction windows following a non-sequentially-ended window and beginning with a branch target or immediately following a branch predictor redirection are assigned to different op cache pipelines.

[0010] In some embodiments, the processor selects an op cache pipeline to process a given prediction window based at least in part on specified operation flow criteria that govern one or more aspects of instruction flow through the processor. By way of example, in some embodiments, the operation flow criteria require that each op cache access available and required to satisfy the stream of incoming prediction windows be evenly distributed across the available op cache pipelines. In other embodiments, each op cache pipeline of the processor may be assigned a different prediction window, with the different prediction windows comprising different numbers of operations provided to the dispatch queue. The operation flow criteria identify an aspect of the number of assigned operations that governs which of the op cache pipelines is selected to process a given prediction window in whole or in part. For example, in some embodiments, the operation flow criteria require the processor to select an op cache pipeline that is assigned fewer operations to process, thereby balancing the instruction load among the different op cache pipelines. In other embodiments, the operation flow criteria require that a minimum or maximum number of operations be provided to one of the op cache pipelines before the prediction window is provided to another of the op cache pipelines. In yet another embodiment, the operational flow criteria require that prediction windows associated with different threads be provided to different ones of the op cache pipelines.

[0011] In some cases, the simultaneous processing of different prediction windows results in an op cache pipeline or a fetch-decode pipeline that provides out-of-order operations relative to the overall program flow. Accordingly, in some embodiments, the processor includes a reordering control module for reordering decoded instruction bytes from different operation pipelines, thereby restoring the program flow before the operations are provided to subsequent units in the instruction pipeline.

[0012] Referring now to the figures, FIG. 1 illustrates a processor 100 implementing multiple op cache pipelines, according to some embodiments. Processor 100 is generally configured to execute a set of instructions (e.g., a computer program) to perform specified tasks on behalf of an electronic device. Accordingly, in different embodiments, processor 100 is part of any of a variety of electronic devices, such as a desktop computer, a laptop computer, a server, a smartphone, a tablet, a game console, etc. For purposes of explanation, it is assumed that processor 100 is a general-purpose processor, such as a central processing unit (CPU). However, in other embodiments, processor 100 is another type of processor, such as a processing unit specifically designed to perform specified types of operations, such as a graphics processing unit (GPU), a machine learning processor, etc.

[0013] 1, the set of instructions executed by processor 100 is represented as instruction stream 101. In particular, instruction stream 101 is a stream of instructions executed by processor 100 over a predetermined period of time, and the instructions may therefore be associated with one or more computer programs, program threads, etc. Instruction stream 101 is made up of variable-length instructions, i.e., the size of one instruction in instruction stream 101 (e.g., the size of the op-code and operands associated with the instruction) is different from the size of at least one other instruction in instruction stream 101.

[0014] To support execution of instructions in instruction stream 101, processor 100 includes a fetch-decode pipeline 112 configured to fetch instructions from instruction cache 110 and decode each fetched instruction into one or more operations. For example, in the embodiment shown in Figure 1, fetch-decode pipeline 112 includes a fetch unit 120 configured to fetch instructions from instruction cache 110 and a decode unit 123 that decodes the fetched instructions into one or more operations and stores the operations in dispatch queue 122. Processor 100 includes additional instruction pipeline units not shown in Figure 1, including a dispatch unit that dispatches the operations stored in dispatch queue 122 to multiple execution units that perform the operations.

[0015] To improve efficient execution of instructions, processor 100 employs an operation cache, referred to herein as op cache 125, configured to store previously decoded operations. In some embodiments, op cache 125 includes multiple storage entries, each entry associated with one or more instruction addresses, and each entry stores one or more decoded operations associated with the one or more instruction addresses.

[0016] As described further herein, processor 100 includes control module 106 that is generally configured to receive an instruction address that represents an instruction to be decoded. In response to receiving the instruction address, control module 106 determines whether an entry in op cache 125 is associated with the received instruction address, and if so, allocates access to an op cache pipeline (126 or 127), which then provides the operation stored in the corresponding entry to dispatch queue 122 for storage and subsequent provision to a dispatch stage. This allows processor 100 to provide operations for frequently executed instructions or instruction sequences to dispatch queue 122 relatively quickly, without having to repeatedly fetch and decode these frequently executed instructions or instruction sequences.

[0017] If op cache 125 does not contain an entry corresponding to the received instruction address, control module 106 retrieves a decoded operation for the received instruction address (e.g., from decode unit 123), and in some embodiments, assigns the received instruction address to an entry according to a specified op cache allocation and replacement policy, and stores the retrieved operation in the assigned entry. Op cache 125 thereby maintains stored operations for a set of instruction addresses based on a specified cache allocation and replacement policy, such as the N most recently received instruction addresses.

[0018] Processor 100 is generally configured to split instruction stream 101 into separate streams and, for each instruction in the stream, provide a decoded operation for dispatch to a set of execution units (not shown), as described further herein. Processor 100 specifies an instruction address associated with a first instruction in the fetch stream and also specifies an end of the fetch stream. In some cases, the end of the fetch stream corresponds to the end of a cache line in instruction cache 110 that contains the first instruction. In other embodiments, the end of the fetch stream corresponds to a defined instruction, such as a branch instruction, as described further herein.

[0019] To identify the fetch stream, processor 100 employs branch predictor 102, which monitors instruction stream 101 for branch patterns (i.e., patterns relating to which branch instructions in instruction stream 101 are taken or not taken, and under what conditions). Based on the patterns, as understood in the art, branch predictor 102 generates branch predictions that indicate the likely outcome of branch instructions in instruction stream 101. In some embodiments, branch predictor 102 provides one of three different prediction types: 1) a taken branch prediction, which represents a prediction that a branch instruction will result in the corresponding branch being taken; 2) a not-taken branch prediction, which represents a prediction that a branch instruction will result in the corresponding branch not being taken; and 3) no prediction, which represents an indication that the branch predictor has not identified a next branch instruction in instruction stream 101 and that the instruction flow of processor 100 should proceed sequentially.

[0020] For each branch prediction, the branch predictor 102 generates a corresponding prediction window 104, where each prediction window includes: 1) an indicator (e.g., a pointer) of the first instruction byte of the window; and 2) an indicator of the last instruction byte of the window. Thus, each prediction window 104 defines the sequence of instruction bytes between the first and last instructions of the window.

[0021] Control module 106 of processor 100 receives prediction window 104 from branch predictor 102 and controls fetch-decode pipeline 112 and op cache 125 to generate operations based on each instruction byte in prediction window 104. Illustratively, in response to receiving a prediction window, control module 106 determines whether the instruction address of the first instruction byte of the window corresponds to an entry in op cache 125. In response to a cache hit (i.e., in response to control module 106 determining that there is an entry matching the instruction address of the first instruction byte), control module 106 instructs op cache 125 to provide the operations for the received prediction window stored in the corresponding entry to dispatch queue 122. In some embodiments, to improve processing efficiency, control module 106 starts fetching and decoding of operations for the prediction window in fetch-decode pipeline 112 simultaneously with determining a cache hit or cache miss in op cache 125, and stops fetching and decoding in response to a cache hit in op cache 125.

[0022] To improve instruction throughput, op cache 125 includes multiple op cache pipelines (e.g., op cache pipelines 126, 127), each configured to independently provide a set of operations from one or more entries in op cache 125 to dispatch queue 122. In some embodiments, each op cache pipeline includes a read port or set of read ports to independently read instruction bytes from op cache 125 entries and provide the read instruction bytes to dispatch queue 122. The read ports are configured to operate simultaneously, so, for example, op cache pipelines 126 and 127 can each simultaneously provide a set of instruction bytes to dispatch queue 122.

[0023] In some embodiments, control module 106 is configured to, in response to a cache hit for the prediction window, retrieve at least a portion of the instruction bytes of the prediction window from the corresponding op cache entry and allocate one of op cache pipelines 126 and 127 to provide the retrieved instruction bytes to dispatch queue 122.

[0024] For example, as described above, prediction windows 104 are classified into two types: 1) sequentially terminated windows, whose final instruction byte corresponds to a defined address or location, such as the end of a cache line, and 2) non-sequentially terminated windows, whose final instruction byte corresponds to a branch instruction. Furthermore, each prediction window follows either a sequentially terminated window or a non-sequentially terminated window. Based on at least a subset of the instructions indicated by prediction window 104, control module 106 identifies a corresponding set of op cache 125 entries that store operations corresponding to the subset of instructions. Control module 106 generates a set of op cache accesses, designated op cache accesses 117, to access the op cache 125 entries. Control module 106 assigns individual accesses to one of op cache pipelines 126 and 127, which generate op streams 115 and 116, respectively. This allows the control module to assign different accesses to different ones of op cache pipelines 126 and 127.

[0025] By way of example, in some embodiments, control module 106 identifies, for a given prediction window 104, a sequence of four different entries in op cache 125 that store operations corresponding to instructions in the prediction window. In response, control module 106 generates op cache accesses 117 to include a workload of four accesses, one for each entry, designated in this example as access A, access B, access C, and access D. Control module 106 assigns each access to one of op cache pipelines 126 and 127. Thus, for example, control module 106 assigns access A and access C to op cache pipeline 126 and access B and access D to op cache pipeline 127. Each of op cache pipelines 126 and 127 performs its assigned access by retrieving the operation in the op cache entry indicated by the corresponding access. Furthermore, in some embodiments, op cache pipelines 126 and 127 perform at least some of their assigned accesses concurrently. For example, in some embodiments, op cache pipeline 126 performs access A at the same time as op cache pipeline 127 performs access B. Processor 100 thereby supports increased throughput in op cache 125 .

[0026] In some embodiments, control module 117 generates op cache accesses 117 to have a specified order that corresponds to the program order of instruction stream 101. Control module 106 assigns individual accesses to op cache pipelines 126 and 127 according to a specified scheme, such that operations retrieved from op cache 125 are stored in dispatch queue 125 in program order. For example, in some embodiments, control module 106 is configured to assign, for a given pair of concurrent accesses, the “older” access (the access corresponding to the earlier operation in program order) to op cache pipeline 126 and the newer access (the access corresponding to the later operation in program order) to op cache pipeline 127.

[0027] In some embodiments, processor 100 is a multi-threaded processor that executes multiple threads simultaneously. In some of these embodiments, control module 106 alternately assigns accesses for multiple threads to op cache pipelines 126 and 127. For example, during one cycle, control module 106 assigns accesses for a given thread to pipelines 126 and 127, with the accesses executing simultaneously in the different pipelines. During the next cycle, control module 106 assigns accesses for a different thread to pipelines 126 and 127, with the accesses again executing simultaneously in the different pipelines.

[0028] In other embodiments, control module 106 allocates access to pipelines 126 and 127 for different threads based on a set of flow constraints or criteria specified as opflow criteria 108. Opflow criteria 108 is a data structure stored in memory or other storage component (not shown) of processor 100 and, in some embodiments, is generated to ensure efficient processor operation, for example, during testing and development of processor 100. In other embodiments, opflow criteria 108 is generated by an operating system running on processor 100 and is adjustable or programmable by a user of an electronic device incorporating processor 100.

[0029] In some embodiments, control module 106 enforces instruction flow policies based in part on fixed policy constraints to prevent execution errors in processor 100 and in part on criteria specified by operational flow criteria 108. Control module 106 first ensures that the fixed policy constraints are satisfied. Once these constraints are satisfied, control module 106 employs operational flow criteria 108 to improve instruction throughput in processor 100.

[0030] In different embodiments, opflow criteria 108 may indicate factors such as downstream resource availability, thread fairness criteria, quality of service settings, etc., or any combination thereof. An operating system or other program sets opflow criteria 108 to govern the flow of accesses provided to pipelines 126 and 127. Thus, for example, in some embodiments, the operating system sets opflow criteria 108 such that control module 106 provides all accesses for a specified thread to a specified set of opcache pipelines. In other embodiments, the operating system sets opflow criteria 108 such that, for each of a specified set of time slices (e.g., each clock cycle), control module 106 allocates all accesses of a single thread per time slice to pipelines 126 and 127, with control module 106 selecting which thread to allocate in a particular time slice based on flow considerations or constraints indicated by the operating system. In other embodiments, control module 106 dynamically allocates accesses from different threads across different opcache pipelines based on flow considerations or constraints indicated by the operating system. This allows the operating system to implement any of a variety of thread flow control schemes.

[0031] In some embodiments, based on op flow criteria 108, control module 106 selects an op cache pipeline to process the branch stream based at least in part on the relative workload pending in each of the multiple op cache pipelines. By way of example, in some embodiments, each of op cache pipelines 126 and 127 includes one or more queues, such as an input queue, for storing operations provided through the corresponding read port. Based on the fullness of these one or more queues, control module 106 determines the relative workload pending in each of op cache pipelines 126 and 127. In response to receiving a prediction window from branch predictor 102, and assuming the fixed constraint is satisfied, control module 106 assigns the corresponding fetch stream to the op cache pipeline with the lower workload (i.e., the op cache pipeline has less pending work).

[0032] In some embodiments, op cache pipelines 126 and 127 are configured such that it is more efficient to allocate a threshold number of prediction windows to one pipeline before allocating prediction windows to the other. Thus, in these embodiments, based on operation flow criteria 108, control module 106 selects one of op cache pipelines 126 and 127 to process prediction windows (i.e., provide operations for the prediction windows) until the number of pending operations in the selected pipeline exceeds the threshold. Control module 106 then begins allocating prediction windows to the other of op cache pipelines 126 and 127.

[0033] In some embodiments, op cache pipelines 126 and 127 are configured such that processing efficiency decreases when a threshold maximum number of prediction windows are assigned to one pipeline. Thus, in these embodiments, based on operation flow criteria 108, control module 106 selects one of op cache pipelines 126 and 127 to process prediction windows until the threshold maximum number of pending instructions in the selected pipeline is reached. Control module 106 then begins assigning prediction windows to the other of op cache pipelines 126 and 127.

[0034] In some embodiments, the prediction windows processed by op cache pipelines 126 and 127 are associated with different program threads executing on processor 100. That is, the instructions for one prediction window are instructions of one thread, and the instructions for another prediction window are instructions of another thread. In these embodiments, control module 106 assigns prediction windows to op cache pipelines 126 and 127 based in part on the thread associated with each prediction window. For example, in some embodiments, each thread is associated with a quality of service (QoS) level set by an operating system executing on processor 100. Control module 106 selects an op cache pipeline to process a given prediction window based on the QoS level of the thread associated with the given prediction window and the QoS level of the thread associated with the prediction window being processed in op cache pipelines 126 and 127. For example, in some embodiments, control module 106 assigns prediction windows to op cache pipelines 126 and 127 such that the prediction windows of threads with higher QoS are not blocked by the prediction windows of threads with lower QoS levels, such as by assigning all prediction windows of threads with higher QoS to op cache pipeline 126 and all prediction windows of threads with lower QoS levels to op cache pipeline 127.

[0035] FIG. 2 is a block diagram illustrating an example of control module 106 allocating op streams 115 and 116 to op cache pipelines, according to some embodiments. In the illustrated example, it is assumed that op streams 115 and 116 each correspond to a different set of prediction windows. Furthermore, in the illustrated example, op cache 125 includes two different read ports, designated read port 226 (corresponding to op cache pipeline 126 in FIG. 1) and read port 227 (corresponding to op cache pipeline 127 in FIG. 1). In the example of FIG. 2, control module 106 selects read port 226 to provide operations for op stream 115. Accordingly, control module 106 instructs op cache 125 to provide stored operations for op stream 115 to dispatch queue 122 via read port 226. Similarly, control module 106 instructs op cache 125 to provide stored operations for op stream 116 via read port 227. In some embodiments, op cache 125 provides at least one operation of op stream 115 via read port 227 while simultaneously providing at least one operation of op stream 116 via read port 226 .

[0036] In some embodiments, op streams 115 and 116 are generated by control module 106 at the granularity of an op cache entry read, such that op streams 115 and 116 correspond to the same prediction window, and control module 106 assigns op cache entry reads to op streams 115 and 116 based on an ordered set of all op cache accesses. To illustrate by way of example, a given prediction window PW0 corresponds to three op cache entries and requires three op cache accesses designated PW0A0, PW0A1, and PW0A2. A subsequent prediction window PW1 corresponds to a single op cache entry and requires one op cache access designated PW1A0. Control module 106 assigns accesses PW0A0 and PW0A2 to op stream 115 and accesses PW0A1 and PW1A0 to op stream 116. Thus, accesses PW0A0 and PW0A1 are performed simultaneously (eg, during the same clock cycle), and accesses PW0A2 and PW1A0 are performed simultaneously (eg, during a subsequent clock cycle).

[0037] As mentioned above, in some embodiments, operation flow criteria 108 indicates that control module 106 assigns op streams to op cache pipelines 126 and 127 based on the relative workload in each pipeline. An example according to some embodiments is shown in FIG. 3. In the example of FIG. 3, control module 106 monitors the fullness of the input queues of read ports 226 and 227, respectively. In the illustrated example, the input queue for read port 226 is fuller than the input queue for read port 227. That is, read port 226 has more operations waiting to be provided to dispatch queue 222 than read port 227, indicating that op cache pipeline 126 has a greater overall workload than op cache pipeline 127. Thus, in response to receiving an indicator of op stream 327 corresponding to a prediction window from branch predictor 102, control module 106 selects read port 227 to provide the operations associated with op stream 327. Control module 106 thereby balances the overall workload of op cache 125 between op cache pipelines 126 and 127 .

[0038] In some embodiments, operation flow criteria 108 indicates that control module 106 allocates op stream 428 to op cache pipeline 226 until the workload in pipeline 226 is equal to or greater than a minimum threshold. An example according to some embodiments is shown in FIG. 4. In the example of FIG. 4, control module 106 monitors the fullness of the input queue of op cache pipeline 226 and compares the fullness to threshold 431. In the illustrated example, the input queue fullness has not reached threshold 431; that is, input queue 224 has fewer pending operations than the number of operations indicated by threshold 431. Thus, in response to receiving an indicator of op stream 428 from branch predictor 102, control module 106 selects op cache pipeline 226 to process op stream 428.

[0039] In some embodiments, control module 106 assigns op streams to op cache pipelines 126 and 127 to be processed out of order with respect to the overall program flow associated with instruction stream 101. In some cases, such out-of-order processing of instructions causes errors in other parts of processor 100, such as one or more execution units of processor 100. Therefore, in some embodiments, processor 100 includes elements for restoring instruction order after op cache pipelines 126 and 127. An example according to some embodiments is shown in FIG. 5. In the illustrated example, processor 100 includes a reordering control module 538 located between op cache pipelines 126 and 127 and dispatch queue 122.

[0040] While the above examples of op flow criteria are described with respect to a single factor, it will be appreciated that in some embodiments, control module 106 employs multiple factors to determine how to allocate op cache accesses to different op streams. Thus, for example, in some embodiments, control module 106 allocates op cache accesses to op cache streams based on a combination of two or more factors, such as quality of service information for different running threads, buffer fullness, a threshold number of accesses allocated to a given stream, etc.

[0041] During operation, op cache pipelines 126 and 127 retrieve operations for op streams 115 and 116, respectively. As described above, in some cases, pipelines 126 and 127 generate these operations out of order relative to the overall program flow of instruction stream 101. Reorder control module 538 is configured to reorder the operations generated by op cache pipelines 126 and 127 to restore the original order of instruction stream 101. For example, in some embodiments, reorder control module 538 reorders the operations such that the reordered sequence of operations corresponds to the sequence that would have occurred if the operations had been provided by a single op cache pipeline (referred to as a designated instruction sequence). In yet other embodiments, reorder module 538 receives decoded instructions from both op cache pipelines 126 and 127 and decode unit 123 and reorders the received decoded instructions according to the designated instruction sequence. Thus, in some embodiments, the designated instruction sequence includes both operations received from op cache 106 and operations received from decode unit 123. The reordering control module 538 provides the reordered operations to the dispatch queue 122. The dispatch unit takes the reordered operations in the reordered sequence and dispatches each operation to a corresponding execution unit (not shown) of the processor 100.

[0042] In different embodiments, reordering control module 538 includes different reordering structures. For example, in some embodiments, reordering control module includes a first-in first-out (FIFO) buffer that restores the ordering of operations received from op cache pipelines 126 and 127 and from decode unit 123. In another embodiment, reordering control module 538 includes a separate FIFO buffer for each of op cache pipelines 126 and 127 and a FIFO buffer for decode unit 123, and also includes a reordering sequence FIFO buffer that stores control information such as an instruction or block sequence number for each operation, a FIFO buffer that stores a block start operation (i.e., the FIFO buffer that stores the operation corresponding to the first instruction of the fetch window), a FIFO start entry address (i.e., the address of the FIFO buffer entry that stores the operation corresponding to the first instruction of the fetch window), a FIFO end entry address (i.e., the address of the FIFO buffer entry that stores the operation corresponding to the last instruction of the fetch window), the number of entries for the fetch block in the corresponding FIFO buffer, etc., or any combination thereof. Control circuitry in reordering control module 538 employs this control information to restore the order of the operations stored in the separate FIFO buffers.

[0043] In another embodiment, reordering control module 538 includes a FIFO buffer that stores the oldest reordered set of operations at the time of FIFO write and further includes another FIFO buffer that stores other, newer blocks of operations from any of multiple op cache pipelines. Reordering control module 538 further includes a reordering sequence FIFO buffer that stores control information such as one or more of an instruction or block sequence number, a FIFO that stores a fetch window start, a FIFO entry start address, a FIFO entry end address, and the number of entries for the fetch window in the corresponding FIFO buffer. Control circuitry in reordering control module 538 employs this control information to restore the order of the operations stored in the respective FIFO buffers.

[0044] In yet another embodiment, each prediction window is associated with an identifier, such as a sequence number. Reordering control module 538 includes a storage structure having multiple entries, where an entry addressed by a branch prediction identifier stores one or more of an indicator of the set of predictions starting with the corresponding branch prediction window, the size of the branch prediction window, and a pointer to the set of operations associated with the next branch prediction window in instruction stream 101. In yet another embodiment, reordering control module 538 employs a linked list of descriptors that identify the correct sequence of decoded operations. Reordering control module 538 employs this sequence information to restore received operations to the sequence indicated by the program flow of instruction stream 101.

[0045] As disclosed herein, in some embodiments, a method includes, in response to receiving a first branch prediction at a processor, selecting a first op cache pipeline from among multiple op cache pipelines of the processor, and providing a first set of operations associated with the first branch prediction to a dispatch stage of the processor via the selected first op cache pipeline. In one aspect, selecting the first op cache pipeline further includes selecting the first op cache pipeline based on operation flow criteria associated with the multiple op cache pipelines. In another aspect, the operation flow criteria includes a quality of service associated with at least one thread executing in the processor. In yet another aspect, the operation flow criteria indicates which threads of the multiple executing threads are assigned to the multiple op cache pipelines for each of a plurality of time slices.

[0046] In one aspect, the operation flow criteria includes a number of prediction windows to be provided to the first op cache pipeline prior to selecting. In another aspect, the operation flow criteria includes a minimum number of prediction windows expected to be provided to each of the multiple op cache pipelines. In yet another aspect, the operation flow criteria includes a maximum number of prediction windows expected to be provided to each of the multiple op cache pipelines. In yet another aspect, each of the multiple op cache pipelines includes a different read port of the processor's op cache.

[0047] In one aspect, the method includes determining a first set of operations based on a first operation and a last operation indicated by the first branch prediction, and reordering the first set of operations after the first set of operations is generated based on a program sequence identified in the processor. In another aspect, selecting the first op cache pipeline further includes selecting the first op cache pipeline based on the first branch prediction.

[0048] In some embodiments, a method includes, in response to identifying a branch prediction in the processor, selecting one of multiple op cache pipelines of the processor based on the branch prediction, each of the multiple op cache pipelines being associated with a different read port of an op cache of the processor, and providing a set of operations associated with the branch prediction to a dispatch stage of the processor via the selected op cache pipeline.

[0049] In some embodiments, a processor includes a branch predictor that generates a first branch prediction, a dispatch queue, a plurality of op cache pipelines, and a control module for selecting a first op cache pipeline from among the processor's plurality of op cache pipelines, the selected op cache pipeline providing operations associated with the first branch prediction to the dispatch queue. In one aspect, the control module selects the first op cache pipeline based on operation flow criteria associated with the plurality of op cache pipelines. In another aspect, the operation flow criteria includes a first amount of operations assigned to the first op cache pipeline. In yet another aspect, the operation flow criteria includes a second amount of operations assigned to a second op cache pipeline from the plurality of op cache pipelines. In yet another aspect, the operation flow criteria includes a number of prediction windows provided to the first op cache pipeline prior to the selecting.

[0050] In one aspect, the operation flow criteria includes a minimum number of prediction windows expected to be provided to each of the multiple op cache pipelines. In another aspect, the operation flow criteria includes a maximum number of prediction windows expected to be provided to each of the multiple op cache pipelines. In yet another aspect, each of the multiple op cache pipelines includes a different read port of the op cache. In yet another aspect, the processor further comprises a reordering module for reordering operations provided through the first op cache pipeline, the reordering being based on a program sequence identified in the processor.

[0051] In some embodiments, certain aspects of the techniques described above are implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied in a non-transitory computer-readable storage medium. The software may include instructions and specific data that, when executed by one or more processors, operate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer-readable storage medium may include, for example, a magnetic or optical disk storage device, a solid-state storage device such as flash memory, a cache, a random access memory (RAM), or other non-volatile memory device(s). The executable instructions stored on the non-transitory computer-readable storage medium may be implemented as source code, assembly language code, object code, or other form of instructions that can be interpreted or otherwise executed by one or more processors.

[0052] In addition to the above, it should be noted that not all activities or elements described in the summary description are required, that some of the particular activities or devices may not be required, that one or more additional activities may be performed, and that one or more additional elements may be included. Furthermore, the order in which the activities are listed is not necessarily the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, those skilled in the art will recognize that various modifications and variations can be made without departing from the scope of the invention as set forth in the claims. Accordingly, the specification and drawings should be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present invention.

[0053] Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and features from which any benefit, advantage, or solution may arise or be manifested are not construed as critical, essential, or essential features of any or all claims. Moreover, the specific embodiments described above are illustrative only, since the disclosed invention may be modified and practiced in different, but similar manners apparent to those skilled in the art having the benefit of the teachings herein. No limitations are intended to the details of construction or design herein shown, other than as described in the appended claims. It is therefore apparent that the specific embodiments described above may be altered or modified, and that all such variations are considered within the scope of the disclosed invention. Accordingly, the protection sought herein is set forth in the appended claims.

Claims

1. 1. A method comprising: In response to receiving a first branch prediction at the processor, selecting a first op cache pipeline from among a plurality of op cache pipelines of the processor; providing a first set of operations associated with the first branch prediction to a dispatch stage of the processor via a selected first op cache pipeline; method.

2. selecting the first op cache pipeline includes selecting the first op cache pipeline based on operation flow criteria associated with the plurality of op cache pipelines.

10. The method of claim 1.

3. the operational flow criteria include a quality of service associated with at least one thread executing on the processor; The method of claim 2.

4. the operational flow criteria indicating, for each of a plurality of time slices, which threads of a plurality of running threads are assigned to the plurality of op cache pipelines; The method of claim 2.

5. the operational flow criteria includes a number of prediction windows provided to the first op cache pipeline prior to the selection; The method of claim 2.

6. the operational flow criteria include a minimum number of prediction windows expected to be provided to each of the plurality of op cache pipelines; The method of claim 5.

7. the operational flow criteria include a maximum number of prediction windows expected to be provided to each of the plurality of op cache pipelines; The method of claim 5.

8. 1. A processor, comprising: a branch predictor that generates a first branch prediction; A dispatch queue; a plurality of op cache pipelines of op caches; a control module for selecting a first op cache pipeline from the plurality of op cache pipelines of the processor; the selected op cache pipeline provides the operation associated with the first branch prediction to the dispatch queue. Processor.

9. the control module selecting the first op cache pipeline based on operation flow criteria associated with the plurality of op cache pipelines. The processor of claim 8.

10. the operation flow criteria includes a first amount of operations to be allocated to the first op cache pipeline; The processor of claim 9.

11. the operation flow criteria includes a second quantity of operations to be assigned to a second op cache pipeline of the plurality of op cache pipelines; The processor of claim 9.

12. the operational flow criteria includes a number of prediction windows provided to the first op cache pipeline prior to the selection; The processor of claim 11.

13. the operational flow criteria include a minimum number of prediction windows expected to be provided to each of the plurality of op cache pipelines; The processor of claim 12.

14. the operational flow criteria include a maximum number of prediction windows expected to be provided to each of the plurality of op cache pipelines; The processor of claim 12.

15. each of the plurality of op cache pipelines includes a different read port of the op cache; The processor of claim 14.

Images