Branch confidence throttling

Abstract

A processing system includes a processor [100] having a branch predictor [108], the branch predictor including one or more branch target buffer tables [110]. The processor also includes a branch prediction pipeline [122], the branch prediction pipeline including a throttling unit [124] and an uncertainty accumulator [130]. The processor assigns an uncertainty value [132] to each of a plurality of branch predictions generated by the branch predictor, and adds the uncertainty value for each of the plurality of branch predictions to an accumulated uncertainty counter [134] associated with the uncertainty accumulator. The throttling unit of the branch prediction pipeline throttles the operation of the branch prediction pipeline based on the accumulated uncertainty counter.

Description

Background Art

[0001] The processing unit uses branch prediction to predict the outcome of a branch instruction so that the processing unit can begin speculatively executing instructions along the predicted branch before the processing unit evaluates the branch instruction. To predict the outcome of a branch instruction, the processing unit uses information in a branch prediction structure, such as an indirect branch predictor that redirects the flow of a program to any instruction, a return address stack that includes return addresses for subroutines executed on the processing unit, a conditional branch predictor that predicts the direction of conditional branches (taken or not taken), and a branch target buffer that includes information about the location, type, and target address of the predicted branch instruction.

[0002] Some implementations of a branch prediction structure use a branch history of the results of branch instructions executed by processes previously or currently executing on the processing unit. For example, if a branch instruction previously directed program flow to a first address 90% of the time and to a second address 10% of the time, the corresponding entry in the branch prediction structure predicts that the branch instruction will direct program flow to the first address, thereby allowing the process to speculatively execute instructions along the branch starting at the first address without waiting for the branch instruction to be evaluated. If the predicted branch proves to be incorrect when the branch instruction is evaluated, speculative execution along the incorrectly predicted branch is suspended, and the state of the processing unit is rolled back to the state at the branch instruction to begin execution along the correct branch. More specifically, both the branch prediction unit and the fetch unit roll back to process from the correct target of the branch or an address after the branch if the branch is not taken. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] By referencing the accompanying drawings, the present disclosure may be better understood, and its numerous features and advantages will be apparent to those skilled in the art. The use of the same reference numerals in different drawings indicates similar or identical items.

[0004] Figure 1 is a block diagram of a processor according to some embodiments.

[0005] Figure 2 is a diagram of predictive pipeline throttling in single-threaded mode, according to some embodiments.

[0006] Figure 3 is a diagram of predictive pipeline throttling in multithreaded mode, according to some embodiments.

[0007] Figure 4 is a flow chart of a method for throttling a branch prediction pipeline, according to some embodiments. DETAILED DESCRIPTION

[0008] Speculative execution causes wasted processor work when the branch predictor incorrectly predicts the sequence of instructions to be fetched. This wasted work wastes power by processing instructions that follow the predicted branch. Incorrect branch predictions also waste processing bandwidth when cores operate in a multithreaded mode where pipelines and resources are shared.

[0009] In various branch prediction pipelines, a branch predictor at the front end of the prediction pipeline predicts the address from which instructions should be fetched every cycle. For many cycles, it is not known whether the prediction made is correct. As long as the prediction is correct, keeping the prediction pipeline full is computationally efficient. However, when a prediction is made incorrectly, the prediction pipeline needs to be flushed, which results in a performance loss associated with the flush. Another loss of making an incorrect prediction is that not only are resources wasted to execute instructions along the wrong path, but other threads running on the same system are also harmed because the incorrectly predicted path consumes resources that could have been used by other threads. For example, in single-threaded mode, current solutions do not address the wasted work caused by redirection. In multi-threaded mode, competitive sharing of computing resources is based on the fairness criterion of assuming that each thread executes instructions on the correct path without considering potential mispredictions from different threads.

[0010] To reduce wasted work in order to save power and improve multi-threaded performance, Figures 1 to 4 Systems and methods for performing branch confidence throttling are described. In one embodiment, a processor includes: a branch predictor having one or more branch target buffer (BTB) tables; and a branch prediction pipeline including a throttling unit and an uncertainty accumulator. The processor assigns an uncertainty value to each of a plurality of branch predictions generated by the branch predictor. The uncertainty accumulator adds the uncertainty value for each of the plurality of branch predictions to an accumulated uncertainty counter associated with the uncertainty accumulator. The throttling unit throttles operation of the branch prediction pipeline based on the accumulated uncertainty counter, thereby slowing down threads that are unlikely to be on the correct path and, in doing so, saving power by not executing instructions that are likely to be flushed.

[0011] Figure 11 is a block diagram of a processor 100 that implements branch confidence throttling according to some embodiments. The processor 100 includes a memory 102, an instruction cache 104, an instruction fetch unit 106, a branch predictor 108, one or more branch target buffer (BTB) tables 110, and a processing pipeline 112. In various embodiments, the processing system is included in a computer processor or otherwise distributed within the computer system. The memory 102 includes any type of volatile or non-volatile memory for storing instructions and data, such as a cache memory. The instruction cache 104 accesses instructions from the memory 102 and stores instructions to be fetched. In various embodiments, the memory 102 and the instruction cache 104 include multiple cache levels. In addition, the processor 100 also includes a data cache (not shown).

[0012] exist Figure 1 , a simplified example of an instruction fetch unit 106 and a processing pipeline 112 is depicted for ease of illustration. The branch predictor 108 of the instruction fetch unit 106 generates branch target addresses that are stored or provided to one or more BTB tables (also commonly referred to as BT buffers and BTBs) 110. The branch predictor 108 is at least partially responsible for the execution of instructions in a processor (e.g., Figure 1 100) is implemented in the instruction fetch unit 106 of the processor 100. Figure 1 108, but the BTB table 110 may or may not be located near the instruction fetch unit 106 or certain elements of the branch predictor 108 in the processor 100. In some embodiments, the processor 100 also includes multiple instruction fetch units 106 and a processing pipeline 112. The processing pipeline 112 includes a decode unit 114, an issue unit 116, an execute stage 118, and a write-back unit 120.

[0013] In various embodiments, the entire instruction fetch unit 106 and / or the branch predictor 108 are also part of the processing pipeline 112. The processing pipeline 112 may also include other features such as error checking and handling logic, one or more parallel paths through the processing pipeline 112, and other features known in the art. Figure 1 A forward path through processing system 112 is depicted in FIG, but other feedback and signaling paths may be included between elements of processor 100.

[0014] The instruction fetch unit 106 fetches instructions from the instruction cache 104 for further processing by the decode unit 114. In one embodiment, the instruction fetch unit 106 includes a branch predictor 108 and also includes other branch prediction logic (not shown). Alternatively, in other embodiments, the branch predictor 108 is located separately from the instruction fetch unit 106. The branch predictor 108 is an example of processing circuitry for implementing branch confidence throttling, as discussed in more detail below.

[0015] Decode unit 114 decodes instructions and passes the decoded instructions, portions of instructions, or other decoded data to issue unit 116. Issue unit 116 analyzes the instructions or other data and, based on the analysis, passes the decoded instructions, portions of instructions, or other data to one or more execution units 118. One or more execution units 118 include execution units for executing instructions, such as a fixed-point execution unit, a floating-point execution unit, a load / store execution unit, a vector execution unit, and the like. Write-back unit 120 writes the results of instruction execution back to a destination resource (not shown). The destination resource can be any type of resource, including registers, cache memory, other memory, I / O circuitry for communicating with other devices, other processing circuitry, or any other type of destination for executed instructions or data.

[0016] The branch predictor 108 includes a branch prediction pipeline 122 that performs predictions by comparing the instruction address with the addresses of previously executed instructions that are already stored in the BTB table 110. In this type of architecture, many instructions can be "in progress"; that is, in various stages of execution at the same time. Figure 2 and Figure 3 The operation of the branch prediction pipeline 122 is presented in more detail. Each stage within the branch prediction pipeline 122 typically occurs sequentially. To achieve high performance, a new instruction enters the branch prediction pipeline 122 every cycle, and each instruction in the pipeline moves to a new stage. Each stage takes input and produces output, which is stored in an output buffer associated with that stage. The output buffer of one stage is typically the input buffer of the next stage. This arrangement allows all stages to operate in parallel and therefore produces greater throughput than if each instruction had to pass through the entire branch prediction pipeline 122 before the next instruction could enter the pipeline. When the branch prediction pipeline 122 is delayed or must be cleared, a delay is generated in the processing of each instruction in the pipeline.

[0017] Prediction generally saves processing time because a successful prediction allows the processor 100 to skip executing the step for obtaining the target address. The processor 100 saves time by looking up the address for the next execution step in the BTB table 110. For example, in some embodiments, the BTB 110 stores the branch address, the target address, and historical data to predict the branch direction (e.g., taken, not taken). The branch address information is stored to identify which BTB entry is used for prediction (e.g., the entry with a branch address equal to or closest to the search address). The target address is used to initiate an instruction fetch for the target of the predicted taken branch. The historical data (taken / not taken) is used to predict the direction of the branch as taken or not taken based on the previous outcome of the branch. In other embodiments, the direction prediction information may come from an additional (e.g., hierarchical) structure accessed in parallel with the BTB 110.

[0018] The branch prediction pipeline 122 includes a throttling unit 124, a plurality of prediction pipeline stages BP0 to BP3 (e.g., BP0 136, BP1 138, BP2 140, BP3 142), a BTB lookup 126, a conditional predictor 128, and an uncertainty accumulator 130. In terms of the pipeline, such as in Figure 1 In the branch prediction pipeline 122, one or more of the BTB tables 110 are read for each prediction flow, unless the read is suppressed for power conservation. According to some embodiments, the first BTB read is initiated in the first stage, designated as BP0. As stages in the branch prediction pipeline 122, subsequent stages are designated as BP1, BP2, and so on. It will be appreciated that in Figure 1 In the drawings and other figures, the pipelines shown are simplified to provide a clear illustration and to avoid unnecessarily obscuring the embodiments of the invention presented in the drawings. For example, the pipeline stages shown may represent multiple actual pipeline stages, or where two or more of the shown pipeline stages can be executed simultaneously, they may be combined into the same pipeline stage in certain implementations or embodiments. It will also be understood that although the functionality of a particular pipeline stage may be explained with reference to a particular portion of a particular thread, the pipeline is actually used to perform such functionality simultaneously on many such portions of one or more threads.

[0019] In one embodiment, a throttling mechanism (e.g., throttling unit 124) is implemented at the front end of the branch prediction pipeline 122 (i.e., the throttling unit 124 is located at the beginning of the branch prediction pipeline 122). In various embodiments, the throttling unit 124 initiates the prediction process based on resource availability and arbitrates between multiple independent processes or execution threads running on the processor 100. As shown, the BTB lookup 126 and conditional predictor 128 stages span multiple cycles of the branch prediction pipeline 122 (i.e., spanning prediction pipeline stages BP0 through BP3).

[0020] The conditional predictor 128 assigns a confidence level to each branch prediction generated by the branch predictor 108, the confidence level being related to the probability that the current prediction made earlier in the speculation flow will not be refreshed. Although described herein in the context of the conditional predictor 128 assigning a confidence level to each of a plurality of branch predictions generated by the branch predictor, those skilled in the art will recognize that various other predictors may be used without departing from the scope of this disclosure. For example, in other embodiments, an indirect predictor (not shown) or a return address predictor (not shown) provides an uncertainty value.

[0021] In one embodiment, the conditional predictor 128 performs prediction confidence grading and classifies each branch prediction as a high confidence prediction, a medium confidence prediction, or a low confidence prediction. For example, for a tag geometry length (TAGE) predictor, a high confidence prediction includes predictions made from a TAGE table using a strong bimodal or saturated 3-bit counter and typically has a misprediction rate of less than 1%. A medium confidence prediction includes predictions made from a TAGE table using a near-saturated 3-bit counter and typically has a misprediction rate in the range of 8% to 12%. A low confidence prediction includes predictions made from a TAGE table using a weak bimodal or weak / near-weak 3-bit counter and typically has a misprediction rate greater than 30%. In addition, as described in further detail below, for each branch prediction generated by the branch predictor 108, the processor 100 assigns an uncertainty value to each prediction.

[0022] In other embodiments, various confidence grading systems may be used, including deriving confidence levels from specific information about each prediction (such as the branch type, the age of the BTB entry used to predict the branch, the "strength" of the prediction from the conditions involved in the prediction or the indirect predictor), or special conditions (such as predicting a return with an underflow return address stack), as understood by those skilled in the art. In various embodiments, the processor 100 assigns an uncertainty value 132 to each branch prediction based at least in part on the above confidence levels assigned by the conditional predictor 128 and the number of dynamic branches predicted. For example, in one embodiment, each branch prediction is assigned an uncertainty value in the range of 0 to 10 by the processor 100 based on the prediction confidence grading. For example, a high confidence prediction is assigned an uncertainty value of 0, a medium confidence prediction is assigned an uncertainty value in the range of 1 to 4, and a low confidence prediction is assigned an uncertainty value in the range of 5 to 10.

[0023] The uncertainty accumulator 130 is positioned at the back end of the branch prediction pipeline 122 (i.e., the uncertainty accumulator 130 is positioned at the end of the branch prediction pipeline 122). The uncertainty accumulator 130 adds the uncertainty value 132 assigned to each prediction to an accumulated uncertainty counter 134 associated with the uncertainty accumulator 130. That is, the uncertainty accumulator 130 is incremented at the back end of the branch prediction pipeline 122. Additionally, the uncertainty value assigned to each prediction is subtracted from the accumulated uncertainty counter 134 when the corresponding branch prediction is retired or flushed from the branch prediction pipeline 122. That is, the accumulated uncertainty counter 134 is decremented when a branch prediction is no longer in progress in the processor 100. Thus, the various stages of the branch prediction pipeline 122 include assigning an uncertainty value to each prediction and accumulating the uncertainty values for all predictions in progress by adding the uncertainty value of the new prediction and subtracting the uncertainty value of the retired or flushed prediction at the uncertainty accumulator 130. In this manner, the uncertainty accumulator 130 represents the contents of the prediction in progress, i.e., instructions that have been predicted but not yet retired.

[0024] Based on whether the accumulated uncertainty counter 134 exceeds one or more uncertainty thresholds, the throttling unit 124 throttles the operation of the branch prediction pipeline 122 and makes performance / power tradeoffs based on the confidence of the branch predictor 108 in being on the correct path. That is, the accumulated uncertainty of the ongoing branch prediction is compared to a set of multiple thresholds, and increasingly stringent throttling is implemented as the accumulated uncertainty exceeds different thresholds.

[0025] In one embodiment, the throttling unit 124 reduces the prediction rate for the first execution thread based on the accumulated uncertainty counter 134 exceeding one or more uncertainty thresholds. For example, when the processor 100 is in single-threaded mode and the accumulated uncertainty counter 134 exceeds one or more of the uncertainty thresholds, the throttling unit 124 instructs the branch predictor 108 to skip predictions in one or more future cycles. In another embodiment, when the processor 100 is in multi-threaded mode and the accumulated uncertainty counter 134 exceeds one or more of the uncertainty thresholds, the throttling unit 124 instructs the branch predictor 108 to allocate processing bandwidth in one or more future cycles by assigning cycles that would otherwise be used to generate branch predictions from the first execution thread to the second execution thread.

[0026] Throttling causes an execution thread to not issue any predictions in cycles in which it would otherwise meet all criteria for issuing predictions, thereby causing the execution thread to yield to another execution thread, or to create a bubble cycle if the other thread cannot take advantage of yielding (e.g., when the processor 100 is operating in single-threaded mode). For example, in single-threaded mode, throttling achieves power savings, while in multi-threaded mode, throttling achieves power savings and performance improvements by yielding to other execution threads. By assigning an uncertainty value to each prediction and selecting a threshold and associated prediction rate, throttling reduces wasted work while minimizing the performance impact on the throttled execution thread. Saving on wasted work translates into power savings and / or multi-threaded performance benefits.

[0027] Figure 2 is a diagram of prediction pipeline throttling in single-threaded mode according to some embodiments. Graph 200 illustrates the value of an accumulated uncertainty counter associated with a first execution thread (i.e., thread A in this example) as a function of time (measured in cycles) during a branch prediction operation. Graph 202 illustrates the value of an accumulated uncertainty counter associated with a first execution thread (i.e., thread A in this example) as a function of time (measured in cycles) during a branch prediction operation. Figure 1 The horizontal axis of graphs 200 and 202 indicates increasing time (in cycles) from left to right. The vertical axis of graph 200 indicates the value of the accumulated uncertainty counter (in arbitrary units).

[0028] As shown in graph 200, the new prediction 204 in cycle 1 and the new prediction 206 in cycle 2 each cause the accumulated uncertainty counter to increment. However, the value of the accumulated uncertainty counter is less than the first uncertainty threshold T1. Therefore, the operation of the branch prediction pipeline continues as usual, with a new prediction entering the branch prediction pipeline 122 each cycle and each instruction already in the pipeline moving to a new stage.

[0029] The uncertainty value associated with the new prediction 208 that enters the branch prediction pipeline 122 in cycle 3 is added to the accumulated uncertainty counter. After adding the uncertainty value associated with the new prediction 208, the value of the accumulated uncertainty counter increases to above the first uncertainty threshold T1. Therefore, the throttling unit 124 of the branch prediction pipeline 122 begins throttling the branch prediction pipeline operation at the first throttling level in the next execution cycle (i.e., cycle 4). Figure 2 The example of assumes that each prediction is made in one cycle, so that the uncertainty accumulator 130 represents the accumulated uncertainty of the front of the pipeline in which the throttling unit 124 operates.

[0030] In cycle 4, the throttling unit 124 inserts a stall in the branch prediction pipeline 122 by not issuing a new prediction. A new prediction 210 enters the branch prediction pipeline 122 in cycle 5. Thus, the throttling unit 124 throttles the operation of the branch prediction pipeline by generating a new prediction every other cycle instead of every cycle when the accumulated uncertainty counter increases above the first uncertainty threshold T1. As shown in graph 200, the new prediction 210 in cycle 5 causes the accumulated uncertainty counter to increase (e.g., because the new prediction is another medium confidence or low confidence prediction). However, the value of the accumulated uncertainty counter is less than the second uncertainty threshold T2 but greater than the first uncertainty threshold T1. Thus, the operation of the branch prediction pipeline proceeds at a first throttling level, wherein a new prediction enters the branch prediction pipeline 122 every other cycle and each instruction already in the pipeline moves to a new stage every cycle.

[0031] In cycle 6, the throttling unit 124 inserts a stall in the branch prediction pipeline 122 by not issuing a new prediction. The new prediction 212 enters the branch prediction pipeline 122 in cycle 7. The uncertainty value associated with the new prediction 212 entering the branch prediction pipeline 122 in cycle 7 is added to the accumulated uncertainty counter. However, after adding the uncertainty value associated with the new prediction 212, the value of the accumulated uncertainty counter increases to above the second uncertainty threshold T2. Therefore, the throttling unit 124 of the branch prediction pipeline 122 begins throttling the branch prediction pipeline operation at the second throttling level in the next execution cycle (i.e., cycle 8).

[0032] In cycles 8 and 9, the throttle unit 124 inserts a stall in the branch prediction pipeline 122 by not issuing a new prediction during cycle 8 or cycle 9. The new prediction 214 does not enter the branch prediction pipeline 122 until cycle 10. Thus, the throttling unit 124 throttles the branch prediction pipeline operation by generating a new prediction every three cycles when the accumulated uncertainty counter increases above the second uncertainty threshold T2. Those skilled in the art will recognize that while throttling is described herein in the context of issuing a new prediction every two cycles (e.g., at a first throttle level) and every three cycles (e.g., at a second throttle level), any approach that reduces the rate at which new predictions are introduced into the branch prediction pipeline 122 and reduces the amount of power and computational resources dedicated to a given execution thread may be used without departing from the scope of the present disclosure.

[0033] For example, Figure 3 is a diagram of predictive pipeline throttling in multithreaded mode according to some embodiments. Figure 3As shown, instead of skipping predictions, processing bandwidth is allocated to other execution threads that can more efficiently utilize execution cycles (eg, other execution threads that have a higher confidence level that the prediction is on the correct path).

[0034] Graph 300 shows the value of an accumulated uncertainty counter associated with a first thread of execution (i.e., thread A in this example) as a function of time (measured in cycles) during a branch prediction operation. Graph 302 shows the value of an accumulated uncertainty counter associated with a first thread of execution (i.e., thread A in this example) as a function of time (measured in cycles) during a branch prediction operation. Figure 1 The horizontal axis of graphs 300 and 302 indicates increasing time (in cycles) from left to right. The vertical axis of graph 300 indicates the value of the accumulated uncertainty counter (in arbitrary units).

[0035] As shown in graph 300, the new prediction 304 in cycle 1 and the new prediction 306 in cycle 2 each cause the accumulated uncertainty counter to increment. However, the value of the accumulated uncertainty counter is less than the first uncertainty threshold T1. Therefore, the operation of the branch prediction pipeline proceeds as usual. In particular, for each cycle of the branch prediction pipeline 122, a new prediction for the first thread (e.g., thread A) and the second thread (e.g., thread B) is obtained, alternating between the following: ABABAB, and each instruction already in the pipeline is moved to a new stage.

[0036] The uncertainty value associated with the new prediction 308 that enters the branch prediction pipeline 122 in cycle 3 is added to the accumulated uncertainty counter. After adding the uncertainty value associated with the new prediction 308, the value of the accumulated uncertainty counter increases to above the first uncertainty threshold T1. Therefore, the throttling unit 124 of the branch prediction pipeline 122 begins throttling the branch prediction pipeline operation for executing thread A at the first throttling level in the next execution cycle (i.e., cycle 4). For example, instead of alternating between thread A and thread B, the throttling unit 124 selects thread B for processing more frequently: for example, thread B may be selected in two-thirds of the branch prediction pipeline cycles, as expressed as follows: ABBABB.

[0037] In cycles 4 and 5, the throttling unit 124 throttles the prediction associated with executing thread A by not issuing a new prediction. Instead, the throttling unit 124 instructs the branch predictor 108 to assign cycles for generating new branch predictions for thread B, and generates new predictions 310 and 312 for thread B. New prediction 314 enters the branch prediction pipeline 122 for thread A in cycle 6. Thus, the throttling unit 124 throttles the branch prediction pipeline operation by generating a new prediction every three cycles instead of every other cycle when the accumulated uncertainty counter for thread A increases above the first uncertainty threshold T1. As shown in the graph 300, the new prediction 314 in cycle 6 causes the accumulated uncertainty counter to increase (e.g., because the new prediction is another medium confidence or low confidence prediction). However, the value of the accumulated uncertainty counter is greater than the second uncertainty threshold T2 but less than the first uncertainty threshold T1. Thus, the branch prediction pipeline operates at a first throttling level, where a new prediction for thread A enters the branch prediction pipeline 122 every three cycles, and each instruction already in the pipeline moves to a new stage every cycle.

[0038] 8 , 316 , 318 , 320 , 321 , 322 , 323 , 324 , 325 , 326 , 327 , 328 , 329 , 330 , 331 , 332 , 333 , 334 , 336 , 337 , 338 , 339 , 340 , 350 , 360 , 371 , 372 , 373 , 374 , 375 , 376 , 377 , 378 , 379 , 400 , 410 , 420 , 430 , 440 , 450 , 460 , 479 , 480 , 490 , 510 , 520 , 530 , 540 , 550 , 560 , 579 , 480 , 490 , 580 , 590 , 610 , 620 , 630 , 640 , 650 , 660 , 679 , 480 , 490 , 580 , 640 , 480 , 490 , 590

[0039] In cycles 10, 11, and 12, the throttling unit 124 throttles the prediction associated with execution thread A by not issuing a new prediction. Instead, the throttling unit 124 instructs the branch predictor 108 to allocate processing bandwidth to execution thread B and generates new predictions 322, 324, and 326 for thread B in cycles 10, 11, and 12, respectively. The new prediction for thread A does not enter the branch prediction pipeline 122 until cycle 13 (not shown). Thus, the throttling unit 124 throttles the branch prediction pipeline operation for thread A by generating a new prediction for thread A every four cycles when the accumulated uncertainty counter increases above the second uncertainty threshold T2. Those skilled in the art will recognize that while throttling is described herein in the context of issuing a new prediction every three cycles (e.g., at a first throttling level) and every four cycles (e.g., at a second throttling level), any approach that reduces the rate at which new predictions are introduced into the branch prediction pipeline 122 and reduces the amount of power and computing resources dedicated to a given execution thread may be used without departing from the scope of the present disclosure. In other embodiments, throttling may include varying the number of periods between issuing new forecasts. However, the amount of throttling generally increases as the accumulated uncertainty increases.

[0040] Figure 4 is a block diagram illustrating a method 400 for throttling a branch prediction pipeline according to some embodiments. Figure 1 The branch prediction pipeline 122 is shown as implemented in some embodiments of the processor 100 .

[0041] At block 402, the branch predictor 108 generates a branch prediction for an executing thread. At block 404, the conditional predictor 128 assigns a confidence level to the branch prediction generated by the branch predictor 108. In some embodiments, the conditional predictor 128 assigns a confidence level to each branch prediction generated by the branch predictor 108 that is related to the probability that the current prediction made earlier in the speculation flow will not be refreshed. In one embodiment, the conditional predictor 128 performs prediction confidence grading and classifies each branch prediction as a high confidence prediction, a medium confidence prediction, or a low confidence prediction.

[0042] At block 406, the processor 100 assigns uncertainty values to the branch predictions generated by the branch predictor 108. In some embodiments, the processor 100 assigns an uncertainty value to each branch prediction based at least in part on the confidence level assigned by the condition predictor 128 at block 404 and the number of dynamic branches predicted.

[0043] At block 408, the uncertainty accumulator 130 adds the uncertainty value of the branch prediction from block 406 to an accumulated uncertainty counter associated with the branch predictor 108. In some embodiments, each different execution thread executing at the processor 100 is associated with a separate accumulated uncertainty counter. The uncertainty accumulator 130 is incremented at the back end of the branch prediction pipeline 122. Additionally, the uncertainty value assigned to the branch prediction is subtracted from the accumulated uncertainty counter when the corresponding branch prediction retires or is flushed from the branch prediction pipeline 122. That is, the accumulated uncertainty counter is decremented when the branch prediction is no longer in progress in the processor 100. In various embodiments, for all prediction blocks in progress, uncertainty values are accumulated in thread-specific counters as follows: the uncertainty value is added at prediction time; the uncertainty value is subtracted at retirement time; and the uncertainty counter is reset upon redirection.

[0044] Thus, the various stages of the branch prediction pipeline 122 include assigning an uncertainty value to each prediction and accumulating the uncertainty value across all predictions in progress by adding the uncertainty value of the new prediction and subtracting the uncertainty of the retired or flushed prediction at the uncertainty accumulator stage 130. The accumulated uncertainty counter provides a measure of confidence that the processor 100 is making the prediction along the correct path.

[0045] At block 410, a new prediction period begins, and the throttling unit 124 determines whether the accumulated uncertainty counter exceeds the first uncertainty threshold. If the throttling unit 124 determines that the accumulated uncertainty counter does not exceed the first uncertainty threshold, the method 400 returns to block 402 for the new prediction period. However, if the throttling unit 124 determines that the accumulated uncertainty counter does exceed the first uncertainty threshold, the method 400 proceeds to block 412.

[0046] At block 412, the throttling unit 124 determines whether the processor 100 is operating in single-threaded mode or multi-threaded mode in order to determine how to throttle the operation of the branch prediction pipeline 122. If the throttling unit 124 determines that the processor 100 is operating in single-threaded mode, the method 400 proceeds to block 414. At block 414, the throttling unit 124 determines whether the accumulated uncertainty counter exceeds a second uncertainty threshold. If the throttling unit 124 determines that the accumulated uncertainty counter exceeds the first uncertainty threshold (from block 410) but does not exceed the second uncertainty threshold, the throttling unit 124 skips branch prediction for a first number of cycles, such as previously specified with respect to the branch prediction pipeline 122, at block 416. Figure 2If the throttling unit 124 determines that the accumulated uncertainty counter exceeds the first uncertainty threshold (from block 410) and also exceeds the second uncertainty threshold, the throttling unit 124 skips branch prediction for a second number of cycles greater than the first number of cycles at block 418, such as previously with respect to Figure 2 Discuss in more detail.

[0047] If the throttling unit 124 determines at block 412 that the processor 100 is operating in multithreaded mode, the method 400 proceeds to block 420. At block 420, the throttling unit 124 determines whether the accumulated uncertainty counter exceeds the second uncertainty threshold. If the throttling unit 124 determines that the accumulated uncertainty counter exceeds the first uncertainty threshold (from block 410) but does not exceed the second uncertainty threshold, the throttling unit 124 allocates a first number of branch prediction cycles to different execution threads (block 422), such as previously allocated with respect to the first branch prediction cycle. Figure 3 If the throttling unit 124 determines that the accumulated uncertainty counter exceeds the first uncertainty threshold (from block 410) and also exceeds the second uncertainty threshold, the throttling unit 124 allocates a second number of branch prediction cycles (block 424) to a different execution thread that is greater than the first number of cycles, such as previously allocated with respect to the branch prediction counter. Figure 3 Discuss in more detail.

[0048] As shown, each in frame 416,418,422 and 424 returns to frame 402 to carry out subsequent new prediction cycle.Although throttling is carried out when the cumulative uncertainty counter increases to be higher than various uncertainty thresholds, if the cumulative uncertainty counter falls back to be lower than the uncertainty threshold, then branch prediction reverts to previous prediction rate.As previously discussed, when each in a plurality of branch predictions exits or is refreshed from the branch prediction pipeline, the uncertainty value of each in a plurality of branch predictions is subtracted from the cumulative uncertainty counter.The cumulative uncertainty counter starts to reduce when the instruction exits.When the instruction exits, if the instruction exits when no redirection is performed, then this means that the prediction made is correct.In that case, prediction is no longer in progress and is therefore subtracted from the cumulative uncertainty counter.

[0049] Thus, in single-threaded mode, when the accumulated uncertainty counter falls below the second uncertainty threshold, the throttling unit 124 instructs the branch predictor 108 to skip fewer predictions. Similarly, in single-threaded mode, when the accumulated uncertainty counter falls below the first uncertainty threshold, the throttling unit 124 instructs the branch predictor 108 to abandon throttling and return to generating new predictions every cycle. In multi-threaded mode, when the accumulated uncertainty counter falls below the second uncertainty threshold, the throttling unit 124 instructs the branch predictor 108 to allocate less processing bandwidth to the second execution thread and return processing bandwidth to the first execution thread. Similarly, in multi-threaded mode, when the accumulated uncertainty counter falls below the first uncertainty threshold, processing bandwidth is returned to the first execution thread, which resumes generating new predictions every other cycle.

[0050] In this way, the branch confidence throttling discussed herein slows down threads that are unlikely to be on the correct path, and in doing so, saves power by not executing instructions that are likely to be flushed. The processor yields power and computing resources to another thread sharing the same hardware within the CPU core (e.g., in multi-threaded mode), or can save power by not generating new predictions in cycles (e.g., in single-threaded mode). Power resources yielded by threads in a given CPU core can be utilized by threads on other CPU cores within an IC package that can dynamically allocate power across multiple CPU cores. In this way, branch confidence throttling enables more efficient use of computing resources.

[0051] As disclosed herein, in some embodiments, a processor includes: a branch predictor including one or more branch target buffer (BTB) tables; a branch prediction pipeline including a throttling unit and an uncertainty accumulator; wherein: the processor is configured to assign an uncertainty value to each of a plurality of branch predictions generated by the branch predictor; the uncertainty accumulator is configured to add the uncertainty value of each of the plurality of branch predictions to an accumulated uncertainty counter associated with the uncertainty accumulator; and the throttling unit is configured to throttle operation of the branch prediction pipeline based on the accumulated uncertainty counter. In one aspect, when each of the plurality of branch predictions retires or is flushed from the processor pipeline, the uncertainty value of each of the plurality of branch predictions is subtracted from the accumulated uncertainty counter. In another aspect, the throttling unit is further configured to reduce a prediction rate for a first execution thread based on the accumulated uncertainty counter exceeding one or more uncertainty thresholds.

[0052] In one aspect, the throttling unit is further configured to, when the processor is in single-threaded mode, instruct the branch predictor to skip predictions for one or more cycles based on the accumulated uncertainty counter exceeding one or more uncertainty thresholds. In another aspect, the throttling unit is further configured to, when the processor is in multi-threaded mode, instruct the branch predictor to allocate processing bandwidth to the second execution thread for one or more cycles based on the accumulated uncertainty counter exceeding one or more uncertainty thresholds. In yet another aspect, the throttling unit is further configured to instruct the branch predictor to return processing bandwidth from the second execution thread to the first execution thread for a first number of cycles in response to the accumulated uncertainty counter falling below a first uncertainty threshold. In yet another aspect, the processor includes: a conditional predictor configured to assign a confidence level to each of a plurality of branch predictions generated by the branch predictor, wherein the processor is configured to assign an uncertainty value to each of the plurality of branch predictions based at least in part on the confidence level.

[0053] As disclosed herein, in some embodiments, a method includes: assigning an uncertainty value to a branch prediction generated by a branch predictor; adding the uncertainty value of the branch prediction to an accumulated uncertainty counter associated with the branch predictor; and throttling the operation of the branch predictor based on the accumulated uncertainty counter exceeding one or more uncertainty thresholds. In one aspect, adding the uncertainty value of the branch prediction further includes: incrementing the accumulated uncertainty counter at the back end of a branch prediction pipeline of the branch predictor. In another aspect, the method includes subtracting the uncertainty value of the branch prediction from the accumulated uncertainty counter when the branch prediction is retired or flushed from the processor pipeline.

[0054] In one aspect, throttling the operation of the branch predictor further comprises reducing a prediction rate for a first execution thread based on an accumulated uncertainty counter exceeding one or more uncertainty thresholds. In another aspect, the method comprises skipping branch prediction for the first execution thread for a first number of cycles based on the accumulated uncertainty counter exceeding a first uncertainty threshold of the one or more uncertainty thresholds. In yet another aspect, the method comprises skipping branch prediction for the first execution thread for a second number of cycles greater than the first number of cycles based on the accumulated uncertainty counter exceeding a second uncertainty threshold greater than the first uncertainty threshold.

[0055] In one aspect, the method includes allocating processing bandwidth associated with a first number of cycles to a second execution thread when a branch predictor operates in a multithreaded mode. In another aspect, the method includes assigning a confidence level to a branch prediction; and assigning an uncertainty value to the branch prediction based at least in part on the confidence level.

[0056] As disclosed herein, in some embodiments, a method includes: incrementing an accumulated uncertainty counter of a first execution thread at an uncertainty accumulator of a branch prediction pipeline by an uncertainty value; and arbitrating between a plurality of execution threads based at least in part on a determination that the accumulated uncertainty counter exceeds one or more uncertainty thresholds. In one aspect, the method includes: allocating processing bandwidth from the first execution thread to a second execution thread for a first number of cycles based on the accumulated uncertainty counter exceeding a first uncertainty threshold.

[0057] In another aspect, the method includes allocating processing bandwidth from the first execution thread to the second execution thread for a second number of cycles greater than the first number of cycles based on an accumulated uncertainty counter exceeding a second uncertainty threshold greater than the first uncertainty threshold. In yet another aspect, the method includes allocating processing bandwidth from the second execution thread to the first execution thread for a first number of cycles based on the accumulated uncertainty counter falling below the first uncertainty threshold. In yet another aspect, the method includes assigning a confidence level to a branch prediction generated by a branch prediction pipeline; and assigning an uncertainty value to the branch prediction based at least in part on the confidence level.

[0058] In some embodiments, the apparatus and techniques described above are implemented in a system that includes one or more integrated circuit (IC) devices (also referred to as integrated circuit packages or microchips), such as those described above with reference to Figures 1 to 4 The system, processor and BTB table described are described. Electronic design automation (EDA) and computer-aided design (CAD) software tools can be used for the design and manufacture of these IC devices. These design tools are typically represented as one or more software programs. The one or more software programs include code that can be executed by a computer system to manipulate the computer system to operate on the code representing the circuit of one or more IC devices so as to perform at least a portion of the process for designing or adapting the manufacturing system to manufacture the circuit. This code may include instructions, data, or a combination of instructions and data. The software instructions representing the design tool or manufacturing tool are typically stored in a computer-readable storage medium that can be accessed by the computing system. Similarly, the code representing one or more stages of the design or manufacture of the IC device can be stored in the same computer-readable storage medium or different computer-readable storage media and accessed therefrom.

[0059] Computer-readable storage media may include any non-transitory storage medium or combination of non-transitory storage media that can be accessed by a computer system during use to provide instructions and / or data to the computer system. Such storage media may include, but are not limited to, optical media (e.g., compact discs (CDs), digital versatile discs (DVDs), Blu-ray discs), magnetic media (e.g., floppy disks, magnetic tapes, or magnetic hard drives), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or microelectromechanical systems (MEMS)-based storage media. Computer-readable storage media may be embedded in a computing system (e.g., system RAM or ROM), fixedly attached to a computing system (e.g., a magnetic hard drive), removably attached to a computing system (e.g., an optical disc or flash memory based on a universal serial bus (USB)), or coupled to a computer system via a wired or wireless network (e.g., a network accessible storage device (NAS)).

[0060] In some embodiments, certain aspects of the above-described technology may be implemented by one or more processors of a processing system that executes software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied on a non-transitory computer-readable storage medium. The software may include instructions and certain data that manipulate one or more processors to perform one or more aspects of the above-described technology when executed by one or more processors. The non-transitory computer-readable storage medium may include, for example, a magnetic or optical disk storage device, a solid-state storage device (such as a flash memory, a cache, a random access memory (RAM), or one or more other non-volatile memory devices), etc. The executable instructions stored on the non-transitory computer-readable storage medium may be source code, assembly language code, object code, or other instruction formats that are interpreted or otherwise executed by one or more processors.

[0061] Not all activities or elements described above in the general description are required, part of a specific activity or device may not be required, and one or more additional activities may be performed, or elements other than those described may be included. In addition, the order in which the activities are listed is not necessarily the order in which the activities are performed. Moreover, the concepts have been described with reference to specific embodiments. However, it will be understood by those skilled in the art that various modifications and changes may be made without departing from the scope of the present disclosure as set forth in the appended claims. Therefore, the description and drawings are to be regarded as illustrative rather than restrictive, and all such modifications are intended to be included within the scope of the present disclosure.

[0062] Benefits, other advantages and solutions to problems have been described above with reference to specific embodiments. However, the benefits, advantages, solutions to problems, and any features that may make any benefit, advantage or solution appear or become more prominent should not be construed as key, required or essential features of any or all of the claims. Furthermore, the specific embodiments disclosed above are illustrative only, as the disclosed subject matter may be modified and practiced in different but equivalent manners that will be apparent to those skilled in the art having the benefit of the teachings herein. No limitation is intended to the details of construction or design shown herein, except as described in the appended claims. Therefore, the specific embodiments disclosed above may be changed or modified, and all such variations are considered to be within the scope of the disclosed subject matter. Accordingly, the protection sought herein is as set forth in the appended claims.

Claims

1. A processor [100], comprising: A branch predictor [108], the branch predictor including one or more branch target buffer (BTB) tables [110]; a branch prediction pipeline [122], the branch prediction pipeline comprising a throttling unit [124] and an uncertainty accumulator [130]; in: The processor is configured to assign an uncertainty value to each of a plurality of branch predictions generated by the branch predictor [132]; the uncertainty accumulator being configured to add the uncertainty value of each of the plurality of branch predictions to an accumulated uncertainty counter associated with the uncertainty accumulator [134]; and The throttling unit is configured to instruct the branch predictor to skip prediction for one or more cycles based on the accumulated uncertainty counter exceeding one or more uncertainty thresholds.

2. The processor of claim 1 , wherein the uncertainty value for each of the plurality of branch predictions is subtracted from the accumulated uncertainty counter when each of the plurality of branch predictions retires or is flushed from a processor pipeline. 3 . The processor of claim 1 , wherein the throttling unit is further configured to reduce a prediction rate for the first thread of execution based on the accumulated uncertainty counter exceeding one or more uncertainty thresholds.

4. The processor according to any one of claims 1 to 3, further comprising: a conditional predictor [128] configured to assign a confidence level to each of the plurality of branch predictions generated by the branch predictor, wherein the processor is configured to assign the uncertainty value to each of the plurality of branch predictions based at least in part on the confidence level.

5. A processor, comprising: A branch predictor, the branch predictor including one or more branch target buffer (BTB) tables; a branch prediction pipeline comprising a throttling unit and an uncertainty accumulator; in: The processor is configured to assign an uncertainty value to each of a plurality of branch predictions generated by the branch predictor; the uncertainty accumulator being configured to add the uncertainty value for each of the plurality of branch predictions to an accumulated uncertainty counter associated with the uncertainty accumulator; and The throttling unit is configured to instruct the branch predictor to allocate processing bandwidth from the first execution thread to the second execution thread for one or more cycles based on the accumulated uncertainty counter exceeding one or more uncertainty thresholds.

6. The processor of claim 5 , wherein the throttling unit is further configured to instruct the branch predictor to return processing bandwidth from the second execution thread to the first execution thread within a certain number of cycles in response to the accumulated uncertainty counter falling below the one or more uncertainty thresholds.

7. A method comprising: Assigning an uncertainty value [132] to a branch prediction generated by a branch predictor [108]; adding the uncertainty value of the branch prediction to an accumulating uncertainty counter associated with the branch predictor [134]; and Branch prediction is skipped for a first number of cycles based on the accumulated uncertainty counter exceeding a first uncertainty threshold.

8. The method of claim 7, wherein adding the uncertainty value of the branch prediction comprises: The accumulated uncertainty counter is incremented at the back end of the branch prediction pipeline [122] of the branch predictor.

9. The method of claim 7, further comprising: The uncertainty value of the branch prediction is subtracted from the accumulated uncertainty counter when the branch prediction is retired or flushed from the processor pipeline.

10. The method according to any one of claims 7 to 9, further comprising: A prediction rate for a first thread of execution is reduced based on the accumulated uncertainty counter exceeding the first uncertainty threshold.

11. The method according to any one of claims 7 to 9, further comprising: Branch prediction for a first thread of execution is skipped for a second number of cycles greater than the first number of cycles based on the accumulated uncertainty counter exceeding a second uncertainty threshold greater than the first uncertainty threshold.

12. The method according to any one of claims 7 to 9, further comprising: Processing bandwidth associated with the first number of cycles is allocated from the first execution thread to the second execution thread.

13. The method according to any one of claims 7 to 9, further comprising: assigning a confidence level to the branch prediction; as well as The uncertainty value is assigned to the branch prediction based at least in part on the confidence level.

14. A method comprising: At the uncertainty accumulator [130] of the branch prediction pipeline [122], incrementing the accumulated uncertainty counter [134] of the first execution thread by the uncertainty value [132]; Based at least in part on a determination that the accumulated uncertainty counter exceeds a first uncertainty threshold, processing bandwidth is allocated from the first thread of execution to a second thread of execution for a first number of cycles.

15. The method of claim 14, further comprising: Based on the accumulated uncertainty counter exceeding a first uncertainty threshold, processing bandwidth is allocated from the first execution thread to a second execution thread for a first number of cycles.

16. The method of claim 14 or claim 15, further comprising: Based on the accumulated uncertainty counter exceeding a second uncertainty threshold that is greater than the first uncertainty threshold, processing bandwidth is allocated from the first thread of execution to the second thread of execution for a second number of cycles that is greater than the first number of cycles.

17. The method of claim 14 or claim 15, further comprising: Based on the accumulated uncertainty counter falling below the first uncertainty threshold, processing bandwidth is allocated from the second thread of execution to the first thread of execution for the first number of cycles.

18. The method of claim 14 or claim 15, further comprising: assigning confidence levels to branch predictions generated by the branch prediction pipeline; as well as The uncertainty value is assigned to the branch prediction based at least in part on the confidence level.

Images