Method and apparatus for implementing adaptive branch prediction throttling

Abstract

Methods and apparatus for implementing adaptive branch prediction throttling are disclosed. In one embodiment, the method includes determining whether a current branch in execution of a single thread is a low confidence branch in which a branch predictor of a processor has a low certainty level at which the result of the current branch is correctly predicted based on lookup of a branch confidence data structure; and comparing the branch misprediction rate, the microinstruction wastage rate, and the cache miss rate of the single thread to their corresponding thresholds. The method further includes throttling the branch prediction of the current branch based on a determination that the current branch is a low confidence branch and one or more thresholds of a branch misprediction rate, a microinstruction wear rate, and a cache miss rate for the single thread are exceeded.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computing; and more particularly, the embodiments relate to apparatus and methods for implementing adaptive branch prediction throttling. Background Art

[0002] To increase single-threaded performance, modern computer processor microarchitectures continue to progress toward wider and deeper pipelines to exploit instruction-level parallelism. While these design choices rely aggressively on speculative execution to improve overall instruction throughput, they also incur high energy costs every time the processor pipeline needs to be flushed to redirect instruction fetches to the correct path due to wrong path execution. This drawback is particularly evident in workloads with high branch misprediction rates and / or long branch resolution latencies.

[0003] Prior work has proposed techniques for gating pipeline stages (e.g., fetch, decode) to reduce the number of instructions that are speculatively issued. These techniques rely on branch confidence estimates to decide when the pipeline should be gated. However, such techniques do not consider the dynamic operating conditions and performance benefits of wrong-path execution when making decisions to limit speculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The present disclosure may be better understood by referring to the following description and the accompanying drawings which are used to illustrate embodiments of the present disclosure.

[0005] Figure 1 Illustrated is an example of computing hardware for implementing adaptive speculative restrictions on execution of instructions in a processor in accordance with some embodiments.

[0006] Figure 2 Illustrated are entries of a branch confidence data structure (BCDS) for adaptive speculation limiting according to some embodiments.

[0007] Figure 3 An adaptive speculation limiting (ASL) configuration data structure is illustrated in accordance with some embodiments.

[0008] Figure 4 Illustrated is throttling in an example of an in-order pipeline and an example of a register renaming, out-of-order issue / execution pipeline in accordance with some embodiments.

[0009] Figure 5 is a block diagram illustrating adaptive speculation limits (ASL) in an in-order architecture core and a register renaming, out-of-order issue / execution architecture core to be included in a processor in accordance with some embodiments.

[0010] Figure 6 An operational flow for implementing adaptive branch prediction throttling according to some embodiments is illustrated.

[0011] Figure 7 Examples of execution unit circuitry, such as execution unit circuitry, are illustrated.

[0012] Figure 8 An example of a computing system is illustrated.

[0013] Fig. 9 A block diagram illustrating an example processor and / or System on a Chip (SoC) that may have one or more cores and an integrated memory controller.

[0014] Fig.10 is a block diagram illustrating a computing system 1000 configured to implement one or more aspects of the examples described herein.

[0015] Fig.11A Figure 1 shows an example of a parallel processor.

[0016] Fig. 11B An example of a block diagram of a partition unit is illustrated.

[0017] Fig. 11C An example of a block diagram illustrating a processing cluster within a parallel processing unit.

[0018] Fig.11D An example of a graphics multiprocessor is illustrated, wherein the graphics multiprocessor is coupled with a pipeline manager of a processing cluster.

[0019] Figure 12A-12C Additional graphics multiprocessors are illustrated according to some examples.

[0020] Fig.13 A parallel computing system 1300 is shown according to some examples.

[0021] Figure 14A-14B Illustrate a hybrid logical / physical view of a split parallel processor according to examples described herein.

[0022] Figure 15A-15B Thread execution logic including an array of processing elements employed in a graphics processor core is illustrated according to examples described herein.

[0023] Fig.16 Additional execution units are illustrated according to an example.

[0024] Fig.17 is a block diagram of another example of a graphics processor.

[0025] Fig.18 is a block diagram illustrating converting binary instructions in a source ISA to binary instructions in a target ISA using a software instruction converter according to an example.

[0026] Fig.19 is a block diagram illustrating an IP core development system 1900 that may be used to fabricate an integrated circuit to perform operations according to some examples. DETAILED DESCRIPTION

[0027] In the following description, a number of specific details are set forth. However, it should be understood that embodiments of the present disclosure may be implemented without these specific details. In other instances, well-known circuits, structures, and techniques are not shown in detail to avoid obscuring the understanding of the present description.

[0028] Bracketed text and boxes with dashed borders (such as long dashes, short dashes, dot-dash lines, and dotted lines) may be used to illustrate optional operations that add additional features to embodiments of the present disclosure. However, such notation should not be taken to mean that these are the only options or the only optional operations, and / or to mean that boxes with solid borders are not optional in some embodiments of the present disclosure.

[0029] References in the specification to "one embodiment," "an embodiment," "an example embodiment," etc. indicate that the described embodiment may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Further, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it is considered to be within the knowledge of those skilled in the art to affect such feature, structure, or characteristic in conjunction with other embodiments, whether or not explicitly described.

[0030] The term "connected" means a direct electrical or magnetic connection between objects that are connected without any intervening devices, while the term "coupled" means a direct electrical or magnetic connection between objects that are connected, or an indirect connection through one or more passive or active intervening devices. The term "circuit" means one or more passive and / or active components arranged to cooperate with each other to provide a desired function. As used herein, "set" refers to any positive integer number of items including an item. Adaptive Speculation Limit (ASL) in processors

[0031] In computer processors, workloads cause frequent branch mispredictions. An experiment showed that fetching and dispatching around pipeline clears averages about 600 uops (micro-operations) for a 24-wide, 2048-deep pipeline design, where 24 instructions can be executed in parallel in a single clock cycle, with up to 2048 instructions / uops that can be in-flight at different stages of the execution pipeline. The large number of uops in pipeline clears due to wrong path fetch / decode / dispatch / execute overhead dominates processor energy consumption.

[0032] Prior techniques have been used to gate pipeline stages to reduce the number of instructions speculatively issued. However, this results in a performance penalty for several reasons. One reason is that execution on the wrong path can result in useful prefetches (e.g., L1 / L2 / L3 prefetches) that can avoid future cache misses. This is an important consideration for workloads with high cache miss rates, which are often measured in cache misses per kilo instructions (MPKI). Aggressively limiting speculation in high MPKI workloads can hurt performance.

[0033] Additionally, if the workload has low cost (high branch prediction accuracy, e.g., due to low branch misprediction rate), the energy spent in wrong path execution is not a significant component of the total processor energy consumption. Limiting speculation in these workloads may incur a performance cost without providing significant energy improvement. Furthermore, when the processor is operating in a high performance state, the primary goal may be to maximize performance, and aggressive speculation restrictions may undermine this goal.

[0034] Embodiments of the present disclosure overcome the performance issues of these prior arts. The proposed adaptive speculation limit (ASL) limits the amount of speculation based on workload behavior and runtime conditions. Such mechanisms dynamically adapt speculation control based on real-time program behavior and / or performance / power tradeoffs, and may also be referred to as dynamic speculation limits, real-time / runtime speculation limits. ASL reduces the energy cost of aggressive speculation while retaining performance benefits.

[0035] Figure 1An example of computing hardware for implementing adaptive speculation restrictions on instruction execution in a processor according to some embodiments is illustrated. As shown, a processor or processor core 100 implements adaptive speculation restrictions (ASL) logic 150. ASL logic 150 may be implemented as hardware circuitry and / or a software module, or it may be in one functional unit (e.g., decoder circuitry 105 or register renaming, register allocation and / or scheduling circuitry 107), or distributed among multiple functional units as discussed in further detail herein. Note that processor and processor core (or cores) are used interchangeably in the discussion of ASL operation, and ASL applies to both.

[0036] A storage device 103, internal or external to the processor / core 100, stores a set of instructions 101 to be executed. The instructions 101 are received by a decoder circuit system 105. For example, the decoder circuit system 105 receives the instructions 101 from a fetch circuit system (not shown). The instructions can be in any suitable format, for example, the instruction includes fields for an opcode, one or more source identifiers, and a destination identifier. In some examples, the sources and destinations are registers, and in other examples, one or more of the sources and destinations are memory locations. In some examples, one or more of the sources may be immediate operands. According to some embodiments, the ASL logic 150 is implemented to throttle the execution of the instructions 101.

[0037] Decoder circuitry 105 decodes the instruction into one or more operations. In some examples, the decoding includes generating a plurality of micro-operations to be executed by execution circuitry, such as execution circuitry 109. Decoder circuitry 105 also decodes instruction prefixes.

[0038] In some examples, the register renaming, register allocation, and / or scheduling circuitry 107 provides functionality for one or more of: 1) renaming logical operand values ​​to physical operand values ​​(e.g., a register alias table in some examples), 2) assigning status bits and flags to decoded instructions, and 3) scheduling decoded instructions from an instruction pool for execution by the execution circuitry (e.g., using a reservation station in some examples).

[0039] Registers (register files) and / or memory 108 store data as an operation target for instructions to be operated by execution circuit system 109. Example register types include packed data registers, general purpose registers (GPRs), and floating point registers.

[0040] The execution circuit system 109 executes the decoded instructions. The execution circuit system of the example detailed includes Figure 5The execution cluster(s) 560 shown in Fig.11A The parallel processing unit 1102 in Fig.11D Graphics multiprocessor 1134, etc.

[0041] In some examples, retirement / writeback circuitry 111 architecturally commits the destination register to register or memory 108 and retires the instruction.

[0042] An example of a format for an instruction is OPCODE DST, SRC1, SRC2. In some examples, OPCODE is the opcode mnemonic for the instruction. DST is a field for the location of the destination operand (such as a general register, a packed data register, or a memory). SRC1 and SRC2 are fields for the location of the source operand (such as a general register, a packed data register, and / or a memory).

[0043] Adaptive speculation limit (ASL) logic 150 allows the processor front end to dynamically control the amount of speculation based on runtime program behavior (e.g., cost and branch confidence information). The goal of ASL is to reduce power consumption for high-cost workloads without giving up too much performance. High performance is retained for low-cost workloads (e.g., through conservative ASL).

[0044] In some embodiments, performance is indicated by performance states (P-states), each of which has a frequency and voltage combination at which the processor can operate. Each P-state represents a specific performance level, which is often associated with different power consumption and thermal characteristics. High-performance P-states are P-states that allow the processor to operate at a higher frequency, thereby providing increased computing power. In higher P-states, the processor can consume higher power and run at higher temperatures, while in lower P-states, the processor consumes less power and runs at lower temperatures.

[0045] The ASL logic 150 can be thought of as making predictions about how much to speculate. Without ASL, the processor might always set the probability of prediction to a maximum with uncontrolled / unthrottled speculation. With ASL, the processor can throttle speculation based on (1) the importance of limiting speculation and / or (2) the confidence level of execution on the correct path. For (1), if the cost is high and / or saving power is important (e.g., the processor is operating in a mobile environment), the processor tends to throttle speculation aggressively (less speculative execution). For (2), if the confidence level is low, the processor also tends to throttle speculation aggressively. If (1) and (2) point to opposite throttling levels, for example, when the cost is low (allowing more speculation to improve execution performance) but the confidence level is low (requiring a lower level of speculation), ASL weights these factors and determines the appropriate throttling level.

[0046] In some embodiments, ASL logic 150 limits speculation based on power management (PM) hints that indicate saving power in relation to Figure 3 Whether this is important in the context of the operation of processor 100 discussed in further detail. These hints are used to select an ASL configuration that specifies the aggressiveness of the speculation restrictions. The configurations range from a conservative configuration, which is the least restricted (i.e., the least restricted on how much speculation is allowed), to an aggressive configuration, which is the most restricted. The least restricted configuration yields the lowest power savings, and the most restricted yields the highest power savings.

[0047] In some embodiments, power savings are measured by the dynamic power consumption of the processor. Dynamic power (also called switching power) is the power consumed by the processor during normal operation of the processor when transistors switch states and perform computing tasks. Cdyn is a measure of the power consumed by the processor due to the charging and discharging of internal capacitance associated with switching transistors. In some embodiments, Cdyn is used as an example of a power savings metric.

[0048] The ASL logic 150 may be viewed as including three components: (i) low confidence detection, which determines that the processor is fetching instructions from a low confidence path; (ii) low confidence throttling, which reduces instruction delivery to execution when low confidence is detected and until the processor is no longer on a low confidence path; and (iii) adaptive threshold adjustment, which is used to fine-tune the threshold for triggering low confidence operations based on runtime program behavior and / or current performance state. These components may be multiple circuit systems / software modules integrated into a single ASL logic 150 in some embodiments, and may be multiple circuit systems / software modules of distributed ASL logic 150 in multiple processor entities in alternative embodiments. For example, in a distributed implementation, (i) and (iii) may be implemented in the decoder circuit system 105, while (ii) may be implemented in the register renaming, register allocation and / or scheduling circuit system 107. Note that these components may also be implemented in other units (e.g., execution circuit system 109).

[0049] In some embodiments, low confidence detection is performed by a branch confidence data structure (BCDS) 192, while the threshold for low confidence throttling is determined using an ASL configuration 194. These entities are discussed in more detail herein. Although BCDS 192 and ASL configuration 194 are shown as being implemented in decoder circuitry 105, in alternative embodiments, one or both of BCDS 192 and ASL configuration 194 may be implemented in another entity within processor 100. Low confidence detection

[0050] To throttle speculative execution in the wrong path, Adaptive Speculation Limit (ASL) detects whether the processor is taking instructions from the wrong path (e.g., a branch that may be mispredicted). A Branch Confidence Data Structure (BCDS) is introduced for this detection. Figure 2 Entries of a branch confidence data structure (BCDS) for adaptive speculation limiting according to some embodiments are illustrated. The BCDS may be stored in a memory location (e.g., a cache), and it tracks the prediction accuracy of recent branches. The BCDS may be implemented as a table, a map, a dictionary, a list, an array, a file, or another similar / different data structure.

[0051] In one embodiment, the fields of BCDS entry 202 are shown with field names, exemplary field sizes, and descriptions. In other embodiments, more or fewer fields, each with the same / different field sizes, may be implemented.

[0052] The BCDS entry 202 is identified by a tag that identifies multiple instances of the same branch. For example, the tag can be the cache line address for an instruction (e.g., the cache line storing the loop condition instruction). The BCDS entry 202 also includes a validity field for indicating whether the entry is valid and an offset field for indicating the offset of the branch in the prediction window. The BCDS entry 202 includes one or more counters for tracking the number of times the branch has been mispredicted relative to being correctly predicted.

[0053] In some embodiments, the BCDS entry is allocated when the branch is first mispredicted. For example, the initial values of the counter fields in the BCDS entry 202 can be: pred_count (prediction_count)=1, mispred_count (misprediction_count)=1.

[0054] Subsequently, whenever an instance of the branch retires, the prediction counter and misprediction counter for that branch are updated: at each branch retirement, pred_count is incremented; at each misprediction, mispred_count is incremented.

[0055] At each update, the low-confidence criterion is recalculated. In a first embodiment, the low confidence is determined based on accuracy: is_low_conf (is_low_confidence)=accuracy (accuracy)<accuracy_thresh (accuracy_threshold). In a second embodiment, the low confidence is also determined based on the sufficiency of the data: is_low_conf=(mispred_count>mispred_thresh (misprediction_threshold))&&(accuracy<accuracy_thresh).

[0056] The above two thresholds are: (1) mispred_thresh: the minimum number of mispredictions caused by the branch before the branch can be considered a low-confidence branch candidate; and (2) accuracy_thresh: the maximum accuracy for a branch that can be considered a low-confidence branch candidate.

[0057] For example, mispred_thresh = 8, accuracy_thresh = 3 / 4 implies that any branch with a prediction accuracy less than 75% and that has caused at least 8 mispredictions will be considered a low-confidence branch.

[0058] Note that the low-confidence criterion in the second embodiment is based not only on the prediction accuracy for a particular branch but also on how many times the branch has retired. This is a deliberate design choice because such a design avoids classifying those branches as low-confidence branches when the branch predictor has not had enough time to train on the branch.

[0059] In some embodiments, in parallel with the branch misprediction structure handling the results of branch mispredictions, the BCDS is looked up on each branch prediction. If there is a hit (an entry for the branch exists in the BCDS) and the is_low_confidence_bit is set (field shown in the figure), the ASL logic 150 treats the branch as a low confidence branch. Upon detection of a low confidence branch, instruction execution enters a low confidence mode in which the processor throttles speculative execution if operating conditions and performance / power tradeoffs also favor throttling, as measured by a set of operating condition thresholds referred to as global ASL thresholds. Global ASL Threshold and ASL Configuration

[0060] In addition to the branch-by-branch accuracy and misprediction thresholds (e.g., mispred_thresh and accuracy_thresh) discussed herein, ASL also implements a set of global ASL thresholds that measure and quantify the operating conditions of the processor. Various thresholds related to branch prediction and cache usage efficiency can be used as global ASL thresholds. Embodiments are explained using three thresholds (thresholds (1) to (3) below) and the effects of exceeding these thresholds when throttling speculation.

[0061] (1) Misprediction rate threshold: measures the number of branches that are mispredicted every X clock cycles (e.g., 1000 clock cycles). For example, speculation is limited only if the misprediction rate is above the misprediction threshold. The default value of the threshold can be set to 10 mispredictions per 1000 cycles.

[0062] (2) Cost threshold: A measure of the fraction of speculative allocation, e.g., allocated uops compared to (ii) retired uops. For example, the fraction may be (allocated uops) / (retired uops), or another formula that includes both (i) and (ii). In that example, speculation may be limited only if the measured cost is above a threshold (where the default value may be set to 2).

[0063] (3) Cache miss rate threshold: measures the number of cache misses at a certain level (e.g., L1, L2, or L3) per Y clock cycles (e.g., 1000 cycles). For example, speculation can be restricted only if the L2 MPKI is below its corresponding threshold.

[0064] Note that while these global ASL thresholds and the thresholds for low confidence detection can be configured with default values, they can be set based on performance / power tradeoffs. For example, when running the processor faster is more important than saving power (e.g., when the processor is a server processor in a data center), the thresholds can be set so that less throttling will be performed. Such conservative throttling may have a lower accuracy threshold, a higher misprediction rate threshold, and a higher cache miss rate threshold. On the other hand, when running the processor more power efficiently is more important than running the processor faster (e.g., when the processor is a mobile processor in a smartphone), the thresholds can be set so that more throttling will be performed. Such aggressive throttling may have a higher accuracy threshold, a lower misprediction rate threshold, and a lower cache miss rate threshold.

[0065] Additionally, these thresholds may be adjusted based on varying performance / power tradeoffs. For example, when a processor in a laptop computer is plugged into a power outlet, the processor may be throttled conservatively to favor high performance. When the same processor is running on battery, the processor may be throttled aggressively to favor power efficiency. In order to configure ASL for a processor in different settings, in some embodiments, these thresholds may be saved in an ASL configuration data structure so that the processor may select an appropriate ASL configuration with a particular threshold. Setting and / or adjusting the ASL configuration may be performed by an adaptive threshold adjustment component of the ASL logic 150.

[0066] Figure 3 An adaptive speculation limit (ASL) configuration data structure is illustrated in accordance with some embodiments. The ASL configuration data structure may be implemented as a table, a mapping, a dictionary, a list, an array, a file, or another similar / different data structure. The accompanying figure illustrates the ASL configuration as a table at reference 302. Thresholds associated with branch-by-branch low confidence detection and three global ASL thresholds are illustrated. Three configurations are presented, from the most conservative (e.g., applicable when higher performance is more important than power efficiency) to the default, and then to the aggressive (e.g., applicable when consuming less power is more important than performance status). Note that while Figure 3 Only an indication of relative high or low for a particular threshold relative to a default value is shown, but in some embodiments, actual values ​​are used in the data structure and these actual values ​​may be adjusted based on performance / power tradeoffs and previous performance of the thresholds when making performance / power tradeoffs. In various embodiments, the thresholds may be set and / or adjusted heuristically or through machine learning or heuristic algorithms.

[0067] Although three configuration settings are shown with 5 thresholds, multiple settings may be selected with more or fewer thresholds for throttling. One of the configurations may be selected by / for processing based on the desired dynamic performance / power tradeoff.

[0068] The selection may be based on the P-state of the processor, which may be indicated in a 2-bit or 3-bit information at a control register (CR). The ASL logic 150 may map the P-state information to one or more supported ASL configurations. Based on the P-state, an appropriate configuration may be selected from an ASL configuration data structure. Note that in some embodiments, as shown at reference 350, the lower the performance (P-state), the higher the level of throttling is implemented. Thus, in these embodiments, a performance state of the processor lower than a default performance state corresponds to a throttling level higher than the default throttling level. Low confidence mode entry and exit

[0069] When the processor is determined to be on a low-confidence branch (e.g., based on a hit in the BCDS) and the global ASL criteria are met (e.g., a threshold for the ASL configuration is exceeded), the processor enters a low-confidence mode in which the processor throttles speculative execution based on the ASL configuration (e.g., aggressively, conservatively, or in an intermediate state). The processor remains in the low-confidence mode until one or more of the following events occurs: (1) the low-confidence branch resolves to being correctly predicted (i.e., the low confidence is mis-asserted); (2) the low-confidence branch resolves to being mis-predicted and causes a pipeline flush (i.e., the low confidence is appropriate and the effects of the misprediction have been waived); and (3) any instruction older than the low-confidence branch causes a pipeline flush (front-end or back-end flush).

[0070] In some embodiments, if the processor is already in low confidence mode, any subsequent hits in the BCDS are disregarded until the processor exits from low confidence mode.

[0071] Note that after exiting low confidence mode, the processor may re-enter low confidence mode shortly after (based on the same or different ASL configuration). For example, consider a difficult-to-predict branch in a long running loop. Since the processor keeps taking that branch repeatedly, the processor may continue through the exit sequence shortly after re-entering low confidence mode. Throttling during low confidence mode

[0072] After a processor enters a low confidence mode, the processor limits the amount of speculation by throttling the corresponding execution pipeline, and such throttling may be performed at one or more stages.

[0073] In some embodiments, ASL is implemented by throttling the processor front-end bandwidth. In these embodiments, instead of actively looking up the branch predictor every cycle, the processor remains active for M cycles, followed by being inactive for another N cycles. The values ​​of M and N can be configured based on post-silicon (post-si) analysis, which is the process of evaluating and debugging a processor after it has been manufactured and fabricated in silicon. In one embodiment, M and N are set to 1 and 3, respectively (activity factor 25%).

[0074] It is worth noting that since branch prediction is the first stage in the processor pipeline, throttling the rate at which the branch prediction pipeline operates automatically throttles later stages of the pipeline as well. In addition to the fetch stage, throttling is also performed in other stages in the execution pipeline.

[0075] Figure 4 The diagram illustrates throttling in an example of an in-order pipeline and an example of a register renaming, out-of-order issue / execution pipeline according to some embodiments. In the figure, a processor pipeline 400 includes a fetch stage 402, an optional length decode stage 404, a decode stage 406, an optional allocation (Alloc) stage 408, an optional rename stage 410, a dispatch (also called dispatch or issue) stage 412, an optional register read / memory read stage 414, an execute stage 416, a write back / memory write stage 418, an optional exception handling stage 422, and an optional commit stage 424.

[0076] One or more operations may be performed in each of these processor pipeline stages. For example, during the fetch stage 402, one or more instructions are fetched from the instruction memory, and during the decode stage 406, the fetched one or more instructions may be decoded, an address using a forwarded register port (e.g., a load store unit (LSU) address) may be generated, and branch forwarding (e.g., an immediate offset or a link register (LR)) may be performed. In one embodiment, the decode stage 406 and the register read / memory read stage 414 may be combined into one pipeline stage. In some examples, during the execute stage 416, decoded instructions may be executed, LSU address / data pipelining may be performed to an Advanced Microcontroller Bus (AMB) interface, multiplication and addition operations may be performed, arithmetic operations with branch results may be performed, and the like.

[0077] Note that throttling may be performed in any stage up to the execute stage 416, as shown at reference 450. That is, slowing down operations at any stage of instruction execution up to the execute stage 416 will reduce the rate of speculative execution. An example of such throttling in the instruction decode stage 406 is to perform instruction decode for M cycles, followed by keeping the decode stage 406 inactive for N cycles. Such alternating active and inactive cycles may also be implemented for the ASL in one or more other stages in the execution pipeline 400 (e.g., the allocation stage, rename stage, schedule stage, register / memory read stage, and execute stage at references 408 to 416).

[0078] Another example of throttling is reducing the rate of operations in one or more stages in the execution pipeline 400, e.g., during low confidence mode, the decode rate (number of instructions decoded per cycle) is set to less than the available decode bandwidth (e.g., 25% of the peak decode bandwidth). Another method for reducing the amount of speculation through the ASL is to limit the number of in-flight instructions in the execution pipeline 400 during low confidence mode. This functionality can be implemented, for example, in the allocation / rename stage (at reference 408 / 410) of the pipeline by constraining the number of available entries in the reorder buffer (ROB). Although the execution pipeline 400 is used as an example of task execution, other processors, execution pipelines of processors, and stages / components (e.g., FIGS. 11 to 12) may be implemented. Fig.17 Multiple) shown in can achieve ASL through the same / similar throttling method.

[0079] Figure 51 is a block diagram illustrating adaptive speculation limit (ASL) in an in-order architecture core and a register renaming, out-of-order issue / execution architecture core to be included in a processor according to some embodiments. The accompanying figure shows that a processor core 590 includes a front end unit circuit system 530 coupled to an execution engine unit circuit system 550, and both are coupled to a memory unit circuit system 570. The core 590 can be a reduced instruction set architecture computing (RISC) core, a complex instruction set architecture computing (CISC) core, a very long instruction word (VLIW) core, or a hybrid or alternative core type. As another option, the core 590 can be a special purpose core, such as a network or communication core, a compression engine, a coprocessor core, a general purpose computing graphics processing unit (GPGPU) core, a graphics core, etc. The ASL logic 150 can be implemented in the front end unit 530 or the execution unit 550. In some embodiments, ASL logic 150 may have a distributed implementation, for example, with a portion in front end unit 530 (e.g., low confidence detection component) and another portion in execution units (e.g., low confidence throttling and adaptive threshold adjustment components).

[0080] The front end unit circuit system 530 may include a branch prediction circuit system 532, which is coupled to an instruction cache unit circuit system 534, which is coupled to an instruction translation lookaside buffer (TLB) 536, which is coupled to an instruction fetch circuit system 538, which is coupled to a decoding circuit system 540. In some examples, the instruction cache circuit system 534 is included in the memory unit circuit system 570 instead of the front end circuit system 530. The decoding circuit system 540 (or decoder) can decode the instruction and generate one or more micro-operations, micro-code entry points, micro-instructions, other instructions or other control signals as output, which are decoded from the original instruction, or otherwise reflect the original instruction, or are derived from the original instruction. The decoding circuit system 540 may also include an address generation unit (AGU, not shown) circuit system. In some examples, the AGU uses the forwarded register port to generate the LSU address and may further perform branch forwarding (e.g., immediate offset branch forwarding, LR register branch forwarding, etc.). The decode circuit system 540 may be implemented using a variety of different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLA), microcode read only memory (ROM), and the like. In some examples, the core 590 includes a microcode ROM (not shown) or other medium that stores microcode for certain macroinstructions (e.g., in the decode circuit system 540 or otherwise within the front-end circuit system 530). In some examples, the decode circuit system 540 includes a micro-op or operation cache (not shown) to save / cache decoded operations, micro-tags, or micro-operations generated during decoding or other stages of the processor pipeline 500. The decode circuit system 540 may be coupled to the rename / allocator unit circuit system 552 in the execution engine circuit system 550.

[0081] Execution engine circuitry 550 includes rename / allocator unit circuitry 552 coupled to retirement unit circuitry 554 and a set of one or more scheduler circuitry 556. Scheduler circuitry 556 represents any number of different schedulers, including reservation stations, central instruction windows, and the like. In some examples, scheduler circuitry(s) 556 may include an arithmetic logic unit (ALU) scheduler / scheduling circuitry, an ALU queue, an address generation unit (AGU) scheduler / scheduling circuitry, an AGU queue, and the like. Scheduler circuitry(s) 556 are coupled to physical register file(s) circuitry 558. Each of the physical register file(s) circuitry 558 represents one or more physical register files, different ones of which store one or more different data types, such as scalar integer, scalar floating point, packed integer, packed floating point, vector integer, vector floating point, status (e.g., instruction pointer, i.e., the address of the next instruction to be executed), and the like. In some examples, the (one or more) physical register file circuit systems 558 include vector register unit circuit systems, write mask register unit circuit systems, and scalar register unit circuit systems. These register units can provide architectural vector registers, vector mask registers, general registers, etc. The (one or more) physical register unit file circuit systems 558 are coupled to the retirement unit circuit system 554 (also called retirement queue) overlap to show various ways that register renaming and out-of-order execution can be implemented (for example, using (one or more) reorder buffers (ROB) and (one or more) retirement register files; using (one or more) future stacks, (one or more) history buffers, and (one or more) retirement register files; using register maps and pools of registers; etc.). The retirement unit circuit system 554 and the (one or more) physical register file circuit system 558 are coupled to (one or more) execution clusters 560. The (one or more) execution clusters 560 include a set of one or more execution unit circuit systems 562 and a set of one or more memory access circuit systems 564. Execution unit circuitry 562 may perform various arithmetic, logical, floating point, or other types of operations (e.g., shifts, additions, subtractions, multiplications) on various types of data (e.g., scalar integer, scalar floating point, packed integer, packed floating point, vector integer, vector floating point). While some examples may include several execution units or execution unit circuitry dedicated to a particular function or set of functions, other examples may include only one execution unit circuitry or multiple execution units / execution unit circuitry that all perform all functions.Scheduler circuitry(s) 556, physical register file circuitry(s) 558, and execution cluster(s) 560 are shown as possibly multiple because some examples create separate pipelines for certain types of data / operations (e.g., a scalar integer pipeline, a scalar floating point / packed integer / packed floating point / vector integer / vector floating point pipeline, and / or a memory access pipeline, each with its own scheduler circuitry, physical register file circuitry(s), and / or execution cluster—and in the case of a separate memory access pipeline, only the execution cluster of that pipeline has memory access unit circuitry(s) 564 in some examples of the implementation). It should also be understood that where separate pipelines are used, one or more of these pipelines may be out-of-order issue / execution, while the rest are in-order.

[0082] In some examples, the execution engine unit circuitry 550 may perform load store unit (LSU) address / data pipelining to an advanced microcontroller bus (AMB) interface (not shown), as well as address stage and writeback, data stage loads, stores, and branches.

[0083] A set of memory access circuitry 564 is coupled to memory cell circuitry 570, which includes data TLB circuitry 572, which is coupled to data cache circuitry 574, which is coupled to level 2 (L2) cache circuitry 576. In some examples, memory access circuitry 564 may include load unit circuitry, store address unit circuitry, and store data unit circuitry, each of which is coupled to data TLB circuitry 572 in memory cell circuitry 570. Instruction cache circuitry 534 is further coupled to level 2 (L2) cache unit circuitry 576 in memory cell circuitry 570. In some examples, instruction cache 534 and data cache 574 are combined into L2 cache circuitry 576, level 3 (L3) cache circuitry (not shown), and / or a single instruction and data cache in main memory (not shown). L2 cache circuitry 576 is coupled to one or more other levels of cache and ultimately to main memory.

[0084] Core 590 may support one or more instruction sets (e.g., x86 instruction set architecture (optionally with some extensions that have been added with newer versions); MIPS instruction set architecture; ARM instruction set architecture (optionally with optional additional extensions, such as NEON)) that include the instruction(s) described herein. In some examples, core 590 includes logic to support packed data instruction set architecture extensions (e.g., AVX1, AVX2), allowing operations used by many multimedia applications to be performed using packed data.

[0085] As an example, Figure 5 An example register renaming, out-of-order issue / execution architecture core may implement pipeline 400 as follows: 1) instruction fetch circuitry 538 performs fetch stage 402 and length decode stage 404; 2) decode circuitry 540 performs decode stage 406; 3) rename / allocator unit circuitry 552 performs allocate stage 408 and rename stage 410; 4) (one or more) scheduler circuitry 556 performs schedule stage 412; 5) (one or more) physical register file circuitry 558 and memory unit circuitry 570 perform register read / memory read stage 414; (one or more) execution clusters 560 perform execute stage 416; 6) memory unit circuitry 570 and (one or more) physical register file circuitry 558 perform write back / memory write stage 418; 7) each circuitry may involve an exception handling stage 422; and 8) retirement unit circuitry 554 and (one or more) physical register file circuitry 558 perform commit stage 424. Operations in Some Embodiments

[0086] Figure 6 An operational flow is illustrated to implement adaptive branch prediction throttling according to some embodiments. The operations in method 600 are performed by adaptive speculation limit (ASL) logic 150 of the processor discussed herein.

[0087] At reference 602, a determination is made based on looking up a branch confidence data structure whether a current branch in the execution of a single thread is a low confidence branch in which the branch predictor of the processor has a low level of certainty that the outcome of the current branch is correctly predicted. At reference 604, the branch misprediction rate, microinstruction cost rate, and cache miss rate of the single thread are compared to their corresponding thresholds. At reference 606, based on a determination that the current branch is a low confidence branch and one or more thresholds of the branch misprediction rate, microinstruction cost rate, and cache miss rate for the single thread are exceeded, branch prediction of the current branch is throttled.

[0088] In some embodiments, when the processor performs branch prediction, a lookup of a branch confidence data structure occurs. Branch confidence data structure 192 is an example of such a data structure.

[0089] In some embodiments, the branch confidence data structure includes an entry for the current branch, and the entry includes an identifier of the branch and an indication of low confidence. For example, the tag and is_low_confidence in the BCDS entry 202 are the identifier of the branch and the indication of low confidence, respectively.

[0090] In some embodiments, the entry further includes one or more of the following: the number of branch predictions that have been completed for the current branch, the count of mispredictions or successful predictions within the branch prediction. For example, Pred_count and Mispred_count are the number of branch predictions that have been completed for the current branch and the count of mispredictions, respectively.

[0091] In some embodiments, throttling branch prediction includes skipping branch prediction for a number of clock cycles.An example of skipping is shown with M (active cycles) and N (inactive cycles) being set to 1 and 3, respectively.

[0092] In some embodiments, the level at which branch prediction is throttled is based on the performance state of the processor. In some embodiments, a performance state of the processor lower than a default performance state corresponds to a throttling level higher than the default throttling level; and a performance state of the processor higher than the default performance state corresponds to a throttling level lower than the default throttling level. For example, the higher the P-state, the lower the level of throttling to be performed, and in conservative throttling, throttling may be set / adjusted to M:N=1:1 instead of M:N=1:3.

[0093] In some embodiments, the throttling level is determined based on a mapping between the performance state of the processor and the throttling level. Figure 3 Explain in detail.

[0094] In some embodiments, a branch misprediction rate (eg, the misprediction rate discussed herein above) for a single thread is determined based on the number of mispredicted micro-ops per number of clock cycles.

[0095] In some embodiments, a microinstruction consumption rate (eg, consumption) for a single thread is determined based on a ratio of micro-ops allocated and micro-ops retired.

[0096] In some embodiments, the cache miss rate for a single thread is determined based on a number of cache misses per number of instructions.

[0097] In some embodiments, the method further includes, at numeral 610 , removing the branch prediction throttle once thread execution of the single thread is no longer in a low confidence branch.

[0098] Note that the operations described may be implemented in a single ASL logic, or in separate circuitry of distributed ASL logic. For example, one circuitry may perform the operations at reference 602 and 604, while another circuitry performs the operations at reference 606 / 608. In other embodiments, alternative distributions of operations between circuitry are applicable.

[0099] Through these embodiments, Adaptive Speculative Limiting (ASL) provides significant improvements in processor energy efficiency. By adapting speculation control based on real-time program behavior and / or performance / power tradeoffs, ASL provides lower electricity bills for processors used on servers (e.g., in data centers) and higher battery life for processors used on mobile devices. As new processors are built with deeper and wider pipelines to improve single-threaded performance, ASL becomes increasingly valuable.

[0100] Adaptive Speculative Limit (ASL) may be used in various execution circuitry processor architectures and systems, and some examples thereof are provided below. Exemplary Execution Unit Circuitry

[0101] Figure 7 An example of execution unit circuitry is shown, such as Figure 5(One or more) execution unit circuit systems 562. As shown, (one or more) execution unit circuit systems 562 may include one or more ALU circuits 701, optional vector / single instruction multiple data (SIMD) circuits 703, load / store circuits 705, branch / jump circuits 707 and / or floating-point unit (FPU) circuits 709. ALU circuits 701 perform integer arithmetic and / or Boolean operations. Vector / SIMD circuits 703 perform vector / SIMD operations on packed data (e.g., SIMD / vector registers). Load / store circuits 705 execute load and store instructions to load data from memory into registers, or store data from registers to memory. Load / store circuits 705 may also generate addresses. Branch / jump circuits 707 cause branches or jumps to a certain memory address depending on the instruction. FPU circuits 709 perform floating-point arithmetic. The width of (one or more) execution unit circuit systems 562 varies depending on the example, and may be in the range of 16 bits to 1024 bits, for example. In some examples, two or more smaller execution units are logically combined to form a larger execution unit (e.g., two 128-bit execution units are logically combined to form a 256-bit execution unit). Note that in some embodiments, ASL logic 150 is implemented within execution unit circuitry(s) 526. Example System

[0102] Figure 8 An example of a computing system is illustrated. A multiprocessor system 800 is a system provided with an interface and includes a plurality of processors or cores, including a first processor 870 and a second processor 880 coupled via an interface 850 such as a point-to-point (PP) interconnect, a fabric, and / or a bus. In some examples, the first processor 870 and the second processor 880 are isomorphic. In some examples, the first processor 870 and the second processor 880 are heterogeneous. Although the example system 800 is shown as having two processors, the system can have three or more processors, or can be a single processor system. In some examples, the computing system is a system on a chip (SoC).

[0103] Processor 870 and processor 880 are shown as including integrated memory controller (IMC) circuit systems 872 and 882, respectively. Processor 870 also includes interface circuit systems 876 and 878; similarly, second processor 880 includes interface circuit systems 886 and 888. Processors 870, 880 can exchange information via interface 850 using interface circuits 878, 888. IMCs 872 and 882 couple processors 870, 880 to respective memories, namely memory 832 and memory 834, which can be portions of main memory locally attached to the respective processors.

[0104] Processors 870, 880 can each use interface circuits 876, 894, 886, 898 to exchange information with a network interface (NW I / F) 890 via separate interfaces 852, 854. The network interface 890 (e.g., one or more of an interconnect, bus, and / or fabric, and in some examples, a chipset) can optionally exchange information with a coprocessor 838 via interface circuit 892. In some examples, the coprocessor 838 is a special-purpose processor such as, for example, a high-throughput processor, a network or communication processor, a compression engine, a graphics processor, a general-purpose graphics processing unit (GPGPU), a neural-network processing unit (NPU), an embedded processor, and the like.

[0105] A shared cache (not shown) may be included in either processor 870, 880, or external to both processors but connected to these processors via an interface (such as, the PP interconnect) so that if the processor is placed in a low power mode, local cache information of either or both processors can be stored in the shared cache.

[0106] The network interface 890 can be coupled to the first interface 816 via the interface circuit 896. In some examples, the first interface 816 can be an interface such as a peripheral component interconnect (PCI) interconnect, a PCI express interconnect, or another I / O interconnect. In some examples, the first interface 816 is coupled to a power control unit (PCU) 817, which may include circuitry, software, and / or firmware for performing power management operations related to the processors 870, 880, and / or the coprocessor 838. The PCU 817 provides control information to a voltage regulator (not shown) so that the voltage regulator generates an appropriate regulated voltage. The PCU 817 also provides control information to control the generated operating voltage. In various examples, the PCU 817 may include various power management logic units (circuitry) for performing hardware-based power management. Such power management may be entirely controlled by the processor (e.g., by various processor hardware, and which may be triggered by workload and / or power, thermal constraints, or other processor constraints), and / or power management may be performed in response to an external source (such as a platform or power management source or system software).

[0107] PCU 817 is illustrated as existing as logic separate from processor 870 and / or processor 880. In other cases, PCU 817 may be executed on a given one or more cores (not shown) of the cores of processor 870 or 880. In some cases, PCU 817 may be implemented as a (dedicated or general purpose) microcontroller or other control logic configured to execute its own dedicated power management code (sometimes referred to as P-code). In still other examples, the power management operations to be performed by PCU 817 may be implemented outside the processor, such as by means of a separate power management integrated circuit (PMIC) or another component external to the processor. In still other examples, the power management operations to be performed by PCU 817 may be implemented within BIOS or other system software.

[0108] Various I / O devices 814 may be coupled to the first interface 816 along with a bus bridge 818, which couples the first interface 816 to a second interface 820. In some examples, one or more additional processors 815 (such as a coprocessor, a high-throughput many integrated core (MIC) processor, a GPGPU, an accelerator (such as a graphics accelerator or a digital signal processing (DSP) unit), a field programmable gate array (FPGA), or any other processor) are coupled to the first interface 816. In some examples, the second interface 820 may be a low pin count (LPC) interface. Various devices may be coupled to the second interface 820, including, for example, a keyboard and / or mouse 822, a communication device 827, and a storage circuit system 828. The storage circuit system 828 may be one or more non-transitory machine-readable storage media as described below, such as a disk drive or other mass storage device, which in some examples may include instructions / codes and data 830 and may implement the storage device 103. Further, audio I / O 824 may be coupled to second interface 820. Note that other architectures are possible in addition to the point-to-point architecture described above. For example, a system such as multiprocessor system 800 may implement a multi-drop interface or other such architectures instead of a point-to-point architecture. Exemplary Core Architectures, Processors, and Computer Architectures

[0109] Processor cores can be implemented in different ways, for different purposes, and in different processors. For example, implementations of such cores may include: 1) a general-purpose in-order core intended for general-purpose computing; 2) a high-performance general-purpose out-of-order core intended for general-purpose computing; 3) a special-purpose core intended primarily for graphics and / or scientific (throughput) computing. Different processor implementations may include: 1) a CPU that includes one or more general-purpose in-order cores intended for general-purpose computing and / or one or more general-purpose out-of-order cores intended for general-purpose computing; and 2) a coprocessor that includes one or more special-purpose cores intended primarily for graphics and / or scientific (throughput) computing. Such different processors lead to different computer system architectures, which may include: 1) a coprocessor on a separate chip from the CPU; 2) a coprocessor in the same package as the CPU but on a separate die; 3) a coprocessor on the same die as the CPU (in which case such a coprocessor is sometimes referred to as dedicated logic or as a dedicated core, such as integrated graphics and / or scientific (throughput) logic); and 4) a system on a chip (SoC), which may include the described CPU (sometimes referred to as application core(s) or application processor(s), the coprocessor described above, and additional functionality on the same die. An example core architecture is described next, followed by a description of an example processor and computer architecture.

[0110] Fig. 9 A block diagram of an example processor and / or SoC 900 that may have one or more cores and an integrated memory controller is illustrated. The solid lined boxes illustrate the processor 900 having a single core 902(A), a system agent unit circuitry 910, and a collection of one or more interface controller unit circuitry 916, while the optional addition of dashed lined boxes illustrates an alternative processor 900 having multiple cores 902(A)-902(N), a collection of one or more integrated memory controller units 914 in the system agent unit circuitry 910, and dedicated logic 908 and a collection of one or more interface controller unit circuitry 916. Note that the processor 900 may be Figure 8 one of the processors 870 or 880, or the coprocessors 838 or 815.

[0111] Thus, different implementations of processor 900 may include: 1) a CPU, where the dedicated logic 908 is integrated graphics and / or scientific (throughput) logic (which may include one or more cores, not shown), and cores 902(A)-902(N) are one or more general purpose cores (e.g., general purpose in-order cores, general purpose out-of-order cores, a combination of the two); 2) a coprocessor, where cores 902(A)-902(N) are a large number of dedicated cores intended primarily for graphics and / or scientific (throughput); and 3) a coprocessor, where cores 902(A)-902(N) are a large number of general purpose in-order cores. Thus, processor 900 may be a general purpose processor, a coprocessor, or a dedicated processor, such as, for example, a network or communications processor, a compression engine, a graphics processor, a GPGPU (general purpose graphics processing unit), a high throughput many integrated core (MIC) coprocessor (including 30 or more cores), an embedded processor, etc. The processor may be implemented on one or more chips. Processor 900 may be part of and / or implemented on one or more substrates using any of a variety of process technologies, such as, for example, complementary metal-oxide-semiconductor (CMOS), bipolar CMOS (BiCMOS), P-type metal oxide semiconductor (PMOS), or N-type metal oxide semiconductor (NMOS).

[0112] The memory hierarchy includes one or more levels of cache unit circuitry 904(A)-904(N) within the cores 902(A)-902(N), a set of one or more shared cache unit circuitry 906, and external memory (not shown) coupled to a set of (one or more) integrated memory controller unit circuitry 914. The set of one or more shared cache unit circuitry 906 may include one or more intermediate levels of cache (such as level 2 (L2), level 3 (L3), level 4 (L4)) or other levels of cache (such as last level cache (LLC)) and / or combinations thereof. Although in some examples, an interface network circuitry 912 (e.g., a ring interconnect) provides an interface to dedicated logic 908 (e.g., integrated graphics logic), the set of (one or more) shared cache unit circuitry 906, and the system agent unit circuitry 910, alternative examples use any number of well-known techniques for providing interfaces to such units. In some examples, coherency is maintained between the shared cache unit circuitry 906 and one or more of the cores 902(A)-902(N). In some examples, the interface controller unit circuitry 916 couples the cores 902(A)-902(N) to one or more other devices 918, such as one or more I / O devices, storage, one or more communication devices (e.g., a wireless network, a wired network, etc.), etc.

[0113] In some examples, one or more of the cores 902(A)-902(N) may be capable of multithreading. System agent unit circuitry 910 includes those components that coordinate and operate the cores 902(A)-902(N). System agent unit circuitry 910 may include, for example, power control unit (PCU) circuitry and / or display unit circuitry (not shown). The PCU may be, or may include, the logic and components required to regulate the power state of the cores 902(A)-902(N) and / or the dedicated logic 908 (e.g., integrated graphics logic). The display unit circuitry is used to drive one or more externally connected displays.

[0114] The cores 902(A)-902(N) may be homogeneous in terms of instruction set architecture (ISA). Alternatively, the cores 902(A)-902(N) may be heterogeneous in terms of ISA; that is, a subset of the cores 902(A)-902(N) may be capable of executing an ISA, while other cores may only be capable of executing a subset of that ISA or another ISA.

[0115] Fig.10 1 is a block diagram illustrating a computing system 1000 configured to implement one or more aspects of the examples described herein. The computing system 1000 includes a processing subsystem 1001 having one or more processors 1002 and a system memory 1004 communicating via an interconnect path, which may include a memory hub 1005. The memory hub 1005 may be a separate component within a chipset component, or may be integrated within one or more processors 1002. The memory hub 1005 is coupled to an I / O subsystem 1011 via a communication link 1006. The I / O subsystem 1011 includes an I / O hub 1007, which may enable the computing system 1000 to receive input from one or more input devices 1008. Additionally, the I / O hub 1007 may enable a display controller (which may be included in one or more processors 1002) to provide output to one or more display devices 1010A. In some examples, one or more display devices 1010A coupled to I / O hub 1007 may include local, internal, or embedded display devices.

[0116] The processing subsystem 1001 includes, for example, one or more parallel processors 1012 coupled to a memory hub 1005 via a bus or other communication link 1013. The communication link 1013 can be one of any number of standard-based communication link technologies or protocols, such as but not limited to PCI Express, or can be a vendor-specific communication interface or communication fabric. The one or more parallel processors 1012 can form a parallel or vector processing system in a computational concentration that can include a large number of processing cores and / or processing clusters, such as, for example, many integrated core (MIC) processors. For example, the one or more parallel processors 1012 form a graphics processing subsystem that can output pixels to one of the one or more display devices 1010A coupled via the I / O hub 1007. The one or more parallel processors 1012 may also include a display controller and a display interface (not shown) for enabling direct connection to one or more display devices 1010B.

[0117] Within the I / O subsystem 1011, a system storage unit 1014 may be connected to the I / O hub 1007 to provide a storage mechanism for the computing system 1000. An I / O switch 1016 may be used to provide an interface mechanism to enable connections between the I / O hub 1007 and other components, such as a network adapter 1018 and / or a wireless network adapter 1019 that may be integrated into the platform, and various other devices that may be added via one or more plug-in devices 1020. The plug-in device(s) 1020 may also include, for example, one or more external graphics processor devices, graphics cards, and / or computing accelerators. The network adapter 1018 may be an Ethernet adapter or another wired network adapter. The wireless network adapter 1019 may include one or more of the following: Wi-Fi, Bluetooth, near field communication (NFC), or other network devices including one or more wireless radio devices.

[0118] The computing system 1000 may include other components not explicitly shown, including USB or other port connections, optical storage drives, video capture devices, etc., which may also be connected to the I / O hub 1007. Fig.10 The communication paths for interconnecting the various components in the system may be implemented using any suitable protocol, such as a PCI (Peripheral Component Interconnect) based protocol (e.g., PCI Express), or any other bus or point-to-point communication interface and / or protocol(s), such as NVLink high-speed interconnect, Compute Express Link, or the like. TM (ComputeExpress Link TM , CXL TM) (e.g., CXL.mem), Infinity Fabric (IF), Ethernet (IEEE802.3), remote direct memory access (RDMA), InfiniBand, Internet Wide Area RDMA Protocol (iWARP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), quick UDP Internet Connections (QUIC), RDMA over Converged Ethernet (RoCE), Intel Quick Path Interconnect (QPI), Intel Ultra Path Interconnect (UPI), Intel On-Chip System Fabric (IOSF), Omnipath, HyperTransport, Advanced Microcontroller Bus Architecture (AMBA) interconnect, OpenCAPI, Gen-Z, Cache Coherent Interconnect for Accelerators The data may be copied or stored to a virtualized storage node using protocols such as NVMe (non-volatile memory express over Fabrics, NVME-oF) or NVMe.

[0119] One or more parallel processors 1012 may include circuitry optimized for graphics and video processing (including, for example, video output circuitry) and constitute a graphics processing unit (GPU). Alternatively or additionally, as described in more detail herein, one or more parallel processors 1012 may include circuitry optimized for general-purpose processing while retaining the underlying computing architecture. The components of the computing system 1000 may be integrated on a single integrated circuit with one or more other system elements. For example, one or more parallel processors 1012, a memory hub 1005, (one or more) processors 1008, and an I / O hub 1007 may be integrated into a system-on-chip (SoC) integrated circuit. Alternatively, the components of the computing system 1000 may be integrated into a single package to form a system-in-package (SIP) configuration. In some examples, at least a portion of the components of the computing system 1000 may be integrated into a multi-chip module (MCM), which may be interconnected with other multi-chip modules into a modular computing system.

[0120] It will be appreciated that the computing system 1000 shown herein is illustrative, and variations and modifications are possible. The connection topology, including the number and arrangement of bridges, the number of (one or more) processors 1002, and the number of (one or more) parallel processors 1012, may be modified as desired. For example, the system memory 1004 may be connected to the (one or more) processors 1002 directly rather than through a bridge, while other devices communicate with the system memory 1004 via the memory hub 1005 and the (one or more) processors 1002. In other alternative topologies, the (one or more) parallel processors are connected to the I / O hub 1007 or directly to one of the one or more processors 1002, rather than to the memory hub 1005. In other examples, the I / O hub 1007 and the memory hub 1005 may be integrated into a single chip. It is also possible for two or more sets of processors 1002 to be attached via multiple slots, which may be coupled to two or more instances of (one or more) parallel processors 1012.

[0121] Some of the specific components shown herein are optional and may not be included in all implementations of computing system 1000. For example, any number of plug-in cards or peripherals may be supported, or some components may be eliminated. In addition, some architectures may be specific to the Fig.10 Different terminology is used for components similar to those illustrated in FIG. 1004. For example, memory hub 1005 may be referred to as a north bridge in some architectures, while I / O hub 1007 may be referred to as a south bridge.

[0122] Fig.11A An example of a parallel processor 1100 is shown. The parallel processor 1100 may be a GPU, a GPGPU, etc. as described herein. The various components of the parallel processor 1100 may be implemented using one or more integrated circuit devices, such as a programmable processor, an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). The illustrated parallel processor 1100 may be Fig.10 One or more of the parallel processor(s) 1012 shown in FIG.

[0123] The parallel processor 1100 includes a parallel processing unit 1102. The parallel processing unit includes an I / O unit 1104 that enables communication with other devices, including other instances of the parallel processing unit 1102. The I / O unit 1104 can be directly connected to other devices. For example, the I / O unit 1104 is connected to other devices via the use of a hub or switch interface (such as, a memory hub 1005). The connection between the memory hub 1005 and the I / O unit 1104 forms a communication link 1013. Within the parallel processing unit 1102, the I / O unit 1104 is connected to a host interface 1106 and a memory cross switch 1116, wherein the host interface 1106 receives commands related to performing processing operations and the memory cross switch 1116 receives commands related to performing memory operations.

[0124] When the host interface 1106 receives the command buffer via the I / O unit 1104, the host interface 1106 can direct the work operations for executing those commands to the front end 1108. In some examples, the front end 1108 is coupled with a scheduler 1110, which is configured to distribute commands or other work items to the processing cluster array 1112. The scheduler 1110 ensures that the processing cluster array 1112 is properly configured and in a valid state before the task is distributed to the processing clusters in the processing cluster array 1112. The scheduler 1110 can be implemented via firmware logic executed on a microcontroller. The microcontroller-implemented scheduler 1110 can be configured to perform complex scheduling and work distribution operations at coarse and fine granularity, thereby achieving fast preemption and context switching of threads executed on the processing cluster array 1112. Preferably, the host software can confirm the workload for scheduling on the processing cluster array 1112 via one of a plurality of graphics processing doorbells. In other examples, polling for new workloads or interrupts may be used to identify or indicate the availability of work to be performed.The workload may then be automatically distributed across the processing cluster array 1112 by the scheduler 1110 logic within the scheduler microcontroller.

[0125] The processing cluster array 1112 may include up to "N" processing clusters (e.g., cluster 1114A, cluster 1114B to cluster 1114N). Each cluster 1114A-1114N in the processing cluster array 1112 may execute a large number of concurrent threads. The scheduler 1110 may use various scheduling and / or work distribution algorithms to assign work to the clusters 1114A-1114N in the processing cluster array 1112, which may vary depending on the workload generated for each type of program or calculation. Scheduling may be handled dynamically by the scheduler 1110, or may be partially assisted by compiler logic during the compilation of program logic configured to be executed by the processing cluster array 1112. Optionally, different clusters 1114A-1114N in the processing cluster array 1112 may be assigned to process different types of programs or to perform different types of calculations.

[0126] Processing cluster array 1112 may be configured to perform various types of parallel processing operations. For example, processing cluster array 1112 is configured to perform general-purpose parallel computing operations. For example, processing cluster array 1112 may include logic for performing processing tasks including filtering of video and / or audio data, performing modeling operations including physical operations, and performing data transformations.

[0127] The processing cluster array 1112 is configured to perform parallel graphics processing operations. In such examples where the parallel processor 1100 is configured to perform graphics processing operations, the processing cluster array 1112 may include additional logic for supporting the execution of such graphics processing operations, including but not limited to texture sampling logic for performing texture operations, and tessellation logic and other vertex processing logic. In addition, the processing cluster array 1112 may be configured to execute graphics processing related shader programs, such as but not limited to vertex shaders, tessellation shaders, geometry shaders, and pixel shaders. The parallel processing unit 1102 can transfer data from the system memory via the I / O unit 1104 for processing. During processing, the transferred data can be stored in an on-chip memory (e.g., parallel processor memory 1122) during processing, and then the data is written back to the system memory.

[0128] In examples where parallel processing units 1102 are used to perform graphics processing, scheduler 1110 may be configured to divide the processing workload into tasks of approximately equal size to better enable distribution of graphics processing operations to multiple clusters 1114A-1114N in processing cluster array 1112. In some of these examples, portions of processing cluster array 1112 may be configured to perform different types of processing. For example, a first portion may be configured to perform vertex shading and topology generation, a second portion may be configured to perform tessellation and geometry shading, and a third portion may be configured to perform pixel shading or other screen space operations to produce a rendered image for display. Intermediate data generated by one or more of clusters 1114A-1114N may be stored in a buffer to allow the intermediate data to be transferred between clusters 1114A-1114N for further processing.

[0129] During operation, the processing cluster array 1112 may receive processing tasks to be executed via the scheduler 1110, which receives commands defining the processing tasks from the front end 1108. For graphics processing operations, the processing tasks may include data to be processed, such as surface (patch) data, primitive data, vertex data, and / or pixel data, and state parameters and commands defining how the data is to be processed (e.g., what program is to be executed). The scheduler 1110 may be configured to fetch an index corresponding to the task, or may receive the index from the front end 1108. The front end 1108 may be configured to ensure that the processing cluster array 1112 is configured to a valid state before a workload specified by an incoming command buffer (e.g., a batch buffer, a push buffer, etc.) is initiated.

[0130] Each of the one or more instances of the parallel processing unit 1102 may be coupled to a parallel processor memory 1122. The parallel processor memory 1122 may be accessed via a memory crossbar 1116, which may receive memory requests from the processing cluster array 1112 and the I / O unit 1104. The memory crossbar 1116 may access the parallel processor memory 1122 via a memory interface 1118. The memory interface 1118 may include a plurality of partition units (e.g., partition unit 1120A, partition unit 1120B, up to partition unit 1120N), each of which may be coupled to a portion (e.g., memory unit) of the parallel processor memory 1122. The number of partition units 1120A-1120N may be configured to be equal to the number of memory units, such that the first partition unit 1120A has a corresponding first memory unit 1124A, the second partition unit 1120B has a corresponding second memory unit 1124B, and the Nth partition unit 1120N has a corresponding Nth memory unit 1124N. In other examples, the number of partition units 1120A-1120N may not be equal to the number of memory devices.

[0131] The memory units 1124A-1124N may include various types of memory devices, including dynamic random-access memory (DRAM) or graphics random access memory, such as synchronous graphics random access memory (SGRAM), including graphics double data rate (GDDR) memory. Optionally, the memory units 1124A-1124N may also include 3D stacked memory, including but not limited to high bandwidth memory (HBM). Those skilled in the art will appreciate that the specific implementation of the memory units 1124A-1124N may vary and may be selected from one of a variety of conventional designs. Rendering targets such as frame buffers or texture maps may be stored across the memory units 1124A-1124N, allowing the partition units 1120A-1120N to write portions of each rendering target in parallel to efficiently use the available bandwidth of the parallel processor memory 1122. In some examples, local instances of parallel processor memory 1122 may be eliminated in favor of a unified memory design utilizing system memory in conjunction with local cache memory.

[0132] Optionally, any of the clusters 1114A-1114N in the processing cluster array 1112 has the capability to process data to be written to any of the memory units 1124A-1124N within the parallel processor memory 1122. The memory crossbar 1116 may be configured to transmit the output of each cluster 1114A-1114N to any partition unit 1120A-1120N or to another cluster 1114A-1114N, which may perform additional processing operations on the output. Each cluster 1114A-1114N may communicate with a memory interface 1118 via the memory crossbar 1116 to read from or write to various external memory devices. In one example with a memory crossbar switch 1116, the memory crossbar switch 1116 has connections to a memory interface 1118 to communicate with the I / O unit 1104 and has connections to a local instance of a parallel processor memory 1122, thereby enabling processing units within different processing clusters 1114A-1114N to communicate with system memory or other memory that is not local to the parallel processing unit 1102. In general, the memory crossbar switch 1116 may be able to separate traffic flows between the clusters 1114A-1114N and the partition units 1120A-1120N using virtual channels, for example.

[0133] Although a single instance of parallel processing unit 1102 is illustrated within parallel processor 1100, any number of instances of parallel processing unit 1102 may be included. For example, multiple instances of parallel processing unit 1102 may be provided on a single plug-in card, or multiple plug-in cards may be interconnected. For example, parallel processor 1100 may be a plug-in device, such as, Fig.10 Plug-in device 1020, which can be a graphics card (such as, a discrete graphics card including one or more GPUs, one or more memory devices, and device-to-device or network or structure interfaces). Different instances of parallel processing unit 1102 can be configured to interoperate even if different instances have different numbers of processing cores, different amounts of local parallel processor memory, and / or other configuration differences. Optionally, some instances of parallel processing unit 1102 may include higher precision floating point units relative to other instances. Systems including one or more instances of parallel processing unit 1102 or parallel processor 1100 can be implemented in various configurations and form factors, including but not limited to desktop computers, laptop computers, or handheld personal computers, servers, workstations, game consoles, and / or embedded systems. The orchestrator can use one or more of the following to form a composite node for workload execution: decomposed processor resources, cache resources, memory resources, storage resources, and networking resources.

[0134] In some examples, the parallel processing unit 1102 can be partitioned into multiple instances. Those multiple instances can be configured to execute workloads associated with different clients in an isolated manner, so that a predetermined quality of service is provided for each client. For example, each cluster 1114A-1114N can be partitioned and isolated from other clusters, thereby allowing the processing cluster array 1112 to be divided into multiple computing partitions or instances. In such a configuration, the workload executed on the isolated partition is protected from errors or errors associated with different workloads executed on different partitions. Partition units 1120A-1120N can be configured to enable dedicated paths and / or isolated paths to the memory of the cluster 1114A-1114N associated with the corresponding computing partition. This data path isolation enables the computing resources within the partition to communicate with one or more assigned memory units 1124A-1124N without being disturbed by the activities of other partitions.

[0135] Fig. 11B is a block diagram of the partition unit 1120. The partition unit 1120 may be Fig.11A 1120N. As illustrated, partition unit 1120 includes L2 cache 1121, frame buffer interface 1125, and ROP 1126 (raster operation unit). L2 cache 1121 is a read / write cache configured to perform load and store operations received from memory crossbar switch 1116 and ROP 1126. Read misses and urgent writeback requests are output by L2 cache 1121 to frame buffer interface 1125 for processing. Updates may also be sent to the frame buffer via frame buffer interface 1125 for processing. In some examples, frame buffer interface 1125 communicates with memory units in parallel processor memory (such as, for example, within parallel processor memory 1122). Fig.11A Partition unit 1120 may also additionally or alternatively interface with one of the memory units in the parallel processor memory via a memory controller (not shown).

[0136] In graphics applications, ROP 1126 is a processing unit that performs raster operations such as stenciling, z-testing, blending, etc. ROP 1126 then outputs processed graphics data, which is stored in graphics memory. In some examples, ROP 1126 includes or is coupled to a codec (CODEC) 1127 that includes compression logic for compressing depth or color data written to memory or L2 cache 1121 and decompressing depth or color data read from memory or L2 cache 1121. The compression logic may be lossless compression logic that utilizes one or more of a variety of compression algorithms. The type of compression performed by CODEC 1127 may vary based on the statistical characteristics of the data to be compressed. For example, in some examples, delta color compression is performed on depth and color data on a slice-by-slice basis. In some examples, CODEC 1127 includes compression and decompression logic that can compress and decompress computational data associated with machine learning operations. CODEC 1127 can, for example, compress sparse matrix data for sparse machine learning operations. CODEC 1127 can also compress sparse matrix data encoded in a sparse matrix format (e.g., coordinate list encoding (COO), compressed sparse row (CSR), compressed sparse column (CSC), etc.) to generate compressed and encoded sparse matrix data. The compressed and encoded sparse matrix data can be decompressed and / or decoded before being processed by a processing element, or the processing element can be configured to consume compressed, encoded, or compressed and encoded data for processing.

[0137] ROP 1126 may be included in each processing cluster (e.g., Fig.11A 114A-1114N) rather than being included in partition unit 1120. In such examples, read and write requests for pixel data rather than pixel fragment data are transmitted through memory crossbar switch 1116. The processed graphics data may be displayed on a display device such as a Fig.10 1010A-1010B), is routed for further processing by (one or more) processors 1002, or is routed for processing by Fig.11A One of the processing entities within the parallel processor 1100 further processes.

[0138] Fig. 11Cis a block diagram of a processing cluster 1114 within a parallel processing unit. For example, a processing cluster is Fig.11A An instance of a processing cluster in the processing clusters 1114A-1114N. Processing cluster 1114 may be configured to execute many threads in parallel, wherein the term "thread" refers to an instance of a specific program executed on a set of specific input data. Optionally, a single-instruction, multiple-data (SIMD) instruction issuance technique may be used to support the parallel execution of a large number of threads without providing multiple independent instruction units. Alternatively, a single-instruction, multiple-thread (SIMT) technique may be used to support the parallel execution of a large number of threads that are generally synchronized using a common instruction unit configured to issue instructions to a set of processing engines within each processing cluster in the processing cluster. Unlike the SIMD execution mechanism in which all processing engines typically execute the same instruction, SIMT execution allows different threads to more easily follow the divergent execution path through a given thread program. Those skilled in the art will appreciate that the SIMD processing mechanism represents a functional subset of the SIMT processing mechanism.

[0139] The operation of the processing cluster 1114 may be controlled via a pipeline manager 1132 that distributes processing tasks to SIMT parallel processors. The pipeline manager 1132 receives Fig.11A The graphics multiprocessor 1134 is an exemplary embodiment of a SIMT parallel processor. However, various types of SIMT parallel processors of different architectures may be included in the processing cluster 1114. One or more instances of the graphics multiprocessor 1134 may be included in the processing cluster 1114. The graphics multiprocessor 1134 may process data, and the data crossbar 1140 may be used to distribute the processed data to one of a plurality of possible destinations, including other shader units. The pipeline manager 1132 may facilitate the distribution of processed data by specifying a destination for the processed data to be distributed via the data crossbar 1140.

[0140] Each graphics multiprocessor 1134 within a processing cluster 1114 may include an identical set of function execution logic (e.g., arithmetic logic units, load-store units, etc.). The function execution logic may be configured in a pipelined manner in which new instructions may be issued before previous instructions are completed. The function execution logic supports a variety of operations, including integer and floating point arithmetic, comparison operations, Boolean operations, bit shifts, and calculation of various algebraic functions. The same functional unit hardware may be utilized to perform different operations, and any combination of functional units may exist.

[0141] The instructions transmitted to the processing cluster 1114 constitute threads. A collection of threads executed across a collection of parallel processing engines is a thread group. A thread group executes the same program on different input data. Each thread within a thread group may be assigned to a different processing engine within the graphics multiprocessor 1134. A thread group may include fewer threads than the number of processing engines within the graphics multiprocessor 1134. When a thread group includes fewer threads than the number of processing engines, one or more of the processing engines may be idle during the cycle during which the thread group is being processed. A thread group may also include more threads than the number of processing engines within the graphics multiprocessor 1134. When a thread group includes more threads than the number of processing engines within the graphics multiprocessor 1134, processing may be performed in consecutive clock cycles. Optionally, multiple thread groups may be executed concurrently on the graphics multiprocessor 1134.

[0142] The graphics multiprocessor 1134 may include internal cache memory to perform load and store operations. Optionally, the graphics multiprocessor 1134 may abandon the internal cache and use cache memory (e.g., first level (level 1, L1) cache 1148) within the processing cluster 1114. Each graphics multiprocessor 1134 also has a partition unit (e.g., Fig.11A 1120N), which are shared between all processing clusters 1114 and can be used to transfer data between threads. Graphics multiprocessor 1134 can also access off-chip global memory, which can include one or more of local parallel processor memory and / or system memory. Any memory external to parallel processing unit 1102 can be used as global memory. Embodiments in which processing cluster 1114 includes multiple instances of graphics multiprocessor 1134 can share common instructions and data, which can be stored in L1 cache 1148.

[0143] Each processing cluster 1114 may include an MMU 1145 (memory management unit) configured to map virtual addresses to physical addresses. In other examples, one or more instances of MMU 1145 may reside in Fig.11A 1148. The MMU 1145 includes a set of page table entries (PTEs) for mapping virtual addresses to physical addresses of the slice, and optionally includes a cache line index. The MMU 1145 may include an address translation lookaside buffer (TLB) or cache that may reside within the graphics multiprocessor 1134 or L1 cache 1148 of the processing cluster 1114. The physical addresses are processed to distribute surface data access locality, thereby allowing efficient request interleaving between partition units. The cache line index can be used to determine whether a request for a cache line is a hit or a miss.

[0144] In graphics and compute applications, the processing clusters 1114 may be configured such that each graphics multiprocessor 1134 is coupled to a texture unit 1136 for performing texture mapping operations, such as determining texture sample locations, reading texture data, and filtering texture data. Texture data is read from an internal texture L1 cache (not shown), or in some examples, from an L1 cache within the graphics multiprocessor 1134 and retrieved from an L2 cache, local parallel processor memory, or system memory as needed. Each graphics multiprocessor 1134 outputs processed tasks to a data crossbar 1140 to provide the processed tasks to another processing cluster 1114 for further processing, or to store the processed tasks in an L2 cache, local parallel processor memory, or system memory via a memory crossbar 1116. A pre-raster operations unit 1142 is configured to receive data from the graphics multiprocessor 1134, direct the data to ROP units, which may be coupled to partition units (e.g., Fig.11A The preROP 1142 unit may perform optimizations for color blending, organize pixel color data, and perform address translation.

[0145] It will be appreciated that the core architecture described herein is illustrative, and variations and modifications are possible. Any number of processing units (e.g., graphics multiprocessors 1134, texture units 1136, preROPs 1142, etc.) may be included within a processing cluster 1114. Further, while only one processing cluster 1114 is shown, a parallel processing unit as described herein may include any number of instances of a processing cluster 1114. Optionally, each processing cluster 1114 may be configured to operate independently of other processing clusters 1114 using separate and distinct processing units, L1 caches, L2 caches, etc.

[0146] Fig.11D An example of a graphics multiprocessor 1134 is shown, where the graphics multiprocessor 1134 is coupled to a pipeline manager 1132 of a processing cluster 1114. The graphics multiprocessor 1134 has an execution pipeline that includes, but is not limited to, an instruction cache 1152, an instruction unit 1154, an address mapping unit 1156, a register file 1158, one or more general purpose graphics processing unit (GPGPU) cores 1162, and one or more load / store units 1166. The GPGPU cores 1162 and the load / store units 1166 are coupled to a cache memory 1172 and a shared memory 1170 via a memory and cache interconnect 1168. The graphics multiprocessor 1134 may additionally include a tensor and / or ray tracing core 1163 that includes hardware logic for accelerating matrix and / or ray tracing operations.

[0147] The instruction cache 1152 may receive an instruction stream to be executed from the pipeline manager 1132. The instructions are cached in the instruction cache 1152 and dispatched for execution by the instruction unit 1154. The instruction unit 1154 may dispatch the instructions as a thread group (e.g., a unit group (warp)), wherein each thread in the thread group is assigned to a different execution unit within the GPGPU core 1162. The instruction may access any of the local address space, the shared address space, or the global address space by specifying an address within the unified address space. The address in the unified address space may be translated into different memory addresses that may be accessed by the load / store unit 1166 using the address mapping unit 1156.

[0148] The register file 1158 provides a collection of registers for the functional units of the graphics multiprocessor 1134. The register file 1158 provides temporary storage for operands of data paths connected to the functional units of the graphics multiprocessor 1134 (e.g., GPGPU core 1162, load / store unit 1166). The register file 1158 may be divided between each of the functional units so that each functional unit is assigned a dedicated portion of the register file 1158. For example, the register file 1158 may be divided between different groups of units executed by the graphics multiprocessor 1134.

[0149] The GPGPU cores 1162 may each include a floating point unit (FPU) and / or an integer arithmetic logic unit (ALU) for executing instructions of the graphics multiprocessor 1134. In some implementations, the GPGPU core 1162 may include hardware logic that may otherwise reside within the tensor and / or ray tracing core 1163. The GPGPU cores 1162 may be similar in architecture, or may be different in architecture. For example and in some examples, the first portion of the GPGPU core 1162 includes a single-precision FPU and an integer ALU, while the second portion of the GPGPU core includes a double-precision FPU. Optionally, the FPU may implement the IEEE 754-2008 standard for floating-point arithmetic, or enable variable-precision floating-point arithmetic. The graphics multiprocessor 1134 may additionally include one or more fixed-function or special-function units for performing specific functions, such as copying rectangles or pixel blending operations. One or more of the GPGPU cores in the GPGPU core may also include fixed-function or special-function logic.

[0150] GPGPU core 1162 may include SIMD logic capable of executing a single instruction on multiple sets of data. Optionally, GPGPU core 1162 may physically execute SIMD4, SIMD8 and SIMD16 instructions, and logically execute SIMD1, SIMD2 and SIMD32 instructions. SIMD instructions for GPGPU cores may be generated by a shader compiler at compile time, or automatically generated when executing a program written and compiled for a single program multiple data (SPMD) or SIMT architecture. Multiple threads of a program configured for a SIMT execution model may be executed via a single SIMD instruction. For example and in some examples, eight SIMT threads may be executed in parallel via a single SIMD8 logic unit, and these eight SIMT threads perform the same or similar operations.

[0151] The memory and cache interconnect 1168 is an interconnect network that connects each of the functional units of the graphics multiprocessor 1134 to the register file 1158 and to the shared memory 1170. For example, the memory and cache interconnect 1168 is a crossbar interconnect that allows the load / store unit 1166 to implement load and store operations between the shared memory 1170 and the register file 1158. The register file 1158 can operate at the same frequency as the GPGPU core 1162, so data transfers between the GPGPU core 1162 and the register file 1158 are very low latency. The shared memory 1170 can be used to enable communication between threads executed on the functional units within the graphics multiprocessor 1134. The cache memory 1172 can be used as a data cache, for example, to cache texture data passed between the functional unit and the texture unit 1136. The shared memory 1170 can also be used as a managed cached program. The shared memory 1170 and the cache memory 1172 can be coupled with the data crossbar 1140 to enable communication with other components of the processing cluster. Threads executing on GPGPU core 1162 can programmatically store data in shared memory in addition to automatically cached data stored in cache memory 1172.

[0152] Figure 12A-12C Additional graphics multiprocessors are illustrated according to some examples. Figure 12A-12B Graphics multiprocessors 1225 and 1250 are shown. Graphics multiprocessors 1225 and 1250 are Fig. 11C 1134 and may be used in place of one of those graphics multiprocessors. Therefore, any disclosure of features herein in conjunction with graphics multiprocessor 1134 also discloses corresponding combinations with graphics multiprocessors 1225, 1250, but is not limited thereto. Fig. 12C A graphics processing unit (GPU) 1280 is illustrated, which includes a collection of dedicated graphics processing resources arranged as multi-core groups 1265A-1265N, which correspond to graphics multiprocessors 1225, 1250. The illustrated graphics multiprocessors 1225, 1250 and multi-core groups 1265A-1265N may be streaming multiprocessors (SMs) capable of executing a large number of execution threads simultaneously.

[0153] Fig. 12A The graphics multiprocessor 1225 includes relative Fig.11DThe graphics multiprocessor 1225 may include multiple additional instances of execution resource units of the graphics multiprocessor 1134. For example, the graphics multiprocessor 1225 may include multiple instances of instruction units 1232A-1232B, register files 1234A-1234B, and (one or more) texture units 1244A-1244B. The graphics multiprocessor 1225 also includes multiple sets of graphics or compute execution units (e.g., GPGPU cores 1236A-1236B, tensor cores 1237A-1237B, ray tracing cores 1238A-1238B) and multiple sets of load / store units 1240A-1240B. The execution resource units have a common instruction cache 1230, texture and / or data cache memory 1242, and shared memory 1246.

[0154] The components may communicate via an interconnect structure 1227. The interconnect structure 1227 may include one or more crossbar switches to enable communication between the components of the graphics multiprocessor 1225. The interconnect structure 1227 is a separate, high-speed network fabric layer on which each component of the graphics multiprocessor 1225 is stacked. The components of the graphics multiprocessor 1225 communicate with remote components via the interconnect structure 1227. For example, the cores 1236A-1236B, 1237A-1237B, and 1238A-1238B may each communicate with the shared memory 1246 via the interconnect structure 1227. The interconnect structure 1227 may arbitrate communications within the graphics multiprocessor 1225 to ensure fair bandwidth allocation between components.

[0155] Fig. 12B The graphics multiprocessor 1250 includes a plurality of execution resource sets 1256A-1256D, wherein Fig.11D and Fig. 12A As shown in FIG. 1 , each set of execution resources includes multiple instruction units, register files, GPGPU cores, and load storage units. Execution resources 1256A-1256D can work in conjunction with (one or more) texture units 1260A-1260D for texture operations while sharing instruction cache 1254 and shared memory 1253. For example, execution resources 1256A-1256D can share multiple instances of instruction cache 1254 and shared memory 1253 and texture and / or data cache memory 1258A-1258B. Each component can be connected to the CPU via the CPU. Fig. 12A The interconnect structure 1227 communicates with a similar interconnect structure 1252 .

[0156] Those skilled in the art will understand that Figure 1 , Figures 11A-11D as well as Figure 12A-12BThe architecture described in is illustrative and not limiting in scope of the current examples. Thus, the techniques described herein may be implemented on any appropriately configured processing unit without departing from the scope of the examples described herein, including but not limited to: one or more mobile application processors; one or more desktop or server central processing units (CPUs), including multi-core CPUs; one or more parallel processor units, such as, Fig.11A A parallel processing unit 1102 and one or more graphics processors or special processing units.

[0157] The parallel processor or GPGPU described herein can be communicatively coupled to a host / processor core to accelerate graphics operations, machine learning operations, pattern analysis operations, and various general-purpose GPU (GPGPU) functions. The GPU can be communicatively coupled to the host processor / core via a bus or another interconnect (e.g., a high-speed interconnect, such as PCIe, NVLink, or other known protocols, standardized protocols, or proprietary protocols). In other examples, the GPU can be integrated on the same package or chip as the core and communicatively coupled to the core via an internal processor bus / interconnect (i.e., inside the package or chip). Regardless of the manner in which the GPU is connected, the processor core can assign work to the GPU in the form of a sequence of commands / instructions contained in a work descriptor. The GPU then uses a dedicated circuit system / logic to efficiently process these commands / instructions.

[0158] Fig. 12C A graphics processing unit (GPU) 1280 is shown, which includes a dedicated set of graphics processing resources arranged as multi-core groups 1265A-1265N. Although details of only a single multi-core group 1265A are provided, it will be appreciated that other multi-core groups 1265B-1265N may be equipped with the same or similar set of graphics processing resources. The details described with respect to multi-core groups 1265A-1265N may also be applied to any of the graphics multiprocessors 1134, 1225, 1250 described herein.

[0159] As illustrated, multi-core group 1265A may include a set 1270 of graphics cores, a set 1271 of tensor cores, and a set 1272 of ray tracing cores. Scheduler / dispatcher 1268 schedules and dispatches graphics threads for execution on the respective cores 1270, 1271, 1272. A set 1269 of register files stores operand values ​​used by cores 1270, 1271, 1272 when executing graphics threads. These register files may include, for example, integer registers for storing integer values, floating point registers for storing floating point values, vector registers for storing packed data elements (integer and / or floating point data elements), and slice registers for storing tensor / matrix values. Slice registers may be implemented as a combined set of vector registers.

[0160] One or more combined first level (L1) caches and shared memory units 1273 store graphics data locally within each multi-core group 1265A, such as texture data, vertex data, pixel data, light data, bounding volume data, etc. One or more texture units 1274 may also be used to perform texture operations, such as texture mapping and sampling. A second level (L2) cache 1275 shared by all multi-core groups 1265A-1265N or a subset of multi-core groups 1265A-1265N stores graphics data and / or instructions for multiple concurrent graphics threads. As illustrated, the L2 cache 1275 may be shared across multiple multi-core groups 1265A-1265N. One or more memory controllers 1267 couple the GPU 1280 to a memory 1266, which may be a system memory (e.g., DRAM) and / or a dedicated graphics memory (e.g., GDDR6 memory).

[0161] Input / output (I / O) circuitry 1263 couples GPU 1280 to one or more I / O devices 1262, such as a digital signal processor (DSP), a network controller, or a user input device. On-chip interconnects may be used to couple I / O devices 1262 to GPU 1280 and memory 1266. One or more I / O memory management units (IOMMUs) 1264 of I / O circuitry 1263 directly couple I / O devices 1262 to system memory 1266. Optionally, IOMMU 1264 manages multiple sets of page tables for mapping virtual addresses to physical addresses in system memory 1266. I / O devices 1262, CPU(s) 1261, and GPU(s) 1280 may then share the same virtual address space.

[0162] In one implementation of the IOMMU 1264, the IOMMU 1264 supports virtualization. In this case, the IOMMU 1264 can manage a first set of page tables for mapping guest / graphics virtual addresses to guest / graphics physical addresses and a second set of page tables for mapping guest / graphics physical addresses to system / host physical addresses (e.g., within system memory 1266). The base address of each of the first set of page tables and the second set of page tables can be stored in a control register and swapped out at context switch (e.g., so that the new context is provided with access to the relevant set of page tables). Although not described in detail in the prior art, the IOMMU 1264 can manage a first set of page tables for mapping guest / graphics virtual addresses to guest / graphics physical addresses and a second set of page tables for mapping guest / graphics physical addresses to system / host physical addresses (e.g., within system memory 1266). Fig. 12C , but each of the cores 1270, 1271, 1272 and / or multi-core groups 1265A-1265N may include a translation lookaside buffer (TLB) for caching guest virtual to guest physical translations, guest physical to host physical translations, and guest virtual to host physical translations.

[0163] (One or more) CPU 1261, GPU 1280 and I / O device 1262 can be integrated on a single semiconductor chip and / or chip package. The illustrated memory 1266 can be integrated on the same chip, or can be coupled to a memory controller 1267 via an off-chip interface. In one implementation, memory 1266 includes GDDR6 memory that shares the same virtual address space as other physical system-level memory, but the basic principles described herein are not limited to this particular implementation.

[0164] Tensor core 1271 may include multiple execution units specifically designed to perform matrix operations, which are basic computational operations for performing deep learning operations. For example, synchronized matrix multiplication operations may be used for neural network training and inference. Tensor core 1271 may perform matrix processing using a variety of operand precisions, including single-precision floating point (e.g., 32 bits), half-precision floating point (e.g., 16 bits), integer words (16 bits), bytes (8 bits), and half bytes (4 bits). For example, a neural network implementation extracts features of each rendered scene, potentially combining details from multiple frames to build a high-quality final image.

[0165] In a deep learning implementation, parallel matrix multiplication work may be scheduled for execution on the tensor core 1271. The training of neural networks in particular requires a large number of matrix dot product operations. To handle the inner product formulation of an N x N x N matrix multiplication, the tensor core 1271 may include at least N dot product processing elements. Before the matrix multiplication begins, a complete matrix is ​​loaded into the slice register, and for each of the N cycles, at least one column of the second matrix is ​​loaded. For each cycle, there are N dot products that are processed.

[0166] Depending on the specific implementation, matrix elements can be stored with different precisions, including 16-bit words, 8-bit bytes (e.g., INT8), and 4-bit nibbles (e.g., INT4). Different precision modes can be specified for the Tensor Core 1271 to ensure that the most efficient precision is used for different workloads (e.g., such as inference workloads, which can tolerate quantization to bytes and nibbles). Supported formats additionally include 64-bit floating point (FP64) and non-IEEE floating point formats, such as the bfloat16 format (e.g., Brain floating point), a 16-bit floating point format with one sign bit, eight exponent bits, and eight significand bits (seven of which are explicitly stored). An example includes support for a reduced precision tensor floating point (TF32) mode that performs calculations using the range of FP32 (8 bits) and the precision of FP16 (10 bits). Reduced-precision TF32 operations can be performed on FP32 inputs and produce FP32 outputs with higher performance relative to FP32 and increased precision relative to FP16. In some examples, one or more 8-bit floating point formats (FP32) are supported.

[0167] In some examples, the tensor core 1271 supports sparse operation modes for matrices in which the vast majority of values ​​are zero. The tensor core 1271 includes support for sparse input matrices encoded in sparse matrix representations (e.g., coordinate list encoding (COO), compressed sparse rows (CSR), compressed sparse columns (CSC), etc.). The tensor core 1271 also includes support for compressed sparse matrix representations in the case where the sparse matrix representation can be further compressed. Compressed matrix data, encoded matrix data, and / or compressed and encoded matrix data and associated compression and / or encoding metadata can be read by the tensor core 1271, and non-zero values ​​can be extracted. For example, for a given input matrix A, non-zero values ​​can be loaded from at least a portion of the compressed and / or encoded representation of matrix A. Based on the position of the non-zero value in matrix A (which can be determined from the index or coordinate metadata associated with the non-zero value), the corresponding value in the input matrix B can be loaded. Depending on the operation to be performed (e.g., multiplication), loading values ​​from input matrix B may be bypassed if the corresponding value is a zero value. In some examples, the pairing of values ​​for certain operations (such as multiplication operations) may be pre-scanned by the scheduler logic, and only operations between non-zero inputs are scheduled. Depending on the dimensions of matrix A and matrix B and the operation to be performed, the output matrix C may be dense or sparse. In the case where the output matrix C is sparse and depending on the configuration of the tensor core 1271, the output matrix C may be output in a compressed format, sparse coding, or compressed sparse coding.

[0168] The ray tracing core 1272 can accelerate ray tracing operations for both real-time ray tracing implementations and non-real-time ray tracing implementations. Specifically, the ray tracing core 1272 may include a ray traversal / intersection circuit system that uses a bounding volume hierarchy (BVH) to perform ray traversals and identify intersections between rays and primitives enclosed within the BVH volume. The ray tracing core 1272 may also include circuit systems for performing depth testing and culling (e.g., using a Z buffer or similar arrangement). In one implementation, the ray tracing core 1272 performs traversal and intersection operations in conjunction with the image denoising techniques described herein, at least portions of which may be performed on the tensor core 1271. For example, the tensor core 1271 may implement a deep learning neural network to perform denoising on frames generated by the ray tracing core 1272. However, the CPU(s) 1261, graphics core 1270, and / or ray tracing core 1272 may also implement all or portions of the denoising and / or deep learning algorithms.

[0169] In addition, as described above, a distributed approach to denoising can be employed, wherein GPU 1280 is in a computing device coupled to other computing devices via a network or high-speed interconnect. According to the distributed approach, the interconnected computing devices can share neural network learning / training data to improve the speed at which the entire system learns to perform denoising for different types of image frames and / or different graphics applications.

[0170] The ray tracing core 1272 can handle all BVH traversals and / or ray-primitive intersections, thereby protecting the graphics core 1270 from being overloaded with thousands of instructions for each ray. For example, each ray tracing core 1272 includes a first set of specialized circuit systems for performing bounding box tests (e.g., for traversal operations) and / or a second set of specialized circuit systems for performing ray-triangle intersection tests (e.g., intersecting rays that have been traversed). Thus, for example, the multi-core group 1265A can simply start ray detection, and the ray tracing core 1272 independently performs ray traversals and intersections and returns hit data (e.g., hits, no hits, multiple hits, etc.) to the thread context. While the ray tracing core 1272 performs traversal and intersection operations, the other cores 1270, 1271 are freed up to perform other graphics or computational work.

[0171] Optionally, each ray tracing core 1272 may include a traversal unit for performing BVH test operations and / or an intersection unit for performing ray-primitive intersection tests. The intersection unit generates a "hit", "no hit", or "multiple hits" response, which it provides to the appropriate thread. During traversal and intersection operations, execution resources of other cores (e.g., graphics core 1270 and tensor core 1271) are freed up to perform other forms of graphics work.

[0172] In some examples described below, a hybrid rasterization / ray tracing approach is used in which the work is distributed between graphics core 1270 and ray tracing core 1272 .

[0173] The ray tracing core 1272 (and / or other cores 1270, 1271) may include hardware support for ray tracing instruction sets, such as Microsoft's DirectX Ray Tracing (DXR), which includes the DispatchRays command; and ray generation shaders, nearest hit shaders, any hit shaders, and miss shaders, which enable unique shader and texture sets to be assigned to each object. Another ray tracing platform that may be supported by the ray tracing core 1272, graphics core 1270, and tensor core 1271 is the Vulkan API (e.g., Vulkan version 1.1.85, or later). However, it should be noted that the basic principles described herein are not limited to any particular ray tracing ISA.

[0174] In general, each core 1272, 1271, 1270 may support a ray tracing instruction set including instructions / functions for one or more of the following: ray generation, nearest hit, any hit, ray-primitive intersection, per-primitive and hierarchy bounding box construction, misses, visits, and exceptions. More specifically, some examples include ray tracing instructions for performing one or more of the following functions: Ray Generation - Ray generation instructions can be executed for each pixel, sample, or other user-defined work assignment. Closest Hit - Closest hit instructions can be executed to locate the closest intersection of a ray with a primitive within the scene. Any Hit - The Any Hit directive identifies multiple intersections between rays and primitives within the scene, potentially identifying a new closest intersection point. Intersect - The Intersect command performs a ray-primitive intersection test and outputs the result. Per-primitive bounding box construction - This instruction builds a bounding box around a given primitive or group of primitives (e.g., when building a new BVH or other acceleration data structure). Miss - Indicates that the ray missed the scene or all geometry within a specified area of ​​the scene. Visits – Indicates the subvolumes that the ray will traverse. Exceptions - includes various types of exception handlers (eg, called for various error conditions).

[0175] In some examples, the ray tracing core 1272 may be adapted to accelerate general computing operations that can be accelerated using computing techniques similar to ray intersection tests. A computing framework may be provided that enables shader programs to be compiled into low-level instructions and / or primitives that perform general computing operations via the ray tracing core. Exemplary computing problems that may benefit from computing operations performed on the ray tracing core 1272 include calculations involving the propagation of beams, waves, rays, or particles within a coordinate space. Interactions associated with that propagation may be calculated relative to geometry or meshes within the coordinate space. For example, calculations associated with the propagation of electromagnetic signals through an environment may be accelerated using instructions or primitives that are executed via the ray tracing core. Refraction and reflection of signals through objects in the environment may be calculated as direct ray tracing simulations.

[0176] The ray tracing core 1272 can also be used to perform calculations that are not directly similar to ray tracing. For example, the ray tracing core 1272 can be used to accelerate mesh projection, mesh refinement, and volume sampling calculations. General coordinate space calculations, such as nearest neighbor calculations, can also be performed. For example, a set of points near a given point can be found by defining a bounding box around the given point in the coordinate space. The BVH and ray detection logic in the ray tracing core 1272 can then be used to determine the set of intersections of points within the bounding box. The intersection constitutes the origin and the nearest neighbor of that origin. The calculations performed using the ray tracing core 1272 can be performed in parallel with the calculations performed on the graphics core 1272 and the tensor core 1271. The shader compiler can be configured to compile a compute shader or other general graphics processing program into a low-level primitive that can be parallelized across the graphics core 1270, the tensor core 1271, and the ray tracing core 1272.

[0177] Building larger and larger silicon dies is challenging for various reasons. As silicon dies get larger, manufacturing yields become smaller, and process technology requirements for different components may vary. On the other hand, in order to have a high-performance system, key components should be interconnected through high-speed, high-bandwidth, low-latency interfaces. These conflicting requirements bring challenges to high-performance chip development.

[0178] The embodiments described herein provide techniques for separating the architecture of a system-on-chip integrated circuit into multiple different chiplets that can be packaged onto a common substrate. In some examples, a graphics processing unit or a parallel processor is composed of various silicon chiplets that are manufactured separately. A chiplet is an integrated circuit that is at least partially packaged, and the at least partially packaged integrated circuit includes different logic units that can be assembled into a larger package with other chiplets. Various chiplet collections with different IP core logic can be assembled into a single device. In addition, the chiplets can be integrated into a base die or base chiplet using active interposer technology. The concepts described herein enable interconnection and communication between different forms of IP within a GPU. IP development on different processes can be mixed. This avoids the complexity of converging multiple IPs to the same process, especially for large SoCs with several styles of IP.

[0179] Enabling the use of multiple process technologies improves time to market and provides a cost-effective way to create multiple product SKUs. For customers, this means getting a product that better suits their requirements in a cost-effective and timely manner. Additionally, the split IP is easier to modify to be independently power-gated, and components that are not in use for a given workload can be turned off, reducing overall power consumption.

[0180] Fig.13 A parallel computing system 1300 according to some examples is shown. In some examples, the parallel computing system 1300 includes a parallel processor 1320, which may be a graphics processor or a computing accelerator as described herein. The parallel processor 1320 includes a global logic unit 1301, an interface 1302, a thread dispatcher 1303, a media unit 1304, a set of computing units 1305A-1305H, and a cache / memory unit 1306. In some examples, the global logic unit 1301 includes global functions for the parallel processor 1320, including device configuration registers, a global scheduler, power management logic, and the like. The interface 1302 may include a front-end interface for the parallel processor 1320. The thread dispatcher 1303 may receive a workload from the interface 1302 and dispatch threads of the workload to the computing units 1305A-1305H. If the workload includes any media operations, at least part of those operations may be performed by the media unit 1304. The media unit may also migrate some operations to the compute units 1305A-1305H. The cache / memory unit 1306 may include cache memory (eg, L3 cache) and local memory (eg, HBM, GDDR) for the parallel processors 1320.

[0181] Figure 14A-14BIllustrate a hybrid logical / physical view of a split parallel processor according to examples described herein. Fig.14A A split parallel computing system 1400 is shown. Fig. 14B A chiplet 1430 of the split parallel computing system 1400 is shown.

[0182] like Fig.14A As shown in , the split computing system 1400 may include a parallel processor 1420, wherein the components of the parallel processor SOC are distributed across multiple chiplets. Each chiplet may be a different IP core that is independently designed and configured to communicate with other chiplets via one or more common interfaces. The chiplets are not limited to computing chiplets 1405, media chiplets 1404, and memory chiplets 1406. Each chiplet can be manufactured separately using different process technologies. For example, the computing chiplet 1405 can be manufactured using the smallest or most advanced process technology available at the time of manufacturing, while the memory chiplet 1406 or other chiplets (e.g., I / O, networking, etc.) can be manufactured using larger or less advanced process technologies.

[0183] The individual chiplets may be bonded to a base die 1410 and configured to communicate with each other and logic within the base die 1410 via an interconnect layer 1412. In some examples, the base die 1410 may include a global logic 1401, which may include a scheduler 1411 and power management 1421 logic unit, an interface 1402, a dispatch unit 1403, and an interconnect fabric module 1408 coupled or integrated with one or more L3 cache blocks 1409A-1409N. The interconnect fabric 1408 may be an inter-chiplet fabric integrated into the base die 1410. The logic chiplets may use the fabric 1408 to relay messages between the individual chiplets. Additionally, the L3 cache blocks 1409A-1409N in the base die and / or the L3 cache blocks within the memory chiplet 1406 may cache data read from and transferred to the DRAM chiplets within the memory chiplet 1406 and to the system memory of the host.

[0184] In some examples, global logic 1401 is a microcontroller that can execute firmware to perform scheduler 1411 and power management 1421 functions for parallel processor 1420. The microcontroller that executes the global logic can be customized for the target use case of parallel processor 1420. Scheduler 1411 can perform global scheduling operations for parallel processor 1420. Power management 1421 functions can be used to enable or disable individual chiplets within a parallel processor when those chiplets are not in use.

[0185] The individual chiplets of parallel processor 1420 may be designed to perform specific functions that would be integrated into a single die in existing designs. A collection of compute chiplets 1405 may include a cluster of compute units (e.g., execution units, streaming multiprocessors, etc.) that include programmable logic for executing compute or graphics shader instructions. Media chiplets 1404 may include hardware logic for accelerating media encoding and decoding operations. Memory chiplets 1406 may include volatile memory (e.g., DRAM) and one or more SRAM cache memory blocks (e.g., L3 blocks).

[0186] like Fig. 14B As shown in , each chiplet 1430 may include common components and dedicated components. The chiplet logic 1436 within the chiplet 1430 may include specific components of the chiplet, such as an array of streaming multiprocessors, computing units, or execution units described herein. The chiplet logic 1436 may be coupled with an optional cache or shared local memory 1438, or a cache or shared local memory may be included within the chiplet logic 1436. The chiplet 1430 may include a fabric interconnect node 1442 that receives commands via an inter-chiplet fabric. Commands and data received via the fabric interconnect node 1442 may be temporarily stored in an interconnect buffer 1439. Data transmitted to and received from the fabric interconnect node 1442 may be stored in the interconnect cache 1440. Power control 1432 and clock control 1434 logic may also be included within the chiplet. The power control 1432 and clock control 1434 logic may receive configuration commands via the fabric and may configure dynamic voltage and frequency scaling for the chiplet 1430. In some examples, each chiplet can have independent clock and power domains and can be clock-gated and power-gated independently of other chiplets.

[0187] At least some of the components within the illustrated chiplet 1430 may also be included in a Fig.14A The base die 1410 may be within logic within the base die 1410. For example, logic within the base die that communicates with the fabric may include some version of the fabric interconnect node 1442. The base die logic that may be independently clock gated or power gated may include some version of the power control 1432 and / or clock control 1434 logic.

[0188] Thus, while various examples described herein use the term SOC to describe devices and systems having a processor and associated circuitry (e.g., input / output (“I / O”) circuitry, power delivery circuitry, memory circuitry, etc.) monolithically integrated into a single integrated circuit (“Integrated Circuit”) die or chip, the present disclosure is not limited in this respect. For example, in various examples of the present disclosure, a device or system may have one or more processors (e.g., one or more processor cores) and associated circuitry (e.g., input / output (“I / O”) circuitry, power delivery circuitry, etc.) arranged in a discrete collection of discrete dies, slices, and / or chiplets (e.g., one or more discrete processor core dies arranged adjacent to one or more other dies (such as a memory die, an I / O die, etc.)). In such discrete devices and systems, individual dies, slices, and / or chiplets may be physically and / or electrically coupled together by a packaging structure including, for example, individual packaging substrates, interposers, active interposers, photonic interposers, interconnect bridges, etc. A discrete collection of discrete dies, slices, and / or chiplets may also be part of a System-on-Package ("SoP"). Graphics Execution Unit

[0189] Figure 15A-15B Thread execution logic 1500 is illustrated including an array of processing elements employed in a graphics processor core according to examples described herein. Figure 15A-15B Elements having the same reference numeral (or name) as elements of any other figure herein may operate or function in any manner similar to that described elsewhere herein, but are not limited thereto. Fig.15A represents an execution unit within a general-purpose graphics processor, and Fig. 15B Represents an execution unit that can be used within a computing accelerator.

[0190] As in Fig.15AAs illustrated in FIG. 1 , in some examples, thread execution logic 1500 includes a shader processor 1502, a thread dispatcher 1504, an instruction cache 1506, a scalable execution unit array including a plurality of execution units 1508A-1508N, a sampler 1510, a shared local memory 1511, a data cache 1512, and a data port 1514. In some examples, the scalable execution unit array can be dynamically scaled by enabling or disabling one or more execution units (e.g., any one of execution units 1508A, 1508B, 1508C, 1508D, up to 1508N-1 and 1508N) based on the computational requirements of the workload. In some examples, the included components are interconnected via an interconnect structure that links to each of the components. In some examples, thread execution logic 1500 includes one or more connections to a memory (such as system memory or cache memory) through instruction cache 1506, data port 1514, sampler 1510, and one or more of execution units 1508A-1508N. In some examples, each execution unit (e.g., 1508A) is a self-contained programmable general-purpose computing unit capable of executing multiple simultaneous hardware threads while processing multiple data elements in parallel for each thread. In various examples, the array of execution units 1508A-1508N is scalable to include any number of separate execution units.

[0191] In some examples, execution units 1508A-1508N are primarily used to execute shader programs. Shader processor 1502 can process various shader programs and can dispatch execution threads associated with shader programs via thread dispatcher 1504. In some examples, thread dispatcher includes logic for arbitrating thread initiation requests from graphics pipelines and media pipelines and instantiating the requested threads on one or more execution units in execution units 1508A-1508N. For example, a geometry pipeline can dispatch vertex shaders, tessellation shaders, or geometry shaders to thread execution logic for processing. In some examples, thread dispatcher 1504 can also handle runtime thread generation requests from executed shader programs.

[0192] In some examples, execution units 1508A-1508N support an instruction set including native support for many standard 3D graphics shader instructions, so that shader programs from graphics libraries (e.g., Direct 3D and OpenGL) are executed with minimal translation. These execution units support vertex and geometry processing (e.g., vertex programs, geometry programs, vertex shaders), pixel processing (e.g., pixel shaders, fragment shaders), and general processing (e.g., compute and media shaders). Each execution unit in execution units 1508A-1508N is capable of multi-issue single instruction multiple data (SIMD) execution, and multi-threaded operations enable an efficient execution environment in the face of high latency memory accesses. Each hardware thread within each execution unit has a dedicated high-bandwidth register file and an associated independent thread state. For pipelines capable of integer operations, single-precision floating-point operations, and double-precision floating-point operations, capable of SIMD branching capabilities, capable of logical operations, capable of transcendental operations, and capable of other miscellaneous operations, execution is multi-issued for each clock. While waiting for data from a shared function in memory or a shared function, dependency logic within execution units 1508A-1508N puts the waiting thread to sleep until the requested data has been returned. While the waiting thread is sleeping, hardware resources can be dedicated to processing other threads. For example, during a delay associated with a vertex shader operation, an execution unit can perform operations for a pixel shader, a fragment shader, or another type of shader program including different vertex shaders. Each example can be applied to use execution using single instruction multiple threads (SIMT) as an alternative to the use of SIMD, or as an addition to the use of SIMD. References to SIMD cores or operations can also be applied to SIMT, or to a combination of SIMD and SIMT.

[0193] Each of the execution units 1508A-1508N operates on an array of data elements. The number of data elements is the "execution size," or number of lanes for an instruction. An execution lane is a logical unit used for the execution of data element access, masking, and flow control within an instruction. The number of lanes may be independent of the number of physical Arithmetic Logic Units (ALUs) or Floating Point Units (FPUs) used for a particular graphics processor. In some examples, the execution units 1508A-1508N support integer and floating point data types.

[0194] The execution unit instruction set includes SIMD instructions. Various data elements can be stored in registers as compact data types, and the execution unit will process various elements based on the data size of the element. For example, when a 256-bit wide vector is operated, the 256 bits of the vector are stored in registers, and the execution unit operates the vector as four independent 64-bit compact data elements (quad-word (Quad-Word, QW) size data elements), eight independent 32-bit compact data elements (double-word (Double Word, DW) size data elements), sixteen independent 16-bit compact data elements (word (Word, W) size data elements) or thirty-two independent 8-bit data elements (byte (byte, B) size data elements). However, different vector widths and register sizes are possible.

[0195] In some examples, one or more execution units may be combined into a fused execution unit 1509A-1509N having a thread control logic (1507A-1507N) common to the fused EUs. Multiple EUs may be fused into a EU group. Each EU in a fused EU group may be configured to execute a separate SIMD hardware thread. The number of EUs in a fused EU group may vary according to the example. Additionally, various SIMD widths may be executed EU by EU, including but not limited to SIMD8, SIMD16, and SIMD32. Each fused graphics execution unit 1509A-1509N includes at least two execution units. For example, a fused execution unit 1509A includes a first EU 1508A, a second EU 1508B, and a thread control logic 1507A common to the first EU 1508A and the second EU 1508B. Thread control logic 1507A controls threads executing on fused graphics execution unit 1509A, allowing each EU within fused execution units 1509A-1509N to execute using a common instruction pointer register.

[0196] One or more internal instruction caches (e.g., 1506) are included in the thread execution logic 1500 to cache thread instructions for the execution unit. In some examples, one or more data caches (e.g., 1512) are included to cache thread data during thread execution. Threads executed on the execution logic 1500 may also store explicitly managed data in the shared local memory 1511. In some examples, a sampler 1510 is included to provide texture sampling for 3D operations and media sampling for media operations. In some examples, the sampler 1510 includes specialized texture or media sampling functions to process texture data or media data during the sampling process before providing the sampled data to the execution unit.

[0197] During execution, the graphics pipeline and the media pipeline send thread initiation requests to the thread execution logic 1500 via the thread generation and dispatch logic. Once the group of geometric objects has been processed and rasterized into pixel data, the pixel processor logic (e.g., pixel shader logic, fragment shader logic, etc.) within the shader processor 1502 is called to further calculate output information and cause the results to be written to the output surface (e.g., color buffer, depth buffer, stencil buffer, etc.). In some examples, the pixel shader or fragment shader calculates the value of each vertex attribute, and the value of each vertex attribute is interpolated across the rasterized object. In some examples, the pixel processor logic within the shader processor 1502 then executes the pixel shader program or fragment shader program supplied by the application programming interface (API). To execute the shader program, the shader processor 1502 dispatches the thread to the execution unit (e.g., 1508A) via the thread dispatcher 1504. In some examples, the shader processor 1502 uses the texture sampling logic in the sampler 1510 to access the texture data in the texture map stored in the memory. Arithmetic operations on texture data and input geometry data compute pixel color data for each geometry fragment, or discard one or more pixels without further processing.

[0198] In some examples, data port 1514 provides a memory access mechanism for thread execution logic 1500 to output processed data to memory for further processing on a graphics processor output pipeline. In some examples, data port 1514 includes or is coupled to one or more cache memories (e.g., data cache 1512) to cache data for memory access via the data port.

[0199] In some examples, execution logic 1500 may also include a ray tracer 1505 that may provide ray tracing acceleration functionality. Ray tracer 1505 may support a ray tracing instruction set that includes instructions / functions for ray generation.

[0200] Fig. 15B1508 . An exemplary internal detail of an execution unit 1508 according to an example is illustrated. Graphics execution unit 1508 may include instruction fetch unit 1537, general register file array (GRF) 1524, architectural register file array (ARF) 1526, thread arbiter 1522, issue unit 1530, branch unit 1532, set of SIMD floating point units (FPUs) 1534, and set of dedicated integer SIMD ALUs 1535 in some examples. GRF 1524 and ARF 1526 include a set of general register files and architectural register files associated with each simultaneous hardware thread that may be active in graphics execution unit 1508. In some examples, per-thread architectural state is maintained in ARF 1526, while data used during thread execution is stored in GRF 1524. The execution state of each thread, including the instruction pointer for each thread, may be saved in thread-specific registers in ARF 1526.

[0201] In some examples, graphics execution unit 1508 has an architecture that is a combination of Simultaneous Multi-Threading (SMT) and fine-grained Interleaved Multi-Threading (IMT). The architecture has a modular configuration that can be fine-tuned at design time based on the target number of simultaneous threads and the number of registers per execution unit, where the execution units are divided across logic for executing multiple simultaneous threads. The number of logical threads that can be executed by graphics execution unit 1508 is not limited to the number of hardware threads, and multiple logical threads can be assigned to each hardware thread.

[0202] In some examples, the graphics execution unit 1508 may issue multiple instructions in coordination, which may each be a different instruction. The thread arbiter 1522 of the graphics execution unit thread 1508 may dispatch the instruction to one of the send unit 1530, the branch unit 1532, or (one or more) SIMD FPUs 1534 for execution. Each execution thread may access 128 general registers within the GRF 1524, wherein each register may store 32 bytes accessible as a SIMD 8 element vector with 32-bit data elements. In some examples, each execution unit thread has access to 4 kilobytes within the GRF 1524, but the examples are not limited thereto, and more or less register resources may be provided in other examples. In some examples, the graphics execution unit 1508 is partitioned into seven hardware threads that may independently perform computing operations, but the number of threads of each execution unit may also vary according to the example. For example, in some examples, a maximum of 16 hardware threads are supported. In an example where seven threads can access 4 kilobytes, GRF 1524 can store a total of 28 kilobytes. Where 16 threads can access 4 kilobytes, GRF 1524 can store a total of 64 kilobytes. Flexible addressing modes can permit registers to be addressed together, effectively creating wider registers or representing strided rectangular block data structures.

[0203] In some examples, memory operations, sampler operations, and other longer latency system communications are dispatched via "send" instructions executed by message passing send unit 1530. In some examples, branch instructions are dispatched to dedicated branch unit 1532 to facilitate SIMD scatter and eventual convergence.

[0204] In some examples, graphics execution unit 1508 includes one or more SIMD floating point units (FPU(s)) 1534 for performing floating point operations. In some examples, FPU(s) 1534 also support integer computations. In some examples, FPU(s) 1534 can SIMD perform up to M 32-bit floating point (or integer) operations, or SIMD perform up to 2M 16-bit integer or 16-bit floating point operations. In some examples, at least one of the FPU(s) provides extended math capabilities that support high throughput transcendental math functions and double precision 64-bit floating point. In some examples, a set 1535 of 8-bit integer SIMD ALUs are also present and can be specifically optimized to perform operations associated with machine learning computations.

[0205] In some examples, an array of multiple instances of graphics execution unit 1508 may be instantiated in graphics sub-core groupings (e.g., sub-slices). For scalability, product architects may choose the exact number of execution units per sub-core grouping. In some examples, execution unit 1508 may execute instructions across multiple execution lanes. In further examples, each thread executed on graphics execution unit 1508 is executed on a different lane.

[0206] Fig.16 An additional execution unit 1600 according to an example is illustrated. In some examples, the execution unit 1600 includes a thread control unit 1601, a thread state unit 1602, an instruction fetch / prefetch unit 1603, and an instruction decode unit 1604. The execution unit 1600 additionally includes a register file 1606, which stores registers that can be assigned to hardware threads within the execution unit. The execution unit 1600 additionally includes a dispatch unit 1607 and a branch unit 1608. In some examples, the dispatch unit 1607 and the branch unit 1608 can be used in conjunction with Fig. 15B The send unit 1530 and branch unit 1532 of graphics execution unit 1508 operate in a similar manner.

[0207] The execution unit 1600 also includes a computing unit 1610, which includes a plurality of different types of functional units. In some examples, the computing unit 1610 includes an ALU unit 1611, which includes an array of arithmetic logic units. The ALU unit 1611 can be configured to perform 64-bit, 32-bit, and 16-bit integer and floating-point operations. Integer and floating-point operations can be performed simultaneously. The computing unit 1610 may also include a systolic array 1612 and a mathematical unit 1613. The systolic array 1612 includes a wide W and deep D network of data processing units, which can be used to perform vector or other data parallel operations in a systolic manner. In some examples, the systolic array 1612 may be configured to perform matrix operations, such as matrix dot product operations. In some examples, the systolic array 1612 supports 16-bit floating-point operations and 8-bit and 4-bit integer operations. In some examples, the systolic array 1612 may be configured to accelerate machine learning operations. In such examples, systolic array 1612 may be configured with support for the bfloat 16-bit floating point format. In some examples, math unit 1613 may be included to perform a specific subset of math operations in an efficient and lower power manner than ALU unit 1611. Math unit 1613 may include a variation of math logic that may be found in the shared functional logic of the graphics processing engine provided by other examples (e.g., Figure 4In some examples, math unit 1613 may be configured to perform 32-bit and 64-bit floating point operations.

[0208] Thread control unit 1601 includes logic for controlling the execution of threads within execution units. Thread control unit 1601 may include thread arbitration logic for starting, stopping, and preempting the execution of threads within execution unit 1600. Thread state unit 1602 may be used to store thread states for threads assigned to execute on execution unit 1600. Storing thread states within execution unit 1600 enables quick preemption of threads when they become blocked or idle. Instruction fetch / prefetch unit 1603 may fetch instructions from an instruction cache of higher level execution logic (e.g., Fig.15A The instruction fetch / prefetch unit 1603 may also issue a prefetch request for instructions to be loaded into the instruction cache based on an analysis of the currently executing thread. The instruction decode unit 1604 may be used to decode instructions to be executed by the computing unit. In some examples, the instruction decode unit 1604 may be used as a secondary decoder to decode complex instructions into constituent micro-operations.

[0209] Execution unit 1600 additionally includes a register file 1606 that can be used by hardware threads executing on execution unit 1600. Registers in register file 1606 can be divided across logic used to execute multiple simultaneous threads within compute unit 1610 of execution unit 1600. The number of logical threads that can be executed by graphics execution unit 1600 is not limited to the number of hardware threads, and multiple logical threads can be assigned to each hardware thread. The size of register file 1606 can vary across examples based on the number of hardware threads supported. In some examples, register renaming can be used to dynamically assign registers to hardware threads. Graphics Pipeline

[0210] Fig.17 is a block diagram of another example of a graphics processor 1700 . Fig.17 Elements having the same reference numeral (or name) as elements of any other figure herein may operate or function in any manner similar to that described elsewhere herein, but are not limited thereto.

[0211] In some examples, graphics processor 1700 includes geometry pipeline 1720, media pipeline 1730, display engine 1740, thread execution logic 1750, and rendering output pipeline 1770. In some examples, graphics processor 1700 is a graphics processor within a multi-core processing system that includes one or more general-purpose processing cores. The graphics processor is controlled by register writes to one or more control registers (not shown) or via commands issued to graphics processor 1700 through ring interconnect 1702. In some examples, ring interconnect 1702 couples graphics processor 1700 to other processing components (such as other graphics processors or general-purpose processors). Commands from ring interconnect 1702 are interpreted by command stream converter 1703, which supplies instructions to various components of geometry pipeline 1720 or media pipeline 1730.

[0212] In some examples, command stream converter 1703 directs the operation of vertex fetcher 1705, which reads vertex data from memory and executes vertex processing commands provided by command stream converter 1703. In some examples, vertex fetcher 1705 provides vertex data to vertex shader 1707, which performs coordinate space transformation and lighting operations on each vertex. In some examples, vertex fetcher 1705 and vertex shader 1707 can execute vertex processing instructions by dispatching execution threads to execution units 1752A-1752B via thread dispatcher 1731.

[0213] In some examples, execution units 1752A-1752B are arrays of vector processors with instruction sets for performing graphics operations and media operations. In some examples, execution units 1752A-1752B may have an attached L1 cache 1751 that is dedicated to each array or shared between arrays. The cache may be configured as a data cache, an instruction cache, or partitioned into a single cache containing data and instructions in different partitions.

[0214] In some examples, the geometry pipeline 1720 includes a tessellation component for performing hardware-accelerated tessellation of 3D objects. In some examples, the programmable shell shader 1711 configures the tessellation operation. The programmable domain shader 1717 provides back-end evaluation of the tessellation output. The tessellation 1713 operates under the direction of the shell shader 1711 and contains dedicated logic for generating a detailed set of geometric objects based on a coarse geometric model that is provided as input to the geometry pipeline 1720. In some examples, if tessellation is not used, the tessellation components (e.g., the shell shader 1711, the tessellation 1713, and the domain shader 1717) can be bypassed.

[0215] In some examples, the complete geometric object may be processed by geometry shader 1719 via one or more threads dispatched to execution units 1752A-1752B, or may proceed directly to clipper 1729. In some examples, the geometry shader operates on the entire geometric object, rather than on vertices or patches of vertices as in previous stages of the graphics pipeline. If tessellation is disabled, geometry shader 1719 receives input from vertex shader 1707. In some examples, geometry shader 1719 is programmable by a geometry shader program to perform geometry tessellation when the tessellation unit is disabled.

[0216] Before rasterization, the clipper 1729 processes the vertex data. The clipper 1729 can be a fixed function clipper or a programmable clipper with clipping and geometry shader functions. In some examples, the rasterizer and depth test component 1773 in the render output pipeline 1770 dispatches a pixel shader to convert the geometric object into a pixel-by-pixel representation. In some examples, the pixel shader logic is included in the thread execution logic 1750. In some examples, the application can bypass the rasterizer and depth test component 1773 and access the unrasterized vertex data via the outflow unit 1723.

[0217] Graphics processor 1700 has an interconnect bus, interconnect structure, or some other interconnect mechanism that allows data and messages to be passed between the main components of the processor. In some examples, execution units 1752A-1752B and associated logic units (e.g., L1 cache 1751, sampler 1754, texture cache 1758, etc.) are interconnected via data port 1756 to perform memory access and communicate with the processor's rendering output pipeline components. In some examples, sampler 1754, cache 1751, 1758, and execution units 1752A-1752B each have a separate memory access path. In some examples, texture cache 1758 can also be configured as a sampler cache.

[0218] In some examples, the render output pipeline 1770 includes a rasterizer and depth test component 1773 that converts vertex-based objects into associated pixel-based representations. In some examples, the rasterizer logic includes a windower / masker unit for performing fixed-function triangle and line rasterization. In some examples, an associated render buffer 1778 and depth buffer 1779 are also available. The pixel operation component 1777 performs pixel-based operations on the data, but in some instances, pixel operations associated with 2D operations (e.g., using hybrid bit block image transfers) are performed by the 2D engine 1741, or replaced by the display controller 1743 using an overlay display plane when displaying. In some examples, a shared L3 cache 1775 can be used for all graphics components, allowing data to be shared without using main system memory.

[0219] In some examples, graphics processor media pipeline 1730 includes media engine 1737 and video front end 1734. In some examples, video front end 1734 receives pipeline commands from command stream converter 1703. In some examples, media pipeline 1730 includes a separate command stream converter. In some examples, video front end 1734 processes media commands before sending them to media engine 1737. In some examples, media engine 1737 includes thread generation functionality for generating threads for dispatching to thread execution logic 1750 via thread dispatcher 1731.

[0220] In some examples, graphics processor 1700 includes display engine 1740. In some examples, the display engine 1740 is external to processor 1700 and is coupled to the graphics processor via ring interconnect 1702, or some other interconnect bus or structure. In some examples, display engine 1740 includes 2D engine 1741 and display controller 1743. In some examples, display engine 1740 contains dedicated logic that can operate independently of the 3D pipeline. In some examples, display controller 1743 is coupled to a display device (not shown), which can be a system-integrated display device such as in a laptop or an external display device attached via a display device connector.

[0221] In some examples, the geometry pipeline 1720 and the media pipeline 1730 may be configured to perform operations based on multiple graphics and media programming interfaces and are not dedicated to any one application programming interface (API). In some examples, the driver software for the graphics processor converts API calls dedicated to a specific graphics or media library into commands that can be processed by the graphics processor. In some examples, support is provided for the Open Graphics Library (OpenGL), the Open Computing Language (OpenCL), and / or the Vulkan graphics and computing APIs all from the Khronos Group. In some examples, support may also be provided for the Direct3D library from Microsoft. In some examples, a combination of these libraries may be supported. Support may also be provided for the Open Source Computer Vision Library (OpenCV). If a mapping from the pipeline of a future API to the pipeline of a graphics processor can be performed, future APIs with compatible 3D pipelines will also be supported. Emulation (including binary translation, code deformation, etc.)

[0222] In some cases, the instruction converter can be used to convert instructions from a source instruction set architecture to a target instruction set architecture. For example, the instruction converter can translate (e.g., using static binary translation, dynamic binary translation including dynamic compilation), deform, simulate or otherwise convert instructions to one or more other instructions to be processed by the core. The instruction converter can be implemented with software, hardware, firmware or a combination thereof. The instruction converter can be on the processor, outside the processor, or partly on the processor and partly outside the processor.

[0223] Fig.18 The block diagram of using a software instruction converter to convert binary instructions in a source ISA into binary instructions in a target ISA according to an example is illustrated. In the illustrated example, the instruction converter is a software instruction converter, but alternatively, the instruction converter can be implemented in software, firmware, hardware, or various combinations thereof. Fig.18 It is shown that a program in a high-level language 1802 can be compiled using a first ISA compiler 1804 to generate first ISA binary code 1806 that can be natively executed by a processor having at least one first ISA core 1816. The processor having at least one first ISA core 1816 represents any processor that is capable of executing or otherwise processing (1) a substantial portion of the first ISA or (2) a program that is natively executed by a processor having at least one first ISA core. The first ISA compiler 1804 represents a compiler operable to generate first ISA binary code 1806 (e.g., object code) that can be executed on a processor 1816 having at least one first ISA core with or without additional linking processing. Similarly, Fig.18 It is shown that a program in a high-level language 1802 can be compiled using an alternative ISA compiler 1808 to generate an alternative ISA binary code 1810 that can be natively executed by a processor 1814 without a first ISA core. An instruction converter 1812 is used to convert the first ISA binary code 1806 into code that can be natively executed by a processor 1814 without a first ISA core. This converted code is not necessarily the same as the alternative ISA binary code 1810; however, the converted code will implement the overall operation and be composed of instructions from the alternative ISA. Thus, the instruction converter 1812 represents software, firmware, hardware, or a combination thereof that allows a processor or other electronic device that does not have a first ISA processor or core to execute the first ISA binary code 1806 through simulation, emulation, or any other process. IP core implementation

[0224] One or more aspects of at least some examples may be implemented by representative code stored on a machine-readable medium that represents and / or defines logic within an integrated circuit (such as a processor). For example, a machine-readable medium may include instructions representing various logic within a processor. When read by a machine, the instructions may cause the machine to manufacture logic for performing the techniques described herein. Such representations (referred to as "IP cores") are reusable units of logic for an integrated circuit that can be stored on a tangible, machine-readable medium as a hardware model describing the structure of the integrated circuit. The hardware model may be supplied to each customer or manufacturing facility that loads the hardware model on a manufacturing machine that manufactures the integrated circuit. The integrated circuit may be manufactured so that the circuit performs the operations described in association with any of the examples described herein.

[0225] Fig.191 is a block diagram illustrating an IP core development system 1900 that can be used to manufacture an integrated circuit to perform operations according to some examples. The IP core development system 1900 can be used to generate modular, reusable designs that can be incorporated into a larger design or used to build an entire integrated circuit (e.g., a SOC integrated circuit). The design facility 1930 can generate a software simulation 1910 of the IP core design in a high-level programming language (e.g., C / C++). The software simulation 1910 can be used to design, test, and verify the behavior of the IP core using a simulation model 1912. The simulation model 1912 can include functional simulation, behavioral simulation, and / or timing simulation. A register transfer level (RTL) design 1915 can then be created or synthesized from the simulation model 1912. The RTL design 1915 is an abstraction of the behavior of an integrated circuit (including associated logic executed using the modeled digital signals) that models the flow of digital signals between hardware registers. In addition to the RTL design 1915, a lower level design at the logic level or transistor level can also be created, designed, or synthesized. Thus, the specific details of the initial design and simulation can be different.

[0226] The RTL design 1915 or equivalent may be further synthesized by the design facility into a hardware model 1920, which may be in a hardware description language (HDL) or some other representation of physical design data. The HDL may be further simulated or tested to verify the IP core design. The IP core design may be stored using a non-volatile memory 1940 (e.g., a hard disk, flash memory, or any non-volatile storage medium) for delivery to a third-party manufacturing facility 1965. Alternatively, the IP core design may be transmitted via a wired connection 1950 or a wireless connection 1960 (e.g., via the Internet). The manufacturing facility 1965 may then manufacture an integrated circuit based at least in part on the IP core design. The manufactured integrated circuit may be configured to perform operations according to at least some of the examples described herein.

[0227] Although some embodiments have been described with reference to specific implementations, other implementations are possible according to some embodiments. Additionally, the arrangement and / or order of elements or other features illustrated in the drawings and / or described herein need not be arranged in the particular manner illustrated and described. According to some embodiments, many other arrangements are possible.

[0228] In each system shown in the drawings, the elements in some cases may each have the same reference numeral or different reference numerals to indicate that the elements represented may be different and / or similar. However, the elements may be flexible enough to have different implementations and work together with some or all of the systems shown or described herein. The various elements shown in the drawings may be the same or different. It is arbitrary which is referred to as the first element and which is referred to as the second element.

[0229] An embodiment is an implementation or example of the present disclosure. References in the specification to "an embodiment," "one embodiment," "some embodiments," or "other embodiments" mean that the particular features, structures, or characteristics described in conjunction with the embodiment are included in at least some embodiments of the present disclosure, but not necessarily in all embodiments of the present disclosure. Various appearances of "an embodiment," "one embodiment," or "some embodiments" do not necessarily refer to the same embodiment.

[0230] Not all components, features, structures, characteristics, etc. described and illustrated herein need to be included in a particular one or more embodiments. For example, if the specification states that a component, feature, structure, or characteristic "may", "might", "can", or "can" be included, then the particular component, feature, structure, or characteristic is not required to be included. If the specification or claims refer to "a" or "an" element, this does not mean that there is only one element. If the specification or claims refer to "additional" elements, this does not exclude the presence of more than one additional element.

[0231] The above description of the illustrated implementations of the present disclosure (including what is described in the Abstract) is not intended to be exhaustive, nor is it intended to limit the present disclosure to the precise forms disclosed. Although specific embodiments and examples of the present disclosure are described herein for illustrative purposes, various equivalent modifications are possible within the scope of the present disclosure, as will be appreciated by those skilled in the relevant art.

[0232] These modifications may be made to the present disclosure in light of the above specific embodiments. The terms used in the appended claims should not be understood to limit the present disclosure to the specific embodiments disclosed in the specification and the drawings. On the contrary, the scope of the present disclosure is determined entirely by the appended claims, which are interpreted in accordance with established principles of claim interpretation. Further examples

[0233] Example 1 provides an exemplary method, which includes: determining whether a current branch in the execution of a single thread is a low-confidence branch based on looking up a branch confidence data structure, in which the branch predictor of the processor has a low level of certainty that the outcome of the current branch is correctly predicted; comparing the branch misprediction rate, microinstruction waste rate, and cache miss rate of the single thread with their corresponding thresholds; and throttling the branch prediction of the current branch based on a determination that the current branch is a low-confidence branch and one or more thresholds of the branch misprediction rate, microinstruction waste rate, and cache miss rate for the single thread are exceeded.

[0234] Example 2 includes the substance of Example 1, wherein the lookup of the branch confidence data structure occurs when the processor performs branch prediction.

[0235] Example 3 includes the substance of Examples 1-2, wherein the branch confidence data structure includes an entry for the current branch, and the entry includes an identifier of the branch and an indication of low confidence.

[0236] Example 4 includes the substance of Examples 1 to 3, wherein the entry further includes one or more of: a number of branch predictions that have been completed for the current branch, a count of mispredictions or successful predictions within the branch prediction.

[0237] Example 5 includes the essence of Examples 1 to 4, wherein throttling branch prediction includes one or more of: skipping operations in one or more stages of an execution pipeline of a processor for a number of clock cycles; reducing a ratio of operations in one or more stages of an execution pipeline of a processor; or limiting a number of in-flight instructions in an execution pipeline of a processor.

[0238] Example 6 includes the substance of Examples 1-5, wherein the level at which branch prediction is throttled is based on a performance state of the processor.

[0239] Example 7 includes the substance of Examples 1 to 6, wherein a performance state of the processor lower than a default performance state corresponds to a throttle level higher than the default throttle level.

[0240] Example 8 includes the substance of Examples 1 to 7, wherein the throttling level is determined based on a mapping between a performance state of the processor and the throttling level.

[0241] Example 9 includes the substance of Examples 1-8, wherein the branch misprediction rate of the single thread is determined based on a number of mispredicted micro-ops per a number of clock cycles.

[0242] Example 10 includes the substance of Examples 1-9, wherein the microinstruction consumption rate of the single thread is determined based on a ratio of allocated micro-ops to retired micro-ops.

[0243] Example 11 includes the substance of Examples 1 to 10, wherein the cache miss rate of the single thread is determined based on a number of cache misses per number of instructions.

[0244] Example 12 includes the substance of Examples 1-11, further comprising removing the branch prediction throttling once thread execution of the single thread is no longer in the low confidence branch.

[0245] Example 13 provides an exemplary processor comprising: a first circuit system for determining whether a current branch in the execution of a single thread is a low-confidence branch based on looking up a branch confidence data structure, in which the branch predictor of the processor has a low level of certainty that the current branch will be taken in the execution of the thread; the first circuit system is used to further compare the branch misprediction rate, microinstruction loss rate, and cache miss rate of the single thread with their corresponding thresholds; and a second circuit system for throttling the branch prediction of the current branch based on a determination that the current branch is a low-confidence branch and that one or more thresholds for the branch misprediction rate, microinstruction loss rate, and cache miss rate for the single thread are exceeded.

[0246] Example 14 includes the substance of Example 13, wherein the lookup of the branch confidence data structure occurs when the processor performs branch prediction.

[0247] Example 15 includes the substance of Example 13 or 14, wherein the branch confidence data structure includes an entry for the current branch, and the entry includes an identifier of the branch and an indication of low confidence.

[0248] Example 16 includes the substance of Examples 13 to 15, wherein throttling branch prediction comprises skipping branch prediction for a number of clock cycles.

[0249] Example 17 includes an exemplary computer-readable storage medium storing instructions that, when executed by a processor of a computing system, enable the computing system to perform the following steps: determining whether a current branch in execution of a single thread is a low-confidence branch based on looking up a branch confidence data structure, in which the branch predictor of the processor has a low level of certainty that the outcome of the current branch is correctly predicted; comparing the branch misprediction rate, microinstruction waste rate, and cache miss rate of the single thread with their corresponding thresholds; and throttling branch prediction of the current branch based on a determination that the current branch is a low-confidence branch and that one or more thresholds of the branch misprediction rate, microinstruction waste rate, and cache miss rate for the single thread are exceeded.

[0250] Example 18 includes the substance of Example 17, wherein the level at which branch prediction is throttled is based on a performance state of the processor.

[0251] Example 19 includes the substance of Example 17 or 18, wherein the branch misprediction rate of the single thread is determined based on a number of mispredicted micro-ops per a number of clock cycles.

[0252] Example 20 includes the substance of Examples 17 to 19, wherein the microinstruction consumption rate of the single thread is determined based on a ratio of allocated micro-ops to retired micro-ops. Additional explanation

[0253] In the above description, embodiments of the present disclosure have been described with reference to specific exemplary embodiments of the present disclosure. However, it will be apparent that various modifications and changes may be made to these implementations without departing from the broader spirit and scope of the present disclosure as described in the appended claims. Therefore, the description and drawings should be regarded as illustrative rather than restrictive.

[0254] Embodiments of the present disclosure may include the steps described above. These steps may be embodied as machine executable instructions, which may be used to cause a general or special processor to perform these steps. Alternatively, these steps may be performed by specific hardware components containing hard-wired logic for performing these steps, or by any combination of programmed computer components and custom hardware components.

[0255] As described herein, instructions may refer to a specific configuration of hardware, such as an application-specific integrated circuit (ASIC) configured to perform certain operations or have predetermined functions, or may refer to software instructions stored in a memory embodied in a non-transitory computer-readable medium. Thus, the techniques shown in the accompanying drawings may be implemented using code and data stored on and executed on one or more electronic devices (e.g., terminal stations, network elements, etc.). Such electronic devices use computer machine-readable media (internally and / or with other electronic devices via a network) to store and transmit code and data, such as non-transitory computer machine-readable storage media (e.g., disks; optical disks; random access memory; read-only memory; flash memory devices; phase change memory) and transient computer machine-readable communication media (e.g., electrical, optical, acoustic or other forms of propagation signals - such as carrier waves, infrared signals, digital signals, etc.). In addition, such electronic devices typically include a collection of one or more processors coupled to one or more other components (such as, one or more storage devices (non-transitory machine-readable storage media), user input / output devices (e.g., keyboards, touch screens and / or displays), and network connections). The coupling of the set of processors to other components is typically through one or more buses and bridges (also referred to as bus controllers). The storage device and signal carrying network traffic represent one or more machine-readable storage media and machine-readable communication media, respectively. Thus, the storage device of a given electronic device typically stores code and / or data for execution on the set of one or more processors of the electronic device. Of course, different combinations of software, firmware, and / or hardware can be used to implement one or more parts of the embodiments of the present disclosure. Throughout this specific embodiment, for the purpose of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure can be implemented without some of these specific details. In some instances, well-known structures and functions are not described in detail to avoid confusing the subject matter of the present disclosure. Therefore, the scope and spirit of the present disclosure should be determined according to the appended claims.

Claims

1. A method comprising: determining, based on looking up a branch confidence data structure, whether a current branch in the execution of the single thread is a low confidence branch in which a branch predictor of the processor has a low level of certainty that an outcome of the current branch is correctly predicted; comparing a branch misprediction rate, a microinstruction waste rate, and a cache miss rate of the single thread with their corresponding thresholds; as well as Branch prediction of the current branch is throttled based on a determination that the current branch is a low confidence branch and one or more thresholds of the branch misprediction rate, the microinstruction waste rate, and the cache miss rate for the single thread are exceeded.

2. The method of claim 1, wherein: When the processor performs a branch prediction, a lookup of the branch confidence data structure occurs.

3. The method according to claim 1 or 2, wherein: The branch confidence data structure includes an entry for the current branch, and the entry includes an identifier of the branch and an indication of low confidence.

4. The method of claim 3, wherein: The items further include one or more of the following: the number of branch predictions that have been completed for the current branch; and A count of mispredictions or success predictions within the branch prediction.

5. The method according to claim 1 or 2, wherein: Throttling the branch prediction includes one or more of the following: skipping operations in one or more stages of an execution pipeline of the processor for a number of clock cycles; reducing a rate of operations in one or more stages of the execution pipeline of the processor; or A number of in-flight instructions in the execution pipeline of the processor is limited.

6. The method according to claim 1 or 2, wherein: The level at which the branch prediction is throttled is based on the performance state of the processor.

7. The method of claim 6, wherein: The performance state of the processor being lower than a default performance state corresponds to a throttling level being higher than a default throttling level.

8. The method of claim 6, wherein: A throttling level is determined based on a mapping between the performance state of the processor and the throttling level.

9. The method according to claim 1 or 2, wherein: The branch misprediction rate of the single thread is determined based on a number of mispredicted micro-ops per a number of clock cycles.

10. The method according to claim 1 or 2, wherein: The microinstruction waste rate for the single thread is determined based on a ratio of allocated micro-ops to retired micro-ops.

11. The method according to claim 1 or 2, wherein: The cache miss rate of the single thread is determined based on a number of cache misses per a number of instructions.

12. The method of claim 1 or 2, further comprising: Once thread execution of the single thread is no longer in a low confidence branch, the branch prediction throttle is removed.

13. A processor, comprising: first circuitry for determining, based on looking up a branch confidence data structure, whether a current branch in execution of a single thread is a low confidence branch in which a branch predictor of the processor has a low level of certainty that the current branch will be taken in thread branching; The first circuit system is used to further compare the branch misprediction rate, microinstruction loss rate and cache miss rate of the single thread with their corresponding thresholds; as well as second circuitry for throttling branch prediction of the current branch based on a determination that the current branch is a low confidence branch and one or more thresholds of the branch misprediction rate, the microinstruction waste rate, and the cache miss rate for the single thread are exceeded.

14. The processor of claim 13, wherein: When the processor performs a branch prediction, a lookup of the branch confidence data structure occurs.

15. A processor as claimed in claim 13 or 14, wherein: The branch confidence data structure includes an entry for the current branch, and the entry includes an identifier of the branch and an indication of low confidence.

16. The processor of claim 13 or 14, wherein: Throttling the branch prediction includes skipping branch prediction for a number of clock cycles.

17. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor of a computing system, enable the computing system to perform the following steps: determining, based on looking up a branch confidence data structure, whether a current branch in the execution of the single thread is a low confidence branch in which a branch predictor of the processor has a low level of certainty that an outcome of the current branch is correctly predicted; comparing a branch misprediction rate, a microinstruction waste rate, and a cache miss rate of the single thread with their corresponding thresholds; as well as Branch prediction of the current branch is throttled based on a determination that the current branch is a low confidence branch and one or more thresholds of the branch misprediction rate, the microinstruction waste rate, and the cache miss rate for the single thread are exceeded.

18. The non-transitory computer readable storage medium of claim 17, wherein: The level at which the branch prediction is throttled is based on the performance state of the processor.

19. The non-transitory computer-readable storage medium of claim 17 or 18, wherein: The branch misprediction rate of the single thread is determined based on a number of mispredicted micro-ops per a number of clock cycles.

20. The non-transitory computer readable storage medium of claim 17 or 18, wherein: The microinstruction waste rate for the single thread is determined based on a ratio of allocated micro-ops to retired micro-ops.

21. The non-transitory computer readable storage medium of claim 17 or 18, wherein: When the processor performs a branch prediction, a lookup of the branch confidence data structure occurs.

22. The non-transitory computer readable storage medium of claim 17 or 18, wherein: The branch confidence data structure includes an entry for the current branch, and the entry includes an identifier of the branch and an indication of low confidence.

23. The non-transitory computer readable storage medium of claim 17 or 18, wherein: Throttling the branch prediction includes one or more of the following: skipping operations in one or more stages of an execution pipeline of the processor for a number of clock cycles; reducing a rate of operations in one or more stages of the execution pipeline of the processor; or A number of in-flight instructions in the execution pipeline of the processor is limited.

24. The non-transitory computer readable storage medium of claim 17 or 18, wherein: The cache miss rate of the single thread is determined based on a number of cache misses per a number of instructions.

25. The non-transitory computer readable storage medium of claim 17 or 18, wherein: The instructions can cause the computing system to perform the following steps: Once thread execution of the single thread is no longer in a low confidence branch, the branch prediction throttle is removed.

Images