Hardware boot for efficiently scheduling workloads to optimal computing modules

Abstract

The invention relates to hardware booting for efficiently scheduling workloads to optimal computing modules. Techniques are described for providing hardware-provided bootstrapping for efficiently scheduling workloads to optimal computing modules. In some examples, hardware includes: a first plurality of physical processor cores of a first type to implement a plurality of logical processor cores of the first type; a second plurality of physical processor cores of a second type, wherein each core of the second type is configured to implement a plurality of logical processor cores of the second type; a power management unit to monitor telemetry data on the first plurality of processor cores and the second plurality of processor cores and to update hardware feedback telemetry data; and thread runtime telemetry circuitry to provide prompts to merge tasks onto one of the core types using hardware feedback telemetry data.

Description

Background Art

[0001] With the advent of mixed processors running on shrink-wrapped operating systems, the operating system needs the ability to understand the different processor cores in the system. On Intel platforms, Intel Thread Director technology provides this capability, providing the operating system with a means to use the optimal core for a given task without requiring the operating system to understand the underlying architecture.

[0002] Products with a large number of compute IPs may package those IPs in modules where all the IPs in a module share common connections to the fabric and the rest of the SOC. Since the power cost of this shared logic may be largely independent of the number of active IPs in the module, there are often power vs. performance tradeoffs associated with how workloads that need to use many of these compute IPs are scheduled across these modules. For example, some types of workloads may benefit when all work is scheduled on a single module, while other types of workloads may benefit from scheduling work across multiple modules.

[0003] In some products, it may be desirable to design and integrate computing modules with very different power and performance capabilities. For example, a product may include a module with very high performance capabilities, but perhaps at the expense of lower energy efficiency. Other modules may be designed to have higher energy efficiency capabilities, but with the tradeoff of providing lower performance capabilities. In such designs, this "low power island" module may provide very high energy efficiency for the work that can be contained on the module, but only for the work that can be fully contained on the module.

[0004] In general-purpose systems, software that schedules work usually doesn't understand or want to understand these lower-level implementation details - the software expects to be able to use common scheduling methods without having to worry about these product-specific details. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Various examples according to the present disclosure will be described with reference to the accompanying drawings, in which:

[0006] Figure 1 The diagram illustrates a computer system including a processor core according to some examples.

[0007] Figure 2 The diagram illustrates thread runtime telemetry circuitry according to examples of the present disclosure.

[0008] Figure 3 Figure illustrates an example format of a control register for enabling thread runtime telemetry according to some examples.

[0009] Figure 4The diagram illustrates a computer system including a first plurality of physical processor cores of a first type and a second plurality of physical processor cores of a second type, wherein each core of the first type is used to implement a plurality of logical processor cores, according to some examples.

[0010] Figure 5A-Figure 5B The diagram illustrates an example format of hardware feedback telemetry data (eg, per logical processor core) according to some examples.

[0011] Figure 6 The diagram illustrates a hardware feedback data structure for hardware feedback telemetry data storing energy efficiency capability values ​​and performance capability values ​​for each logical processor core of a computer system according to some examples.

[0012] Figure 7 is a flow diagram illustrating the operation of a method of performing dynamic simultaneous multi-threading (SMT) scheduling (eg, including SMT core isolation) according to some examples.

[0013] Figure 8 is a flow diagram illustrating the operation of another method of performing dynamic simultaneous multithreading (SMT) scheduling according to some examples.

[0014] Fig. 9 The figure shows an example of a processing device.

[0015] Fig.10 The figure shows an example of a block diagram of a portion of a system.

[0016] Fig.11 The figure shows an example of a system.

[0017] Fig.12 The figure shows an example of task movement.

[0018] Fig.13 An example of the method is shown.

[0019] Fig.14 The figure illustrates an example of interaction between actors in the potential update of telemetry data.

[0020] Fig.15 The figure illustrates an example of interaction between actors in the potential update of telemetry data.

[0021] Fig.16 An example computing system is illustrated.

[0022] Fig.17 A block diagram illustrating an example processor and / or System on a Chip (SoC) that may have one or more cores and an integrated memory controller.

[0023] Fig.18 is a block diagram illustrating a computing system 1800 configured to implement one or more aspects of the examples described herein.

[0024] Fig.19A Figure 1 shows an example of a parallel processor.

[0025] Fig.19B An example of a block diagram of a partition unit is illustrated.

[0026] Fig.19C An example of a block diagram illustrating a processing cluster within a parallel processing unit.

[0027] Fig.19D An example of a graphics multiprocessor is illustrated, wherein the graphics multiprocessor is coupled with a pipeline manager of a processing cluster.

[0028] Figure 20A-20C Additional graphics multiprocessors are illustrated according to examples.

[0029] Fig.21 A parallel computing system according to some examples is shown.

[0030] Figure 22A-22B Illustrate a hybrid logical / physical view of a split parallel processor according to examples described herein.

[0031] 23(A) is a block diagram illustrating both an example in-order pipeline and an example register renaming, out-of-order issue / execution pipeline according to an example.

[0032] 23(B) is a block diagram illustrating both an example in-order architecture core and an example register renaming, out-of-order issue / execution architecture core to be included in a processor according to an example.

[0033] Fig.24 Examples of execution unit circuitry, such as execution unit circuitry, are illustrated.

[0034] Fig.25 is a block diagram of a register architecture according to some examples.

[0035] Figure 26A-26B Thread execution logic including an array of processing elements employed in a graphics processor core is illustrated according to examples described herein.

[0036] Fig. 27 Additional execution units are illustrated according to an example.

[0037] Fig.28 is a block diagram of another example of a graphics processor.

[0038] Fig.29 is a block diagram illustrating an IP core development system that may be used to fabricate an integrated circuit to perform operations according to some examples. DETAILED DESCRIPTION

[0039] The present disclosure relates to methods, apparatus, systems, and non-transitory computer-readable storage media for biasing.

[0040] Some systems provide operating system guidance about the performance and energy efficiency capabilities of each of the cores in the system. The operating system uses this information to determine where to schedule software threads that are ready to run. But it currently has no ability to direct the operating system to schedule a certain class of work only on (one or more) specific modules. Existing solutions can direct the OS to spread work across multiple modules, which may be less performant and / or less energy efficient than containing the work on one or more specific modules.

[0041] The examples detailed in this article add the ability to monitor (over time) the behavior of all work running on compute IP on the system, determine the compute power required, and provide hints to the operating system to consolidate all work onto specific modules when the work runs better when contained on those specific modules.

[0042] In addition to adding the telemetry and logic required to make these decisions, some examples also create new hints to be passed to the OS to consolidate current work in the system onto a subset of available compute modules.

[0043] The example extension provides the operating system with the ability to provide guidance regarding optimal scheduling for the entire set of currently active threads, rather than making suboptimal decisions at the individual thread level.

[0044] As an example, consolidating energy-efficient adaptation work into low-power islands on a product is expected to provide hundreds of milliwatts less power than would otherwise be achieved using existing scheduling hints and OS scheduler behavior. This significantly improves energy efficiency for key classes of workloads.

[0045] In some examples, the power management unit (Punit / PCODE) coordinates IP states to achieve the lowest power state for the processor and exit lower power states (e.g., after low power and / or thermal issues are relaxed). Additionally, the SoC power management unit decides when each of the throttling actions is engaged. In some examples, the PCODE waits a certain amount of time between actions (the wait time / hysteresis may be unique for each action) and / or the PCODE may choose to observe system power / temperature and then measure the impact of the previous action before engaging in the next action. Additionally, the PCODE may engage in many actions in parallel.

[0046] A (e.g., hardware) processor (e.g., having one or more cores) may execute instructions (e.g., threads of instructions) to operate on data, for example, to perform arithmetic, logical, or other functions. For example, software may request an operation, and a hardware processor (e.g., one or more cores of the hardware processor) may perform the operation in response to the request. Software may request execution of a (e.g., software) thread. An operating system (OS) may include a scheduler (e.g., an "OS scheduler") for scheduling execution of (e.g., software) threads on a hardware processor, for example, for scheduling execution of (e.g., software) threads on one or more logical processors (e.g., one or more logical processor cores) of a hardware processor. Each logical processor may be referred to as a corresponding central processing unit (CPU).

[0047] In some examples, the hardware processor implements multithreading (e.g., multithreading), e.g., executing multiple threads simultaneously on one physical processor core. In some examples, the multithreading is temporal multithreading (e.g., hyperthreading), e.g., where only one thread can execute instructions at any given pipeline stage at a time. In some examples, the multithreading is simultaneous multithreading (SMT) (e.g., Hyperthreading), for example, where instructions from more than one thread can be executed in any given pipeline stage at a time. In some examples, SMT allows two (or more) concurrent threads to run on a single physical processor core, for example, where the single physical processor core is exposed to software (e.g., an operating system) as a first logical processor core to execute a first thread and is exposed to software (e.g., an operating system) as a second logical processor core to execute a second thread.

[0048] In some examples, SMT improves multi-threaded (MT) performance by virtualizing a physical processor core (e.g., an SMT physical processor core) into multiple logical processors (e.g., logical processor cores). In some examples, all logical processors (e.g., logical processor cores) of a hardware processor are exposed to an operating system (executing on the hardware processor) as individual logical processors (e.g., logical processor cores). In some examples, this abstraction allows the operating system to schedule software threads across all available logical processors (e.g., logical processor cores), thereby maximizing throughput and multi-threaded (MT) performance. However, in some examples, there is a problem that the resources (e.g., fetch circuits, decode circuits, execution circuits, etc.) of the underlying SMT physical processor core are shared between logical processors, and therefore when other (one or more) "peer" logical threads are active on the same physical SMT core (e.g., when multiple logical processor cores are active on the same physical SMT core), the performance of each individual active logical processor (e.g., logical processor core) is significantly lower than the performance of the physical SMT core. This can lead to poor performance and responsiveness for certain workloads (e.g., lightly threaded workloads initiated by users) when concurrent background threads begin to contend for processor (e.g., central processing unit (CPU)) time on the same SMT physical processor core. Additionally, some processors (e.g., as returned by a core type request from the OS) do not distinguish between logical cores and physical (e.g., SMT) cores.

[0049] In some examples, applications (e.g., software) that are launched and / or interacted with by a user are referred to as foreground applications, for example, and applications that run independently of the user are referred to as background applications. In some examples, foreground and background are priority levels assigned to programs that are running (e.g., not "stopped") in a multitasking environment, for example, where the foreground (FG) includes the application(s) on which the user is working (e.g., an application that receives input(s) from the user and / or provides output to the user, such as via a graphical user interface (GUI), and the background (BG) includes the application(s) that are running behind the system (e.g., without user interaction).

[0050] Examples herein relate to methods and circuit systems that allow threads of a (e.g., foreground) application to use physical SMT cores in isolation (e.g., disabling all logical processor cores except a single logical processor core of the physical SMT core that the thread is using), for example, but if the (e.g., foreground) application only uses a certain threshold number (e.g., 2) of cores, then allow another (e.g., background) (e.g., MT) application to use the remaining (one or more) idle (e.g., unused) physical SMT cores for its use, for example, thereby maximizing foreground and background performance.

[0051] In some examples, an asymmetric platform (e.g., a processor) utilizes different types of cores, e.g., (i) a first type of processor core (e.g., a lower power, lower maximum frequency, and / or more energy efficient core) (e.g., an efficient core (“E core”)) (e.g., a “little” core or “small” core) and (ii) a second, higher performance type of processor core (e.g., a higher power and / or higher frequency core) (e.g., a performance core (“P core”)) (e.g., a “big” core). In some examples, one of these types of cores utilizes SMT (e.g., each of the physical processor cores of the type of core implements multiple logical processor cores), for example, and the other type of core does not use SMT (e.g., each of the physical processor cores of the other type of core implements only a single logical processor core). In some examples, the efficient cores (“E cores”) run at a (maximum) lower frequency, and therefore execute instructions at a lower performance than the performance cores (“P cores”).

[0052] In some examples, the resources of the underlying SMT physical processor core are shared among the logical processors, resulting in the performance of each individual active logical processor (e.g., logical processor core) being significantly lower than the performance of the physical SMT core when other (one or more) "peer" logical threads are active on the same physical SMT core. This problem is more prevalent on a hybrid platform (e.g., a hybrid processor) that includes a first set of cores that do not support SMT and a second set of cores that support SMT. For example, in order to maximize the performance of foreground applications (e.g., foreground processes) on a hybrid platform (e.g., a hybrid processor), some OSs attempt to restrict background tasks to non-SMT cores (e.g., E cores) via corresponding (e.g., "only small") scheduling policies. However, such scheduling policies can cause significant performance degradation of user-initiated multi-threaded workloads (e.g., compilers, renderers, etc.) that run in the "background". Therefore, a dynamic solution is needed that delivers core isolation for lightly threaded foreground tasks when no critical foreground tasks are active on the system, while not compromising the performance of user-initiated MT background tasks.

[0053] Examples herein relate to methods and circuit systems for maximizing SMT performance on hybrid system (e.g., processor) platforms by: (i) providing necessary SMT core isolation (e.g., disabling all but a single logical processor core of a physical SMT core to be used) for user-initiated (e.g., lightly threaded) critical computationally intensive tasks in the foreground while these tasks are concurrently running in a multithreaded background (e.g., "noisy") environment, and / or (ii) allowing user-initiated critical multithreaded background tasks (e.g., compiling, rendering, etc.) to run on SMT core(s) (e.g., certain P-cores) when needed, e.g., without being restricted by a static (e.g., "only small") scheduling configuration for background tasks. In some examples, the scheduling configuration is selected using an operating system (e.g., a scheduler of the operating system).

[0054] A software-based solution to this problem includes a static OS core parking policy and a static scheduling policy that attempts to provide core isolation by parking logical threads based on thread concurrency and utilization, while limiting background tasks only to (one or more) cores that do not support SMT (e.g., certain E cores). However, when critical threads are running concurrently in a multithreaded background environment (e.g., high concurrency and total utilization (e.g., average CPU utilization, such as "C0")), such static OS parking policies fail to deliver the necessary core isolation for critical threads. Even in the absence of critical tasks in the foreground, configuring the static OS scheduling policy for background tasks to "only small" will significantly reduce the performance of user-initiated MT tasks (e.g., compilation, rendering, etc.) that require high performance. Certain examples in this article allow the OS, for example, to implement SMT isolation support when running concurrent scenarios (e.g., foreground and background applications) with mixed quality of service (QoS).

[0055] Certain examples herein detect when core isolation is used based on the concurrency (e.g., of threads running on a processor) and / or the utilization of user-initiated (e.g., as opposed to system-initiated) critical foreground tasks running on the system and the nature of the system (e.g., system-on-a-chip, SoC) workload (e.g., sustained SoC workload due to heavy multithreaded background activity) running on the system. When a lightly threaded compute-intensive critical task is detected running in a noisy, sustained background environment, certain examples herein isolate the resources of the SMT core by forcibly parking (one or more) sibling logical processors that share the resources of the SMT core to dedicate them to the critical task scheduled on the active logical processor of the SMT core, for example, which temporarily limits the computational resources of the multithreaded background tasks running on the system to a subset of the remaining available cores. When the computational requirements for the critical task change due to low utilization and / or high concurrency, certain examples herein do not apply core isolation via SMT sibling parking, for example, and the OS uses a less restrictive (e.g., small or idle) scheduling policy. In one example, a "small or idle" scheduling policy causes the scheduling of threads to attempt to schedule tasks (e.g., threads) to idle high-efficiency cores (e.g., E cores) (e.g., small cores) (e.g., non-SMT cores), and if no high-efficiency cores are available (e.g., no high-efficiency cores are idle), then attempts to schedule tasks to idle performance cores (e.g., P cores) (e.g., large cores) (e.g., SMT cores). In another example, the scheduling policy causes the scheduling of threads to attempt to schedule tasks (e.g., threads) to idle non-SMT physical cores, and if no non-SMT physical cores are available (e.g., no non-SMT cores are idle), then attempts to schedule tasks to idle SMT physical cores, and, for example, if none of these SMT physical cores are available, then attempts to schedule tasks to idle logical cores of SMT cores.

[0056] In some examples, the processor generates a "capability" value to distinguish logical processors (e.g., CPUs) with different (e.g., current) computing capabilities (e.g., computing throughput). In some examples, the processor generates capability values ​​normalized within a range (e.g., 256, 512, 1024, etc.). In some examples, the processor is able to estimate how busy and / or energy efficient a logical processor (e.g., CPU) is via the capability value (e.g., on a per-category basis), for example, and the OS scheduler is used to utilize the capability value when evaluating the performance and energy tradeoffs for scheduling threads.

[0057] In some examples, the performance (Perf) capability value of a logical processor (e.g., CPU) indicates the amount of work it can absorb when running at its highest frequency (e.g., compared to the most capable logical processor (e.g., CPU) of the system). In some examples, the performance (Perf) capability value of a single logical processor (e.g., CPU) is a value (e.g., an 8-bit value indicating a value from 0 to 255) that specifies the relative performance level of the logical processor (e.g., where a higher value indicates higher performance and / or a minimum performance level of 0 indicates a recommendation to the OS not to schedule any threads thereon for performance reasons).

[0058] In some examples, the energy efficiency (EE) capability value of a logical processor (e.g., a CPU) indicates its energy efficiency (e.g., when executing a process). In some examples, the energy efficiency (EE) capability value of a single logical processor (e.g., a CPU) is a value (e.g., an 8-bit value indicating a value from 0 to 255) that specifies the relative energy efficiency level of the logical processor, for example, where a higher value indicates a higher energy efficiency and / or a minimum energy efficiency capability of 0 indicates a recommendation to the OS not to schedule any software threads thereon for efficiency reasons. In some examples, the energy efficiency capability of the maximum value (e.g., 255) indicates which logical processors have the highest relative energy efficiency capability. In some examples, the maximum value (e.g., 255) is an explicit recommendation for the OS to consolidate these jobs onto these logical processors for energy efficiency reasons.

[0059] In some examples, the functionality discussed herein (e.g., core isolation via parking of one or more SMT sibling logical cores) is implemented as a hardware-based solution, e.g., using thread runtime telemetry (e.g., at nanosecond granularity) circuitry (e.g., In some examples, the processor (e.g., via a non-transitory machine-readable medium storing power management code (e.g., p-code)) uses each energy performance preference (EPP) group utilization and quality of service (QoS) to determine whether there is limited threaded high QoS and / or low EPP activity (e.g., foreground threads) and multi-threaded low QoS and / or high EPP activity (e.g., background threads). In some examples, if so, the processor (e.g., via a non-transitory machine-readable medium storing power management code (e.g., p-code)) will populate a data structure storing telemetry data (e.g., for each logical processor core) to dynamically park the (one or more) logical core siblings of the SMT core. In some examples, such a data structure stores data of thread runtime telemetry circuitry, such as (i) hardware guide scheduler (HGS) (or HGS+) circuitry or (ii) thread guide circuitry. In some examples, the processor is configured to cause a (e.g., capability) value (e.g., zero or approximately zero) to be written to one or more entries of (one or more) sibling logical processor cores of a logical processor core of the SMT physical processor core to hint to the OS (e.g., to the OS scheduler) to avoid using the sibling logical processor core(s), e.g., to avoid scheduling threads on the sibling logical processor core(s).

[0060] In some examples, thread runtime telemetry circuitry (e.g., (i) hardware boot scheduler (HGS) (or HGS+) circuitry or (ii) thread boot circuitry) communicates to the OS (e.g., via their corresponding data structures) in real time the digital performance capability and digital power efficiency capability of each logical core in a range of (e.g., 0 to 255) (e.g., 0 to 511) (e.g., 0 to 1023). In some examples, when the performance or energy efficiency capability of a logical processor core (e.g., CPU) is zero, the hardware dynamically adapts the current instruction mix and recommends not scheduling any tasks on such logical core.

[0061] In some examples, the functionality discussed herein (e.g., core isolation via parking one or more SMT peer logical cores) is implemented as a non-transitory machine-readable medium storing system code, e.g., system code that dynamically parks (one or more) logical core peers of an SMT core when executed. In one example, the non-transitory machine-readable medium stores a system software driver (e.g., Dynamic Tuning Technology (DTT) software driver), for example, a system software driver that dynamically optimizes the system for performance, battery life, and thermals when executed.

[0062] Thus, the examples herein deliver significant performance gains by better utilizing SMT (e.g., hyperthreading) enabled cores, thereby delivering unique (e.g., utilizing SMT cores and non-SMT cores) hybrid processor differentiation. The examples herein utilize core isolation via parking one or more SMT sibling logical cores, thereby delivering significant responsiveness and performance gains during concurrent use involving lightly threaded tasks (e.g., application launch, page load, speedometer (e.g., testing browser web application responsiveness through timed simulated user interactions), etc.) running with multithreaded background tasks (e.g., compiling and / or rendering in the background). The examples herein involve less restrictive scheduling for a processor (e.g., a platform), allowing user-initiated multithreaded background tasks (e.g., compilers and / or renderers) to utilize SMT processor cores when needed.

[0063] Some (e.g., default) OS scheduling policies (e.g., utilizing SMT cores and non-SMT cores) on hybrid platforms do not provide flexibility to customers. In some examples, scheduling (one or more) background threads only on less powerful non-SMT physical processor cores (e.g., efficient cores (E cores)) (e.g., small cores) is too restrictive because user-initiated (e.g., multithreaded) background work (e.g., compilation and / or rendering) cannot utilize more powerful SMT physical processor cores (e.g., performance cores (P cores)) (e.g., large cores). In some examples, scheduling (one or more) background threads on less powerful non-SMT physical processor cores (e.g., efficient cores (E cores)) (e.g., small cores) or idle SMT physical processor cores (e.g., performance cores (P cores)) (e.g., large cores) affects foreground (FG) performance during concurrent use (e.g., sharing SMT cores with critical threads due to lack of core isolation). The above shortcomings are overcome by the dynamic SMT scheduling disclosed herein, which, for example, provides core isolation via forced core parking of logical SMT peer processors when needed (e.g., when necessary), while allowing less restrictive (e.g., "small or idle") scheduling policies to be adopted for user-initiated background tasks (e.g., compilers / renderers, etc.) running on the system to utilize SMT physical processor cores (e.g., performance cores (P cores)) (e.g., large cores). Being able to dynamically implement SMT isolation at runtime allows the OS (e.g., OS scheduler) to use less restrictive scheduling policies (e.g., "small or idle") for user-initiated background tasks without worrying about the impact on foreground responsiveness.

[0064] Some of the examples in this document do not disable SMT completely (e.g., for the entire processor), for example, by a hardware initialization manager (e.g., Basic Input / Output System (BIOS) firmware or Unified Extensible Firmware Interface (UEFI) firmware) or by having the OS schedule work on only one of the threads.

[0065] Figure 1The figure illustrates a computer system including a processor core according to some examples. The processor core 109 includes multiple components (e.g., microarchitecture prediction and caching mechanisms) that can be shared by multiple contexts (e.g., virtualized as multiple logical processors implemented on a single SMT core). For example, a branch target buffer (BTB) 124, an instruction cache 132, and / or a return stack buffer (RSB) 144 can be shared by multiple contexts. Certain examples include a context manager circuit 110 for simultaneously maintaining multiple unique states associated with multiple contexts and switching active contexts between those contexts tracked by the context manager circuit. In some examples, the processor core 109 is an instance of the processor core 2390 in Figure 23 (B).

[0066] The depicted computer system 100 includes a branch predictor 120 and a branch address calculator 142 (BAC) in pipelined processor cores 109 ( 1 )- 109 (N) according to examples of the present disclosure. Figure 1 , a pipelined processor core (e.g., 109(1)) includes an instruction pointer generation (IPtr Gen) stage 111, a fetch stage 130, a decode stage 140, and an execute stage 150. In one example, computer system 100 includes a plurality of cores 109(1-N), where N is any positive integer. In another example, computer system 100 includes a single core.

[0067] In some examples, each processor core 109(1-N) instance supports multithreading (e.g., executing two or more concurrent operations or sets of threads on first and second logical cores), and may do so in a variety of ways, including time-division multithreading, simultaneous multithreading (e.g., where a single physical core provides a logical core for each of the threads that the physical core simultaneously multithreads), or a combination thereof (e.g., time-division fetch and decode followed by simultaneous multithreading). In the depicted example, each individual processor core 109(1) to 109(N) includes an instance of a branch predictor 120. The branch predictor 120 may include a branch target buffer (BTB) 124.

[0068] In some examples, a branch target buffer 124 stores (e.g., in a branch predictor array) predicted target instructions corresponding to each of a plurality of branch instructions (e.g., branch instructions of a code segment that has been executed multiple times). In the depicted example, a branch address calculator (BAC) 142 is included, which accesses (e.g., includes) a return stack buffer 144 (RSB). In some examples, the return stack buffer 144 is used to store the return address of any CALL instruction (e.g., push its return address onto the stack) (e.g., in a stack data structure where the last data in is the first data out (LIFO)).

[0069] Branch address calculator (BAC) 142 is used to calculate the address of certain types of branch instructions and / or verify the branch prediction made by the branch predictor (e.g., BTB). In some examples, the branch address calculator performs branch target and / or next continuous linear address calculations. In some examples, the branch address calculator performs static predictions on branches based on address calculations.

[0070] In some examples, the branch address calculator 142 includes a return stack buffer 144 to track the return address of the CALL instruction. In one example, the branch address calculator attempts to correct any inappropriate predictions made by the branch predictor 120 to reduce the damage of branch mispredictions. As an example, the branch address calculator verifies branch predictions for those branches whose targets can be determined only from the branch instruction and the instruction pointer.

[0071] In some examples, the branch address calculator 142 maintains a return stack buffer 144, which is used as a branch prediction mechanism for determining the target address of a return instruction, for example, where the return stack buffer operates by monitoring all "call subroutine" and "return from subroutine" branch instructions. In one example, when the branch address calculator detects a "call subroutine" branch instruction, the branch address calculator pushes the address of the next instruction into the return stack buffer, for example, using a stack top pointer to mark the top of the return stack buffer. The return stack buffer contains a stack of return addresses in this example by pushing the address immediately following each "call subroutine" instruction into the return stack buffer. When the branch address calculator later detects a "return from subroutine" branch instruction, the branch address calculator pops the top return address from the return stack buffer, for example, to verify the return address predicted by the branch predictor 120. In one example, for a direct branch type, the branch address calculator is used to (e.g., always) predict that the conditional branch has been taken, for example, and if the branch predictor does not predict that the direct branch has been taken, the branch address calculator overrides a miss prediction or an improper prediction of the branch predictor.

[0072] In some examples, the core 109 includes circuitry for verifying branch predictions made by the branch predictor 120. Each branch predictor 120 entry (e.g., in the BTB 124) may also include a valid field and a bundle address (BA) field, which are used to improve accuracy and verify branch predictions performed by the branch predictor 120, as discussed in more detail below. In one example, the valid field and the BA field are each composed of a one-bit field. However, in other examples, the size of the valid field and the BA field may vary. In one example, the fetched instruction (e.g., by the BAC 142 from the line 137) is sent to the decoder 146 for decoding, and the decoded instruction is sent to the execution circuit (e.g., unit) 154 for execution.

[0073] The depicted computer system 100 includes a network device 101 , input / output (I / O) circuitry 103 (eg, keyboard), a display 105 , and a system bus (eg, interconnect) 107 .

[0074] In one example, the branch instructions stored in the branch predictor 120 are pre-selected by the compiler as the branch instructions to be taken. Figure 1 As shown in the compiler code 104 stored in the memory 102 of , the compiler code 104 includes a sequence of codes that, when executed, converts the source code of a program written in a high-level language into executable machine code. In one example, the compiler code 104 also includes additional branch predictor code 106, which predicts the target instruction for a branch instruction (e.g., a branch instruction that may be taken (e.g., a pre-selected branch instruction)). Subsequently, the branch predictor 120 (e.g., its BTB 124) is updated with the target instruction for the branch instruction. In one example, the software (e.g., by specifying a prediction mode by software, or by setting a mode bit in the entry by a prediction mode implicitly defined by the mode of the instruction written to the BTB).

[0075] The memory 102 may include operating system (OS) code 160 , virtual machine monitor (VMM) code 162 , first application (eg, program) code 168 , second application (eg, program) code 170 , or any combination thereof.

[0076] In some examples, OS code 160 is used to implement OS scheduler 162, for example, to utilize thread runtime telemetry circuitry 116 (e.g., (i) hardware guide scheduler (HGS) (or HGS+) circuitry or (ii) thread guide circuitry) of processor core 109 to schedule one or more threads for processing in core 109 (e.g., a logical core of a plurality of logical cores implemented by core 109). In some examples, OS scheduler 162 is used to implement (e.g., select from a plurality of scheduling modes) one or more scheduling modes. In some examples, the scheduling mode enables scheduling (one or more) threads utilizing dynamic SMT scheduling disclosed herein, for example, to provide SMT core isolation via forced core parking of logical SMT sibling processors when needed (e.g., when necessary), for example, while allowing a less restrictive (e.g., "small or idle") scheduling policy for user-initiated background tasks (e.g., compiler / renderer, etc.) running on the system to utilize SMT physical processor cores (e.g., performance cores (P cores)) (e.g., large cores). In some examples, OS 160 includes a control value 164, for example, for setting the number of logical processors that can be in a non-parked (or idle) state at any given time. In some examples, a control value 164 (e.g., "CPMaxCores") is set (e.g., by a user) to specify a maximum percentage of logical processors that can be in a non-parked state at any given time (e.g., with respect to logical processors within each non-uniform memory access (NUMA) node, for example, as discussed below). In an example with 16 logical processors (e.g., in a NUMA node), configuring the value of the setting to 50% ensures that no more than eight logical processors are always in a non-parked state at the same time. In some examples, the value of the "CPMaxCores" setting will be automatically rounded up to a minimum number of cores (e.g., "CPMinCores"), which specifies a minimum percentage of logical processors that can be in a non-parked state at any given time (e.g., with respect to all logical processors enabled on the system within each NUMA node). In an example with 16 logical processors (e.g., in a NUMA node), configuring the value of the "CPMinCores" setting to 25% ensures that at least four logical processors are always in a non-parked state. In some examples, if the value of this setting is 100%, the core parking feature is disabled.

[0077] In some examples, non-uniform memory access (NUMA) is a computer system architecture that is used with a multiprocessor design in which some memory areas have greater access latency, for example, due to how the system memory and physical processors (e.g., processor cores) are interconnected. In some examples, some memory areas are directly connected to one or more physical processors, where all physical processors are connected to each other through various types of interconnect structures. In some examples, for large multiprocessor (e.g., multi-core) systems, this arrangement results in less memory contention and improved system performance. In some examples, the NUMA architecture divides memory and processors into groups, referred to as NUMA nodes. In some examples, from the perspective of any single processor in the system, memory in the same NUMA node as the processor is referred to as local memory, and memory contained in another NUMA node is referred to as remote memory (e.g., where the processor (e.g., core) can access local memory faster).

[0078] In some examples, a virtual machine monitor (VMM) code 166 is used to implement one or more virtual machines (VMs) as a simulation of a computer system. In some examples, VMs are based on a specific computer architecture and provide the functionality of an underlying physical computer system. Their implementation may involve specialized hardware, firmware, software, or a combination. In some examples, a virtual machine monitor (VMM) (also referred to as a hypervisor) is a software program that, when executed, enables the creation, management, and control of VM instances, and manages the operation of a virtualized environment on a physical host machine. In some examples, VMM is the main software behind the virtualized environment and implementation. In some examples, when installed on a host machine (e.g., a processor), VMM facilitates the creation of VMs, e.g., each VM has a separate operating system (OS) and application. VMM can manage the backend operations of these VMs by allocating necessary computing, memory, storage, and other input / output (I / O) resources (such as, but not limited to, an input / output memory management unit (IOMMU)). A VMM may provide a centralized interface for managing the complete operation, status, and availability of VMs installed on a single host machine or spread across different and interconnected hosts.

[0079] As discussed below, the depicted core (eg, its branch predictor 120 ) includes access to one or more registers. In some examples, the core includes one or more general registers 108 and / or one or more status / control registers 112 .

[0080] In some examples, each entry of the branch predictor 120 (e.g., in its BTB 124) includes a tag field and a target field. In one example, the tag field of each entry in the BTB stores at least a portion of an instruction pointer (e.g., a memory address) that identifies a branch instruction. In one example, the tag field of each entry in the BTB stores an instruction pointer (e.g., a memory address) that identifies a branch instruction in the code. In one example, the target field stores at least a portion of an instruction pointer for the target of the branch instruction identified in the tag field of the same entry. In addition, in other examples, the entry of the branch predictor 120 (e.g., in its BTB 124) includes one or more other fields. In some examples, the entry does not include a separate field for assisting in the prediction of whether a branch instruction is taken (e.g., if there is a branch instruction (e.g., in the BTB), the branch instruction is considered to be taken).

[0081] like Figure 1 As shown, IPtr Gen (pointer generation) multiplexer 113 of IPtr generation stage 111 receives an instruction pointer from line 115A. The instruction pointer provided via line 115A is generated by incrementer circuit 115, which receives a copy of the latest instruction pointer from path 113A. Incrementer circuit 115 can increment the current instruction pointer by a predetermined amount to obtain the next sequential instruction from the program sequence currently being executed by the core.

[0082] In one example, upon receiving the IPtr from the IPtr Gen multiplexer 113, the branch predictor 120 compares a portion of the IPtr with a tag field of each entry in the branch predictor 120 (e.g., the BTB 124). If no match is found between the IPtr and the tag field of the branch predictor 120, the IPtr Gen multiplexer will proceed to select the next sequential IPtr as the next instruction to be fetched in this example. Conversely, if a match is detected, the branch predictor 120 reads the valid field of the branch predictor entry that matches the IPtr. If the valid field is not set (e.g., has a logical value of 0), the branch predictor 120 considers the corresponding entry to be "invalid", and in this example, the match between the IPtr and the tag of the corresponding entry will be ignored, for example, and the branch target of the corresponding entry will not be forwarded to the IPtr Gen multiplexer. On the other hand, if the valid field of the matching entry is set (e.g., has a logical value of 1), then in this example, the branch predictor 120 proceeds to perform a logical comparison between a predetermined portion of the instruction pointer (IPtr) and the branch address (BA) field of the matching branch predictor entry. If an "allowable condition" exists, the branch target of the matching entry will be forwarded to the IPtr Gen multiplexer, otherwise, the branch predictor 120 ignores the match between the IPtr and the tag of the branch predictor entry. In some examples, the entry indicator is formed not only by the current branch IPtr, but also by at least a portion of the global history.

[0083] More specifically, in one example, the BA field indicates where the corresponding branch instruction is stored within a line of cache memory 132. In some examples, the processor can initiate execution of multiple instructions per clock cycle, where the instructions are not dependent on each other and do not use the same execution resources.

[0084] For example, Figure 1 Each line of instruction cache 132 shown in includes multiple instructions (e.g., six instructions). In addition, in this example, in response to a fetch operation of fetch unit 134, instruction cache 132 responds by providing the entire cache line to fetch unit 134 (e.g., in the case of a "hit"). Instructions within a cache line may be grouped into separate "bundles." For example, Figure 1 As shown, the first three instructions in cache line 133 can be addressed as bundle 0, and the last three instructions can be addressed as bundle 1. Each instruction in the instructions within the bundle is independent of each other (e.g., can be issued simultaneously for execution). In some examples, the BA field provided in the branch predictor 120 entry is used to identify the bundle address of the branch instruction corresponding to the corresponding entry. For example, in one example, the BA identifies whether the branch instruction is stored in the first bundle or the second bundle of a particular cache line.

[0085] In one example, the branch predictor 120 performs a logical comparison between the BA field of the matching entry and a predetermined portion of the IPtr to determine whether a "permissible condition" exists. For example, in one example, the fifth bit position of the IPtr (e.g., IPtr[4]) is compared to the BA field of the matching (e.g., BTB) entry. In one example, when IPtr[4] is not greater than BA, a permissible condition exists. Such permissible conditions help prevent apparently unnecessary predictions of branch instructions that may not be executed. That is, when less than all IPtrs are considered when comparing with the tag of the branch predictor 120, it is possible to have a match with the tag, but this may not be a true match. Nevertheless, the match between the IPtr and the tag of the branch predictor indicates that a particular cache line including the branch instruction corresponding to the corresponding branch predictor entry may be about to be executed. Specifically, if the bundle address of the IPtr is not greater than the BA field of the matching branch predictor entry, the branch instruction in the corresponding cache line is quickly executed. Therefore, in some examples, performance benefits can be achieved by continuing to obtain the target of the branch instruction.

[0086] As discussed above, if there is an "allowable condition", then in this example, the branch target of the matching entry will be forwarded to the IPtr Gen multiplexer. Otherwise, the branch predictor will ignore the match between the IPtr and the tag. In one example, the branch target forwarded from the branch predictor is initially sent to the branch prediction (BP) redirection multiplexer 128 and then to the IPtr Gen multiplexer. Figure 1 As shown, the BP redirect multiplexer 128 may also receive instruction pointers from other branch prediction devices. In one example, the input lines received by the BP redirect multiplexer are prioritized to determine which input line will be allowed to pass through the BP redirect multiplexer to reach the IPtr Gen multiplexer.

[0087] In addition to forwarding the branch target to the BP redirection multiplexer, upon detecting a match between the IPtr and the tag of the branch predictor, the BA of the matching branch predictor entry is forwarded to the branch address calculator (BAC) 142. The BAC 142 Figure 1 140 , but may also be located in other stages. The BAC may also receive cache lines from fetch unit 134 via line 137 .

[0088] In this example, the IPtr selected by the IPtr Gen multiplexer is also forwarded to the fetch unit 134 via data line 135. Once the IPtr is received by the fetch unit 134, the cache line corresponding to the IPtr is fetched from the instruction cache 132. The cache line received from the instruction cache is forwarded to the BAC via data line 137.

[0089] In this example, upon receiving the BA, the BAC will read the BA to determine where a pre-selected branch instruction (e.g., a branch instruction identified in a matching branch predictor entry) is located in the next cache line to be received by the BAC (e.g., the first or second bundle of cache lines). In one example, the location at which the branch instruction is located within the bundle of cache lines (e.g., in a bundle of three instructions, the branch instruction will be stored as the second instruction) is predetermined.

[0090] In an alternative example, the BA includes additional bits for more specifically identifying the address of the branch instruction within the cache line.Thus, the branch instruction is not restricted to a particular instruction location within the bundle.

[0091] After the BAC determines the address of the pre-selected branch instruction within the cache line and has received the corresponding cache line from the acquisition unit 134, the BAC will decode the corresponding instruction to verify whether the IPtr actually corresponds to the branch instruction. If the instruction addressed by the BA in the received cache line is a branch instruction, there is no need to correct the branch prediction. On the contrary, if the corresponding instruction in the cache line is not a branch instruction (i.e., the IPtr does not correspond to a branch instruction), the BAC will send a message to the branch predictor to invalidate the corresponding branch predictor entry to prevent similar mispredictions of the same branch predictor entry. Thereafter, the invalidated branch predictor entry will be overwritten by the new branch predictor entry.

[0092] Additionally, in one example, the BAC will increment the IPtr by a predetermined amount and forward the incremented IPtr to the BP redirection multiplexer 128 via data line 145 (e.g., data line 145 from the BAC will take precedence over data line from the branch predictor). Thus, the incremented IPtr will be forwarded to the IPtr Gen multiplexer and passed to the fetch unit to correct the branch misprediction by fetching instructions that sequentially follow the IPtr.

[0093] In some examples, context manager circuitry 110 allows multiple contexts to utilize one or more of the shared components discussed above, e.g., while mitigating cross-context information leakage by directly or indirectly observing stored information. Computing system 100 (e.g., core 109) may include control registers (e.g., model-specific register(s)) 112 (e.g., as described below with reference to Figure 3), segment registers 114 (e.g., indicating the current privilege level), thread runtime telemetry circuitry 116 (e.g., as described below with reference to Figure 2-Figure 6 ), or any combination thereof. Segment register 114 may store a value indicating a current privilege level of software operating on the logical core (e.g., separately for each logical core). In one example, the current privilege level is stored in a current privilege level (CPL) field of a code segment selector register of segment register 114. In some examples, processor core 109 requires a certain level of privilege to perform certain actions, such as actions requested by a particular logical core (e.g., actions requested by software running on the particular logical core).

[0094] Each thread may have a context. In some examples, the context is identified by one or more of the following attributes: 1) a hardware thread identifier, such as a value identifying one of multiple logical processors (e.g., logical cores) implemented on the same physical core through a technology such as simultaneous multithreading (SMT); 2) a privilege level such as implemented by a ring; 3) a page table base address or code segment configuration such as implemented in a control register (e.g., CR3) or a code segment (CS) register; 4) an address space identifier (ASID) such as implemented by a process context ID (PCID) or a virtual process ID (VPID) that semantically distinguishes virtual-to-physical mapping used by the CPU; 5) a key register containing a cryptographically sealed asset (e.g., a token) used to determine the privileges of the executing software; and / or 6) a transient—context change, such as a random reset of the context.

[0095] During any significant period of time, many threads (e.g., their contexts) may be active within a physical core. In some examples, system software time slices between applications and system software functions, potentially allowing many context accesses to microarchitectural predictions and / or caching mechanisms.

[0096] An instance of thread runtime telemetry circuitry 116 (e.g., (i) hardware boot scheduler (HGS) (or HGS+) circuitry or (ii) thread guide circuitry) may be located in each core 109(1-N) of computer system 100 (e.g., for each logical processor implemented by a core). A single instance of thread runtime telemetry circuitry 116 may be located anywhere in computer system 100, e.g., a single instance of thread runtime telemetry circuitry for all cores 109(1-N) present.

[0097] In one example, state / control registers 112 include (one or more) state registers for indicating the state of the processor core and / or (one or more) control registers for controlling the functionality of the processor core. In one example, one or more (e.g., control) registers are written (e.g., only) in response to a request from an OS running on the processor, e.g., when the OS is operating in privileged (e.g., system) mode, but not for code running in non-privileged (e.g., user) mode. In one example, the control registers are writable only by software running in hypervisor mode and not by software running in user mode. In some examples, control registers 112 include fields for enabling thread runtime telemetry circuitry 116, such as Figure 3 shown.

[0098] In some examples, decoder 146 decodes the instruction, and the decoded instruction is executed by execution circuitry 154 , eg, to perform an operation according to an opcode of the instruction.

[0099] In some examples, decoder 146 decodes the instruction, and the decoded instruction is executed by execution circuitry 154 , for example, to reset one or more capabilities (or one or more software thread runtime attribute histories) of, for example, thread runtime telemetry circuitry 116 .

[0100] The computer system 100 may include a performance monitoring circuit system 172, which includes, for example, any number of performance counters therein for counting, monitoring and / or recording performance-related events, activities and / or other measurements. In various examples, the performance counters may be programmed by software running on the core to record performance monitoring information. For example, any of the performance counters may be programmed to increment for each occurrence of a selected event, or to increment for each clock cycle during a selected event. The event may include any of a variety of events related to the execution of program code on the core, such as branch mispredictions, cache hits, cache misses, translation lookaside buffer hits, translation lookaside buffer misses, etc. Therefore, the performance counters may be used to adjust or profile program code to improve or optimize performance. In some examples, the thread runtime telemetry circuit system 116 is part of the performance monitoring circuit system 172. In some examples, the thread runtime telemetry circuit system 116 is separate from the performance monitoring circuit system 172. The computer system 100 may include a power management unit 174 to monitor power usage, etc.

[0101] In some examples, thread runtime telemetry circuitry 116 (e.g., (i) hardware boot scheduler (HGS) (or HGS+) circuitry or (ii) thread boot scheduler circuitry) is used to generate a "capability" value to distinguish logical processors (e.g., CPUs) of each physical processor core 109 having different (e.g., current) computing capabilities (e.g., computing throughput). In some examples, thread runtime telemetry circuitry 116 generates capability values ​​normalized within a range (e.g., 256, 512, 1024, etc.). In some examples, thread runtime telemetry circuitry 116 is able to estimate how busy and / or energy efficient a logical processor (e.g., CPU) is via the capability values ​​(e.g., on a per-category basis), for example, and OS scheduler 162 is used to utilize the capability values ​​when evaluating performance and energy tradeoffs for scheduling threads.

[0102] In some examples, the performance (Perf) capability value of a logical processor (e.g., CPU) indicates the amount of work it can absorb when running at its highest frequency (e.g., compared to the most capable logical processor (e.g., CPU) of system 100). In some examples, the performance (Perf) capability value of a single logical processor (e.g., CPU) of system 100 is a value (e.g., an 8-bit value indicating a value from 0 to 255) that specifies the relative performance level of the logical processor, e.g., where a higher value indicates higher performance and / or a minimum performance level of 0 indicates a recommendation to the OS to not schedule any threads thereon for performance reasons.

[0103] In some examples, the energy efficiency (EE) capability value of a logical processor (e.g., a CPU) of the system 100 indicates its energy efficiency (e.g., when executing a process). In some examples, the energy efficiency (EE) capability value of a single logical processor (e.g., a CPU) is a value (e.g., an 8-bit value indicating a value from 0 to 255) that specifies the relative energy efficiency level of the logical processor, for example, where a higher value indicates a higher energy efficiency and / or a minimum energy efficiency capability of 0 indicates a recommendation to the OS not to schedule any software threads thereon for efficiency reasons. In some examples, the energy efficiency capability of the maximum value (e.g., 255) indicates which logical processors have the highest relative energy efficiency capability. In some examples, the maximum value (e.g., 255) is an explicit recommendation for the OS to consolidate these jobs onto these logical processors for energy efficiency reasons.

[0104] In some examples, the functionality discussed herein (e.g., core isolation via parking of one or more SMT sibling logical cores) is accomplished by using thread runtime telemetry circuitry 116 (e.g., The processor (e.g., via a non-transitory machine-readable medium storing power management code (e.g., p-code)) uses each energy performance preference (EPP) group utilization and quality of service (QoS) to determine whether there is limited threaded high QoS and / or low EPP activity (e.g., foreground threads) and multi-threaded low QoS and / or high EPP activity (e.g., background threads). In some examples, if so, the processor (e.g., via a non-transitory machine-readable medium storing power management code (e.g., p-code)) will populate a data structure storing telemetry data of the thread runtime telemetry circuit system 116 (e.g., for each logical processor core) to dynamically park the (one or more) logical core siblings of the SMT core. In some examples, such a data structure stores data of (i) the hardware boot scheduler (HGS) (or HGS+) circuit system or (ii) the thread guide circuit system. In some examples, the thread runtime telemetry circuitry 116 is used to cause a (e.g., capability) value (e.g., zero or approximately zero) to be written to one or more entries of (one or more) sibling logical processor cores of the logical processor core of the SMT physical processor core to hint to the OS 160 (e.g., to the OS scheduler 162) to avoid using the sibling logical processor core(s), e.g., to avoid scheduling threads on the sibling logical processor core(s).

[0105] In some examples, thread runtime telemetry circuitry 116 (e.g., (i) hardware boot scheduler (HGS) (or HGS+) circuitry or (ii) thread guide circuitry) communicates to the OS in real time (e.g., via its corresponding data structures) the digital performance capability and digital power efficiency capability of each logical core in a range of (e.g., 0 to 255) (e.g., 0 to 511) (e.g., 0 to 1023). In some examples, when the performance or energy efficiency capability of a logical processor core (e.g., CPU) is zero, thread runtime telemetry circuitry 116 adapts the current instruction mix and recommends not scheduling any tasks on such logical core.

[0106] In some examples, thread runtime telemetry circuitry 116 reduces hardware implementation costs for performance monitoring by, for example, providing ISA-level counters (e.g., number of load instructions) that can be shared among cores, predicting capability values ​​based on dynamic characteristics of the system (e.g., thereby eliminating the need to run a workload on each core to measure its workload), and by providing a single counter based on multiple performance monitoring events.

[0107] Each core 109 of the computer system 100 can be identical (e.g., symmetric cores), or a suitable subset of one or more of the cores can be different from the other cores (e.g., asymmetric cores). In one example, the set of asymmetric cores includes a first type of core (e.g., lower power cores) and a second, higher performance type of core (e.g., higher power cores). In some examples, an asymmetric processor is a hybrid processor that includes one or more less powerful non-SMT physical processor cores (e.g., high efficiency cores (E cores)) (e.g., small cores) and one or more SMT physical processor cores (e.g., performance cores (P cores)) (e.g., big cores).

[0108] In some examples, a computer system includes multiple cores that all execute the same instruction set architecture (ISA). In some examples, a computer system includes multiple cores, each core having an instruction set architecture (ISA) according to which the core executes instructions issued by software or provided to it and / or the system. In this specification, use of the term "instruction" may generally refer to this type of instruction (which may also be referred to as a macroinstruction or an ISA-level instruction), rather than: (1) a microinstruction or micro-operation, which may be provided to execution and / or scheduling hardware as a result of decoding a macroinstruction (e.g., by a hardware instruction decoder), and / or (2) a command, procedure, routine, subroutine, or other software construct, the execution and / or implementation of which involves the execution of multiple ISA-level instructions.

[0109] In some such systems, the system may be heterogeneous in that it includes cores with different ISAs. The system may include a first core having hardware, hard wiring, microcode, control logic, and / or other microarchitectures designed to execute specific instructions according to a specific ISA (or an extension or other subset of an ISA), and the system may also include a second core that does not have such a microarchitecture. In other words, the first core may be able to execute these specific instructions (except for decoding macroinstructions into microinstructions and / or microoperations) without any translation, emulation, or other conversion of instructions, while the second core cannot. In this case, the specific ISA (or an extension or subset of an ISA) may be referred to as being supported (or natively supported) by the first core and not supported by the second core, and / or the system may be referred to as having a heterogeneous ISA.

[0110] In other such systems, the system may be heterogeneous in that it includes cores that have the same ISA but differ in performance, power consumption, and / or some other processing metric or capability. The differences may be provided by the size, speed, and / or microarchitecture of the cores and / or their features. In a heterogeneous system, one or more cores may be referred to as "big" because they are able to provide, they can be used to provide, and / or their use can provide and / or result in a higher level of performance (e.g., more instructions per cycle (IPtrC)), power consumption (e.g., less energy efficient), and / or some other metric than one or more other "small" or "small" cores in the system.

[0111] In these and / or other heterogeneous systems, it may be possible for tasks to be executed by different types of cores. In addition, it may be possible for a scheduler (e.g., a hardware scheduler and / or software scheduler 162 of an operating system 160 executing on a processor) to schedule or dispatch tasks to different cores and / or migrate tasks between / among different cores (generally a "task scheduler"). Thus, optimizing, balancing, or otherwise affecting throughput, latency, response time, delay, fairness, quality of service, performance, power consumption, and / or some other measured workload on a heterogeneous system may include task scheduling decisions.

[0112] For example, if a particular task is mostly stopped due to long latency memory access, it may be more efficient to schedule it on a "small" core (e.g., an E core) and save power on other larger cores (e.g., a P core). On the other hand, heavy tasks may be scheduled on a large core (e.g., a P core) to complete calculations faster, for example, and to put the system into sleep / idle faster. Due to the diversity of workloads that can be performed by a system (e.g., a client), the dynamic nature of the workload, and the conditions of the system itself, a pure software solution may not be able to make such decisions directly. Therefore, the use of examples (e.g., telemetry circuitry during thread runtime) in this article may be expected to provide information, and such decisions may be based in part or in whole on this information. In addition, the use of these examples may be expected to be used to optimize and / or adjust applications based on the information that can be provided.

[0113] The processor may include (for example, as described below with reference to Figure 2-Figure 6The processor may include other shared structures for processing state, including, for example, prediction structures, cache structures, physical register files (rename state), and buffer state (store buffers). Prediction structures (such as branch predictors or prefetchers) may store state related to past execution behavior, which is used to predict future behavior. The processor may use these predictions to guide speculative execution, thereby achieving performance that would otherwise be impossible. Cache structures (such as caches or TLBs) may keep local copies of shared state to allow the processor to access (e.g., very) quickly.

[0114] Figure 2 The figure shows the thread runtime telemetry circuit system 116 according to an example of the present disclosure. The thread runtime telemetry circuit system 116 (and / or the hybrid scaling predictor 240) can be implemented in logic gates and / or any other type of circuit system, all or part of which can be included in discrete components (e.g., microcontrollers) and / or integrated into the circuit system of a processing device or any other device in a computer or other information processing system, for example, implemented in a core in a heterogeneous SoC (such as Figure 1 in the core 109) and / or in the system agent.

[0115] In some examples, the thread runtime telemetry circuitry 116 generates one or more software thread runtime attribute histories (e.g., including weight values ​​and / or HCNT counter values ​​discussed herein). Figure 2 In the example, any number of unweighted event counts (shown as E0 210A to E N Each of 210N) represents an unweighted event count or any other output (typically, each output is an "unweighted event count") of a performance counter such as Figure 1 In various examples, E0 210A to E0 210B may be any performance counter in the performance monitoring circuitry 172 and / or the thread runtime telemetry circuitry 116. N 210N may represent any number of sets of unweighted event counts, including any number of subsets of unweighted event counts from different (e.g., logical) cores. For example, the unweighted event counts may be from all performance counters in one (e.g., logical) core, from one or more performance counters in a first (e.g., logical) core plus one or more performance counters in a second (e.g., logical) core, from one or more performance counters in a first (e.g., logical) core plus one or more performance counters in a second (e.g., logical) core plus one or more performance counters in a third (e.g., logical) core, and so on. In addition, event counts (e.g., E0 210A to E10N) may be from all performance counters in one (e.g., logical) core, from one or more performance counters in a first (e.g., logical) core plus one or more performance counters in a second (e.g., logical) core plus one or more performance counters in a third (e.g., logical) core, and so on.N Any one or more of 210N) can represent an output of (e.g., feedback from) an active runtime (e.g., work) counter, such as work counter 230 (described below), as in examples in which a hierarchy of performance and work counters is implemented (note that in such examples, the event count may be referred to as an unweighted event count, even though it may be generated by a work counter based on a weighted event count).

[0116] exist Figure 2 , weight register 220 represents a programmable or configurable register or other storage location (or combination of storage locations) for storing any number of weight values ​​(shown as W0 222A through W0 222B). N 222N), each weight value corresponds to one of the unweighted event counts and is used by a corresponding weighting unit (shown as weighting units 224A to 224N) to weight the corresponding unweighted event count and generate a weighted event count. The weight values ​​may be a set of adjusted values. For example, software or firmware may assign a weight value of 1 to E0 and a weight value of 224A to E1. N Assign a weight value of 2, in which case weighting unit 224A may weight (eg, scale or multiply) E0 by a factor of 1, and weighting unit 224N may weight E0 by a factor of 2. N The weighting is performed (e.g., scaling or multiplication). In various examples, any weight value (including 0), range of weight values, and / or weighting method (e.g., multiplication, division, addition, etc.) can be used. In various examples, the implementation of the weight register and / or weighting unit can limit the selection of the weight value to one of multiple possible weight values.

[0117] exist Figure 2 , weighted event counts (shown as outputs of weighting units 224A to 224N) are received for processing by a work counter (shown as a heterogeneous (e.g., hybrid) counter (HCNT) 230, but usable with homogeneous or heterogeneous processors / systems). In an example, processing of the weighted event counts may include summing the weighted event counts to generate a measure of the workload (generally a "measured workload"). Various examples may provide the measured workload for use based on various performance measures or other parameters, each of which is scaled or manipulated in various ways and for various purposes. In an example, the work counter may be used to provide a dynamic overview of the current workload.

[0118] For example, HCNT 230 can be used to generate a weighted sum of various types of performance monitoring events, which can be dynamically estimated by all cores in a system (e.g., SoC). HCNT 230 can be used to predict thread runtime telemetry circuitry (e.g., HGS or thread guide) categories, for example, HCNT 230 can be used as a source for hybrid scaling predictor 240 and / or for any software with access to HCNT 230. Events can be subclasses of ISA (e.g., AVX floating point, AVX2 integer), special instructions (e.g., repeated strings), or bottleneck categories (e.g., front-end constraints for top-down analysis). Weights can be selected to reflect the type of code being executed (e.g., memory stalls or branch code) and / or performance ratios (e.g., 2 for instruction classes that execute twice as fast on large cores and 1 for all other instruction classes), scalars of workload (e.g., 2 for fused multiplication instructions), etc.

[0119] Certain examples provide any of a variety of events to be counted and / or summed, including events related to arithmetic floating-point (e.g., 128-bit) vector instructions, arithmetic integer (e.g., 256-bit) vector instructions, arithmetic integer vector neural network instructions, load instructions, store instructions, repeated strings, top-down micro-architectural analysis (TMA) level 1 metrics (e.g., front-end bound, back-end bound, bad speculation, retirement), and / or any performance monitoring event counted by any counter.

[0120] In addition to the working counter according to the example of the present disclosure, Figure 2 The diagram illustrates a representation of the use of work counters according to examples of the present disclosure, including use of a hybrid scaling predictor 240 and / or use by any software (e.g., OS code 160) that has access to the work counters. In an example, a hybrid scaling predictor 240 (e.g., implemented in hardware or firmware) provides information to the OS 160 or other system software (e.g., directly or indirectly, such as by enabling a range of indices based on counter values), and / or may be used to predict performance scaling (e.g., between large cores (e.g., P cores) and small cores (e.g., E cores), such as by providing history-based hints to the hardware (e.g., via writing to a hardware feedback data structure 250 that is read by the OS). In some examples, the hardware feedback data structure 250 is written by the power management unit 174.

[0121] In some examples, hybrid scaling predictor 240 is used to generate one or more capability values ​​(e.g., for each logical processor core) 242. In some examples, capability values ​​242 include performance capability 242P (e.g., for each logical processor core) and / or energy efficiency capability 242E (e.g., for each logical processor core).

[0122] In some examples, data generated by the thread runtime telemetry circuitry 116 is stored in a hardware feedback data structure 250, for example, having one or more sets of entries for each logical processor core. In some examples, the data structure is based on Figure 5A-Figure 5B . In some examples, hardware feedback data structure 250 (e.g., accessible by OS code 160 or at least OS scheduler 162 of OS code 160) is stored in a storage device of thread runtime telemetry circuitry 116 (e.g., within thread runtime telemetry circuitry 116 or separate from thread runtime telemetry circuitry 116 (e.g., in system memory 102 of system 100)). In some examples, data in hardware feedback data structure 250 is modifiable (e.g., modifiable by thread runtime telemetry circuitry 116 and / or power management unit 174) to achieve core isolation via forced core parking of logical SMT sibling processors when needed.

[0123] In an example, a work counter may be used to provide a hint (e.g., written to a hardware feedback data structure 250) (e.g., a capability value) to an operating system running on a heterogeneous (e.g., or homogeneous) SoC or system, wherein the hint may provide a task scheduling that may improve performance and / or quality of service. For example, a homogeneous system includes one or more instances of the same core for optimal multi-core thread scheduling. For example, a heterogeneous client system including one or more large cores (e.g., P cores) and one or more small cores (e.g., E cores) may be used to run an artificial intelligence (AI) application (e.g., a machine learning model) that includes a particular class of instructions that may accelerate processing of instruction types typically used in AI applications, e.g., particularly or only when executed on a large core (e.g., P core). Using a work counter programmed to monitor the execution of such instructions may provide a hint to the OS 160 for directing the OS scheduler 162 to schedule threads including these instructions on a large core (e.g., P core) rather than on a small core (e.g., E core), thereby improving performance and / or quality of service.

[0124] In some examples, the weight values ​​in register 220 are programmable to provide adjustments to the weights based on actual results (e.g., in a lab). In an example, one or more weights of zero may be used to disconnect specific events or event categories. In an example, one or more weights of zero may be used to isolate components that feed into a work counter. Examples herein may support options for enabling / disabling work counters for hardware and / or software (e.g., OS) for any of a variety of reasons (e.g., to avoid power leakage when the work counter is not in use).

[0125] In one example, Figure 1 The scheduler 162 of the operating system code 160 in the embodiment of the present invention uses the thread runtime telemetry circuitry 116 (and / or the hybrid scaling predictor 240) to schedule a software thread (e.g., a first application code (e.g., Figure 1 168) or the second application code (e.g., Figure 1 In some examples, Figure 1 The scheduler 162 of the operating system code 160 in the system uses capability values ​​242 (e.g., stored in the hardware feedback data structure 250) for implementing the dynamic SMT scheduling disclosed herein (e.g., performance capability 242P of each logical processor core and / or energy efficiency capability 242E of each logical processor core), for example, to provide core isolation via forced core parking of logical SMT sibling processors when needed (e.g., when necessary), for example, while allowing less restrictive (e.g., “small or idle”) scheduling policies for user-initiated background tasks (e.g., compiler / renderer, etc.) running on the system to utilize SMT physical processor cores (e.g., performance cores (P cores)) (e.g., big cores).

[0126] In some examples, the software thread runtime attribute history (e.g., including weight values ​​and / or HCNT counter values ​​discussed herein) of the thread runtime telemetry circuitry 116 may be useful to a first software thread, but not to a subsequent second software thread. In other examples, it may be desirable to clear (e.g., set to zero) certain software thread runtime attribute histories (e.g., capability values), for example, to provide core isolation via forced core parking of logical SMT sibling processors when needed.

[0127] Thus, some examples herein provide instructions (and methods) to clear software thread runtime attribute history, e.g., to clear the capability value of a certain logical processor (e.g., and not (one or more) other logical processors), e.g., to provide core isolation via forced core parking of logical SMT sibling processors. For example, clearing the current value of the HCNT counter (e.g., and thus clearing the effect of that value on the full prediction flow). For example, clearing Figure 2 Counter E0...E N and / or the current value of HCNT 230.

[0128] In one example, the instruction mnemonic is "HRESET", but for other examples, it can be another mnemonic. The use opcode of HRESET can include an immediate operand, other types of operands, or a zero explicit operand (e.g., defined without using any operand). In one example, hardware (e.g., processor core) (e.g., without causing an exception (e.g., a fault)) ignores any immediate operand value and / or any request specific settings. It should be understood that other examples can utilize immediate operand values ​​(e.g., so that the value is retained for other purposes). In another example in which an instruction includes an immediate operand, it is possible to define that the immediate operand will only include zero (e.g., otherwise an exception (e.g., a fault) will be caused when executing the instruction). Other operand values ​​may not be supported, and incorrect settings can generate exceptions similar to invalid opcodes (e.g., undefined opcodes or general protection errors).

[0129] In one example, an instruction is used to ignore explicit (e.g., immediate) operands, while its implicit operands (e.g., not explicitly specified in a field of the instruction) may be (e.g., Figure 1 Another option is to define instructions without explicit immediate operands, and in this case, the effective use may be indicated by the opcode (e.g., a mnemonic corresponding to HRESET), for example, while its implicit operand (e.g., not explicitly specified in a field of the instruction) may be (e.g., Figure 1 In some examples, the implicit operation object is a single register (e.g., EAX) or a concatenation of multiple registers (e.g., EAX:EDX is used to concatenate the contents of register EAX with the contents of the following register EDX (e.g., to enable the 64-bit mask configuration option)).

[0130] In some examples, the instruction utilizes a new opcode (e.g., a legacy opcode that is not a legacy instruction), for example, such that hardware that does not support the instruction will not be able to execute the instruction (e.g., and an exception undefined instruction will occur in this case). In some examples, the use of the instruction can include software (e.g., an OS) checking whether the hardware supports execution of the instruction before scheduling execution of the instruction. In one example, the software is used to check whether the hardware supports execution of the instruction by performing a check (e.g., with a mnemonic of CPUID) instruction feature bit setting.

[0131] In some examples, execution of instructions is allowed only for certain privilege levels, e.g., supervisor level (e.g., ring 0) and / or user level (e.g., ring 3). In examples where instructions are restricted to use only by supervisor level (e.g., OS) (e.g., only in ring 0), a request for execution of instructions at user level (e.g., user application) generates an exception, e.g., a general protection exception.

[0132] Certain examples herein define instructions in which the OS is able to select processor components to be cleared (e.g., to (e.g., only) clear the history of one or more logical processors) (e.g., to (e.g., only) clear one or more of the software thread runtime attribute histories). In one example, the instruction includes control parameters for enabling software (e.g., the OS) to control the supported precise history reset at runtime (e.g., in a method much faster than writing to the MSR). In some examples, control of the instruction is accomplished by parameters of the instruction (e.g., a data register to enable a 32-bit control option and / or a data register set to enable a 64-bit control option). In some examples, the instruction also defines OS control of the support capabilities of the instruction (e.g., select-in). In some examples, the instruction takes an implicit operand (e.g., EAX) or an explicit operand.

[0133] In examples where instructions are supported in user mode (e.g., ring 3), the OS may have the ability to control and select what capabilities (e.g., what capabilities among multiple capabilities) the instruction includes and / or what type of history the instruction may reset and in what manner. To support this, in some examples, OS assistance (e.g., OS system calls of an application programming interface (API)) may be requested and used to enable instructions to user-level code, indicate which reset (e.g., HRESET) support capabilities are enabled by the OS (e.g., and supported by the hardware), and / or for controlling any reset (e.g., HRESET) instruction parameters (e.g., at the hypervisor level).

[0134] In one example, the OS sets the instruction as part of OS scheduler runtime support, for example, to clear the capability value of a certain logical processor (e.g., but not other logical processor(s)) to provide core isolation via forced core parking of logical SMT sibling processors (e.g., as Figure 7 ). In some examples, the instruction is defined with a new opcode, so the software (e.g., OS) first checks whether the hardware supports the instruction and what its capabilities are before being able to use the instruction. Therefore, in one example, different code paths are defined by software to support the instruction. For example, where the check whether the hardware supports the instruction is performed by reading (one or more) (e.g., CPUID) feature bits to determine whether the hardware supports the instruction. In one example, the software only uses the instruction if the hardware supports the instruction, as indicated by its enumeration method.

[0135] In one example of a processor, execution is done in a speculative manner. To avoid speculative history resets, it is possible to take an action as a pre-serialization action instruction (e.g., where all previous (in program order) instructions have been completed locally before the history reset is completed) when an instruction (e.g., HRESET) is executed for a history reset (e.g., when all checks for resetting history have occurred, but the history reset itself has not occurred). In one example, HRESET is used to avoid history leaks (e.g., in a core that does not execute instructions in program order). Another possible support option is to enable pre-serialization instructions to support only a subset of history reset types that can be affected by the processor's speculative execution method. In yet another option, instructions are supported as serialization. It is also possible to define support as serialization instructions only for specific HRESET capabilities and only when these HRESET capabilities are enabled for use. For example, the option of selecting a pre-serialized instruction support method or a serialized instruction support method for an appropriate subset of history reset types may be used to limit any negative performance side effects of pre-serialized or serialized instruction support, such as in situations where all prior (e.g., in program order) instructions have completed locally before a history reset is performed.

[0136] In one example, a reset (e.g., HRESET) instruction includes a control register (e.g., used by the OS) to enable different support functions. In one example, as a default, all of the support functions are disabled. In one example, the OS is used to enable a subset or all of the support functions. In one example, only a lower (e.g., 32) appropriate subset of bits is allocated for HRESET use.

[0137] In some examples, thread runtime telemetry circuitry 116 is enabled by control register 112. An example format of this register is Figure 3 shown.

[0138] Figure 3 An example format of a control register 112 for enabling thread runtime telemetry is shown according to some examples. The format of the control register 112 (e.g., IA32_HW_FEEDBACK_CONFIG) of a logical processor core may include bit index [63:2] 306 as reserved, bit index one (bit position two) 304 for turning on thread runtime telemetry (e.g., the corresponding function of the thread runtime telemetry circuit system 116), and / or bit index zero (bit position one) 302 for turning on the hardware feedback interface (HFI). In some examples, both bits 0 and 1 must be set for the thread runtime telemetry circuit system 116 (e.g., thread director circuit system) to be enabled. In some examples, after setting these two bits, the hardware immediately updates the (e.g., additional) "category" column in the runtime telemetry (e.g., thread director) hardware feedback data structure 250 (e.g., table). In one example, its control register 112 (eg, bit 0 302 and / or bit 1 304 ) is only set (or reset) for requests made in the hypervisor mode. In some examples, the hardware feedback data structure 250 may be written by the power management unit 174 .

[0139] Figure 4 The figure shows a computer system 100 according to some examples, the computer system 100 includes a first plurality of physical processor cores 401 of a first type and a second plurality of physical processor cores 402 of a second type, wherein each core of the first type is used to implement multiple logical processor cores. In some examples, the core 401 of the first type is an SMT physical processor core (e.g., a performance core (P core)) (e.g., a large core). In some examples, the core 402 of the second type is a less powerful non-SMT physical processor core (e.g., an efficient core (E core)) (e.g., a small core).

[0140] In some examples, the computer system 100 includes multiple physical cores 401 of the SMT type of the first physical core type, for example, "X" physical cores 401, where X is an integer greater than one. In some examples, each first physical core 401 of the SMT type implements multiple logical cores, for example, even if two logical cores are implemented by the same physical core, the operating system (and applications) treat each logical core as if it were its own discrete core. Figure 4In the embodiment, (e.g., performance) physical core 109_P1 implements logical core 109_P1A and logical core 109_P1B, (e.g., performance) physical core 109_P2 implements logical core 109_P2A and logical core 109_P2B, (e.g., performance) physical core 109_P(X) implements logical core 109_P(X)A and logical core 109_P(X)B, and so on.

[0141] In some examples, the computer system 100 includes a plurality of non-SMT type (or SMT type in other examples) physical cores 402 of the second physical core type, for example, "Y" physical cores 402, where Y is an integer greater than one (e.g., where X and Y are equal in some examples and X and Y are not equal in other examples). In some examples, each non-SMT type second physical core 402 implements only a single logical core. Figure 4 In the example, (e.g., energy efficiency) physical core 109_E1 implements a single logical core, (e.g., energy efficiency) physical core 109_E2 implements a single logical core, physical core 109_E(Y) implements a single logical core, etc. In one example, computer system 100 includes six SMT physical processor cores 401 of a first type (e.g., 12 logical processor cores) and eight non-SMT physical processor cores 402 of a second type, so for such computer system 100, there are a total of 14 (6+8) physical processor cores, but there are 20 (12+8) logical processor cores.

[0142] In some examples, thread runtime telemetry circuitry 116 (e.g., thread director circuitry) is used to Figure 4 The computer system 100 in the embodiment generates runtime telemetry data, which includes, for example, one or more capability values ​​generated for each logical core. In some examples, the performance monitoring circuit system 172 is used to generate Figure 4 The performance data of the computer system 100 in FIG. 1 may include, for example, one or more capability values ​​of each logical core.

[0143] Figure 5A-Figure 5BThe figure illustrates example formats 500A-500B of hardware feedback telemetry data (e.g., per logical processor core) according to some examples. In some examples, the hardware feedback telemetry data according to the formats 500A-500B is generated by the thread runtime telemetry circuitry 116 (e.g., thread director circuitry). In some examples, the hardware feedback telemetry data is stored in a runtime telemetry (e.g., thread director) hardware feedback data structure 250 (e.g., table). In some examples, uppercase CL is a category and uppercase CP is a capability defined for the processor. In some examples, the first capability is a performance capability and the second capability is an energy efficiency capability. In some examples, the individual categories (CL) indicate (e.g., performance) differences between cores (e.g., different core functions), for example, a category in which some cores (e.g., P cores) provide higher performance than other cores (e.g., E cores). For example, where the first category (e.g., Category 1) indicates support for ISA extensions (such as, but not limited to, vector extensions (e.g., AVX) (e.g., AVX2-FP32), matrix extensions (e.g., AMX), etc.), and Category 2 indicates higher Vector Neural Network Instruction (VNNI) (e.g., AVX512 VNNI) performance differences. Some examples include categories for tracking waits (e.g., UMWAIT / TPAUSE / PAUSE, etc.) to prevent performance cores (e.g., P cores) from being idle while actual work goes to high-efficiency cores (e.g., E cores). Categories can also be used to indicate support for one or more accelerators or accelerator subtypes (e.g., matrix accelerators, in-memory computing accelerators, etc.).

[0144] Figure 6 FIG. 2 illustrates a hardware feedback data structure 250 for hardware feedback telemetry data according to some examples, the hardware feedback data structure 250 storing energy efficiency capability values ​​and performance capability values ​​for each logical processor core of a computer system. In some examples, the thread runtime telemetry circuitry 116 (e.g., thread director circuitry) is used to populate during runtime a processor including logical processor cores LP0 through LPn-1 (e.g., for a 20 logical processor core example computer system 100 including six SMT physical processor cores 401 of a first type (e.g., 12 logical processor cores) and eight non-SMT physical processor cores 402 of a second type, this would be LP0 through 19). Figure 6In some examples, the thread runtime telemetry circuitry 116 (e.g., its hybrid scaling predictor 240) is used to generate a performance capability (Perf Cap) 242P (e.g., for each logical processor core) and / or an energy efficiency capability (EE Cap) 242E (e.g., for each logical processor core) and populate (e.g., at runtime) Figure 6 250 in the hardware feedback data structure 250. In some examples, the predicted capability is for the current time. In some examples, the predicted performance capability (Perf Cap) and / or the predicted energy efficiency capability (EE Cap) is generated (and populated in the hardware feedback data structure 250) for each logical processor core and / or for each category (e.g., Category D 600, Category 1 601, Category 2 602, Category 3 603, etc.).

[0145] In some examples, an operating system (e.g., an OS scheduler) is used to select between using a predicted performance capability (PerfCap) and / or a predicted energy efficiency capability (EE Cap) to schedule threads on a particular logical processor (LP) (e.g., an LP core) (e.g., depending on parameters such as a power policy, a battery slider, etc.).

[0146] In some examples, the operating system can determine the index of the logical processor entry within the hardware feedback data structure 250 (e.g., the thread leader table) by executing a CPU identification (CPUID) instruction on the logical processor (e.g., using the corresponding ID value of CPUID.06H.0H:EDX[31:16] returned to the logical processor). Check / trigger for SMT core isolation

[0147] Certain examples herein implement the dynamic SMT scheduling disclosed herein, such as to provide core isolation via forced core parking of logical SMT peer processors when needed (e.g., when necessary), for example, while allowing a less restrictive (e.g., "small or idle") scheduling policy for user-initiated background tasks (e.g., compilers / renderers, etc.) running on the system to utilize SMT physical processor cores (e.g., performance cores (P cores)) (e.g., big cores). For example, to avoid completely disabling simultaneous multithreading (SMT) and / or processing background tasks only on less powerful (e.g., non-SMT) physical processor cores (e.g., E cores) and / or because some applications spawn threads based on logical core counts rather than just physical core counts (e.g., the OS scheduler does not have a physical core count).

[0148] In some examples, the determination of when to deliver core isolation depends on (i) the utilization of foreground tasks and thread concurrency (e.g., threads of foreground applications, e.g., Figure 1 Application 1 code 168 in (ii) overall workload characteristics based on package power and system-wide processor core utilization (e.g., where the Advanced Configuration and Power Interface (ACPI) standard "C0" operating state utilization percentage is referred to as "C0%").

[0149] Figure 7 700 is a flowchart illustrating a method of performing dynamic synchronous multithreading (SMT) scheduling (e.g., including SMT core isolation) according to some examples. Some or all of operation 700 (or other processes described herein, or variants and / or combinations thereof) are performed under the control of one or more computer systems configured with executable instructions and implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is jointly executed on one or more processors, by hardware or a combination thereof. The code is stored on a computer-readable storage medium, for example, in the form of a computer program including instructions executable by one or more processors. The computer-readable storage medium is non-transitory. An example is that a machine (e.g., a processor) can use EPP information to determine which codes are high priority / high QoS and which are low priority / low QoS, and if these tasks are to be scheduled on the SMT peer of the same physical core, isolation of higher priority / higher QoS code is provided.

[0150] Operation 700 includes determining, at block 702, whether an application (e.g., an application requesting an operating system to execute a thread on a processing system) is a foreground application. In some examples, such determination at block 702 includes checking whether the application has a class of service (CLOS) below a threshold (e.g., stored in a CLOS register of a processor (e.g., in IA32_PQR_ASSOC MSR (e.g., 0xC8F)), for example, where a CLOS value below the threshold (e.g., CLOS=0) means that it is a foreground application (e.g., with high quality of service (high QoS)), for example, and a CLOS value above the threshold means that it is not a foreground application (e.g., it is a background application). In some examples, such determination at block 702 includes checking whether the application has an energy performance preference (EPP) value (e.g., stored in a hardware-controlled performance state (HWP) register (e.g., 0198H)) below a threshold, for example, where an EPP value below the threshold means it is a foreground application, and an EPP value above the threshold means it is not a foreground application (e.g., it is a background application). In some examples, if the application (e.g., an application requesting an operating system to execute a thread on a processing system) is not a foreground application, operation 700 stops (e.g., until another application requests an operating system to execute a thread on the processing system), and if it is a foreground (FG) application, operation 700 continues to block 704.

[0151] Operations 700 also include, at block 704, determining whether the foreground application is CPU intensive (e.g., whether the foreground application uses more than a threshold number (e.g., a single) of logical processor cores), and if not, proceeding back to block 702, and if yes, proceeding to block 706. In some examples, such determination at block 704 includes checking whether the average CPU utilization (e.g., the application's C0) of the application (e.g., as tracked by performance monitoring circuitry 172) is greater than a threshold number of logical processor cores, e.g., greater than 100% of the logical processor cores.

[0152] Operations 700 also include, at block 706, determining whether the foreground application is lightly threaded (e.g., whether the foreground application uses less than or equal to the number of physical cores that support multithreading (e.g., SMT P-cores)), and if not, proceeding back to block 702, and if so, proceeding to block 708. In another example, instead of proceeding to block 708, operations proceed to block 710 for core isolation, e.g., where block 708 is optional or not included. In some examples, such a determination at block 706 includes checking whether the concurrency of the foreground application (e.g., the number of threads to be executed concurrently by the application) is less than the SMT core count (e.g., the SMT core count determined from a status register (e.g., MSR 0x35)).

[0153] Operation 700 also includes, at box 708, determining whether the system workload is persistent (e.g., whether there are background activities (e.g., (one or more) background applications) that will compete with the foreground application for cores) based on package power and / or CPU utilization (e.g., system-wide C0%), and if not, returning to box 702, and if so, proceeding to box 710.

[0154] Operations 700 also include, at block 710 , applying SMT core isolation.

[0155] In some examples, the SMT core isolation at block 710 includes disabling all but one logical core of each SMT physical core (e.g., of all SMT physical cores of the system), e.g., the remainder of those logical cores of a single physical core are referred to as "peers" of the one (non-disabled) logical core. Figure 4 As an example, in some examples, this may disable (eg, not allow use of) logical cores 109_P1B, 109_P2B, through 109_P(X)B.

[0156] In some examples, the SMT core isolation at block 710 includes disabling sibling logical cores of only those SMT physical cores to be used by the foreground application (e.g., not disabling sibling logical cores of all SMT physical cores of the system). Figure 4 As an example, in some examples, the foreground application uses only logical core 109_P1A of physical core 109_P1, and thus the operation at block 710 may include disabling (e.g., not allowing use of) logical core 109_P1B, but not disabling logical cores 109_P2B through 109_P(X)B. Figure 4As another example, in some examples, the foreground application only uses logical core 109_P1A of physical core 109_P1 and logical core 109_P2A of physical core 109_P2, and therefore the operation at box 710 may include disabling (e.g., not allowing use of) logical core 109_P1B and logical core 109_P2B, but not disabling logical core 109_P(X)B.

[0157] In some examples, SMT core isolation is triggered (e.g., at block 710) for a request (e.g., a request to schedule a thread of an application) by checking: Is X% < the utilization of the foreground application? <Y% Where X and Y represent foreground utilization thresholds between which the SMT peer logical core can be parked during sustained workloads. In some examples, if the foreground application's usage falls within this range, the work of the foreground application will not overflow to the SMT peer logical core, e.g., so that the SMT peer can be parked to improve performance. In some examples, the check also includes checking "C0" on the platform (e.g., where C0 is the active time of the core / CPU) and whether the system (e.g., SoC) workload detection based on package power is sustained, thereby, for example, indicating sustained background activity that may affect the responsiveness of the foreground application.

[0158] Referring again to the example of computer system 100, which includes six SMT physical processor cores 401 of a first type (e.g., 12 logical processor cores) and eight non-SMT physical processor cores 402 of a second type, so that for such computer system 100 there are a total of 14 (6+8) physical processor cores, but 20 (12+8) logical processor cores, the trigger for SMT core isolation is to check whether the foreground application utilization (e.g., C0%) is between 100% usage of 1 thread and 100% usage of 14 threads when thread concurrency is <14 and there is continued background activity, and if the check passes, appropriate action is taken to park the SMT peers to improve foreground performance during concurrent workloads.

[0159] In some examples, the trigger for SMT core isolation is a check if the following condition is met: 100% of 1 thread < foreground application utilization < 100% of (total physical cores, e.g., via MSR 0x35), and a check is made for ongoing background activity, and if that check passes, appropriate action is taken to park the SMT peer to improve foreground performance during concurrent workloads. SMT Core Isolation

[0160] In some examples, after determining that SMT core isolation is triggered, SMT core isolation is achieved by configuring platform-specific trigger(s) and action(s) (e.g., disabling all logical cores except one logical core on one or more SMT physical core sets). In some examples, after determining that SMT core isolation is triggered, SMT core isolation is achieved by updating a runtime core parking configuration on a platform (e.g., a computer system) (e.g., disabling all logical cores except one logical core on an SMT physical core).

[0161] In some examples, SMT core parking is implemented by updating runtime processor power management configuration settings (e.g., of an OS) to implement SMT core isolation. In some examples, such forced core parking of sibling logical processor cores of an SMT physical processor core is implemented by limiting the number of logical processors (e.g., CPUs) available for scheduling, for example, by setting a corresponding value to a control value 164 (e.g., a "CPMaxCores" value) of OS 160 (e.g., a processor power management (PPM) control value), for example, a control value representing a maximum percentage of processors that are not parked on the platform. In some examples, this includes setting control value 164 (e.g., CPMaxCores) = (number of physical cores / total number of threads) * 100.

[0162] Referring again to the example of computer system 100, which includes six SMT physical processor cores 401 of a first type (e.g., 12 logical processor cores) and eight non-SMT physical processor cores 402 of a second type, such that for such computer system 100 there are a total of 14 (6+8) physical processor cores but 20 (12+8) logical processor cores, setting control value 164 (e.g., CPMaxCores) to 70% = (14 / 20)*100 will prevent OS 160 (e.g., OS scheduler 162) from scheduling on the remaining 30% (i.e., 6) SMT peers.

[0163] In some examples, SMT core isolation (e.g., core parking) is implemented via hardware (e.g., thread runtime telemetry circuitry 116 (e.g., (i) hardware guided scheduler (HGS) (or HGS+) circuitry or (ii) thread guide circuitry)). In some examples, SMT core isolation (e.g., core parking) is implemented by utilizing hardware guided scheduling for each logical thread entry. In some examples, hardware is used to make a hint (or other value) readable by the OS to avoid (e.g., not use) SMT sibling cores (e.g., even if they are actually available to perform the work). In some examples, a processor (e.g., via a non-transitory machine-readable medium storing power management code (e.g., p-code) is used to cause thread runtime telemetry circuitry 116 (e.g., (i) hardware guided scheduler (HGS) (or HGS+) circuitry or (ii) thread guide circuitry) to implement SMT core isolation (e.g., core parking), for example, by modifying a value in a hardware feedback data structure 250. Reference Figure 6 , in some examples, if the SMT physical core implements logical processor (LP) (e.g., logical processor core) 0 and logical processor (LP) (e.g., logical processor core) 1, and it is desired to disable logical processor (LP) (e.g., logical processor core) 1 (and not disable LP 0), then the LP 1 row ( Figure 6 0 (e.g., in all categories or a subset of applicable categories). In some examples, an indication to use LP 0 may also be written to hardware feedback data structure 250 to fully enable (e.g., encourage) the use of LP0, for example, to LP 0 row ( Figure 6 , the first row in the table in FIG. 1 ) to perform a corresponding write (e.g., a maximum value, e.g., 255) with a predicted performance capability (Perf Cap) and / or a predicted energy efficiency capability (EE Cap) (e.g., in all categories or a subset of applicable categories).

[0164] An example of a hardware feedback data structure 250 being used for telemetry data (e.g., capability values) is discussed above, however, it should be understood that telemetry data (e.g., capability values) may be sourced in other ways (e.g., directly from the hybrid scaling predictor 240), for example, and telemetry data therefrom may be modified in accordance with the present disclosure to implement SMT core isolation (e.g., core parking).

[0165] Figure 8800 is a flowchart illustrating another method of performing dynamic synchronous multithreading (SMT) scheduling according to some examples. Some or all of operation 800 (or other processes described herein, or variations and / or combinations thereof) are performed under the control of one or more computer systems configured with executable instructions and implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed together on one or more processors, through hardware or a combination thereof. The code is stored on a computer-readable storage medium, for example, in the form of a computer program that includes instructions that can be executed by one or more processors. The computer-readable storage medium is non-transitory.

[0166] Operation 800 includes: at block 802, receiving a request to execute a set of threads of a foreground application on a hardware processor, the hardware processor including a first plurality of physical processor cores of a first type and a second plurality of physical processor cores of a second type, the first plurality of physical processor cores of the first type implementing a plurality of logical processor cores of a first type, wherein each core of the second type implements a plurality of logical processor cores of a second type. Operation 800 also includes: at block 804, determining whether the set of threads of the foreground application is to use a number of logical processor cores greater than a threshold number and less than or equal to the total number of the first plurality of physical processor cores of the first type and the second plurality of physical processor cores of the second type. Operation 800 also includes: at block 806, in response to determining that the set of threads of the foreground application is to use a number of logical processor cores greater than a threshold number and less than or equal to the total number of the first plurality of physical processor cores of the first type and the second plurality of physical processor cores of the second type, disabling a second logical core of a physical processor core of the second type, and not disabling a first logical core of a physical processor core of the second type.

[0167] Fig. 9 The figure illustrates an example of a processing device. The processing device includes at least one or more of the following: a system on a chip (SOC) 922, a processor complex 912, one or more accelerators 902, a power management controller 938 and / or 940, etc. In some examples, the above aspects are connected to a die-to-die (D2D) interconnect 900. In some examples, some SOCs 922 include additional components, such as more cores, accelerators, etc. Note that the "boxes" of one or more accelerators 902, processor complexes 912, and SOCs 922, etc. can be redrawn. For example, one or more of the accelerators can be part of the SOC 922 or the processor complex 912, etc. (e.g., see (one or more) accelerators 926).

[0168] The one or more accelerators 902 may include one or more of the following: one or more graphics processing units (GPUs) 903, one or more matrix processing units 904, one or more data flow accelerators 905 (e.g., for storage network, and / or data intensive workloads such as copying and transforming data), one or more in-memory analytics accelerators 906 (e.g., for performing compression and decompression in conjunction with analytics), one or more data encryption and compression accelerators 907, and / or other accelerators 908 that may include, but are not limited to, vision, machine learning model specific hardware (e.g., accelerators for transformer-based models), field programmable gate arrays (FPGAs), etc.

[0169] The processor complex 912 may include one or more performance cores 914 and one or more high-efficiency cores 916. In some examples, the one or more performance cores 914 and the one or more high-efficiency cores 916 share an instruction set architecture. In some examples, the one or more performance cores 914 and the one or more high-efficiency cores 916 have different instruction set architectures (although there may be some overlap). A DMU 918 (e.g., data management unit) that runs "dcode" firmware and / or software may also be included.

[0170] The SoC 922 includes at least one or more of the following: a power control unit 924 (PCU or p-unit) that runs pcode (p-code), one or more accelerators 926, one or more CPU cores 928 (e.g., high-efficiency cores), a memory controller and / or memory 930, one or more display controllers and connections 932 (e.g., display drivers, ports, etc.), and / or one or more interconnect controllers and / or connections 934.

[0171] In some examples, a power management controller (e.g., power management controller (PMC) 938 or 940) controls the supply of power to components such as processor cores, accelerators, etc. The PMC may have access to one or more storage devices for storing information related to the operation of the logic. The PMC may be coupled to a voltage regulator (VR) and / or other components of the system (such as a core). For example, the PMC may be coupled to receive information (e.g., in the form of one or more bits or signals) indicating the state of one or more sensors 936 (where (one or more) sensors (e.g., thermal diodes) may be close to components of the system (or other computing systems discussed herein) (such as core interconnects, etc.) to sense changes in temperature, operating frequency, operating voltage, power consumption, inter-core communication activity, etc.) and / or information from one or more power monitoring logics (e.g., the information may indicate the operating state of various components of the system, such as operating temperature, operating frequency, operating voltage, operating state (e.g., active or inactive), power consumption (immediately or over a period of time), etc.). The PMC may issue instructions to the VR, power source, and / or various components of the system to modify their operation. In embodiments, changes may be sensed in such a manner that takes into account leakage and active power. For example, the PMC may instruct the VR and / or power source to adjust its output. In some embodiments, the PMC may request a core, etc. to modify its operating frequency, power consumption, etc. Note that a processor complex, accelerator, etc. may include a PMC.

[0172] In some examples, at least a subset of the following share a unified memory (eg, shared virtual memory): processor complex 912 , SOC 922 , and one or more accelerators 902 .

[0173] Fig.10 The figure shows an example of a block diagram of a portion of a system. Fig.10 As shown in , system 1000 may include various components, including processor 1010, which is shown as a multi-core processor. Processor 1010 may be coupled to power supply 1050 via external voltage regulator 1060, which may perform a first voltage conversion to provide a main regulated voltage to processor 1010.

[0174] As can be seen, the processor 1010 may include multiple cores 1020 a –1020 n In addition, each core can be connected to an integrated voltage regulator (IVR) 1025 a –1025 n Associated, IVR1025 a –1025n A main regulated voltage is received and an operating voltage is generated to be provided to one or more agents of a processor associated with the IVR. Thus, an IVR implementation can be provided that allows fine-grained control of voltage, and thereby fine-grained control of power and performance of each individual core. Thus, each core can operate at an independent voltage and frequency, thereby achieving great flexibility and providing a wide range of opportunities for balancing power consumption and performance. In some embodiments, the use of multiple IVRs enables grouping of components into separate power planes, such that power is regulated by the IVRs and power is provided by the IVRs only to those components in the group. During power management, when a processor is placed in a low power state, a given power plane of one IVR may be powered down or powered off, while another power plane of another IVR remains active or fully powered.

[0175] Still refer to Fig.10 , there may be additional components within the processor, including an input / output interface 1032, another interface 1034, and an integrated memory controller 1036. As can be seen, each of these components may be provided by another integrated voltage regulator 1025 x In one embodiment, the interface 1032 can be used to The operation of the Quick Path Interconnect (QPI) interconnect provides a point-to-point (PtP) link in a cache coherence protocol that includes multiple layers, including a physical layer, a link layer, and a protocol layer. In turn, the interface 1034 can be connected via a Peripheral Component Interconnect Express (PCIe TM ) protocol to communicate.

[0176] Also shown is a power control unit (PCU) 1038, which may include hardware, software, and / or firmware for performing power management operations related to the processor 1010. As seen, the PCU 1038 provides control information to an external voltage regulator 1060 via a digital interface to cause the voltage regulator to generate an appropriate regulated voltage. The PCU 1038 also provides control information to the IVR 1025 via another digital interface to control the generated operating voltage (or to cause the corresponding IVR to be disabled in a low power mode). In various embodiments, the PCU 1038 may include various power management logic units for performing hardware-based power management. Such power management may be completely processor controlled (e.g., controlled by various processor hardware, and it may be triggered by workload and / or power, thermal constraints, or other processor constraints), and / or power management may be performed in response to an external source (such as, a platform or management power management source or system software).

[0177] In embodiments herein, PCU 1038 may be configured to control a domain within processor 1010 to operate at one of a plurality of different operating points. More specifically, PCU 1038 may determine operating points for different domains. Additionally, it should be understood that a fast mechanism (e.g., external to PCU 1038) may be involved in performing throttling operations in response to platform events. Thus, if Fig.10 As further illustrated in , throttling circuitry 1039 (which in some cases may be a distributed hardware circuit) may further communicate with IVR 1025 (and / or core 1020 itself). In this manner, throttling circuitry 1039 may translate processor throttling power levels to corresponding throttling levels for various domains (such as core 1020, etc.) based at least in part on translation information received from PCU 1038, as further described herein.

[0178] Although not shown for ease of illustration, it is to be understood that additional components may be present within processor 1010, such as uncore (external to the core) logic and other components such as internal memory, e.g., one or more levels of a cache memory hierarchy, etc. Fig.10 The implementation of is shown as having an integrated voltage regulator, but the embodiments are not limited thereto.

[0179] The processor described herein may utilize power management techniques that may be independent of and complementary to an operating system (OS)-based power management (OSPM) mechanism. According to an example OSPM technique, a processor may operate in various performance states or levels (so-called P states, i.e., from P0 to PN). In general, a P1 performance state may correspond to the highest guaranteed performance state that an OS can request. In addition to this P1 state, an OS may also request a higher performance state, i.e., a P0 state. This P0 state may thus be a speculative or turbo mode state in which, when power and / or thermal budgets are available, the processor hardware may configure the processor or at least a portion thereof to operate at a frequency higher than the guaranteed frequency. In many implementations, a processor may include multiple so-called bin frequencies that are higher than the P1 guaranteed maximum frequency, exceed the maximum peak frequency of a particular processor, such as burned in or otherwise written into the processor during manufacturing. In addition, according to an OSPM mechanism, a processor may operate in various power states or levels. In terms of power states, the OSPM mechanism may specify different power consumption states, generally referred to as C-states, C0, C1 to Cn states. When a core is active, it runs in the C0 state, and when a core is idle, it may be placed in a core low power state, also referred to as a core non-zero C-state (e.g., C1-C6 states), where each C-state is at a low power consumption level (such that C6 is a deeper low power state than C1, etc.).

[0180] It is to be understood that many different types of power management techniques may be used, alone or in combination, in different embodiments. As a representative example, a power controller may control a processor to be power managed via some form of dynamic voltage frequency scaling (DVFS), in which the operating voltage and / or operating frequency of one or more cores or other processor logic may be dynamically controlled to reduce power consumption in certain situations. In an example, DVFS may use Enhanced Intel SpeedStep 4 available from Intel Corporation of Santa Clara, California. TM In another example, DVFS can use Intel TurboBoost TM The technology is performed to enable one or more cores or other computing engines to operate at an operating frequency higher than the guaranteed operating frequency based on conditions (e.g., workload and availability).

[0181] Fig.11 The figure shows an example of a system. Fig.11 As illustrated, in one embodiment, system 1100 includes a processor 1110, which in an embodiment may be a heterogeneous multi-core processor coupled to a system memory 1160 implemented as DRAM. As further illustrated, an OS 1170 executes on processor 1110. Processor 1110 includes at least a single core 1120, which is shown for representative purposes. Core 1120 is coupled to PCU 1130 via a mailbox interface 1150, which in an embodiment may be implemented as a microcode mailbox interface. Of course, other interfaces for implementing communication between core 1120 and PCU 1130 are also possible.

[0182] As further illustrated, processor 1110 also includes a register set 1140. Registers 1140 may include various MSRs, status registers, configuration registers, etc. Fig.11 In the particular embodiment shown, such registers include a CPUID register 1142. Notably, the CPUID register 1142 may include a hardware feedback field that, when set, indicates that the processor is capable of providing dynamic hardware feedback information to the OS. The hardware feedback pointer MSR 1144 may include a hardware feedback pointer field for storing a pointer written by the OS to a location in memory where the hardware feedback information structure is to be stored. The hardware feedback configuration MSR 1146 may store a hardware feedback configuration enable field that, when set, indicates that the processor is used to perform the hardware feedback processing described herein. The package thermal interrupt register 1148 may include a hardware feedback notification enable indicator for enabling the triggering of an interrupt in response to an update of the hardware feedback information. And finally, the package thermal status register 1149 may include a hardware feedback notification log indicator that may be set when the PCU 1130 writes updated hardware feedback information and may be reset by the OS when the OS reads the updated information.

[0183] Still reference Fig.11 , PCU 1130 includes a hardware feedback control 1135 (e.g., as part of a PCODE), which can calculate hardware feedback information and trigger writing of the information to memory 1160. More specifically, the information can be passed to core 1120 via mailbox interface 1150. In turn, the microcode of core 1120 can receive the hardware feedback information and write it to system memory 1160, and then send a confirmation of writing the information to PCU 1130.

[0184] Note that in other embodiments, a direct interface may be provided to enable the PCU 1130 to write the hardware feedback information directly to the system memory 1160. In an embodiment, the hardware feedback information may be temporarily cached as write-back information, for example, in the last level cache of the processor, before actually being written to the memory. Thus, the OS 1170 may access the hardware feedback information from the LLC (if present) or from the memory 1160. It should be understood that although in Fig.11 The embodiments are shown at this high level, but many variations and alternatives are possible.

[0185] In some examples, the system includes multiple hybrid core layers (e.g., Fig. 9 912). In some examples, there are differences in the power cores and / or efficiency cores presented (e.g., a first type of power core and a second type of power core, etc.). In some cases, the cores on the SoC die (low power domain) are expected to be more efficient than the compute die (e.g., processor complex 912) efficiency cores only when the workload is fully "fitted" on the SOC core ((one or more) CPU cores 928), so that the compute die can remain powered off for a long time. It is expected that some common workloads (such as video playback) will gain platform-level battery life improvements by running exclusively on the SOC core 928.

[0186] In some examples, the hint in telemetry data format 500A and / or 500B is defined as a request that the OS consolidate all energy efficiency oriented tasks onto cores with energy efficiency capability=255.

[0187] In some examples, the hint in telemetry data format 500A and / or 500B is defined as a request that the OS consolidate performance-oriented tasks onto cores with performance capability=255.

[0188] In some examples, telemetry data formats 500A and / or 500B include other capabilities for each category, such as power, latency (e.g., memory bandwidth), compute type (e.g., accelerator, accelerator type (such as matrix), etc.). In some examples, the value of this field indicates that the operating system should perform an action (such as, use a lower power core with a value, use a higher power core, avoid cores with low memory bandwidth, use cores with high memory bandwidth, use a specific type of accelerator, etc.).

[0189] Fig.12 The figure shows an example of task movement. On the left hand side, the task is shown as running on the high power core 1203. The thread runtime telemetry circuitry 116 is updated (in some examples, by the PCODE) to provide a hint to the OS 1201 to merge the task to the low power core.

[0190] On the right hand side, tasks are shown as being migrated by the OS 1201 to the low power core 1205. Note that this is an example and tasks may go from low power to high power, from general purpose cores to accelerators, from accelerators to performance cores, etc.

[0191] In some examples, there will be restrictions on the use of SOC cores 928, only scenarios where all work fits on these SOC cores 928 so that the compute die can be powered off. In some examples, the exception to this behavior is for scenarios where SOC cores 928 are used in highly multithreaded scenarios where there are more threads than can run on the available cores on the compute die.

[0192] In some examples, the performance capabilities and energy efficiency capabilities of natural computing may bias OS usage toward compute die cores.

[0193] In some examples, the PCODE monitors activity on the compute die cores over time to detect operating windows in which all work will fit on the SOC cores.

[0194] In some examples, when the PCODE determines that all work on the system will fit on the SOC die core and the workload is expected to be stable, the next telemetry data update (of the thread runtime telemetry circuitry 116) will be populated with a special hint (described in detail above) that requests the OS to merge all energy efficiency oriented work to run exclusively on the SOC core. For example, in some instances, the special HFI replaces the energy efficiency capability of the SOC core with a value of 255 for class 0 and class 3 workloads. All other table values ​​will be calculated as defined elsewhere in this document.

[0195] In some examples, it is expected that class 1 and class 2 work will be more efficient when run on the compute die core. Therefore, in some examples, the EE=255 hint will only apply to class 0 and class 3 workloads.

[0196] In some examples, when the PCODE firmware determines that the work no longer "fits" on the SOC core, the next telemetry data table update will be populated with the normally calculated values ​​of the SOC core. Work no longer fits when the operating frequency or voltage of the SOC core is pushed above a predetermined threshold.

[0197] For both the "fit" and "don't fit" cases, in some examples, a hysteresis window can be applied to create inertia to minimize changes to the operating system's prompts.

[0198] Fig.13An example of a method is shown. In some examples, PCODE and / or DCODE performs these actions. In some examples, thread runtime telemetry circuitry provides telemetry feedback structure 250. In some examples, PCOD and / or DCODE updates telemetry feedback structure 250 using thread runtime telemetry circuitry. At 1301, the state of components of the system is maintained. For example, core usage, average VCC (VCCsa) of the system agent, C-states (e.g., ACPI states, such as processor C6 states), core and / or core die parking states, voltage minimums, etc.

[0199] At 1303, the highest operating point and number of cores (e.g., core concurrency) required for each thread classification (e.g., the categories discussed above with respect to telemetry feedback data structure 250) is determined, where low power island (e.g., SOC) modules are more efficient than all other computing modules. The highest operating point and cores can be considered hardware telemetry data. Other hardware telemetry data can include system operating characteristics, such as one or more of temperature (system and / or component), core power, input / output power, power state, etc.

[0200] The classification can be considered as thread runtime telemetry data. Other thread runtime telemetry data can include but is not limited to latency per class, power per class, ISA support, memory and / or cache topology, etc.

[0201] Based on the SoC and compute telemetry, the current operating point and number of active cores for each thread classification is determined at 1305 .

[0202] A determination is made at 1306 whether the hardware feedback telemetry data is to be updated. For example, whether work currently running on a high power plateau should be moved to a low power island and a prompt generated. In some examples, telemetry data maintained by thread runtime telemetry circuitry 116 is updated. The determination may be made based on one or more factors such as workload type (bursty, continuous, inferred, etc.), core availability, system power, system temperature, latency, concurrency, etc. The determination may also be based on a desired outcome (e.g., speed, energy efficiency, etc.).

[0203] In some examples, the workload type and / or characteristics are considered. In some examples, if the workload type is inferred, a determination is made as to whether the workload is bursty or sustained. For example, if the workload is a bursty or sustained workload, then hints should not be provided to the system software to consolidate the work onto a particular set of cores for energy efficiency. In these examples, both EE and performance are set to 00. In some examples, the class of the core is considered when determining when to generate hints. For example, a class 0 or class 3 core may be set, but for class 1 and class 2, the performance and EE information comes from the thread runtime telemetry circuitry.

[0204] In some examples, a bias is provided (a hint is provided) if the workload type is not inference, bursty, or sustained. In some examples, a hint is provided when the total core count of a particular type of system (e.g., efficiency cores) can be satisfied with only the cores in the SoC.

[0205] In some examples, it is determined that a certain amount of time needs to pass before the prompt is updated.

[0206] In some examples, thermal information is used in this determination. For example, if the thermal level of the system and / or a particular computing element is too high, a merge hint may be used to move threads.

[0207] In some examples, latency information is used in this determination. For example, if the latency value allows merging without adverse effects, a merge hint can be used to move the thread.

[0208] Fig.14 and Fig.15 An example of determining when to update telemetry data is illustrated.

[0209] Fig.14 The figure illustrates an example of interaction between actors in potential updates of telemetry data. As shown, compute telemetry 1401 is provided to DCODE 1403, which determines whether a core should be parked based on compute telemetry 1401. DCODE 1403 also determines the ACPI state of the processor complex. Note that the "watermarks" in the figure can include one or more of the following: performance core utilization, efficiency core maximum frequency, efficiency core average frequency, average efficiency core concurrency, system agent VMin (Vmin), and average VCCsa.

[0210] PCODE 1405 determines if the parking indication (park flag) has been set for a set number of cycles, and if the average VCCSA (provided by SOC 1407) is less than some minimum voltage. If this is true, then the hardware feedback telemetry 1409 is updated to indicate consolidation on the SOC cores (e.g., setting efficiency to 255). If not, the consolidation hint may be removed depending on the processor complex C-state. Note that a stable state may also occur without changing the hardware feedback telemetry 1409.

[0211] Fig.14 An example of pseudo code for compute (processor complex 912) migration is shown below. / *pCode stream and dCode stream run concurrently The averaging windows are different: pCode samples and averages at 16ms granularity, while dCode works at about 100ms granularity (note the example MS) The constants in the following pseudocode are adjustable parameters* / @dCode telemetry_aggregation(): 1.Performance core utilization 2.Efficiency core max frequency 3.Efficiency core avg frequency 4.Avg Efficiency core concurrency(tracked from C-state residency) decision_tree(); / / Note that the numbers are examples IF((Performance core utilization=0)& (Efficiency core max frequency<=1.8GHz)& (Efficiency core avg frequency<=1.6GHz)& (Avg Efficiency core concurrency<=2.2)) Set ConsolidateFlag Trigger pCode<->dCode mailbox @pCode Telemetry_aggregation(); 1.Avg VccSA 2.Accum ConsolidateFlag status decision_tree(); / / Note that the numbers are examples IF((ConsolidateFlag set for 5consecutive cycles)& (avg VccSA <Vmin+0.05v) Trigger Telemetry Data Update(consolidate work on SOC cores) / / Merge the work to the SOC die core->Set the EE of the SOC die core of category 0 and category 3 to = 255 / / If the OS does not put all the compute die cores in C6 within 100ms, reset the SOC chip work merge IF((Telemetry Data SOC die work consolidation status updated)&Computedie did not enter C6)) Trigger Telemetry Data Update(undo consolidate work on SOC cores)

[0212] Fig.15 The figure shows an example of the interaction between the actors in a potential update of telemetry data. As shown, PCODE 1505 determines whether the average VCCSA (provided by SOC 1507) is less than some minimum voltage, or whether the computing die (processor complex) is active. If so, the hardware feedback telemetry data 1509 is updated to remove the indication of merging on the SOC core (e.g., setting the efficiency to a value other than 255). It should be noted that a stable state without changing the hardware feedback telemetry data 1509 may also occur.

[0213] Fig.15 An example of pseudo code for compute (processor complex 912) migration is shown below. / / pCode is sampled at 16ms granularity and averaged / / The constants in the following pseudocode are adjustable parameters @dCode Die C6 exit flow @pCode Telemetry_aggregation(); 1.Avg VccSA(over~100ms time window) decision_tree(); IF(avg VccSA>Vmin+0.010v) Trigger Telemetry Data Update(undo consolidate work on SOC die cores) / / Set the performance and EE capabilities of all cores according to normal behavior IF (Cdie active for>16ms) Trigger Telemetry Data Update(undo consolidate work on SOC die

[0214] The (updated) telemetry data is provided to system software (e.g., OS, VMM, hypervisor, etc.) at 1307. In some examples, the hint provided by the updated telemetry data is a request for the OS to consolidate all energy efficiency oriented tasks onto cores with energy efficiency capability = 255. In some examples, the hint provided by the updated telemetry data is a request for the OS to consolidate performance oriented tasks onto cores with performance capability = 255.

[0215] In some examples, at 1307, the merge hint is reset based on telemetry data such as power state, voltage level, thermal considerations (e.g., thermal issues no longer exist), etc. For example, if the system software does not place the required processing device into a reduced power state. In some examples, the merge hint is deleted if the compute die has been active for a determined period of time. In some examples, the merge hint is deleted if the average VCCsa of the SoC is greater than a determined value.

[0216] Some examples utilize the instruction formats described herein. Some examples are implemented in one or more computer architectures, cores, accelerators, etc. Some examples are generated or are IP cores. Some examples utilize simulation and / or translation. Example Architecture

[0217] The example computer architecture is described in detail below. Other system designs and configurations for laptop computers, desktop computers, handheld personal computers (personal computers, PCs), personal digital assistants, engineering workstations, servers, decomposed servers, network devices, network hubs, switches, routers, embedded processors, digital signal processors (digitalsignal processors, DSPs), graphics devices, video game devices, set-top boxes, microcontrollers, cellular phones, portable media players, handheld devices, and various other electronic devices known in the art are also appropriate. In a word, various systems or electronic devices that can include processors and / or other execution logic disclosed herein are generally appropriate. Example System

[0218] Fig.16 An example computing system is illustrated. Multiprocessor system 1600 is an interfaced system and includes multiple processors or cores, including a first processor 1670 and a second processor 1680 coupled via an interface 1650 (e.g., a point-to-point (PP) interconnect, a fabric, and / or a bus). In some examples, first processor 1670 and second processor 1680 are isomorphic. In some examples, first processor 1670 and second processor 1680 are heterogeneous. Although example system 1600 is shown as having two processors, the system may have three or more processors, or may be a single processor system. In some embodiments, the computing system is a system on chip (SoC).

[0219] Processors 1670 and 1680 are shown as including integrated memory controller (IMC) circuitry 1672 and 1682, respectively. Processor 1670 also includes interface circuits 1676 and 1678; similarly, second processor 1680 includes interface circuits 1686 and 1688. Processors 1670, 1680 may exchange information via interface 1650 using interface circuits 1678, 1688. IMCs 1672 and 1682 couple processors 1670, 1680 to respective memories, namely memory 1632 and memory 1634, which may be part of the main memory locally attached to the respective processors.

[0220] Processors 1670, 1680 can each utilize interface circuits 1676, 1694, 1686, 1698 to exchange information with a network interface (NW I / F) 1690 via respective interfaces 1652, 1654. Network interface 1690 (e.g., one or more of an interconnect, bus, and / or structure, a chipset in some examples) can optionally exchange information with a coprocessor 1638 via interface circuit 1692. In some examples, coprocessor 1638 is a special-purpose processor, such as a high-throughput processor, a network or communication processor, a compression engine, a graphics processor, a general-purpose graphics processing unit (GPGPU), a neural-network processing unit (NPU), an embedded processor, and the like.

[0221] A shared cache (not shown) may be included in either processor 1670, 1680, or may be external to both processors but connected to the processors via an interface (e.g., a PP interconnect) such that if a processor is placed in a low power mode, local cache information of either or both processors may also be stored in the shared cache.

[0222] The network interface 1690 can be coupled to the first interface 1616 via the interface circuit 1696. In some examples, the first interface 1616 can be an interface such as a Peripheral Component Interconnect (PCI) interconnect, a PCI Express interconnect, or another I / O interconnect. In some examples, the first interface 1616 is coupled to a power control unit (PCU) 1617, which can include circuit systems, software, and / or firmware to perform power management operations with respect to the processors 1670, 1680, and / or the coprocessor 1638. The PCU 1617 provides control information to a voltage regulator (not shown) so that the voltage regulator generates an appropriate regulated voltage. The PCU 1617 also provides control information to control the generated operating voltage. In various examples, the PCU 1617 can include various power management logic units (circuitry systems) to perform hardware-based power management. Such power management may be entirely processor controlled (e.g., controlled by various processor hardware and may be triggered by workload and / or power constraints, thermal constraints, or other processor constraints), and / or power management may be performed in response to an external source (e.g., a platform or power management source or system software).

[0223] PCU 1617 is illustrated as existing as logic separate from processor 1670 and / or processor 1680. In other cases, PCU 1617 may execute on a given one or more cores of a core (not shown) of processor 1670 or 1680. In some cases, PCU 1617 may be implemented as a microcontroller (dedicated or general purpose) or other control logic that is configured to execute its own dedicated power management code (sometimes referred to as P-code). In still other examples, the power management operations to be performed by PCU 1617 may be implemented outside the processor, such as by a separate power management integrated circuit (PMIC) or another component external to the processor. In still other examples, the power management operations to be performed by PCU 1617 may be implemented within a BIOS or other system software.

[0224] Various I / O devices 1614 and a bus bridge 1618 can be coupled to the first interface 1616, and the bus bridge 1618 couples the first interface 1616 to the second interface 1620. In some examples, (one or more) additional processors 1615 are coupled to the first interface 1616, such as a coprocessor, a high-throughput many integrated core (MIC) processor, a GPGPU, an accelerator (such as a graphics accelerator or a digital signal processing (DSP) unit), a field programmable gate array (FPGA), or any other processor. In some examples, the second interface 1620 can be a low pin count (LPC) interface. Various devices can be coupled to the second interface 1620, including, for example, a keyboard and / or mouse 1622, a communication device 1627, and a storage circuit system 1628. Storage circuitry 1628 may be one or more non-transitory machine-readable storage media described below, such as a disk drive or other mass storage device, which in some examples may include instructions / code and data 1630 and may implement storage device `ISAB03. Additionally, audio I / O 1624 may be coupled to second interface 1620. Note that other architectures are possible in addition to the point-to-point architecture described above. For example, a system such as multiprocessor system 1600 may implement a multi-drop interface or other such architecture instead of a point-to-point architecture.

[0225] Example core architectures, processors, and computer architectures.

[0226] Processor cores can be implemented in different ways, for different purposes, and in different processors. For example, implementations of these cores may include: 1) general-purpose in-order cores for general-purpose computing purposes; 2) high-performance general-purpose out-of-order cores for general-purpose computing purposes; 3) special-purpose cores, mainly for graphics and / or scientific (throughput) computing purposes. Implementations of different processors may include: 1) a CPU including one or more general-purpose in-order cores for general-purpose computing purposes and / or one or more general-purpose out-of-order cores for general-purpose computing purposes; and 2) a coprocessor including one or more special-purpose cores mainly for graphics and / or scientific (throughput) computing purposes. These different processors result in different computer system architectures, which may include: 1) coprocessors on separate chips from the CPU; 2) coprocessors on separate dies in the same package as the CPU; 3) coprocessors on the same die as the CPU (in which case such coprocessors are sometimes referred to as dedicated logic, such as integrated graphics and / or scientific (throughput) logic, or as dedicated cores); and 4) systems on a chip (SoCs), which may include the described CPU (sometimes referred to as application core(s) or application processor(s), the coprocessors described above, and additional functionality on the same die. An example core architecture is described next, followed by a description of an example processor and computer architecture.

[0227] Fig.17 A block diagram of an example processor and / or SoC 1700 is illustrated, which may have one or more cores and have an integrated memory controller. The processor 1700 illustrated in solid line blocks has a single core 1702(A), a system agent unit circuitry 1710, and a set of one or more interface controller unit circuitry 1716, while the optionally added dashed line blocks illustrate an alternative processor 1700 as having multiple cores 1702(A)-(N), a set of one or more integrated memory control unit circuitry 1714 in the system agent unit circuitry 1710, dedicated logic 1708, and a set of one or more interface controller unit circuitry 1716. Note that the processor 1700 may be Fig.16 One of the processors 1670 or 1680 or the coprocessors 1638 or 1615.

[0228] Thus, different implementations of processor 1700 may include: 1) a CPU, where dedicated logic 1708 is integrated graphics and / or scientific (throughput) logic (which may include one or more cores, not shown), and cores 1702(A)-(N) are one or more general purpose cores (e.g., general purpose in-order cores, general purpose out-of-order cores, or a combination of the two); 2) a coprocessor, where cores 1702(A)-(N) are a large number of dedicated cores primarily for graphics and / or scientific (throughput) purposes; and 3) a coprocessor, where cores 1702(A)-(N) are a large number of general purpose in-order cores. Thus, processor 1700 may be a general purpose processor, a coprocessor, or a special purpose processor, such as a network or communication processor, a compression engine, a graphics processor, a GPGPU (general purpose graphics processing unit), a high throughput many integrated core (MIC) coprocessor (including 30 or more cores), an embedded processor, etc. The processor may be implemented on one or more chips. Processor 1700 may be part of one or more substrates and / or may be implemented on one or more substrates using any of a variety of process technologies, such as complementary metal oxide semiconductor (CMOS), bipolar CMOS (BiCMOS), P-type metal oxide semiconductor (PMOS), or N-type metal oxide semiconductor (NMOS).

[0229] The memory hierarchy includes one or more levels of cache unit circuitry 1704(A)-(N) within the cores 1702(A)-(N), a set of one or more shared cache unit circuitry 1706, and external memory (not shown) coupled to the set of integrated memory controller unit circuitry 1714. The set of one or more shared cache unit circuitry 1706 may include one or more intermediate level caches, such as level 2 (L2), level 3 (L3), level 4 (4), or other levels of cache, such as last level cache (LLC), and / or combinations thereof. While in some examples an interface network circuitry 1712 (e.g., a ring interconnect) provides an interface to dedicated logic 1708 (e.g., integrated graphics logic), the set of shared cache unit circuitry 1706, and the system agent unit circuitry 1710, alternative examples use any number of well-known techniques to provide interfaces to these units. In some examples, coherency is maintained between the (one or more) shared cache unit circuitry 1706 and one or more of the cores 1702(A)-(N). In some examples, interface controller unit circuitry 1716 couples the cores 1702 to one or more other devices 1718, such as one or more I / O devices, storage, one or more communication devices (e.g., wireless network, wired network, etc.), and the like.

[0230] In some examples, one or more of the cores 1702(A)-(N) have multi-threading capabilities. System agent unit circuitry 1710 includes those components that coordinate and operate the cores 1702(A)-(N). System agent unit circuitry 1710 may include, for example, a power control unit (PCU) circuitry and / or a display unit circuitry (not shown). The PCU may be (or may include) logic and components required to regulate the power state of the cores 1702(A)-(N) and / or the dedicated logic 1708 (e.g., integrated graphics logic). The display unit circuitry is used to drive one or more externally connected displays.

[0231] The cores 1702(A)-(N) may be homogeneous with respect to instruction set architecture (ISA). Alternatively, the cores 1702(A)-(N) may be heterogeneous with respect to ISA; that is, a subset of the cores 1702(A)-(N) may be capable of executing one ISA, while other cores may be capable of executing only a subset of the ISA or may be capable of executing another ISA.

[0232] Fig.181800 is a block diagram illustrating a computing system 1800 configured to implement one or more aspects of the examples described herein. The computing system 1800 includes a processing subsystem 1801 having one or more processors 1802 and a system memory 1804 communicating via an interconnect path, which may include a memory hub 1805. The memory hub 1805 may be a separate component within a chipset component, or may be integrated within one or more processors 1802. The memory hub 1805 is coupled to an I / O subsystem 1811 via a communication link 1806. The I / O subsystem 1811 includes an I / O hub 1807, which may enable the computing system 1800 to receive input from one or more input devices 1808. In addition, the I / O hub 1807 may enable a display controller (which may be included in one or more processors 1802) to provide output to one or more display devices 1810A. In some examples, one or more display devices 1810A coupled to I / O hub 1807 may include local, internal, or embedded display devices.

[0233] The processing subsystem 1801 includes, for example, one or more parallel processors 1812 coupled to a memory hub 1805 via a bus or other communication link 1813. The communication link 1813 can be one of any number of standard-based communication link technologies or protocols, such as but not limited to PCI Express, or can be a vendor-specific communication interface or communication fabric. The one or more parallel processors 1812 can form a parallel or vector processing system in a computational concentration that can include a large number of processing cores and / or processing clusters, such as, for example, many integrated core (MIC) processors. For example, the one or more parallel processors 1812 form a graphics processing subsystem that can output pixels to one of the one or more display devices 1810A coupled via the I / O hub 1807. The one or more parallel processors 1812 may also include a display controller and a display interface (not shown) for enabling direct connection to one or more display devices 1810B.

[0234] Within the I / O subsystem 1811, a system storage unit 1814 may be connected to the I / O hub 1807 to provide a storage mechanism for the computing system 1800. An I / O switch 1816 may be used to provide an interface mechanism to enable connections between the I / O hub 1807 and other components, such as a network adapter 1818 and / or a wireless network adapter 1819 that may be integrated into the platform, and various other devices that may be added via one or more plug-in devices 1820. The plug-in device(s) 1820 may also include, for example, one or more external graphics processor devices, graphics cards, and / or computing accelerators. The network adapter 1818 may be an Ethernet adapter or another wired network adapter. The wireless network adapter 1819 may include one or more of the following: Wi-Fi, Bluetooth, near field communication (NFC), or other network devices including one or more wireless radio devices.

[0235] The computing system 1800 may include other components not explicitly shown, including USB or other port connections, optical storage drives, video capture devices, etc., which may also be connected to the I / O hub 1807. Fig.18 The communication paths for interconnecting the various components in the system may be implemented using any suitable protocol, such as a PCI (Peripheral Component Interconnect) based protocol (e.g., PCI Express), or any other bus or point-to-point communication interface and / or protocol(s), such as NVLink high-speed interconnect, Compute Express Link, or the like. TM (ComputeExpress Link TM , CXL TM) (e.g., CXL.mem), Infinity Fabric (IF), Ethernet (IEEE802.3), remote direct memory access (RDMA), InfiniBand, Internet Wide Area RDMA Protocol (iWARP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), quick UDP Internet Connections (QUIC), RDMA over Converged Ethernet (RoCE), Intel Quick Path Interconnect (QPI), Intel Ultra Path Interconnect (UPI), Intel On-Chip System Fabric (IOSF), Omnipath, HyperTransport, Advanced Microcontroller Bus Architecture (AMBA) interconnect, OpenCAPI, Gen-Z, Cache Coherent Interconnect for Accelerators The data may be copied or stored to a virtualized storage node using protocols such as NVMe (non-volatile memory express over Fabrics, NVME-oF) or NVMe.

[0236] One or more parallel processors 1812 may include circuits optimized for graphics and video processing (including, for example, video output circuits) and constitute a graphics processing unit (GPU). Alternatively or additionally, described in more detail herein, one or more parallel processors 1812 may include circuits optimized for general-purpose processing while retaining the underlying computing architecture. The components of the computing system 1800 may be integrated on a single integrated circuit with one or more other system elements. For example, one or more parallel processors 1812, a memory hub 1805, (one or more) processors 1802, and an I / O hub 1807 may be integrated into a system-on-chip (SoC) integrated circuit. Alternatively, the components of the computing system 1800 may be integrated into a single package to form a system-in-package (SIP) configuration. In some examples, at least a portion of the components of the computing system 1800 may be integrated into a multi-chip module (MCM), which may be interconnected with other multi-chip modules into a modular computing system.

[0237] It will be appreciated that the computing system 1800 shown herein is illustrative, and variations and modifications are possible. The connection topology, including the number and arrangement of bridges, the number of (one or more) processors 1802, and the number of (one or more) parallel processors 1812, may be modified as desired. For example, the system memory 1804 may be connected to the (one or more) processors 1802 directly rather than through a bridge, while other devices communicate with the system memory 1804 via the memory hub 1805 and the (one or more) processors 1802. In other alternative topologies, the (one or more) parallel processors are connected to the I / O hub 1807 or directly to one of the one or more processors 1802, rather than to the memory hub 1805. In other examples, the I / O hub 1807 and the memory hub 1805 may be integrated into a single chip. It is also possible for two or more sets of processors 1802 to be attached via multiple slots, which may be coupled to two or more instances of (one or more) parallel processors 1812.

[0238] Some of the specific components shown herein are optional and may not be included in all implementations of computing system 1800. For example, any number of plug-in cards or peripherals may be supported, or some components may be eliminated. In addition, some architectures may be specific to the Fig.18 Different terminology is used for components similar to those illustrated in FIG. 1804. For example, memory hub 1805 may be referred to as a north bridge in some architectures, while I / O hub 1807 may be referred to as a south bridge.

[0239] Fig.19AAn example of a parallel processor 1900 is shown. The parallel processor 1900 may be a GPU, a GPGPU, etc. as described herein. The various components of the parallel processor 1900 may be implemented using one or more integrated circuit devices, such as a programmable processor, an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). The illustrated parallel processor 1900 may be Fig.18 One or more of the parallel processor(s) 1812 shown in .

[0240] Parallel processor 1900 includes parallel processing unit 1902. Parallel processing unit includes I / O unit 1904 that enables communication with other devices, including other instances of parallel processing unit 1902. I / O unit 1904 can be directly connected to other devices. For example, I / O unit 1904 is connected to other devices via the use of a hub or switch interface (such as memory hub 1805). The connection between memory hub 1805 and I / O unit 1904 forms a communication link 1813. Within parallel processing unit 1902, I / O unit 1904 is connected to host interface 1906 and memory cross switch 1916, wherein host interface 1906 receives commands related to performing processing operations and memory cross switch 1916 receives commands related to performing memory operations.

[0241] When the host interface 1906 receives the command buffer via the I / O unit 1904, the host interface 1906 can direct the work operations for executing those commands to the front end 1908. In some examples, the front end 1908 is coupled with a scheduler 1910, which is configured to distribute commands or other work items to the processing cluster array 1912. The scheduler 1910 ensures that the processing cluster array 1912 is properly configured and in a valid state before the task is distributed to the processing clusters in the processing cluster array 1912. The scheduler 1910 can be implemented via firmware logic executed on a microcontroller. The microcontroller-implemented scheduler 1910 can be configured to perform complex scheduling and work distribution operations at coarse and fine granularity, thereby achieving fast preemption and context switching of threads executed on the processing cluster array 1912. Preferably, the host software can confirm the workload for scheduling on the processing cluster array 1912 via one of a plurality of graphics processing doorbells. In other examples, polling for new workloads or interrupts may be used to identify or indicate the availability of work to be performed. The workload may then be automatically distributed across the processing cluster array 1912 by the scheduler 1910 logic within the scheduler microcontroller.

[0242] The processing cluster array 1912 may include up to "N" processing clusters (e.g., cluster 1914A, cluster 1914B to cluster 1914N). Each cluster 1914A-1914N in the processing cluster array 1912 may execute a large number of concurrent threads. The scheduler 1910 may use various scheduling and / or work distribution algorithms to assign work to the clusters 1914A-1914N in the processing cluster array 1912, which may vary depending on the workload generated for each type of program or calculation. Scheduling may be handled dynamically by the scheduler 1910, or may be partially assisted by compiler logic during the compilation of program logic configured for execution by the processing cluster array 1912. Optionally, different clusters 1914A-1914N in the processing cluster array 1912 may be assigned to process different types of programs or to perform different types of calculations.

[0243] Processing cluster array 1912 may be configured to perform various types of parallel processing operations. For example, processing cluster array 1912 is configured to perform general-purpose parallel computing operations. For example, processing cluster array 1912 may include logic for performing processing tasks including filtering of video and / or audio data, performing modeling operations including physical operations, and performing data transformations.

[0244] Processing cluster array 1912 is configured to perform parallel graphics processing operations. In such examples where parallel processor 1900 is configured to perform graphics processing operations, processing cluster array 1912 may include additional logic for supporting the execution of such graphics processing operations, including but not limited to texture sampling logic for performing texture operations, and tessellation logic and other vertex processing logic. In addition, processing cluster array 1912 may be configured to execute graphics processing related shader programs, such as but not limited to vertex shaders, tessellation shaders, geometry shaders, and pixel shaders. Parallel processing unit 1902 may transfer data from system memory via I / O unit 1904 for processing. During processing, the transferred data may be stored in on-chip memory (e.g., parallel processor memory 1922) during processing, and then the data may be written back to system memory.

[0245] In examples where parallel processing units 1902 are used to perform graphics processing, scheduler 1910 may be configured to divide the processing workload into tasks of approximately equal size to better enable distribution of graphics processing operations to multiple clusters 1914A-1914N in processing cluster array 1912. In some of these examples, portions of processing cluster array 1912 may be configured to perform different types of processing. For example, a first portion may be configured to perform vertex shading and topology generation, a second portion may be configured to perform tessellation and geometry shading, and a third portion may be configured to perform pixel shading or other screen space operations to produce a rendered image for display. Intermediate data generated by one or more of clusters 1914A-1914N may be stored in a buffer to allow the intermediate data to be transferred between clusters 1914A-1914N for further processing.

[0246] During operation, the processing cluster array 1912 may receive processing tasks to be executed via the scheduler 1910, which receives commands defining the processing tasks from the front end 1908. For graphics processing operations, the processing tasks may include data to be processed, such as surface (patch) data, primitive data, vertex data, and / or pixel data, and state parameters and commands defining how the data is to be processed (e.g., what program is to be executed). The scheduler 1910 may be configured to fetch an index corresponding to a task, or may receive an index from the front end 1908. The front end 1908 may be configured to ensure that the processing cluster array 1912 is configured to a valid state before a workload specified by an incoming command buffer (e.g., a batch buffer, a push buffer, etc.) is initiated.

[0247] Each of the one or more instances of parallel processing unit 1902 may be coupled to parallel processor memory 1922. Parallel processor memory 1922 may be accessed via memory crossbar switch 1916, which may receive memory requests from processing cluster array 1912 and I / O unit 1904. Memory crossbar switch 1916 may access parallel processor memory 1922 via memory interface 1918. Memory interface 1918 may include a plurality of partition units (e.g., partition unit 1920A, partition unit 1920B, up to partition unit 1920N), each of which may be coupled to a portion (e.g., memory unit) of parallel processor memory 1922. The number of partition units 1920A-1920N may be configured to be equal to the number of memory units, such that a first partition unit 1920A has a corresponding first memory unit 1924A, a second partition unit 1920B has a corresponding second memory unit 1924B, and an Nth partition unit 1920N has a corresponding Nth memory unit 1924N. In other examples, the number of partition units 1920A-1920N may not be equal to the number of memory devices.

[0248] The memory units 1924A-1924N may include various types of memory devices, including dynamic random-access memory (DRAM) or graphics random access memory, such as synchronous graphics random access memory (SGRAM), including graphics double data rate (GDDR) memory. Optionally, the memory units 1924A-1924N may also include 3D stacked memory, including but not limited to high bandwidth memory (HBM). Those skilled in the art will appreciate that the specific implementation of the memory units 1924A-1924N may vary and may be selected from one of a variety of conventional designs. Rendering targets such as frame buffers or texture maps may be stored across the memory units 1924A-1924N, allowing the partition units 1920A-1920N to write portions of each rendering target in parallel to efficiently use the available bandwidth of the parallel processor memory 1922. In some examples, local instances of parallel processor memory 1922 may be eliminated in favor of a unified memory design utilizing system memory in conjunction with local cache memory.

[0249] Optionally, any of the clusters 1914A-1914N in the processing cluster array 1912 has the capability to process data to be written to any of the memory units 1924A-1924N within the parallel processor memory 1922. The memory crossbar 1916 may be configured to transfer the output of each cluster 1914A-1914N to any partition unit 1920A-1920N or to another cluster 1914A-1914N, which may perform additional processing operations on the output. Each cluster 1914A-1914N may communicate with a memory interface 1918 through the memory crossbar 1916 to read from or write to various external memory devices. In one example with a memory crossbar 1916, the memory crossbar 1916 has connections to a memory interface 1918 to communicate with the I / O unit 1904 and has connections to a local instance of a parallel processor memory 1922, thereby enabling processing units within different processing clusters 1914A-1914N to communicate with system memory or other memory that is not local to the parallel processing unit 1902. In general, the memory crossbar 1916 may be able to separate traffic flows between the clusters 1914A-1914N and the partition units 1920A-1920N using virtual channels, for example.

[0250] Although a single instance of parallel processing unit 1902 is illustrated within parallel processor 1900, any number of instances of parallel processing unit 1902 may be included. For example, multiple instances of parallel processing unit 1902 may be provided on a single plug-in card, or multiple plug-in cards may be interconnected. For example, parallel processor 1900 may be a plug-in device, such as, Fig.18 Plug-in device 1820, which can be a graphics card (such as a discrete graphics card including one or more GPUs, one or more memory devices, and device-to-device or network or structure interfaces). Different instances of parallel processing unit 1902 can be configured to interoperate even if different instances have different numbers of processing cores, different amounts of local parallel processor memory, and / or other configuration differences. Optionally, some instances of parallel processing unit 1902 may include higher precision floating point units relative to other instances. Systems including one or more instances of parallel processing unit 1902 or parallel processor 1900 can be implemented in various configurations and form factors, including but not limited to desktop computers, laptop computers, or handheld personal computers, servers, workstations, game consoles, and / or embedded systems. The orchestrator can use one or more of the following to form a composite node for workload execution: decomposed processor resources, cache resources, memory resources, storage resources, and networking resources.

[0251] In some examples, the parallel processing unit 1902 can be partitioned into multiple instances. Those multiple instances can be configured to execute workloads associated with different clients in an isolated manner, so that a predetermined quality of service is provided for each client. For example, each cluster 1914A-1914N can be partitioned and isolated from other clusters, thereby allowing the processing cluster array 1912 to be divided into multiple computing partitions or instances. In such a configuration, the workload executed on the isolated partition is protected from errors or errors associated with different workloads executed on different partitions. Partition units 1920A-1920N can be configured to enable dedicated paths and / or isolated paths to the memory of the cluster 1914A-1914N associated with the corresponding computing partition. This data path isolation enables the computing resources within the partition to communicate with one or more assigned memory units 1924A-1924N without being disturbed by the activities of other partitions.

[0252] Fig.19B is a block diagram of the partition unit 1920. The partition unit 1920 may be Fig.19A 1920A-1920N. As illustrated, partition unit 1920 includes L2 cache 1921, frame buffer interface 1925, and ROP 1926 (raster operation unit). L2 cache 1921 is a read / write cache configured to perform load and store operations received from memory crossbar switch 1916 and ROP 1926. Read misses and urgent writeback requests are output by L2 cache 1921 to frame buffer interface 1925 for processing. Updates may also be sent to the frame buffer via frame buffer interface 1925 for processing. In some examples, frame buffer interface 1925 communicates with memory units in parallel processor memory (such as, for example, within parallel processor memory 1922). Fig.19A Partition unit 1920 may also additionally or alternatively interface with one of the memory units in the parallel processor memory via a memory controller (not shown).

[0253] In graphics applications, ROP 1926 is a processing unit that performs raster operations such as stenciling, z-testing, blending, etc. ROP 1926 then outputs processed graphics data, which is stored in graphics memory. In some examples, ROP 1926 includes or is coupled to a codec (CODEC) 1927 that includes compression logic for compressing depth or color data written to memory or L2 cache 1921 and decompressing depth or color data read from memory or L2 cache 1921. The compression logic may be lossless compression logic that utilizes one or more of a variety of compression algorithms. The type of compression performed by CODEC 1927 may vary based on the statistical characteristics of the data to be compressed. For example, in some examples, delta color compression is performed on depth and color data on a slice-by-slice basis. In some examples, CODEC 1927 includes compression and decompression logic that can compress and decompress computational data associated with machine learning operations. CODEC 1927 can, for example, compress sparse matrix data for sparse machine learning operations. CODEC 1927 can also compress sparse matrix data encoded in a sparse matrix format (e.g., coordinate list encoding (COO), compressed sparse row (CSR), compressed sparse column (CSC), etc.) to generate compressed and encoded sparse matrix data. The compressed and encoded sparse matrix data can be decompressed and / or decoded before being processed by a processing element, or the processing element can be configured to consume compressed, encoded, or compressed and encoded data for processing.

[0254] ROP 1926 may be included in each processing cluster (e.g., Fig.19A 1914A-1914N) rather than being included in partition unit 1920. In such examples, read and write requests for pixel data rather than pixel fragment data are transmitted through memory crossbar switch 1916. The processed graphics data may be displayed on a display device (such as, Fig.18 1810A-1810B), is routed for further processing by (one or more) processors 1802, or is routed for processing by Fig.19A One of the processing entities within parallel processor 1900 further processes.

[0255] Fig.19Cis a block diagram of a processing cluster 1914 within a parallel processing unit. For example, a processing cluster is Fig.19A An instance of a processing cluster in the processing clusters 1914A-1914N. Processing cluster 1914 can be configured to execute many threads in parallel, wherein the term "thread" refers to an instance of a specific program executed on a set of specific input data. Optionally, a single-instruction, multiple-data (SIMD) instruction issuance technology can be used to support the parallel execution of a large number of threads without providing multiple independent instruction units. Alternatively, a single-instruction, multiple-thread (SIMT) technology can be used to use a common instruction unit configured to issue instructions to a set of processing engines in each processing cluster in the processing cluster to support the parallel execution of a large number of threads that are generally synchronized. Unlike the SIMD execution mechanism in which all processing engines typically execute the same instruction, SIMT execution allows different threads to more easily follow the divergent execution path through a given thread program. Those skilled in the art will understand that the SIMD processing mechanism represents a functional subset of the SIMT processing mechanism.

[0256] The operation of the processing cluster 1914 may be controlled via a pipeline manager 1932 that distributes processing tasks to SIMT parallel processors. The pipeline manager 1932 receives data from Fig.19A The graphics multiprocessor 1934 may be used to process the graphics multiprocessor 1934 and / or the texture unit 1936. The graphics multiprocessor 1934 is an exemplary embodiment of a SIMT parallel processor. However, various types of SIMT parallel processors of different architectures may be included in the processing cluster 1914. One or more instances of the graphics multiprocessor 1934 may be included in the processing cluster 1914. The graphics multiprocessor 1934 may process data, and the data crossbar 1940 may be used to distribute the processed data to one of a plurality of possible destinations, including other shader units. The pipeline manager 1932 may facilitate the distribution of processed data by specifying a destination for the processed data to be distributed via the data crossbar 1940.

[0257] Each graphics multiprocessor 1934 within a processing cluster 1914 may include an identical set of function execution logic (e.g., arithmetic logic units, load-store units, etc.). The function execution logic may be configured in a pipelined manner in which new instructions may be issued before previous instructions are completed. The function execution logic supports a variety of operations, including integer and floating point arithmetic, comparison operations, Boolean operations, bit shifts, and calculation of various algebraic functions. The same functional unit hardware may be utilized to perform different operations, and any combination of functional units may exist.

[0258] The instructions transmitted to the processing cluster 1914 constitute threads. A collection of threads executed across a collection of parallel processing engines is a thread group. A thread group executes the same program on different input data. Each thread within a thread group can be assigned to a different processing engine within the graphics multiprocessor 1934. A thread group may include fewer threads than the number of processing engines within the graphics multiprocessor 1934. When a thread group includes fewer threads than the number of processing engines, one or more of the processing engines may be idle during the cycle during which the thread group is being processed. A thread group may also include more threads than the number of processing engines within the graphics multiprocessor 1934. When a thread group includes more threads than the number of processing engines within the graphics multiprocessor 1934, processing may be performed in consecutive clock cycles. Optionally, multiple thread groups may be executed concurrently on the graphics multiprocessor 1934.

[0259] The graphics multiprocessor 1934 may include internal cache memory to perform load and store operations. Optionally, the graphics multiprocessor 1934 may abandon the internal cache and use cache memory (e.g., first level (level 1, L1) cache 1948) within the processing cluster 1914. Each graphics multiprocessor 1934 also has a partition unit (e.g., Fig.19A 1920A-1920N), which are shared between all processing clusters 1914 and can be used to transfer data between threads. Graphics multiprocessor 1934 can also access off-chip global memory, which can include one or more of local parallel processor memory and / or system memory. Any memory external to parallel processing unit 1902 can be used as global memory. Embodiments in which processing cluster 1914 includes multiple instances of graphics multiprocessor 1934 can share common instructions and data, which can be stored in L1 cache 1948.

[0260] Each processing cluster 1914 may include an MMU 1945 (memory management unit) configured to map virtual addresses to physical addresses. In other examples, one or more instances of MMU 1945 may reside in Fig.19A 1918. The MMU 1945 includes a set of page table entries (PTEs) for mapping virtual addresses to physical addresses of the slice, and optionally includes a cache line index. The MMU 1945 may include an address translation lookaside buffer (TLB) or cache that may reside within the graphics multiprocessor 1934 or L1 cache 1948 of the processing cluster 1914. The physical addresses are processed to distribute surface data access locality, thereby allowing efficient request interleaving between partition units. The cache line index can be used to determine whether a request for a cache line is a hit or a miss.

[0261] In graphics and compute applications, the processing clusters 1914 may be configured such that each graphics multiprocessor 1934 is coupled to a texture unit 1936 for performing texture mapping operations, such as determining texture sample locations, reading texture data, and filtering texture data. Texture data is read from an internal texture L1 cache (not shown), or in some examples, from an L1 cache within the graphics multiprocessor 1934 and retrieved from an L2 cache, local parallel processor memory, or system memory as needed. Each graphics multiprocessor 1934 outputs processed tasks to a data crossbar 1940 to provide the processed tasks to another processing cluster 1914 for further processing, or to store the processed tasks in an L2 cache, local parallel processor memory, or system memory via a memory crossbar 1916. A pre-raster operations unit 1942 is configured to receive data from the graphics multiprocessor 1934, direct the data to ROP units, which may be coupled to partition units (e.g., Fig.19A The preROP 1942 unit may perform optimizations for color blending, organize pixel color data, and perform address translation.

[0262] It will be appreciated that the core architecture described herein is illustrative, and variations and modifications are possible. Any number of processing units (e.g., graphics multiprocessors 1934, texture units 1936, preROPs 1942, etc.) may be included within a processing cluster 1914. Further, while only one processing cluster 1914 is shown, a parallel processing unit as described herein may include any number of instances of a processing cluster 1914. Optionally, each processing cluster 1914 may be configured to operate independently of other processing clusters 1914 using separate and distinct processing units, L1 caches, L2 caches, etc.

[0263] Fig.19D An example of a graphics multiprocessor 1934 is shown, where the graphics multiprocessor 1934 is coupled to a pipeline manager 1932 of a processing cluster 1914. The graphics multiprocessor 1934 has an execution pipeline that includes, but is not limited to, an instruction cache 1952, an instruction unit 1954, an address mapping unit 1956, a register file 1958, one or more general purpose graphics processing unit (GPGPU) cores 1962, and one or more load / store units 1966. The GPGPU cores 1962 and the load / store units 1966 are coupled to a cache memory 1972 and a shared memory 1970 via a memory and cache interconnect 1968. The graphics multiprocessor 1934 may additionally include a tensor and / or ray tracing core 1963 that includes hardware logic for accelerating matrix and / or ray tracing operations.

[0264] The instruction cache 1952 may receive an instruction stream to be executed from the pipeline manager 1932. The instructions are cached in the instruction cache 1952 and dispatched for execution by the instruction unit 1954. The instruction unit 1954 may dispatch the instructions as a thread group (e.g., a unit group (warp)), wherein each thread in the thread group is assigned to a different execution unit within the GPGPU core 1962. The instruction may access any of the local address space, the shared address space, or the global address space by specifying an address within the unified address space. The address mapping unit 1956 may be used to translate the address in the unified address space into different memory addresses that may be accessed by the load / store unit 1966.

[0265] The register file 1958 provides a collection of registers for the functional units of the graphics multiprocessor 1934. The register file 1958 provides temporary storage for operands of data paths connected to the functional units of the graphics multiprocessor 1934 (e.g., GPGPU core 1962, load / store unit 1966). The register file 1958 may be divided between each of the functional units so that each functional unit is assigned a dedicated portion of the register file 1958. For example, the register file 1958 may be divided between different groups of units executed by the graphics multiprocessor 1934.

[0266] The GPGPU cores 1962 may each include a floating point unit (FPU) and / or an integer arithmetic logic unit (ALU) for executing instructions of the graphics multiprocessor 1934. In some implementations, the GPGPU core 1962 may include hardware logic that may otherwise reside within the tensor and / or ray tracing core 1963. The GPGPU cores 1962 may be similar in architecture, or may be different in architecture. For example and in some examples, the first portion of the GPGPU core 1962 includes a single-precision FPU and an integer ALU, while the second portion of the GPGPU core includes a double-precision FPU. Optionally, the FPU may implement the IEEE 754-2008 standard for floating-point arithmetic, or enable variable-precision floating-point arithmetic. The graphics multiprocessor 1934 may additionally include one or more fixed-function or special-function units for performing specific functions, such as copying rectangles or pixel blending operations. One or more of the GPGPU cores in the GPGPU core may also include fixed-function or special-function logic.

[0267] GPGPU core 1962 may include SIMD logic capable of executing a single instruction on multiple sets of data. Optionally, GPGPU core 1962 may physically execute SIMD4, SIMD8 and SIMD16 instructions, and logically execute SIMD1, SIMD2 and SIMD32 instructions. SIMD instructions for GPGPU cores may be generated by a shader compiler at compile time, or automatically generated when executing a program written and compiled for a single program multiple data (SPMD) or SIMT architecture. Multiple threads of a program configured for a SIMT execution model may be executed via a single SIMD instruction. For example and in some examples, eight SIMT threads may be executed in parallel via a single SIMD8 logic unit, and these eight SIMT threads perform the same or similar operations.

[0268] The memory and cache interconnect 1968 is an interconnect network that connects each of the functional units of the graphics multiprocessor 1934 to the register file 1958 and to the shared memory 1970. For example, the memory and cache interconnect 1968 is a crossbar interconnect that allows the load / store unit 1966 to implement load and store operations between the shared memory 1970 and the register file 1958. The register file 1958 can operate at the same frequency as the GPGPU core 1962, so data transfers between the GPGPU core 1962 and the register file 1958 are very low latency. The shared memory 1970 can be used to enable communication between threads executing on the functional units within the graphics multiprocessor 1934. The cache memory 1972 can be used as a data cache, for example, to cache texture data passed between the functional unit and the texture unit 1936. The shared memory 1970 can also be used as a managed cached program. The shared memory 1970 and the cache memory 1972 can be coupled with the data crossbar 1940 to enable communication with other components of the processing cluster. Threads executing on GPGPU core 1962 can programmatically store data in shared memory in addition to the automatically cached data stored in cache memory 1972.

[0269] Figure 20A-20C An additional graphics multiprocessor is illustrated according to an example. Figure 20A-20B Graphics multiprocessors 2025 and 2050 are shown in the figure. Graphics multiprocessors 2025 and 2050 are Fig.19C The graphics multiprocessor 1934 of FIG. 1 is related to the graphics multiprocessor 1934 of FIG. 1 and can be used instead of one of those graphics multiprocessors. Therefore, the disclosure of any feature in this article in conjunction with the graphics multiprocessor 1934 also discloses the corresponding combination with the graphics multiprocessors 2025 and 2050, but is not limited to this. Fig. 20C A graphics processing unit (GPU) 2080 is illustrated, which includes a collection of dedicated graphics processing resources arranged as multi-core groups 2065A-2065N, which correspond to the graphics multiprocessors 2025, 2050. The illustrated graphics multiprocessors 2025, 2050 and multi-core groups 2065A-2065N may be streaming multiprocessors (SMs) capable of executing a large number of execution threads simultaneously.

[0270] Fig. 20A The graphics multiprocessor 2025 includes relative Fig.19DThe graphics multiprocessor 2025 may include multiple additional instances of execution resource units of the graphics multiprocessor 1934. For example, the graphics multiprocessor 2025 may include multiple instances of instruction units 2032A-2032B, register files 2034A-2034B, and (one or more) texture units 2044A-2044B. The graphics multiprocessor 2025 also includes multiple sets of graphics or compute execution units (e.g., GPGPU cores 2036A-2036B, tensor cores 2037A-2037B, ray tracing cores 2038A-2038B) and multiple sets of load / store units 2040A-2040B. The execution resource units have a common instruction cache 2030, texture and / or data cache memory 2042, and shared memory 2046.

[0271] The components may communicate via the interconnect structure 2027. The interconnect structure 2027 may include one or more crossbar switches to enable communication between the components of the graphics multiprocessor 2025. The interconnect structure 2027 is a separate, high-speed network fabric layer on which each component of the graphics multiprocessor 2025 is stacked. The components of the graphics multiprocessor 2025 communicate with remote components via the interconnect structure 2027. For example, the cores 2036A-2036B, 2037A-2037B, and 2038A-2038B may each communicate with the shared memory 2046 via the interconnect structure 2027. The interconnect structure 2027 may arbitrate communications within the graphics multiprocessor 2025 to ensure fair bandwidth allocation between components.

[0272] Fig. 20B The graphics multiprocessor 2050 includes a plurality of execution resource sets 2056A-2056D, wherein Fig.19D and Fig. 20A As shown in FIG. 1 , each set of execution resources includes multiple instruction units, register files, GPGPU cores, and load storage units. Execution resources 2056A-2056D can work in conjunction with (one or more) texture units 2060A-2060D for texture operations while sharing instruction cache 2054 and shared memory 2053. For example, execution resources 2056A-2056D can share multiple instances of instruction cache 2054 and shared memory 2053 and texture and / or data cache memory 2058A-2058B. Each component can be connected to the CPU via the CPU. Fig. 20A The interconnect structure 2027 communicates with a similar interconnect structure 2052 .

[0273] Those skilled in the art will understand that Fig.18 , Figures 19A-19D as well as Figure 20A-20BThe architecture described in is illustrative and not limiting in scope of the current examples. Thus, the techniques described herein may be implemented on any appropriately configured processing unit without departing from the scope of the examples described herein, including but not limited to: one or more mobile application processors; one or more desktop or server central processing units (CPUs), including multi-core CPUs; one or more parallel processor units, such as, Fig.19A Parallel processing unit 1902 and one or more graphics processors or special processing units.

[0274] The parallel processor or GPGPU described herein can be communicatively coupled to a host / processor core to accelerate graphics operations, machine learning operations, pattern analysis operations, and various general-purpose GPU (GPGPU) functions. The GPU can be communicatively coupled to the host processor / core via a bus or another interconnect (e.g., a high-speed interconnect, such as PCIe, NVLink, or other known protocols, standardized protocols, or proprietary protocols). In other examples, the GPU can be integrated on the same package or chip as the core and communicatively coupled to the core via an internal processor bus / interconnect (i.e., inside the package or chip). Regardless of the manner in which the GPU is connected, the processor core can assign work to the GPU in the form of a sequence of commands / instructions contained in a work descriptor. The GPU then uses dedicated circuits / logic to efficiently process these commands / instructions.

[0275] Fig. 20C A graphics processing unit (GPU) 2080 is shown, which includes a dedicated set of graphics processing resources arranged as multi-core groups 2065A-2065N. Although details of only a single multi-core group 2065A are provided, it will be appreciated that other multi-core groups 2065B-2065N may be equipped with the same or similar set of graphics processing resources. The details described with respect to multi-core groups 2065A-2065 may also be applied to any graphics multiprocessor 1934, 2025, 2050 described herein.

[0276] As illustrated, the multi-core group 2065A may include a set of graphics cores 2070, a set of tensor cores 2071, and a set of ray tracing cores 2072. The scheduler / dispatcher 2068 schedules and dispatches the graphics thread for execution on the respective cores 2070, 2071, 2072. The set of register files 2069 stores operand values ​​used by the cores 2070, 2071, 2072 when executing the graphics thread. These register files may include, for example, integer registers for storing integer values, floating point registers for storing floating point values, vector registers for storing packed data elements (integer and / or floating point data elements), and slice registers for storing tensor / matrix values. The slice registers may be implemented as a combined set of vector registers.

[0277] One or more combined first level (L1) caches and shared memory units 2073 store graphics data locally within each multi-core group 2065A, such as texture data, vertex data, pixel data, light data, bounding volume data, etc. One or more texture units 2074 may also be used to perform texture operations, such as texture mapping and sampling. A second level (L2) cache 2075 shared by all multi-core groups 2065A-2065N or a subset of multi-core groups 2065A-2065N stores graphics data and / or instructions for multiple concurrent graphics threads. As shown, the L2 cache 2075 may be shared across multiple multi-core groups 2065A-2065N. One or more memory controllers 2067 couple the GPU 2080 to a memory 2066, which may be a system memory (e.g., DRAM) and / or a dedicated graphics memory (e.g., GDDR6 memory).

[0278] Input / output (I / O) circuitry 2063 couples GPU 2080 to one or more I / O devices 2062, such as a digital signal processor (DSP), a network controller, or a user input device. On-chip interconnects may be used to couple I / O devices 2062 to GPU 2080 and memory 2066. One or more I / O memory management units (IOMMUs) 2064 of I / O circuitry 2063 directly couple I / O devices 2062 to system memory 2066. Optionally, IOMMU 2064 manages a plurality of page table sets for mapping virtual addresses to physical addresses in system memory 2066. I / O devices 2062, CPU(s) 2061, and GPU(s) 2080 may then share the same virtual address space.

[0279] In one implementation of the IOMMU 2064, the IOMMU 2064 supports virtualization. In this case, the IOMMU 2064 can manage a first set of page tables for mapping guest / graphics virtual addresses to guest / graphics physical addresses and a second set of page tables for mapping guest / graphics physical addresses to system / host physical addresses (e.g., within system memory 2066). The base address of each of the first set of page tables and the second set of page tables can be stored in a control register and swapped out at context switch (e.g., so that the new context is provided with access to the relevant set of page tables). Although not described in detail in the prior art, the IOMMU 2064 can manage a first set of page tables for mapping guest / graphics virtual addresses to guest / graphics physical addresses and a second set of page tables for mapping guest / graphics physical addresses to system / host physical addresses (e.g., within system memory 2066). Fig. 20C , but each of the cores 2070, 2071, 2072 and / or multi-core groups 2065A-2065N may include a translation lookaside buffer (TLB) for caching guest virtual to guest physical translations, guest physical to host physical translations, and guest virtual to host physical translations.

[0280] (One or more) CPU 2061, GPU 2080 and I / O device 2062 can be integrated on a single semiconductor chip and / or chip package. The illustrated memory 2066 can be integrated on the same chip, or can be coupled to a memory controller 2067 via an off-chip interface. In one implementation, memory 2066 includes GDDR6 memory that shares the same virtual address space as other physical system-level memories, but the basic principles described herein are not limited to this particular implementation.

[0281] Tensor core 2071 may include multiple execution units specifically designed to perform matrix operations, which are basic computational operations for performing deep learning operations. For example, synchronized matrix multiplication operations may be used for neural network training and inference. Tensor core 2071 may perform matrix processing using a variety of operand precisions, including single-precision floating point (e.g., 32 bits), half-precision floating point (e.g., 16 bits), integer words (16 bits), bytes (8 bits), and half bytes (4 bits). For example, a neural network implementation extracts features of each rendered scene, potentially combining details from multiple frames to build a high-quality final image.

[0282] In a deep learning implementation, parallel matrix multiplication work may be scheduled for execution on the tensor core 2071. The training of neural networks in particular requires a large number of matrix dot product operations. To handle the inner product formulation of the N x N x N matrix multiplication, the tensor core 2071 may include at least N dot product processing elements. Before the matrix multiplication begins, a complete matrix is ​​loaded into the slice register, and for each of the N cycles, at least one column of the second matrix is ​​loaded. For each cycle, there are N dot products that are processed.

[0283] Depending on the specific implementation, matrix elements can be stored with different precisions, including 16-bit words, 8-bit bytes (e.g., INT8), and 4-bit nibbles (e.g., INT4). Different precision modes can be specified for the Tensor Core 2071 to ensure that the most efficient precision is used for different workloads (e.g., such as inference workloads, which can tolerate quantization to bytes and nibbles). Supported formats additionally include 64-bit floating point (FP64) and non-IEEE floating point formats, such as the bfloat16 format (e.g., Brain floating point), a 16-bit floating point format with one sign bit, eight exponent bits, and eight significand bits (seven of which are explicitly stored). An example includes support for a reduced precision tensor floating point (TF32) mode that performs calculations using the range of FP32 (8 bits) and the precision of FP16 (10 bits). Reduced-precision TF32 operations can be performed on FP32 inputs and produce FP32 outputs with higher performance relative to FP32 and increased precision relative to FP16. In some examples, one or more 8-bit floating point formats (FP32) are supported.

[0284] In some examples, the tensor core 2071 supports sparse operation modes for matrices in which the vast majority of values ​​are zero. The tensor core 2071 includes support for sparse input matrices encoded with sparse matrix representations (e.g., coordinate list encoding (COO), compressed sparse rows (CSR), compressed sparse columns (CSC), etc.). The tensor core 2071 also includes support for compressed sparse matrix representations in the case where the sparse matrix representation can be further compressed. Compressed matrix data, encoded matrix data, and / or compressed and encoded matrix data and associated compression and / or encoding metadata can be read by the tensor core 2071, and non-zero values ​​can be extracted. For example, for a given input matrix A, non-zero values ​​can be loaded from at least a portion of the compressed and / or encoded representation of matrix A. Based on the position of the non-zero value in matrix A (which can be determined from the index or coordinate metadata associated with the non-zero value), the corresponding value in the input matrix B can be loaded. Depending on the operation to be performed (e.g., multiplication), loading values ​​from input matrix B may be bypassed if the corresponding value is a zero value. In some examples, the pairing of values ​​for certain operations (such as multiplication operations) may be pre-scanned by the scheduler logic, and only operations between non-zero inputs are scheduled. Depending on the dimensions of matrix A and matrix B and the operation to be performed, the output matrix C may be dense or sparse. In the case where the output matrix C is sparse and depending on the configuration of the tensor core 2071, the output matrix C may be output in a compressed format, sparse coding, or compressed sparse coding.

[0285] The ray tracing core 2072 can accelerate ray tracing operations for both real-time ray tracing implementations and non-real-time ray tracing implementations. Specifically, the ray tracing core 2072 may include ray traversal / intersection circuitry for performing ray traversals and identifying intersections between rays and primitives enclosed within a bounding volume hierarchy (BVH) using a bounding volume hierarchy. The ray tracing core 2072 may also include circuitry for performing depth testing and culling (e.g., using a Z buffer or similar arrangement). In one implementation, the ray tracing core 2072 performs traversal and intersection operations in conjunction with the image denoising techniques described herein, at least portions of which may be performed on the tensor core 2071. For example, the tensor core 2071 may implement a deep learning neural network to perform denoising on frames generated by the ray tracing core 2072. However, the CPU(s) 2061, graphics core 2070, and / or ray tracing core 2072 may also implement all or portions of the denoising and / or deep learning algorithms.

[0286] In addition, as described above, a distributed approach to noise reduction may be employed, wherein GPU 2080 is in a computing device coupled to other computing devices via a network or high-speed interconnect. According to the distributed approach, the interconnected computing devices may share neural network learning / training data to improve the speed at which the entire system learns to perform noise reduction for different types of image frames and / or different graphics applications.

[0287] The ray tracing core 2072 can handle all BVH traversals and / or ray-primitive intersections, thereby preventing the graphics core 2070 from being overloaded with thousands of instructions for each ray. For example, each ray tracing core 2072 includes a first set of specialized circuits for performing bounding box tests (e.g., for traversal operations) and / or a second set of specialized circuits for performing ray-triangle intersection tests (e.g., intersecting rays that have been traversed). Thus, for example, the multi-core group 2065A can simply start ray detection, and the ray tracing core 2072 independently performs ray traversals and intersections and returns hit data (e.g., hits, no hits, multiple hits, etc.) to the thread context. While the ray tracing core 2072 performs traversal and intersection operations, the other cores 2070, 2071 are freed up to perform other graphics or computational work.

[0288] Optionally, each ray tracing core 2072 may include a traversal unit for performing BVH test operations and / or an intersection unit for performing ray-primitive intersection tests. The intersection unit generates a "hit", "no hit", or "multiple hits" response, which it provides to the appropriate thread. During traversal and intersection operations, execution resources of other cores (e.g., graphics core 2070 and tensor core 2071) are freed up to perform other forms of graphics work.

[0289] In some of the examples described below, a hybrid rasterization / ray tracing approach is used in which the work is distributed between graphics core 2070 and ray tracing core 2072.

[0290] The ray tracing core 2072 (and / or other cores 2070, 2071) may include hardware support for ray tracing instruction sets, such as Microsoft's DirectX Ray Tracing (DXR), which includes the DispatchRays command; and ray generation shaders, nearest hit shaders, any hit shaders, and miss shaders, which enable unique shader and texture sets to be assigned to each object. Another ray tracing platform that can be supported by the ray tracing core 2072, graphics core 2070, and tensor core 2071 is the Vulkan API (e.g., Vulkan version 1.1.85, or later). However, it should be noted that the basic principles described herein are not limited to any particular ray tracing ISA.

[0291] In general, each core 2072, 2071, 2070 may support a ray tracing instruction set including instructions / functions for one or more of the following: ray generation, nearest hit, any hit, ray-primitive intersection, per-primitive and hierarchy bounding box construction, misses, visits, and exceptions. More specifically, some examples include ray tracing instructions for performing one or more of the following functions: - Light Generation - Ray generation instructions can be executed for each pixel, sample, or other user-defined work assignment. - Recent Hits - A nearest hit command may be executed to locate the closest intersection of a ray with a primitive within the scene. - Any hit - Any hit command identifies multiple intersections between rays and primitives within the scene, potentially identifying a new closest intersection point. - intersect ——The intersection instruction performs a ray-primitive intersection test and outputs the result. - Per-primitive bounding box construction - This instruction builds a bounding box around a given primitive or group of primitives (e.g., when building a new BVH or other acceleration data structure). - miss - Indicates that the ray missed the scene or all geometry within a specified region of the scene. - visit ——Indicates the subvolume that the ray will traverse. - abnormal - Includes various types of exception handlers (e.g., called for various error conditions).

[0292] In some examples, the ray tracing core 2072 may be adapted to accelerate general computing operations that can be accelerated using computing techniques similar to ray intersection testing. A computing framework may be provided that enables shader programs to be compiled into low-level instructions and / or primitives that perform general computing operations via the ray tracing core. Exemplary computing problems that may benefit from computing operations performed on the ray tracing core 2072 include calculations involving the propagation of beams, waves, rays, or particles in a coordinate space. Interactions associated with that propagation may be calculated relative to geometry or meshes in the coordinate space. For example, calculations associated with the propagation of electromagnetic signals through an environment may be accelerated using instructions or primitives that are executed via the ray tracing core. Refraction and reflection of signals through objects in the environment may be calculated as direct ray tracing simulations.

[0293] The ray tracing core 2072 can also be used to perform calculations that are not directly similar to ray tracing. For example, the ray tracing core 2072 can be used to accelerate mesh projection, mesh refinement, and volume sampling calculations. General coordinate space calculations, such as nearest neighbor calculations, can also be performed. For example, a set of points near a given point can be found by defining a bounding box around the given point in the coordinate space. The BVH and ray detection logic in the ray tracing core 2072 can then be used to determine the set of points that intersect within the bounding box. The intersection constitutes the origin and the nearest neighbor of that origin. The calculations performed using the ray tracing core 2072 can be performed in parallel with the calculations performed on the graphics core 2072 and the tensor core 2071. The shader compiler can be configured to compile a compute shader or other general graphics processing program into a low-level primitive that can be parallelized across the graphics core 2070, the tensor core 2071, and the ray tracing core 2072.

[0294] Building larger and larger silicon dies is challenging for various reasons. As silicon dies get larger, manufacturing yields become smaller, and process technology requirements for different components may vary. On the other hand, in order to have a high-performance system, key components should be interconnected through high-speed, high-bandwidth, low-latency interfaces. These conflicting requirements bring challenges to high-performance chip development.

[0295] The embodiments described herein provide techniques for separating the architecture of a system-on-chip integrated circuit into multiple different chiplets that can be packaged onto a common substrate. In some examples, a graphics processing unit or a parallel processor is composed of various silicon chiplets that are manufactured separately. A chiplet is an integrated circuit that is at least partially packaged, and the at least partially packaged integrated circuit includes different logic units that can be assembled into a larger package with other chiplets. Various chiplet collections with different IP core logic can be assembled into a single device. In addition, the chiplets can be integrated into a base die or base chiplet using active interposer technology. The concepts described herein enable interconnection and communication between different forms of IP within a GPU. IP development on different processes can be mixed. This avoids the complexity of converging multiple IPs to the same process, especially for large SoCs with several styles of IP.

[0296] Enabling the use of multiple process technologies improves time to market and provides a cost-effective way to create multiple product SKUs. For customers, this means getting a product that better suits their requirements in a cost-effective and timely manner. Additionally, the split IP is easier to modify to be independently power-gated, and components that are not in use for a given workload can be turned off, reducing overall power consumption.

[0297] Fig.21 A parallel computing system 2100 according to some examples is shown. In some examples, the parallel computing system 2100 includes a parallel processor 2120, which may be a graphics processor or a computing accelerator as described herein. The parallel processor 2120 includes a global logic unit 2101, an interface 2102, a thread dispatcher 2103, a media unit 2104, a set of computing units 2105A-2105H, and a cache / memory unit 2106. In some examples, the global logic unit 2101 includes global functions for the parallel processor 2120, including device configuration registers, a global scheduler, power management logic, and the like. The interface 2102 may include a front-end interface for the parallel processor 2120. The thread dispatcher 2103 may receive a workload from the interface 2102 and dispatch threads of the workload to the computing units 2105A-2105H. If the workload includes any media operations, at least part of those operations may be performed by the media unit 2104. The media unit may also migrate some operations to the compute units 2105A-2105H. The cache / memory unit 2106 may include cache memory (eg, L3 cache) and local memory (eg, HBM, GDDR) for the parallel processors 2120.

[0298] Figure 22A-22BIllustrate a hybrid logical / physical view of a decomposed parallel processor according to examples described herein. Fig.22A A split parallel computing system 2200 is shown. Fig. 22B A chiplet 2230 of the split parallel computing system 2200 is shown.

[0299] like Fig.22A As shown in , the separate computing system 2200 may include a parallel processor 2220, wherein the components of the parallel processor SOC are distributed across multiple chiplets. Each chiplet may be a different IP core that is independently designed and configured to communicate with other chiplets via one or more common interfaces. The chiplets include, but are not limited to, a computing chiplet 2205, a media chiplet 2204, and a memory chiplet 2206. Each chiplet can be manufactured separately using different process technologies. For example, the computing chiplet 2205 can be manufactured using the smallest or most advanced process technology available at the time of manufacturing, while the memory chiplet 2206 or other chiplets (e.g., I / O, networking, etc.) can be manufactured using larger or less advanced process technologies.

[0300] The individual chiplets may be bonded to a base die 2210 and configured to communicate with each other and logic within the base die 2210 via an interconnect layer 2212. In some examples, the base die 2210 may include a global logic 2201, which may include a scheduler 2211 and power management 2221 logic unit, an interface 2202, a dispatch unit 2203, and an interconnect fabric module 2208 coupled or integrated with one or more L3 cache blocks 2209A-2209N. The interconnect fabric 2208 may be an inter-chiplet fabric integrated into the base die 2210. The logic chiplets may use the fabric 2208 to relay messages between the individual chiplets. Additionally, the L3 cache blocks 2209A-2209N in the base die and / or the L3 cache blocks within the memory chiplet 2206 may cache data read from the DRAM chiplets within the memory chiplet 2206 and transferred to the DRAM chiplets and to the system memory of the host.

[0301] In some examples, the global logic 2201 is a microcontroller that can execute firmware to perform scheduler 2211 and power management 2221 functions for the parallel processor 2220. The microcontroller that executes the global logic can be customized for the target use case of the parallel processor 2220. The scheduler 2211 can perform global scheduling operations for the parallel processor 2220. The power management 2221 function can be used to enable or disable individual chiplets within the parallel processor when those chiplets are not in use.

[0302] The individual chiplets of the parallel processor 2220 may be designed to perform specific functions that would be integrated into a single die in existing designs. The collection of compute chiplets 2205 may include a cluster of compute units (e.g., execution units, streaming multiprocessors, etc.) that include programmable logic for executing compute or graphics shader instructions. The media chiplet 2204 may include hardware logic for accelerating media encoding and decoding operations. The memory chiplet 2206 may include volatile memory (e.g., DRAM) and one or more SRAM cache memory blocks (e.g., L3 blocks).

[0303] like Fig. 22B As shown in , each chiplet 2230 may include common components and dedicated components. The chiplet logic 2236 within the chiplet 2230 may include specific components of the chiplet, such as an array of streaming multiprocessors, computing units, or execution units described herein. The chiplet logic 2236 may be coupled with an optional cache or shared local memory 2238, or a cache or shared local memory may be included within the chiplet logic 2236. The chiplet 2230 may include a fabric interconnect node 2242 that receives commands via an inter-chiplet structure. Commands and data received via the fabric interconnect node 2242 may be temporarily stored in an interconnect buffer 2239. Data transmitted to and received from the fabric interconnect node 2242 may be stored in the interconnect cache 2240. Power control 2232 and clock control 2234 logic may also be included in the chiplet. The power control 2232 and clock control 2234 logic may receive configuration commands via the structure and may configure dynamic voltage and frequency scaling for the chiplet 2230. In some examples, each chiplet can have independent clock and power domains and can be clock-gated and power-gated independently of other chiplets.

[0304] At least some of the components within the illustrated chiplet 2230 may also be included in a Fig.22A The base die 2210 may be within logic within the base die 2210. For example, logic within the base die that communicates with the fabric may include some version of the fabric interconnect node 2242. The base die logic that may be independently clock gated or power gated may include some version of the power control 2232 and / or clock control 2234 logic.

[0305] Thus, while various examples described herein use the term SOC to describe a device or system having a processor and associated circuitry (e.g., input / output (“I / O”) circuitry, power delivery circuitry, memory circuitry, etc.) monolithically integrated into a single integrated circuit (“IC”) die or chip, the present disclosure is not limited in this regard. For example, in various examples of the present disclosure, a device or system may have one or more processors (e.g., one or more processor cores) and associated circuitry (e.g., input / output (“I / O”) circuitry, power delivery circuitry, etc.) arranged in a discrete collection of discrete dies, slices, and / or chiplets (e.g., one or more discrete processor core dies arranged adjacent to one or more other dies (such as memory dies, I / O dies, etc.)). In such discrete devices and systems, the individual dies, slices, and / or chiplets may be physically and / or electrically coupled together by a packaging structure that includes, for example, individual packaging substrates, interposers, active interposers, photonic interposers, interconnect bridges, and the like. A separate collection of discrete dies, slices, and / or chiplets may also be part of a System-on-Package (“SoP”).

[0306] Example core architectures—in-order and out-of-order core block diagrams.

[0307] The block diagram of Figure 23(A) illustrates both an example in-order pipeline and an example register renaming, out-of-order issue / execution pipeline according to some examples. The block diagram of Figure 23(B) illustrates both an example in-order architecture core and an example register renaming, out-of-order issue / execution architecture core to be included in a processor according to an example. The solid-line boxes in Figures 23(A)-(B) illustrate the in-order pipeline and the in-order core, while the optionally added dashed-line boxes illustrate the register renaming, out-of-order issue / execution pipeline and core. Considering that the in-order aspect is a subset of the out-of-order aspect, the out-of-order aspect will be described.

[0308] In FIG. 23(A), the processor pipeline 2300 includes a fetch stage 2302, an optional length decode stage 2304, a decode stage 2306, an optional allocation (Alloc) stage 2308, an optional rename stage 2310, a dispatch (also called dispatch or issue) stage 2312, an optional register read / memory read stage 2314, an execute stage 2316, a write back / memory write stage 2318, an optional exception handling stage 2322, and an optional commit stage 2324. One or more operations may be performed in each of these processor pipeline stages. For example, during the fetch stage 2302, one or more instructions are fetched from the instruction memory, and during the decode stage 2306, the fetched one or more instructions may be decoded, an address using a forwarding register port (e.g., a load store unit (LSU) address) may be generated, and branch forwarding (e.g., an immediate offset or a link register (LR)) may be performed. In some examples, the decode stage 2306 and the register read / memory read stage 2314 can be combined into one pipeline stage. In some examples, during the execute stage 2316, decoded instructions can be executed, LSU address / data pipelining to an Advanced Microcontroller Bus (AMB) interface can be performed, multiplication and addition operations can be performed, arithmetic operations with branch results can be performed, and the like.

[0309] As an example, the example register renaming, out-of-order issue / execution architecture core of FIG. 23(B) may implement pipeline 2300 by: 1) instruction fetch circuit system 2338 executing fetch and length decoding stages 2302 and 2304; 2) decoding circuit system 2340 executing decoding stage 2306; 3) renaming / allocator unit circuit system 2352 executing allocation stage 2308 and renaming stage 2310; 4) (one or more) scheduler circuit system 2356 executing scheduling stage 2312; 5) (one or more) physical register stacks Circuit system 2358 and memory unit circuit system 2370 perform register read / memory read stage 2314; (one or more) execution clusters 2360 perform execution stage 2316; 6) memory unit circuit system 2370 and (one or more) physical register file circuit system 2358 perform write back / memory write stage 2318; 7) various circuit systems may be involved in the exception handling stage 2322; and 8) retirement unit circuit system 2354 and (one or more) physical register file circuit system 2358 perform commit stage 2324.

[0310] 23(B) shows that a processor core 2390 includes a front end unit circuit system 2330 coupled to an execution engine unit circuit system 2350, and both are coupled to a memory unit circuit system 2370. The core 2390 may be a reduced instruction set architecture computing (RISC) core, a complex instruction set architecture computing (CISC) core, a very long instruction word (VLIW) core, or a hybrid or alternative core type. As another option, the core 2390 may be a special purpose core, such as a network or communication core, a compression engine, a coprocessor core, a general purpose computing graphics processing unit (GPGPU) core, a graphics core, and the like.

[0311] The front end unit circuit system 2330 may include a branch prediction circuit system 2332, which is coupled to an instruction cache circuit system 2334, which is coupled to an instruction translation lookaside buffer (TLB) 2336, which is coupled to an instruction fetch circuit system 2338, which is coupled to a decoding circuit system 2340. In some examples, the instruction cache circuit system 2334 is included in the memory unit circuit system 2370 instead of the front end circuit system 2330. The decoding circuit system 2340 (or decoder) can decode the instruction and generate one or more micro-operations, micro-code entry points, micro-instructions, other instructions or other control signals as output, which are decoded from the original instruction, or otherwise reflect the original instruction, or are derived from the original instruction. The decoding circuit system 2340 may also include an address generation unit (AGU, not shown) circuit system. In some examples, the AGU uses the forwarded register port to generate the LSU address and may further perform branch forwarding (e.g., immediate offset branch forwarding, LR register branch forwarding, etc.). The decode circuit system 2340 may be implemented using a variety of different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLA), microcode read only memory (ROM), and the like. In some examples, the core 2390 includes a microcode ROM (not shown) or other medium that stores microcode for certain macroinstructions (e.g., in the decode circuit system 2340 or otherwise within the front end circuit system 2330). In some examples, the decode circuit system 2340 includes a micro-op or operation cache (not shown) to save / cache decoded operations, micro-tags, or micro-operations generated during decoding or other stages of the processor pipeline 2300. The decode circuit system 2340 may be coupled to the rename / allocator unit circuit system 2352 in the execution engine circuit system 2350.

[0312] Execution engine circuitry 2350 includes rename / allocator unit circuitry 2352, which is coupled to retirement unit circuitry 2354 and a set of one or more scheduler circuitry 2356. Scheduler circuitry 2356 represents any number of different schedulers, including reservation stations, central instruction windows, etc. In some examples, scheduler circuitry(s) 2356 may include an arithmetic logic unit (ALU) scheduler / scheduling circuitry, an ALU queue, an address generation unit (AGU) scheduler / scheduling circuitry, an AGU queue, etc. Scheduler circuitry(s) 2356 are coupled to physical register file(s) circuitry 2358. Each of the (one or more) physical register file circuit systems 2358 represents one or more physical register files, different ones of which store one or more different data types, such as scalar integers, scalar floating points, packed integers, packed floating points, vector integers, vector floating points, status (e.g., instruction pointers, i.e., the address of the next instruction to be executed), etc. In some examples, the (one or more) physical register file circuit systems 2358 include vector register unit circuit systems, write mask register unit circuit systems, and scalar register unit circuit systems. These register units can provide architectural vector registers, vector mask registers, general registers, etc. The (one or more) physical register file circuit systems 2358 are coupled to the retirement unit circuit system 2354 (also called a retirement queue) to demonstrate various ways that register renaming and out-of-order execution can be implemented (e.g., using (one or more) reorder buffers (ROBs) and (one or more) retirement register files; using (one or more) future stacks, (one or more) history buffers, and (one or more) retirement register files; using register maps and pools of registers; etc.). Retirement unit circuitry 2354 and physical register file(s) circuitry 2358 are coupled to execution cluster(s) 2360. Execution cluster(s) 2360 include a set of one or more execution unit circuitry 2362 and a set of one or more memory access circuitry 2364. Execution unit circuitry 2362 may perform various arithmetic, logical, floating point, or other types of operations (e.g., shifts, additions, subtractions, multiplications) on various types of data (e.g., scalar integer, scalar floating point, packed integer, packed floating point, vector integer, vector floating point).While some examples may include several execution units or execution unit circuit systems dedicated to a particular function or set of functions, other examples may include only one execution unit circuit system or multiple execution units / execution unit circuit systems that all perform all functions. (One or more) scheduler circuit systems 2356, (one or more) physical register file circuit systems 2358, and (one or more) execution clusters 2360 are shown as possibly multiple because some examples create separate pipelines for certain types of data / operations (e.g., scalar integer pipelines, scalar floating point / packed integer / packed floating point / vector integer / vector floating point pipelines, and / or memory access pipelines, each of which has its own scheduler circuit system, (one or more) physical register file circuit systems, and / or execution clusters - and in the case of a separate memory access pipeline, in some examples implemented, only the execution cluster of the pipeline has (one or more) memory access unit circuit systems 2364). It should also be understood that in the case of using separate pipelines, one or more of these pipelines can be out-of-order issue / execution, while the rest are in-order.

[0313] In some examples, the execution engine unit circuitry 2350 may perform load store unit (LSU) address / data pipelining to an advanced microcontroller bus (AMB) interface (not shown), as well as address stage and writeback, data stage loads, stores, and branches.

[0314] A set of memory access circuitry 2364 is coupled to memory cell circuitry 2370, which includes data TLB circuitry 2372, which is coupled to data cache circuitry 2374, which is coupled to level 2 (L2) cache circuitry 2376. In some examples, memory access circuitry 2364 may include load unit circuitry, store address unit circuitry, and store data unit circuitry, each of which is coupled to data TLB circuitry 2372 in memory cell circuitry 2370. Instruction cache circuitry 2334 is further coupled to level 2 (L2) cache circuitry 2376 in memory cell circuitry 2370. In some examples, instruction cache 2334 and data cache 2374 are combined into L2 cache circuitry 2376, level 3 (L3) cache circuitry (not shown), and / or a single instruction and data cache in main memory (not shown). The L2 cache circuitry 2376 is coupled to one or more other levels of cache and ultimately to main memory.

[0315] Core 2390 may support one or more instruction sets (e.g., x86 instruction set architecture (optionally with some extensions that have been added with updated versions); MIPS instruction set architecture; ARM instruction set architecture (optionally with optional additional extensions, such as NEON)) that include the instruction(s) described herein. In some examples, core 2390 includes logic to support packed data instruction set architecture extensions (e.g., AVX1, AVX2), allowing operations used by many multimedia applications to be performed using packed data. Example execution unit circuitry.

[0316] Fig.24 An example of (one or more) execution unit circuit systems is illustrated, such as (one or more) execution unit circuit systems 2362 of Figure 23 (B). As shown, (one or more) execution unit circuit systems 2362 may include one or more ALU circuit systems 2401, optional vector / single instruction multiple data (single instruction multiple data, SIMD) circuit systems 2403, load / store circuit systems 2405, branch / jump circuit systems 2407, and / or floating-point unit (Floating-point unit, FPU) circuit systems 2409. ALU circuit system 2401 performs integer arithmetic and / or Boolean operations. Vector / SIMD circuit system 2403 performs vector / SIMD operations on packed data (e.g., SIMD / vector registers). Load / store circuit system 2405 executes load and store instructions to load data from memory into registers, or store data from registers to memory. Load / store circuit system 2405 may also generate addresses. Branch / jump circuitry 2407 causes a branch or jump to a memory address depending on the instruction. FPU circuitry 2409 performs floating point arithmetic. The width of execution unit(s) circuitry 2362 varies depending on the example and may range, for example, from 16 bits to 1024 bits. In some examples, two or more smaller execution units are logically combined to form a larger execution unit (e.g., two 128-bit execution units are logically combined to form a 256-bit execution unit).

[0317] Example register architecture.

[0318] Fig.252500 is a block diagram of a register architecture 2500 according to some examples. As shown, the register architecture 2500 includes a vector / SIMD register 2510, whose width varies from 128 bits to 1024 bits. In some examples, the vector / SIMD register 2510 is physically 512 bits, and only some low-order bits are used depending on the mapping. For example, in some examples, the vector / SIMD register 2510 is a 512-bit ZMM register: the low-order 256 bits are used for the YMM register, and the low-order 128 bits are used for the XMM register. Therefore, there is a register coverage. In some examples, the vector length field selects between the maximum length and one or more other shorter lengths, wherein each such shorter length is half the length of the previous length. Scalar operations are operations performed on the lowest-order data element position in the ZMM / YMM / XMM register; higher-order data element positions are either maintained as the same as before the instruction, or are zeroed, depending on the example.

[0319] In some examples, the register architecture 2500 includes a write mask / predicate register 2515. For example, in some examples, there are 8 write mask / predicate registers (sometimes referred to as k0 to k7), each of which is 16 bits, 32 bits, 64 bits, or 128 bits in size. The write mask / predicate register 2515 can allow merging (e.g., allowing any set of elements in the destination to be protected from update during the execution of any operation) and / or zeroing (e.g., a zeroing vector mask allows any set of elements in the destination to be zeroed during the execution of any operation). In some examples, each data element position in a given write mask / predicate register 2515 corresponds to a data element position at the destination. In other examples, the write mask / predicate register 2515 is scalable and consists of a set number of enable bits for a given vector element (e.g., 8 enable bits for each 64-bit vector element).

[0320] The register architecture 2500 includes a plurality of general registers 2525. These registers may be 16 bits, 32 bits, 64 bits, etc., and may be used for scalar operations. In some examples, these registers are referred to by the names RAX, RBX, RCX, RDX, RBP, RSI, RDI, RSP, and R8 to R15.

[0321] In some examples, the register architecture 2500 includes a scalar floating-point (FP) register file 2545, which is used to perform scalar floating-point operations on 32 / 64 / 80-bit floating-point data using the x87 instruction set architecture extension, or as an MMX register to perform operations on 64-bit packed integer data, and to store operands for some operations performed between MMX and XMM registers.

[0322] One or more flag registers 2540 (e.g., EFLAGS, RFLAGS, etc.) store status and control information for arithmetic, comparison, and system operations. For example, one or more flag registers 2540 can store condition code information, such as carry, parity, auxiliary carry, zero, sign, and overflow. In some examples, one or more flag registers 2540 are referred to as program status and control registers.

[0323] Segment registers 2520 contain segment points for accessing memory. In some examples, these registers are referred to by the names CS, DS, SS, ES, FS, and GS.

[0324] Model-specific registers or machine-specific registers (MSRs) 2535 control and report processor performance. Most MSRs 2535 handle system-related functions and are not accessible to applications. For example, MSRs can provide control of one or more of the following: performance monitoring counters, debug extensions, memory type range registers, thermal and power management, instruction-specific support, and / or processor feature / mode support. Machine check registers 2560 consist of control, status, and error reporting MSRs for detecting and reporting hardware errors. (One or more) control registers 2555 (e.g., CR0-CR4) determine the operating mode of the processor (e.g., processors 1670, 1680, 1638, 1615, and / or 1700) and the characteristics of the task currently being executed. In some examples, MSRs 2535 are a subset of control registers 2555.

[0325] One or more instruction pointer registers 2530 store instruction pointer values. Debug registers 2550 control and allow monitoring of debug operations of the processor or core.

[0326] Memory (mem) management registers 2565 specify the location of data structures used in protected mode memory management. These registers may include global descriptor table registers (GDTR), interrupt descriptor table registers (IDTR), task registers, and local descriptor table registers (LDTR).

[0327] Alternative examples may use wider or narrower registers. In addition, alternative examples may use more, fewer or different register files and registers. Register architecture 2500 may be used, for example, in register file / memory 1958, or in (one or more) physical register file circuit systems 2358. Graphics Execution Unit

[0328] Figure 26A-26B Thread execution logic 2600 is illustrated including an array of processing elements employed in a graphics processor core according to examples described herein. Figure 26A-26B Elements having the same reference numeral (or name) as elements of any other figure herein may operate or function in any manner similar to that described elsewhere herein, but are not limited thereto. Fig.26A represents an execution unit within a general-purpose graphics processor, and Fig.26B Represents an execution unit that can be used within a computing accelerator.

[0329] As in Fig.26A, in some examples, thread execution logic 2600 includes a shader processor 2602, a thread dispatcher 2604, an instruction cache 2606, a scalable execution unit array including a plurality of execution units 2608A-2608N, a sampler 2610, a shared local memory 2611, a data cache 2612, and a data port 2614. In some examples, the scalable execution unit array can be dynamically scaled by enabling or disabling one or more execution units (e.g., execution units 2608A, 2608B, 2608C, 2608D up to any of 2608N-1 and 2608N) based on the computational requirements of the workload. In some examples, the included components are interconnected via an interconnect structure that links to each of the components. In some examples, thread execution logic 2600 includes one or more connections to a memory (such as system memory or cache memory) through instruction cache 2606, data port 2614, sampler 2610, and one or more of execution units 2608A-2608N. In some examples, each execution unit (e.g., 2608A) is a self-contained programmable general-purpose computing unit capable of executing multiple simultaneous hardware threads while processing multiple data elements in parallel for each thread. In various examples, the array of execution units 2608A-2608N is scalable to include any number of separate execution units.

[0330] In some examples, execution units 2608A-2608N are primarily used to execute shader programs. Shader processor 2602 can process various shader programs, and can dispatch execution threads associated with shader programs via thread dispatcher 2604. In some examples, thread dispatcher includes logic for arbitrating thread initiation requests from graphics pipelines and media pipelines and instantiating the requested threads on one or more execution units in execution units 2608A-2608N. For example, a geometry pipeline can dispatch vertex shaders, tessellation shaders, or geometry shaders to thread execution logic for processing. In some examples, thread dispatcher 2604 can also handle runtime thread generation requests from executed shader programs.

[0331] In some examples, execution units 2608A-2608N support an instruction set including native support for many standard 3D graphics shader instructions, so that shader programs from graphics libraries (e.g., Direct 3D and OpenGL) are executed with minimal translation. These execution units support vertex and geometry processing (e.g., vertex programs, geometry programs, vertex shaders), pixel processing (e.g., pixel shaders, fragment shaders), and general processing (e.g., compute and media shaders). Each execution unit in execution units 2608A-2608N is capable of multi-issue single instruction multiple data (single instruction multiple data, SIMD) execution, and multi-threaded operations enable efficient execution environments in the face of high latency memory accesses. Each hardware thread within each execution unit has a dedicated high-bandwidth register file and an associated independent thread state. For pipelines capable of integer operations, single-precision floating-point operations, and double-precision floating-point operations, capable of SIMD branching capabilities, capable of logical operations, capable of transcendental operations, and capable of other miscellaneous operations, execution is multi-issued for each clock. While waiting for data from a shared function in memory or a shared function, dependency logic within execution units 2608A-2608N puts the waiting thread to sleep until the requested data has been returned. While the waiting thread is sleeping, hardware resources can be dedicated to processing other threads. For example, during a delay associated with a vertex shader operation, an execution unit can perform operations for a pixel shader, a fragment shader, or another type of shader program including different vertex shaders. Each example can be applied to use execution utilizing single instruction multiple threads (SIMT) as an alternative to the use of SIMD, or as an addition to the use of SIMD. References to SIMD cores or operations can also be applied to SIMT, or to a combination of SIMD and SIMT.

[0332] Each execution unit in execution units 2608A-2608N operates on an array of data elements. The number of data elements is the "execution size", or the number of lanes for an instruction. An execution lane is a logical unit used for the execution of data element access, masking, and flow control within an instruction. The number of lanes may be independent of the number of physical arithmetic logic units (ALUs) or floating point units (FPUs) used for a particular graphics processor. In some examples, execution units 2608A-2608N support integer and floating point data types.

[0333] The execution unit instruction set includes SIMD instructions. Various data elements can be stored in registers as compact data types, and the execution unit will process various elements based on the data size of the element. For example, when a 256-bit wide vector is operated, the 256 bits of the vector are stored in registers, and the execution unit operates the vector as four independent 64-bit compact data elements (quad-word (Quad-Word, QW) size data elements), eight independent 32-bit compact data elements (double-word (Double Word, DW) size data elements), sixteen independent 16-bit compact data elements (word (Word, W) size data elements) or thirty-two independent 8-bit data elements (byte (byte, B) size data elements). However, different vector widths and register sizes are possible.

[0334] In some examples, one or more execution units may be combined into a fused execution unit 2609A-2609N having a thread control logic (2607A-2607N) common to the fused EU. Multiple EUs may be fused into a EU group. Each EU in a fused EU group may be configured to execute a separate SIMD hardware thread. The number of EUs in a fused EU group may vary according to the example. Additionally, various SIMD widths may be executed EU by EU, including but not limited to SIMD8, SIMD16, and SIMD32. Each fused graphics execution unit 2609A-2609N includes at least two execution units. For example, a fused execution unit 2609A includes a first EU 2608A, a second EU 2608B, and a thread control logic 2607A common to the first EU 2608A and the second EU 2608B. Thread control logic 2607A controls threads executing on fused graphics execution unit 2609A, allowing each EU within fused execution units 2609A-2609N to execute using a common instruction pointer register.

[0335] One or more internal instruction caches (e.g., 2606) are included in the thread execution logic 2600 to cache thread instructions for the execution unit. In some examples, one or more data caches (e.g., 2612) are included to cache thread data during thread execution. Threads executed on the execution logic 2600 may also store explicitly managed data in the shared local memory 2611. In some examples, a sampler 2610 is included to provide texture sampling for 3D operations and media sampling for media operations. In some examples, the sampler 2610 includes specialized texture or media sampling functions to process texture data or media data during the sampling process before providing the sampled data to the execution unit.

[0336] During execution, the graphics pipeline and the media pipeline send thread initiation requests to the thread execution logic 2600 via the thread generation and dispatch logic. Once the group of geometric objects has been processed and rasterized into pixel data, the pixel processor logic (e.g., pixel shader logic, fragment shader logic, etc.) within the shader processor 2602 is called to further calculate output information and cause the results to be written to the output surface (e.g., color buffer, depth buffer, stencil buffer, etc.). In some examples, the pixel shader or fragment shader calculates the value of each vertex attribute, and the value of each vertex attribute is interpolated across the rasterized object. In some examples, the pixel processor logic within the shader processor 2602 then executes the pixel shader program or fragment shader program supplied by the application programming interface (API). To execute the shader program, the shader processor 2602 dispatches the thread to the execution unit (e.g., 2608A) via the thread dispatcher 2604. In some examples, the shader processor 2602 uses the texture sampling logic in the sampler 2610 to access the texture data in the texture map stored in the memory. Arithmetic operations on texture data and input geometry data compute pixel color data for each geometry fragment, or discard one or more pixels without further processing.

[0337] In some examples, data port 2614 provides a memory access mechanism for thread execution logic 2600 to output processed data to memory for further processing on a graphics processor output pipeline. In some examples, data port 2614 includes or is coupled to one or more cache memories (e.g., data cache 2612) to cache data for memory access via the data port.

[0338] In some examples, execution logic 2600 may also include a ray tracer 2605 that may provide ray tracing acceleration functionality. Ray tracer 2605 may support a ray tracing instruction set that includes instructions / functions for ray generation.

[0339] Fig.26B26. The graphics execution unit 2608 may include an instruction fetch unit 2637, a general register file array (GRF) 2624, an architectural register file array (ARF) 2626, a thread arbiter 2622, an issue unit 2630, a branch unit 2632, a set of SIMD floating point units (FPUs) 2634, and a set of dedicated integer SIMD ALUs 2635 in some examples. The GRF 2624 and ARF 2626 include a set of general register files and architectural register files associated with each synchronous hardware thread that can be active in the graphics execution unit 2608. In some examples, per-thread architectural state is maintained in the ARF 2626, while data used during thread execution is stored in the GRF 2624. The execution state of each thread, including the instruction pointer for each thread, can be saved in thread-specific registers in the ARF 2626.

[0340] In some examples, graphics execution unit 2608 has an architecture that is a combination of Simultaneous Multi-Threading (SMT) and fine-grained Interleaved Multi-Threading (IMT). The architecture has a modular configuration that can be fine-tuned at design time based on the target number of simultaneous threads and the number of registers per execution unit, where execution unit resources are divided across logic for executing multiple simultaneous threads. The number of logical threads that can be executed by graphics execution unit 2608 is not limited to the number of hardware threads, and multiple logical threads can be assigned to each hardware thread.

[0341] In some examples, the graphics execution unit 2608 may issue multiple instructions in coordination, each of which may be a different instruction. The thread arbiter 2622 of the graphics execution unit 2608 may dispatch the instruction to one of the send unit 2630, the branch unit 2632, or (one or more) SIMD FPUs 2634 for execution. Each execution thread may access 128 general registers within the GRF 2624, each of which may store 32 bytes accessible as a SIMD 8 element vector with 32-bit data elements. In some examples, each execution unit thread has access to 4 kilobytes within the GRF 2624, but the examples are not limited thereto, and more or less register resources may be provided in other examples. In some examples, the graphics execution unit 2608 is partitioned into seven hardware threads that can independently perform computing operations, but the number of threads per execution unit may also vary from example to example. For example, in some examples, up to 16 hardware threads are supported. In an example where seven threads can access 4 kilobytes, GRF 2624 can store a total of 28 kilobytes. Where 16 threads can access 4 kilobytes, GRF 2624 can store a total of 64 kilobytes. Flexible addressing modes can permit registers to be addressed together, effectively creating wider registers or representing strided rectangular block data structures.

[0342] In some examples, memory operations, sampler operations, and other longer latency system communications are dispatched via "send" instructions executed by message passing send unit 2630. In some examples, branch instructions are dispatched to dedicated branch unit 2632 to facilitate SIMD scatter and eventual convergence.

[0343] In some examples, graphics execution unit 2608 includes one or more SIMD floating point units (FPU(s)) 2634 for performing floating point operations. In some examples, FPU(s) 2634 also support integer computations. In some examples, FPU(s) 2634 can SIMD perform up to M 32-bit floating point (or integer) operations, or SIMD perform up to 2M 16-bit integer or 16-bit floating point operations. In some examples, at least one of the FPU(s) provides extended math capabilities that support high throughput transcendental math functions and double precision 64-bit floating point. In some examples, a set 2635 of 8-bit integer SIMD ALUs are also present and can be specifically optimized to perform operations associated with machine learning computations.

[0344] In some examples, an array of multiple instances of graphics execution unit 2608 may be instantiated in graphics sub-core groupings (e.g., sub-slices). For scalability, product architects may choose the exact number of execution units per sub-core grouping. In some examples, execution unit 2608 may execute instructions across multiple execution lanes. In further examples, each thread executed on graphics execution unit 2608 is executed on a different lane.

[0345] Fig. 27 An additional execution unit 2700 according to an example is illustrated. In some examples, the execution unit 2700 includes a thread control unit 2701, a thread state unit 2702, an instruction fetch / prefetch unit 2703, and an instruction decode unit 2704. The execution unit 2700 additionally includes a register file 2706 that stores registers that can be assigned to hardware threads within the execution unit. The execution unit 2700 additionally includes a dispatch unit 2707 and a branch unit 2708. In some examples, the dispatch unit 2707 and the branch unit 2708 can be used in conjunction with Fig.26B The sending unit 2630 and the branching unit 2632 of the graphics execution unit 2608 operate in a similar manner.

[0346] The execution unit 2700 also includes a computing unit 2710, which includes a plurality of different types of functional units. In some examples, the computing unit 2710 includes an ALU unit 2711, which includes an array of arithmetic logic units. The ALU unit 2711 can be configured to perform 64-bit, 32-bit, and 16-bit integer and floating-point operations. Integer and floating-point operations can be performed simultaneously. The computing unit 2710 may also include a systolic array 2712 and a mathematical unit 2713. The systolic array 2712 includes a wide W and deep D network of data processing units, which can be used to perform vector or other data parallel operations in a systolic manner. In some examples, the systolic array 2712 may be configured to perform matrix operations, such as matrix dot product operations. In some examples, the systolic array 2712 supports 16-bit floating-point operations and 8-bit and 4-bit integer operations. In some examples, the systolic array 2712 may be configured to accelerate machine learning operations. In such examples, systolic array 2712 may be configured with support for the bfloat 16-bit floating point format. In some examples, math unit 2713 may be included to perform a specific subset of math operations in an efficient and lower power manner than ALU unit 2711. Math unit 2713 may include math logic that may be found in shared functional logic of a graphics processing engine provided by other examples (e.g., Figure 4In some examples, math unit 2713 may be configured to perform 32-bit and 64-bit floating point operations.

[0347] Thread control unit 2701 includes logic for controlling the execution of threads within execution units. Thread control unit 2701 may include thread arbitration logic for starting, stopping, and preempting the execution of threads within execution unit 2700. Thread state unit 2702 may be used to store thread states for threads assigned to execute on execution unit 2700. Storing thread states within execution unit 2700 enables quick preemption of threads when they become blocked or idle. Instruction fetch / prefetch unit 2703 may fetch instructions from instruction caches of higher level execution logic (e.g., Fig.26A The instruction fetch / prefetch unit 2703 may also issue a prefetch request for instructions to be loaded into the instruction cache based on an analysis of the current execution thread. The instruction decode unit 2704 may be used to decode instructions to be executed by the computing unit. In some examples, the instruction decode unit 2704 may be used as a secondary decoder to decode complex instructions into constituent micro-operations.

[0348] Execution unit 2700 additionally includes a register file 2706 that can be used by hardware threads executing on execution unit 2700. Registers in register file 2706 can be divided across logic used to execute multiple simultaneous threads within compute unit 2710 of execution unit 2700. The number of logical threads that can be executed by graphics execution unit 2700 is not limited to the number of hardware threads, and multiple logical threads can be assigned to each hardware thread. The size of register file 2706 can vary across examples based on the number of hardware threads supported. In some examples, register renaming can be used to dynamically assign registers to hardware threads.

[0349] Graphics Pipeline

[0350] Fig.28 is a block diagram of another example of a graphics processor 2800 . Fig.28 Elements having the same reference numeral (or name) as elements of any other figure herein may operate or function in any manner similar to that described elsewhere herein, but are not limited thereto.

[0351] In some examples, graphics processor 2800 includes geometry pipeline 2820, media pipeline 2830, display engine 2840, thread execution logic 2850, and rendering output pipeline 2870. In some instances, graphics processor 2800 is a graphics processor within a multi-core processing system that includes one or more general-purpose processing cores. The graphics processor is controlled by register writes to one or more control registers (not shown) or via commands issued to graphics processor 2800 through ring interconnect 2802. In some examples, ring interconnect 2802 couples graphics processor 2800 to other processing components (such as other graphics processors or general-purpose processors). Commands from ring interconnect 2802 are interpreted by command stream converter 2803, which supplies instructions to various components of geometry pipeline 2820 or media pipeline 2830.

[0352] In some examples, command stream converter 2803 directs the operation of vertex fetcher 2805, which reads vertex data from memory and executes vertex processing commands provided by command stream converter 2803. In some examples, vertex fetcher 2805 provides vertex data to vertex shader 2807, which performs coordinate space transformation and lighting operations on each vertex. In some examples, vertex fetcher 2805 and vertex shader 2807 execute vertex processing instructions by dispatching execution threads to execution units 2852A-2852B via thread dispatcher 2831.

[0353] In some examples, execution units 2852A-2852B are arrays of vector processors with instruction sets for performing graphics operations and media operations. In some examples, execution units 2852A-2852B have an attached L1 cache 2851 that is dedicated to each array or shared between arrays. The cache can be configured as a data cache, an instruction cache, or partitioned into a single cache containing data and instructions in different partitions.

[0354] In some examples, the geometry pipeline 2820 includes a tessellation component for performing hardware-accelerated tessellation of 3D objects. In some examples, the programmable hull shader 2811 configures the tessellation operation. The programmable domain shader 2817 provides back-end evaluation of the tessellation output. The tessellation 2813 operates under the direction of the hull shader 2811 and contains dedicated logic for generating a detailed set of geometric objects based on a coarse geometric model that is provided as input to the geometry pipeline 2820. In some examples, if tessellation is not used, the tessellation components (e.g., the hull shader 2811, the tessellation 2813, and the domain shader 2817) can be bypassed.

[0355] In some examples, the complete geometric object may be processed by the geometry shader 2819 via one or more threads dispatched to execution units 2852A-2852B, or may proceed directly to the clipper 2829. In some examples, the geometry shader operates on the entire geometric object, rather than on vertices or patches of vertices as in previous stages of the graphics pipeline. If tessellation is disabled, the geometry shader 2819 receives input from the vertex shader 2807. In some examples, the geometry shader 2819 is programmable by a geometry shader program to perform geometry tessellation when the tessellation unit is disabled.

[0356] Before rasterization, the clipper 2829 processes the vertex data. The clipper 2829 can be a fixed function clipper or a programmable clipper with clipping and geometry shader functions. In some examples, the rasterizer and depth test component 2873 in the render output pipeline 2870 dispatches a pixel shader to convert the geometric object into a pixel-by-pixel representation. In some examples, the pixel shader logic is included in the thread execution logic 2850. In some examples, the application can bypass the rasterizer and depth test component 2873 and access the unrasterized vertex data via the outflow unit 2823.

[0357] The graphics processor 2800 has an interconnect bus, an interconnect structure, or some other interconnect mechanism that allows data and messages to be passed between the main components of the processor. In some examples, the execution units 2852A-2852B and associated logic units (e.g., L1 cache 2851, sampler 2854, texture cache 2858, etc.) are interconnected via data ports 2856 to perform memory accesses and communicate with the rendering output pipeline components of the processor. In some examples, the sampler 2854, cache 2851, 2858, and execution units 2852A-2852B each have a separate memory access path. In some examples, the texture cache 2858 can also be configured as a sampler cache.

[0358] In some examples, the render output pipeline 2870 includes a rasterizer and depth test component 2873 that converts vertex-based objects into associated pixel-based representations. In some examples, the rasterizer logic includes a windower / masker unit for performing fixed-function triangle and line rasterization. In some examples, associated rendering buffers 2878 and depth buffers 2879 are also available. The pixel operation component 2877 performs pixel-based operations on data, but in some instances, pixel operations associated with 2D operations (e.g., using mixed bit block image transfers) are performed by the 2D engine 2841, or replaced by the display controller 2843 using an overlay display plane when displaying. In some examples, a shared L3 cache 2875 can be used for all graphics components, allowing data to be shared without using main system memory.

[0359] In some examples, the graphics processor media pipeline 2830 includes a media engine 2837 and a video front end 2834. In some examples, the video front end 2834 receives pipeline commands from the command stream converter 2803. In some examples, the media pipeline 2830 includes a separate command stream converter. In some examples, the video front end 2834 processes the media commands before sending them to the media engine 2837. In some examples, the media engine 2837 includes a thread generation function for generating threads for dispatching to the thread execution logic 2850 via the thread dispatcher 2831.

[0360] In some examples, the graphics processor 2800 includes a display engine 2840. In some examples, the display engine 2840 is external to the processor 2800 and is coupled to the graphics processor via a ring interconnect 2802, or some other interconnect bus or structure. In some examples, the display engine 2840 includes a 2D engine 2841 and a display controller 2843. In some examples, the display engine 2840 contains dedicated logic that can operate independently of the 3D pipeline. In some examples, the display controller 2843 is coupled to a display device (not shown), which can be a system-integrated display device such as in a laptop or an external display device attached via a display device connector.

[0361] In some examples, the geometry pipeline 2820 and the media pipeline 2830 may be configured to perform operations based on multiple graphics and media programming interfaces and are not dedicated to any one application programming interface (API). In some examples, the driver software for the graphics processor converts API calls dedicated to a specific graphics or media library into commands that can be processed by the graphics processor. In some examples, support is provided for the Open Graphics Library (OpenGL), the Open Computing Language (OpenCL) and / or the Vulkan graphics and computing APIs all from the Khronos Group. In some examples, support may also be provided for the Direct3D library from Microsoft. In some examples, a combination of these libraries may be supported. Support may also be provided for the Open Source Computer Vision Library (OpenCV). If a mapping from the pipeline of a future API to the pipeline of a graphics processor can be performed, future APIs with compatible 3D pipelines will also be supported.

[0362] IP core implementation

[0363] One or more aspects of at least some examples may be implemented by representative code stored on a machine-readable medium that represents and / or defines logic within an integrated circuit (such as a processor). For example, a machine-readable medium may include instructions representing various logic within a processor. When read by a machine, the instructions may cause the machine to manufacture logic for performing the techniques described herein. Such representations (referred to as "IP cores") are reusable units of logic for an integrated circuit that can be stored on a tangible, machine-readable medium as a hardware model describing the structure of the integrated circuit. The hardware model may be supplied to each customer or manufacturing facility that loads the hardware model on a manufacturing machine that manufactures the integrated circuit. The integrated circuit may be manufactured so that the circuit performs the operations described in association with any of the examples described herein.

[0364] Fig.292900 is a block diagram illustrating an IP core development system 2900 that can be used to manufacture an integrated circuit to perform operations according to some examples. The IP core development system 2900 can be used to generate modular, reusable designs that can be incorporated into a larger design or used to build an entire integrated circuit (e.g., a SOC integrated circuit). The design facility 2930 can generate a software simulation 2910 of the IP core design using a high-level programming language (e.g., C / C++). The software simulation 2910 can be used to design, test, and verify the behavior of the IP core using a simulation model 2912. The simulation model 2912 can include functional simulation, behavioral simulation, and / or timing simulation. A register transfer level (RTL) design 2915 can then be created or synthesized from the simulation model 2912. The RTL design 2915 is an abstraction of the behavior of an integrated circuit (including associated logic executed using the modeled digital signals) that models the flow of digital signals between hardware registers. In addition to the RTL design 2915, a lower level design at the logic level or transistor level can also be created, designed, or synthesized. Thus, the specific details of the initial design and simulation can be different.

[0365] The RTL design 2915 or equivalent may be further synthesized by the design facility into a hardware model 2920, which may be in a hardware description language (HDL) or some other representation of physical design data. The HDL may be further simulated or tested to verify the IP core design. The IP core design may be stored using a non-volatile memory 2940 (e.g., a hard disk, flash memory, or any non-volatile storage medium) for delivery to a third-party manufacturing facility 2965. Alternatively, the IP core design may be transmitted via a wired connection 2950 or a wireless connection 2960 (e.g., via the Internet). The manufacturing facility 2965 may then manufacture an integrated circuit based at least in part on the IP core design. The manufactured integrated circuit may be configured to perform operations according to at least some of the examples described herein.

[0366] References to "some examples," "examples," etc. indicate that the examples described may include a particular feature, structure, or characteristic, but not every example may necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same example. In addition, when a particular feature, structure, or characteristic is described in conjunction with an example, it is considered to be within the knowledge of those skilled in the art to affect such feature, structure, or characteristic in conjunction with other examples, whether or not explicitly described.

[0367] Furthermore, in the examples described above, unless specifically noted otherwise, separating language such as phrases "at least one of A, B, or C" or "A, B and / or C" are intended to be understood to mean A, B, or C, or any combination thereof (i.e., A and B, A and C, B and C, and A, B and C).

[0368] Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the disclosure as set forth in the claims.

Claims

1. A device, comprising: a first plurality of physical processing devices of a first type; a second plurality of physical processing devices of a second type; power management unit means for monitoring telemetry data on said first plurality of physical processing devices and said second plurality of physical processing devices and for updating hardware feedback telemetry data based at least in part on hardware telemetry data and thread runtime telemetry data to provide a hint to merge tasks onto at least one core of said second type; as well as The thread runtime telemetry device is used to maintain the hardware feedback telemetry data.

2. The apparatus of claim 1 , wherein the hardware telemetry data of the first plurality of cores comprises the number of cores of each type that are concurrently active and the operating frequency of the active cores, and the hardware telemetry data of the second plurality of cores is a maximum frequency, an average frequency, and an average concurrency.

3. The apparatus of any one of claims 1-2, wherein the hint is used to merge all energy efficiency-oriented tasks onto a core whose energy efficiency capability is a set value.

4. The apparatus of any one of claims 1-2, wherein the hint is used to merge all performance-oriented tasks onto a core whose performance capability is a set value.

5. The apparatus of any one of claims 1-4, wherein the thread runtime telemetry data comprises a compute type capability for each category.

6. The apparatus of any one of claims 1-5, wherein the thread runtime telemetry data comprises wait times for each category.

7. The apparatus of any one of claims 1-6, wherein the thread runtime telemetry data comprises a power value for each category.

8. The apparatus of any one of claims 1-7, wherein the hardware telemetry data comprises system operating characteristics.

9. A system, comprising: a first plurality of physical processing devices of a first type; a second plurality of physical processing devices of a second type; power management unit means for monitoring threads and hardware telemetry data on the first plurality of physical processing devices and the second plurality of physical processing devices, and for updating hardware feedback telemetry data based at least in part on the hardware telemetry data and thread runtime telemetry data to provide a hint to merge tasks onto at least one core of the second type; Thread runtime telemetry means for maintaining said hardware feedback telemetry data; as well as The memory is used to store system software for receiving the hardware feedback telemetry data.

10. The system of claim 9, wherein the hardware telemetry data of the first plurality of cores includes the number of cores of each type that are concurrently active and the operating frequency of the active cores, and the hardware telemetry data of the second plurality of cores is a maximum frequency, an average frequency, and an average concurrency.

11. The system of any one of claims 9-10, wherein the hint is used to merge all energy efficiency oriented tasks onto a core whose energy efficiency capability is a set value.

12. The system of any one of claims 9-10, wherein the hint is used to merge all performance-oriented tasks onto a core having a performance capacity of a set value.

13. The system of any one of claims 9-12, wherein the thread runtime telemetry data includes a compute type capability for each category.

14. The system of any one of claims 9-13, wherein the thread runtime telemetry data includes wait times for each category.

15. The system of any one of claims 9-14, wherein the thread runtime telemetry data includes a power value for each category.

16. A method comprising: monitoring hardware and thread telemetry data of a first plurality of physical processor cores of a first type and a second plurality of physical processor cores of a second type; determining to update hardware feedback telemetry data based at least in part on the monitored hardware and thread telemetry data; as well as The hardware feedback telemetry data is used to provide a hint to the operating system to consolidate tasks onto one of the core types.

17. The method of claim 16, wherein the hint is used to merge all energy efficiency oriented tasks onto a core having a set value of energy efficiency capability.

18. The method of claim 16, wherein the hint is used to merge all performance-oriented tasks onto cores having a performance capability of a set value.

19. The method of any one of claims 16-18, wherein the thread runtime telemetry data includes a compute type capability for each category.

20. The method of any of claims 16-19, wherein the thread runtime telemetry data includes wait times for each category.

Images