Holistic Global Performance and Power Management

Abstract

A method and apparatus are provided that provides holistic global performance and power management. In an embodiment, logic (e.g., coupled to each of the plurality of compute nodes) triggers the determination of policies for power and performance management across the plurality of compute nodes. The policies are coordinated across the plurality of compute nodes to manage jobs toward one or more objective functions, where the jobs include multiple tasks running simultaneously on the multiple compute nodes.

Description

[Technical field]

[0001] The present disclosure relates generally to the field of electronics and, more particularly, in some embodiments, to power management for servers and other computing devices. [Background technology]

[0002] High performance computing (HPC) systems can include many nodes connected by a fabric for distributed computing. Furthermore, applications are divided into tasks that run simultaneously across the nodes in the HPC system. These tasks are broken down into sequential milestones, and the tasks are expected to reach each of these milestones simultaneously.

[0003] Unfortunately, if any node completes its work towards the next milestone slower than the others, progress across the application will halt until the slowest task completes its work. When this happens, the application loses potential performance and power is wasted on the node that has to wait. [Brief description of the drawings]

[0004] The detailed description is provided with reference to the accompanying drawings, in which the left-most digit of a reference number identifies the figure in which the reference number first appears. Use of the same reference number in different drawings indicates similar or identical items.

[0005] [Figure 1] FIG. 1 illustrates a block diagram of various computing systems according to some embodiments. [Diagram 2] FIG. 2 illustrates a block diagram of a Holistic Global Performance and Power Management (HGPPM) system, according to an embodiment. [Diagram 3]FIG. 3 illustrates a block diagram of hierarchical partially observable Markov decision process (H-POMDP) ​​agent interaction details, according to an embodiment. [Figure 4] FIG. 4 illustrates a block diagram of various computing systems according to some embodiments. [Diagram 5] FIG. 5 illustrates a block diagram of various computing systems according to some embodiments. [Figure 6] FIG. 6 illustrates a block diagram of various computing systems according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006] In the following description, numerous specific details are set forth in order to provide a thorough understanding of various embodiments. However, various embodiments may be practiced without the specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to obscure certain embodiments. Furthermore, various aspects of the embodiments may be implemented using a variety of means, such as integrated semiconductor circuitry ("hardware"), computer-readable instructions organized into one or more programs ("software"), some combination of hardware and software, etc. For purposes of this disclosure, "logic" shall mean hardware, software, or some combination thereof.

[0007] As mentioned above, a high performance computing (HPC) system may include many nodes coupled by a high speed network fabric for distributed computing. As described herein, a "node" generally refers to a compute element (which may include one or more processors, such as general purpose processors, graphics processors, etc., as described herein), a connection to the network fabric, a login component, a service component, and possibly memory, IO (input / output) devices, or other components. Generally, an application (also referred to herein as a "job") is divided into tasks that run simultaneously across many nodes (e.g., tens of thousands) in an HPC system. There may be one or many tasks mapped to each node, or a single task may run across one or more cores. Tasks may consist of the same program running on different data of the same problem set. Tasks are broken down into sequential milestones, and all tasks are expected to complete their computations between milestones in the same amount of time, leading to a so-called bulk synchronization style of computation. In the milestones, tasks may be synchronized through operations such as global barriers.

[0008] Unfortunately, if any core or node completes its work between synchronizations more slowly than others (for any one of many reasons), progress across the application stops until the slowest task completes its work. When this happens, the application loses potential performance and power is wasted on the core or node that must wait. This problem has many names, including load imbalance, application or operating system jitter, and performance variation. Load imbalance can have many causes, ranging from static factors such as manufacturing variability that leads to a distribution of performance in hardware components, to dynamic factors such as page faults occurring at different times on different cores, operating system interference that affects only some cores, recoverable hardware errors that temporarily cripple only one core or node, uneven distribution of work among tasks within an application, etc.

[0009] As HPC systems continue to grow in size and complexity, load imbalance is becoming a significant source of performance degradation and power waste. Manufacturing variability is particularly problematic. Modern processors cannot run floating-point intensive workloads at their maximum core frequency without exceeding thermal design and power limits. Furthermore, two processors of the same model and stepping require different power to achieve the same core frequency. The industry expects processor performance variations of over 20% for a given power budget.

[0010] To this end, some embodiments provide a holistic global performance and power management. More specifically, a new performance and power management framework is described for coordinating software and hardware policies across (e.g., all) nodes in a job while managing the job toward configurable objective functions (e.g., maximize performance within the job power cap, maximize efficiency within the job power cap, etc.). One application of the framework is to solve the load balancing problem mentioned above.

[0011] Additionally, some embodiments provide a Holistic Global Performance and Power Management (HGPPM) framework that coordinates performance and power management decisions across (e.g., all) nodes in a job while managing the job toward a job power cap or other configurable objective function, such as performance maximization, efficiency maximization (e.g., minimum energy-delay product), performance maximization while managing toward a job power cap, efficiency maximization while managing toward a job power cap, etc. HGPPM technology is based, at least in part, on a hierarchical feedback-guided control system implemented with a fast hierarchical partially observable Markov decision process (H-POMDP) ​​reinforcement learning (RL) method. Such embodiments can mitigate application load imbalance by introducing key capabilities for steering power between hierarchical system domains and coordinating other broader optimizations across software and hardware abstraction boundaries. For example, in some embodiments, HGPPM can achieve higher performance or efficiency by simultaneously coordinating and optimizing the selection of power allocations between hierarchical system domains to achieve load balancing, and the selection of application algorithms from a repertoire to find the best performing option for a given system architecture, problem size, or power allocation.

[0012] Further, the techniques described herein can be provided in a variety of computing systems (e.g., non-mobile computing devices such as desktops, workstations, servers, rack systems, etc., and mobile computing devices such as smartphones, tablets, UMPCs (ultra-mobile personal computers), laptop computers, Ultrabooks, smart watches, smart glasses, etc.), including those described with reference to FIGS. 1 through 6. More specifically, FIG. 1 illustrates a block diagram of a computing system 100 according to an embodiment. FIG. 1 is a schematic representation and is not meant to reflect a physical configuration. System 100 includes one or more processors 102-1 through 102-N (generally referred to herein as "multiple processors 102" or "processor 102"). Processors 102 can communicate via an interconnect (fabric) 104. It is also possible that one or more processors can share a connection to the interconnect fabric. Each processor can include various components, only some of which are described with reference to processor 102-1 for clarity. And each of the remaining processors 102-2 through 102-N may include components that are the same as or similar to those described with reference to processor 102-1.

[0013] In an embodiment, processor 102-1 may include one or more processor cores 106-1 through 106-M (referred to herein as "multiple cores 106" or more generally as "cores 106"), cache 108 (which in various embodiments may be a shared cache or a private cache), and / or router 110. Processor core 106 may be implemented on a single integrated circuit (IC) chip. It may also be implemented with multiple integrated circuits in the same package. Additionally, the chip may include one or more shared and / or private caches (such as cache 108), buses or interconnects (such as buses or interconnects 112), logic 150, memory controllers (including, for example, flash memory, NVM (non-volatile memory) such as SSD (solid state drive)) or other components as described with reference to Figures 4-6. In other embodiments, the components of Figure 1 may be arranged in a different manner, for example, the router may be external to the processor while the VR, memory controller, and main memory may be internal to the processor.

[0014] In one embodiment, router 110 may be used to communicate between various components of processor 102-1 and / or system 100. Additionally, processor 102-1 may include multiple routers 110. Additionally, multiple routers 110 may be in communication to enable data routing between various components internal or external to processor 102-1. In some embodiments, if there are multiple routers, some may be internal to the processor and some may be external.

[0015] The cache 108 may store data (including, e.g., instructions) utilized by one or more components of the processor 102-1, such as the cores 106. For example, the cache 108 may locally cache data stored in (volatile and / or non-volatile) memory 114 (also interchangeably referred to herein as “main memory”) for fast access by the components of the processor 102. As shown in FIG. 1, the memory 114 may be in communication with the processor 102 via the interconnect 104. In an embodiment, the (possibly shared) cache 108 may have various levels, e.g., the cache 108 may be a mid-level cache and / or a last-level cache (LLC) (L1, L2 cache, etc.). Additionally, each core 106 may include a level 1 (L1) cache (116-1) (generally referred to herein as “L1 cache 116”). Various components of the processor 102-1 may communicate with the cache 108 directly, through a bus fabric (eg, bus 112) and / or through a memory controller hub.

[0016] System 100 also includes a (e.g., platform) power source 125 (e.g., a direct current (DC) power source or an AC power source) for providing power to one or more components of system 100. Power source 125 may include PV (photovoltaic) panels, wind generators, thermal generators, water / hydro turbines, etc. In some embodiments, power source 125 includes one or more battery packs (e.g., charged by one or more of the PV panels, wind generators, thermal generators, water / hydro turbines, plug-in power sources (e.g., coupled to an AC power grid)) and / or a plug-in power source.

[0017] The power supply 125 may be coupled to the components of the system 100 via a voltage regulator (VR) 130. Additionally, while FIG. 1 illustrates one power supply 125 and a single voltage regulator 130, additional power supplies and / or voltage regulators may be utilized. For example, one or more processors 102 may have corresponding voltage regulators and / or power supplies. Also, the voltage regulator 130 may be coupled to the processor 102 via a single power plane (e.g., powering all cores 106) or multiple power planes (e.g., each power plane may power a different core or group of cores and / or other components of the system 100). Additionally, while FIG. 1 illustrates the power supply 125 and the voltage regulator 130 as separate components, the power supply 125 and the voltage regulator 130 may be incorporated into other components of the system 100. For example, all or a portion of the VR 130 may be incorporated into the source 125, the SOC (such as that described with reference to FIG. 6), and / or the processor 102.

[0018] As shown in FIG. 1, memory 114 may be coupled to other components of system 100 through memory controller 120. System 100 also includes logic 150 to facilitate and / or perform one or more operations with reference to the HGPPM techniques / embodiments described herein. For example, logic 150 may perform operations corresponding to performance and / or power management of one or more compute nodes and / or components of system 100 (e.g., processor 102, memory controller 120, memory 114 (sometimes referred to herein as “external memory”), caches 116, 108, and / or interconnect fabric 104, 112, etc.). Additionally, although logic 150 is shown in several arbitrary locations within system 100, logic 150 may be located elsewhere within system 100.

[0019] Further, embodiments provide a scalable dynamic technique that adjusts performance and power management policies across all nodes in a job and across software and hardware abstraction layers while managing the job holistically toward a configurable objective function, which may include, but is not limited to, maximizing performance while meeting power caps, minimizing performance differences between compute elements (nodes or cores) while meeting power caps (mitigating load imbalance), maximizing efficiency, etc. Such techniques are generally referred to herein as Holistic Global Performance and Power Management (HGPPM), which in one embodiment is based at least in part on hierarchical machine learning algorithms.

[0020] Traditional HPC power managers have many limitations. First, applying a uniform power cap to each node exacerbates performance differences between nodes, resulting in uneven frequencies for each node and reduced application performance. Since the industry typically expects node performance to vary by 20% or more at a given power cap, it is important to mitigate these performance differences so they do not exacerbate them. Second, traditional power managers lack the ability to coordinate software and hardware policies. Unfortunately, software and hardware policies have historically been coordinated through independent control systems. This leads to interference between the control systems that degrades results. It is important to coordinate software and hardware policies under a unified control system. Third, traditional HPC power managers lack scalability. They employ a centralized design that cannot be adapted to coordinate policies across a large number (e.g., tens of thousands) of nodes in future systems. A fundamentally different design is needed. Finally, traditional HPC systems lack flexibility. To meet the highest performance and efficiency challenges of exascale systems, new policy knobs need to be designed and more opportunities need to be exploited to optimize the system, but current solutions lack a power manager framework that can understand and control new policies. Also, flexibility is needed in programming the tradeoffs between performance and power that the power manager makes (e.g., in some cases, efficiency takes precedence over performance), but traditional power managers tend to only support performance-biased objective functions. Current management techniques are too brittle.

[0021] Embodiments of HGPPM are groundbreaking in that they simultaneously address the problems of load imbalance, scalability, and flexibility while introducing a critical capability to align software and hardware policies, which are considered key requirements for increasing performance and efficiency of exascale systems. More specifically, HGPPM improves upon the prior art in a number of important ways. HGPPM introduces the capability to detect and mitigate load imbalance within a job by balancing the power allocated to the compute elements and equalizing their loads. HGPPM is the first technology that can mitigate a variety of causes of load imbalance ranging from manufacturing variability to application or operating system jitter, hardware errors, and intrinsic causes of imbalance where the application or operating system, etc., do not divide work equally among the compute elements (also interchangeably referred to herein as "nodes" or "cores"). Furthermore, at least one embodiment provides a load balancing technique that is synergistic with concurrently managing jobs toward power caps.

[0022] Embodiments of the HGPPM also introduce significant new capabilities: (a) tuning policy optimization across software and hardware abstraction boundaries, (b) extensibility to new types of policies with robust techniques for policy optimization, and / or (c) flexibility through support for management towards configurable objective functions. No other performance or power manager supports these capabilities while simultaneously scaling to tune policies across all compute elements in a job. The scalability, robustness, and flexibility of such embodiments are groundbreaking.

[0023] Examples of new policies and optimizations enabled by embodiments of HGPPM include, but are not limited to, (a) tuning applications for maximum performance or efficiency through new policy knobs that control the number of cores each application task can utilize, and (b) tuning processors for better performance or efficiency through new policy knobs that control how aggressively the processor performs speculative arithmetic operations, memory prefetch operations, etc. The design of new policies and optimizations is believed to be critical to meeting the performance and efficiency challenges of exascale systems, and HGPPM embodiments are the first performance and power management framework that can orchestrate such optimizations. Additionally, HGPPM's hierarchical learning framework further improves scalability and makes load balancing more responsive to better application performance or efficiency.

[0024] In an embodiment, the HGPPM is used to globally adjust performance and power policies across (e.g., all) nodes in a job while managing the job toward a configurable objective function. There is an HGPPM instance bound to each job. The HGPPM approach solves the scalability challenge of determining policies for many nodes (e.g., tens of thousands of nodes) in a job by employing a hierarchical design based on Hierarchical Partially Observable Markov Decision Process (H-POMDP) ​​machine learning. In particular, one embodiment employs a hierarchy of identical reinforcement learning agents. As described herein, the use of "node," "core," "computational element," etc. are interchangeable, e.g., each such unit refers to a computing component capable of performing one or more of the operations described herein with reference to that unit.

[0025] Reinforcement learning agents optimize policies not through models, but through interactions with the environment and empirical experimentation. They continuously evaluate the outcomes of actions to adapt them to the best outcomes, where the quality of the outcomes is defined by an objective function. Experiments are selected in a systematic way to efficiently navigate the space of all possible policy options. According to an embodiment, the reinforcement learning algorithm employs a technique called Stochastic Policy Gradient to efficiently navigate while still achieving good or acceptable outcomes.

[0026] In H-POMDP, each agent is identical and operates independently on a subproblem of the overall problem, but all agents use the same objective function, and the subproblems are defined hierarchically so that the decisions of one agent constrain the option space that its children can consider. That way, parents can guide their children's actions to help them identify the best policy more quickly, or to preclude them from selecting certain policy options. In this hierarchical fashion, performance and power policy decisions are coordinated from the root to the leaves of the reinforcement learning agent tree hierarchy.

[0027] FIG. 2 illustrates a block diagram of an HGPPM system, according to an embodiment. Although FIG. 2 illustrates a tree depth of three, this is for illustrative purposes only, and deeper or shallower tree depths can be used in various embodiments. In an embodiment, each computing node of the system illustrated in FIG. 2 can include one or more components described in the computing systems of FIG. 1, FIG. 4, FIG. 5, and / or FIG. 6. More specifically, global coordination is enabled by a scalable, hierarchical, k-variable tree design. System policies are decomposed hierarchically (e.g., allocating power among cabinets 202-0 to 202-m (where "cabinet" generally refers to multiple nodes), and then allocating power among nodes). As described further herein, good global power allocation decisions are obtained via H-POMDP reinforcement learning agents operating at each node of the tree. Aggregation of power and / or performance telemetry (back up the tree from leaves toward the root) and distribution of control (from parents to children) are provided by a scalable overlay network (SCON) 204. SCON is a logical network implemented on top of one of the physical networks in an HPC system. The physical network may be an in-band network used by the application (e.g., network fabric) or an out-of-band network such as Ethernet (e.g., following the IEEE (Institute of Electrical and Electronics Engineers) 802.3 standard). In one embodiment, the physical network may be the same as or similar to the network 403 described with reference to Figures 4 and 5. In Figure 2, "DN" indicates a dedicated node (e.g., a node reserved and not used by an application), and the small box in each DN and compute node indicates an H-PO MDP agent. As described herein, in an embodiment, the H-PO MDP agent may be a reinforcement learning agent. As described herein, per-core (or per-portion of a core) policies (such as allocation of power to each core or core portion) may be provided. As shown in Figure 2, each compute node may include one or more cores.Also, in an embodiment, the leaf H-POMDP agent is responsible for coordinating any policies within the compute node (which may include coordinating per-core policies).

[0028] In one embodiment, a stochastic policy gradient technique is used. Each node of the HGPPM tree implements the reinforcement learning POMDP algorithm and applies a policy gradient method to search for an optimal policy. For example, a policy can be thought of as a probability distribution over a set of discrete knob settings. For example, the knobs contain a set of options for how a parent agent can allocate power among its children. The policy is evaluated by sampling knob settings from the distribution, testing them several times, and measuring the resulting rewards. The policy is improved through a method similar to gradient ascent in the embodiment. To use a gradient-based method, the policy can be made differentiable by adapting a stochastic softmax policy, which is made differentiable. The gradient of the reward is then estimated with respect to the policy, and the policy can be stepped in the gradient direction to move to a policy that maximizes the reward. The stochastic policy can be adapted to avoid the "local maxima" pitfalls of simpler gradient methods and balance the tradeoff between exploration and exploitation.

[0029] In one embodiment, the following operations can be used to implement the stochastic policy gradient: The stochastic policy gradient algorithm can be implemented using the Natural Actor-Critic framework: where a is the action (aka knob setting), r is the reward, and α is the step size.

number

[0030] As mentioned before, a probabilistic softmax policy can be used to make the policy differentiable and suitable for gradient-based learning. In particular, an exponential family parameterization of the multinomial distribution is used to assign real weights θ i The probability of getting knob setting j out of n possibilities when sampling from the probability distribution for knob i is given by

number

[0031] The gradient required for the Natural Actor Critic algorithm is

number

number

number

number

[0032] As described herein, HGPPM can be applied to correct load imbalance of an application by balancing power among nodes in a job. In one example of how a load balancing problem can be decomposed hierarchically, the load balancing problem for an entire job is divided into load balancing among cabinets used in the job, then among nodes in each cabinet, then among tasks mapped to each node, then among cores running each task. At each granularity, performance is dynamically compared and power is directed from leading computational elements to lagging elements (with reference to reaching the next milestone and reaching the barrier in the sequence) to maximize or improve the performance of the application.

[0033] One embodiment maps the process of job load balancing to a reinforcement learning abstraction by having each agent learn the best allocation of its input power budget among its children and defining an objective function such that a) mismatch in child agent performance is penalized and b) aggregate performance is rewarded. Here, the aggregate performance is taken as the minimum performance obtained by the child agents. The performance of each child agent is the average or median (or other function) of some number of samples. Each agent learns how to divide its input budget (from its parent) among its children to get the best performance from them. The children, in turn, take their budget and divide it among their children, etc. Decisions at the lowest level of the tree can specify how the hardware should divide the power among different types of processors and external memory resources. Performance can be defined based on a number of metrics. For example, at the lowest level of the H-POMDP tree, the metrics could be core frequency, a measurement of each core's progress towards the next application milestone (provided to HGPPM via programmer-created annotations or automatically inferred by analysis of performance counters or other means), the runtime of the application's phases completed so far between milestones, the rate of instructions retired, the rate of main memory accesses, etc.

[0034] In general, the objective function evaluated by each reinforcement learning agent in the H-POMDP is an aggregate of the objective function values ​​of each child. Many aggregations are possible, including the minimum, average, variance, etc. of the children's objective function values. In one embodiment, if the objective function is to maximize performance, the performance of a node can be defined as the minimum performance of any active core of a processor in the node, the performance of a cabinet can be defined as the minimum performance of any active node in the cabinet, and the performance of a job can be defined as the minimum performance of active cabinets in the job. The aggregate calculations can be performed by HGPPM techniques and / or through the assistance of SCONs in the HPC system.

[0035] As described herein, embodiments of HGPPM can tune different kinds of policies (beyond power budget) and tune multiple types at once. In this mode, HGPPM technology composes policies into a joint policy. A reinforcement learning agent experiments with the joint policy options and optimizes the joint policy according to an objective function. In one example, consider a hierarchy ending with one reinforcement learning agent for each node. Consider the case where the agent is tasked with jointly learning two policies: how to split the node's power budget among the various types of hardware resources at the node, and how many threads / cores each software process on the node should utilize. The agent creates a joint policy with one option for each combination of power budget options and parallelism options. The learning agent tests new power budget options and new parallelism options together to measure the combined effect on the objective function and navigates towards the best joint policy over time.

[0036] Figure 3 zooms in on a leaf H-POMDP RL agent of one of the compute nodes of Figure 2. The interaction between the RL agent, the application, and the processors in the node is shown, including the inputs to the RL agent (labeled Observables in Figure 3) and the new policy settings output by RL. This figure shows management of the example policies discussed above: number of threads per application process and partitioning of the node power budget among hardware resources. In one embodiment, the H-POMDP RL agent captures performance and phase signals from the application. In other embodiments, the performance and phase signals may be inferred automatically by the HGPPM without programmer annotations of the application (as described above). From the processor, the H-POMDP RL agent captures energy signals. The output of the RL agent is new policy settings (e.g., new settings for number of threads per application process, or a new repartition of the node power budget among the hardware components of the node).

[0037] The observables are combined in various ways to define a desired objective function. The objective function is evaluated and fed into a learning algorithm as a reward (as described above). The RL agent uses this reward signal to measure the impact of different policy settings. The RL agent explores the policy space over time, experimenting with policy settings by enacting policies (in Figure 3, the output policy settings are labeled as Actions) and measuring the impact on the observables and reward signals. As described above, the RL agent explores and navigates the policy space over time in an efficient manner until it identifies the best policy settings.

[0038] The diagram in Figure 3 is an example of an embodiment of the HGPPM where multiple policies are composed. As more policies are composed, the size of the search space can grow exponentially. There are also dependencies between policies that complicate the search space. The combined policy described above (division of node power budget among node hardware resources and number of threads / cores per software process) is an example of a combined policy with complex interdependencies. The optimal power division among resources depends on the balance of communication, memory, and computation within the application process, which in turn depends on the number of threads / cores employed by the application process, and the optimal number of threads / cores an application process uses depends on the available communication, memory, and computation bandwidth (although the amount of bandwidth varies depending on how much power each resource is allocated).

[0039] One embodiment of HGPPM scales to handle large search spaces with complex tradeoffs by employing a stochastic policy gradient reinforcement learning technique. The stochastic policy gradient technique estimates the gradient of the objective function index with respect to the policy and then takes a step in the gradient direction. In the next round, the combined policy option tried is a step away (in the gradient direction) from the previous one. The stochastic policy gradient method navigates the exponential search space by taking steps in the gradient direction, trying only those directions that are expected to improve the results, rather than exhaustively searching the entire space.

[0040] One drawback of gradient-based search techniques is that they tend to assume that the search space is convex. If this is not the case, the method is not guaranteed to land on a globally optimal decision. For this reason, a stochastic policy gradient algorithm may be employed instead of the usual policy gradient algorithm. Instead of selecting the next policy option based on the gradient direction, a probability is assigned to samples from all options and distributions. If there is a non-zero probability, a step is taken in a direction not indicated by the gradient, which is optimal. Doing so allows this embodiment to avoid local maxima. In a stochastic policy gradient algorithm, instead of learning a policy option that maximizes the objective function, a policy distribution that maximizes it is learned. The step in the gradient direction updates the probability distribution, which tends to assign more probability in the direction that is consistent with the gradient, but there is still a chance of selecting other options.

[0041] The approach adopted in one embodiment also addresses three classical challenges using reinforcement learning techniques and H-POMDP. The first challenge has to do with reinforcement learning balancing between exploring new regions of the policy space and exploiting the best known optimal policy. Methods that overemphasize exploration will use suboptimal policies most of the time. Methods that overemphasize exploitation may settle for suboptimal policies and leave potential gains on the table. The stochastic policy gradient technique used in some embodiments ensures that new regions of the search space are tried since all policy options have nonzero probability in the policy distribution.

[0042] Furthermore, taking gradient steps to update the distribution to add weight to promising directions seems to risk slowly creating a strong bias that undermines the "best known" option and exploration, but one embodiment incorporates a regularizer component that counterbalances the excessive bias by slowly making the distribution more uniform. The strength of these opposing forces is systematically adjusted at runtime. One embodiment measures how stable and predictable the relationship is between the objective function indicators and the policy. The more stable and predictable the relationship, the stronger the power of the gradient step (note: the amount of bias added at each step is constant, but frequent gradient steps can effectively strengthen the power). If the relationship is unstable or unpredictable, the regularizer wins.

[0043] The second challenge is one that affects all control systems, not just reinforcement learning control systems: the noise challenge. The embodiments address the noise challenge in part through the opposing force mechanisms described above and in part through digital signal processing techniques. In particular, noise causes the relationship between the objective function index and the policy to be poorly predicted. The embodiments take gradient steps only when the noise is at a low level in the immediate future and the relationship is predictable (in other words, when the gradient results can be trusted to lead towards a better policy). In periods of high noise, regularization will prevail. One embodiment has free parameters that set the bias strength of each gradient step and each regularization step. These parameters are set such that as long as the application experiences periods with a stable and predictable relationship between the objective function index and the policy, the bias of the gradient steps will prevail overall and the policy distribution will approach the optimum over time. Many methods can be used to set the bias and regularization step sizes. Some are manual, but can follow standard techniques, and some are automatic, on-line methods.

[0044] Another mechanism that some embodiments use to solve the noise problem is digital signal processing. Many signal processing methods can be used. In one example, a moving average or median filter is applied to the objective function signal. This type of filter tends to smooth the signal using a short history of previous values. It is possible to apply a filter to the input of the objective function. This can have advantages when the objective function divides by a signal of relatively small amplitude, since noise in the denominator will be amplified by the division operation.

[0045] The third challenge is ensuring stability of control despite distributed adjustments across the hierarchy of H-POMDP reinforcement learning. Consider the following example: Suppose a parent changes to a new power budget before the children have a chance to explore an optimal split of that budget among their children. If that happens, the parent may base its guesses on incomplete data from the children about how well the power budget will work. In reality, the parents may never get complete data from the children, and the data is probabilistic. Nevertheless, unless the children are given time to find a good split of the budget, the H-POMDP may never converge, or it may take too long to converge to a good overall policy.

[0046] There are many solutions to this problem, according to some embodiments. One example involves arranging for the reinforcement learning agent to operate at a predefined time interval with durations that become coarsening as the movement up the hierarchy (from leaves to root). Another approach is to allow the entire hierarchy of agents to self-configure the adjustment timescale. Levels of the hierarchy operate as fast as possible, but the speed is rate-limited to ensure accuracy. That is, the parent block waits for input from the child, and the child only transmits performance or other information required to evaluate the objective function up to the parent when the child has achieved a good policy (e.g., a good allocation of power). As a result, the parent cannot set a new policy (e.g., a new power budget for the child) before the child is ready. This self-configuring strategy has the advantage of maximizing the responsiveness of the overall policy optimization. According to some embodiments, there are many ways to determine when a good policy has been reached. One standard method is a convergence test: if the change in policy in the last k iterations is less than epsilon, then a good policy has been reached. k and epsilon are free parameters that can be adjusted according to an offline manual procedure.

[0047] At the last level of the H-POMDP hierarchy, the reinforcement learning agent has no children. At this hierarchical level, the agent can choose when to sample the objective function indicators and try new policy options. The objective function can be sampled, for example, at regular time intervals coarser than the phase duration, or at phase change events, at regular time intervals finer than the phase duration. A new policy can be tested after one or more samples have been collected, although the number of samples taken for each test can be variable or fixed.

[0048] Furthermore, some embodiments can be synergistic with phase-based optimization. In phase-based optimization, one copy of state is instantiated for the reinforcement learning agent for each application phase. A similar or identical H-POMDP hierarchy of reinforcement learning agents can be used, but they operate in different states depending on which phase the application is in. At any given time, the embodiment determines the current application phase and loads the exact state. The reinforcement learning agent is tasked with optimizing for the same objective function in all phases, but different phases may have their own policies. Furthermore, the definition of the current phase and the policies for each phase can be determined in many ways, as described above. Some examples include obtaining information from the programmer via annotations in the application (or other software layers). Determining the current phase can also be inferred through dynamic analysis of activity in different computational resources (through the use of hardware event counters).

[0049] 4 illustrates a block diagram of a computing system 400 according to an embodiment. The computing system 400 may include one or more central processing units (CPUs) 402 or processors that communicate over an interconnection network (or bus) 404. The processor 402 may be a general purpose processor, a network processor (such as those used in mobile phones, smart phones, etc.), or other type of processor (including a reduced instruction set computer (RISC) processor or a complex instruction set computer (CISC) processor).

[0050] Various types of computer networks 403 may be utilized, including wired (e.g., Ethernet, Gigabit, fiber, etc.) or wireless networks (cellular, 3G (third generation mobile phone technology or third generation wireless format (UWCC)), 4G (fourth generation (wireless / mobile communications)), low power embedded (e.g., LPE), etc.). Furthermore, the processor 402 may have a single or multiple core design. A processor 402 with a multi-core design may integrate different types of processor cores on the same integrated circuit (IC) die. Also, a processor 402 with a multi-core design may be implemented as a symmetric or asymmetric multi-processor.

[0051] In an embodiment, the one or more processors 402 may be the same as or similar to the processor 102 of Figure 1. For example, the one or more processors 402 may include one or more cores 106 and / or caches 108. Additionally, the operations described with reference to Figures 1-3 may be performed by one or more components of the system 400.

[0052] The chipset 406 may also be in communication with the interconnect network 404. The chipset 406 may include a graphics and memory control hub (GMCH) 408. The GMCH 408 may include a memory controller 410 (which may be the same as or similar to the memory controller 120 of FIG. 1) that communicates with the memory 114. The system 400 may also include logic 150 in various locations, such as those shown in FIG. 4, but which may be located elsewhere (not shown) in the system 400. The memory 114 may store data, including a set of instructions, to be executed by the CPU 402, or any other device included in the computing system 400. In one embodiment, memory 114 may include one or more volatile / non-volatile storage (or memory) devices such as random access memory (RAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), static RAM (SRAM), or other types of storage devices such as hard disks, nanowire memory, ferroelectric transistor random access memory (FeTRAM), magnetoresistive random access memory (MRAM), flash memory, spin transfer memory (STTRAM), resistive random access memory, 3D cross-point memory such as PCM (phase change memory), solid state drives (SSD) with NAND / NOR memory, etc. Additional devices may communicate via interconnect network 404, such as multiple CPUs and / or multiple system memories.

[0053] GMCH 408 may also include a graphics interface 414 that communicates with a graphics accelerator 416. In one embodiment, graphics interface 414 may communicate with graphics accelerator 416 via an Accelerated Graphics Port (AGP) or Peripheral Component Interconnect (PCI) (or PCI Express (PCIe) interface). In an embodiment, display device 417 (such as a flat panel display, touch screen, etc.) may communicate with graphics interface 414 via a signal converter that converts digital representations of images stored in a storage device, such as video memory, system memory, etc., into display signals that are interpreted and displayed by the display. The display signals generated by the display device may pass through various control devices before being interpreted and subsequently displayed by display device 417.

[0054] Hub interface 418 may enable GMCH 408 to communicate with an input / output control hub (ICH) 420. ICH 420 may provide an interface to I / O devices that communicate with computing system 400. ICH 420 may communicate with bus 422 through a peripheral bridge (or controller) 424, such as a peripheral component interconnect (PCI) bridge, a universal serial bus (USB) controller, or other type of peripheral bridge or controller. Bridge 424 may provide a data path between CPU 402 and peripheral devices. Other types of topologies may be utilized. Multiple buses may also communicate with ICH 420, for example, through multiple bridges or controllers. Additionally, other peripherals that communicate with ICH 420 may include integrated drive electronics (IDE) or small computer system interface (SCSI) hard drives, USB ports, keyboards, mice, parallel serial ports, floppy disk drives, digital output support (e.g., digital video interface (DVI)), or other devices, in various embodiments.

[0055] The bus 422 can communicate with an audio device 426, one or more disk drives 428, and a network interface device 430 (e.g., communicating with the computer network 403 via a wired or wireless interface). As shown, the network interface device 430 can be coupled to an antenna 431 and communicates with the network 403 wirelessly (e.g., via an Institute of Electrical and Electronics Engineers (IEEE) 802.111 interface (including IEEE 802.11a / b / g / n, etc.), a cellular interface, 3G, 4G, LPE, etc.). Other devices can communicate over the bus 422. Also, in some embodiments, various components (e.g., the network interface device 430) can communicate with the GMCH 408. Furthermore, the processor 402 and the GMCH 408 can be combined to form a single chip. Additionally, in other embodiments, the graphics accelerator 416 can be included within the GMCH 408.

[0056] Additionally, computing system 400 may include volatile and / or non-volatile memory (or storage). For example, non-volatile memory includes read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrical EPROM (EEPROM), disk drives (e.g., 428), floppy disks, compact disk ROM (CD-ROM), digital versatile disks (DVD), flash memory, magneto-optical disks, or other types of non-volatile machine-readable media capable of storing electronic data (e.g., including instructions).

[0057] FIG 5 illustrates a computing system 500 arranged in a point-to-point (PtP) configuration, according to an embodiment. In particular, FIG 5 illustrates a system in which processors, memory, and input / output devices are interconnected by multiple point-to-point interfaces. The operations described with reference to FIGS 1-4 may be performed by one or more components of the system 500.

[0058] As shown in Figure 5, system 500 may include several processors, of which only two processors 502 and 504 are shown for clarity. Processors 502 and 504 may include local memory controller hubs (MCHs) 506 and 508 that enable communication with memories 510 and 512, respectively. Memories 510 and / or 512 may store various data, such as those described with reference to memory 114 of Figures 1 and / or 4. Additionally, in some embodiments, MCHs 506 and 508 may include memory controller 120 and / or logic 150 of Figures 1-4.

[0059] In an embodiment, the processors 502 and 504 may be one of the processors 402 described with reference to FIG. 2. The processors 502 and 504 may exchange data via a point-to-point (PtP) interface 514 using PtP interface circuits 516 and 518, respectively. The processors 502 and 504 may also exchange data with a chipset 520 via respective PtP interfaces 522 and 524 using point-to-point interface circuits 526, 528, 530 and 532, respectively. The chipset 520 may exchange data with a high performance graphics circuit 534 via a high performance graphics interface 536, e.g., using a PtP interface circuit 537. As described in FIG. 4, in some embodiments, the graphics interface 536 may be coupled to a display device (e.g., display 417).

[0060] As shown in Figure 5, one or more of the cores 106 and / or caches 108 of Figure 1 may be located within processors 502 and 504. However, other embodiments may be present in other circuits, logic units, or devices within system 500 of Figure 1. Additionally, other embodiments may be distributed throughout several of the circuits, logic units, or devices shown in Figure 5.

[0061] Chipset 520 can communicate with bus 540 using PtP interface circuitry 541. Bus 540 can have one or more devices, such as bus bridge 542, I / O devices 543, communicating with it. Through bus 544, bridge 542 can communicate with keyboard / mouse 545, communication devices 546 (e.g., modems, network interface devices, other communication devices capable of communicating with computer network 403, including via antenna 431, as described with reference to network interface device 430), audio I / O devices, and / or data storage 548. Data storage 548 can store code 549 that can be executed by processors 502 and / or 504.

[0062] In some embodiments, one or more of the components described herein may be embodied on a system-on-chip (SOC) device. FIG. 6 illustrates a block diagram of a SOC package according to an embodiment. As shown in FIG. 6, SOC 602 includes one or more central processing unit (CPU) cores 620, one or more graphics processor unit (GPU) cores 630, an input / output (I / O) interface 640, and a memory controller 120. Various components of SOC package 602 may be interconnected or coupled to a bus / network, such as SCON 204 described herein with reference to other figures. SOC package 602 may also include more or fewer components, such as those described herein with reference to other figures. Furthermore, each component of SOC package 602 may include one or more other components, such as those described herein with reference to other figures. In one embodiment, SOC package 602 (and its components) may be provided on one or more integrated circuit (IC) dies, for example, packaged on a single semiconductor device.

[0063] 6, the SOC package 602 is coupled to a main memory 114 (external to the SOC package 602) via an interface, such as a memory controller 120. In an embodiment, the memory 114 (or a portion thereof) may be integrated onto the SOC package 602.

[0064] I / O interface 640 may be coupled to one or more I / O devices 670, for example, via interconnects and / or buses as described herein with reference to other figures. I / O devices 670 may include one or more of a keyboard, mouse, touchpad, display, image / video capture device (camera, video camera / video recorder, etc.), touch screen, speaker, etc. Additionally, in an embodiment, SOC package 602 may include / integrate logic 150. Alternatively, logic 150 may be provided external to SOC package 602 (i.e., as discrete logic).

[0065] The following examples relate to further embodiments. Example 1 includes an apparatus including logic coupled to each node of a plurality of nodes and causing determination of a policy for power and performance management for transmission to the plurality of nodes, the policy causing coordination of power and performance management across the plurality of nodes, the policy managing a job toward one or more objective functions, the job including a plurality of tasks operating simultaneously on the plurality of nodes. Example 2 includes the apparatus of example 1, wherein the logic determines a separate policy for each of the plurality of nodes. Example 3 includes the apparatus of example 1, wherein the logic determines a distinct policy for at least a portion of each of the plurality of nodes. Example 4 includes the apparatus of example 1, wherein the one or more objective functions include one or more of maximizing performance while meeting a power cap, maximizing energy efficiency while meeting a power cap, minimizing performance difference between the multiple nodes while meeting the power cap, maximizing performance or maximizing efficiency, or maximizing efficiency while maximizing performance while meeting a power cap. Example 5 includes the apparatus of example 1, wherein the logic operates according to a hierarchical machine learning operation. Example 6 includes the apparatus of example 1, wherein the logic performs one or more operations to address one or more of a load imbalance problem, a scalability problem, or a flexibility problem. Example 7 includes the apparatus of example 1, wherein the policy coordinates power and performance management across all nodes in the job. Example 8 includes the apparatus of example 1, wherein the policy coordinates power and performance management across all nodes in the job and across software and hardware abstraction layers. Example 9 includes the apparatus of example 1, wherein the logic determines the policy according to a stochastic policy gradient method. Example 10 includes the apparatus of example 1, wherein the plurality of nodes form a cabinet, and the policy is hierarchically decomposed among one or more cabinets and then decomposed among the plurality of nodes. Example 11 includes the apparatus of example 1, further including a scalable overlay network coupling the plurality of nodes. Example 12 further includes a scalable overlay network coupling the plurality of nodes, the scalable overlay network including the apparatus of example 1 providing aggregation of power or performance telemetry and distribution of control. Example 13 includes the apparatus of example 1, wherein a system on a chip (SOC) integrated circuit includes the logic and the memory. Example 14 includes the apparatus of Example 1, wherein each node of the plurality of nodes includes a processor having one or more processor cores, an image processing unit having one or more processor cores, and one or more of a connection to a network fabric, a login component, a service component, a memory, or an input / output device.

[0066] Example 15 includes a method including causing a determination of a policy for power and performance management for each node of a plurality of nodes and transmitting the policy to the plurality of nodes, the policy causing coordination of power and performance management across the plurality of nodes, the policy managing a job toward one or more objective functions, the job including a plurality of tasks operating simultaneously on the plurality of nodes. Example 16 includes the method of example 15, further including determining a separate policy for each of the plurality of nodes. Example 17 includes the method of example 15, further including determining a distinct policy for at least a portion of each of the plurality of nodes. Example 18 includes the method of example 15, wherein the one or more objective functions include one or more of maximizing performance while meeting a power cap, maximizing energy efficiency while meeting a power cap, minimizing performance difference between the multiple nodes while meeting the power cap, maximizing performance or maximizing efficiency, or maximizing efficiency while maximizing performance while meeting a power cap. Example 19 includes the method of example 15, wherein the determining operates according to a hierarchical machine learning operation. Example 20 includes the method of example 15, wherein the determining is performed to address one or more of a load imbalance problem, a scalability problem, or a flexibility problem. Example 21 includes the method of example 15, wherein the policy further includes coordinating power and performance management across all nodes in the job. Example 22 includes the method of example 15, wherein the policy further includes coordinating power and performance management across all nodes in the job and across software and hardware abstraction layers. Example 23 includes the method of example 15, further including determining the policy according to a stochastic policy gradient method. Example 24 includes the method of example 15, wherein the plurality of nodes form a cabinet, and the policy is hierarchically decomposed among one or more cabinets and then decomposed among the plurality of nodes. Example 25 includes the method of example 15, further including coupling the plurality of nodes via a scalable overlay network. Example 26 includes the method of example 15, further including coupling the plurality of nodes via a scalable overlay network, the scalable overlay network providing aggregation of power or performance telemetry and distribution of control.

[0067] Example 27 is a computer-readable medium including one or more instructions that, when executed on a processor, configure the processor to perform one or more operations, including causing determination of a policy for power and performance management for each node of a plurality of nodes, and transmitting the policy to the plurality of nodes, the policy causing coordination of power and performance management across the plurality of nodes, the policy managing a job toward one or more objective functions, the job including a plurality of tasks operating simultaneously on the plurality of nodes. Example 28 includes the computer-readable medium of Example 27, including the one or more instructions, when executed on the processor, to configure the processor to perform the one or more operations, the one or more instructions further including determining a separate policy for each of the plurality of nodes. Example 29 includes the method of Example 27, including the one or more instructions, when executed on the processor, configuring the processor to perform the one or more operations, the one or more instructions further including determining a distinct policy for at least a portion of each of the plurality of nodes.

[0068] Example 30 includes an apparatus including means for carrying out the method according to any one of the previous examples.

[0069] Example 31 includes a machine-readable medium that includes machine-readable instructions that, when executed, implement a method or device according to any one of the previous examples.

[0070] In various embodiments, for example, with reference to Figures 1-6, the operations described herein may be implemented in hardware (e.g., circuitry), software, firmware, microcode, or a combination thereof, and provided as a computer program product including a tangible (e.g., non-transitory) machine-readable or computer-readable medium storing instructions (or software procedures) used to program a computer to perform the processes described herein. Also, the term "logic" may include, by way of example, software, hardware, or a combination of software and hardware. Machine-readable media may include storage devices such as those described with respect to Figures 1-3.

[0071] Furthermore, such tangible computer-readable media may be downloaded as a computer program product, where the program is transferred by data signals (e.g., in a carrier wave, other propagation medium, etc.) from a remote computer (e.g., a server) to a requesting computer (e.g., a client) via a communications link (e.g., a bus, modem, or network connection).

[0072] The term "one embodiment" or "embodiment" in the specification means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one implementation. The appearances of the phrase "in one embodiment" in various places in the specification may or may not all refer to the same embodiment.

[0073] Also, in the specification and claims, the terms "coupled" and "connected," along with their derivatives, may be used. In some embodiments, "connected" may be used to indicate that two or more elements are in direct physical or electrical contact with each other. "Coupled" may mean that two or more elements are in direct physical or electrical contact. However, "coupled" can also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other.

[0074] Thus, although the embodiments have been described in language specific to structural features and / or methodological acts, it is to be understood that the claimed subject matter is not limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms for implementing the claimed subject matter.

Claims

1. 1. An apparatus including a multi-chip integrated circuit (IC) package, the multi-chip IC package comprising: one or more graphics processing unit (GPU) IC chips including one or more GPU cores for performing graphics operations; A plurality of processor IC chips, each processor IC chip comprising: A plurality of processor cores; a power manager for controlling power consumption and performance of the processor IC chip, the power manager managing one or more clock frequencies associated with one or more of the plurality of processor cores; an interconnect IC chip coupled to at least one of the plurality of processor IC chips, the interconnect IC chip comprising: a memory controller for coupling a memory device to a system component including the plurality of processor cores; an input / output (I / O) interface for coupling one or more I / O devices to the interconnect IC chip; and another power manager for coordinating performance and power policies across all IC chips in the multi-chip IC package, including the plurality of processor IC chips, the one or more GPU IC chips, and the interconnect IC chip; the other power manager aggregates power and / or performance telemetry data received from each power manager of the plurality of processor IC chips and sends control signals to the plurality of power managers of the plurality of processor IC chips to indicate one or more power and / or performance constraints to the plurality of power managers based on the aggregated power and / or performance telemetry data; and an interconnect IC chip, wherein each power manager of the plurality of processor IC chips controls the power consumption and / or performance of its respective processor IC chip in accordance with power and / or performance limitations implied by the other power managers of the other processor IC chips, independent of other power managers of the other processor IC chips.

2. 2. The apparatus of claim 1, further comprising a scalable control fabric coupling to said plurality of processor IC chips and said interconnect IC chips.

3. 3. The apparatus of claim 2, wherein the scalable control fabric comprises a scalable overlay operable on a physical communications fabric that supports data signals in addition to control signals.

4. The apparatus of claim 1 , wherein in response to receiving a first control signal, the power manager modifies a frequency of at least one associated core.

5. 2. The apparatus of claim 1, wherein the other power manager includes circuitry executing firmware or software for aggregating the power and / or performance telemetry data received from the power manager and for sending the control signals to the power manager.

6. 6. The apparatus of claim 5, wherein the power manager includes circuitry that executes firmware or software for controlling the power consumption and / or performance of each processor integrated circuit chip in accordance with the power and / or performance limits implied by the other power manager.

7. The apparatus of claim 6 , wherein the another power manager and the power manager include identical circuitry but perform different functions based on a hierarchical relationship between the another power manager and the power manager.

8. The apparatus of claim 7 , wherein to perform the different functions, the another power manager and the power manager execute different program code sequences.

9. The apparatus of claim 8 , wherein the different program code sequences comprise firmware or software program code.

10. 10. The apparatus of claim 9, wherein the firmware or software comprises machine learning firmware or software for learning how to allocate power among execution resources of the system components and to individual processors.

11. The apparatus of claim 10 , wherein the machine learning firmware or software comprises reinforcement learning firmware or software.

12. System memory; a multi-chip integrated circuit (IC) package coupled to the system memory, the multi-chip IC package comprising: one or more graphics processing unit (GPU) IC chips including one or more GPU cores for performing graphics operations; A plurality of processor IC chips, each processor IC chip comprising: A plurality of processor cores; a power manager for controlling power consumption and performance of the processor IC chip, the power manager managing one or more clock frequencies associated with one or more of the plurality of processor cores; an interconnect IC chip coupled to at least one of the plurality of processor IC chips, the interconnect IC chip comprising: a memory controller for coupling the plurality of processor cores to the system memory; an input / output (I / O) interface for coupling one or more I / O devices to the interconnect IC chip; and another power manager for coordinating performance and power policies across all IC chips in the multi-chip IC package, including the plurality of processor IC chips, the one or more GPU IC chips, and the interconnect IC chip; the other power manager aggregates power and / or performance telemetry data received from a plurality of power managers including the power manager, and transmits control signals to the plurality of power managers to indicate one or more power and / or performance constraints to the plurality of power managers based on the aggregated power and / or performance telemetry data; and an interconnect IC chip, wherein each power manager of the plurality of processor IC chips controls the power consumption and / or performance of its respective processor IC chip in accordance with power and / or performance limits implied by the other power managers of the other processor IC chips, independent of other power managers of the other processor IC chips.

13. 13. The system of claim 12, further comprising a scalable control fabric coupling to said plurality of processor IC chips and said interconnect IC chips.

14. 14. The system of claim 13, wherein the scalable control fabric comprises a scalable overlay operable on a physical communications fabric that supports data signals in addition to control signals.

15. The system of claim 12 , wherein in response to receiving a first control signal, the power manager modifies a frequency of at least one associated core.

16. 13. The system of claim 12, wherein the other power manager includes circuitry executing firmware or software for aggregating the power and / or performance telemetry data received from the power manager and for sending the control signals to the power manager.

17. 17. The system of claim 16, wherein the power manager includes circuitry that executes firmware or software for controlling the power consumption and / or performance of each processor integrated circuit chip in accordance with the power and / or performance limits implied by the other power manager.

18. 20. The system of claim 17, wherein the another power manager and the power manager include identical circuitry but perform different functions based on a hierarchical relationship between the another power manager and the power manager.

19. 20. The system of claim 18, wherein to perform the different functions, the another power manager and the power manager execute different program code sequences.

20. 20. The system of claim 19, wherein the different program code sequences include firmware or software program code.

21. 21. The system of claim 20, wherein the firmware or software comprises machine learning firmware or software for learning how to allocate power among execution resources of the system and to individual processors.

22. 22. The system of claim 21, wherein the machine learning firmware or software comprises reinforcement learning firmware or software.

23. A computer-readable medium having program code stored thereon, the program code, when executed by a machine, causing the machine to: executing graphics operations by one or more graphics processing unit (GPU) cores of one or more GPU IC chips; managing, with a plurality of power managers, the power consumption and performance of a corresponding plurality of processor IC chips of a multi-chip package, each processor IC chip including a plurality of processor cores, a corresponding power manager of the plurality of power managers managing one or more clock frequencies associated with one or more of the plurality of processor cores; coupling said plurality of processor cores to a memory controller of a memory device by an interconnect IC chip; coupling one or more input / output (I / O) devices to the memory controller through an I / O interface of the interconnect IC chip; coordinating performance and power policies across all IC chips in the multi-chip package, including the one or more processor IC chips, the one or more GPU IC chips, and the interconnect IC chip, by a separate power manager of the interconnect IC chip; aggregating, by the other power manager, power and / or performance telemetry data received from a plurality of power managers of the plurality of power managers and sending control signals to the plurality of power managers of the plurality of processor IC chips based on the aggregated power and / or performance telemetry data to indicate one or more power and / or performance constraints to the plurality of power managers; and an interconnected IC chip, wherein each power manager of the plurality of processor IC chips controls the power consumption and / or performance of its respective processor IC chip in accordance with power and / or performance limitations implied by the other power managers of the other processor IC chips, independent of other power managers of the other processor IC chips.

24. 24. The machine-readable medium of claim 23, further comprising a scalable control fabric coupling to said plurality of processor IC chips and said interconnect IC chips.

25. 25. The machine-readable medium of claim 24, wherein the scalable control fabric comprises a scalable overlay operable on a physical communications fabric that supports data signals in addition to control signals.

26. 24. The machine-readable medium of claim 23, wherein in response to receiving a first control signal, the power manager modifies a frequency of at least one associated core.

27. 24. The machine-readable medium of claim 23, wherein the other power manager includes circuitry that executes firmware or software for aggregating the power and / or performance telemetry data received from the power manager and for sending the control signals to the power manager.

28. 28. The machine-readable medium of claim 27, wherein the power manager includes circuitry that executes firmware or software for controlling the power consumption and / or performance of each processor integrated circuit chip in accordance with the power and / or performance limits implied by the other power manager.

29. 30. The machine-readable medium of claim 28, wherein the another power manager and the power manager include identical circuitry but perform different functions based on a hierarchical relationship between the another power manager and the power manager.

30. 30. The machine-readable medium of claim 29, wherein to perform the different functions, the another power manager and the power manager execute different program code sequences.

31. 31. The machine-readable medium of claim 30, wherein the different program code sequences comprise firmware or software program code.

32. 32. The machine-readable medium of claim 31, wherein the firmware or software comprises machine learning firmware or software for learning how to allocate power among execution resources of a system and to individual processors.

33. 33. The machine-readable medium of claim 32, wherein the machine learning firmware or software comprises reinforcement learning firmware or software.

Images