Power distribution in a computer system

Abstract

A method for power distribution in a computer system associates several tokens with multiple frequency domains (e.g., cores) of a central processing unit (CPU) computer chip. The number of tokens assigned to a CPU is based on the CPU's power budget. The cores are configured in a ring topology. A token pool travels around the ring, picking up excess tokens from cores with excess tokens and granting them to cores needing additional tokens. Tokens acquired by a core allow the core to increase its operating frequency by the increment represented by the token. Thus, power usage is weighted toward heavily loaded cores and away from lightly loaded cores. The CPU's total power usage remains within the power budget. The method allocates power optimally to maintain turbo frequencies over longer periods by not allowing the control unit to increase frequency if there are no useful higher frequencies that would benefit the workload.

Description

Technical Field

[0001] The present invention generally relates to the field of power usage in computers, and more particularly to efficient power distribution between frequency domains, such as in central processing unit (CPU) cores, and between CPUs in a multi-CPU computer system.

Background Art

[0002] Electronic components consume (dissipate) power to operate, thereby generating heat within the component. This heat raises the temperature within the component. A cooling system that carries away the heat helps manage the component temperature. Another way to manage temperature is to regulate the power used by the component. For example, a central processing unit (CPU) can be associated with a power budget that limits the power usage above which the CPU cannot operate. An on-chip controller (OCC) throttles the CPU operating frequency to prevent exceeding the power budget limit.

Summary of the Invention

[0003] According to an aspect of the present invention, (although not necessarily in the following order) (i) receiving, by a token pool, a power allowance request from a first power-consuming device among a plurality of power-consuming devices of an integrated circuit; (ii) determining, in response to receiving the power allowance request, that a second power-consuming device among the plurality of power-consuming devices has a power allowance surplus; (iii) receiving, in response to the second power-consuming device having the power allowance surplus, the power allowance surplus from the second power-consuming device; (iv) transmitting the power allowance surplus to the first power-consuming device via the token pool; and (v) in response to transmitting the power allowance surplus to the first power-consuming device, (a) increasing a first power usage limit corresponding to the first power-consuming device based on the power allowance surplus, and (b) decreasing a second power usage limit corresponding to the second power-consuming device based on the power allowance surplus. A method, computer program product, and system for performing the above operations are provided.

Brief Description of the Drawings

[0004]

Figure 1

Figure 2

Figure 3

Figure 4A

Figure 4B

Figure 4C

Figure 4D

Figure 4E

Figure 4F

Figure 5

Figure 6A

Figure 6B

Figure 6C

Figure 6D

DETAILED DESCRIPTION OF THE INVENTION

[0005] In some embodiments of the present invention, the power management system uses a ring topology (distributed token passing mechanism) to distribute a fixed number of power tokens among the frequency domains of the electronic modules. Each power token represents a portion of the total power that an electronic module is permitted to dissipate. A fixed number of power tokens collectively represent the predetermined maximum consumption (dissipation) power allocated to the electronic modules.

[0006] An electronic module can have several frequency domains. Each frequency domain operates at a controllable clock frequency that is independent of other frequency domains of the electronic module. A higher operating frequency can be achieved by the frequency domain processing the workload at a higher speed, but this means dissipating more power as a result. In some embodiments, a CPU core (in a multi-core CPU) is an example of such a frequency domain. The power dissipated by several frequency domains varies according to their respective workloads. The power management system provides power tokens from a rotating token pool, which in turn polls and addresses each frequency domain in a round-robin fashion (using a ring topology). A frequency domain can, in conjunction with communicating with the token pool, accept additional tokens from the token pool or donate unwanted tokens to the token pool if necessary (and available). A frequency domain that requires more power tokens when the token pool has no (or not enough) available power tokens (in a "power shortage" state) waits for tokens to be relinquished to the token pool by other frequency domains. Other frequency domains that communicate with the token pool in each of their respective turns can respond by placing unwanted (surplus) power tokens in the token pool. When the token pool makes a full circle and communicates with the power shortage region again, the power shortage region can acquire the available power tokens from the token pool.

[0007] A frequency domain operates at a frequency according to the number of power tokens held by the frequency domain. Overall, the electronic module (having multiple frequency domains) executes its workload more efficiently while staying within the allocated power budget (represented by the total number of power tokens allocated among the multiple frequency domains).

[0008] This detailed description section is divided into subsections such as (i) hardware and software environment, (ii) exemplary embodiments, (iii) further comments or embodiments or both, and (iv) definitions. I. Hardware and Software Environment

[0009] The present invention can be a system, method, or computer program product, or a combination thereof, in the integration at any possible technical detail level. The computer program product can include a computer-readable storage medium (or media) having therein computer-readable program instructions for causing a processor to execute aspects of the present invention.

[0010] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. The computer-readable storage medium can be, for example, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing, but is not limited thereto. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded devices such as punch cards, or raised structures within grooves storing instructions, and any suitable combination of the foregoing. A computer-readable storage medium as used herein should not be construed as being a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., an optical pulse passing through an optical fiber cable), or an electrical signal transmitted through an electrical wire.

[0011] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to respective computing / processing devices, or may be downloaded from an external computer or an external storage device via a network, such as, for example, the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium within each respective computing / processing device.

[0012] The computer-readable program instructions for carrying out the operation of the present invention may be source code or object code written in any combination of one or more programming languages, including assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or object-oriented programming languages such as Smalltalk(R) or C++ and procedural programming languages such as the "C" programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer as a stand-alone software package, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, in order to carry out aspects of the present invention, an electronic circuit, including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), can execute the computer-readable program instructions by utilizing the state information of the computer-readable program instructions to personalize the electronic circuit.

[0013] Aspects of the invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0014] These computer readable program instructions may be provided to a computer processor or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the computer processor or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more blocks of the flowchart and / or block diagram. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the computer readable storage medium containing instructions comprises a manufacture including instructions which implement aspects of the function / act specified in one or more blocks of the flowchart and / or block diagram.

[0015] The computer readable program instructions may also be loaded onto a computer, other programmable apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus, or other devices implement the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0016] The flowcharts and block diagrams in the figures illustrate the possible architectures, functionality, and operation of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of a module that includes one or more executable instructions for implementing the specified logical function. In some alternative embodiments, the functions shown in the blocks may be performed in an order different from that shown in the drawings. For example, two blocks shown in succession may, in fact, be executed simultaneously, substantially simultaneously, in a partially or wholly temporally overlapping manner depending on the functionality involved, or the blocks may, in some cases, be executed in reverse order. It should also be noted that each block of the block diagram or flowchart diagram, or both, and combinations of blocks in the block diagram or flowchart diagram, or combinations thereof, may be implemented by a special-purpose hardware-based system that performs a particular function or act or that implements a combination of special-purpose hardware instructions and computer instructions.

[0017] Next, an embodiment of a possible hardware and software environment for software or method or both according to the present invention will be described in detail with reference to the figures. FIG. 1 is a functional block diagram showing various parts of a networked computer system 100 including a server system 102, a client computer 104, a communication network 114, a server computer 200, a communication unit 202, a processor set 204, an input / output (I / O) interface set 206, a memory 208, a persistent storage 210, a display 212, an external device 214, a random access memory (RAM) 230, a cache memory 232, and a power management program 300.

[0018] Server system 102 represents various computer subsystems in the present invention in many respects. Therefore, some parts of server system 102 will be described in the following paragraphs.

[0019] Server system 102 can be a laptop computer, a tablet computer, a netbook computer, a personal computer (PC), a desktop computer, a personal digital assistant (PDA), a smart phone, or any programmable electronic device capable of communicating with a client subsystem via communication network 114. Power management program 300 is the totality of machine-readable instructions or data or both used to generate, manage, and control certain software functions detailed in the exemplary embodiments subsection of this detailed description section below.

[0020] Server system 102 can communicate with other computer subsystems via communication network 114. Communication network 114 can be, for example, a local area network (LAN), a wide area network (WAN) such as the Internet, or a combination of the two, and can include a wired connection, a wireless connection, or an optical fiber connection. Generally, communication network 114 can be any combination of connections and protocols that support communication between a server and a client subsystem.

[0021] Server system 102 is shown as a block diagram having many double arrows. These double arrows (the reference numbers are not separate) represent a communication fabric that enables communication between various components of server system 102. This communication fabric can be implemented in any architecture designed to pass data or control information or both between processors (e.g., microprocessors, communication and network processors) within the system, system memory, peripheral devices, and any other hardware components. For example, the communication fabric can be implemented, at least in part, using one or more buses.

[0022] Memory 208 and persistent storage 210 are computer-readable storage media. Generally, memory 208 can include any suitable volatile or non-volatile computer-readable storage media. It should be further noted that currently or in the near future or both, (i) external device 214 may be able to supply some or all of the memory for server system 102, and / or (ii) devices external to server system 102 may be able to provide memory for server system 102.

[0023] Power management program 300 is typically stored in persistent storage 210 through one or more memories of memory 208 for access or execution or both by one or more of each computer processor set 204. Persistent storage 210 is (i) at least as persistent as a propagating signal, (ii) stores a program (including its software logic or data or both) on a tangible medium (e.g., magnetic or optical region), and (iii) is not as substantially persistent as persistent storage. Alternatively, the data storage may be more persistent or permanent or both than the type of storage provided by persistent storage 210.

[0024] The power management program 300 may include both machine-readable instructions and executable instructions or actual data (i.e., the type of data stored in a database) or both. In this particular embodiment, the persistent storage 210 includes a magnetic hard disk drive. By way of some possible variations, the persistent storage 210 may include a solid state hard drive, a semiconductor memory device, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, or any other computer-readable storage medium capable of storing program instructions or digital information.

[0025] The medium used by the persistent storage 210 can also be removable. For example, a removable hard drive may be used for the persistent storage 210. Other examples include optical disks and magnetic disks, thumb drives, and smart cards that are inserted into a drive for transfer to another computer-readable storage medium, which is also part of the persistent storage 210.

[0026] In these examples, the communication unit 202 communicates with other data processing systems or devices external to the server system 102. In these examples, the communication unit 202 includes one or more network interface cards. The communication unit 202 can communicate by using one or both of a physical communication link and a wireless communication link. Any software module described herein may be downloaded to a persistent storage device (e.g., the persistent storage 210) through the communication unit (e.g., the communication unit 202).

[0027] The I / O interface set 206 enables the input and output of data to other devices that can be locally connected in data communication with the server computer 200. For example, the I / O interface set 206 provides a connection to an external device 214. Typically, the external device 214 includes devices such as a keyboard, keypad, touch screen, or some other suitable input device, or a combination thereof. The external device 214 can also include a portable computer-readable storage medium, such as a thumb drive, portable optical or magnetic disk, and memory card, etc. The software and data used to implement embodiments of the present invention, such as the power management program 300, can be stored on such a portable computer-readable storage medium. In these embodiments, the relevant software may (or may not) be loaded in whole or in part into the persistent storage 210 via the I / O interface set 206. The I / O interface set 206 also connects in data communication with the display 212.

[0028] The display 212 provides a mechanism for displaying data to the user and can be, for example, a computer monitor or a smart phone display screen.

[0029] The programs described herein are identified based on the applications implementing the programs in particular embodiments of the invention. However, it should be understood that in this specification, any specific program nomenclature is used for convenience only, and thus the present invention should not be limited to use only in any specific application identified or suggested or a combination thereof by such nomenclature.

[0030] The description of various embodiments of the present invention is presented for illustrative purposes and is not intended to be comprehensive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein is chosen to best explain the principles of the embodiments, the practical application, or the technical improvements found in the marketplace, or to enable those skilled in the art to understand the embodiments disclosed herein. II. Exemplary Embodiments

[0031] FIG. 2 shows a flowchart 250 illustrating a method according to the present invention. FIG. 3 shows a power management program 300 that executes at least some of the method operations of flowchart 250. Next, this method and related software are described in the following paragraphs with extensive reference to FIG. 2 (for method operation blocks) and FIG. 3 (for software blocks).

[0032] When the token pool 420 of the power management program 300 proceeds through a plurality of frequency regions of the integrated circuit chip, processing begins at operation S252. In some embodiments, the integrated circuit chip is a central processing unit (CPU), and each frequency region, e.g., the first frequency region 306 and the second frequency region 312, is a core of the CPU. The frequency regions are configured in a ring topology. The token pool 420 proceeds around the ring indefinitely and communicates with each frequency region. In each communication between the token pool 420 and a frequency region, various events can occur, including, but not limited to, (i) the token pool 420 receiving a token from the frequency region, (ii) the token pool 420 giving a token to the frequency region, and (iii) the token pool 420 receiving a "lack" flag from the frequency region, which indicates that the frequency region needs more tokens. In some communications, neither a token nor a "lack" flag is passed between the token pool 420 and the frequency region.

[0033] When the token pool 420 progresses around the ring, tokens tend to move away from frequency regions with excess tokens and towards frequency regions that require more tokens. A token represents an increment in the power or operating frequency that the frequency region owning the token can use. The total number of tokens assigned to the CPU represents the total power that the CPU can consume, and thus maintains the CPU within its power budget so as not to exceed the CPU's physical cooling capacity (and thus temperature limit).

[0034] Since the CPU is constrained by the total number of tokens, the power management program 300 distributes tokens among the frequency regions based on the relative workloads of the frequency regions. Thus, high-load frequency regions tend to own more tokens and can therefore operate at a higher frequency (consume more power) to execute the workload more quickly. The opposite is true for low-load frequency regions.

[0035] The process proceeds to operation S255, and the token pool 420 of the power management program 300 communicates with the first control module 308 of the first frequency region 306 of the power management program 300. In connection with the communication, the token pool 420 receives a starvation flag 318 from the first control module 308. The starvation flag 318 is sometimes referred to herein as a "starvation token", "power request", "power allowance request", etc. In some embodiments, the starvation flag 318 includes information indicating the magnitude of the power allowance requested by the first control module 308. The starvation flag 318 indicates that the first control module 308 requests permission to consume more power so as to process the current workload (or its backlog) more quickly due to the current workload. In some embodiments, a control module that does not execute a high workload spontaneously donates excess tokens to the token pool even in the absence of a starvation flag. The starvation flag is a new feature added to a conventional system that operates under the principle of "altruistic" (fair distribution based on workload requirements). The starvation flag can indicate that the associated control unit urgently needs additional tokens to start executing an operation.

[0036] The token pool 420 communicates alternately with the first control module 308 and the second control module 314. Some embodiments have multiple frequency regions and multiple corresponding control units respectively. Regardless of how many frequency regions (and corresponding control units) exist in an embodiment, the frequency regions are configured in a ring topology. The token pool 420 communicates with many control units in order around the ring topology and communicates with the control units infinitely.

[0037] The embodiments shown in FIGS. 2 and 3 have two frequency regions (a first frequency region 306 and a second frequency region 312) and two corresponding power control modules (a first control module 308 and a second control module 314). The token pool 420 communicates with the first control module 308 and the second control module 314, repeating this sequence infinitely, and shifts the relative power usage between the two control units according to the current relative workload being processed by the corresponding frequency regions (306 and 312). This approach keeps the total power usage of the two frequency regions within a predetermined limit, such as (i) so that the cooling load does not exceed the capacity of the cooling system, (ii) so that the device junction temperature does not exceed the design limit, (iii) so that the device temperature is not exceeded, and / or (iv) so that the reliability requirements are not negatively affected.

[0038] The process proceeds to operation S260, where the token pool 420 communicates with the second control module 314 of the second frequency region 312 of the power management program 300. In connection with the communication, the second control module 314 has a surplus power allowance available and, when detecting the deficit flag 318 present in the token pool 420, relinquishes the surplus power allowance 320 to the token pool 420. In some embodiments, the surplus power allowance 320 has the magnitude of the power allowance up to the magnitude required by the deficit flag 318.

[0039] By abandoning the surplus power allowance 320, the second frequency module 316 reduces the operating frequency of the second frequency region 312. The amount of reduction in the operating frequency is based on the surplus power allowance 320. The reduction in the operating frequency means that the second frequency region 312 operates at a slower speed, consumes less power, while executing the assigned workload more slowly. Since the current workload assigned to the second frequency region is such that the frequency region 312 is in an idle state for some time, this is a desirable result of abandoning the surplus power allowance 320. Thus, the frequency region 312 can still process the workload in a timely manner with less (or no) idle time.

[0040] The process proceeds to operation S265, and the token pool 420 communicates with the first control module 308 again. In relation to the communication, the token pool 420 transmits the surplus power allowance 320 to the first control module 308.

[0041] The process proceeds to operation S270, and the first frequency module 310 of the first frequency region 306 of the power management program 300 increases the operating frequency of the first frequency region 306. The amount of increase in the operating frequency is based on the surplus power allowance 320. The increase in the operating frequency means that the first frequency region 306 operates at a faster speed, consumes more power, and executes the assigned workload more quickly.

[0042] In some embodiments, the first frequency region 306 and the second frequency region 312 are physically integrated on a common device such as, without limitation, a central processing unit (CPU) or other integrated circuit chip (not shown). The increase in power used by the first frequency region 306 is offset by the decrease in power used by the second frequency region 312. Thus, the total power used by both the frequency regions and the cooling load of the common device remains substantially unchanged without significant change. III. Further Comments or Embodiments or Both

[0043] An electronic module can include a single integrated circuit chip, such as (but not limited to) a central processing unit (CPU), a set of integrated circuit chips attached in a common package, a circuit card such as a computer motherboard with many components, a circuit in a smartphone, a single board computer, a memory card, a storage controller, and any electronic device having an identifiable set of electronic devices that are operatively coupled or operate together or both. For simplicity herein, the terms "CPU" and "chip" are synonymous and should be understood to include at least the devices described above.

[0044] Some embodiments of the present invention include one or more of the following features, characteristics, or advantages, or combinations thereof: (i) the total number of power tokens assigned to a chip is at least partially based on the power budget for the chip; (ii) shifting the power requirements for each frequency region while maintaining the chip within the power budget limit by passing tokens (and corresponding power usage allowances) between frequency regions on the chip; (iii) a distributed frequency control algorithm ensures that the power budget limit of the system is adhered to; (iv) minimizing the suppression of on-chip control (OCC) due to power budget constraints; (v) maintaining an operating load optimization frequency (WOF) range over a longer period; and / or (vi) maximizing the operating load throughput across frequency regions.

[0045] In some embodiments of the present invention, the power management system comprises a token pool that executes a token-passing strategy in a round-robin fashion using a plurality of distributed control units to efficiently shift power allowances (power tokens) between frequency regions corresponding to each control unit. The power management system initializes (i) a fixed number of power tokens in the token pool and (ii) a "shortage flag" associated with (and thus "owned" by) each control unit that can request a fair policy in the case of a longer shortage. This system minimizes the metadata required for configuration and control and can thus scale to systems of any size without increasing complexity.

[0046] In some embodiments, each control unit (CU) holds only the number of tokens that can be utilized by the CU. This number of "useful" tokens is determined based on CU utilization and instructions per second (IPS) values. The combination of utilization and IPS values determines whether it is useful to give tokens to the CU. If the CU has excess tokens (tokens not being used by the CU), it can donate the excess tokens to the token pool, ensuring that the CU does not hold unnecessary tokens. If the CU is in a shortage state for a length of time above a threshold, the CU can set a shortage flag, after which the CU does not hold more than an average number of tokens. The shortage flag is removed when the "starving" CU obtains sufficient tokens (this is guaranteed since the control unit switches to operating under a "fair" policy).

[0047] In some embodiments, the frequency domain can set the maximum operating frequency (and thus power consumption) based on the number of tokens held by the corresponding control unit. The frequency domain does not always have to operate at the maximum operating frequency, but can operate at (i) the maximum frequency, (ii) a low frequency, or (iii) a workload-optimized frequency (Turbo WOF) that can exceed the "maximum frequency". The control unit that delivers power tokens to the token pool reduces the maximum operating frequency of the corresponding frequency domain based on the number of tokens delivered.

[0048] In some embodiments, power tokens represent an increment in the allowable amount of power usage by any power-consuming device, whether the operations are electronic, mechanical, electromechanical, chemical, hydraulic, etc., alone or in any combination. The power usage can be controlled by modulating electrical characteristics, chemical characteristics, or physical characteristics, or combinations thereof, such as operating speed, torque, force, frequency, vibration amplitude, intensity (such as electromagnetic or acoustic radiation), impedance, flow rate, temperature, duty cycle, pressure, etc. Embodiments disclosed herein with respect to the frequency domain of an electronic device should not be construed to exclude any other embodiments such as those described in this paragraph above.

[0049] Some embodiments of the present invention identify parameters (such as instructions per second (IPS) and task utilization rate) that are necessary and sufficient to determine the number of power tokens required to meet the energy requirements of a given frequency domain. Power tokens represent the allowance for using an increment of power.

[0050] The control unit can assign the right to use power up to the amount represented by the number of power tokens held by the control unit to the associated frequency region. Some embodiments assign a default (base) amount of power to the frequency region. The power tokens held by the control unit allow the frequency region to increase the power usage by increments above the base up to the amount represented by the number of power tokens held by the control unit. Some embodiments assign a default number of power tokens to the control unit and a default amount of power to the frequency region. If the frequency region uses less than the default amount of power (which means the frequency region has a power surplus), the control unit can transfer some or all of the power tokens held by the control unit to the token pool.

[0051] In some embodiments, a given frequency region is in a power deficit state when the power requirement of the frequency region exceeds the power allowance. In this case, the workload assigned to the given frequency region cannot be processed in a timely manner. Accordingly, when the token pool communicates with a given control unit (associated with the given frequency region), for example, if the control unit faces a frequency shortage over a certain time threshold, the control unit passes a shortage flag to the token pool. Subsequently, when the token pool communicates with other control units as it moves around the ring, the other control units (i) detect the shortage flag, (ii) limit themselves to the upper limit of the power tokens they can consume, and (iii) discard excess (surplus) power tokens to the token pool. When the token pool comes back to the given control unit again, the control unit picks up the surplus power tokens and removes the requested tokens from the token pool (assuming that sufficient power tokens have been obtained to satisfy the power deficit). The control unit then increases the power usage allowance for the given frequency region. Accordingly, the frequency region operates at an increased frequency (and thus consumes more power) to process the assigned workload more quickly. Thus, the power deficit is partially or completely eliminated within the time it takes for the token pool to make one pass around the ring.

[0052] Figure 4A is a schematic diagram of a ring topology according to some embodiments of the present invention, including control unit 401, control unit 402, control unit 403, and token pool 420. The token pool 420 circulates repeatedly (iteratively) through the ring topology and communicates with all control units during each cycle. Each control unit is associated with a frequency region (not shown), manages, or regulates, or a combination thereof, the allowable power usage for each frequency region. Each control unit is associated with a corresponding deficit flag and a token count (the number of tokens held by the control unit). The ring topology can comprise any number of control units and their respective corresponding frequency regions.

[0053] In some embodiments, the order in which the token pool 420 communicates with the control units is dynamic, effectively changing the ordering of the control units within the ring topology. In at least one embodiment, the change in the ordering of the control units is triggered by a higher need for a particular control unit for more tokens. In an exemplary scenario, the token pool 420 receives a signal indicating that a higher-priority control unit needs more power tokens. In response, the token pool 420 services the higher-priority mission as follows: (i) exits the normal ring sequence, (ii) picks up surplus tokens from one or more donor control units, (iii) delivers the surplus tokens to the higher-priority control unit, and / or (iv) resumes normal processing at the location in the normal ring sequence from which it exited to service the higher-priority mission.

[0054] In another exemplary embodiment, consider a control unit that processes high-priority tasks and requires more tokens. When a donor exists, when the token pool reaches the donor, the donor is guaranteed to donate all excess tokens to the token pool. When the token pool comes back to the control unit with high-priority tasks, the token pool transmits excess tokens to the control unit. In some embodiments, high-priority tasks are identified based on (i) instructions per second (IPS) processed by the frequency domain and (ii) the usage rate of the frequency domain. The control unit calculates the required number of tokens locally based on the IPS and usage rate parameters.

[0055] In some embodiments, some control units have a given relative weighting (based on importance). The token pool communicates with the control units at a certain frequency based on the weighting. For example, considering a ring topology with five control units (C1, C2, C3, C4, and C5), C1 is given a weighting of "2", and C2 to C5 are each given a weighting of "1". The token pool communicates with the control units in a sequence such as C1, C2, C3, C1, C4, C5, C1, C2... and communicates with C1 twice as frequently as each other.

[0056] In some embodiments, the control unit manages two or more frequency domains with respect to power distribution and power allocation among the frequency domains. In some embodiments, a single control unit manages multiple frequency domains on an integrated circuit chip. In some embodiments, a single control unit manages multiple frequency domains distributed among one or more integrated circuit chips.

[0057] In some embodiments, two or more control units manage a single frequency region. For example, a redundancy scheme sets up three control units that are responsible for the frequency region. The control units operate on a voting system that requires agreement between at least two control units in order to take an action (e.g., donate a token to the token pool or accept a token from the token pool). In this way, the failure of a single control unit does not affect the operation of the power management system with respect to the frequency region.

[0058] In some embodiments of the present invention, with each communication with a control unit, the token pool 420 can perform one or more of the following actions (but not limited to), namely, (i) passing a token to the control unit, (ii) receiving a token from the control unit, (iii) receiving a shortage flag from the control unit, (iv) returning a power request token to the control unit, (v) exchanging status information with the control unit (in one direction or both directions), and / or (vi) taking no action.

[0059] FIG. 4B is a schematic diagram showing a control unit that accepts a power token from a token pool according to some embodiments of the present invention. Specifically, the token pool 420 initially has 100 power tokens, and the control unit 401 has none. The token pool 420 transmits 10 out of the 100 power tokens to the control unit 401. After the transmission, the token pool 420 has 90 tokens, and the control unit 401 has 10 tokens. Therefore, the control unit 401 can increase its power consumption by an increment that does not exceed the power represented by 10 tokens for the corresponding frequency region (not shown).

[0060] Figure 4C is a schematic diagram showing a control unit that donates power tokens to a token pool according to some embodiments of the present invention. Specifically, first, the token pool 420 has 90 power tokens, and the control unit 402 has 50. The control unit 402 donates 20 tokens to the token pool 420 (which may or may not represent surplus power tokens). After transmission, the token pool 420 has 110 tokens (90 + 20 = 110), and the control unit 402 has 30 tokens (50 - 20 = 30). Thus, the control unit 402 reduces the power usage limit in the corresponding frequency region (not shown) by the increment represented by the 20 donated tokens. Note that when a CU donates tokens to the token pool, the tokens can be surplus tokens (either not needed or not available or both by the CU), or tokens exceeding the average number of tokens donated due to the presence of a deficit flag.

[0061] Figure 4D is a schematic diagram showing the placement of power request tokens by a control unit to a token pool according to some embodiments of the present invention. Specifically, the control unit 403 is power - deficient and places a deficit flag 405 requesting 50 power tokens to the token pool 420. The token pool 420 continues to cycle through the ring - topology and now communicates one - by - one with other control units.

[0062] In some embodiments, token pool 420 communicates with the control unit based on control unit activity. That is, token pool 420 assumes an idle state until the control unit issues an interrupt signal. In response, token pool 420 communicates with the signaling control unit. When token pool 420 determines why the control unit signaled (e.g., to donate excess power tokens to token pool 420), token pool 420 takes appropriate action (e.g., receives the excess power tokens and forwards the donated power tokens to the "starving" control unit). When no further action is required, token pool 420 re-enters the idle state.

[0063] In some embodiments, the deficit flag identifies a number of tokens required by a related control unit called the "remaining to be filled" (RTBF) number. The token pool accumulates up to the RTBF number of power tokens, holds them for delivery to the requesting control unit, and delivers the power tokens to the requesting control unit when the deficit flag is satisfied. The token pool then returns the deficit flag to the requesting control unit (or otherwise clears the deficit flag).

[0064] In some embodiments, the token pool is unable to accumulate the requested number of power tokens on the first trip around the ring. The token pool continues to circle the ring, accumulating tokens, holding them so they become available to other control units, and transmitting the accumulated power tokens to the requesting control unit. When the accumulated power tokens are transmitted, the token pool reduces the RTBF number by the number of power tokens transmitted to the requesting control unit. When the RTBF number reaches zero, the request is fulfilled and the token pool transmits to return the power request tokens to the requesting control unit.

[0065] In some embodiments, the token pool receives a plurality of power request tokens from a plurality of corresponding requesting control units. When the token pool collects surplus tokens, the token pool can use any suitable method to hold the surplus tokens collected among the plurality of requesting control units until all requests are executed. In some embodiments, the surplus tokens are preferentially allocated to higher-priority control units. In some embodiments, the surplus tokens are allocated in proportion to the number requested by the plurality of control units (and / or based on the respective RTBF numbers).

[0066] FIG. 4E is a schematic diagram showing the abandonment of power tokens in response to the lack flag 405 according to some embodiments of the present invention. As described above with reference to FIG. 4D, the control unit 403 placed the lack flag 405 on the token pool 420. Subsequently, the token pool 420 moves around the ring topology and then communicates with the control unit 404. The control unit 404 has 50 surplus power tokens and transmits 50 surplus tokens to the token pool 420 in response to detecting the lack flag 405 present in the token pool 420. The token pool 420 holds 50 tokens for delivery to the control unit 403 in the practice of the lack flag 405.

[0067] In some embodiments, if the token pool cannot acquire all the tokens requested by the control unit in a power deficit state (a "starving" control unit) within a given time interval, the control unit sets a deficit flag. In response, the power management system initiates a token leveling process based on "fair token distribution" to alleviate the power deficit. In the mitigation process, the power request tokens placed in the token pool by the starving control unit inform the other control units to relinquish some amount of tokens (whether surplus or not) in order to effect a "fair token distribution" across all control units. When the starving control unit places a request token in the token pool, the other control units (the contributing control units) relinquish surplus tokens to the token pool. If some (or all) of the other control units do not have surplus tokens, the other control units still transfer some tokens. The token pool transfers the tokens relinquished by the contributing control units to the starving control unit, thereby effectively shifting some of the power usage from the contributing control units to the starving control unit. This flattens the power usage to some extent among all control units while remaining within the total power allotted to the control units in a lump sum.

[0068] FIG. 4F is a schematic diagram showing the execution of the deficit flag 405 when the token pool 420 returns to the control unit 403. As described above with reference to FIG. 4E, the control unit 404 relinquished 50 tokens in response to the deficit flag 405. When the token pool 420 returns to the control unit 403, the token pool 420 transfers the deficit flag 405 and the 50 surplus tokens received from the contribution by the control unit 404 to the control unit 403.

[0069] The control unit 403 grants the right to increase the operating frequency by an amount based on 50 additional power tokens currently owned by the control unit 403 to the associated frequency region. Subsequently, the frequency region processes the workload more quickly at the expense of consuming more power, but the total power used by all frequency domain participants in the ring topology remains within the established power budget overall.

[0070] FIG. 5 is a schematic diagram showing the operation of a system according to some embodiments of the present invention. The token-passing model can be applied to various layers of a computing stack such as a hardware stack or a software stack. In a hardware stack, the control unit can be, for example, a discrete module or a physical circuit on an integrated chip. At the hardware level, in some embodiments, each on-chip control (OCC) represents the control unit and the shared memory represents the token-passing mechanism.

[0071] In contrast, a control unit implemented within the user space layer of a software stack equips threads to an open multi-processing application programming interface (MP-API) that passes tokens using a shared memory-based approach. Further regarding the user space, in some embodiments, as shown in FIG. 5, MP-API threads (e.g., MP-API thread 501, MP-API thread 502, and MP-API thread 503... MP-API thread N) are treated as control units and use shared memory to pass the token pool to adjacent threads. Also, in a software implementation, both user space applications and kernel space applications can inherit the token-passing model.

[0072] Some embodiments use an MP - API process as a control unit and pass tokens between systems that use a Message Passing Interface (MPI).

[0073] In some embodiments, the scheduling policy within the kernel layer of the operating system represents a control unit that uses shared memory to pass tokens.

[0074] In some embodiments, the MP - API process includes a control unit, and the token pool is communicated by using the Message Passing Interface.

[0075] Some embodiments of the present invention include devices other than the control unit of the integrated circuit chip. For example, the techniques described herein may be applied to servers in a rack of servers, and the tokens distributed among the servers in the rack regulate the power usage allowed for each server. A high - load server can have more tokens (and thus dissipate more power but is allowed to) than a server with a relatively low load in the rack. In this way, the servers in the rack, as a whole, do not exceed the total power budget limit for the rack, and moreover, maximize the workload executed by the servers therein.

[0076] The same techniques can be applied throughout the data center in a nested fashion, such that (i) the first - level tokens regulate power distribution among portions of the integrated circuit chips of the servers in the racks within the data center, (ii) the second - level tokens regulate power distribution among components of the servers (e.g., integrated circuit chips, circuit modules, circuit cards and circuit boards, storage devices, power supplies, network adapters, memory, storage, etc.), (iii) the third - level tokens regulate power distribution among multiple servers in the rack, or (iv) the fourth - level tokens regulate power distribution among groups of racks, or a combination thereof.

[0077] Some embodiments of the present invention use token passing in a ring topology to regulate power distribution, for example, among, between, or in combination with other types of power-consuming devices such as transportation systems, automobiles, communication systems, networking systems, manufacturing systems, power generation systems, ships, aircraft, spacecraft, etc.

[0078] FIG. 6A is a pseudo-code list showing a method for initializing a token pool according to at least one embodiment of the present invention.

[0079] FIG. 6B is a pseudo-code list showing a method for calculating at 1 million instructions per second (MIPS) according to at least one embodiment of the present invention.

[0080] FIG. 6C is a pseudo-code list showing a method for requesting additional tokens according to at least one embodiment of the present invention.

[0081] FIG. 6D is a pseudo-code list showing a method for a daemon thread corresponding to a frequency domain (FD) according to at least one embodiment of the present invention.

[0082] Some embodiments of the present invention include: (i) some central processing units (CPUs) having a clearly defined power budget above which some central processing units (CPUs) become inoperable; (ii) an on-chip controller (OCC) that suppresses the CPU frequency to prevent the CPU from exceeding the limits imposed by the power budget; (iii) some CPU frequency governors that do not consider the power budget but instead vary the core operating frequency according to the core utilization without feedback as to whether the CPU benefits from the changed frequency; (iv) a distributed approach that can result in an indeterminate power deficit across the frequency domain; (v) not taking into account the problem of resource shortage; and / or (vi) some resources can be over-utilized while other resources are not fully utilized or are idle, recognizing one or more of the facts, potential problems, or potential improvement areas, or combinations thereof, related to the current state of the art as described above.

[0083] At least one embodiment of the present invention includes one or more of the following features, characteristics, or advantages, or combinations thereof: (i) implementing an efficient scheme for setting the operating frequency of each of various CPU cores so as not to exceed the total system power budget; (ii) avoiding a centralized bottleneck by selectively distributing power among the cores; (iii) redirecting power / energy to the parts of the system that most need it without starving other parts of the system; (iv) allowing a subset of the CPU cores to operate above the rated frequency while keeping the system power budget under control; (v) using a distributed token passing mechanism across frequency regions to determine the achievable frequencies according to the operating conditions and their changes; (vi) allocating some tokens for a circuit chip (e.g., a CPU chip) based on the power budget for the chip; (vii) shifting tokens (and thus the power usage allowance) among the frequency regions of the chip while maintaining the total power usage of the chip within the limits of the allocation; (viii) implementing a distributed frequency control algorithm to ensure compliance with the system's power budget limit; (ix) minimizing the on-chip controller (OCC) suppressed by the power budget constraint; (x) maintaining the workload optimization frequency (WOF) range over a longer period; (xi) avoiding indefinite power starvation across frequency regions, which is a known problem using a distributed approach; and / or (xii) requiring minimal metadata for implementation and thus being scalable to systems of any size without significantly increasing the complexity of the messages.

[0084] Some tokens assigned to the chip determine at least in part the granularity (in terms of power increments) for power distribution among the components of the chip. For example, consider the coarse granularity for a chip with a 20-watt power budget to which 20 tokens are assigned. The power distribution among the various frequency regions of the chip can be adjusted in increments of 1.0 watt (20 watts / 20 tokens = 1.0 watt / token). In a more fine-grained embodiment, 100 tokens are assigned to the chip. Then, the power distribution can be adjusted in increments of 0.2 watt (20 watts / 100 tokens = 0.2 watt / token).

[0085] In some embodiments, some tokens assigned to the chip determine at least in part the granularity (in terms of operating frequency increments) for adjusting the power distribution among the components of the chip. For example, consider the coarse granularity for a chip with a nominal operating frequency of 3.0 gigahertz (3.0 GHz, meaning 3,000,000,000 cycles per second) to which 1,000 tokens are assigned. The operating frequencies of the various frequency regions of the chip can be adjusted in increments of 3.0 megahertz (3.0 MHz, meaning 3,000,000 cycles per second) (3.0 GHz / 1000 tokens = 3.0 MHz / token). In a more fine-grained embodiment, 300 million (300,000,000) tokens are assigned to the chip. Then, the operating frequencies of the various frequency regions can be adjusted in increments of 10 Hz (3 GHz / 300,000,000 tokens = 10 Hz / token).

[0086] In some embodiments of the present invention, the ring topology arranges (i) a distributed control unit that efficiently performs distributed power shifting and / or (ii) defines a token-passing strategy in the form of a rotating token pool that “moves” around the ring. The system (comprising the control unit and the token pool) is initialized with a fixed number of tokens in the token pool. Each control unit has a special token (deficiency flag) to indicate power deficiency.

[0087] Some embodiments are necessary and sufficient to identify parameters (e.g., instructions per second (IPS), task utilization rate, duty cycle, etc.) necessary to determine how many token control units from the pool are required to meet the energy requirements.

[0088] In some embodiments of the present invention, the mechanism by which a control unit acquires tokens without unlimited lack of other power control units is the following procedure, namely, (i) when one (or more) of the energy requirements of the control unit is not met based on the number of available tokens, the control unit is in a "power shortage" state, and correspondingly, the control unit drops a special "power request token" into the token pool; (ii) when the token pool advances around the ring and communicates with other control units, the other control units detect the power request token and respond by discarding excess tokens that they can consume (based on the current power usage) back into the pool; (iii) finally, the token pool returns to the control unit in the "power shortage" state and transmits the excess tokens to the control unit, thus curing or at least alleviating the "power shortage" within a single trip around the ring, and / or (iv) a control unit that is no longer in a power shortage state removes its special power request token from the token pool.

[0089] In some embodiments of the present invention, the strategy for implementing global energy constraints through local decisions (e.g., at the frequency domain level) (e.g., for the entire chip having multiple frequency domains) includes 1) maintaining a power budget and 2) maintaining at a frequency higher than the rated frequency sometimes called the turbo frequency or the workload optimization frequency as described below.

[0090] 1) Maintaining the power budget: Depending on the power budget for a module (e.g., CPU), in some embodiments, the number of tokens in the pool is increased or decreased so that the total power consumed by the individual control units does not exceed the power budget. The number of tokens held by each control unit limits the power consumed by the control unit. In other words, the maximum allowable power consumed by a control unit is a function of the number of tokens held by the control unit.

[0091] In some embodiments, the function is linear, meaning that the power limit is directly proportional to the number of tokens. In some embodiments, a non - linear functional relationship between tokens and power consumption limits is used to skew the power usage unevenly towards the higher - load control units, enabling them to process the workload much faster, or to skew the power usage unevenly towards the lower - load control units so that more work can be directed there.

[0092] 2) Maintaining the turbo (WOF) frequency: When one or more cores (e.g., of a CPU or any module's frequency domain) are idle and the number of tokens held by the active core exceeds the default minimum number of tokens, the active core can maintain a frequency higher than the rated frequency.

[0093] In some embodiments, the operating algorithm is essentially general - purpose so that it can be implemented across any system infrastructure stack. For example, the algorithm can be easily scaled across a cluster of nodes or can be implemented on several system cores that can act as control units. This algorithm is scalable without (or at least minimizing) the problem of power starvation among the control units.

[0094] The operating algorithms according to some embodiments of the present invention include components such as 1) initialization of a token pool, 2) useful load calculation, 3) tokens to frequency mapping, or 4) a power shortage flag (sometimes referred to herein as a "lack token"), or a combination thereof. These components are described below.

[0095] 1) Initialization of a token pool: Determine the total number of tokens available in the system. The combined value of the tokens is mapped to the power budget constraints of the tokens.

[0096] 2) Useful load calculation: Each control unit calculates the load from the system utilization rate and the 1 million instructions per second (MIPS) value that can distinguish between the "frequency variable" workload and the "frequency invariant" workload. This results in a better token allocation among the control units because the frequency invariant tasks do not require tokens and can instead be given to other tasks to maintain a higher frequency over a longer period.

[0097] 3) Tokens to frequency mapping: Each control unit receives tokens from the rotating token pool or releases tokens to the rotating token pool respectively based on the combination of useful loads that can be processed by the corresponding frequency region. Then, the operating algorithm maps the available tokens to the frequencies achievable by the control units.

[0098] 4) Power shortage flag: Each control unit is considered to be in a power shortage state if it cannot obtain the tokens required to meet its current usage needs with respect to a certain threshold. Accordingly, such a control unit places a "lack flag" in the rotating token pool (see FIG. 4D). The power shortage flag commands each control unit to release extra tokens so that all control units still have a "fair" number of tokens. This equalizes the frequency demands (and thus the power allowance distribution) among competing control units.

[0099] Some embodiments of the present invention can include one or more of the following features, characteristics, or advantages, or a combination thereof: (i) when implemented in a kernel space (e.g., an operating system kernel), enabling the system to maintain a higher turbo frequency for a longer period of time compared to conventional techniques; (ii) resulting in higher throughput and less latency; (iii) being based on the global power constraints of the system; (iv) utilizing the MIPS metric to calculate useful workload capacity; or (v) reducing the energy consumption of the system compared to conventional techniques.

[0100] Some embodiments of the present invention allocate a fixed number of power tokens to an integrated circuit chip. The fixed number of power tokens is based on a power budget allocated to the integrated circuit chip that the circuit chip must not exceed. A rotating token pool distributes the power tokens among multiple frequency regions of the integrated circuit chip based on the corresponding workload requirements (e.g., instructions per second and task usage rate) of each frequency region. When shifting the relative workload requirements between frequency regions, the rotating token pool moves power tokens away from lightly loaded frequency regions and towards more heavily loaded frequency regions. By this method, the sharing of power tokens (which means sharing the power usage allowance) is advantageously skewed towards the more heavily loaded frequency regions. The more heavily loaded frequency regions can operate at the workload optimization frequency (WOF) when they receive the corresponding number of tokens from the token pool. Thus, the workload can be processed more quickly while not exceeding the power budget allocated to the integrated circuit chip.

[0101] All means or steps plus function elements in the following claims, corresponding structures, materials, acts, and equivalents thereof, are intended to include any structure, material, or act that performs the function in combination with other claim elements that are specifically claimed. The description of the present disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or to limit the disclosure to the forms disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the present disclosure. Embodiments are chosen and described in order to best explain the principles of the present disclosure and its practical application, to enable others of ordinary skill in the art to understand the present disclosure for various embodiments with various modifications as are suited to the particular use contemplated. IV. Definitions

[0102] The present invention: The subject matter described by the term "the present invention" should not be taken as an absolute indication that it is encompassed by either the claims as filed or the claims that may ultimately issue after patent examination. The term "the present invention" is used to help the reader gain a general sense that the disclosure herein is potentially new. However, as indicated by the use of the term "the present invention", this understanding is hypothetical and provisional, and as the relevant information evolves and the claims may be amended, it may change during the course of patent examination.

[0103] Embodiment: See the above definition of "the present invention". Similar considerations apply to the term "embodiment".

[0104] And / or: inclusive, or, for example, A, B "and / or" C means that at least one of A or B or C is true and applicable.

[0105] Including / include / include: Unless otherwise expressly stated, it means "including but not necessarily limited to".

[0106] The user includes, but is not necessarily limited to, (i) a single individual, (ii) an artificial intelligence entity having sufficient intelligence to act as a user or consumer, or (iii) a group of related users or consumers, or a combination thereof.

[0107] Data communication: Any type of data communication method currently known or to be developed in the future, including wireless communication, wired communication, and communication paths having both wireless and wired portions. Data communication is not necessarily limited to (i) direct data communication, (ii) indirect data communication, and / or (iii) data communication in which the format, packetization status, medium, encryption status, or protocol, or a combination thereof, remains constant throughout the entire process of data communication.

[0108] Receiving / providing / transmitting / inputting / outputting / reporting: Unless otherwise expressly specified, these terms should not be construed as (i) indicating any particular degree of directness in the relationship between their object and subject, and / or (ii) suggesting the absence of intervening intermediate components, actions, or things, or a combination thereof, between their object and subject.

[0109] Module / sub-module: Any set of hardware, firmware, or software, or a combination thereof, that is operable to perform a certain function, regardless of whether the module is (i) in a single local vicinity, (ii) distributed over a wide area, (iii) in a single vicinity within a larger portion of software code, (iv) located within a single portion of software code, (v) located on a single storage device, memory, or medium, (vi) mechanically connected, (vii) electrically connected, and / or (viii) connected by data communication.

[0110] A computer: any device having significant data processing, or the ability to read machine-readable instructions, or both, including, but not limited to, desktop computers, mainframe computers, laptop computers, field programmable gate array (FPGA)-based devices, smartphones, personal digital assistants (PDAs), body-worn or implantable computers, embedded device-type computers, or application specific integrated circuit (ASIC)-based devices, or combinations thereof.

Claims

1. A method implemented by a computer, comprising: receiving, by a token pool, a power allowance request from a first power-consuming device of a plurality of power-consuming devices of an integrated circuit; in response to receiving the power allowance request, determining that a second power-consuming device among the plurality of power-consuming devices has a power allowance surplus; in response to determining that the second power-consuming device has the power allowance surplus, receiving the power allowance surplus from the second power-consuming device; transmitting the power allowance surplus to the first power-consuming device via the token pool; in response to transmitting the power allowance surplus to the first power-consuming device, (i) increasing a first power usage limit corresponding to the first power-consuming device based on the power allowance surplus, and (ii) decreasing a second power usage limit corresponding to the second power-consuming device based on the power allowance surplus; A method comprising the above steps.

2. further comprising repeatedly communicating with the plurality of power-consuming devices including the first power-consuming device and the second power-consuming device The method according to claim 1, further comprising the above step.

3. Communicating with the plurality of power-consuming devices comprises: communicating with the first power-consuming device; determining a power usage status of the first power-consuming device; in response to determining the power usage status of the first power-consuming device, performing an action selected from the group consisting of (i) transmitting a first power allowance increment from the token pool to the first power-consuming device, and (ii) receiving, by the token pool, a second power allowance increment from the first power-consuming device; The method according to claim 2, further comprising the above steps.

4. Repeatedly communicating with the plurality of power-consuming devices comprises: performing a first round of communication with each power-consuming device of the plurality of power-consuming devices in a first specific order, one power-consuming device at a time; performing a second round of communication with each power-consuming device of the plurality of power-consuming devices in the first specific order, one power-consuming device at a time; The method according to claim 3, further comprising the above steps.

5. Repeatedly communicating with the plurality of power-consuming devices comprises: Performing a third round of communication with a subset of the plurality of power-consuming devices in a second specific order, one power-consuming device at a time The method according to claim 4, further comprising this.

6. The method according to any one of claims 1 to 5, wherein the first power-consuming device is within a first frequency region of the electronic module, and the second power-consuming device is within a second frequency region of the electronic module.

7. Further in response to transmitting the power allowance surplus to the first power-consuming device Operating the first power-consuming device at a workload optimization frequency that can exceed the maximum frequency; and Keeping the power consumption of the integrated circuit within a predetermined maximum power consumption The method according to any one of claims 1 to 6, further comprising this.

8. One or more computer-readable storage media, and program instructions stored together on the one or more computer-readable storage media A computer program product comprising: The program instructions are Receiving a power allowance request from a first power-consuming device of a plurality of power-consuming devices of an integrated circuit by a token pool; Determining that a second power-consuming device among the plurality of power-consuming devices has a power allowance surplus in response to receiving the power allowance request; Receiving the power allowance surplus from the second power-consuming device in response to determining that the second power-consuming device has the power allowance surplus; Transmitting the power allowance surplus to the first power-consuming device via the token pool; In response to transmitting the power allowance surplus to the first power-consuming device, (i) increasing a first power usage limit corresponding to the first power-consuming device based on the power allowance surplus, and (ii) decreasing a second power usage limit corresponding to the second power-consuming device based on the power allowance surplus; A computer program product comprising instructions programmed to perform this.

9. Repeatedly communicating with the plurality of power-consuming devices including the first power-consuming device and the second power-consuming device The computer program product according to claim 8, further comprising program instructions programmed to perform this.

10. communicating with the plurality of power consumption devices comprises communicating with the first power consumption device, determining a power usage status of the first power consumption device, in response to determining the power usage status of the first power consumption device, performing an action selected from the group consisting of: (i) transmitting a first power allowance increment from the token pool to the first power consumption device; (ii) receiving, by the token pool, a second power allowance increment from the first power consumption device, The computer program product according to claim 9, comprising program instructions programmed to perform the above.

11. repeatedly communicating with the plurality of power consumption devices comprises performing a first round of communication with each power consumption device of the plurality of power consumption devices in a first specific order, one power consumption device at a time, performing a second round of communication with each power consumption device of the plurality of power consumption devices in the first specific order, one power consumption device at a time, The computer program product according to claim 10, comprising program instructions programmed to perform the above.

12. repeatedly communicating with the plurality of power consumption devices comprises performing a third round of communication with a subset of the plurality of power consumption devices in a second specific order, one power consumption device at a time The computer program product according to claim 11, further comprising program instructions programmed to perform the above.

13. The computer program product according to any one of claims 8 to 12, wherein the first power consumption device is within a first frequency range of the electronic module and the second power consumption device is within a second frequency range of the electronic module.

14. further in response to transmitting the power allowance surplus to the first power consumption device, operating the first power consumption device at a workload optimization frequency that can exceed the maximum frequency, keeping the power consumption of the integrated circuit within a predetermined maximum power consumption, The computer program product according to any one of claims 8 to 12, further comprising program instructions programmed to perform the above.

15. a set of processors, One or more computer-readable storage media and A computer system comprising: The set of processors is constructed, installed, connected, or programmed, or a combination thereof, to execute program instructions stored in the one or more computer-readable storage media, The program instructions are Receiving, by a token pool, a power allowance request from a first power-consuming device of a plurality of power-consuming devices of an integrated circuit; In response to receiving the power allowance request, determining that a second power-consuming device among the plurality of power-consuming devices has a power allowance surplus; In response to determining that the second power-consuming device has the power allowance surplus, receiving the power allowance surplus from the second power-consuming device; Transmitting the power allowance surplus to the first power-consuming device via the token pool; In response to transmitting the power allowance surplus to the first power-consuming device, (i) increasing a first power usage limit corresponding to the first power-consuming device based on the power allowance surplus, and (ii) decreasing a second power usage limit corresponding to the second power-consuming device based on the power allowance surplus; A computer system comprising instructions programmed to perform the above.

16. Repeatedly communicating with the plurality of power-consuming devices including the first power-consuming device and the second power-consuming device The computer system according to claim 15, further comprising program instructions programmed to perform the above.

17. Communicating with the plurality of power-consuming devices includes Communicating with the first power-consuming device; Determining a power usage status of the first power-consuming device; In response to determining the power usage status of the first power-consuming device, performing an action selected from the group consisting of (i) transmitting a first power allowance increment from the token pool to the first power-consuming device, and (ii) receiving, by the token pool, a second power allowance increment from the first power-consuming device; The computer system according to claim 16, comprising program instructions programmed to perform the above.

18. Said repeatedly communicating with said plurality of power consumption devices comprises: performing a first round of communication with each of said plurality of power consumption devices in a first specific order, one power consumption device at a time; performing a second round of communication with each of said plurality of power consumption devices in said first specific order, one power consumption device at a time; The computer system according to claim 17, comprising program instructions programmed to perform the above.

19. The computer system according to any one of claims 15 to 18, wherein said first power consumption device is within a first frequency range of the electronic module, and said second power consumption device is within a second frequency range of the electronic module.

20. Further in response to transmitting said power allowance surplus to said first power consumption device, operating said first power consumption device at a workload optimization frequency capable of exceeding the maximum frequency; keeping the power consumption of said integrated circuit within a predetermined maximum power consumption; The computer system according to any one of claims 15 to 18, further comprising program instructions programmed to perform the above.

Images