System and method for energy saving control of thermal management

Abstract

Systems and methods for energy saving control for thermal management are provided. A direct to chip (D2C) cooling system maintains the microchip assembly within its optimal junction temperature range by circulating a liquid coolant into contact with the microchip assembly to transfer heat from the microchip assembly. A coolant distribution unit (C DU) controller directly monitors (via a power distribution unit that supplies operating power to the server) the power consumed by each of a set of servers, switches, or other information technology (IT) devices and / or microchip components disposed therein, determines, on a continuous basis, which IT device currently consumes the most power, and direct the CDU of the D2C system to dynamically adjust the flow rate setpoint of the liquid coolant supply based on the current maximum power consumption.

Description

[0001] Cross-reference to related applications

[0002] This application claims the benefit of U.S. Provisional Patent Application Serial No. 63 / 694,348, filed September 13, 2024, entitled “SYSTEM ANDMETHOD FOR ENERGY SAVING CONTROL OF THERMAL MANAGEMENT,” filed pursuant to 35 U.S. SC §119(e). U.S. Provisional Patent Application Serial No. 63 / 694,348 is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure generally relates to the field of thermal management for data centers and other enclosed or internal spaces, and specifically to the management of direct-to-chip (D2C) thermal management systems. Background Technology

[0004] Applications such as generative artificial intelligence (AI) and other similar algorithms and applications require high-performance CPUs and GPUs for smooth, efficient, and fast operation. For this purpose, data center chips can include, for example, over 80 billion transistors per chip, which translates to increased thermal design power (TDP), or the maximum heat generated by the chip, all of which must be dissipated regardless of workload. For example, assuming nearly 100% of silicon's TDP is converted into heat, the increased heat flux per square centimeter of such high-performance, high-density chips exceeds the point where air cooling alone is effective. Single-phase direct-to-chip (D2C) liquid cooling is a viable solution for heat collection and dissipation, where a dedicated coolant distribution unit (CDU) delivers liquid coolant to a cooling plate or heat exchanger near the chip at a set flow rate and temperature. However, the flow rate and supply temperature are typically driven by peak IT load and maximum chip temperature, and the flow rate exceeds the required level as IT load varies over time. Summary of the Invention

[0005] In a first aspect, a computer-aided method for direct-to-chip (D2C) thermal management is disclosed. In an embodiment, the method includes providing a cluster of servers, switches, or other information technology (IT) equipment, each IT equipment comprising one or more microchip components. The method includes providing operating power to each IT equipment in the cluster via a power distribution unit (PDU). The method includes regulating the junction temperature of each microchip component by circulating liquid coolant through the microchip components of the cluster via a coolant distribution unit (CDU) and according to a flow rate setpoint. The method includes monitoring at least two power draws from two or more different IT equipment within the cluster via power sensors within the PDU. The method includes determining the highest or maximum power draw among the measured power draws; for example, a server is determined to consume the most operating power. The method includes adjusting the coolant flow rate setpoint via a CDU controller based on the determined highest power draw to maintain the highest-consuming microchip component or server, as well as any microchip component or server with lower power draws, below its maximum junction temperature.

[0006] In some implementations, the method includes increasing the coolant flow rate based on the determined maximum power consumption.

[0007] In some implementations, the controller (after the current power consumption measurement) measures an additional (subsequent) set of two or more power consumptions corresponding to two or more IT devices within the cluster (which may be the same IT devices as the previous power consumption set, or a different subset of IT devices). The method includes determining the highest subsequent power consumption, such as the highest measured power consumption in a subsequently collected set of power consumptions.

[0008] In some implementations, when the highest subsequent power consumption is associated with a decrease in the flow rate setpoint (e.g., a highest power consumption lower than the last or most recently monitored highest power consumption), the controller will maintain the flow rate setpoint adjusted based on the initial highest power consumption for at least a threshold duration.

[0009] In some implementations, where the highest subsequent power consumption is associated with a further increase in the flow rate setpoint (e.g., still higher than the highest power consumption monitored last or most recently), the controller further increases the flow rate setpoint based on the highest subsequent power consumption.

[0010] In some implementations, the CDU circulates liquid coolant through the IT equipment / microchip components based on a flow rate setpoint and a supply temperature setpoint. For example, the controller may adjust the supply temperature setpoint and flow rate setpoint based on a determined maximum power consumption.

[0011] In another aspect, a system for D2C thermal management is disclosed. In one embodiment, the system includes a coolant distribution unit (CDU) controller connected to a power distribution unit (PDU) that supplies operating power to a cluster of servers, switches, or other IT devices, each IT device including one or more microchip components. The CDU regulates the temperature of each microchip component by circulating liquid coolant through the microchip components according to a predetermined coolant flow rate, for example, keeping the microchip components below their maximum junction temperature. The system includes power sensors (e.g., within the PDU) for monitoring the operating power consumed by each IT device (e.g., or at least two different IT devices in a cluster) and reporting the set of measured power consumptions to the controller. For each set of measured power consumptions, the controller determines the highest power consumption in the set and adjusts the coolant flow rate based on the determined highest power consumption, for example, to ensure that the IT device or microchip component associated with the highest power consumption is maintained at its maximum junction temperature along with all other IT devices or microchip components currently consuming less power.

[0012] In some implementations, the controller increases the flow rate setpoint based on the determined maximum power consumption.

[0013] In some implementations, the set of two or more measured power consumptions is the initial power consumption, and a power sensor measures subsequent power consumption sets, for example, from the same or different sets of IT devices but at a time after the initial power consumption set. The controller then determines the highest power consumption from the subsequent sets of two or more power consumptions.

[0014] In some implementations, the subsequent peak power consumption is associated with a reduction in flow rate (e.g., relative to the initial peak power consumption), but the controller maintains the flow rate setpoint associated with the initial peak power consumption for at least a threshold duration (e.g., before reducing the flow rate).

[0015] In some implementations, the subsequent maximum power consumption is associated with a further increase in flow rate (e.g., relative to the initial maximum power consumption), and the controller immediately increases the flow rate setpoint further based on the subsequent maximum power consumption.

[0016] In some implementations, the coolant flow rate setpoint can be adjusted between a maximum flow rate and a minimum flow rate, with the maximum flow rate based on the peak workload in the IT equipment cluster and the minimum flow rate based on the minimum required flow rate associated with one or more IT equipment within the cluster.

[0017] In some implementations, the CDU regulates the junction temperature of the microchip assembly by adjusting the coolant supply temperature and the coolant flow rate. For example, based on a determined maximum power consumption, the controller can adjust the coolant supply temperature setpoint and the coolant flow rate setpoint.

[0018] In some implementations, the environment includes multiple clusters of IT equipment. For example, the system may include a network switch that connects a CDU controller to multiple PDUs, each PDU supplying a cluster of IT equipment. The CDU controller may monitor the junction temperature within each cluster individually (e.g., each cluster may be associated with a different maximum junction temperature, coolant flow rate setpoint, and / or coolant supply temperature setpoint), or the multiple clusters may be viewed as a single group of IT equipment and / or microchip components.

[0019] In some implementations, the PDU supplies operating power to IT equipment via a set of sockets, and the power sensor is a socket-level sensor located in each socket of the PDU and senses the power consumption of the IT equipment plugged into that socket.

[0020] The present invention is provided solely as an introduction to the subject matter fully described in the detailed description and accompanying drawings. The summary should not be construed as describing essential features, nor should it be used to define the scope of the claims. Furthermore, it should be understood that the foregoing summary and the following detailed description are merely exemplary and explanatory, and are not necessarily intended to limit the claimed subject matter. Attached Figure Description

[0021] Specific embodiments are described with reference to the accompanying drawings. The use of the same reference numerals in different instances in the specification and drawings may indicate similar or identical items. Various embodiments or examples (“multiple examples”) of this disclosure are disclosed in the following detailed description and drawings. The drawings are not necessarily drawn to scale. Generally, unless otherwise provided in the claims, the operations of the disclosed processes can be performed in any order. In the drawings:

[0022] Figure 1 This is a schematic diagram of a direct-to-chip (D2C) cooling system environment for a set of IT devices and / or microchip components, based on an exemplary embodiment of the inventive concept disclosed herein.

[0023] Figure 2 yes Figure 1 A block diagram of the microchip components in the D2C cooling system environment;

[0024] Figure 3 It is shown Figure 1 A block diagram of dynamic thermal management operation in a D2C system environment; and

[0025] Figures 4A to 4CThis is a processing flowchart of a computer-aided method for dynamic D2C thermal management, corresponding to an exemplary embodiment of the inventive concept disclosed herein. Detailed Implementation

[0026] Before explaining one or more embodiments of this disclosure in detail, it should be understood that the embodiments, in their application, are not limited to the details of the construction and arrangement of the components, steps, or methods set forth in the following description or shown in the accompanying drawings. In the following detailed description of embodiments, many specific details may be set forth to provide a more thorough understanding of this disclosure. However, it will be apparent to those skilled in the art who will benefit from this disclosure that the embodiments disclosed herein can be practiced without some of these specific details. In other instances, well-known features may not be described in detail to avoid unnecessarily complicating this disclosure.

[0027] As used herein, the letters following the reference numerals are intended to designate embodiments of features or elements that may be similar to, but not necessarily identical to, previously described elements or features having the same reference numerals (e.g., 1, 1a, 1b). Such shorthand symbols are used for convenience only and should not be construed as limiting the scope of this disclosure in any way unless expressly stated otherwise.

[0028] Furthermore, unless explicitly stated to the contrary, "or" refers to inclusive or not exclusive or. For example, condition A or B is satisfied by any of the following: A is true (or exists) and B is false (or does not exist), A is false (or does not exist) and B is true (or exists), and both A and B are true (or exist).

[0029] Additionally, the terms "a" or "an" may be used to describe elements and components of the embodiments disclosed herein. This is done merely for convenience, and "a" and "an" are intended to include "one" or "at least one," and the singular includes the plural, unless it is obvious otherwise.

[0030] Finally, as used herein, any reference to “one embodiment” or “some embodiments” means that a particular element, feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment disclosed herein. The phrase “some embodiments” appearing in various places in the specification does not necessarily refer to the same embodiment, and an embodiment may include one or more features explicitly described or inherent in this document, or any combination or sub-combination of two or more such features, as well as any other features that may not necessarily be explicitly described or inherent in this disclosure.

[0031] Figure 1 -environment

[0032] In a broader sense, embodiments of the inventive concepts disclosed herein relate to systems and computer-aided methods for energy-efficient single-phase direct-to-chip (D2C) thermal management of servers, switches, and / or other IT equipment. The response time of the D2C coolant distribution unit (CDU) to changes in IT load and the microchip junction temperature (T) are also relevant considerations. j The corresponding changes can be delayed based on a variety of factors. For example, the thermal resistance between the microchip assembly itself (where the junction temperature is measured) and the liquid coolant supplied by the CDU can introduce delays. Furthermore, the physical distance between the CDU and the microchip assembly introduces additional delays. These associated sources of delay complicate the CDU's response to rapid increases in junction temperature, for example, as associated with a sudden and rapid increase in IT load due to generative AI or similar algorithmic processing. The system avoids these delays by directly monitoring the power consumption of each server and identifying the maximum current power consumption. The CDU can then be instructed to adjust the coolant flow rate and / or supply temperature based on the chip or server associated with the highest power consumption, even if the highest power consumption may not represent the maximum possible power consumption or junction temperature. By continuously monitoring the power consumption of each microchip assembly or server, the CDU dynamically and immediately adjusts the coolant flow rate and supply temperature to keep each microchip below its temperature threshold (above which clock frequency and processing performance may be adversely affected). However, by adjusting the flow rate or supply temperature based on the current local maximum (as opposed to an overly conservative estimate of potential delays in expected response time, for example), the CDU can avoid consuming more energy than is required to keep all microchips below their temperature thresholds. For example, given a technology cooling loop (TCL) served by the CDU, the TCL comprises a cluster of servers and / or microchip components, each server potentially including a single or multiple microchip components. In any case, the CDU can use the lowest threshold temperature within the cluster as a local maximum, below which all clusters, servers, and / or microchip components will be maintained.

[0033] Reference Figure 1The diagram illustrates a data center environment 100. Environment 100 may include one or more racks, collections, or clusters 102 of servers, switches, or any other suitable information technology (IT) equipment 102a to 102b; one or more power distribution units 104 (PDUs; also referred to as rack PDUs (rPDUs)) for supplying operating power to the IT equipment 102a to 102b (e.g., via power supply 106); coolant distribution units 108 (CDUs) for circulating liquid coolant (e.g., fluid refrigerant, water, or similar working fluid; single-phase or two-phase / phase change fluid) through the IT equipment via supply conduits 110 and return conduits 112 (or via their microchip components 116, as described below); and controllers 114 (e.g., CDU controllers) for managing the CDUs 108. In some embodiments, environment 100 may include, for example, an external controller 114a connected to and controlling one or more CDUs 108 within the environment.

[0034] In implementations, each IT device 102a to 102b may incorporate one or more microchip components 116. For example, an air-cooling system removes heat generated within the data center environment 100 by circulating cooling air through the IT devices 102a to 102b, thereby transferring heat from the server to the circulating air and thus guiding the heated air out of the environment to remove the transferred heat (and subsequently recirculating the cooling air). However, a direct-to-chip (D2C) liquid cooling system transfers heat more directly and efficiently from the microchip components 116 via a liquid manifold and indirect contact with a cold plate (as described in more detail below). For example, a D2C cooling system implemented by the data center environment 100 guides liquid coolant through a supply pipe 110 and across the microchip components 116 via a CDU 108 at a predetermined flow rate and supply temperature. Furthermore, the heat generated by each microchip component 116 (e.g., assuming each microchip component converts all or nearly all of its supplied operating power into heat) is transferred to a liquid coolant, which circulates from IT devices 102a to 102b and returns to the CDU 108 via return pipe 112, where the transferred heat is removed from the liquid coolant and the recooled coolant is recirculated to the server. The liquid coolant returning to the CDU 108 via return pipe 112 can reach a return temperature, where the increment or difference between the return temperature and the supply temperature can indicate the cooling capacity of the CDU.

[0035] In this implementation, the increase in IT load managed by IT devices 102a to 102b results in an increase in the operating power supplied by PDU 104, as well as the increase in clock frequency and junction temperature T within each microchip assembly 116. JThe corresponding increase. For example, each microchip component 116 may have a specific junction temperature threshold. If the temperature threshold is exceeded, the clock frequency (and corresponding chip performance) within the microchip component 116 is drastically reduced as a built-in protection. In a conventional D2C system, the CDU 108 detects these increases in junction temperature via a supply temperature sensor within the CDU. For example, an increase in the supply temperature of the liquid coolant (T... 2a ≈T SS T SS The supply temperature setpoint (which can be adjusted by CDU 108) can be interpreted by the CDU as an increase in junction temperature. The CDU responds to the increase in junction temperature by increasing the flow rate and / or supply temperature of the liquid coolant supply.

[0036] In the implementation method, under steady-state conditions, the junction temperature T J With supply temperature setpoint T SS The increment ΔT between them can be the resistance R within the microchip component 116. chip and the resistor R associated with the supply temperature sensor 118 within CDU 108 sensor A constant function. As described above (and as follows via...) Figure 2 (Shown in more detail), the resistor R within each microchip assembly 116 chip This introduces latency into the response time of CDU 108. Furthermore, CDU 108 can be remotely located relative to IT devices 102a to 102b, for example, outside or at a distance from the data center environment 100 itself. Therefore, the liquid coolant returning from IT devices 102a to 102b to CDU 108 must be within a distance L between the IT devices 102a to 102b and CDU 108. pipe The upper part passes through the return pipe 112, such that the increment ΔT = Δ(T) J ,T SS )=f(R chip +R sensor +L pipe ), where the distance L associated with return pipe 112 pipe Introducing significant additional latency can adversely affect the CDU's response time to spikes or rapid changes in IT load.

[0037] In an implementation, the controller 114 can eliminate the correlation with R by directly sensing the power consumption supplied by the power supply 106 via the PDU 104 to each IT device 102a to 102b and / or microchip assembly 116. chip and L pipeAssociated latency. For example, when a rapid increase or change in IT load requires a rapid change in the power consumption of one or more IT devices 102a to 102b and / or their microchip components, controller 114 can immediately detect these changes and guide the response of CDU 108 (e.g., the flow rate setpoint of the liquid coolant and, possibly, the supply temperature setpoint T). SS (Adjustments) without waiting for the temperature sensor 118 of the CDU to detect and confirm the changes.

[0038] Figure 2 - Chip component thermal resistance

[0039] Now refer to Figure 2 This shows the IT equipment server ( Figure 1 Microchip component 116 within 102a to 102b).

[0040] In an implementation, each microchip assembly 116 may have a specified junction temperature (T). J ) range, for example, between the minimum and maximum (e.g., the heat dissipation (sink)) junction temperatures (e.g., 80°C ≤ T J Within the temperature range of ≤100℃, the chip performance should not be adversely affected. For example, as mentioned above, ΔT = Δ(T J T SS It can include parameter R chip The parameter R chip The presence of thermal resistance within the microchip assembly 116, located between junction 200 and liquid coolant 202 (e.g., circulating via supply and return pipes 110, 112, respectively), and this thermal resistance affecting the CDU (Cellular Duplex Unit) is considered. Figure 1 The responsiveness of 108) to rapid spikes or shifts in IT load.

[0041] In an embodiment, the microchip assembly 116 may additionally include a silicon package 204, a microchip housing 206, a thermally insulating material 208 (TIM), and a cooling plate 210. For example, the cooling plate 210 can transfer heat (212) directly from the microchip assembly 116 to the liquid coolant 202 (therefore, for example, the liquid coolant 202a can be supplied at a setpoint T higher than the supply temperature T of the liquid coolant 202 leaving the CDU). SS Higher temperatures return to CDU 108, but the cooling plate can provide a certain amount of thermal resistance due to its thickness. Similarly, the microchip housing 206 encapsulating the silicon package 204 can be no more than a fraction of a millimeter thick, but the silicon package itself and TIM 208 can each contribute R... chip Some parts.

[0042] Besides R chip and L pipeIn addition to (e.g., the distance or length of the return pipe 112 between IT devices 102a to 102b and CDU 108), other parameters can further affect the CDU's responsiveness to rapid spikes or shifts in IT loads, such as the CDU temperature sensor 118. Figure 1 The time constant and / or CDU proportional / integral / derivative (PID) control parameters / setpoint T SS Dead zone on.

[0043] Figure 3 -CDU diagram

[0044] Now refer to Figure 3 The illustration shows a data center environment 100. In an implementation, IT devices 102a to 102d (and corresponding microchip components 116a to 116d disposed therein) can be implemented and can be similar to Figure 1 The IT equipment 102a to 102b shown are in operation.

[0045] In an implementation, the controller 114 can bypass, for example, R as described above, by directly monitoring the power consumption of each microchip component 116a to 116d and / or IT device 102a to 102d. chip R sensor and / or L pipe The associated responsiveness delay. Furthermore, based on the real-time power consumed by each IT device 102a to 102b and / or its microchip components 116a to 116d, the controller 114 can continuously guide the CDU 108 to adjust its flow rate setpoint and / or supply temperature setpoint T. SS For example, each IT device 102a to 102d may include one or more microchip components 116a to 116d served by the D2C cooling system, such that the CDU 108 adjusts according to the current flow rate setpoint and the supply temperature setpoint T. SS Liquid coolant 202 (e.g., water or some other single-phase or two-phase fluid) is piped through supply pipe 110. Furthermore, liquid coolant 202 can be guided through a network of IT devices 102a to 102d and into contact with each microchip assembly 116a to 116d, absorbing heat from the microchip assembly and then returning to CDU 108 at an elevated temperature via return pipe 112.

[0046] In this implementation, each IT device 102a to 102d may consume operating power from power supply 106 via rPDU 104, and each server may be connected to rPDU 104 via rPDU socket 300 (e.g., C13, C19, or any other suitable socket type). For example, each socket 300 of rPDU 104 may include a socket-level power sensor 302 capable of sensing the power consumption of each IT device 102a to 102d (e.g., the IT device plugged into the socket).

[0047] In this implementation, the control logic within controller 114 can continuously receive power consumption data (e.g., a periodic set of concurrent power consumed by each active IT device 102a to 102d) from each power sensor 302 of rPDU 104 (and any other rPDU operating within data center environment 100 and / or under controller 114, e.g., via Ethernet or a similar network connection). For example, controller 114 can identify each IT device 102a to 102d associated with the measured power consumption, for example via the server's IP address, and for each concurrent set of two or more power consumptions, determine which IT device 102a to 102d is currently consuming the highest amount of power. For example, IT device 102b could be responsible for power consumption 306b that is significantly higher than the power consumed by IT devices 102a, 102c, or 102d, respectively.

[0048] In an implementation, controller 114 may instruct CDU 108 to adjust the flow rate setpoint and / or supply temperature setpoint T of the liquid coolant 202 to IT devices 102a-102d based on the highest current power consumption 306b associated with IT device 102b. SS For example, if the maximum power consumption of 306b represents a sudden increase in the clock frequency relative to IT device 102b, an increase in coolant flow rate (and / or a decrease in coolant supply temperature) can be expected to quickly offset the increase in junction temperature T relative to the IT device 102b. JAny increase in power consumption will, while maintaining IT devices 102a, 102c, and 102d (e.g., all of which currently consume less power, indicating lower clock frequencies and / or lower junction temperatures) within their respective junction temperature ranges. Furthermore, while CDU 108 can be programmed to overly conservatively maintain the junction temperatures of microchip components 116a to 116d, for example, by adjusting the flow rate and / or supply temperature setpoint based on the maximum possible power consumption 308, controller 114 can also prevent unnecessary excessive energy consumption by dynamically adjusting the flow rate and / or supply temperature setpoint based on the actual sensed maximum power consumption 306b rather than the maximum power consumption 308. This ensures that at any given time, CDU 108 can supply liquid coolant 202 at a sufficient flow rate and / or supply temperature to maintain IT device 102b, which currently consumes the most power (and therefore generates the most heat), within its junction temperature range, while also maintaining IT devices 102a, 102c, and 102d, which currently consume less power (and therefore generate less heat). Therefore, over time, the controller 114 can maintain maximum and minimum flow rate setpoints for the CDU 108 as a linear function of the power consumed by the IT devices 102a to 102d, for example, where the minimum flow rate setpoint corresponds to the minimum flow rate required by a particular IT device, and the maximum flow rate corresponds to the peak workload.

[0049] In this implementation, controller 114 can also manage CDU 108 to avoid unnecessary long-term stress on the inverter-driven pumps within the CDU. For example, a critical increase in junction temperature (e.g., capable of driving microchip components 116a to 116d into a critical mode with a sharp decrease in clock frequency) can be associated with a rapid spike or increase in IT load. Therefore, controller 114 can instruct CDU 108 to increase its flow rate setpoint if necessary. For example, when a first set of concurrent power consumption readings from rPDU 104 indicates an increase in maximum power consumption 306b (corresponding to an increase in clock frequency and junction temperature, and indicating a load spike), causing CDU 108 to increase its flow rate setpoint, and subsequent sets of concurrent power consumption readings indicate a further increase relative to the maximum power consumption, controller 114 can again instruct CDU 108 to increase its flow rate setpoint as quickly as possible.

[0050] In this implementation, regarding the offset rather than the spike in the IT load, which may cause fluctuations in the maximum power consumption 306b, the controller 114 can instruct the CDU 108 to maintain the increased flow rate for at least a minimum threshold duration before reducing the flow rate, even if the sensed maximum power consumption continues to decrease (e.g., indicating a reduced IT load).

[0051] In some implementations, CDU 108 and controller 114 can monitor multiple clusters 102 of IT devices 102a to 102d. Figure 1 For example, environment 100 may include network switch 310 that connects controller 114 to cluster 102 of IT devices 102a to 102s via rPDU 104, and also to additional clusters of IT devices 312a, 312b...312n via rPDU 314 and separate technology control loops (TCLs; e.g., supply pipes 316, return pipes 318). Therefore, controller 114 can also monitor the power consumed by each IT device 312a to 312n via power sensor 302 and adjust coolant flow rate and / or supply temperature as needed. In one implementation, the controller 114 may treat each cluster 102a to 102d, 312a to 312n as a discrete group of IT devices and / or microchip components 116a to 116d with their own temperature and workload parameters; alternatively, the controller 114 may monitor all IT devices 102a to 102d, 312a to 312n as a single group.

[0052] Figures 4A to 4C -method

[0053] Now refer to Figure 4A Method 400 can be implemented by a D2C system including controller 114 and CDU 108, and can include the following steps.

[0054] At step 402, a cluster of servers, switches or other similar IT equipment is provided, wherein each server includes one or more microchip components configured for D2C cooling.

[0055] At step 404, a power distribution unit (PDU; also known as a rack PDU (rPDU)) is provided such that each server in the cluster consumes operating power from a power source via the PDU. In some embodiments, multiple server clusters are provided, for example, each cluster is connected to a network switch via a PDU, which in turn is connected to a CDU controller. In some embodiments, each server consumes operating power from a socket on the PDU, for example, by plugging the server into the socket or otherwise engaging the server with the socket.

[0056] At step 406, the CDU regulates the junction temperature of each server or microchip component (e.g., maintaining the microchip component within a predetermined temperature range) by circulating liquid coolant through the server or microchip component according to a flow rate setpoint and a supply temperature setpoint.

[0057] At step 408, a power sensor located within or connected to the rPDU senses the power level consumed by each (or, for example, at least two) of the servers. In some embodiments, the power sensor is a socket-level sensor, for example, dedicated to a server plugged into the power socket of the PDU.

[0058] At step 410, the controller receives a set of sensed power consumption and determines the server currently associated with the highest power consumption in the server cluster.

[0059] At step 412, the controller instructs the CDU to adjust the flow rate setpoint based on the highest current power consumption (e.g., within the set of currently sensed power consumption). In some embodiments, the CDU additionally or alternatively adjusts the coolant supply temperature setpoint based on the highest current power consumption.

[0060] Also refer to Figure 4B Method 400 may include additional steps 414 and 416. At step 414, for example, after an initial or most recently sensed set of power consumption, the sensor measures a subsequent set of power consumption with respect to the set of servers.

[0061] At step 416, the controller receives a set of subsequent power consumptions and determines the highest subsequent power consumption in the set of subsequent power consumptions (e.g., the server that currently consumes the most power at a later time, which may or may not be the same server associated with the highest initial power consumption in the most recently received set of sensed power consumptions).

[0062] Also refer to Figure 4C Method 400 may include additional steps 418 and 420. At step 418, when the most recently determined highest power consumption causes the CDU to increase the flow rate setpoint, but the current highest power consumption is associated with a decrease in the highest power consumption (indicating a decrease in load and suggesting a decrease in the flow rate setpoint), the controller may instruct the CDU to maintain the current flow rate setpoint for at least a minimum threshold duration before decreasing the flow rate setpoint.

[0063] However, at step 420, when the most recently determined maximum power consumption causes the CDU to increase the flow rate setpoint and the current maximum power consumption is associated with a further increase in the maximum power consumption (indicating a further increase in the flow rate setpoint), the controller may instruct the CDU to immediately increase the flow rate setpoint again.

[0064] Implementations of the inventive concepts disclosed herein are expected to offer numerous advantages. For example, as described above, the controller enables the CDU to respond immediately to rapid spikes or shifts in IT load (and corresponding increases in junction temperature), rather than waiting to detect these shifts via supply temperature. Furthermore, the controller can keep each microchip component below its specific temperature threshold without consuming unnecessary energy when anticipating future temperature shifts.

[0065] Those skilled in the art will recognize that the prior art has evolved to the point where there is little difference between the hardware and software implementations of various aspects of a system; the use of hardware or software is often (but not always, as the choice between hardware and software may become important in certain contexts) a design choice representing a trade-off between cost and efficiency. Those skilled in the art will understand that various carriers (e.g., hardware, software, and / or firmware) exist that can implement the processing and / or system and / or other technologies described herein, and the preferred carrier will vary depending on the context in which the processing and / or system and / or other technologies are deployed. For example, if the implementer determines that speed and accuracy are of paramount importance, the implementer may choose a primarily hardware and / or firmware carrier; alternatively, if flexibility is of paramount importance, the implementer may choose a primarily software implementation; or, again alternatively, the implementer may choose some combination of hardware, software, and / or firmware. Thus, there are several possible carriers that can implement the processing and / or device and / or other technologies described herein, none of which is inherently superior to another, because any carrier to be utilized is a choice dependent on the context in which the carrier will be deployed and the implementer's specific concerns (e.g., speed, flexibility, or predictability), any of which can vary. Those skilled in the art will recognize that the optical aspects of the implementation will typically employ optically oriented hardware, software, and / or firmware.

[0066] The foregoing detailed description has illustrated various implementations of the device and / or processing using block diagrams, flowcharts, and / or examples. Within the scope of such block diagrams, flowcharts, and / or examples containing one or more functions and / or operations, those skilled in the art will understand that each function and / or operation in such block diagrams, flowcharts, or examples can be implemented individually and / or collectively by various hardware, software, firmware, or virtually any combination thereof. In one implementation, several portions of the subject matter described herein can be implemented via an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or other integration format. However, those skilled in the art will recognize that some aspects of the implementations disclosed herein can be implemented, wholly or partially equivalently, in an integrated circuit as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more microprocessors), as firmware, or as virtually any combination thereof, and that designing the circuit system and / or writing the code for the software and / or firmware in accordance with this disclosure will be entirely within the skill of those skilled in the art. Furthermore, those skilled in the art will understand that the mechanisms of the subject matter described herein can be distributed as a program product in various forms, and that the illustrative embodiments of the subject matter described herein are applicable to the signal-bearing medium in which the distribution is actually performed, regardless of the specific type of signal-bearing medium. Examples of signal-bearing media include, but are not limited to, the following: recordable media, such as floppy disks, hard disks, CDs, digital video discs (DVDs), digital magnetic tapes, computer memory, etc.; and transmission media, such as digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

[0067] In a general sense, those skilled in the art will recognize that the various aspects described herein, which can be implemented individually and / or jointly by a wide range of hardware, software, firmware, or any combination thereof, can be considered as comprising various types of "electronic circuit systems." Therefore, as used herein, "electronic circuit system" includes, but is not limited to, electronic circuit systems having at least one discrete electronic circuit, electronic circuit systems having at least one integrated circuit, electronic circuit systems having at least one application-specific integrated circuit, electronic circuit systems forming general-purpose computing devices configured by computer programs (e.g., a general-purpose computer configured by a computer program that at least partially performs the processes and / or devices described herein, or a microprocessor configured by a computer program that at least partially performs the processes and / or devices described herein), electronic circuit systems forming memory devices (e.g., in the form of random access memory), and / or electronic circuit systems forming communication devices (e.g., modems, communication switches, or optoelectronic devices). Those skilled in the art will recognize that the subject matter described herein can be implemented in analog or digital modes, or some combination thereof.

[0068] Those skilled in the art will recognize that describing devices and / or processes in the manner set forth herein is common in the art, and that such described devices and / or processes are subsequently integrated into data processing systems using engineering practice. That is, at least a portion of the devices and / or processes described herein can be integrated into data processing systems through a reasonable amount of experimentation. Those skilled in the art will recognize that a typical data processing system typically includes one or more of the following: a system unit housing, a video display device, memory (such as volatile and non-volatile memory), a processor (such as a microprocessor and a digital signal processor), computing entities (such as an operating system, drivers, a graphical user interface, and applications), one or more interactive devices (such as a touchpad or screen), and / or a control system including feedback loops and control motors (e.g., feedback for sensing position and / or speed; control motors for moving and / or adjusting components and / or quantities). A typical data processing system can be implemented using any suitable commercially available components, such as those commonly found in data computing / communication and / or network computing / communication systems.

[0069] The topics described herein sometimes illustrate different components contained within or connected to different other components. It should be understood that such architectures depicted are merely exemplary, and many other architectures that achieve the same functionality can indeed be implemented. Conceptually, any arrangement of components that achieve the same functionality is effectively “associated” to achieve the desired functionality. Therefore, any two components combined herein to achieve a particular function can be considered “associated” with each other to achieve the desired functionality, regardless of the architecture or intermediate components. Similarly, any two such associated components can also be considered “operably connected” or “operably coupled” to each other to achieve the desired functionality, and any two components that can be suchly associated can also be considered “operably coupled” to each other to achieve the desired functionality. Specific examples of being operationally coupled include, but are not limited to, physically matchable and / or physically interactable components and / or wirelessly interactable components and / or logically interactable and / or logically interactable components.

[0070] While specific aspects of the subject matter described herein have been shown and described, it will be apparent to those skilled in the art that changes and modifications can be made based on the teachings herein without departing from the subject matter and its broader aspects, and therefore the appended claims are intended to cover within their scope all such changes and modifications that fall within the true spirit and scope of the subject matter described herein. Furthermore, it should be understood that the invention is defined by the appended claims.

Claims

1. A computer-aided method for direct-to-chip thermal management, the method comprising: Provide at least one cluster of two or more information technology (IT) devices, each IT device including at least one microchip component; Provides a power distribution unit (PDU) configured to supply operating power to each IT device in the cluster; The junction temperature associated with the at least one microchip component is regulated by circulating liquid coolant to each IT device in the cluster via a coolant distribution unit (CDU) based at least on a flow rate setpoint. Two or more power consumptions are measured via at least one power sensor of the PDU, each power consumption corresponding to an IT device in the cluster; The highest power consumption among the two or more measured power consumptions is determined via the controller of the CDU; as well as The flow rate setpoint is adjusted by the controller based on the maximum power consumption.

2. The computer-aided method according to claim 1, wherein, The adjustments include: The flow rate setpoint is increased based on the highest power consumption.

3. The computer-aided method according to claim 2, wherein, The highest power consumption is the highest initial power consumption, and the two or more power consumptions are two or more initial power consumptions. The method further includes: Measuring two or more subsequent power consumptions via the at least one power sensor and after the two or more initial power consumptions; and The controller determines the highest subsequent power consumption among the two or more subsequent power consumptions.

4. The computer-aided method according to claim 3, wherein, The highest subsequent power consumption is associated with a decrease in the flow rate setpoint, and the method further includes: The flow rate setpoint is maintained at a duration not less than a threshold based on the highest initial power consumption.

5. The computer-aided method according to claim 3, wherein, The highest subsequent power consumption is associated with the increase of the flow rate setpoint, and the method further includes: The flow rate setpoint is further increased based on the highest subsequent power consumption.

6. The computer-aided method according to claim 1, wherein, The adjustment includes: Based on at least one of the flow rate setpoint or supply temperature setpoint, liquid coolant is circulated to the at least one microchip assembly via a coolant distribution unit (CDU); and The adjustments include: The controller adjusts at least one of the flow rate setpoint and the supply temperature setpoint based on the maximum power consumption.

7. A system for direct-to-chip thermal management, the system comprising: A controller associated with and operatively coupled to a power distribution unit (PDU) associated with a cluster of information technology (IT) equipment, wherein the CDU is configured to regulate the temperature of at least one microchip component disposed within each IT device in the cluster of IT equipment by circulating liquid coolant to each IT device in the cluster of IT equipment, at least based on a flow rate setpoint, and wherein the PDU is configured to supply operating power to each IT device in the cluster of IT equipment; and Multiple power sensors are disposed within at least one of the PDUs or the cluster of IT devices, and the multiple power sensors are configured to measure two or more power consumptions, each power consumption corresponding to an IT device in the cluster. The controller is configured to: Receive the measured power consumption of the two or more sources. Determine the highest power consumption among the two or more measured power consumptions, and The flow rate setpoint is adjusted based on the highest power consumption.

8. The system according to claim 7, wherein, The controller is configured to increase the flow rate setpoint based on the highest power consumption.

9. The system according to claim 8, wherein, The two or more power consumptions are initial power consumptions, and the highest power consumption is the highest initial power consumption, and: The at least one power sensor is configured to measure two or more subsequent power consumptions following the two or more initial power consumptions; The controller is configured to determine the highest subsequent power consumption among the two or more subsequent power consumptions.

10. The system according to claim 9, wherein: The highest subsequent power consumption is associated with the decrease in the flow rate setpoint, and The controller is configured to maintain the flow rate setpoint associated with the highest initial power consumption for a duration not less than a threshold value.

11. The system according to claim 9, wherein: The highest subsequent power consumption is associated with a further increase in the flow rate setpoint, and The controller is configured to immediately increase the flow rate setpoint based on the highest subsequent power consumption.

12. The system according to claim 7, wherein, The flow rate setpoint is associated with at least one of the following: The maximum flow rate corresponding to the peak workload of the cluster of the IT equipment, or Minimum flow rate associated with at least one IT device in the cluster.

13. The system according to claim 7, wherein: The CDU is configured to regulate the temperature of the at least one microchip component by circulating liquid coolant to the at least one microchip component based at least on a flow rate setpoint or a supply temperature setpoint. as well as The controller is configured to adjust at least one of the flow rate setpoint and the supply temperature setpoint based on the maximum power consumption.

14. The system according to claim 7, wherein, The PDU is the first PDU, and the cluster is the first cluster, further comprising: At least one network switch, operatively coupled to the CDU; and at least one second PDU associated with a second cluster of IT equipment. The controller is operatively coupled to the at least one second cluster via the at least one second PDU.

15. The system according to claim 7, wherein: The PDU is configured to supply operating power to each IT device in the cluster of IT devices via multiple outlets; as well as The at least one power sensor includes a power sensor disposed in each of the plurality of sockets.

Images