Thermo-mechanical power smoothing for distributed computing servers

Abstract

In large-scale machine learning (ML) and / or artificial intelligence (AI) model training, a large number of GPU servers are responsible for distributed periodic computing workloads. This causes the power consumption of the GPU servers to periodically, repeatedly fluctuate from near zero to full load. The presently disclosed thermo-mechanical power smoothing devices and techniques utilize a distributed network of high-speed fans as thermo-mechanical energy storage devices for consuming underutilized power and storing it in the form of thermal and mechanical energy for future reuse.

Description

Background Technology

[0001] Training large-scale machine learning (ML) and / or artificial intelligence (AI) models can involve distributed computation involving thousands of graphics processing units (GPUs) interconnected via high-bandwidth networks, such as Infinite Bandwidth (IB) networks. For example, to train a large language model, the computational workload is divided across thousands of interconnected GPUs in a GPU cluster. At certain stages of this computation, a collection operation (e.g., Allreduce) collects and combines the information generated by the GPUs. The GPUs remain largely idle until the collection operation is complete, and then the GPUs begin the next computational workload. Summary of the Invention

[0002] The implementation described and claimed herein addresses the problems described below by providing a server comprising a processor array that operates over time with synchronously fluctuating workloads and correspondingly fluctuating net power consumption; a variable-speed cooling fan array for supplying cooling air to the processors and operating as a thermomechanical power smoothing device for the server; and a controller. The controller sets the cooling fan speed to maximum in response to a low-power state of the processor and sets the cooling fan speed to automatic in response to a high-power state of the processor.

[0003] This article also describes and documents other implementation methods. Attached Figure Description

[0004] Figure 1 The illustration shows an example graphics processing unit (GPU) server with an integrated cooling fan that operates as a thermomechanical power smoothing device.

[0005] Figure 2 The illustration shows an example server rack with a set of graphics processing unit (GPU) servers, each GPU server having an integrated set of cooling fans that operate as a thermomechanical power smoothing device.

[0006] Figure 3 The illustration shows an example reactive feedback control scheme for achieving thermomechanical power smoothing using a cooling fan set.

[0007] Figure 4 An example predictive feedback control scheme is illustrated for achieving thermomechanical power smoothing using a cooling fan set.

[0008] Figure 5 This is an example synchronous load power diagram illustrating the operation of a cooling fan that functions as a thermomechanical power smoothing device.

[0009] Figure 6 The illustration shows an example operation using a cooling fan that operates as a thermomechanical power smoothing device. Detailed Implementation

[0010] A Graphics Processing Unit (GPU) server is a server with one or more Graphics Processing Units (GPUs) that provide increased power and speed for running computationally intensive tasks such as video rendering, data analytics, and machine learning. In data centers responsible for training large-scale machine learning (ML) and / or artificial intelligence (AI) models, numerous GPU servers are deployed in clusters and handle distributed computing workloads. Once a computing workload is complete, a collection operation (e.g., Allreduce) gathers data from different GPU servers and combines the data into a global result. This result is then distributed back to the GPU servers, and the next computing workload begins. Therefore, computing workloads occur in stages, with collection operations completing one stage. While the collection operation is running, the GPU servers are largely idle, waiting for the next computing workload to begin.

[0011] Therefore, the computational workload on a GPU server is periodic, and the GPUs cycle together between on and off states. This creates a synchronous workload, causing the power consumption of the GPU server to fluctuate periodically and repeatedly from near zero to full load. This can cause problems with the power grid or other power delivery systems, putting stress on uninterruptible power supply (UPS) batteries and generators, causing voltage oscillations, and potentially propagating the resulting noise back to the power grid.

[0012] "Purely electrical" solutions to power oscillations caused by GPU server clusters involve expensive storage technologies (e.g., batteries and / or capacitors) or wasted energy in "dummy loads" (e.g., resistor arrays and / or heaters). Currently disclosed thermomechanical power smoothing devices and techniques utilize a distributed network of high-speed fans as thermomechanical energy storage devices to dissipate underutilized power and store it as thermal energy (e.g., supercooled air cooling components of a GPU server) and mechanical energy (e.g., fan rotors rotating at above-normal speeds) for future reuse. Currently disclosed thermomechanical power smoothing devices and techniques can be implemented at low cost in existing server designs. For even greater energy storage capacity, fans in existing GPU servers can be retrofitted with weighted rotors, defined herein as rotors made of a base material and / or combined with counterweights that, in total, give the rotor a mass significantly greater than required for fan operation.

[0013] The following thermomechanical power smoothing devices and methods are technically superior to the aforementioned "purely electrical" solutions and other solutions because they require little or no change to the GPU server design. Implementing these devices and methods may not require new hardware and data center infrastructure upgrades. Furthermore, power management software can be updated to utilize the disclosed technologies without any hardware changes. In contrast, resistor banks and UPS-based solutions require power infrastructure upgrades, while local battery-based solutions require changes to the PSU / server chassis. Moreover, assuming the use of low-friction hydrodynamic bearings on the cooling fans, the following thermomechanical power smoothing devices and methods exhibit very low wear compared to chemically based storage devices (e.g., batteries and UPS). Furthermore, the following thermomechanical power smoothing devices utilize high-speed fans for the dual purpose of cooling and energy storage. By replacing existing fans with high-speed fans, no additional points of failure are introduced (compared to additional batteries, load switching devices, etc.).

[0014] Furthermore, the following thermomechanical power smoothing devices and methods can achieve net power savings while being as reliable as or even more reliable than resistance heating solutions.

[0015] Figure 1 An example graphics processing unit (GPU) server 100 is illustrated, featuring an integrated cooling fan (e.g., cooling fan 102) operating as a thermomechanical power smoothing device. Server 100 includes a system board 104 to which various microelectronic components are attached and interconnected via various ports (e.g., a peripheral component interconnect rapid (PCIe) port 106). Processors 108, 110 (e.g., discrete or integrated microelectronic chips and / or discrete but integrated processor cores, including but not limited to a central processing unit (CPU) and a graphics processing unit (GPU)) and at least one memory device (e.g., a dual in-line memory module (DIMM) 114) are integrated components of server 100. Server 100 may also include data storage devices (e.g., solid-state drives (SSDs) 112 and / or flash or hard disk drives) and other input / output (I / O) devices (not shown). Any or all of the above-described components of server 100 may be integrated as a chip of server 100 or as discrete devices connected to server 100.

[0016] I / O devices allow users to input commands and information (e.g., via a keyboard or mouse). These and other input devices can be coupled to server 100 via one or more I / O interfaces, such as serial port interfaces, parallel ports, and / or Universal Serial Bus (USB) interfaces. Multiple memory devices and / or multiple data storage devices may include one or both of volatile memory (e.g., random access memory (RAM)) and non-volatile memory (e.g., flash memory or magnetic storage). An operating system (OS) (such as one variant of Linux or Microsoft Windows® OS) resides in the multiple memory devices and / or storage media and is executed by at least one of processors 108, 110, although other OSs may be used. Other software applications may also be loaded into the multiple memory devices and / or storage media and executed within the OS by at least one of processors 108, 110.

[0017] Server 100 may be a remotely controlled and / or physically controlled device, and a network-connected and / or network-capable device. A network adapter 116 is connected to a networking port 118 (e.g., a quad small form factor pluggable (QSFP) networking port) to provide network connectivity to one or more other server and / or client devices within a data network (such as a wide area network (WAN) or local area network (LAN)). Server 100 may also include a power supply 120 (or be connected to an external power source), which is powered by one or more batteries or other power sources and provides power to server 100. Power supply 120 may also include its own battery or capacitor to store energy for momentary power outages. Power supply 120 may also be connected to an external power source that covers or recharges the internal battery or capacitor.

[0018] In some implementations, the cooling fan includes a regenerative braking mechanism connected to power supply 120 or system board 104. These regenerative braking mechanisms can recover mechanical storage capacity as power for the server to bridge transient power outages. Furthermore, the regenerative braking mechanism can be used to partially power server 100 even in the absence of a power outage. Still further, the regenerative braking mechanism can cause the fan to decelerate faster than it would otherwise occur, resulting in power efficiency gains and safety advantages. Regenerative braking mechanisms are technically advantageous because they provide UPS, thermomechanical power smoothing, and / or power efficiency benefits that would not otherwise be available within server 100.

[0019] Processors 108, 110, and other components within server 100 are conductively cooled by a network of heat pipes (e.g., heat pipe 122) connected to a radiator 124 or heat exchanger, such as a cold plate (e.g., a carbon graphite or metal structure designed to diffuse heat), which is convectively cooled. The heat pipes are attached at one end to processors 108, 110 and extend from processors 108, 110 to radiator 124. In various implementations, the heat pipes may be vapor chambers (or planar heat pipes), thermosiphons, etc. The various types, numbers, and configurations of heat pipes, radiators, and cold plates are collectively referred to herein as heat transfer devices and can vary considerably from those depicted, while maintaining the convective cooling aspect discussed in further detail below.

[0020] In various implementations, when processors 108 and 110 are idle, heat sink 124 can be used to add thermal energy storage capacity to server 100. This thermal energy storage capacity can then be recovered when processors 108 and 110 again handle computational workloads that generate the corresponding thermal energy to be dissipated. Heat sink 124 is technically advantageous because it provides an additional source for storing energy as heat, which can contribute to thermomechanical power smoothing and power efficiency benefits that would not otherwise be available within server 100. Regardless of the presence, type, and arrangement of heat transfer devices within server 100, cooling fans are ultimately used in conjunction with heat transfer devices (if present) to cool server 100 and its internal components.

[0021] Cooling fans draw air through a forward-facing perforated grille 126, which serves as an air inlet. Air passes over a convection-cooled radiator 124 and other internal components of the server 100 designed for convection cooling, as well as other components that discharge heated air at the rear of the server 100. Therefore, the cooling airflow generally moves from the front to the rear of the server 100, and convectively cools the internal components of the server 100 as it moves through the server 100.

[0022] System board 104 includes a baseboard management controller (BMC) 128, which manages the interface between the server 100 hardware and the software running on it. Various sensors built into the server 100 report measured parameters to the BMC 128, such as processor workload, temperature, cooling fan speed, power status, operating system (OS) status, etc. The BMC 128 monitors the sensors and can responsively take thermomechanical power smoothing actions.

[0023] When processors 108 and 110 are used for ML and / or AI model training, they operate under synchronous and fluctuating computational workloads. At certain stages of the computational workload, collection operations (e.g., Allreduce) collect and combine information generated by a connected array of servers (such as server 100). This results in synchronous and fluctuating net power consumption, which oscillates between minimum and maximum power consumption. At minimum power consumption, the processor array is essentially idle, while at maximum power consumption, processors 108 and 110 are fully loaded with computational workloads before and after each collection operation. The near-idleness of the processor array occurs during collection operations that are running between computational workloads.

[0024] The fan can operate at a variable speed according to the requirements of the BMC 128. One action the BMC 128 can take to achieve thermomechanical power smoothing in response to (or predict) the synchronous and fluctuating net power consumption of the processors 108 and 110 is to change the fan's operating speed. This action turns the fan into a thermomechanical power smoothing device to adapt to the synchronous and fluctuating net power consumption of the processors 108 and 110.

[0025] The fans are configured to operate in at least two modes. In automatic or AUTO mode, fan speed fluctuations are allowed to maintain the desired temperature within server 100. More specifically, the BMC 128 monitors one or more temperature sensors (not shown, see example...). Figure 2 Temperature sensors 240 measure the temperature of the airflow, heatsink 124, and / or processors 108 and 110. In automatic operation, the BMC 128 allows fan speed fluctuations to maintain the monitored temperature readings within the desired operating range. Therefore, one or more temperature sensors provide feedback control for fan speed in automatic mode. Automatic operation can be used as the default state for the fan.

[0026] In maximum operating state, the fan speed is set to maximum, allowing the actual fan speed to rise to its maximum level. The BMC 128 can utilize the maximum operating state when it detects (or predicts) a low-power state of processors 108 and 110 (i.e., processors 108 and 110 are essentially idle). This consumes additional power, which can be helpful from a power smoothing perspective due to the low-power state of processors 108 and 110. It also stores mechanical energy in the fan rotors, which can be subsequently consumed at a later time (e.g., when processors 108 and 110 enter a high-power state). Furthermore, it results in a decrease in temperature within server 100 due to increased airflow through server 100. This lower temperature can be used later when processors 108 and 110 again handle computational workloads that generate corresponding heat energy to be dissipated (e.g., when processors 108 and 110 enter a high-power state).

[0027] In LOW or OFF operating mode, the fan speed is set to minimum, which can be zero when the monitored temperature(s) are below the desired operating range. The LOW or OFF operating mode can immediately follow the maximum operating mode when the processors 108 and 110 resume computational workload, but the corresponding thermal energy has not yet increased the monitored temperature(s) to a level requiring convection cooling. In some implementations, the LOW or OFF operating mode is covered by an automatic operating mode, which sets the fan speed to minimum whenever the monitored temperature(s) are below the desired operating range.

[0028] Server 100 is arranged with standard form factors, having height (h), width (w), and depth (d) for inclusion in racks (not shown), such as those found in various data centers. For example, if the rack is a standard 19-inch rack, the width (w) dimension is approximately 19 inches and the depth is approximately 37 inches. The height (h) dimension is typically expressed in rack units (U), which are multiples of 1.75 inches. The height (h) of server 100 can be 1U or more. Other rack standards (e.g., 10-inch racks, European Telecommunications Standards Institute (ETSI) racks, open racks, etc.) are also considered in this document, and the height (h), width (w), and depth (d) can be varied accordingly to accommodate other rack standards. This is on a case-by-case basis.

[0029] Although processors 108 and 110 are explicitly disclosed herein as GPUs, and server 100 is explicitly disclosed herein as a GPU server, other server and processor types operating with periodic and fluctuating workloads and the resulting power consumption can similarly employ cooling fans operating as thermomechanical power smoothing devices. Furthermore, although Figure 1The diagram and description above illustrate integrated cooling fans, but external fans (e.g., rack-mounted fans) can be used to achieve a similar effect across several connected servers (e.g., all servers sharing a rack with rack-mounted fans).

[0030] Figure 2 An example server rack 230 is illustrated, which has a set of graphics processing unit (GPU) servers (e.g., GPU server 200), each server having an integrated set of cooling fans (e.g., cooling fan set 202) operating as a thermomechanical power smoothing device. As an example, server rack 230 includes a rack controller 232, seven GPU servers, and a power supply 220. Other and varying numbers of components are also mounted to server rack 230 because server rack 230 is inherently modular. Furthermore, server rack 230 is envisioned as one of many server racks (e.g., server racks 234, 236, etc.) within a data center.

[0031] Each GPU server includes a system board (not shown, see example...) Figure 1 The system board 104), various microelectronic components (including processors, such as processors 208, 209, 210) are attached to the system board and interconnected via various ports. The GPU server may also each include additional connection components (e.g., heat transfer devices), such as... Figure 1 As shown and as described above, GPU servers may be the same or different in the number and type of processors or other connectivity components they include.

[0032] Power supply 220 is located outside the GPU server but inside the server rack 230, and powers the various components mounted to the server rack 230. Power supply 220 is powered by mains power, one or more batteries, or other external power sources. Power supply 220 may also include its own internal power source, such as a battery or capacitor, to store energy for momentary power interruptions. When power is available, the external power source can recharge the internal battery or capacitor. GPU servers may also include their own power sources (e.g., Figure 1 (Power supply 120), as Figure 2 A supplement or replacement for the rack-mount power supply 220.

[0033] In some implementations, the fan includes a regenerative braking mechanism connected to power supply 220 or the corresponding GPU server. This regenerative braking mechanism can recover mechanical storage capacity as power for the server, bridging transient power outages. Furthermore, the regenerative braking mechanism can be used to partially power server rack 230 even in the absence of power outages. Regenerative braking mechanisms are technically advantageous because they provide UPS, thermomechanical power smoothing, and / or power efficiency benefits that would otherwise be unavailable within server rack 230.

[0034] Cooling fans draw air through a forward-facing perforated grille 226 in server rack 230 (or a perforated grille on each GPU server in the GPU server), which serves as an air intake for the GPU server. Air passes through internal components of the GPU server designed to be convectively cooled (such as GPU processors 208, 209, 210) and is exhausted at the rear of server rack 230, as indicated by the dashed arrow (e.g., dashed arrow 238). Therefore, the cooling airflow generally moves from the front to the rear of server rack 230, convectively cooling the internal components of the GPU server as it moves through server rack 230.

[0035] In various implementations, the cooling fan is constructed from a material heavier than that required for normal fan operation (e.g., a metal alloy instead of plastic), or includes a weighted rotor (e.g., a heavy (metal) rotor or a lightweight rotor (plastic) with embedded counterweights) to increase the mechanical power storage capacity of the server rack 230 and individual GPU servers. The mechanical storage capacity of the cooling fan is largely limited by the rotor weight and fan speed. Since modern server cooling fans typically operate at very high speeds for power and aerodynamic efficiency reasons (e.g., around 38,000 RPM for some 1U designs and around 18,300 RPM for some 2U designs), this is helpful when adding weight to the rotor to provide additional mechanical storage capacity.

[0036] In addition, the angular acceleration (or speed-up) of the cooling fans consumes additional energy, which can be used later when the cooling fans decelerate (or slow down), thus operating as a flywheel energy storage device. This is technically advantageous because it increases mechanical power storage capacity without occupying any additional physical space within server rack 230 and the individual GPU servers. Other implementations could add separate weighted flywheels within the GPU servers, operating similarly to the cooling fans, as a supplement or alternative to the cooling fan operation described herein. In one example implementation, four 80mm fans with metal alloy rotors can buffer 500W of power per second, speeding up from 4,000 RPM to 18,000 RPM. In another example implementation, ten 40mm fans with metal alloy rotors can buffer 500W of power per second, speeding up from 8,000 RPM to 38,000 RPM.

[0037] One example implementation in a 2U application uses 80mm fans with acrylonitrile butadiene styrene (ABS) rotors (33g rotors) to smooth 500W of GPU power. The steady-state expected fan speed at automatic operation is 4300 RPM, with each fan consuming 2W. When the GPU enters a low-power state, to smooth the power drop, the 80mm fans can ramp up to 18300 RPM at maximum operation, consuming up to 58W per fan. Therefore, nine fans are used to smooth 500W of GPU power.

[0038] Another example implementation uses 80mm fans with steel rotors (256g rotors) in a 2U application to smooth 500W GPU power. The steady-state expected fan speed at automatic operation is 4300 RPM, with each fan consuming 2W. When the GPU enters a low-power state, to smooth the power drop, the 80mm fans can be increased to 18300 RPM at maximum operation, consuming up to 58W per fan. However, due to the increased inertial mass of the steel rotors, four fans can be used to smooth the 500W GPU power.

[0039] Another example implementation uses 40mm fans with acrylonitrile butadiene styrene (ABS) rotors (7g rotors) in a 1U application to smooth 500W GPU power. The steady-state expected fan speed at automatic operation is 8000 RPM, with each fan consuming 1.4W. When the GPU enters a low-power state, to smooth the power drop, the 40mm fans can ramp up to 38,000 RPM at maximum operation, consuming up to 31W per fan. Therefore, sixteen fans are used to smooth 500W of GPU power.

[0040] Another example implementation uses 40mm fans with steel rotors (52g rotors) in a 1U application to smooth 500W of GPU power. The steady-state expected fan speed at automatic operation is 8000 RPM, with each fan consuming 1.4W. When the GPU enters a low-power state, to smooth the power drop, the 40mm fans can ramp up to 38,000 RPM at maximum operation, consuming up to 31W per fan. Therefore, ten fans are used to smooth 500W of GPU power.

[0041] Rack controller 232 controls the GPU servers, and for each GPU server system board within the GPU servers, a Baseboard Management Controller (BMC), such as BMC 228, is included, which manages the interface between the GPU server hardware and the software running on it. Various sensors built into the servers (such as temperature sensor 240) report measured parameters to the BMC, such as GPU power consumption, GPU workload, temperature, cooling fan speed, power status, operating system (OS) status, etc. Rack controller 232 and / or the BMC monitor the sensors and can responsively take thermomechanical power smoothing actions. For example, the temperature sensor provides feedback control for fan speed in automatic operation.

[0042] When GPU processors are used for ML and / or AI model training, they operate with synchronous and fluctuating computational workloads. This results in synchronous and fluctuating net power consumption, which ranges between minimum and maximum power consumption. At minimum power consumption, the processor array is essentially idle, while at maximum power consumption, the GPU processors are fully loaded with computational workloads before and after each collection operation. The processor array is essentially idle during collection operations performed between computational workloads, either by rack controller 232 or elsewhere.

[0043] The fan can operate at a variable speed according to the requirements of the rack controller 232 and / or the BMC. The action that the rack controller 232 and / or the BMC can take to achieve thermomechanical power smoothing in response to (or predict) the synchronous and fluctuating net power consumption of the GPU processor is to change the fan's operating speed. This action turns the fan into a thermomechanical power smoothing device to adapt to the synchronous and fluctuating net power consumption of the GPU processor.

[0044] The fans are configured to operate in at least two states: automatic and maximum, as described above. The rack controller 232 and / or BMC monitors various sensors within the GPU server and indicators in response to (or predict) the GPU workload to select the fan operating state. Furthermore, in various implementations, the fan operating state can be selected as the same for fans across server racks within a data center, across individual GPU servers within a server rack, or across a fan set within individual GPU servers.

[0045] In some implementations, synchronous RPM variations of numerous fans in a data center can introduce undesirable harmonics into the mechanical system. In such cases, the rack controller 232 can guide asynchronous fan speed increases and decreases. Therefore, the change between the aforementioned maximum and automatic operating states can occur synchronously or asynchronously across all fans controlled by the rack controller 232. Furthermore, asynchronous fan speed increases and decreases can include setting slightly different maximum fan speeds to avoid harmonic resonance.

[0046] Although processors 208, 209, and 210 are explicitly disclosed as GPUs in this document, and the servers within server rack 230 are explicitly disclosed as GPU servers in this document, other server and processor types operating with periodic and fluctuating workloads and the resulting power consumption can similarly employ cooling fans operating as thermomechanical power smoothing devices. Furthermore, although Figure 2 The diagram and description above illustrate integrated cooling fans, but external fans (e.g., rack-mounted fans) can be used to achieve a similar effect across several connected servers (e.g., all servers sharing a rack with rack-mounted fans).

[0047] Furthermore, although the fans are explicitly described above as being specific to individual GPU servers or server racks 230 containing multiple GPU servers, the thermomechanical power smoothing devices disclosed herein can be similarly applied to discrete fan units, such as fan walls within server racks 230, external fans used to cool heat exchangers which in turn are used to cool GPU servers (e.g., via liquid cooling), or even air handling fans used to provide heating or air conditioning to data center facilities.

[0048] Figure 3 The illustration shows an example reactive feedback control scheme 342 for achieving thermomechanical power smoothing using a cooling fan set 302. A GPU 308 and a GPU server (not shown, see example) are also included. Figure 1 Other components inside the server (100) are convectively cooled by cooling fans 302, which draw air through inlets, passing it over the GPU 308 and other internal components in the GPU server designed to be convectively cooled, and expelling the heated air from the GPU server. A baseboard management controller (BMC) 328 manages the interface between the GPU server hardware and the software running on it. Various sensors built into the GPU server report measurement parameters to the BMC 328, such as GPU workload, temperature, cooling fan speed, power status, operating system (OS) status, etc. The BMC 328 monitors the sensors and can responsively take thermomechanical power smoothing actions.

[0049] When GPUs 308 are used for distributed ML and / or AI model training, they operate with synchronous and fluctuating computational workloads. This results in synchronous and fluctuating net power consumption, which oscillates between minimum and maximum power consumption. At minimum power consumption, the processor array is essentially idle, while at maximum power consumption, the GPUs 308 are fully loaded with computational workloads before and after each collection operation, with the processor array essentially idle during collection operations that occur between computational workloads. The BMC328 is capable of monitoring the power consumption of the GPUs 308 using various sensors, such as voltage or current sensors at the power inputs to the GPUs 308, as shown by dashed line 344.

[0050] BMC 328 can also control the operation of cooling fan 302, as shown by dashed line 346. Cooling fan 302 can then operate at a variable speed according to the requirements of BMC 328. BMC 328 can take actions in response to (or predict, see...) Figure 4 The action to achieve thermomechanical power smoothing for the synchronous and fluctuating net power consumption of GPU 308 is to change the operating speed of cooling fan 302. This action turns cooling fan 302 into a thermomechanical power smoothing device to adapt to the synchronous and fluctuating net power consumption of GPU 308.

[0051] Cooling fan 302 is configured to operate in at least two states: maximum and automatic. When BMC 328 detects a low-power state of GPU 308, indicating that GPU 308 is essentially idle, as shown by solid line 348, BMC 328 instructs cooling fan 302 to operate in maximum state, as shown by solid line 350. In maximum state, the fan speed is set to maximum, allowing the actual fan speed to rise to its maximum level. This consumes additional power, which can be helpful from a power smoothing perspective due to the low-power state of GPU 308. This also stores mechanical energy in the rotor of cooling fan 302, which can subsequently be consumed at a later time (e.g., when GPU 308 enters a high-power state). This also results in a temperature drop within the GPU server due to increased airflow through the GPU server. This lower temperature can be used later when GPU 308 again handles computational workloads that generate the corresponding heat energy to be dissipated (e.g., when GPU 308 enters a high-power state).

[0052] When the BMC 328 detects a high-power state in the GPU 308 and instructs the GPU 308 to handle the computational workload again, as shown by solid line 352, the BMC 328 instructs the cooling fan 302 to operate in automatic or automatic mode, as shown by solid line 354. In automatic mode, fan speed fluctuations are allowed to maintain the desired temperature within the GPU server. More specifically, the BMC 328 monitors one or more temperature sensors (not shown, see example...) Figure 2 Temperature sensors 240 measure the temperature of airflow, heatsink(s) (not shown, see, for example, heatsink 124), and / or GPU 308. In automatic operation, the BMC 328 allows fan speed fluctuations to maintain the monitored temperature(s) within the desired operating range. Automatic operation can be used as the default state for cooling fan 302.

[0053] In LOW or OFF operating mode, the fan speed is set to minimum, which can be zero when the monitored temperature(s) are below the desired operating range. The LOW or OFF operating mode can immediately follow the maximum operating mode when the GPU 308 resumes computational workload, but the corresponding thermal activity has not yet increased the monitored temperature(s) to a level requiring convection cooling. In some implementations, the LOW or OFF operating mode is overridden by an automatic operating mode, which sets the fan speed to minimum whenever the monitored temperature(s) are below the desired operating range.

[0054] Figure 4 An example predictive feedback control scheme 442 is illustrated for achieving thermomechanical power smoothing using a cooling fan set 402. Although Figure 3 The reactive feedback control scheme 342 is typically implemented at the GPU server level, but Figure 4 The predictive feedback control scheme 442 is typically implemented at the rack level. Other implementations are similar to... Figure 4 The predictive feedback control scheme of scheme 442 can be implemented at the GPU server level, or similar to... Figure 3 The reactive feedback control scheme of scheme 342 can be implemented at the rack level.

[0055] The rack manager or controller (RM) 432 controls the set of GPU servers within the rack, and the system board for each GPU server in the GPU server set includes a baseboard management controller (BMC), such as BMC 428, which is responsible for managing the interface between the GPU server hardware and the software running thereon. GPU (not shown, see example...) Figure 2 GPUs 208, 209, and 210) and GPU servers (not shown, see example) Figure 1 Other components inside the server 100 are convectively cooled by cooling fan 402, which draws air through an air inlet, passes the air over the GPU and other internal components in the GPU server designed to be convectively cooled, and exhausts the heated air out of the GPU server.

[0056] When GPUs are used for ML and / or AI model training, they operate with synchronous and fluctuating computational workloads. This results in synchronous and fluctuating net power consumption, which oscillates between minimum and maximum power consumption. At minimum power consumption, the processor array is essentially idle, while at maximum power consumption, the GPUs are fully loaded with computational workloads before and after each collection operation. The processor array is essentially idle during collection operations that occur between computational workloads. The RM 432 monitors the rack management interface, as shown by dashed line 444, to anticipate when a collection operation is about to begin or is nearing its start, thus predicting that the GPUs will soon be idle and that GPU power consumption will decrease accordingly.

[0057] RM 432 can also control the operation of the cooling fan via the associated BMC 428, as shown by dashed line 446. The cooling fan 402 can then operate at a variable speed as required by RM 432 via BMC 428. RM 432 can take, predict (or respond to, as per BMC 428) actions, predictions, or responses. Figure 3 The action to achieve thermomechanical power smoothing for the synchronous and fluctuating net power consumption of the GPU is to change the operating speed of the cooling fan 402. This action turns the cooling fan 402 into a thermomechanical power smoothing device to adapt to the synchronous and fluctuating net power consumption of the GPU.

[0058] Cooling fan 402 is configured to operate in at least two states: maximum and automatic. When RM 432 predicts that the GPU is about to enter a low-power state based on a stop GPU workload signal (indicating an upcoming collection operation), as shown by solid line 448, RM 432 commands BMC 428 to instruct cooling fan 402 to operate in maximum state, as shown by solid line 450. In maximum state, the fan speed is set to maximum, allowing the actual fan speed to rise to its maximum level. This consumes additional power, which can be helpful from a power smoothing perspective due to the GPU's low-power state. This also stores mechanical energy in the rotor of cooling fan 402, which can be subsequently consumed at a later time (e.g., when the GPU enters a high-power state). This also results in a decrease in temperature within the GPU server due to increased airflow through the GPU server. This lower temperature can be used later when the GPU again solves a computational workload that generates the corresponding heat energy to be dissipated (e.g., when the GPU enters a high-power state).

[0059] When RM 432 predicts that the GPU is about to enter a high-power state based on the GPU workload initiation signal (indicating the computing workload to be allocated), as shown by solid line 452, RM 432 issues a command to BMC 428 to instruct cooling fan 402 to operate in automatic or automatic mode, as shown by solid line 454. In automatic mode, fan speed fluctuations are allowed to maintain the desired temperature within the GPU server. More specifically, BMC 428 monitors one or more temperature sensors (not shown, see example...) Figure 2 Temperature sensors 240 measure the temperature of airflow, heatsink(s) (not shown, see, for example, heatsink 124), and / or the GPU. In automatic operation, the BMC 428 allows fan speed fluctuations to maintain the monitored temperature(s) within the desired operating range. Automatic operation can be used as the default state for cooling fan 402.

[0060] In LOW or OFF operating mode, the fan speed is set to minimum, which can be zero when the monitored temperature(s) are below the desired operating range. The LOW or OFF operating mode can immediately follow the maximum operating mode when the GPU resumes computational workload, but the corresponding thermal activity has not yet increased the monitored temperature(s) to a level requiring convection cooling. In some implementations, the LOW or OFF operating mode is overridden by an automatic operating mode, which sets the fan speed to minimum whenever the monitored temperature(s) are below the desired operating range.

[0061] Figure 5 Figure 556 is an example of synchronous load power diagram, illustrating the operation of a cooling fan as a thermomechanical power smoothing device. Figure 556 illustrates three distinct graphs: fan speed 558, air-cooled component temperature 560, and power consumption 562, aligned in time 564. Figure 556 is qualitative in nature and is not intended to represent specific fan speeds, temperatures, or power consumption precisely to scale.

[0062] Between time t0 and t1, the GPU server, along with its associated GPU 508 and cooling fan 502, operates in an automatic steady-state state. The GPU server handles the computational workload, in which the GPU operates at high power. Fan 502 uses a feedback control loop and runs at a certain speed to maintain the temperature of the air-cooled components at T1.

[0063] At time t1, GPU 508 abruptly enters an idle state in response to the collection operation of the interrupted computational workload. This causes GPU 508 power consumption to drop rapidly to a low-power state. To offset this effect and achieve thermomechanical power smoothing, the GPU server triggers cooling fan 502 to consume the additional power now available. Between times t1 and t2, cooling fan 502 operates at maximum speed, which allows cooling fan 502 to consume the power previously consumed by GPU 508 and avoids any significant drop in overall power consumption (if any). Maximum operation speeds fan 558 up to its maximum speed (V0). max This allows mechanical energy to be stored within the cooling fan 502. This also causes a supercooling effect within the GPU server, as shown by the air-cooled component temperature 560 dropping below T1. This supercooling effect can also be considered thermal energy storage, since the GPU server does not need to operate at supercooled temperatures, and the supercooling effect can generate a power saving 566 between times t2 and t3.

[0064] At time t2, GPU 508 was assigned a new computational workload and suddenly returned to a high-power state. This triggered cooling fan 502 to return to automatic operation. Since the air-cooled component temperature 560°C was lower than T1, the fan speed decreased to V. min This speed can be relatively low or zero. Without a regenerative braking mechanism, the fan spins freely to gradually decelerate. With a regenerative braking mechanism, the fan decelerates even faster and returns a portion of its energy as electricity to the GPU server. The regenerative braking mechanism can also act as a UPS to bridge momentary power outages. Since the fan consumes very little power (if any), this results in the aforementioned power saving of 566 until time t3.

[0065] At time t3, the air-cooled component temperature 560 approaches that of T1. This causes the fan speed to rebound, approaching the initial steady-state speed between times t0 and t1. This increase in fan speed 558 causes a similar increase in fan power consumption 562 between times t3 and t4. At time t4, fan speed 558, air-cooled component temperature 560, and power consumption 562 have all returned to a steady state similar to or the same as the steady state between times t0 and t1. This situation can persist until GPU 508 suddenly enters an idle state again in response to another interrupted computational workload's collection operation, similar to the situation at time t1. Then, as GPU 508 transitions between high-power and low-power states, the above process iteratively repeats.

[0066] Figure 6The illustration shows an example operation 600 using a cooling fan operating as a thermomechanical power smoothing device. Operation 600 is designed to achieve thermomechanical power smoothing for one or more servers, possibly within a server rack or even across a data center.

[0067] Operation 605 causes the processor array to operate over time with synchronously fluctuating workloads and correspondingly fluctuating net power consumption. A variable-speed cooling fan array supplies cooling air to the processors and operates as a thermomechanical power smoothing device for the servers(s).

[0068] Monitoring operation 610 monitors the net power consumption of the processor array. In various implementations, monitoring operation 610 can be reactive or predictive. For example, monitoring operation 610 can monitor and react to detected changes in the processor's net power consumption. For another example, monitoring operation 610 can monitor stop and start workload signals and, in response to receiving stop and start workload signals, predict changes in the processor's net power consumption.

[0069] A controller (e.g., a rack manager or BMC) can change the fan operating state in response to monitoring operation 610. A first setting operation 615, in response to a low-power state of the processor, sets the cooling fan speed to maximum. A second setting operation 620, in response to a high-power state of the processor, sets the cooling fan speed to automatic. Monitoring operation 610 runs continuously, and setting operations 615 and 620 are repeated as indicated by the controller based on changes in power consumption of the processor array. Although specific examples of controllers (e.g., rack managers and BMCs) are provided herein, controllers are envisioned to operate similarly at various scales, such as server-scale, rack-scale, row-scale, co-location-scale, data center-scale, etc. The main difference lies in how many fans the controller is responsible for, but the control scheme described herein can be scaled accordingly.

[0070] The operations that constitute embodiments of the invention described herein are referred to herein as operations, steps, objects, or modules. Unless otherwise expressly required by the claim language or a particular order is inherently necessary by the claim language, operations may be performed in any order, and operations may be added or omitted as desired.

[0071] The servers and computing devices disclosed herein may include various tangible computer-readable storage media (e.g., multiple memory devices and multiple storage medium devices) and intangible computer-readable communication signals. Tangible computer-readable storage may be embodied by any available medium accessible to the computing device and includes volatile and non-volatile storage media, as well as removable and non-removable storage media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Tangible computer-readable storage media include, but are not limited to, RAM, read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, optical disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage devices, magnetic cartridges, magnetic tapes, disk storage devices or other magnetic storage devices, or any other tangible medium that can be used to store desired information and is accessible to the computing device. Tangible computer-readable storage media do not include intangible communication signals.

[0072] Intangible computer-readable communication signals can be embodied as computer-readable instructions, data structures, program modules, or other data residing in modulated data signals (such as carrier waves or other signal transmission mechanisms). The term "modulated data signal" means a signal in which one or more characteristics of a signal are set or altered in such a way as to encode information in the signal. By way of example and not limitation, intangible communication signals include signals propagated through wired media (such as wired networks or direct wired connections) and wireless media (such as acoustic, radio frequency (RF), infrared (IR), and other wireless media).

[0073] Some implementations may include an article of writing. The article of writing may include a tangible storage medium that stores logic. Examples of storage media may include one or more types of computer-readable storage media capable of storing electronic data, including volatile or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, etc. Examples of logic may include various software elements, such as software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, operation segments, methods, procedures, software interfaces, application programming interfaces (APIs), instruction sets, computational code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof. In one embodiment, for example, the article of writing may store executable computer program instructions that, when executed by a computer, cause the computer to perform methods and / or operations according to the embodiments described. Executable computer program instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, etc. Executable computer program instructions can be implemented according to predefined computer languages, methods, or syntaxes to instruct the computer to perform a segment of operations. Instructions can be implemented using any appropriate high-level, low-level, object-oriented, visual, compiled, and / or interpreted programming languages.

[0074] The implementation described herein includes a server comprising a processor array for operating with a synchronously fluctuating workload and correspondingly fluctuating net power consumption over time; a variable-speed cooling fan array for supplying cooling air to the processors and operating as a thermomechanical power smoothing device for the server; and a controller. The controller is configured to set the cooling fan speed to maximum in response to a low-power state of the processor, and to set the cooling fan speed to automatic in response to a high-power state of the processor.

[0075] The processor can be a graphics processing unit (GPU) used to handle computational workloads for training one or both of machine learning (ML) and artificial intelligence (AI) models.

[0076] The net power consumption of synchronous fluctuations can fluctuate between minimum power consumption and maximum power consumption. At minimum power consumption, the processor array is basically idle, while at maximum power consumption, the processor array is fully loaded with computing workload.

[0077] The controller can be a baseboard management controller (BMC) for servers.

[0078] The processor's power consumption status can be detected or predicted by the BMC.

[0079] The server may also include one or more heat sinks to increase the server's heat storage capacity.

[0080] Variable speed cooling fans may include weighted rotors to increase mechanical storage capacity in servers.

[0081] The server may also include a regenerative braking mechanism connected to one or more cooling fans in the cooling fan assembly, which is used to recover mechanical storage capacity as power for the server.

[0082] Regenerative braking mechanisms can be used as uninterruptible power supplies (UPS) to bridge momentary power outages.

[0083] The server may also include one or more temperature sensors to provide feedback control of fan speed in automatic mode.

[0084] When running at maximum fan speed, additional mechanical energy can be stored in the cooling fan and then consumed in response to the processor entering a high-power state.

[0085] When the cooling fan is running at maximum fan speed, the additional heat is released into the convective airflow and subsequently recovered in response to the processor entering a high-power state.

[0086] The implementation described herein includes a method for performing thermomechanical power smoothing for a server. The method includes: operating a processor array over time with synchronously fluctuating workloads and correspondingly fluctuating net power consumption, wherein a variable-speed cooling fan array supplies cooling air to the processors and operates as a thermomechanical power smoothing device for the server; monitoring the net power consumption of the processor array; setting the cooling fan speed to maximum in response to a low-power state of the processor; and setting the cooling fan speed to automatic in response to a high-power state of the processor.

[0087] The monitoring operation can react to detected changes in the processor's net power consumption.

[0088] Monitoring operations can respond to received stop and start workload signals and predict changes in the processor's net power consumption.

[0089] The implementation described herein includes a server rack comprising a server array, each server including a processor array for operating over time with synchronously fluctuating workloads and correspondingly fluctuating net power consumption; a variable-speed cooling fan array for supplying cooling air to the processors and operating as a thermomechanical power smoothing device for the server array; and a controller. The controller is configured to set the cooling fan speed to maximum in response to a low-power state of the processors, and to set the cooling fan speed to automatic in response to a high-power state of the processors.

[0090] Variable speed cooling fan arrays can cool a server rack or one of the servers in a server array.

[0091] The controller can be one of the following: a data center controller for a server rack and multiple other server racks, a rack controller for a server rack, or a baseboard management controller (BMC) for a server in a server array within a server rack.

[0092] The controller can perform asynchronous changes in cooling fan speed across the variable speed cooling fan array.

[0093] The processor can be a graphics processing unit (GPU) used to handle computational workloads for training one or both of machine learning (ML) and artificial intelligence (AI) models.

[0094] The foregoing specification, examples, and data provide a complete description of the structure and use of exemplary embodiments of the present invention. Since many embodiments of the invention can be made without departing from the spirit and scope of the invention, the invention is contained within the appended claims. Furthermore, structural features of different embodiments can be combined into yet another embodiment without departing from the stated claims.

Claims

1. A server (100), comprising: The processor array (210)(209)(208)(110)(108) is used to operate with synchronously fluctuating workload and corresponding synchronously fluctuating net power consumption over time; A variable-speed cooling fan array (502) (402) (302) (202) (102) is used to supply cooling air to the processor and operates as a thermomechanical power smoothing device for the server; and Controller (232) (128), used to: In response to the processor's low-power state, the cooling fan speed is set to maximum; and In response to the processor's high power consumption state, the cooling fan speed is set to automatic.

2. The server of claim 1, wherein the processor is a graphics processing unit (GPU) for handling computational workloads for training one or both of machine learning (ML) and artificial intelligence (AI) models.

3. The server according to claim 1, wherein the net power consumption of the synchronous fluctuation fluctuates between a minimum power consumption and a maximum power consumption, wherein the processor array is substantially idle at the minimum power consumption and the processor array is fully loaded with computing workload at the maximum power consumption.

4. The server of claim 1, wherein the controller is a baseboard management controller (BMC) for the server.

5. The server of claim 4, wherein the power consumption state of the processor is detected or predicted by the BMC.

6. The server according to claim 1, further comprising: One or more heat sinks are used to increase the thermal storage capacity of the server.

7. The server of claim 1, wherein the variable speed cooling fan includes a weighted rotor for increasing the mechanical storage capacity of the server.

8. The server according to claim 1, further comprising: A regenerative braking mechanism, connected to one or more of the cooling fans, is used to recover mechanical storage capacity for power use in the server.

9. The server of claim 8, wherein the regenerative braking mechanism is used as an uninterruptible power supply (UPS) to bridge transient power outages.

10. The server according to claim 1, further comprising: One or more temperature sensors are used to provide feedback control for fan speed in automatic mode.

11. The server according to claim 1, wherein: When running at maximum fan speed, additional mechanical energy is stored in the cooling fan and is subsequently consumed in response to the processor entering the high-power state.

12. The server according to claim 1, wherein: When the cooling fan is running at maximum fan speed, the additional heat energy is released into the convective airflow and subsequently recovered in response to the processor entering the high-power state.

13. A method (600) for performing thermomechanical power smoothing on a server, comprising: The processor array is operated over time with a synchronously fluctuating workload and a corresponding synchronously fluctuating net power consumption, wherein a variable-speed cooling fan array supplies cooling air to the processor and operates as a thermomechanical power smoothing device for the server (605). Monitor the net power consumption of the processor array (610). In response to the processor's low-power state, the cooling fan speed is set to maximum (615); and In response to the processor's high power consumption state, the cooling fan speed is set to automatic (620).

14. The method of claim 13, wherein the monitoring operation responds to a detected change in the net power consumption of the processor.

15. The method of claim 13, wherein the monitoring operation predicts the change in the processor's net power consumption in response to receiving a stop and start workload signal.

Images