Liquid cooling apparatus, data center, and method of conditioning electronic equipment

Abstract

A liquid cooling apparatus for a data center is disclosed. The apparatus includes two cooling loops, a primary cooling loop with fixed cooling capacity and a secondary cooling loop that supplements the primary cooling loop when the cooling capacity of the primary cooling loop is insufficient. Both cooling loops can use a phase change fluid whose vapor pressure is monitored to control the cooling capacity of the secondary cooling loop in response to fluctuating thermal loads. The apparatus includes two types of energy sources, such as a photovoltaic system and an electrical power storage. The photovoltaic system can power the secondary cooling loop based on the vapor pressure to control the fluid flow rate or air flow rate through the secondary cooling loop. The photovoltaic system can charge the electrical power storage when not powering the secondary cooling loop. A data center and a method of regulating thermal and electrical loads of an electronic device are also disclosed.

Description

TECHNICAL FIELD

[0001] Embodiments of the invention generally relate to server and data center cooling. More specifically, embodiments of the invention relate to an integrated thermal and electrical system design with self-regulating capabilities to provide cooling and power to data centers with high power density racks and clusters. BACKGROUND

[0002] Cooling is a critical consideration in computer system and data center design. The number of high performance electronic components, such as high performance processors packaged within servers, has steadily increased, increasing the heat generated and dissipated during regular operation of the servers. If the thermal environment in which the data center operates is allowed to increase in temperature over time, the reliability of the servers used within the data center decreases. Maintaining an appropriate thermal environment is critical for the normal operation of these servers in the data center and for maximizing server performance, reliability, and lifetime. It requires more effective and efficient cooling solutions, particularly in the case of cooling these high performance servers.

[0003] Servers and other high performance electronic components such as central processing units (CPUs), graphics processing units (GPUs), etc. are typically tightly packed in highly integrated clusters of chips, boards, or racks to produce very high power and thermal densities. In addition to meeting the increasing cooling capacity, increasing power density also requires data centers to be able to flexibly increase their power capacity. There is a need to improve energy and power efficiency to reduce the operating costs of powering high performance data centers, mitigate environmental impact, and meet sustainability goals and regulatory requirements. Existing solutions to meet the dual demands of increased power density and cooling capacity can require the design of large power and cooling buffers that are difficult to scale or inefficiently used during the lifetime of the data center. The power and cooling systems can also not be integrated and controlled to meet the demands of short term fluctuations. Therefore, there is a need for an integrated thermal and electrical system design that is scalable, reliable, effective, serviceable, and low cost to meet the thermal and power management needs of high performance electronic components in data centers. Furthermore, due to environmental regulations, it is becoming more important to develop and deploy renewable energy sources for powering data centers. SUMMARY

[0004] According to one aspect of the present invention, a liquid cooling apparatus for a data center is disclosed. The liquid cooling apparatus comprises: a primary cooling loop having a primary condenser to remove heat from one or more information technology components using a cooling liquid; a sensor to monitor a vapor pressure of the cooling liquid; a secondary cooling loop having a secondary condenser connected to close the secondary cooling loop to supplement the primary cooling loop to remove heat from the information technology components when the vapor pressure of the cooling liquid exceeds a threshold; and an electrical subsystem comprising a renewable energy source and an energy storage system, the electrical subsystem configured to distribute power to connect the secondary condenser to the secondary cooling loop in response to the vapor pressure and to control a cooling capacity of the secondary cooling loop.

[0005] According to another aspect of the present invention, a data center is disclosed. The data center comprises: an electrical subsystem comprising at least one renewable energy source and at least one energy storage system; a plurality of electronic racks, each electronic rack containing a plurality of server enclosures and each server enclosure corresponding to one or more servers, wherein each electronic rack comprises: a primary cooling loop having a primary condenser to remove heat from the servers using a cooling liquid; a sensor to monitor a vapor pressure of the cooling liquid; and a secondary cooling loop having a secondary condenser connected to close the secondary cooling loop to supplement the primary cooling loop to remove heat from the servers when the vapor pressure of the cooling liquid exceeds a threshold; wherein the electrical subsystem is configured to distribute power to connect the secondary condenser to the secondary cooling loop in response to the vapor pressure and to control a cooling capacity of the secondary cooling loop.

[0006] According to yet another aspect of the present invention, a method of regulating a thermal load and an electrical load of an electronic device is disclosed. The method comprises: connecting a primary condenser of a primary cooling loop to circulate a cooling liquid to remove heat from the electronic device; connecting a renewable energy source to charge an energy storage system; monitoring a vapor pressure of the cooling liquid; connecting a secondary condenser to a secondary cooling loop to supplement the primary cooling loop in removing heat from the electronic device when the vapor pressure of the cooling liquid exceeds a threshold; and distributing power from the renewable energy source and the energy storage system to regulate a cooling capacity of the secondary cooling loop based on the vapor pressure. BRIEF DESCRIPTION OF DRAWINGS

[0007] Embodiments of the invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like reference numerals refer to similar elements.

[0008] Figure 1 is a block diagram illustrating an example of a data center facility according to one embodiment.

[0009] Figure 2 is a block diagram illustrating an example of an electronic rack according to one embodiment.

[0010] Figure 3 is a block diagram illustrating an example of a cold plate configuration according to one embodiment.

[0011] Figure 4 An example of a thermal, power, and control architecture of an integrated system for providing enhanced cooling capacity and auxiliary power for racks of electronic components of a data center according to one embodiment is shown.

[0012] Figure 5 An example of a thermal, power, and control architecture of an integrated system according to one embodiment that shares a photovoltaic system and an energy storage system to regulate cooling capacity and electrical load among multiple racks is shown.

[0013] Figure 6 An example of a thermal, power, and control architecture of an integrated system according to one embodiment that provides a dedicated photovoltaic system for each rack, but shares an energy storage system among multiple racks to regulate cooling capacity and electrical load of the racks is shown.

[0014] Figure 7 An example of a thermal, power, and control architecture of an integrated system according to one embodiment that shares a photovoltaic system and an energy storage system to regulate cooling capacity and electrical load among multiple racks, in which the racks also share a condenser of a secondary cooling loop and a fluid storage system in the integrated system is shown.

[0015] Figure 8 is a flowchart illustrating an example of a method for regulating cooling capacity and electrical load of electronic equipment in a data center or computer system in response to thermal and power density according to one embodiment. DETAILED DESCRIPTION

[0016] Various embodiments and aspects of the application will be described in detail with reference to the drawings and the following detailed description. The drawings are provided to illustrate various embodiments of the application and are not intended to limit the application. Numerous specific details are described to provide a thorough understanding of various embodiments of the application. However, in certain instances, well-known or conventional details are not described in order to provide a concise discussion of embodiments of the application.

[0017] Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

[0018] Disclosed is a design for an integrated thermal and electrical system with self-regulating capabilities to provide enhanced cooling capacity and auxiliary power to support high thermal and power density requirements of computer systems or data centers. In one aspect, the integrated design's thermal system includes two cooling loops, a primary cooling loop with fixed cooling capacity for normal operation and a secondary cooling loop to supplement the primary cooling loop when the fixed cooling capacity of the primary cooling loop is insufficient. Both the primary and secondary cooling loops have respective condensers that act as heat exchange units. Each condenser can have a secondary loop to deliver cooling liquid to electronic racks of a data center to remove heat generated by servers or other electronic components of the racks. Each condenser can have a primary loop connected to external cooling liquid supply and return lines to carry fluid for heat exchange with the secondary loop. The secondary loop of each condenser can use a phase change fluid whose vapor pressure changes due to the thermal load of the servers or electronic components. When the secondary cooling loop is connected to supplement the cooling capacity of the primary cooling loop, the system can control the flow rate of fluid in the primary loop of the secondary condenser or the air flow rate through the secondary condenser based on the vapor pressure of the phase change fluid to respond to fluctuations in the thermal load.

[0019] In one aspect, the electrical system includes a photovoltaic system and a power storage system to provide auxiliary power to supplement the primary utility power. The photovoltaic system can power the primary loop of the secondary condenser based on the vapor pressure of the phase change fluid to control the flow rate of fluid or the air flow rate through the secondary condenser. The photovoltaic system can charge the power storage system when it is not powering the secondary cooling loop. The power storage system can be a battery to power the primary loop of the secondary condenser when power from the photovoltaic system is unavailable or insufficient. In one aspect, the power storage system can power the primary cooling loop. In one aspect, the power storage system can provide auxiliary power to the electronic components in response to increased power demands of the computer system or data center. The integrated thermal and electrical system with self-regulating capabilities provides efficient thermal and power management of high performance IT clusters, increasing cooling performance, cooling reliability, power efficiency, system scalability, sustainability requirements, and reducing costs to meet high thermal and power density requirements.

[0020] In one aspect, the photovoltaic system and the power storage system are shared among multiple racks or IT clusters to regulate the cooling capacity and electrical load of the racks or IT clusters. A controller can receive vapor pressures to individually connect and control the cooling capacity of the secondary cooling loop to each rack in response to changes in the vapor pressure due to their respective thermal loads. The controller can also receive voltages and availability of the photovoltaic system and the power storage system to be used in conjunction with the vapor pressure information from each rack to regulate power from the photovoltaic system or the power storage system to the primary loop of the secondary condenser of each rack.

[0021] In one aspect, the photovoltaic systems are dedicated to each rack, but the power storage systems are shared among multiple racks to regulate the cooling capacity and electrical load of the racks. A controller can receive the vapor pressure to individually connect and control the secondary cooling loop to the cooling capacity of each rack in response to changes in the vapor pressure due to their respective thermal loads. The controller can also receive the voltage of the dedicated photovoltaic systems and shared power storage systems to be used in conjunction with the vapor pressure information from each rack to regulate the power from the dedicated photovoltaic systems or shared power storage systems of each rack to the primary loop of the secondary condenser of each rack.

[0022] In one aspect, the secondary condenser of the secondary cooling loop and the fluid storage system are shared among multiple racks. The multiple racks also share the photovoltaic systems and the power storage systems. A controller can receive the vapor pressure from each rack to individually connect and control the secondary cooling loop from the shared secondary condenser and fluid storage system to the cooling capacity of each rack. The fluid vapor line of the secondary cooling loop from each rack is connected to the shared secondary condenser. The return of the shared secondary condenser is connected to the fluid storage system. The liquid fluid supply line of the secondary cooling loop for each rack is connected to the fluid storage system and regulated by the controller based on the respective vapor pressure of the rack in response to its thermal load. The controller can regulate the power from the photovoltaic systems or power storage systems to the primary loop of the shared secondary condenser. The power storage systems can be charged by the photovoltaic systems or the primary utility power to power the primary cooling loop or electronic components of the racks.

[0023] Figure 1 is a block diagram illustrating an example of a data center or data center unit according to one embodiment. In this example, Figure 1 A top view of at least a portion of a data center is shown. Referring to Figure 1 In this embodiment, each row includes an array of electronic racks such as electronic racks 110A-110N. However, more or fewer rows of electronic racks can be implemented. Typically, the rows 101-102 are aligned in parallel with their front ends facing each other and their back ends facing away from each other, forming an aisle 103 in the middle to allow a manager to walk in. However, other configurations or arrangements can also be applied. For example, two rows of electronic racks can face each other back to back without forming an aisle in the middle while their front ends face away from each other. The back ends of the electronic racks can be connected to a room cooling liquid manifold.

[0024] In one embodiment, each of the electronic racks (e.g., electronic racks 110A-110N) includes a housing to house a plurality of IT components arranged in a stack running therein. The electronic rack can include a cooling liquid manifold, a plurality of server slots (e.g., configured with a standard rack or chassis of the same or similar form factor), and a plurality of server chassis (also referred to as server blades or server racks) capable of being inserted into and removed from the server slots. Each server chassis represents a computing node having one or more processors, memory, and / or persistent storage (e.g., hard disk), where the computing node can include one or more servers running therein. At least one processor is attached to a liquid cold plate (also referred to as a cold plate assembly) to receive cooling liquid. In addition, one or more optional cooling fans are associated with the server chassis to provide air cooling to the computing node contained therein. Note that the cooling system 120 can be connected to a plurality of data center systems, such as the data center system 100.

[0025] In one embodiment, the cooling system 120 includes an external liquid loop that is connected to a cooling tower or dry cooler outside of the building / housing container. The cooling system 120 can include, but is not limited to, evaporative cooling, free air, rejection to a large thermal mass, and waste heat recovery designs. The cooling system 120 can include or be connected to a cooling liquid source that provides cooling liquid.

[0026] In one embodiment, each server chassis is modularly connected to the cooling liquid manifold such that the server chassis can be removed from the electronic rack without affecting the operation of the remaining server chassis and the cooling liquid manifold in the electronic rack. In another embodiment, each server chassis is connected to the cooling liquid manifold by a quick release connection assembly having a server liquid inlet connector and a server liquid outlet connector connected to a flexible hose to distribute cooling liquid to the processors. The server liquid inlet connector receives cooling liquid from the cooling liquid manifold mounted on the back end of the electronic rack via a rack liquid inlet connector. The server liquid outlet connector ejects warmer or hotter liquid carrying heat exchanged from the processors via a rack liquid outlet connector to the cooling liquid manifold and then back to a chiller distribution unit (CDU) within the electronic rack.

[0027] In one embodiment, a cooling liquid manifold disposed on the back end of each electronic rack is connected to a liquid supply line 132 (also referred to as a room supply manifold) to receive cooling liquid from the cooling system 120. The cooling liquid is distributed through liquid distribution loops attached to cold plate assemblies on which processors are mounted to remove heat from the processors. The cold plates are configured similar to heat sinks with liquid distribution tubes attached or embedded therein. The resulting warmer or hotter liquid carrying heat exchanged from the processors is transported back to the cooling system 120 via a liquid return line 131 (also referred to as a room return manifold).

[0028] The liquid supply / return lines 131-132 are referred to as data center or room liquid supply / return lines (e.g., global liquid supply / return lines) that supply cooling liquid to all electronic racks of the rows 101-102. The liquid supply line 132 and the liquid return line 131 are connected to heat exchangers of the CDUs located within each electronic rack, forming a primary loop. Secondary loops of the heat exchangers are connected to the cold plates of the processors in each server chassis in the electronic rack. In one embodiment, the liquid supply / return lines 131-132 can be connected to a primary loop of a condenser of a primary cooling loop or a secondary cooling loop of an integrated thermal and electrical system having two cooling loops to be discussed.

[0029] In one embodiment, the data center system 100 also includes an optional airflow delivery system 135 (also referred to as an air supply system) for generating an airflow to pass through the air spaces of the server chassis of the electronic racks to exchange heat generated by the compute nodes (e.g., servers) due to operation of the compute nodes and to exhaust the heat exchanged by the airflow to an external environment or a cooling system (e.g., an air-to-liquid heat exchanger) to reduce the temperature of the airflow. For example, the air supply system 135 generates an airflow of cool / cold air to circulate through the electronic racks 110A-110N from the aisle 103 to carry away the heat exchanged. In one embodiment, the airflow of cool / cold air can be delivered to a condenser of a primary cooling loop or a secondary cooling loop of an integrated thermal and electrical system having two cooling loops to be discussed.

[0030] The cool airflow enters the electronic racks through the front end of the electronic racks, while the warm / hot airflow exits the electronic racks from the back end of the electronic racks. The warm / hot air with the heat exchanged is exhausted from the room / building or cooled using a separate cooling system such as an air-to-liquid heat exchanger. Thus, the cooling system is a hybrid liquid-air cooling system in which a portion of the heat generated by the processors is removed by the cooling liquid through the respective cold plates, while the remaining portion of the heat generated by the processors (or other electronic or processing devices) is removed by airflow cooling.

[0031] Figure 2 This is a block diagram illustrating an electronics rack according to one embodiment. The electronics rack 200 can represent, for example... Figure 1 Any electronic rack shown, such as electronic rack 110A-110N as an example. See reference. Figure 2 According to one embodiment, the electronic rack 200 includes, but is not limited to, a CDU 201, a rack management unit (RMU) 202, and one or more server chassis 203A-203E (collectively referred to as server chassis 203). The server chassis 203 can be inserted into an array of server slots (e.g., standard racks) from either the front end 204 or the rear end 205 of the electronic rack 200. Note that although five server chassis 203A-203E are shown here, more or fewer server chassis can be housed within the electronic rack 200. It should also be noted that the specific locations of the CDU 201, RMU 202, and / or server chassis 203 are shown for illustrative purposes only; other arrangements or configurations of the CDU 201, RMU 202, and / or server chassis 203 may also be implemented. In one embodiment, the electronic rack 200 may be open to the environment or partially contained within a rack container, provided that cooling fans can generate airflow from the front to the rear end.

[0032] In addition, for at least some of the server chassis 203, optional fan modules (not shown) are associated with the server chassis. Each fan module includes one or more cooling fans. The fan modules may be mounted at the rear end or on the electronic rack of the server chassis 203 to generate airflow that exits from the front end 204, travels through the air space of the server chassis 203, and is present at the rear end 205 of the electronic rack 200.

[0033] In one embodiment, CDU 201 primarily includes a heat exchanger 211, a liquid pump 212, and a pump controller (not shown), as well as other components such as a reservoir, a power supply, monitoring sensors, etc. The heat exchanger 211 may be a liquid-to-liquid heat exchanger. The heat exchanger 211 includes a first loop having an inlet port and an outlet port, the inlet and outlet ports having a first pair of liquid connectors connected to external liquid supply / return lines 131-132 to form a main loop. The connectors connected to the external liquid supply / return lines 131-132 may be disposed on or mounted on the rear end 205 of the electronics rack 200. The liquid supply / return lines 131-132, also referred to as room liquid supply / return lines, may be connected to the cooling system 120 as described above.

[0034] Further, the heat exchanger 211 also includes a secondary loop having two ports with a second pair of liquid connectors connected to a liquid manifold 225 (also referred to as a rack manifold) to form a secondary loop, which can include a supply manifold (also referred to as a rack liquid supply line or a rack supply manifold) and a return manifold (also referred to as a rack liquid return line or a rack return manifold), the supply manifold supplies cooling liquid to the server chassis 203, the return manifold returns the warmer liquid to the CDU 201. Note that the CDU 201 can be any kind of commercially available or custom made CDU. Therefore, the details of the CDU 201 will not be described here. In one embodiment, the heat exchanger 211 in the CDU 201 can be a condenser of a primary cooling loop or a secondary cooling loop of an integrated thermal and electrical system having two cooling loops to be discussed.

[0035] Each server chassis 203 can include one or more IT components (e.g., central processing units or CPUs, general purpose / graphics processing units (GPUs), memories, and / or storage devices). Each IT component can perform data processing tasks, where the IT component can include software installed in a storage device, loaded into a memory, and executed by one or more processors to perform the data processing tasks. The server chassis 203 can include a host server (referred to as a host node) connected to one or more compute servers (also referred to as compute nodes), such as CPU servers and GPU servers. The host server (with one or more CPUs) is typically coupled with a client over a network (e.g., the Internet) to receive a request for a particular service, such as a storage service (e.g., a cloud-based storage service, such as backup and / or recovery), execution of an application to perform certain runs (e.g., image processing, deep data learning algorithms or modeling, etc., as part of a software as a service or SaaS platform). In response to the request, the host server allocates the task to one or more compute nodes or compute servers (with one or more GPUs) managed by the host server. The compute servers perform the actual tasks, which can generate heat during the runs.

[0036] The electronic rack 200 also includes an optional RMU 202 configured to provide and manage the power supplied to the servers 203 and the CDU 201. The RMU 202 can be connected to a power supply unit (not shown) to manage the power consumption of the power supply unit. The power supply unit can include necessary circuitry (e.g., alternating current (AC) to direct current (DC) or DC to DC power converters, batteries, transformers, or regulators, etc.) to provide power to the rest of the components of the electronic rack 200.

[0037] In one embodiment, the RMU 202 includes an optimization module 221 and a rack management controller (RMC) 222. The RMC 222 can include a monitor to monitor the operating conditions of various components within the electronic rack 200, such as the compute nodes 203, the CDU 201, and the fan modules, for example. Specifically, the monitor receives operating data from various sensors that represent the operating environment of the electronic rack 200. For example, the monitor can receive operating data that represents the temperature of the processors, the cooling liquid, and the airflow, which can be captured and collected via various temperature sensors. The monitor can also receive data that represents the fan power and the pump power generated by the fan modules and the liquid pump 212, which can be proportional to their respective speeds. These operating data are referred to as real-time operating data. Note that the monitor can be implemented as a separate module within the RMU 202. In one embodiment, the RMU 202 or the RMC 222 can be a controller that monitors the vapor pressure of the phase change fluid in the secondary loop of the condenser of the secondary cooling loop of the integrated thermal and electrical system with two cooling loops to be discussed.

[0038] Based on the operating data, the optimization module 221 performs optimization using a predetermined optimization function or optimization model to derive a set of optimal fan speeds for the fan modules and optimal pump speeds for the liquid pump 212 such that the total power consumption of the liquid pump 212 and the fan modules is minimized while the operating data associated with the cooling fans of the liquid pump 212 and the fan modules are within their respective design specifications. Once the optimal pump speeds and the optimal fan speeds have been determined, the RMC 222 configures the cooling fans of the liquid pump 212 and the fan modules based on the optimal pump speeds and the fan speeds. In one embodiment, the RMU 202, the RMC 222, or the optimization module 222 can be a controller that controls the flow rate of the fluid in the primary loop of the condenser of the secondary cooling loop of the integrated thermal and electrical system with two cooling loops to be discussed, or can control the airflow rate through the condenser of the secondary cooling loop.

[0039] As an example, based on the optimal pump speeds, the RMC 222 communicates with the pump controller of the CDU 201 to control the speed of the liquid pump 212, which in turn controls the liquid flow rate of the cooling liquid supplied to the liquid manifold 225 for distribution to at least some of the server enclosures 203. Similarly, based on the optimal fan speeds, the RMC 222 communicates with each fan module to control the speed of each cooling fan of the fan module, which in turn controls the airflow rate of the fan module. Note that each fan module can be controlled individually with its specific optimal fan speed, and different fan modules and / or different cooling fans within the same fan module can have different optimal fan speeds.

[0040] Note that, as Figure 2The rack configuration shown is illustrated and described for purposes of illustration only; other configurations or arrangements may be applied. For example, CDU 201 may be an optional unit. The cold plate of server chassis 203 may be connected to a rack manifold that can be directly connected to room manifolds 131-132 without using a CDU. Although not shown, a power supply unit may be located within the electronics rack 200. The power supply unit may be implemented as a standard chassis identical or similar to the server chassis, wherein the power supply chassis can be inserted into any standard rack in place of any server chassis 203. Furthermore, the power supply chassis may also include a battery backup unit (BBU) to provide battery power to server chassis 203 when mains power is unavailable. The BBU may include one or more battery packs, and each battery pack includes one or more battery cells, as well as the necessary charging and discharging circuitry for charging and discharging the battery cells.

[0041] Figure 3 This is a block diagram illustrating a processor cold plate configuration according to one embodiment. The processor / cold plate assembly 300 can be represented as follows: Figure 2 Any processor / cold plate structure of the server chassis 203 shown. (Refer to...) Figure 3 The processor 301 (also referred to as a processor chip) is inserted into a processor socket on a printed circuit board (PCB) or motherboard 302 mounted on other electronic components or circuits connected to the data processing system or server. The processor 301 also includes a cold plate 303 attached thereto, which is connected to a rack manifold that is connected to a liquid supply line and / or a steam return line. In one embodiment, the cold plate 303 may be a cooling device connected to a liquid supply line and / or a steam return line. Figure 4 The secondary circuit of the condenser is discussed in the main or secondary cooling circuit. A liquid supply line supplies the phase change fluid (PCF) coolant to the cooling unit. A return line returns the PCF heated vapor from the cooling unit. A portion of the heat generated by the processor 301 is removed by the coolant via the cold plate 303. The remaining heat enters the lower or upper air space, which can be removed by airflow generated by the cooling fan 304.

[0042] Figure 4 An example of the thermal, electrical, and control architecture of an integrated system 400 for providing enhanced cooling and auxiliary power to an electronic component rack in a data center, according to one embodiment, is shown. The integrated system includes: a thermal subsystem for removing heat generated by the electronic components; an electrical subsystem for supplying power to the thermal subsystem and the electronic components; and a control subsystem for controlling the operation of the thermal and electrical subsystems in response to fluctuations in thermal and electrical loads. The electronic components in the rack may include a server 401 or other IT components on a server chassis inserted into a corresponding standard rack within the rack.

[0043] The thermal subsystem includes two cooling loops, a primary cooling loop with a fixed cooling capacity for normal operation and a secondary cooling loop that supplements the primary cooling loop when the fixed cooling capacity of the primary cooling loop is insufficient to remove heat generated by the electronic components. The control subsystem only controls or regulates the cooling capacity of the secondary cooling loop in response to fluctuations in the thermal load. Both the primary and secondary cooling loops can use a phase change fluid that changes its phase from liquid to vapor when thermal energy is absorbed and changes its phase from vapor to liquid when thermal energy is dissipated. The vapor pressure of the phase change fluid can correspond to the amount of thermal energy absorbed.

[0044] Both the primary and secondary cooling loops have respective condensers that act as heat exchangers. Each condenser can have a secondary loop that connects to the server enclosures in the electronic rack to deliver cooling liquid to the cooling devices 403 or cold plates to remove heat generated by the servers 401 or other IT components. Each condenser can have a primary loop that connects to external liquid supply and return lines to carry fluid to exchange heat with the secondary loop. For example, the primary cooling loop has a primary condenser 415 that has a primary loop (not shown) that connects to external liquid supply and return lines that recirculate fluid to remove heat carried by the heated vapor of the phase change fluid of the secondary loop to change the phase of the phase change fluid in the secondary loop from vapor to liquid. After heat exchange, the cooling liquid in the secondary loop of the primary condenser 415 is returned to the cooling devices 403 through the primary liquid return loop 411. The flow rate of the cooling liquid in the primary condenser 415 or the cooling air flow rate through the primary condenser 415 is constant to provide a constant cooling capacity for the primary cooling loop.

[0045] Similarly, the secondary cooling loop has a secondary condenser 425 that has a primary loop 431 that connects to external liquid supply and return lines that recirculate fluid to remove heat carried by the heated vapor of the secondary loop. In one aspect, the control subsystem controls the cooling capacity of the primary loop 431 of the secondary condenser 425 by regulating the flow rate of liquid supplied to the primary loop 431 through the secondary condenser valve 433. In one aspect, the control subsystem can vary the flow rate of the fluid by regulating the pumping speed or frequency of a variable speed pump (not shown) to control the cooling capacity of the primary loop of the secondary condenser 425. In one aspect, the control subsystem can control the cooling capacity of the primary loop of the secondary condenser 425 by regulating the speed of the secondary condenser fan 429 to vary the air flow rate to the secondary condenser 425.

[0046] After exchanging heat with the primary loop 431, the cooled liquid in the secondary loop of the secondary condenser 425 is returned to the cooling device 403 through the secondary liquid return loop 421. The liquid from the primary liquid return loop 411 combines with the liquid from the secondary liquid return loop 421 to form the rack liquid system 405, which supplies phase change fluid to the cooling devices 403 in the plurality of server enclosures in the rack. The rack liquid system 405 also returns phase change fluid from the plurality of server enclosures through the primary vapor line 413 to the primary condenser 415 to complete the secondary loop of the primary condenser 415 or through the secondary vapor line 423 to the secondary condenser 425 to complete the secondary loop of the secondary condenser 425. The secondary vapor line 423 is connected to the secondary condenser through a pressure valve 427. When the vapor pressure is low, the pressure valve 427 is closed under normal operation to prevent phase change fluid in the secondary vapor line 423 from returning to the secondary condenser 425. When the vapor pressure reaches a trigger pressure threshold, the pressure valve 427 can open to discharge phase change fluid in the secondary vapor line 423 to the secondary condenser 425.

[0047] As the phase change fluid in the rack liquid system 405 absorbs heat generated within the servers 401 from the cooling devices 403, the phase change fluid can change its phase from liquid to vapor. Under normal operating conditions, the phase change fluid is returned through the primary vapor line 413 to the primary condenser 415 to exchange heat with the primary loop of the primary condenser 415. If the cooling capacity of the primary cooling loop is sufficient based on the current rack density, the vapor pressure in the secondary vapor line 423 remains below the trigger pressure threshold of the pressure valve 427. As a result, the pressure valve 427 is closed and the phase change fluid is not circulated through the secondary loop of the secondary condenser 425. When the secondary condenser valve 433 is closed, the primary loop 431 of the secondary condenser 425 can also be closed to save power. In one embodiment, the secondary condenser fan 429 is powered off. In one embodiment, the secondary condenser 425 is equipped with a secondary condenser fan 429 or a secondary condenser primary loop 431. If the secondary condenser is a liquid-cooled condenser that uses cooled liquid to cool the vapor in the secondary loop, the secondary condenser primary loop 431 is used to supply the cooled liquid. If the secondary condenser is an air-cooled condenser that uses a cooled airflow to cool the vapor in the secondary loop, such as data center room cooling air, the secondary condenser fan 429 is used to blow the airflow to cool the vapor back to liquid. In this case, the secondary condenser primary loop 431 is not needed.

[0048] The electrical subsystem includes a photovoltaic system 407 and a storage system 409 (also referred to as an energy storage system) to power the primary loop of the secondary condenser 425 in response to the thermal load of the rack. The photovoltaic system 407 or the storage system 409 can power the primary loop of the secondary condenser 425 through the electrical wires 455 to the secondary condenser valve 433 or the secondary condenser fan 429. The storage system 409 can be a battery-based energy storage system that includes one or more battery cells, and the necessary charging and discharging circuitry to charge and discharge the battery cells. A pressure sensor 441 that measures the vapor pressure of the secondary vapor line 423 on the discharge side of the pressure valve 427 controls the power distribution of the photovoltaic system 407 and the storage system 409. During normal operating conditions where the primary cooling loop provides sufficient cooling capacity, the pressure valve 427 is closed because the vapor pressure in the secondary vapor line 423 does not exceed the trigger pressure threshold of the pressure valve 427. A low pressure reading 443 from the pressure sensor 441 places the electrical switch in pole 1 (451) to pole 2 (452) switch position to enable the photovoltaic system 407 to charge the storage system 409. In this case, the photovoltaic system 407 does not power the secondary cooling loop.

[0049] When the cooling capacity of the primary cooling loop is insufficient to remove heat from the secondary loop of the primary condenser 415, such as when the ambient temperature is high, the vapor pressure in the secondary vapor line 423 increases. When the vapor pressure exceeds the trigger pressure threshold of the pressure valve 427, the pressure valve 427 opens to discharge the heated vapor of the secondary vapor line 423 to the secondary loop of the secondary condenser 425 and the pressure sensor 441. A high pressure reading 443 of the vapor pressure from the pressure sensor 441 can cause the electrical switch to switch to the pole 1 (451) to pole 3 (453) position to power the primary loop 431 of the secondary condenser 425. The secondary condenser valve 433 opens to supply cooling fluid to the primary loop 431 of the secondary condenser 425, thereby enabling the secondary cooling loop to augment the cooling capacity of the primary cooling loop. In one embodiment, the secondary condenser fan 429 is energized to supply an air flow to cool the secondary condenser 425.

[0050] In one aspect, the pressure reading 443 controls the amount of power to the primary loop 431 of the secondary condenser 425 to change the volumetric flow rate of the cooling liquid through the secondary condenser valve 433 or change the airflow rate from the secondary condenser fan 429 in response to the vapor pressure of the secondary vapor line 423 to adjust the cooling capacity of the secondary cooling loop. In one aspect, an electrical switch can switch to pole 2 (452) to pole 3 (453) position to use the storage system 409 to power the primary loop 431 of the secondary condenser 425 when the power from the photovoltaic system 407 is not enough. In one aspect, the storage system 409 can provide auxiliary power to the IT load as well as power to the secondary cooling loop if the increased power load of the server 401 or other IT components of the server chassis causes the pressure reading 443 to increase beyond the pressure reading 443 associated with the nominal power load. In one aspect, the storage system 409 can power the primary cooling loop or the primary loop of the primary condenser 415. The integrated thermal and power system 400 with self-regulating capability provides efficient thermal and power management, enhances cooling performance, cooling reliability, power efficiency, system scalability, sustainability requirements, and reduces cost.

[0051] Figure 5 An example of a thermal, power, and control architecture of an integrated system 500 that co-regulates cooling capacity and electrical load among multiple racks from a common photovoltaic system 407 and energy storage system 409 is shown according to one embodiment. Two racks are shown, a first rack 561 and a second rack 563. Each rack has a primary cooling loop and a secondary cooling loop that enhances the cooling capacity of the primary cooling loop. Both the primary cooling loop and the secondary cooling loop have respective condensers to exchange heat between the primary loop and the secondary loop, as described in Figure 4 For brevity, the detailed operation of the valves and fans for liquid and air cooled secondary condensers is omitted. That is, the valves for liquid cooled secondary condensers and the fans for air cooled secondary condensers are shown in Figure 5 For brevity, the detailed operation of the valves and fans for liquid and air cooled secondary condensers is omitted. That is, the valves for liquid cooled secondary condensers and the fans for air cooled secondary condensers are shown in Figure 4 For brevity, the detailed operation of the valves and fans for liquid and air cooled secondary condensers is omitted. That is, the valves for liquid cooled secondary condensers and the fans for air cooled secondary condensers are shown in

[0052] The first rack 561 has a first rack secondary condenser valve 533 to regulate the volumetric flow rate of fluid supplied to the primary loop of its secondary condenser. In one embodiment, the first rack 561 has a first rack secondary condenser fan 529 to regulate the airflow rate to the secondary condenser. Similarly, the second rack 563 has a second rack secondary condenser valve 573 to regulate the volumetric flow rate of fluid supplied to the primary loop of its secondary condenser. In one embodiment, the second rack 563 has a second rack secondary condenser fan 569 to regulate the airflow rate to its secondary condenser.

[0053] During normal operating conditions where the primary cooling loop provides sufficient cooling capacity to the first rack 561 or the second rack 563, the first rack pressure valve 527 or the second rack pressure valve 567 is closed, respectively, because the vapor pressure in the secondary loop of the secondary condenser does not exceed the trigger pressure threshold. The first rack pressure sensor 541 can measure a low pressure value on the first rack pressure reading 543 because the vapor pressure is below the trigger pressure threshold for opening the first rack pressure valve 527. The second rack pressure sensor 581 can measure a low pressure value on the second rack pressure reading 583 because the vapor pressure is below the trigger pressure threshold for opening the second rack pressure valve 567. The controller 562 can read the first rack pressure reading 543 or the second rack pressure reading 583 to place the electrical switch in the pole 1 (551) to pole 2 (552) switch position to enable the photovoltaic system 407 to charge the storage system 409. In this case, the photovoltaic system 407 does not power the secondary cooling loop of the first rack 561 or the second rack 563.

[0054] When the cooling capacity of the primary cooling loop of the first rack 561 or the second rack 563 is insufficient to remove heat from the secondary loop of the respective primary condenser, the vapor pressure in the secondary loop of the respective secondary condenser can exceed the trigger pressure threshold of the first rack pressure valve 527 or the second rack pressure valve 567, respectively. The first rack pressure valve 527 or the second rack pressure valve 567 can open to discharge heated vapor to the secondary loop of the respective secondary condenser. The first rack pressure sensor 541 or the second rack pressure sensor 581 can generate a high pressure reading.

[0055] The controller 562 can read the first rack pressure reading 543 to switch the electrical switch to the pole 1 (551) to pole 4 (554) position to power the primary loop of the secondary condenser of the first rack 561 from the photovoltaic system 407. The first rack secondary condenser valve 533 opens to supply cooling fluid to the primary loop of the secondary condenser of the first rack 561 to enable the secondary cooling loop to augment the cooling capacity of the primary cooling loop of the first rack 561. In one embodiment, the first rack secondary condenser fan 529 is energized to supply an air flow to cool the secondary condenser of the first rack 561.

[0056] Similarly, the controller 562 can read the second rack pressure reading 583 to switch the electrical switch to the pole 1 (551) to pole 3 (553) position to power the primary loop of the secondary condensers of the second rack 563 from the photovoltaic system 407. The second rack secondary condenser valve 573 opens to supply cooling fluid to the primary loop of the secondary condensers of the second rack 563, enabling the secondary cooling loop to augment the cooling capacity of the primary cooling loop of the second rack 563. In one embodiment, the second rack secondary condenser fan 569 is powered to supply air flow to cool the secondary condensers of the second rack 563.

[0057] The controller 562 controls the amount of electrical power to the primary loop of the secondary condensers of the first rack 561 or the second rack 563 to change the volumetric flow rate of cooling liquid through the first rack secondary condenser valve 533 or the second rack secondary condenser valve 573 in response to the vapor pressure measured at a designated location in the respective rack (e.g., the first rack pressure sensor 541 or the second rack pressure sensor 583) to individually adjust the cooling capacity of the secondary cooling loop of each rack. In one embodiment, the controller 562 adjusts the air flow rate from the first rack secondary condenser fan 529 or the second rack secondary condenser fan 569 in response to the vapor pressure in the respective rack, respectively. In one aspect, the controller 562 can monitor the voltage of the photovoltaic system 407 and the storage system 409 to switch the electrical switch to the pole 2 (552) to pole 4 (554) and / or the pole 2 (552) to pole 3 (553) position to power the primary loop of the secondary condensers of one or both of the first rack 562 and the second rack 563 using the storage system 409 when power from the photovoltaic system 407 is insufficient. In one aspect, the storage system 409 can provide auxiliary power to the IT load, as well as to the secondary cooling loop of the first rack 561 or the second rack 563.

[0058] Figure 6 An example of the thermal, power, and control architecture of an integrated system 600 that provides a dedicated photovoltaic system for each rack, but shares an energy storage system among multiple racks to adjust the cooling capacity and electrical load of the racks is shown according to one embodiment. Two racks, a first rack 561 and a second rack 563, are shown. The thermal architecture of the first rack 561 and the second rack 563 is similar to the thermal architecture in Figure 5 with detailed operation omitted for brevity.

[0059] The electrical architecture includes a first-rack photovoltaic system 407 dedicated to the first rack 561, a second-rack photovoltaic system 408 dedicated to the second rack 563, and a storage system 409 shared between the first rack 561 and the second rack 563. During normal operation in which the primary cooling loop provides sufficient cooling capacity to the first rack 561, the controller 661 can read a low pressure value of the first-rack pressure reading 543 from the first-rack pressure sensor 541 to generate the first-rack switch control signal 655. The first-rack switch control signal 655 can cause the electrical switches of the electrical subsystems of the first rack 561 to be in a pole 1 (651) to pole 2 (652) switch position to enable the first-rack photovoltaic system 407 to charge the storage system 409. In this case, the first-rack photovoltaic system 407 does not power the secondary cooling loop of the first rack 561.

[0060] When the cooling capacity of the primary cooling loop of the first rack 561 is insufficient to remove heat from the secondary loop of the primary condenser, the controller 661 can read a high pressure value of the first-rack pressure reading 543 to generate the first-rack switch control signal 655. The first-rack switch control signal 655 can switch the electrical switches of the electrical subsystems of the first rack 561 to a pole 1 (651) to pole 3 (653) position to power the primary loop of the secondary condenser of the first rack 561 from the first-rack photovoltaic system 407. The first-rack secondary condenser valve 533 opens to supply cooling fluid to the primary loop of the secondary condenser of the first rack 561, thereby enabling the secondary cooling loop to augment the cooling capacity of the primary cooling loop of the first rack 561. In one embodiment, the first-rack secondary condenser fan 529 is powered on to supply an air flow to cool the secondary condenser of the first rack 561.

[0061] Similarly, when the primary cooling loop provides sufficient cooling capacity to the second rack 563, the controller 661 can read a low pressure value of the second-rack pressure reading 583 from the second-rack pressure sensor 581 to generate the second-rack switch control signal 695. The second-rack switch control signal 695 can cause the electrical switches of the electrical subsystems of the second rack 563 to be in a pole 1 (691) to pole 2 (692) switch position to enable the second-rack photovoltaic system 408 to charge the storage system 409. In this case, the second-rack photovoltaic system 408 does not power the secondary cooling loop of the second rack 563.

[0062] When the cooling capacity of the primary cooling loop of the second rack 563 is insufficient to remove heat from the secondary loop of the primary condenser, the controller 661 can read the high pressure value of the second rack pressure reading 583 to generate a second rack switch control signal 695. The second rack switch control signal 695 can switch the electrical switch of the electrical subsystem of the second rack 563 to the pole 1 (691) to pole 3 (693) position to power the primary loop of the secondary condenser of the second rack 563 from the second rack photovoltaic system 408. The second rack secondary condenser valve 573 opens to supply cooling fluid to the primary loop of the secondary condenser of the second rack 563, enabling the secondary cooling loop to augment the cooling capacity of the primary cooling loop of the second rack 563. In one embodiment, the second rack secondary condenser fan 569 is powered on to supply an air flow to cool the secondary condenser of the second rack 563.

[0063] The controller 661 controls the amount of electrical power flowing to the primary loop of the secondary condenser of the first rack 561 through the first rack switch control signal 655 to change the volumetric flow rate of cooling liquid through the first rack secondary condenser valve 533 in response to the first rack pressure reading 543, thereby regulating the cooling capacity of the secondary cooling loop of the first rack 561. In one embodiment, the first rack switch control signal 655 regulates the air flow rate from the first rack secondary condenser fan 529 in response to the first rack pressure reading 543. Similarly, the controller 661 controls the amount of electrical power flowing to the primary loop of the secondary condenser of the second rack 563 through the second rack switch control signal 695 to change the volumetric flow rate of cooling liquid through the second rack secondary condenser valve 573 in response to the second rack pressure reading 583, thereby regulating the cooling capacity of the secondary cooling loop of the second rack 563. In one embodiment, the second rack switch control signal 695 regulates the air flow rate from the second rack secondary condenser fan 569 in response to the second rack pressure reading 583.

[0064] In one aspect, the controller 661 can monitor the voltage of the storage system 409 through the storage system status signal 663 and monitor the voltage of the first rack photovoltaic system 407 to generate the first rack switch control signal 655. When power from the first rack photovoltaic system 407 is insufficient, the first rack switch control signal 655 can switch the electrical switch of the electrical subsystems of the first rack 561 to the pole 2 (652) to pole 3 (653) position to power the primary loop of the secondary condenser of the first rack 561 from the storage system 409. Similarly, the controller 661 can monitor the voltage of the storage system 409 through the storage system status signal 663 and monitor the voltage of the second rack photovoltaic system 408 to generate the second rack switch control signal 695. When power from the second rack photovoltaic system 408 is insufficient, the second rack switch control signal 695 can switch the electrical switch of the electrical subsystems of the second rack 563 to the pole 2 (692) to pole 3 (693) position to power the primary loop of the secondary condenser of the second rack 563 from the storage system 409. In one aspect, the storage system 409 can provide auxiliary power to the IT loads, as well as power the secondary cooling loop of the first rack 561 or the second rack 563.

[0065] Figure 7 An example of a thermal, power, and control architecture of an integrated system 700 that co-locates photovoltaic systems 407 and energy storage systems 409 to regulate cooling capacity and electrical loads among multiple racks, in which the racks also co-locate secondary condensers 725 and fluid storage systems 726 of secondary cooling loops, is shown in accordance with one embodiment. Each rack can have a dedicated primary cooling loop (not shown) with a corresponding condenser to perform heat exchange between the primary and secondary loops. As in the other configurations of Figures 4-6 In other configurations, the cooling capacity of the primary cooling loop is fixed for each rack. However, unlike the other architectures, the secondary cooling loop is co-located among the racks to augment the cooling capacity of each primary cooling loop.

[0066] For the secondary cooling loop, the controller 761 can control the secondary condenser primary loop valve 733 to supply cooling liquid to the primary loop of the secondary condenser 725. In one embodiment, the controller 761 can control the speed of the secondary condenser fan 729 to vary the airflow rate to the secondary condenser 725. After heat exchange with the primary loop, the cooling liquid in the secondary loop of the secondary condenser 725 returns to the fluid storage system 726. The fluid storage system 726 acts as a system-level liquid buffer for the phase change fluid in the secondary loop of the secondary condenser 725. The secondary liquid supply line 721 supplies phase change fluid from the fluid storage system 726 to each rack through the respective secondary condenser secondary loop valve 722. The secondary vapor return line 723 returns heated vapor from each rack to the secondary condenser 725 to complete the secondary loop of the secondary condenser 725.

[0067] Each secondary vapor return line 723 is connected to the secondary condenser 725 through a pressure valve 727. The pressure valve 727 is closed under normal operation when the vapor pressure is low. When the vapor pressure reaches a trigger pressure threshold, the pressure valve 727 can open to discharge heated vapor in the secondary vapor return line 725 from the rack to the secondary condenser 725. A pressure sensor 741 measures the vapor pressure of each secondary vapor return line 723 on the discharge side of the pressure valve 727. The controller 761 can read the pressure reading 743 from each pressure sensor 741 to generate a secondary condenser secondary loop control signal 757. The secondary condenser secondary loop control signal 757 controls the secondary condenser secondary loop valve 722 to adjust the volumetric flow rate of phase change fluid supplied to the secondary liquid supply line 721 for the rack in response to the vapor pressure of the respective secondary vapor return line 723 for the rack. As shown, each individual secondary liquid supply line 721 between the fluid storage system 726 and the rack is assembled with the secondary condenser secondary loop valve 722 since there is a shared fluid storage system 726 between the racks. The secondary condenser 725 is then connected to the fluid storage system 726. Each primary condenser dedicated to each rack can be connected directly to the rack liquid cooling distribution unit. The individual control of the cooling capacity of the secondary cooling loop for each rack from the shared secondary condenser 725 improves cooling efficiency and performance while reducing cost.

[0068] During normal operating conditions where the primary cooling loop at each rack provides sufficient cooling capacity, the pressure valve 727 for all racks can be closed. The controller 761 can read low pressure values on the pressure reading 743 from all racks to place the electrical switch in the pole 1 (751) to pole 3 (753) switch position to enable the photovoltaic system 407 to charge the energy storage system 409. In this case, the photovoltaic system 407 does not power the secondary cooling loop for all racks.

[0069] When the cooling capacity of the primary cooling loop for the rack is insufficient to remove heat from the secondary loop of the corresponding primary condenser, the vapor pressure in the secondary vapor return line 723 for the rack can exceed the trigger pressure threshold of the corresponding pressure valve 727. The pressure valve 727 can open to discharge heated vapor to the secondary condenser 725. The pressure sensor 741 for the rack can produce a high pressure reading. The controller 761 can read the pressure reading 743 of the rack to switch the electrical switch to the pole 1 (751) to pole 2 (752) position to power the primary loop of the secondary condenser 725. The secondary condenser primary loop valve 733 opens to supply cooling liquid to the primary loop of the secondary condenser 725. As discussed, the controller 761 can also open the secondary condenser secondary loop valve 722 to regulate the volumetric flow rate of phase change fluid supplied to the rack in response to the pressure reading 743 to enable the secondary cooling loop to augment the cooling capacity of the primary cooling loop for the rack. In one embodiment, the first rack secondary condenser fan 729 is powered on to supply an air flow to cool the secondary condenser 725.

[0070] In one aspect, the controller 761 can monitor the voltage of the photovoltaic system 407 and the energy storage system 409 to switch the electrical switch to the pole 3 (753) and pole 2 (752) positions to power the primary loop of the secondary condenser 725 using the energy storage system 409 when power from the photovoltaic system 407 is insufficient. In one aspect, the energy storage system 409 can be powered by the primary utility power. In one aspect, the energy storage system 409 can provide auxiliary power to the IT load as well as power the secondary cooling loop of the rack. In one aspect, the energy storage system 409 can power the primary cooling loop of the rack. The photovoltaic system 407 and the energy storage system 409 act as system-level electrical buffers for the electrical subsystems.

[0071] Figure 8 is a flowchart illustrating an example of a method 800 for regulating cooling capacity and electrical load of electronic equipment in a data center or computer system in response to heat and power density according to one embodiment. In one embodiment, the method 800 can be performed by the integrated thermal and electrical system 400, 500, 600, 700 of Figure 4 、 Figure 5 、 Figure 6 or Figure 7 In one aspect, the method 800 can be performed using hardware logic or a combination of hardware logic and programmable registers storing configuration values.

[0072] In operation 801, the method 800 connects a primary condenser of a primary cooling loop that circulates a cooling fluid to a thermal load of an electronic device. In one aspect, the cooling liquid can be a phase change fluid. Operation 801 can connect a liquid supply line and a vapor return line of a secondary loop of the primary condenser to remove heat from the thermal load.

[0073] In operation 803, the method 800 connects a renewable energy source to charge an energy storage system. In one aspect, the renewable energy source can be a photovoltaic system and the energy storage system can be a rechargeable battery.

[0074] In operation 805, the method 800 monitors a vapor pressure of the cooling liquid, such as a vapor pressure of the phase change fluid at a discharge side of a pressure-based valve. The vapor pressure can increase when the cooling capacity of the primary cooling loop is insufficient for the thermal load or when there is a high power load associated with the electronic device.

[0075] In operation 807, the method 800 connects a secondary condenser of a secondary cooling loop to the thermal load when the vapor pressure exceeds a threshold pressure value. In one aspect, the pressure valve can be triggered to open when the vapor pressure exceeds the threshold pressure value. When the pressure valve opens, a secondary loop of the secondary condenser can be connected to the thermal load and a primary loop of the secondary condenser can be connected to the liquid supply line. The cooling capacity of the secondary cooling loop supplements the cooling capacity of the primary cooling loop.

[0076] In operation 809, the method 800 controls the distribution of power from the renewable energy source and the energy storage system based on the vapor pressure to regulate the secondary cooling loop. In one aspect, the renewable energy source can power the secondary cooling loop to control the cooling capacity of the secondary cooling loop in response to the vapor pressure if the power from the renewable energy source is sufficient. In one aspect, the energy storage system can power the secondary cooling loop to control the cooling capacity of the secondary cooling loop in response to the vapor pressure if the power from the renewable energy source is insufficient.

[0077] The various configurations, layouts, and components of the integrated thermal and electrical system with self-regulating capabilities as described provide enhanced cooling capacity and auxiliary power to support high thermal and power density requirements of a computer system or data center. The integrated design of the thermal system for the electrical subsystem, the photovoltaic system, and the energy storage system of the primary and secondary cooling loops and the self-regulating control of the thermal and storage systems provide efficient thermal and power management of high performance IT clusters with improved cooling performance, cooling reliability, power efficiency, system scalability, sustainability requirements, and reduced cost to meet high thermal and power density requirements.

[0078] In the foregoing specification, embodiments of the application have been described with reference to specific exemplary embodiments thereof. It will be evident that various modifications can be made thereto without departing from the broader spirit and scope of the application as set forth in the following claims. For example, the primary cooling loop and the secondary cooling loop can monitor a fluid temperature of the cooling fluid to regulate the cooling capacity of the secondary cooling loop instead of monitoring a vapor pressure of the phase change fluid. The renewable energy source can be another renewable energy source powered by thermal energy from the cooling loop, wind, biomass energy, etc. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

Claims

1. A liquid cooling apparatus for a data center, comprising: a primary cooling loop having a primary condenser to remove heat from one or more information technology components using a cooling liquid; a secondary cooling loop having a secondary condenser, the cooling liquid of the primary cooling loop combining with the cooling liquid of the secondary cooling loop to form a rack liquid system, the rack liquid system returning a phase change fluid of the cooling liquid to the primary condenser through a primary vapor line or to the secondary condenser through a secondary vapor line; wherein the secondary vapor line is connected to the secondary condenser through a pressure valve; a pressure sensor to monitor a vapor pressure of the cooling liquid of the secondary vapor line on a discharge side of the pressure valve; and an electrical subsystem including a renewable energy source and an energy storage system, the electrical subsystem configured to distribute power to connect the secondary condenser to a main loop of the secondary cooling loop in response to the vapor pressure and to control a cooling capacity of the secondary cooling loop; wherein, when the vapor pressure exceeds a threshold, the secondary condenser is connected to close the secondary cooling loop, supplementing the primary cooling loop to remove heat from the information technology components; when the vapor pressure does not exceed the threshold, the secondary condenser is disconnected from the secondary cooling loop, and wherein the renewable energy source is configured to charge the energy storage system when the secondary condenser is disconnected.

2. The liquid cooling device of claim 1, wherein, the renewable energy source is configured to distribute power to connect the secondary condenser to the secondary cooling loop in response to the vapor pressure and to control the cooling capacity of the secondary cooling loop when power of the renewable energy source is sufficient, and the energy storage system is configured to distribute power to connect the secondary condenser to the secondary cooling loop in response to the vapor pressure and to control the cooling capacity of the secondary cooling loop when the power of the renewable energy source is insufficient.

3. The liquid cooling apparatus of claim 1, further comprising a secondary condenser valve configured to supply the cooling liquid to the secondary condenser, and wherein, the electrical subsystem is configured to open the secondary condenser valve when the secondary condenser is connected to the secondary cooling loop to allow the cooling liquid to flow into the secondary condenser.

4. The liquid cooling device of claim 3, wherein, the electrical subsystem is configured to control the secondary condenser valve to adjust a volumetric flow rate of the cooling liquid into the secondary condenser in response to the vapor pressure, thereby controlling the cooling capacity of the secondary cooling loop.

5. The liquid cooling apparatus of claim 1, further comprising a secondary condenser fan configured to supply airflow to the secondary condenser, and wherein, the electrical subsystem is configured to open the secondary condenser fan when the secondary condenser is connected to the secondary cooling loop.

6. The liquid cooling device of claim 5, wherein, the electrical subsystem is configured to control the secondary condenser fan to adjust a rate of the airflow in response to the vapor pressure, thereby controlling the cooling capacity of the secondary cooling loop.

7. A data center, comprising: an electrical subsystem including at least one renewable energy source and at least one energy storage system; a plurality of electronic racks, each electronic rack containing a plurality of server enclosures and each server enclosure corresponding to one or more servers, wherein each electronic rack includes: a primary cooling loop having a primary condenser to remove heat from the servers using a cooling liquid; a secondary cooling loop having a secondary condenser, the cooling liquid of the primary cooling loop combining with the cooling liquid of the secondary cooling loop to form a rack liquid system, the rack liquid system returning a phase change fluid of the cooling liquid to the primary condenser through a primary vapor line or to the secondary condenser through a secondary vapor line; wherein the secondary vapor line is connected to the secondary condenser through a pressure valve; a pressure sensor to monitor a vapor pressure of the cooling liquid of the secondary vapor line on a discharge side of the pressure valve; and an electrical subsystem including a renewable energy source and an energy storage system, the electrical subsystem configured to distribute power to connect the secondary condenser to a main loop of the secondary cooling loop in response to the vapor pressure and to control a cooling capacity of the secondary cooling loop; wherein, when the vapor pressure exceeds a threshold, the secondary condenser is connected to close the secondary cooling loop, supplementing the primary cooling loop to remove heat from the information technology components; when the vapor pressure does not exceed the threshold, the secondary condenser is disconnected from the secondary cooling loop, and wherein the renewable energy source is configured to charge the energy storage system when the secondary condenser is disconnected. the renewable energy source is configured to distribute power to connect the secondary condenser to the secondary cooling loop in response to the vapor pressure and to control the cooling capacity of the secondary cooling loop when power of the renewable energy source is sufficient, and the energy storage system is configured to distribute power to connect the secondary condenser to the secondary cooling loop in response to the vapor pressure and to control the cooling capacity of the secondary cooling loop when the power of the renewable energy source is insufficient. the electrical subsystem is configured to open the secondary condenser valve when the secondary condenser is connected to the secondary cooling loop to allow the cooling liquid to flow into the secondary condenser. the electrical subsystem is configured to control the secondary condenser valve to adjust a volumetric flow rate of the cooling liquid into the secondary condenser in response to the vapor pressure, thereby controlling the cooling capacity of the secondary cooling loop. the electrical subsystem is configured to open the secondary condenser fan when the secondary condenser is connected to the secondary cooling loop. the electrical subsystem is configured to control the secondary condenser fan to adjust a rate of the airflow in response to the vapor pressure, thereby controlling the cooling capacity of the secondary cooling loop. a secondary cooling loop having a secondary condenser, the cooling liquid of the primary cooling loop combining with the cooling liquid of the secondary cooling loop to form a rack liquid system, the rack liquid system returning the phase change fluid of the cooling liquid to the primary condenser through a primary vapor line or to the secondary condenser through a secondary vapor line; wherein the secondary vapor line is connected to the secondary condenser through a pressure valve; a pressure sensor for monitoring the vapor pressure of the cooling liquid of the secondary vapor line on the discharge side of the pressure valve; wherein the electrical subsystem is configured to distribute power to connect the secondary condenser to a primary loop of the secondary cooling loop in response to the vapor pressure and to control the cooling capacity of the secondary cooling loop; and wherein the secondary condenser is connected to close the secondary cooling loop, supplementing the primary cooling loop to remove heat from the server, when the vapor pressure exceeds a threshold, and wherein the secondary condenser is disconnected from the secondary cooling loop when the vapor pressure does not exceed the threshold, and wherein the renewable energy source for the electronic rack is configured to charge the energy storage system for the electronic rack when the secondary condenser is disconnected.

8. The data center of claim 7, wherein, Each electronic rack further comprises one of the at least one renewable energy source and one of the at least one energy storage system.

9. The data center of claim 7, wherein, Each electronic rack further comprises one of the at least one renewable energy source and one of the at least one energy storage system, and wherein the renewable energy source for the electronic rack is configured to distribute power to connect the secondary condenser to the secondary cooling loop in response to the vapor pressure and to control the cooling capacity of the secondary cooling loop when power of the renewable energy source is sufficient, and the energy storage system for the electronic rack is configured to distribute power to connect the secondary condenser to the secondary cooling loop in response to the vapor pressure and to control the cooling capacity of the secondary cooling loop when the power of the renewable energy source is insufficient.

10. The data center of claim 7, wherein, Each electronic rack further comprises a secondary condenser valve configured to supply the cooling liquid to the secondary condenser, and wherein the electrical subsystem is configured to open the secondary condenser valve to allow the cooling liquid to flow into the secondary condenser when the secondary condenser is connected to the secondary cooling loop.

11. The data center of claim 10, wherein, The electrical subsystem is configured to control the secondary condenser valve to regulate the volumetric flow rate of the cooling liquid into the secondary condenser in response to the vapor pressure to control the cooling capacity of the secondary cooling loop.

12. The data center of claim 7, wherein, Each electronic rack further comprises a secondary condenser fan configured to supply an airflow to the secondary condenser, and wherein the electrical subsystem is configured to control the secondary condenser fan to regulate the rate of the airflow in response to the vapor pressure to control the cooling capacity of the secondary cooling loop.

13. The data center of claim 7, wherein, The flow rate of the cooling liquid in the primary condenser or the cooling gas flow rate through the primary condenser is constant to provide a constant cooling capacity for the primary cooling loop.

14. The data center of claim 7, wherein, The at least one renewable energy source and the at least one energy storage system are configured to be shared among the plurality of electronic racks to connect the secondary condensers to the secondary cooling loops, respectively, and to control the cooling capacity of the secondary cooling loop of each electronic rack, respectively, in response to the vapor pressure monitored by the pressure sensor.

15. The data center of claim 7, wherein, Each electronic rack further includes one of the at least one renewable energy source, wherein the at least one energy storage system is shared among the plurality of electronic racks, and wherein the electrical subsystem is configured to distribute power from the at least one renewable energy source for the electronic rack or the shared at least one energy storage system to connect the secondary condensers to the secondary cooling loops, respectively, and to control the cooling capacity of the secondary cooling loop of each electronic rack, respectively, in response to the vapor pressure monitored by the pressure sensor.

16. The data center of claim 7, wherein, Each electronic rack further includes a valve, wherein the plurality of electronic racks share the secondary condensers, and wherein the electrical subsystem is configured to distribute power to the valve for each electronic rack to control the cooling capacity of the secondary cooling loop of each electronic rack, respectively, in response to the vapor pressure monitored by the pressure sensor.

17. A method of regulating thermal and electrical loads of an electronic device, comprising: connecting a primary condenser of a primary cooling loop to circulate a cooling liquid to remove heat from the electronic device; connecting a renewable energy source to charge an energy storage system; monitoring a vapor pressure of the cooling liquid; connecting a secondary condenser to a primary loop of a secondary cooling loop to supplement the primary cooling loop in removing heat from the electronic device when the vapor pressure of the cooling liquid exceeds a threshold; and distributing power from the renewable energy source and the energy storage system to regulate a cooling capacity of the secondary cooling loop based on the vapor pressure; wherein the cooling liquid of the primary cooling loop combines with the cooling liquid of the secondary cooling loop to form a rack liquid system that returns a phase change fluid of the cooling liquid to the primary condenser through a primary vapor line or to the secondary condenser through a secondary vapor line; the monitoring the vapor pressure of the cooling liquid includes monitoring a vapor pressure of the cooling liquid of the secondary vapor line; the method further comprises: disconnecting the secondary condenser from the secondary cooling loop when the vapor pressure does not exceed the threshold; and charging the energy storage system from the renewable energy source when the secondary condenser is disconnected.

18. The method of claim 17, wherein, the distributing power from the renewable energy source and the energy storage system to regulate the cooling capacity of the secondary cooling loop includes: controlling a volumetric flow rate of the cooling liquid into the secondary condenser based on the vapor pressure to regulate the cooling capacity of the secondary cooling loop; or controlling a flow rate of the cooling gas into the secondary condenser based on the vapor pressure to regulate the cooling capacity of the secondary cooling loop. Controlling a flow rate of gas into the secondary condenser based on the vapor pressure to regulate the cooling capacity of the secondary cooling loop.

Images