Rack architecture for multiple hybrid thermal systems

Abstract

An electronic rack cooling system is disclosed, including a two-phase cooling system having a first cooling circuit and a single-phase system including a second cooling circuit. A main coolant source, such as a facility cooling fluid, is coupled to a condenser unit of the two-phase cooling system. A branch of the facility cooling fluid is directed to the single-phase cooling circuit. The coolant flow to the single-phase cooling circuit is controlled by a flow control valve and a coolant pump. The facility cooling fluid is managed between a single-phase cycle and a phase change cycle. A rack management unit in the electronic rack controls the flow rate of the facility cooling fluid using a flow control device and a pump.

Description

Technical Field

[0001] Embodiments of the present invention generally relate to cooling of data centers. More specifically, embodiments of the present invention relate to a multi-phase, multi-cooling system architecture for cooling servers in a rack, cluster, or data center. Background Art

[0002] Cooling is a prominent element in the design of computer systems and data centers. The number of high-performance electronic components, such as high-performance processors, packaged within servers has steadily increased, increasing the amount of heat generated and dissipated during normal operation of the servers. If the environment in which the servers used within a data center operate is allowed to increase in temperature over time, the reliability of the servers will decrease. Maintaining a proper thermal environment is critical to the proper operation of these servers in a data center, as well as the performance and life of the servers. More effective and efficient cooling solutions are needed, especially in the case of cooling high-performance servers.

[0003] Server racks in a data center may contain servers with different types of electronic components that generate heat that needs to be removed by a cooling system. Different electronic components may have substantially different heat loads. Typically, a single-phase cooling system is used to remove heat generated within a server system or a rack of server systems. A single-phase cooling system is a type of cooling system in which the coolant or working fluid remains in a liquid state. Rack-level coolant distribution for heat removal is a single point of failure.

[0004] In the prior art, rack-level cooling is performed by a single-phase system that pumps coolant to the server rack and transfers heat to the coolant. The coolant then returns to the cooling system to remove heat from the coolant. The speed of the pump that pumps coolant to the server rack may need to vary to remove different amounts of heat generated by components within the server rack. As the heat load increases, the speed of the pump may not be able to be increased further. Further, increasing the speed of the pump beyond a certain design point becomes an inefficient way to remove heat. In addition, the pump may fail, resulting in the single-phase system being unable to remove heat from the heat-generating components. Therefore, whether it is an operational failure of the pump or the pump is unable to operate quickly enough to remove the generated heat, the single-phase system may include a single point of failure in cooling the heat-generating components within the server rack in the existing cooling system. Summary of the invention

[0005] The present invention aims to solve one of the above-mentioned technical problems in the related art.

[0006] An embodiment of the present invention provides an electronic rack cooling system, which is configured to cool heat-generating components in multiple servers in the electronic rack, and the cooling system includes: a condenser of a two-phase cooling system connected to a facility cooling fluid supply, the condenser being configured to cool steam returned from a two-phase cooling circuit connected between the two-phase cooling system and a two-phase coolant distribution manifold in the electronic rack; a single-phase cooling circuit, the single-phase cooling circuit being connected to the facility cooling fluid supply via a flow control system, the single-phase cooling circuit being connected to the single-phase coolant distribution manifold in the electronic rack; and a controller, the controller being used to control the flow rate of the single-phase cooling fluid from the flow control system to the single-phase cooling circuit in response to a temperature determined by the controller.

[0007] In some embodiments, the flow control system includes: a two-way control valve; a coolant pump; or a two-way control valve and a coolant pump.

[0008] In some embodiments, the two-phase cooling system is integrated with the electronics rack.

[0009] In some embodiments, the controller controls the flow control system to prevent single-phase cooling fluid from flowing to the single-phase cooling circuit in response to the temperature being below a predetermined threshold.

[0010] In some embodiments, the controller controls the flow control system to adjust an amount of the single-phase cooling fluid delivered to the single-phase cooling circuit in response to changes in the temperature.

[0011] In some embodiments, the single-phase coolant loop is coupled to the plurality of servers within the electronics rack via a single-phase distribution manifold in the electronics rack, and the two-phase coolant loop is coupled to the plurality of servers via a two-phase distribution manifold in the electronics rack.

[0012] In some embodiments, the single-phase cooling loop and the two-phase cooling loop are both coupled to a same heat generating component within a server in the plurality of servers.

[0013] In some embodiments, the facility cooling fluid supply is controlled to cool the electronics racks directly, or to cool the electronics racks with a combination of direct cooling using the single-phase cooling loop and indirect cooling using the condenser of the two-phase cooling system and the two-phase cooling loop.

[0014] An embodiment of the present invention also provides an electronic rack cooling system, which is configured to cool heat-generating components in multiple servers in an electronic rack, and the cooling system includes: a coolant pump connected to a facility cooling system fluid supply; a condenser of a two-phase cooling system connected to an output end of the coolant pump, the condenser being configured to cool steam returned from a two-phase cooling loop connected between the two-phase cooling system and a two-phase coolant distribution manifold in the electronic rack; a flow control device connected to the output end of the coolant pump; a single-phase cooling loop, the single-phase cooling loop being connected to the facility cooling fluid supply via the flow control device, the single-phase cooling loop being connected to the single-phase coolant distribution manifold in the electronic rack; and a controller, the controller being configured to control the flow rate of the single-phase cooling fluid from the flow control device to the single-phase cooling loop in response to a temperature determined by the controller.

[0015] In some embodiments, the flow control device comprises a two-way control valve.

[0016] In some embodiments, the two-phase cooling system, the coolant pump, and the flow control device are integrated with the electronics rack.

[0017] In some embodiments, the controller controls the flow control device to prevent the single-phase cooling fluid from flowing to the single-phase cooling circuit in response to the temperature being below a predetermined threshold.

[0018] In some embodiments, the controller controls the control system to adjust an amount of the single-phase cooling fluid delivered to the single-phase cooling circuit in response to a change in the temperature.

[0019] In some embodiments, the flow rate of the coolant pump is set to a preset value.

[0020] In some embodiments, the controller controls the speed of the coolant pump to vary the flow rate of the coolant pump.

[0021] In some embodiments, the single-phase coolant loop is coupled to the plurality of servers within the electronics rack via a single-phase distribution manifold in the electronics rack, and the two-phase coolant loop is coupled to the plurality of servers via a two-phase distribution manifold in the electronics rack.

[0022] In some embodiments, the single-phase cooling loop and the two-phase cooling loop are both coupled to a same heat generating component within a server of the plurality of servers.

[0023] An embodiment of the present invention also provides a data center, comprising: a facility cooling fluid supply; a plurality of electronic racks, each electronic rack comprising a plurality of heat-generating components; and an indirect cooling system for each electronic rack, the indirect cooling system comprising a condenser connected to the facility cooling fluid supply, wherein the indirect cooling system comprises a two-phase cooling loop connected between the electronic rack and the condenser, the two-phase cooling loop recirculating the two-phase cooling fluid to the electronic rack and returning it to the indirect cooling system.

[0024] In some embodiments, for each electronic rack, the data center further includes: a direct single-phase cooling loop, which is connected to the facility cooling fluid supply via a cooling fluid management component and to the electronic rack, wherein the single-phase cooling loop recirculates the single-phase cooling fluid to the electronic rack and returns to the facility cooling fluid return, and the cooling fluid management component includes a coolant pump and a flow control device to regulate the flow of the single-phase cooling fluid within the single-phase cooling loop.

[0025] In some embodiments, the data center further includes, for each electronic rack, a rack controller that controls each cooling fluid management component to adjust a flow rate of the single-phase cooling fluid within the single-phase cooling loop. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Embodiments of the invention are illustrated by way of example and not by way of limitation in the accompanying figures and in which like references indicate similar elements.

[0027] Figure 1A is a block diagram illustrating an example of a rack architecture for a multiple hybrid thermal system according to one implementation.

[0028] Figure 1B is a block diagram illustrating an example of a rack architecture for a multiple hybrid thermal system according to one implementation.

[0029] Figure 1C is a block diagram illustrating an example of a data center architecture for a multiple hybrid thermal system according to one embodiment.

[0030] Figure 2 is a block diagram illustrating an example of a side view of an electronics rack according to one embodiment.

[0031] Figure 3 is a block diagram illustrating an example of a top view of an electronics rack according to one embodiment.

[0032] Figure 4 is a block diagram illustrating an example of a cold plate configuration according to one embodiment. DETAILED DESCRIPTION

[0033] Various embodiments and aspects of the present invention will be described with reference to the details discussed below, and the accompanying drawings will illustrate various embodiments. The following description and drawings are illustrative of the present invention and should not be construed as limiting the present invention. Many specific details are described to provide a thorough understanding of various embodiments of the present invention. However, in some cases, in order to concisely discuss embodiments of the present invention, known or conventional details are not described.

[0034] References to "one embodiment" or "an embodiment" in the specification mean that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The phrase "in one embodiment" appearing in different places in the specification does not necessarily refer to the same embodiment.

[0035] In a first aspect, an electronics rack cooling system is configured to cool heat generating components in a plurality of servers in an electronics rack. The cooling system includes a condenser of a two-phase cooling system, the condenser being coupled to a facility cooling fluid supply. The condenser is configured to continuously cool steam returned from a two-phase cooling loop coupled between the two-phase cooling system and a two-phase coolant distribution manifold within the electronics rack. The cooling system also includes a single-phase cooling loop, the single-phase cooling loop being coupled to a facility cooling fluid supply via a flow control system. The single-phase cooling loop is coupled to a single-phase coolant distribution manifold within the electronics rack. The cooling system also includes a controller to control a flow rate of the single-phase cooling fluid from the flow control system to the single-phase cooling loop in response to a temperature determined by the controller. The flow control system may include a two-way control valve, a coolant pump, or a two-way control valve and a coolant pump. The two-phase cooling system may be integrated with the electronics rack. The controller may control the flow control system in response to a temperature below a predetermined threshold to prevent the single-phase cooling fluid from flowing to the single-phase cooling loop. The controller may also control the flow control system to adjust the amount of the single-phase cooling fluid delivered to the single-phase cooling loop in response to a change in temperature. In one embodiment, the single-phase coolant loop is coupled to a plurality of servers within the electronics rack via a single-phase distribution manifold in the electronics rack, and the two-phase coolant loop is coupled to the plurality of servers via a two-phase distribution manifold in the electronics rack. In one embodiment, both the single-phase cooling loop and the two-phase cooling loop are coupled to the same heat generating component within one of the plurality of servers. The controller may be integrated into the electronics rack.

[0036] In a second aspect, an electronics rack cooling system may be configured to cool heat generating components in a plurality of servers in an electronics rack. The cooling system may include a coolant pump coupled to a facility cooling system fluid supply. A condenser of the two-phase cooling system may be coupled to an output of the coolant pump. The condenser is configured to continuously cool steam returned from a two-phase cooling loop coupled between the two-phase cooling system and a two-phase coolant distribution manifold within the electronics rack. The cooling system may also include a flow control device coupled to an output of the coolant pump. The single-phase cooling loop may be coupled to a facility cooling fluid supply via the flow control device, and the single-phase cooling loop is coupled to a single-phase coolant distribution manifold within the electronics rack. The cooling system may also include a controller configured to control a flow rate of the single-phase cooling fluid from the flow control device to the single-phase cooling loop in response to a temperature determined by the controller. In one embodiment, the flow control device may be a two-way control valve. The two-phase cooling system, the coolant pump, and the flow control device may be integrated with the electronics rack. In one embodiment, the controller may control the flow control device in response to a temperature below a predetermined threshold to prevent the single-phase cooling fluid from flowing to the single-phase cooling loop. In an embodiment, the control system can adjust the amount of single-phase cooling fluid delivered to the single-phase cooling loop in response to changes in temperature. In an embodiment, the flow rate of the coolant pump can be set to a preset value. The single-phase coolant loop can be connected to multiple servers in the electronics rack via a single-phase distribution manifold in the electronics rack. The two-phase coolant loop is connected to the multiple servers via a two-phase distribution manifold in the electronics rack. In an embodiment, the single-phase cooling loop and the two-phase cooling loop can both be connected to the same heat generating component in one of the multiple servers.

[0037] In a third aspect, a server cluster cooling system may include a plurality of electronic racks, each of which has an electronic rack cooling system as described in the first aspect. Each electronic rack may include one or more servers, and the servers in the plurality of electronic racks may be configured to operate as a server cluster.

[0038] In a fourth aspect, a server cluster cooling system may include multiple electronic racks, each of which has an electronic rack cooling system as described in the second aspect. Each electronic rack may include one or more servers, and the servers in the multiple racks may be configured to operate as a server cluster.

[0039] In a fifth aspect, a data center has a plurality of electronic racks, each of which includes a plurality of heat generating components cooled by a cooling system. Each electronic rack includes an indirect cooling system having a condenser coupled to a facility cooling fluid supply. The indirect cooling system has a two-phase cooling loop coupled to the condenser and the electronic rack. The two-phase cooling loop recirculates the two-phase cooling fluid to the electronic rack and back to the condenser of the indirect cooling system. In one embodiment, each electronic rack in the data center further includes a direct single-phase cooling loop coupled to the facility cooling fluid supply via a cooling fluid management component and coupled to the electronic rack. The single-phase cooling loop recirculates the single-phase cooling fluid to the electronic rack and back to the facility cooling fluid return. The cooling fluid management component includes a flow control device and a coolant pump that regulates the flow of the single-phase cooling fluid within the single-phase cooling loop. In one embodiment, each electronic rack in the data center further includes a rack controller that controls each cooling fluid management component to regulate the flow of the single-phase cooling fluid within the single-phase cooling loop.

[0040] Figure 1A 1 is a block diagram illustrating an example of a cooling system 100 for an electronic rack 200 according to one embodiment. The cooling system 100 for the electronic rack 200 includes a two-phase cooling system ("condenser system") 121 and a single-phase cooling loop 132 / 131. A facility cooling fluid supply 137 and a facility fluid cooling return 136 flow continuously through the condenser system 121. The facility cooling fluid supply 137 is a single-phase cooling fluid that is selected so that the single-phase cooling fluid does not evaporate at the design temperature of the cooling system 100. The flow rate of the single-phase cooling loop 132 / 131 is controlled by a flow control device 140 and / or a coolant pump 146. The flow control device 140 receives the single-phase cooling fluid supply from the facility cooling fluid supply 137.

[0041] The two-phase cooling system 121 has a two-phase cooling circuit 134 / 133, which includes a two-phase coolant supply 134 and a vapor return 133, which flow to and flow out of the electronic rack 200, respectively. The two-phase cooling circuit 134 / 133 flows continuously to and from the condenser system 121 and the electronic rack 200. The single-phase cooling circuit 132 / 131 includes a single-phase cooling liquid supply 132 and a single-phase cooling fluid return 131, which flow to and flow out of the electronic rack 200, respectively, at flow rates determined by the flow control device 140 and the coolant pump 146. In one embodiment, the condenser system 121 may be integrated into or onto the electronic rack 200. In one embodiment, the flow control device 140 and the coolant pump 146 may also be integrated into or onto the electronic rack 200 .

[0042] The two-phase coolant supply 134 of the condenser system 121 is coupled to a two-phase coolant supply distribution manifold 144 installed in the electronic rack 200. The vapor return 133 is coupled to a vapor return manifold 143 installed in the electronic rack 200. The single-phase cooling fluid supply 132 is coupled to a single-phase cooling supply liquid distribution manifold 142 installed in the electronic rack 200. The single-phase cooling fluid return 131 is coupled to a single-phase cooling return liquid manifold 141 installed in the electronic rack 200.

[0043] The electronic rack 200 includes a rack management unit (RMU) 202 and one or more servers, for example, servers 203A-203D (collectively referred to as "servers 203" unless otherwise specified). The RMU 202 includes a rack management controller (RMC) 222, which is within the electronic rack 200 and performs control functions for the condenser system 121, the flow control device 140, and the coolant pump 146. The flow rate of the flow control device 140 can be set by the rack management controller (RMC) 222 in the rack management unit (RMU) 202. The flow rate of the single-phase cooling fluid in the single-phase cooling loop 132 / 131 can be set in response to the temperature determined by the RMC 222. Referring to Figure 2 The RMU 202 is described in detail.

[0044] Each server 203 has connections on the server 203 chassis to connect to each of the single-phase cooling fluid supply distribution manifold 142, the single-phase cooling fluid return manifold 141, the two-phase coolant supply distribution manifold 144, and the steam return manifold 143 via connection points 150. Inside each server 203, one or more cold plates (not shown) are connected to the server 203 chassis coolant connection points (not shown) for the single-phase cooling fluid supply 132, the single-phase cooling fluid return 131, the two-phase coolant supply 134, and the steam return 133. Inside each server 203, one or more heat-generating components (not shown) are connected to the cold plate (not shown). The heat-generating components may include general-purpose processors, graphics processors, data processing or artificial intelligence processors, memory, communication devices, etc. Some of these heat-generating components are connected to one or more cold plates or other heat removal devices in each server 203. The cold plate removes heat from one or more heat-generating components using a two-phase cooling loop and a single-phase cooling loop. In one embodiment, air flow, thermoelectric (TE) coolers, or other devices may additionally be used to remove heat from the heat generating components.

[0045] A two-phase cooling fluid 134 is continuously provided to the electronics racks 200 by the condenser system 121. Heat from the heat generating components is returned to the condenser system 121 as, for example, steam 133 returned by the two-phase cooling loop. The steam 133 returned to the condenser system 121 is cooled at the condenser system 121 using a facility cooling fluid supply 137 that continuously flows to the condenser system 121. Heat is transferred by the condenser system 121 to a facility cooling fluid return 136. The facility cooling fluid return 136 is returned to a facility heat removal system, such as a cooling tower, countercurrent heat exchanger, chiller, cooling pond, or other facility-scale cooling system (not shown). The facility cooling fluid return 136, cooled by the facility heat removal system, is recycled to the condenser system 121 as a facility cooling fluid source 137.

[0046] In one embodiment, the flow control device 140 can be a flow control valve, such as a controllable gate valve, a plug valve, a needle valve, or other valve body type. The coolant pump 146 can be a centrifugal pump, a constant volume pump, or other type of coolant pump 146. One or more control signals 147 are generated by the RMC 222 to set the flow rate of the flow control device 140 and the flow rate of the coolant pump 146. Figure 2The RMC 222 is described. The control signal 147 for controlling the flow rate of the flow control device 140 and the flow rate of the coolant pump 146 can be generated directly from a temperature sensor in the server 203, such as a temperature sensor (not shown) in a chip box or cold plate (not shown) in the server 203. The RMC 222 can determine the control signal 147 based on the output of one or more temperature sensors (not shown) from one or more chip boxes or cold plates (not shown) in the servers 203A-203D. The RMC 222 can also control the on and off state of the flow control device 140 and the coolant pump 146.

[0047] In one embodiment, the single-phase cooling fluid in the single-phase cooling loop 132 / 131 is a first working fluid having a vaporization temperature that is higher than the design temperature of the cooling loop, such that the single-phase cooling fluid in the first cooling loop is not designed to vaporize during normal, design, heat loads. In one embodiment, the working fluid in the two-phase coolant supply 134 is selected so that the working fluid will vaporize at a temperature that is within the design operating range of multiple heat generating components within one or more servers 203 in the electronics rack 200.

[0048] Figure 1B 1 is a block diagram illustrating an example of a cooling system 101 for an electronic rack 200 according to one embodiment. The cooling system 101 for the electronic rack 200 may be substantially as described above with reference to Figure 1A The disclosed cooling system 100 has the following additional and / or different features.

[0049] The cooling system 100 for the electronics rack 200 also includes a two-phase "condenser" cooling system 121 having a two-phase cooling circuit with a two-phase coolant supply 134 and a vapor return 133. The cooling system 101 further includes a separate single-phase cooling circuit with a single-phase cooling fluid supply 132 and a single-phase cooling fluid return 131.

[0050] The facility cooling fluid source 137 is a single-phase cooling fluid. Figure 1B, the flow of facility cooling fluid from source 137 to condenser system 121 and to the single-phase cooling loop is determined by the flow rate setting of coolant pump 146. The inlet port of coolant pump 146 is coupled to facility cooling fluid source 137. The outlet port of coolant pump 146 is coupled to the inlet connection of condenser system 121 (1) and to flow control device ("FCD") 140 (2). The outlet port of flow control device 140 is coupled to a single-phase cooling fluid supply manifold 142 mounted in electronics rack 200. The amount of facility cooling fluid 137 provided to condenser system 121 is first determined by the flow rate setting of coolant pump 146. The flow rate of coolant pump 146 sets the maximum flow rate of single-phase cooling fluid 137 that will be shared between two-phase condenser system 121 and single-phase cooling loop 132 / 131. The flow rate setting of the flow control device 140 determines how much of the maximum flow rate from the coolant pump 146 will be delivered to the single-phase cooling fluid 132 / 131 coupled to the distribution manifold 142 / 141. In one embodiment, the single-phase cooling circuit 132 / 131 is a supplement to the continuously circulating two-phase cooling circuit 134 / 133, so that the flow control device can be set to completely closed (zero flow rate). In one embodiment, the RMC 222 can also control the opening and closing state of the flow control device 140 and / or the coolant pump 146. The RMC 222 also controls the variable opening position of the flow control device 140 and the variable speed of the coolant pump 146. The RMC 222 can set the flow control device 140 to the closed position, set the coolant pump to the open state, and adjust the speed of the coolant pump 146. In one embodiment, when the temperature rises, the RMC 222 can first start to increase the speed of the pump and still maintain the flow control device 140 in the closed position (closed state). As the temperature continues to rise to higher values, RMC 222 may open flow control device 140 to direct the single-phase cooling fluid directly to the electronics rack via single-phase cooling loop 132 / 131 .

[0051] In one embodiment, the flow rate of the coolant pump 146 may be manually set. In one embodiment, a control signal (not shown) from a rack management controller (RMC) 222 within the rack management unit 202 may control the flow rate setting of the coolant pump 146 via a control signal set by management software. In one embodiment, the coolant pump 146 flow rate may be set by the RMC 222 in response to one or more temperatures 145 of a chip cassette in the server 203, as follows: Figure 4 shown.

[0052] As described above, one or more heat generating components within each server 203 in the electronic rack 200 are coupled to a cold plate, and a temperature sensor may be provided in or on the chip box of the heat generating component. The cold plate may use a single-phase coolant loop 132 / 131 and a two-phase cooling loop 134 / 133 to cool the heat generating device coupled to the cold plate. In one embodiment, the single-phase flow control device 140 may be set to a zero gpm flow rate so that only a two-phase cooling fluid flows through the cold plate. The chip box temperature may be sent to the rack RMC 222 to generate a control signal to the flow control device 140 to set a flow rate for the flow control device 140. When the RMC 222 determines that the temperature is rising, the flow rate of the flow control device 140 may be increased.

[0053] In one embodiment, the flow rate setting of the flow control device 140 may be a fixed percentage of the flow rate set for the coolant pump 146. For example, the flow control device 140 may be set to a flow rate that is 50% of the flow rate of the coolant pump 146. In another embodiment, the flow rate of the flow control device may be determined based on a calibration table that correlates the temperature determined by the RMC 222 to the flow rate of the coolant pump 146. In one embodiment, the RMC 222 may include one or more artificial intelligence ("AI") models trained to correlate temperature changes of the RMC 222, the flow rate of the coolant pump 146, and the flow rate of the flow control device 140 to minimize the temperature determined by the RMC 222. In one embodiment, the one or more AI models may additionally or alternatively minimize energy consumption, such as due to the operation of the coolant pump 146 and the energy consumption for actuating the flow control device 140.

[0054] Figure 1C As shown above Figure 1B In another embodiment, the above can be added Figure 1A or additional systems of the type described in 1B. In such an embodiment, according to Figure 1A A large number of servers and / or 1B can be configured as a server cluster in a data center (not shown). In one embodiment, a large number of servers according to Figure 1A The electronic racks of 1B and / or 1B can be configured as a data center system (not shown). The number of electronic racks configured as a cluster or data center is unlimited. Figure 1C, a facility cooling fluid source 137 provides facility cooling fluid 137 to a coolant pump 146 of each electronics rack 200A and 200B. The electronics racks 200A and 200B may have different numbers of servers 203 in their respective racks. Each server 203 within each rack may have a different number and type of heat generating components than the other servers 203. The single-phase cooling fluid return 131 from the electronics racks is returned to the facility cooling fluid return 136. The single-phase cooling fluid 136 returned from the condenser system 121 is also returned to the facility cooling fluid return 136. The facility cooling fluid return 136 is delivered to a facility cooling system (not shown). If the electronics racks 200A and 200B or other additional racks (not shown) are added Figure 1C In the embodiment of the present invention, other features of the electronic racks 200A and 200B or other additional racks (not shown) are similar to Figure 1B (exist Figure 1C In the case of racks 200A and 200B shown) or Figure 1A In one embodiment, each electronics rack 200 in a server cluster or data center may include different fluid control components coupled between the facility cooling fluid supply 137 and facility cooling fluid return 136 and the single-phase cooling loop manifolds 141 and 142 in each electronics rack 200 .

[0055] Figure 2 2 is a block diagram showing an electronic rack 200 according to one embodiment. The electronic rack 200 may represent any electronic rack 200, such as Figures 1A to 1C As shown in . Figure 2 According to one embodiment, the electronic rack 200 includes, but is not limited to, a rack management unit (RMU) 202 and one or more server chassis 203A-203F (collectively referred to as server chassis 203 unless otherwise specified). The server chassis 203 can be inserted into a server slot array (e.g., a standard rack) from the front end 204 or the rear end 205 of the electronic rack 200, respectively. Note that although six server chassis 203A-203F are shown here, more or fewer server chassis can be maintained within the electronic rack 200. In one embodiment, the electronic rack 200 can be open to the environment or partially contained by a rack container, as long as the cooling fans can generate airflow from the front end to the rear end.

[0056] In addition, for at least some of the server chassis 203, an optional fan module (not shown) is associated with the server chassis. Each fan module includes one or more cooling fans. The fan module can be mounted on the rear end of the server chassis 203 or on the electronics rack to generate an airflow that flows from the front end 204, passes through the air space of the server chassis 203, and exists at the rear end 205 of the electronics rack 200.

[0057] Each server chassis 203 may include one or more IT components (e.g., a central processing unit or CPU, a general / graphics processing unit (GPU), memory, and / or storage). Each IT component may perform data processing tasks, wherein the IT component may include software installed in a storage device, loaded into the memory, and executed by one or more processors to perform the data processing tasks. The server chassis 203 may include a host server (referred to as a host node) connected to one or more computing servers (also referred to as computing nodes, such as CPU servers and GPU servers). The host server (having one or more CPUs) typically interfaces with clients over a network (e.g., the Internet) to receive requests for specific services, such as storage services (e.g., cloud-based storage services, such as backup and / or recovery), execute applications to perform certain operations (e.g., image processing, deep data learning algorithms or modeling, etc., as part of a software as a service or SaaS platform). In response to the request, the host server assigns the task to one or more computing nodes or computing servers (with one or more GPUs) managed by the host server. The computer server performs the actual task, which may generate heat during operation.

[0058] The electronics rack 200 further includes an optional RMU 202 configured to provide and manage power supplied to the servers 203. The RMU 202 can be connected to a power supply unit (not shown) to manage the power consumption of the power supply unit. The power supply unit may include necessary circuitry (e.g., an alternating current (AC) to direct current (DC) or DC to DC power converter, a battery, a transformer or a regulator, etc.) to provide power to the remaining components of the electronics rack 200. The RMU may include a temperature or control signal 147 that is sent to a flow control device (FCD) 140 and, optionally, to a coolant pump 146. The temperature signal 147 may be derived from one or more temperature signals (not shown) from a chip box (not shown) of a cold plate 400 and / or a heat generating component. For more details on the temperatures of the chip box, cold plate, and heat generating components, see below. Figure 3 .

[0059] In one embodiment, the RMU 202 includes an optimization module 221 and a rack management controller (RMC) 222. The RMC 222 may include a monitor to monitor the operating status of various components (e.g., servers 203 and fan modules) within the electronic rack 200. Specifically, the RMC 222 receives operating data from various sensors representing the operating environment of the electronic rack 200. For example, the monitor may receive operating data representing the temperature of the processor, cooling fluid, and airflow, which may be captured and collected via various temperature sensors. The monitor may also receive data representing the fan power and pump power generated by the fan module 231 and the liquid pump 212, which may be proportional to their respective speeds. These operating data are referred to as real-time operating data. Please note that the monitor may be implemented as a separate module within the RMU 202.

[0060] Notice, Figure 2 The rack configuration shown in is shown and described for illustrative purposes only; other configurations or arrangements may also be applicable. The cold plate 400 of the server chassis 203 can be connected to a rack manifold, such as the coolant distribution and return manifolds 141-144 of the single-phase ("1-phase") cooling loop 132 / 131 and the two-phase ("2-phase") cooling loop 134 / 133 described above. Each server 203 can be connected to the manifold 141-143 at the connection 150 using a connecting hose or pipe 128. Each server 203 can deliver a single-phase cooling fluid supply and return 132 / 131 ("single-phase cooling loop") to one or more cold plates, and can also deliver a two-phase coolant supply and vapor return 134 / 133 ("two-phase cooling loop") to one or more cold plates in the server 203. In one embodiment, one or more or all of the cold plates can be connected to the single-phase cooling loop and the two-phase cooling loop.

[0061] Figure 32 is a block diagram showing a top view of a server 203A in an electronics rack 200 according to one embodiment. The front of the electronics rack 200 is labeled 204, and the rear of the electronics rack is labeled 205. As described above, the single-phase cooling loop may include a single-phase cooling fluid supply line 132 and a single-phase cooling fluid return line 131. The single-phase cooling fluid supply line 132 may be coupled to a manifold 142 mounted in the rear 205 of the electronics rack 200 using, for example, a hose connection. The single-phase cooling fluid return line 131 may be coupled to the manifold 141 using, for example, a hose connection. Similarly, the two-phase cooling loop has a two-phase cooling fluid supply 134, which may be coupled to the manifold 144. The steam return line 133 of the two-phase loop may be coupled to the manifold 143 using, for example, a hose connection. Any heat generating device (not shown) within the server 203 may be coupled to a cold plate (not shown) within the server 203, which is cooled by the single-phase cooling loop and the two-phase cooling loop. In one embodiment, the heat generating components may additionally be cooled by means of air flow, thermoelectric (TE) coolers, etc. The design of the cooling plate for running two cooling circuits is not the subject of this disclosure.

[0062] Figure 4 4 is a block diagram illustrating a processor cold plate configuration according to one embodiment. The processor / cold plate assembly 400 may represent any of the processor / cold plate configurations of the server chassis 203 described above. Figure 4 , the processor 401 is plugged into a processor socket mounted on a printed circuit board (PCB) or motherboard 402, which is connected to other electrical components or circuits of the data processing system or server. The processor chip 401 can be mounted to a chip box with a temperature sensor 145, and the temperature sensor 145 sends a temperature signal to the RMC 222. The RMC 222 can use mathematical derivation to determine a single temperature value for determining the control signal for the flow control device 140. The single temperature can be selected by the RMC 222 as the highest temperature of all chip box temperatures received from the server 203 chip box. The single temperature can be an average or weighted average of all chip box temperatures received from the server 203 chip box. In one embodiment, the RMC 222 can select one or more specific chip box temperatures from all available chip box temperatures for determining the single temperature to determine the control signal for the flow control device 140.

[0063] The processor 401 also includes a cold plate 403 attached thereto, which is coupled to a plurality of rack manifolds (not shown) that are coupled to the single-phase cooling fluid supply line 132 and the single-phase cooling fluid return line 131 as well as the two-phase coolant supply line 134 and the vapor return line 133. Some of the heat generated by the processor 401 is removed by the cooling liquid via the cold plate 403. The remaining heat enters the air space below or above and can be removed by the airflow 139 generated by the cooling fan 404. The cold air 139 passes through the air space, which heats the air into warm air 138, which is exhausted from the back (not shown) of the electronics rack 200 in which the server 203 is mounted.

[0064] In the foregoing description, embodiments of the present invention have been described with reference to specific exemplary embodiments of the present invention. Although the rack orientation is shown as horizontal, this is not limiting. Different server rack orientations (e.g., vertical, or upward / downward) can be implemented using the present disclosure. It will be apparent that various modifications may be made to the present invention without departing from the broader spirit and scope of the present invention as set forth in the following claims. Therefore, the present description and drawings should be regarded as illustrative rather than restrictive.

[0065] In the foregoing description, embodiments of the present invention have been described with reference to specific exemplary embodiments of the present invention. Those skilled in the art in possession of the present disclosure may implement different options for male / female connectors, connector types, hoses, tubes, pipes, and structural frame members and component orientations. Although the rack orientation is shown as horizontal, this is not restrictive. Different server rack orientations (e.g., vertical, or upward / downward) may be implemented using the present disclosure. It will be apparent that various modifications may be made to the present invention without departing from the broader spirit and scope of the present invention as set forth in the following claims. Therefore, the present specification and drawings should be regarded as illustrative, rather than restrictive.

Claims

1. An electronics rack cooling system, configured to cool heat generating components in a plurality of servers in an electronics rack, the cooling system comprising: a condenser of a two-phase cooling system coupled to a facility cooling fluid supply, the condenser configured to cool vapor returned from a two-phase cooling loop coupled between the condenser of the two-phase cooling system and a two-phase coolant distribution manifold within the electronics rack, wherein the two-phase cooling loop includes a two-phase coolant supply and a vapor return; a single-phase cooling circuit coupled to the facility cooling fluid supply via a flow control system, the single-phase cooling circuit coupled to a single-phase coolant distribution manifold within the electronics rack; and A controller is configured to control a flow rate of a single-phase cooling fluid from the flow control system to the single-phase cooling loop in response to a temperature of a heat generating component in the server determined by the controller.

2. The electronic rack cooling system according to claim 1, wherein: The flow control system comprises: Two-way control valve; Coolant pump; or Two-way control valve and coolant pump.

3. The electronic rack cooling system of claim 1, wherein: The two-phase cooling system is integrated with the electronics rack.

4. The electronic rack cooling system of claim 1, wherein: The controller controls the flow control system to prevent single-phase cooling fluid from flowing to the single-phase cooling circuit in response to the temperature being below a predetermined threshold.

5. The electronic rack cooling system of claim 1, wherein: The controller controls the flow control system to adjust an amount of the single-phase cooling fluid delivered to the single-phase cooling circuit in response to the change in temperature.

6. The electronic rack cooling system of claim 5, wherein: The single-phase coolant loop is coupled to the plurality of servers within the electronics rack via a single-phase distribution manifold in the electronics rack, and the two-phase coolant loop is coupled to the plurality of servers via a two-phase distribution manifold in the electronics rack.

7. The electronic rack cooling system of claim 1, wherein: The single-phase cooling loop and the two-phase cooling loop are both coupled to a same heat generating component within a server of the plurality of servers.

8. The electronic rack cooling system of claim 7, wherein: The single-phase cooling loop is controlled to cool the electronic rack directly, or to cool the electronic rack with a combination of direct cooling using the single-phase cooling loop and indirect cooling using the condenser of the two-phase cooling system and the two-phase cooling loop.

9. An electronics rack cooling system configured to cool heat generating components in a plurality of servers in an electronics rack, the cooling system comprising: a coolant pump coupled to the facility cooling system fluid supply; a condenser of a two-phase cooling system coupled to an output of the coolant pump, the condenser configured to cool vapor returned from a two-phase cooling loop coupled between the condenser of the two-phase cooling system and a two-phase coolant distribution manifold within the electronics rack, wherein the two-phase cooling loop includes a two-phase coolant supply and a vapor return; a flow control device coupled to the output end of the coolant pump; a single-phase cooling circuit coupled to the facility cooling system fluid supply via the flow control device, the single-phase cooling circuit coupled to a single-phase coolant distribution manifold within the electronics rack; as well as A controller is configured to control a flow rate of a single-phase cooling fluid from the flow control device to the single-phase cooling circuit in response to a temperature of a heat generating component in the server determined by the controller.

10. The electronic rack cooling system of claim 9, wherein: The flow control device comprises a two-way control valve.

11. The electronic rack cooling system of claim 9, wherein: The two-phase cooling system, the coolant pump, and the flow control device are integrated with the electronics rack.

12. The electronic rack cooling system of claim 9, wherein: The controller controls the flow control device to prevent the single-phase cooling fluid from flowing to the single-phase cooling circuit in response to the temperature being below a predetermined threshold.

13. The electronic rack cooling system of claim 9, wherein: The controller controls the flow control device to adjust an amount of the single-phase cooling fluid delivered to the single-phase cooling circuit in response to the change in temperature.

14. The electronic rack cooling system of claim 13, wherein: The flow rate of the coolant pump is set to a preset value.

15. The electronic rack cooling system of claim 9, wherein: The controller controls the speed of the coolant pump to vary a flow rate of the coolant pump.

16. The electronic rack cooling system of claim 9, wherein: The single-phase coolant loop is coupled to the plurality of servers within the electronics rack via a single-phase distribution manifold in the electronics rack, and the two-phase coolant loop is coupled to the plurality of servers via a two-phase distribution manifold in the electronics rack.

17. The electronics rack cooling system of claim 9, wherein: The single-phase cooling loop and the two-phase cooling loop are both coupled to a same heat generating component within a server of the plurality of servers.

18. A data center, comprising: facility cooling fluid supply; a plurality of electronic racks, each electronic rack including a plurality of heat generating components; as well as an indirect cooling system for each electronics rack, the indirect cooling system comprising a condenser coupled to the facility cooling fluid supply, wherein the indirect cooling system comprises a two-phase cooling loop coupled between the electronics rack and the condenser, the two-phase cooling loop recirculating a two-phase cooling fluid to the electronics rack and back to the indirect cooling system; Each electronic rack further comprises: a direct single-phase cooling loop coupled to the facility cooling fluid supply via a cooling fluid management component and to the electronics rack, The single-phase cooling loop recirculates single-phase cooling fluid to the electronic racks and returns to the facility cooling fluid return, and the cooling fluid management component includes a coolant pump and a flow control device to regulate the flow of the single-phase cooling fluid in the single-phase cooling loop.

19. The data center according to claim 18, further comprising a rack controller for each electronic rack, wherein the rack controller controls each cooling fluid management component to adjust the flow rate of the single-phase cooling fluid in the single-phase cooling loop; wherein The controller is used to control a flow rate of a single-phase cooling fluid from the fluid management component to the single-phase cooling loop in response to a temperature of a heat generating component in the server determined by the controller.

Images