Electronic rack cooling system based on high elastic deployment

Abstract

Disclosed herein is a cooling system for electronic racks in a data center such as an IT rack or a PoD. The system includes a coolant distribution unit (CDU) connected to a rack manifold of the electronic rack by a fluid cooling loop. The CDU supplies cooling fluid distributed to the IT rack or PoD and receives a returned warm / hot fluid from the IT rack or PoD. The system also includes an enhanced cooling unit to receive a first portion of a first distribution portion of the cooling fluid, and further cools the first portion of the first distribution portion of the cooling fluid to a lower temperature value than one of the supplied cooling fluids by enhancing the cooling loop. The system also includes a first external cooling unit connected to the CDU by a cooling fluid loop to supply cooling fluid to the CDU. The system also includes an air cooling unit to receive a second distribution portion of the cooling fluid and use the second distribution portion of the cooling fluid to cool the airflow for the IT rack or PoD. The system is arranged in five parts, and the fluid and cooling unit are shared between the parts in the distribution loop.

Description

Technical Field

[0001] Embodiments of the present invention generally relate to data center cooling. More specifically, embodiments of the present invention relate to a highly resilient deployment based cooling system for electronics racks of information technology (IT) components. Background Art

[0002] Cooling is a prominent factor in the design of computer systems and data centers. The number of high-performance electronic components, such as high-performance processors, packaged within servers has steadily increased, increasing the amount of heat generated and dissipated during ordinary operation of the servers. The reliability of servers used within a data center decreases if the environment in which the servers operate is allowed to increase in temperature over time. Maintaining a proper thermal environment is critical to the proper operation of these servers in a data center, as well as server performance and lifespan, energy efficiency, and cost, especially in the case of cooling these high-performance servers, which requires more effective and efficient cooling solutions.

[0003] For high-density electronic racks of IT components, liquid cooling has become an important technology for high power density servers and electronic devices such as central processing units (CPUs) / graphics processing units (GPUs) / application-specific integrated circuits (ASICs). Such cooling systems need to be able to provide liquid distribution to those components. For other components or IT equipment, they can continue to be air cooled. This means that the rack uses mixed cooling of air and liquid at the same time, so air cooling is still required. This means that the system needs to be able to circulate both air and liquid. Therefore, the cooling system needs to provide cooling / cold fluid to the air cooling equipment.

[0004] Typically, traditional cooling systems focus mainly on liquid cooling, or are optimized on liquid cooling or air cooling. Without any system modifications, they cannot support different types of configurations or changes in racks or IT equipment, and are not cost-effective. In addition, they only target energy efficiency improvements on the infrastructure side, not on the rack or IT side. For example, when the operating temperature increases, those conventional cooling systems can be used to improve the energy efficiency of the equipment side. Unfortunately, this does not support performance and power optimization on the IT and rack sides. Therefore, facility and IT hardware performance and energy efficiency co-design are critical in modern data centers. Summary of the invention

[0005] According to one aspect of the present application, a cooling system for an electronic rack of a data center is provided, which may include:

[0006] a coolant distribution unit (CDU) coupled to a rack manifold of the electronic rack via a fluid cooling loop, the CDU supplying cooling fluid distributed to the rack manifold and receiving warm / hot fluid returned from the rack manifold;

[0007] an enhanced cooling unit for receiving a first portion of the first distributed portion of the cooling fluid and further cooling the first portion of the first distributed portion of the cooling fluid to a temperature lower than a temperature of the cooling fluid supplied by a refrigeration circuit;

[0008] a first external cooling unit connected to the CDU via a cooling fluid circuit to supply the cooling fluid to the CDU; and

[0009] An air cooling unit is configured to receive the second allocated portion of the cooling fluid and to cool the electronics rack using the second allocated portion of the cooling fluid.

[0010] According to another aspect of the present application, a cooling system for an electronic rack of a data center is provided, which may include:

[0011] a plurality of coolant distribution units (CDUs) coupled to rack manifolds of the electronics racks, the plurality of CDUs comprising:

[0012] a first CDU that supplies a first cooling fluid distributed to the rack manifold and receives a return warm / hot fluid from the rack manifold, the first CDU comprising a boost cooling unit that receives a first portion of the first distributed portion of the first cooling fluid and further cools the first portion of the first distributed portion of the first cooling fluid through a refrigeration circuit, and

[0013] a second CDU for supplying a second cooling fluid distributed to the rack manifold on a fluid cooling distribution loop and receiving a return warm / hot fluid from the rack manifold, the second CDU comprising a boost cooling unit for receiving a first portion of the first distributed portion of the second cooling fluid and further cooling the first portion of the first distributed portion of the second cooling fluid through a cold fluid distribution loop, and

[0014] An air cooling unit is configured to receive the second distributed portion of the first cooling fluid and the second distributed portion of the second cooling fluid and to cool the electronic rack through airflow using the second distributed portion of the first cooling fluid and the second distributed portion of the second cooling fluid.

[0015] According to another aspect of the present application, a data center system is provided, which may include:

[0016] Multiple independently deployable parts, including:

[0017] A rack portion, comprising: (i) an electronics rack accommodating a plurality of server chassis, each server chassis corresponding to one or more servers, wherein the electronics rack comprises a rack manifold to receive cooling fluid and return warm / hot fluid from the electronics rack, and (ii) an air cooling unit for cooling the electronics rack,

[0018] a secondary distribution portion, comprising: (i) a fluid cooling distribution loop for delivering the cooling fluid to the electronic rack, (ii) a system return distribution loop for delivering returned warm / hot fluid from the electronic rack, and (iii) an air cooling distribution loop for delivering a distribution portion of the cooling fluid to the air cooling unit, and

[0019] An enhanced cooling section comprises: (i) a first coolant distribution unit (CDU) for supplying a first portion of the cooling fluid to the rack manifold, receiving a first portion of the warm / hot fluid from the rack manifold, and supplying another distribution portion of the cooling fluid to the air cooling unit through the air cooling distribution loop, and (ii) a second CDU for supplying a second portion of the cooling fluid to the rack manifold through the fluid cooling distribution loop, receiving the second portion of the warm / hot fluid through the system return distribution loop, and supplying a distribution portion of the cooling fluid to the air cooling unit through the air cooling distribution loop. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Embodiments of the invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like references indicate similar elements.

[0021] Figure 1 is a block diagram illustrating an example of a data center facility according to one embodiment.

[0022] Figure 2 is a schematic diagram showing an example of a cooling system according to one embodiment.

[0023] Figure 3 is a schematic diagram showing another example of a cooling system according to one embodiment.

[0024] Figure 4 is a schematic diagram illustrating an example of a cooling system with an alternative coolant distribution unit (CDU) according to one embodiment.

[0025] Figure 5 is a schematic diagram showing still another example of a cooling system according to an embodiment.

[0026] Figure 6 is a schematic diagram illustrating an example of a cooling system with different data center sections according to one embodiment. DETAILED DESCRIPTION

[0027] Various embodiments and aspects of the present invention will be described with reference to the details discussed below, and the accompanying drawings will illustrate various embodiments. The following description and accompanying drawings are explanations of the present invention and do not constitute limitations of the present invention. Many specific details are described to provide a thorough understanding of various embodiments of the present invention. However, in some cases, in order to provide a concise discussion of embodiments of the present invention, known or conventional details are not described.

[0028] Reference to "one embodiment" or "an embodiment" in the specification means that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The phrase "in one embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment.

[0029] According to some embodiments, a cooling system with an innovative cooling unit for a data center with multiple high-density racks is provided. The cooling system can be used in the next generation cooling architecture of the electronic rack of the IT component, which, as described above, is mixed cooled by air cooling and liquid cooling. The cooling system also meets the requirements of the dynamic configuration of the rack or IT equipment by providing different liquid / fluid cooling and air cooling configurations. The embodiments of the present invention provide a very needed elastic solution in the modern data center industry and are an important trend. In addition, the embodiments of the present invention solve the problems existing in traditional cooling systems, such as dynamic rack configuration with hybrid (liquid-air) cooling technology, dynamic changes in power density and IT equipment, flexibility, elasticity and other requirements, and dynamic changes in operating conditions due to performance considerations. The embodiments of the present invention can also provide a large temperature range for the liquid cooling loop, because for some electronic components (e.g., CPU, ASIC or GPU), better energy sufficiency or performance enhancement may require lower temperatures while maintaining the entire cooling system at a lower total cost of ownership (TCO). In addition, the embodiments of the present invention provide a strong residence for different types of scenarios, such as from the component level to the full system level.

[0030] In some embodiments, the cooling system disclosed herein is used for data center IT equipment. The electronic racks of IT components use a mixture of cooling water and cooling air. The cooling system provides cooling fluid to liquid cooling devices and air cooling devices by using one or more cooling distribution units. An enhanced cooling loop is also used in the system for the liquid / fluid cooling loop. It is used to provide an enhanced thermal environment for the liquid cooling components when needed. The method can be understood as designing and operating the system to provide most of the required enhanced conditions that are generally used for most types of IT equipment or clusters. This is to increase the flexibility and elasticity of the system design. The enhanced cooling unit is connected to a refrigeration cycle loop or a similar loop, which provides similar functions within a certain period of time. In one embodiment, the enhanced cooling unit can also be embedded in the cooling distribution unit. A system-level structure is also disclosed herein, which includes a secondary distribution part and a main loop part used between the IT and cooling units and an enhanced cooling part.

[0031] According to one aspect, a cooling system for electronic racks in a data center is described. The system includes a coolant distribution unit (CDU) connected to a rack manifold of the electronic rack by a fluid cooling loop. The CDU supplies cooling fluid distributed to the rack manifold and receives returning warm / hot fluid from the rack manifold. The system also includes an enhanced cooling unit to receive a first portion of a first distribution portion of the cooling fluid and further cool the first portion of the first distribution portion of the cooling fluid to a temperature lower than the temperature of the cooling fluid supplied by the refrigeration circuit. The system also includes a first external cooling unit connected to the CDU by a cooling fluid loop to supply cooling fluid to the CDU. The system also includes an air cooling unit to receive a second distribution portion of the cooling fluid and use the second distribution portion of the cooling fluid to cool hot exhaust gas for the electronic rack.

[0032] In one embodiment, the system includes a first three-way valve and a second three-way valve, the first three-way valve is used to receive the supplied cooling fluid and distribute the supplied cooling fluid into the first distribution part and the second distribution part, the second three-way valve is used to receive the first distribution part of the cooling fluid, and deliver the first part of the first distribution part of the cooling fluid to the enhanced cooling unit, and deliver the second part of the first distribution part of the cooling fluid to the rack manifold on the fluid distribution circuit. The enhanced cooling unit can supply the further cooled first part of the first distribution part of the cooling fluid to the fluid cooling circuit. The system also includes a condensing unit connected to the enhanced cooling unit through a refrigeration circuit. The enhanced cooling unit and the first three-way valve and the second three-way valve are included in the CDU. The CDU may also include a heat exchanger that transfers heat between the returned warm / hot fluid and the supplied cooling fluid, and a liquid pump that delivers the supplied cooling fluid to the first three-way valve. The CDU also includes a plurality of inlets and outlets connected to the enhanced cooling unit, the second three-way valve, the heat exchanger, and the enhanced cooling unit.

[0033] According to another aspect, a cooling system for electronic racks in a data center is described. The system includes a plurality of coolant distribution units (CDUs) coupled to a rack manifold of the electronic rack. The CDU includes a first CDU to supply a first cooling fluid distributed to the rack manifold and receive a returned warm / hot fluid from the rack manifold. The first CDU includes an enhanced cooling unit to receive a first portion of a first distribution portion of the first cooling fluid and further cool a first portion of the first distribution portion of the first cooling fluid through a refrigeration circuit. The CDU also includes a second CDU for supplying a second cooling fluid distributed to the rack manifold through a fluid cooling distribution circuit and receiving a returned warm / hot fluid from the rack manifold. The second CDU includes an enhanced cooling unit to receive a first portion of a first distribution portion of the second cooling fluid and further cool a first portion of the first distribution portion of the second cooling fluid through a cold fluid distribution circuit. The system also includes an air cooling unit to receive a second distribution portion of the first cooling fluid and a second distribution portion of the second cooling fluid, and use the second distribution portion of the first cooling fluid and the second distribution portion of the second cooling fluid to cool the electronic rack. In one embodiment, there may be a plurality of air cooling units receiving a plurality of cooling fluid sources of second distribution portions of cooling fluid from a plurality of CDUs.

[0034] In one embodiment, the system further includes a first external cooling unit connected to the first CDU through a cooling fluid circuit to supply a first cooling fluid to the first CDU. The first external cooling unit may also be connected to the second CDU through a cooling fluid distribution circuit to supply a second cooling fluid to the second CDU. The system may also include a second external cooling unit connected to the enhanced cooling unit of the first CDU through a refrigeration circuit. The second external cooling unit may also be connected to the enhanced cooling unit of the second CDU through a cooling fluid distribution circuit.

[0035] In one embodiment, the first CDU further includes a first three-way valve to receive the supplied first cooling fluid and distribute the supplied first cooling fluid into a first distribution portion and a second distribution portion of the first cooling fluid. The first CDU may also include a second three-way valve to receive the first distribution portion of the first cooling fluid and deliver a first portion of the first distribution portion of the first cooling fluid to the enhanced cooling unit of the first CDU through the fluid cooling distribution circuit, and deliver the second portion of the first distribution portion of the first cooling fluid to the rack manifold.

[0036] In one embodiment, the second CDU further includes a first three-way valve to receive the supplied second cooling fluid and distribute the supplied second cooling fluid into a first distribution portion and a second distribution portion of the second cooling fluid. The second CDU may also include a second three-way valve to receive the first distribution portion of the second cooling fluid and deliver a first portion of the first distribution portion of the second cooling fluid to the enhanced cooling unit of the second CDU, and deliver the second portion of the first distribution portion of the second cooling fluid to the rack manifold on the fluid cooling distribution loop.

[0037] According to another aspect, a data center system is described. The system includes an independently deployable portion, the independently deployable portion including a rack portion, the rack portion having: (i) an electronics rack accommodating a plurality of server chassis, each server chassis corresponding to one or more servers, wherein the electronics rack includes a rack manifold for receiving cooling fluid and returning warm / hot fluid from the electronics rack, and (ii) an air cooling unit for cooling airflow for the electronics rack. The system also includes a second distribution portion, the second distribution portion including: (i) a fluid cooling distribution loop for delivering cooling fluid to the electronics rack; (ii) a system return distribution loop for delivering returned warm / hot fluid from the electronics rack and the air cooling unit; and (iii) an air cooling distribution loop for delivering a distributed portion of the cooling fluid to the air cooling unit. The system also includes an enhanced cooling part, which includes: (i) a first coolant distribution unit (CDU), which is used to supply a first part of the cooling fluid to the rack manifold, receive a first part of the warm / hot fluid from the rack manifold, and supply another distribution part of the cooling fluid to the air cooling unit through an air cooling distribution loop, and receive the warm / hot fluid through a system return distribution loop; and (ii) a second CDU, which supplies a second part of the cooling fluid to the rack manifold through a fluid cooling distribution loop to receive the second part of the warm / hot fluid through a system return distribution loop, and supplies a distribution part of the cooling fluid to the air cooling unit through the air cooling distribution loop, and receives the warm / hot fluid through the system return distribution loop.

[0038] Figure 1 is a block diagram illustrating an example of a data center or data center unit according to one embodiment. In this example, Figure 1 A top view of at least a portion of a data center is shown. Figure 1 According to one embodiment, a data center system 100 includes one or more rows of electronic racks (IT) of IT components, equipment or tools 101-102, such as computer servers or computing nodes that provide data services to various clients through a network (e.g., the Internet). In this embodiment, each row includes an array of electronic racks such as electronic racks 110A-110N. However, more or fewer rows of electronic racks may be implemented. Typically, rows 101-102 are arranged in parallel, with their front ends facing each other, and their rear ends facing away from each other, thereby forming aisles 103 between the rows to allow managers to walk therein. However, other configurations or arrangements may also be applied. For example, two rows of electronic racks may face each other back to back without forming an aisle between them, while their front ends are facing away from each other. The rear end of the electronic rack may be connected to a room cooling liquid manifold. The current method proposed in the present disclosure may be implemented in different configurations, including configurations not discussed herein.

[0039] In one embodiment, each of the electronic racks (e.g., electronic racks 110A-110N) includes an enclosure to accommodate a plurality of IT components arranged in a stacked portion for operation therein. The electronic rack may include a cooling liquid manifold, a plurality of server slots (e.g., standard racks or chassis configured with the same or similar form factors), and a plurality of server chassis (also referred to as blade servers or server racks) that can be inserted into and removed from the slots. Each server chassis represents a computing node having one or more processors, memory, and / or permanent storage devices (e.g., hard disks), wherein the computing node may include one or more servers operating therein. At least one of the processors is attached to a liquid cold plate (also referred to as a cold plate assembly) to receive cooling liquid. In addition, one or more optional cooling fans are associated with the server chassis to provide air cooling to the computing nodes housed therein. Note that the cooling system 120 may be coupled to a plurality of data center systems, such as the data center system 100. There may be a plurality of cooling systems coupled to a data center or cluster.

[0040] In one embodiment, the cooling system 120 includes an external liquid loop connected to a cooling tower or dry cooler outside the building / containment vessel. The cooling system 120 may include, but is not limited to, evaporative cooling, free air, rejection of large thermal mass, and chiller systems designed for waste heat recovery. The cooling system 120 may include or be coupled to a cooling liquid source that provides cooling liquid.

[0041] In one embodiment, each server chassis is modularly coupled to the cooling liquid manifold so that the server chassis can be removed from the electronics rack without affecting the operation of the remaining server chassis on the electronics rack and the cooling liquid manifold. In another embodiment, each server chassis is coupled to the cooling liquid manifold by a quick release coupling component having a server liquid outlet connector and a server liquid inlet connector that are coupled to a flexible hose to distribute cooling liquid to the processor. The server liquid inlet connector will receive heat exhaust liquid from a heat exhaust liquid manifold mounted on the rear end of the electronics rack through the rack liquid inlet connector. The server liquid outlet connector is used to transmit the warmer or hotter liquid exchanged from the processor to the cooling liquid manifold through the rack liquid outlet connector back to the coolant distribution unit (CDU) within the electronics rack.

[0042] In one embodiment, a cooling liquid manifold is disposed on the rear end of each electronic rack and is coupled to a liquid supply line 132 (also referred to as a room supply manifold) to receive cooling liquid from the cooling system 120. The cooling liquid is distributed through a liquid distribution loop that is attached to a cold plate assembly on which the processors are mounted to remove cooling energy from the processors. The cold plate is configured similar to a radiator with liquid distribution pipes connected or embedded therein. The resulting warmer or hotter liquid carrying heat exchanged from the processors is passed back to the cooling system 120 via a liquid return line 131 (also referred to as a room return manifold).

[0043] Liquid supply / return lines 131-132, referred to as data center or room liquid supply / return lines (e.g., overall liquid supply / return lines), supply cooling liquid to all of the electronic racks in rows 101-102. Liquid supply lines 132 and liquid return lines 131 are coupled to a heat exchanger of a CDU located within each of the electronic racks, thereby forming a primary loop. A secondary loop of the heat exchanger is coupled to each of the server chassis in the electronic racks to deliver cooling liquid to the cold plates of the processors.

[0044] In one embodiment, the data center system 100 further includes an optional airflow delivery system 135 to generate airflow so that the airflow passes through the air space of the server chassis of the electronic rack to exchange the heat generated by the computing nodes (e.g., servers) due to the operation of the computing nodes, and discharge the heat exchanged by the airflow to the external environment outside the housing / room / building or a cooling system (e.g., an air-liquid heat exchanger) to reduce the temperature of the airflow. For example, the air supply system 135 generates a cool / cold air flow to circulate from the channel 103 through the electronic racks 110A-110N to carry away the exchanged heat.

[0045] A cold air flow enters the electronics rack through the front end of the electronics rack, and a warm / hot air flow leaves the electronics rack from the rear end of the electronics rack. The warm / hot air with the exchanged heat is exhausted from the room / building or cooled using a separate cooling system such as a liquid-to-air heat exchanger. Thus, the cooling system is a hybrid liquid-to-air cooling system, in which part of the heat generated by the processor is removed by a cooling liquid, for example, through a corresponding cold plate, while the remaining part of the heat generated by the processor (or other electronic device or processing device) is removed by air flow cooling.

[0046] Figure 2 is a schematic diagram showing an example of a cooling system according to one embodiment. Figure 2 , the electronic rack 201 is coupled to the cooling system 200 to receive the cooling liquid / fluid from the cooling system 200. The electronic rack 201 represents Figure 1Any of the electronic racks shown, for example, electronic racks 110A-110N. As shown, electronic rack 201 may include server chassis 211 and rack (or cooling liquid) manifold 213. Server chassis 211 may include liquid cold plate 212 (also referred to as a cold plate assembly) that is coupled to manifold 213 to receive cooling fluid from manifold 213 and return warm / hot fluid to manifold 213. Using the cooling fluid received from manifold 213, cold plate 212 (on which IT components of server chassis 211 are mounted) can remove heat generated by the IT components.

[0047] Continue to refer Figure 2 , the cooling system 200 may include, but is not limited to, an enhanced cooling unit 221, a condensing unit (or condenser, or external cooling unit) 222, a CDU 223, an air cooling unit 224, and an external cooling unit 225. The CDU 223 supplies cooling fluid to the rack 201 by supplying the fluid to the rack manifold 213 via three-way valves 242-243, wherein the rack manifold 213 delivers the fluid to the cold plate 212. The CDU 223 may also supply cooling fluid to the air cooling unit 224 (liquid-to-air heat exchanger), so that the air cooling unit 224 may use the supplied cooling fluid to cool recirculation air for the rack 201. For example, in one embodiment, the cooling liquid flows through the air cooling unit 224, and performs heat exchange with the warm / hot air exhausted from the rack 201 (server chassis 211), and circulates into the air cooling unit 224. Due to the heat exchange in the air cooling unit, the hot air is cooled, and cool / cold air may be generated and supplied to the rack 201 to cool the rack 201, while the warm / hot fluid is exhausted from the air cooling unit 224 to mix with the warm / hot fluid generated by the manifold 213. In one embodiment, the air cooling unit 224 may be integrated as part of the rack 201. It should be noted that the cooling airflow is not shown in the figure. The cooling air is supplied to the rack 201, and then the hot exhaust air enters the air cooling unit 224. The air cooling unit 224 supplies the cooling air back to the rack 201.

[0048] In some embodiments, the three-way valve 242 is configured to distribute different amounts of fluid to the rack manifold 213 and the air cooling unit 224. That is, the valve 242 can be used to adjust the cooling capacity delivery of different cooling parts, such as an air cooling part and a liquid cooling part. For example, on a liquid / fluid distribution circuit attached or coupled to the rack manifold 213, the three-way valve 243 is used to direct all or part of the cooling fluid to the enhanced cooling unit 221 (e.g., a liquid-liquid heat exchanger). The enhanced cooling unit 221 can be used to enhance the cooling capacity on the liquid distribution circuit or the entire cooling circuit. Within the enhanced cooling unit 221, the cooling fluid can be further cooled to a much lower temperature because the enhanced cooling unit 221 is cooled by a refrigeration circuit connected or coupled to the condensing unit 222. The cool / cold fluid delivered from the condensing unit 222 to the enhanced cooling unit 221 can then be delivered to the fluid cooling loop 250 via the two-way valve 244 for delivery to the rack manifold 213 (the valve 244 is preferably a unit in the system 200, although it may be optional in some embodiments, but it may also provide additional features for enhanced system control). Note that the condensing unit 222 can be any kind of commercially available or custom condensing unit or (condenser). Therefore, the details of the condensing unit 222 will not be described here. In one embodiment, the cool / cold fluid supplied by the enhanced cooling unit 221 can be mixed with the cold fluid supplied by the CDU 223 on the fluid cooling loop 250, and the fluid mixture is delivered to the manifold 213.

[0049] In one embodiment, the CDU 223 may include a heat exchanger 231 and a liquid pump 232. The CDU 223 includes a pump controller (not shown), as well as some other components such as a liquid reservoir, a power supply, monitoring sensors, etc. Note that the CDU 223 may be any type of commercially available or custom CDU. Thus, the details of the CDU 223 will not be described here. The heat exchanger 231 may be connected to the external cooling unit 225 via a cooling fluid loop. The heat exchanger 231 may receive cooling fluid from the external cooling unit 225 via a liquid pump 241. In one embodiment, heat from the warm fluid delivered by the manifold 213 (and delivered by the air cooling unit 224) may be transferred to the cold fluid before being extracted to the external cooling unit 225 while passing through the heat exchanger 231. The cooling fluid from the external cooling unit 225 is transferred to the liquid distribution loop using the liquid pump 232. In one embodiment, the heat exchanger 231 may be a liquid-liquid heat exchanger. In one embodiment, the external cooling unit 225 may be a cooling tower (open or closed loop), evaporative cooling coils, dry coolers, etc., external to the building / containment vessel.

[0050] Notice, Figure 2Only a simplified representation of the cooling system 200 and corresponding key structures is shown, wherein only one of each component (units 221-225) is shown. As previously mentioned, the data center system (data center system 100) may include multiple rows of electronic racks of IT components. Thus, the system 200 may include each of the multiple units 221-225 for cooling those rows of electronic racks.

[0051] Figure 3 is a schematic diagram showing another example of a cooling system according to one embodiment. Figure 3 In the cooling system 300 of FIG. 2 , the enhanced cooling unit 221 is integrated into the CDU 223. Three-way valves 342-343 are also included in the CDU 223 and are used to manipulate the fluid. For example, valve 342 is used in a similar manner to valve 242 by distributing different amounts of fluid to the rack manifold 213 and the air cooling unit 224. Thus, valve 342 can be used to adjust the cooling capacity delivery of different cooling sections, such as an air cooling section and a liquid cooling section. Similar to valve 243, three-way valve 343 is used to direct all or part of the cooling fluid distributed by valve 342 to the enhanced cooling unit 221 (as previously described and therefore not described again herein). However, in Figure 3 , the CDU 223 includes a plurality of inlet ports and outlet ports 361-367. For example, outlet port 361 connects the enhanced cooling unit 221 to the rack manifold 213, and outlet port 362 connects the valve 343 to the rack manifold 213. Ports 361-362 may independently supply cool / cold fluid to the manifold 213 simultaneously at different or the same fluid temperatures, although they may provide fluid in an alternating manner at different or the same fluid temperatures. In some embodiments, the fluids supplied by ports 361-362 are mixed in the manifold 213. In one embodiment, inlet port 363 connects the manifold 213 to the heat exchanger 231 to receive return warm / hot fluid from the manifold 213 and the air cooling unit 224. Outlet port 364 and inlet port 365 connect the cooling circuit between the heat exchanger 231 and the external cooling unit 325A, as previously described. Figure 2 The outlet port 366 and the inlet port 367 connect the refrigeration circuit between the enhanced cooling unit 221 and the external cooling unit 325B. In the cooling system 300, within the enhanced cooling unit 221, the cooling fluid can be further cooled to a lower temperature because the enhanced cooling unit 221 is cooled by being connected or coupled to the refrigeration circuit of the external cooling unit 325B. In some embodiments, the external cooling unit 325B can be a condensing unit (e.g., Figure 2 In one embodiment, each of the external cooling units 325A-B is Figure 2 The external cooling unit 225. Therefore, the details of the external cooling units 325A-B will not be described here.

[0052] Figure 4 is a schematic diagram showing an example of a cooling system with alternating cooling distribution units (CDUs) according to one embodiment. Figure 4 In the embodiment, the cooling system 400 is similar to Figure 3 The cooling system 300. However, Figure 3 The ports 361-362 of the CDU 223 are combined in the CDU 223 based on internal design variations within the CDU 223. That is, the three-way valve 442 can be combined with Figure 3 The valve 342 of FIG. 40 distributes the cool / cold fluid to the air cooling unit 224 and the three-way valve 443 in a similar manner. However, in the cooling system 400, the valve 443 is configured to distribute the fluid back to the enhanced cooling unit 221 (similar to Figure 3 462-467 is configured similarly to the valve 343 of FIG. 464 and also distributed back to port 461 (rather than directly distributed to manifold 213). Thus, port 461 can carry a fluid mixture of fluid supplied by enhanced cooling unit 221 and fluid supplied by valve 443. Figure 3 Therefore, the details of ports 462-467 are not described here. In some embodiments, Figure 4 The corresponding design of the rack manifold 213 in is different from Figure 3 The design of the rack manifold 213. For example, Figure 3 Manifold 213 may require additional inlet ports to receive cooling fluid for CDU 223 .

[0053] Figure 5 is a schematic diagram showing another example of a cooling system according to an embodiment. Figure 5 In the cooling system 500, an air cooling distribution circuit 503, a cooling fluid distribution circuit 504 and a cold fluid distribution circuit 505 are introduced. This will improve the flexibility of the cooling system design. Figure 5In the system 500, multiple CDUs 523A-B can be used to distribute fluid to the fluid cooling distribution loop 501. For example, in the system 500, the coolant distribution loop may include a fluid cooling distribution loop 501, a system return distribution loop 502, and an air cooling distribution loop 503. Loops 501-503 are used between the electronic rack 201 and the CDUs 523A-B. Connection ports are assembled on these distribution loops, and cooling units, CDUs, and other fluid loops are connected to these distribution loops. Cooling fluid distribution loops 504 and cold fluid distribution loops 505 are used between CDUs 523A-B and external cooling units 325A-B. In some embodiments, the fluids used in each of the loops 501-505 may be different from each other, thereby having different temperatures from each other. However, in other embodiments, the fluids used in some or all of the loops 501-505 may be the same fluid. Note that additional control valves or check valves may be used on the distribution loops not shown in the figure.

[0054] Note that in system 500, CDU 523A-B and external cooling unit 325A-B can be detachably coupled, and loops 501-505 are used as a pool for distributing and receiving fluid. External cooling unit 325A-B and CDU 523A-B can exchange fluid through the pool. Thus, this not only provides flexibility in redundant design, but also ease of service, while improving infrastructure cost efficiency.

[0055] Figure 6 is a schematic diagram showing an example of a cooling system and data center system layout with different parts according to one embodiment. Figure 6 In the embodiment, the system 500 is divided into five parts, namely the rack part (or IT part) 601, the auxiliary distribution part 602, the enhanced cooling part 603, the main loop part 604 and the cooling unit part 605. This provides the boundaries of different parts or modules in the system 500. The classification of such parts (or parts / modules) will simplify the system design and deployment process of the system 500, because the parts 601-605 can be designed and deployed separately and independently, thereby improving efficiency. This is also very important for developing system standardization, because for the module, only the capacity, or the connection method and some key parameters or specifications are needed to be connected to the entire system and work properly. In addition, this will significantly improve reliability, because most of the functions are components that are mobile encapsulated in the CDU (CDU523A-B). The CDU can be easily replaced and serviced in the system 500 without causing a major interruption in the continuous operation of the system. Using the main loop part 604, the external cooling unit 325B can be used to provide enhanced cooling capacity to the server chassis 211 in CDU 523A or CDU 523B or both. Similar functionality is provided by portion 602 for various other cooling units in the cooling circuit.

[0056] In some embodiments, the auxiliary distribution section 602 and the main loop section 604 are used as source pools to increase the resilience of the system. As an example, the refrigeration circuit (between the CDU and the external cooling unit) can be used for different enhancement circuits in the enhanced cooling section 603 in multiple CDU configurations. Valves (three-way valves 442-443) can be used on the distribution section 602 and the main loop section 604 to control the flow. Figure 6 Concepts for compact and efficient methods for designing cooling systems for hybrid (liquid-air) cooling racks are also provided. For example, rack section 601 can be considered a point of delivery (PoD). Thus, system 500 can be used for computing PoDs, storage PoDs, or heterogeneous high-performance computing PoDs, and a common enhanced cooling section (enhanced cooling section 603) can be used, dedicated to a single PoD or multiple PoDs.

[0057] Each part can be deployed independently of each other. This means that even if the corresponding design and specifications of each part can be provided by different suppliers, they can be integrated together and work properly.

[0058] In the foregoing description, embodiments of the present invention have been described with reference to specific exemplary embodiments of the present invention. It is apparent that various modifications may be made thereto without departing from the broader spirit and scope of the present invention as set forth in the appended claims. Therefore, the description and drawings are to be regarded as illustrative rather than restrictive.

Claims

1. A cooling system for an electronic rack in a data center, comprising: a coolant distribution unit CDU, connected to a rack manifold of the electronic rack through a fluid cooling loop, the CDU supplying cooling fluid distributed to the rack manifold and receiving warm / hot fluid returned from the rack manifold; a boost cooling unit for receiving a first portion of a first distributed portion of cooling fluid supplied by the CDU and further cooling the first portion of the first distributed portion of cooling fluid to a temperature lower than a temperature of cooling fluid supplied by a refrigeration circuit; a first external cooling unit connected to the CDU via a cooling fluid circuit to supply the cooling fluid to the CDU; as well as An air cooling unit is configured to receive a second distributed portion of the cooling fluid supplied by the CDU and to cool the electronic rack using the second distributed portion of the cooling fluid.

2. The cooling system according to claim 1, further comprising: a first three-way valve for receiving a supply of cooling fluid and distributing the supplied cooling fluid into the first distribution portion and the second distribution portion; as well as A second three-way valve is configured to receive the first distributed portion of the cooling fluid and deliver the first portion of the first distributed portion of the cooling fluid to the enhanced cooling unit through the fluid cooling circuit and deliver a second portion of the first distributed portion of the cooling fluid to the rack manifold.

3. The cooling system according to claim 2, wherein: The enhanced cooling unit supplies a further cooled first portion of the first distributed portion of the cooling fluid to a fluid distribution circuit. 4 . The cooling system according to claim 2 , further comprising a condensing unit connected to the enhanced cooling unit through the refrigeration circuit.

5. The cooling system according to claim 2, wherein: The enhanced cooling unit and the first and second three-way valves are included in the CDU.

6. The cooling system according to claim 5, wherein: The CDU further includes a heat exchanger for exchanging heat between the returning warm / hot fluid and the supplied cooling fluid and a liquid pump for delivering the supplied cooling fluid into the first three-way valve.

7. The cooling system according to claim 6, wherein: The CDU includes a plurality of inlet ports and an outlet port coupled to the enhanced cooling unit, the second three-way valve, and the heat exchanger.

8. The cooling system according to claim 7, wherein: The plurality of inlet ports and the outlet port include: connecting the enhanced cooling unit to a first outlet port of the rack manifold, connecting the second three-way valve to the second outlet port of the rack manifold, connecting the rack manifold to a first inlet port of the heat exchanger, connecting the cooling fluid circuit to the third outlet port and the second inlet port between the heat exchanger and the first external cooling unit, and The refrigeration circuit is connected between the enhanced cooling unit and the second external cooling unit at the fourth outlet port and the third inlet port.

9. The cooling system according to claim 7, wherein: The plurality of inlet ports and the outlet port include: connecting the enhanced cooling unit and the second three-way valve to the first outlet port of the rack manifold, connecting the rack manifold to a first inlet port of the heat exchanger, connecting the cooling fluid circuit to the second outlet port and the second inlet port between the heat exchanger and the first external cooling unit, and The refrigeration circuit is connected to a third outlet port and a third inlet port between the enhanced cooling unit and a second external cooling unit.

10. A cooling system for an electronic rack in a data center, comprising: a plurality of coolant distribution units (CDUs) coupled to rack manifolds of the electronics racks, the plurality of CDUs comprising: a first CDU, supplying a first cooling fluid distributed to the rack manifolds, and receiving a return warm / hot fluid from the rack manifold, the first CDU comprising a boost cooling unit that receives a first portion of the first distributed portion of the first cooling fluid and further cools the first portion of the first distributed portion of the first cooling fluid through a refrigeration circuit, and a second CDU for supplying a second cooling fluid distributed to the rack manifold on a fluid cooling distribution loop and receiving a return warm / hot fluid from the rack manifold, the second CDU comprising a boost cooling unit for receiving a first portion of the first distributed portion of the second cooling fluid and further cooling the first portion of the first distributed portion of the second cooling fluid through a cold fluid distribution loop, and An air cooling unit is configured to receive the second distributed portion of the first cooling fluid and the second distributed portion of the second cooling fluid and to cool the electronic rack through airflow using the second distributed portion of the first cooling fluid and the second distributed portion of the second cooling fluid.

11. The cooling system according to claim 10, further comprising a first external cooling unit connected to the first CDU through a cooling fluid circuit to supply the first cooling fluid to the first CDU, wherein The first external cooling unit is also connected to the second CDU through a cooling fluid distribution circuit to supply the second cooling fluid to the second CDU.

12. The cooling system of claim 11, further comprising a second external cooling unit connected to the enhanced cooling unit of the first CDU through the refrigeration circuit, wherein: The second external cooling unit is also connected to the enhanced cooling unit of the second CDU through the cold fluid distribution circuit.

13. The cooling system according to claim 10, wherein: The first CDU also includes a first three-way valve for receiving a supply of a first cooling fluid and distributing the supply of the first cooling fluid into the first distribution portion and the second distribution portion of the first cooling fluid.

14. The cooling system according to claim 13, wherein: The first CDU also includes a second three-way valve, which is configured to receive the first distributed portion of the first cooling fluid and deliver the first portion of the first distributed portion of the first cooling fluid to the enhanced cooling unit of the first CDU and deliver a second portion of the first distributed portion of the first cooling fluid to the rack manifold.

15. The cooling system according to claim 14, wherein: The second CDU further includes a first three-way valve for receiving a supply of a second cooling fluid and distributing the supply of the second cooling fluid into the first distribution portion and the second distribution portion of the second cooling fluid.

16. The cooling system according to claim 15, wherein: The second CDU also includes a second three-way valve, which is used to receive the first distribution portion of the second cooling fluid and deliver the first portion of the first distribution portion of the second cooling fluid to the enhanced cooling unit of the second CDU, and deliver the second portion of the first distribution portion of the second cooling fluid to the rack manifold on the fluid cooling distribution loop.

17. A data center system, comprising: Multiple independently deployable parts, including: The rack part includes: (i) an electronic rack that accommodates multiple server chassis, Each server chassis corresponds to one or more servers, wherein the electronic rack includes a rack manifold to receive cooling fluid and return warm / hot fluid from the electronic rack, and (ii) an air cooling unit for cooling the electronic rack, a secondary distribution portion, comprising: (i) a fluid cooling distribution loop for delivering the cooling fluid to the electronic rack, (ii) a system return distribution loop for delivering returned warm / hot fluid from the electronic rack, and (iii) an air cooling distribution loop for delivering a distribution portion of the cooling fluid to the air cooling unit, and An enhanced cooling section, comprising: (i) a first coolant distribution unit (CDU), and (ii) a second CDU for supplying a second portion of the cooling fluid to the rack manifold through the fluid cooling distribution circuit, receiving the second portion of the warm / hot fluid through the system return distribution circuit, and supplying a distributed portion of the cooling fluid to the air cooling unit through the air cooling distribution circuit.

18. The data center system according to claim 17, wherein: The multiple independently deployable parts also include: a main circuit portion, comprising: (i) a cooling fluid distribution circuit connecting the second CDU to a first external cooling unit, and (ii) a cooling fluid distribution circuit connecting the second CDU to a second external cooling unit, wherein the first external cooling unit supplies a first cooling fluid to the first CDU and a second cooling fluid to the second CDU, and the second external cooling unit supplies a first cold fluid having a lower temperature than the first cooling fluid to the first CDU and supplies a second cold fluid having a lower temperature than the second cooling fluid to the second CDU.

19. The data center system according to claim 18, wherein: The plurality of independently deployable sections also include a cooling unit section including the first external cooling unit and the second external cooling unit.

20. The data center system according to claim 17, wherein: The rack portion is the point of delivery (PoD).

Images