Information technology cooling system with fluid distribution configuration

Abstract

An embodiment of an apparatus comprising multiple IT units arranged in an information technology (IT) cluster is disclosed. Each IT unit includes an IT container paired with a corresponding cooler. The IT cluster includes a first row and a second row, each row having an upstream end and a downstream end, and includes one or more IT units positioned adjacent to and abutting each other. The cooler of each IT unit in each row is fluidly connected to the IT container of the next downstream IT unit in the same row via an intra-row fluid connector, or fluidly connected to the IT container of an IT unit in the second row via an inter-row fluid connector. The cluster includes at least one pair of inter-row fluid connectors, such that the pair of inter-row fluid connectors, the intra-row fluid connectors in the first row, and the intra-row fluid connectors in the second row form at least one fluid loop within the IT cluster. The internal loop and the external loop are arranged in different modular designs.

Description

Technical Field

[0001] The disclosed embodiments generally relate to liquid cooling systems for temperature control of electronic devices, and particularly, but not exclusively, to a cooling system having a fluid distribution configuration. Background Technology

[0002] Many modern information technology (IT) devices, such as servers, blade servers, routers, and edge servers, generate significant amounts of heat during operation. The heat generated by individual components (especially high-power components such as processors and System-on-a-Chip (SoC) comprising multiple small chips) makes it impossible or difficult to effectively cool many of these individual components using air-cooling systems. Therefore, modern IT equipment requires liquid cooling or a liquid-air hybrid cooling system.

[0003] Due to the need for liquid cooling, some IT equipment has onboard liquid cooling systems that are thermally connected to individual components requiring cooling. However, these onboard liquid cooling systems typically do not operate in isolation. They are usually connected to at least one larger cooling system, such as a liquid cooling system in an electronics rack, and the rack's cooling system may be further connected to the liquid cooling system of a larger facility, such as a data center. In such a system, the data center's cooling system circulates the working fluid through the rack cooling system, which in turn circulates the working fluid through the cooling systems on the IT equipment.

[0004] Previous solutions designed IT racks / containers and facility cooling infrastructure separately, meaning the cooling infrastructure and coolant loop connections were facility-side infrastructure. This design has long been used and adopted in industry, but it is inflexible and lacks the resilience required for modern IT infrastructure, especially with constantly changing IT configurations. Several existing solutions have proposed integrating cooling equipment with the rack, but each individual rack-level cooling system remains independent, meaning facility-level distribution and cooling infrastructure is still required. The resulting hardware and control systems are complex and costly. Summary of the Invention

[0005] An apparatus includes: a plurality of IT units arranged in an IT cluster, each IT unit including an IT container, the IT container being paired with and fluidly connected to a corresponding cooler adjacent to the IT container, each IT container being upstream of its corresponding cooler. The IT cluster includes: a first row having an upstream end and a downstream end, and including one or more IT units positioned adjacent to and abutting each other; and a second row having an upstream end and a downstream end, the second row being adjacent to the first row and including one or more IT units positioned adjacent to and abutting each other, wherein the cooler of each IT unit in the first row is fluidly connected to the IT container of the next downstream IT unit in the first row via an intra-row fluid connector, or fluidly connected to the IT container of one of the one or more IT units in the second row via an inter-row fluid connector.

[0006] According to some embodiments, the cooler of each IT unit in the second row is fluidly connected to the IT container of the next downstream IT unit in the second row via an intra-row fluid connector, or fluidly connected to the IT container of one of the one or more IT units in the first row via an inter-row fluid connector, and the IT cluster includes at least one pair of inter-row fluid connectors, wherein the at least one pair of inter-row fluid connectors, the intra-row fluid connectors in the first row, and the intra-row fluid connectors in the second row form at least one fluid loop within the IT cluster.

[0007] According to some embodiments, the first row and the second row are parallel to each other, and the upstream end of the first row is adjacent to the downstream end of the second row, and the downstream end of the first row is adjacent to the upstream end of the second row.

[0008] According to some embodiments, the at least one pair of inter-row fluid connectors includes a first inter-row fluid connector that fluidly connects the upstream end of the first row to the downstream end of the second row and a second inter-row fluid connector that fluidly connects the downstream end of the first row to the upstream end of the second row, thereby forming a single fluid loop in the IT cluster.

[0009] According to some embodiments, the at least one pair of inter-row fluid connectors includes two or more pairs of inter-row fluid connectors, wherein the two or more pairs of inter-row fluid connectors, the intra-row fluid connectors in the first row, and the intra-row fluid connectors in the second row form multiple fluid loops within the IT cluster.

[0010] According to some embodiments, each cooler includes: a cooler inlet fluidly connected to a corresponding IT container, an open-loop outlet fluidly connected to an intra-row or inter-row fluid connector, and a closed-loop outlet; a fluid bypass line having a first three-way valve and a second three-way valve connected therein, wherein the fluid bypass line is fluidly connected to a unit inlet and fluidly connected to the open-loop outlet and the closed-loop outlet via the second three-way valve; and a main cooling loop fluidly connected to the fluid bypass line.

[0011] According to some embodiments, the main cooling loop includes: a heat exchanger having an inlet and an outlet, the inlet of the heat exchanger being fluidly connected to the fluid bypass line at or near the unit inlet via a supply line, and the outlet of the heat exchanger being connected to the fluid bypass line between the first three-way valve and the second three-way valve via a return line; a pump fluidly connected to the return line; and a bidirectional line fluidly connected to the first three-way valve and upstream of the pump to the return line.

[0012] According to some embodiments, the heat exchanger of at least one cooler further includes an external inlet adapted to be connected to a facility supply line via a control valve and an external outlet adapted to be fluidly connected to a facility return line.

[0013] According to some embodiments, the heat exchanger of at least one cooler is a multiphase heat exchanger, the multiphase heat exchanger further comprising an external inlet adapted for connection to a facility supply line and an external outlet adapted for fluid connection to a facility steam return line.

[0014] A data center facility includes: one or more IT clusters and one or more data center cooling systems. Each IT cluster includes multiple IT units, each IT unit including an IT container, the IT container being paired with and fluidly connected to a corresponding cooler adjacent to the IT container, each IT container being upstream of its corresponding cooler. Each IT cluster includes: a first row having an upstream end and a downstream end, and including one or more IT units positioned adjacent to and abutting each other; and a second row having an upstream end and a downstream end, the second row being adjacent to the first row, and including one or more IT units positioned adjacent to and abutting each other, wherein the cooler of each IT unit in the first row is fluidly connected to the IT container of the next downstream IT unit in the first row via an intra-row fluid connector, or fluidly connected to the IT container of one of the one or more IT units in the second row via an inter-row fluid connector. Each data center cooling system includes a facility heat exchanger connected to at least one facility loop, the facility loop including facility supply lines and facility return lines, wherein each cooler in the one or more IT clusters is connected to the facility supply lines and facility return lines of one of the one or more data center cooling systems.

[0015] According to some embodiments, the cooler of each IT unit in the second row is fluidly connected to the IT container of the next downstream IT unit in the second row via an intra-row fluid connector, or fluidly connected to the IT container of one of the one or more IT units in the first row via an inter-row fluid connector, and the IT cluster includes at least one pair of inter-row fluid connectors, wherein the at least one pair of inter-row fluid connectors, the intra-row fluid connectors in the first row, and the intra-row fluid connectors in the second row form at least one fluid loop within the IT cluster.

[0016] According to some embodiments, pairs of coolers in each cluster share one of the one or more data center cooling systems, such that pairs of coolers, including one cooler from the first row and one cooler from the second row, are connected to the facility supply line and the facility return line of the same facility loop.

[0017] According to some embodiments, the one or more data center cooling systems include a first data center cooling system and a second data center cooling system, wherein the first data center cooling system uses a first cooling fluid and is fluidly connected to a subset of coolers in the cluster, and the second data center cooling system uses a second cooling fluid and is connected to another subset of coolers in the cluster.

[0018] According to some embodiments, paired clusters share one of the one or more data center cooling systems, such that coolers in the first row of the first cluster and coolers in the second row of the second cluster are fluidly connected to a single facility loop.

[0019] According to some embodiments, each cluster includes an integrated distribution module having supply and return lines adapted for fluid connection to at least one facility heat exchanger.

[0020] According to some embodiments, each cooler is fluidly connected to a single facility heat exchanger via supply and return lines.

[0021] According to some embodiments, the supply line for each cooler is fluidly connected to the facility supply line via a control valve.

[0022] According to some embodiments, each cooler includes a temperature sensor to sense the outlet temperature of the cooler, and the control valve is set based on the sensed outlet temperature.

[0023] According to some embodiments, at least one cluster includes two or more pairs of inter-row fluid connections, wherein the two or more pairs of inter-row fluid connections, the intra-row fluid connections in the first row, and the intra-row fluid connections in the second row form multiple fluid loops within the IT cluster.

[0024] According to some embodiments, at least one cooler includes: a cooler inlet fluidly connected to a corresponding IT container, an open-loop outlet fluidly connected to an intra-row or inter-row fluid connection, and a closed-loop outlet; a fluid bypass line having a first three-way valve and a second three-way valve connected therein, wherein the fluid bypass line is fluidly connected to a unit inlet and fluidly connected to the open-loop outlet and the closed-loop outlet via the second three-way valve; and a main cooling loop fluidly connected to the fluid bypass line. The main cooling loop includes: a heat exchanger having an inlet and an outlet, the inlet of the heat exchanger being fluidly connected to the fluid bypass line at or near the unit inlet via a supply line, and the outlet of the heat exchanger being connected to the fluid bypass line between the first three-way valve and the second three-way valve via a return line; a pump fluidly connected to the return line; and a bidirectional line fluidly connected to the first three-way valve and upstream of the pump to the return line. Attached Figure Description

[0025] Non-limiting and non-exhaustive embodiments of the invention are described with reference to the following figures, wherein similar reference numerals refer to similar parts throughout the various views unless otherwise stated.

[0026] Figure 1 This is a block diagram of an embodiment of an IT container, which includes an electronic rack housing electronic components and a cooling system.

[0027] Figure 2 This is a block diagram of an embodiment of an IT unit, including an IT container and its corresponding cooler or cooling unit.

[0028] Figures 3A to 3B This is a diagram of an embodiment of the cooling system; Figure 3A It's a floor plan. Figure 3B It is a side view.

[0029] Figures 4A to 4C This is a side view of an embodiment of a modular cooling system.

[0030] Figures 5A to 5B This is a plan view of an embodiment of the cooling system.

[0031] Figures 6A to 6C This is a plan view of an embodiment of a cooling system and the modular configurations that can be formed therein.

[0032] Figure 7 This is a plan view of an embodiment of the cooling system.

[0033] Figure 8 This is a flowchart of an embodiment of a process for controlling the flow of cooling fluid between an internal recirculation loop and an external loop. Detailed Implementation

[0034] An embodiment of a multi-loop cooling configuration for high-density server racks is described. Specific details are described to provide an understanding of the embodiments; however, those skilled in the art will recognize that the invention can be practiced without one or more of the described details or using other methods, components, materials, etc. In some instances, well-known structures, materials, or operations are not shown or described in detail, but are still included within the scope of the invention.

[0035] Throughout this specification, references to "one embodiment" or "embodiment" mean that the described features, structures, or characteristics may be included in at least one of the described embodiments; therefore, the appearance of "in one embodiment" or "in an embodiment" does not necessarily refer to the same embodiment. Furthermore, in one or more embodiments, specific features, structures, or characteristics may be combined in any suitable manner. As used herein, directional terms such as "front," "rear," "top," "bottom," "side," "lateral," "longitudinal," etc., refer to the orientation of the embodiments as they are presented in the drawings, but no directional term should be construed as implying or requiring any particular orientation of the described embodiments in actual use.

[0036] Modular design is becoming increasingly important and popular, especially for many application scenarios. Different approaches exist for designing IT clusters using the modular concept. The modular concept is no longer limited to IT containers; it can be used for any part of a system or partition of a cluster. Developing modular cooling and power modules, as well as combined modules of IT modules, to achieve greater system flexibility and resilience is becoming increasingly important. In particular, modular liquid-cooled architectures are needed to support IT clusters or IT containers filled with liquid-cooled IT equipment or liquid-cooled units between IT systems. The described embodiments provide designs for configuring different types of liquid-cooled distribution clusters for different scenarios. This is crucial for the ever-increasing power density generated within IT racks.

[0037] Examples of IT infrastructure designs with novel cooling loop management configurations are described. These examples are used to design and develop modular compute and storage clusters, either for hyperscale data center buildings or in container data center systems. The examples primarily focus on the system architecture and corresponding solutions for implementing IT racks and cooling hardware, particularly heat transfer systems and fluid management designs to extract heat loads from each IT rack to the outside of the cluster. The examples include cooling units integrated with IT containers, forming part of a fluid transport loop. Multiple such components are then combined to create different types of internal recirculation loops. Each internal recirculation loop is formed within the IT cluster. The cooling units are connected to external loops to deliver cooling capacity to the internal loops. The internal loops are configured differently depending on system layout, modular layout, and IT layout. Furthermore, the internal loops can be dynamically adjusted before and after deployment throughout the cluster, allowing for different coolant recirculation between IT clusters. Modular configuration schemes are also proposed for both external and internal layouts and applications. Flow, pressure, and temperature are controlled according to IT requirements.

[0038] A simplified system design offers several benefits, one of which is rapid deployment. The disclosed embodiments integrate complex fluid systems and management into an IT shell, rather than designing the system as a separate facility infrastructure. This design simplifies deployment, and configuration and thermal system layout can be dynamically adjusted. Hardware and system configurations may need to be tailored to different use cases and application scenarios. Especially in cloud computing, system configurations may need to be tailored to customer requirements. Therefore, infrastructure resilience requires not only the ability to support different types of hardware deployments and operations, including power and cooling requirements, but also the ability to support system-wide PoD / cluster deployments and system reconfigurations.

[0039] As mentioned earlier, modular design is important, and the methodology for designing and deploying modules is paramount. The disclosed embodiments are intended to provide effective modular design and methodology.

[0040] The disclosed embodiments offer flexible configuration for IT racks, cooling hardware, and fluid management while eliminating the need for primary facility-level cooling infrastructure for internal fluid distribution. This significantly simplifies facility-level design for heat transfer loops. This design and operation approach enables high flexibility and resilience in system loop configuration and fluid management; rack cluster grouping; and modular design methods and system partitioning layouts. Another proposed feature is that internal loops within the cluster are no longer completely separated and independent of the facility, but rather formed by IT containers and their corresponding cooling units. Furthermore, the internal loop scheme can be dynamically reconfigured as needed. The design can be easily adapted through modular design. In addition, the current design provides robust system redundancy to cope with different failure scenarios or other anomalies, and another innovation is the simplification of room facility-side design to form different heat transfer loops in the cooling system.

[0041] Figure 1 This is a block diagram showing a side view of an embodiment of an electronic rack, a type of IT container commonly used in data centers. In one embodiment, the electronic rack 100 includes a CDU 101, a rack management unit (RMU) 102, and one or more server blades 103A-103D (collectively referred to as server blades 103). The server blades 103 can be inserted into an array of server slots from the front end 904 of the electronic rack 100. Note that although only four server blades 103A-103D are shown, more or fewer server blades may be held within the electronic rack 100. Also note that the specific locations of the CDU 101, CMU 102, and server blades 103 are shown for illustrative purposes only; other arrangements or configurations of the CDU 101, CMU 102, and server blades 103 may also be implemented. Furthermore, a front door located at the front end 104 and a rear door located at the rear end 105 are optional. In some embodiments, there may be no doors at the front end 104 and / or the rear end 105.

[0042] In one embodiment, CDU 101 includes a heat exchanger 111, a liquid pump 112, and a pump controller 110. The heat exchanger 111 may be a liquid-to-liquid heat exchanger. The heat exchanger 111 includes a first tube having a first pair of liquid connectors coupled to an external liquid supply line 132 / return line 131 to form a main loop, wherein the connectors coupled to the external liquid supply line 132 / return line 131 may be disposed on or mounted on the rear end 105 of the electronics rack 100. Furthermore, the heat exchanger 111 includes a second tube having a second pair of liquid connectors coupled to a liquid manifold 125, which may include a supply manifold for supplying coolant to the server blade 103 and a return manifold for returning warmer liquid to the CDU 101. The processor may be mounted on a cold plate, wherein the cold plate includes embedded liquid distribution channels to receive coolant from the liquid manifold 125 and return coolant carrying heat exchanged from the processor to the liquid manifold 125. Rack 100 is an example of an IT container, which can be used with, for example, Figure 2 The cooling units shown are connected to form an IT unit, which can then be combined with other IT units to form a cluster, such as... Figure 3A As shown in the figure.

[0043] Each server blade 103 may include one or more IT components (e.g., CPU, GPU, memory, and / or storage devices). Each IT component can perform data processing tasks, wherein the IT component may include software installed on storage devices, loaded into memory, and executed by one or more processors to perform data processing tasks. Server blade 103 may include a host server (referred to as a host node) connected to one or more compute servers (also referred to as compute nodes). The host server (having one or more CPUs) typically interfaces with clients via a network (e.g., the Internet) to receive requests for specific services such as storage services (e.g., cloud-based storage services such as backup and / or recovery), and to execute applications to perform certain operations (e.g., image processing, deep data learning algorithms, or modeling, as part of a Software as a Service or SaaS platform). In response to the request, the host server assigns the task to one or more compute servers (having one or more GPUs) managed by the host server. The compute servers perform the actual tasks, which may generate heat during operation.

[0044] The electronic rack 100 also includes an RMU 102, which is configured to provide and manage power supplied to the server blades 103 and CDU 101. The RMU 102 may be coupled to a power supply unit (not shown) to manage the power supply unit's power consumption and other thermal management features (e.g., cooling fans). The power supply unit may include necessary circuitry (e.g., AC-to-DC or DC-to-AC power converters, batteries, transformers, or regulators) to provide power to the remaining components of the electronic rack 100.

[0045] In one embodiment, RMU 102 includes optimal control logic 111 and rack management controller (RMC) 122. Optimal control logic 111 is coupled to at least some of the server blades 103 to receive the operational status of each of the server blades 103, such as processor temperature, current pump speed of liquid pump 112, and coolant temperature. Based on this information, optimal control logic 111 determines the optimal pump speed of liquid pump 112 by optimizing a predetermined objective function, such that the output of the objective function is maximized while satisfying a set of predetermined constraints. Based on the optimal pump speed, RMC 122 is configured to send a signal to pump controller 110 to control the pump speed of liquid pump 112 based on the optimal pump speed.

[0046] Figure 2 An embodiment of an IT unit 200 is shown, which includes a cooler or cooling unit 202 fluidly connected to an information technology (IT) container 232 to cool components within the IT container. As the name suggests, the IT container 232 contains heat-generating IT components and includes a container flow path 233 through which cooling fluid entering the IT container 232 is delivered to the heat-generating components. The container flow path 233 can be understood as a cooling fluid management design for the IT container. Figure 1An embodiment of container flow path 233 in a data center rack is shown, but other embodiments of the IT container 232 may have different container flow paths than those shown. The cooling unit 202 is not limited to any particular container flow path but can be used with any flow path within the IT container 232. The IT container 232 also includes an inlet 236 and an outlet 238, and the container flow path 233 is fluidly connected to both inlet 236 and outlet 238 via fluid lines. The IT container 232 also includes another closed-loop container inlet 240, which is fluidly connected to a closed-loop flow path 244 between the closed-loop container inlet 240 and a location upstream of the container flow path 233, located between the container inlet 236 and the inlet of the container flow path. Similarly, the cooling unit 202 includes an inlet 204 connected to a bypass line B and an outlet 242 fluidly connected to a closed-loop outlet 208. The fluid lines within the IT container 232 may be made of rigid tubing, flexible tubing, or both, and may be pre-installed within the IT container to make it more modular.

[0047] IT container 232 is positioned upstream of cooling unit 202, with its container outlet 238 fluidly coupled to unit inlet 204 and its closed-loop container inlet 240 fluidly coupled to outlet 242 (if it will be used in closed-loop mode). In one embodiment, the fluid connections at container inlet 236, between container outlet 238 and unit inlet 204, and between outlet 242 and closed-loop container inlet 240 can be achieved using quick-release fittings or blind-fit fittings, but other embodiments may of course use different fluid connections.

[0048] The cooler or cooling unit 202 is fluidly connected to its corresponding upstream IT container 232, and may further be connected to a downstream IT container (not shown in the figure, but see, for example) Figures 3A to 3BA fluid connection is established to cool the electronics within both upstream and downstream IT containers. Cooling unit 202 includes a unit inlet 204 and two cooling outlets: an open-loop outlet 206 and a closed-loop outlet 208. Within cooling unit 202, a bypass line B fluidly connects inlet 204 to a second three-way valve V2. Valve V2, in turn, fluidly connects bypass line B to both open-loop outlet 206 and closed-loop outlet 208, allowing valve V2 to be used to switch between open-loop and closed-loop configurations. A first three-way valve V1 is also fluidly connected to bypass line B between inlet 204 and valve V2. Both three-way valves V1 and V2 have three fluid ports, numbered #1 to #3 in the figure. In both three-way valves V1 and V2, fluid can be directed from any port to one or both of the other ports; the valve can also change the proportion of flow entering each outlet port as it directs flow from one port to two other ports. Furthermore, in each valve V1 and V2, the fluid flow from one port to the other can be independently regulated and controlled with minimal impact on the other port, and in one operating mode, all three ports of the valve can be closed.

[0049] Cooling unit 202 also includes a main loop or cooling loop fluidly connected to bypass line B. The main loop includes a heat exchanger (HX) 210 having an inlet that is fluidly connected to bypass line B at or near inlet 204 via supply line S. A return line R fluidly connects the outlet of heat exchanger 210 to bypass line B between first valve V1 and second valve V2; in this arrangement, supply line S, heat exchanger 210, return line R, and a portion of bypass line B form the main cooling loop. Pump 212 is fluidly connected to return line R to circulate cooling fluid through at least a portion of the main loop and bypass line. In the illustrated embodiment, heat exchanger 210 also includes an external inlet and an external outlet. The external outlet may include a valve 211 for opening and closing the external outlet. Additionally, valve 211 can be used to regulate flow rate. In another embodiment, the pump can be used to regulate fluid flow rate. Both the external outlet and external inlet can be connected to the data center cooling system to supply cooling fluid to the heat exchanger 210. The external inlet is connected to the facility supply source, and the external outlet is connected to the facility return. Other embodiments of the cooling unit 202 may be completely self-sufficient and do not require the use of an external inlet or external outlet. Other embodiments of the cooling unit 202 may be self-sufficient in terms of liquid cooling, but still use an external cooling source, such as airflow through the cooling unit.

[0050] The bidirectional line 214 is fluidly connected to the return line R upstream of the three-way valve V1 and pump 212. Fluid can flow bidirectionally through the bidirectional line 214: from the return line R to valve V1, or from valve V1 to the return line R in the other direction, depending on the unit's operating mode. The bypass line B, together with the bidirectional line 214 and the three-way valve V1, is designed to handle abnormal operating modes, thereby improving the system's robustness, maintainability, and flexibility. The bidirectional line 214, as well as the bypass line B, the supply line S, and the return line R, can all be made of flexible tubing, rigid tubing, or some combination of both.

[0051] In addition to the aforementioned flow hardware, cooling unit 202 may include various sensors to monitor conditions within the unit. Cooling unit 202 includes a flow meter M1, a pressure sensor P1, and a temperature sensor T, although other embodiments may include additional or different sensors for measuring these or other quantities within the unit. In the illustrated embodiment, flow meter M1 is located near inlet 204, while pressure sensor P1 and temperature sensor T are located downstream of the return line R in bypass line B where it connects to the bypass line—specifically, in this embodiment, between the location where return line R fluidly connects to bypass line B and three-way valve V2. In other embodiments, the sensors may be located differently (e.g., see Figure 3). The various sensors in unit 202 are communicatively coupled to control system 216, and control system 216 is also communicatively coupled to the flow hardware within unit 202 (in this embodiment, pump 212 and three-way valves V1 and V2), which can be controlled during operation. The dashed lines in the figure illustrate the signal connections between the sensors, control system, and flow hardware. Therefore, in the illustrated embodiment, the control system 216 can change the speed of the pump 212 and the positions of the three-way valves V1 and V2 based on inputs from the flow meter M1, the pressure sensor P1, and the temperature sensor T, as well as any additional or different sensors that may be present.

[0052] In operation, cooling unit 202 can operate in closed-loop, open-loop, or hybrid closed-loop / open-loop modes, primarily depending on the setting of three-way valve V2. When valve V2 directs fluid from port #1 to port #2, it forms a closed loop within the IT housing; when V2 directs fluid from port #1 to port #3, it forms an open loop connecting the unit to a larger recirculation system. Three-way valve V1, used in conjunction with bidirectional line 214, is primarily used to regulate the internal operation of cooling unit 202.

[0053] Cooling fluid enters IT container 232, for example, from another cooling unit (not shown) located upstream, through inlet 236. The cooling fluid flows through container flow path 233, where it absorbs heat from IT components and electronics, and the now-hot cooling fluid exits the IT container through outlet 238. As it exits outlet 238, the now-hot cooling fluid enters cooling unit 202 through inlet 204 and flows into heat exchanger 210 via supply line S, where the fluid is cooled. After leaving the heat exchanger, it flows through return line R, pump 212, and bypass line B, and the now-cooled fluid flows to valve V2, which is configured to direct the fluid through port #2 into closed-loop outlet 208. Closed-loop outlet 208 is fluidly coupled to cooling unit outlet 242, such that the now-cooled fluid flows through the closed-loop outlet to unit outlet 242. The fluid flows to unit outlet 242 and is received into IT container 232 through inlet 240, and then directed back into container flow path 233. Then, in this closed-loop mode, cooling unit 202 recirculates the cooling fluid to cool the upstream IT container 232. The fluid line connecting inlet 240 and container flow path 233 can be assembled separately or considered as part of container flow path 233. This fluid line functions similarly to the closed-loop outlet 208: completing the closed loop for the IT container and cooling unit.

[0054] Figures 3A to 3B An embodiment of the cooling system 300 is shown together. Figure 3A It's a floor plan. Figure 3B This is a side view. The cooling system 300 includes at least one cluster 302 of IT units arranged in a first row (row 1) and a second row (row 2). In the illustrated embodiment, these rows are positioned so that the back faces of the IT units face each other, but in other embodiments, the rows may be positioned differently relative to each other. Row 1 has an upstream end and a downstream end, and row 2 similarly has an upstream end and a downstream end. In the illustrated embodiment, the upstream end of row 1 is positioned next to the downstream end of row 2, and the downstream end of row 1 is positioned next to the upstream end of row 2, but in other embodiments, the row arrangement may differ from the arrangement shown.

[0055] Row 1 includes one or more IT units 304, and each IT unit includes an IT housing 306 and a corresponding cooler or cooling unit 308. Therefore, the illustrated embodiment includes N IT units 304_1 to 304_N, including N IT containers 306_1 to 306_N, each IT container having its corresponding cooler 308_1 to 308_N, where N ≥ 1. The IT units 304 are arranged in series within this row such that all IT units have the same orientation, with each IT container 306 upstream of its corresponding cooler 308. As a result, row 1 has an IT container 306 at its upstream end, a cooler 308 at its downstream end, and alternates between IT containers and coolers. Row 2 includes one or more IT units 310, each IT unit including an IT housing 312 and a corresponding cooler or cooling unit 314. Therefore, the illustrated embodiment includes M IT units 310_1 to 310_M, including M IT containers 312_1 to 312_M, each IT container having its corresponding cooler 314_1 to 314_M, where M ≥ 1. IT units 304 are arranged in series within this row, with all IT units in the same orientation, each IT container 312 upstream of its corresponding cooler 314. Consequently, row 2 has an IT container 312 at its upstream end and a cooler 314 at its downstream end. In the illustrated embodiment, both row 1 and row 2 have the same number of IT units (i.e., M = N), but in other embodiments, rows do not need to have the same number of IT units (i.e., M ≠ N).

[0056] In addition to the upstream fluid connection to its corresponding IT container, each cooler in rows 1 and 2 also has a downstream fluid connection to another IT container, for example via its open-loop outlet (see [link to relevant documentation]). Figure 2 The downstream fluid connection can be an intra-row fluid connection (i.e., a fluid connection to a downstream IT container in the same row) or an inter-row fluid connection (i.e., a fluid connection to an IT container in another row). Therefore, in row 1, fluid connections 316a and 316b are intra-row fluid connections that fluidly connect cooler 308_1 to downstream IT container 306_2, while fluid connections 318a and 318b are inter-row fluid connections that fluidly connect cooler 308_N to downstream IT container 312_1. Similarly, in row 2, fluid connections 320a and 320b are intra-row fluid connections that fluidly connect cooler 314_1 to IT container 312_2, while fluid connections 322a and 322b are inter-row fluid connections that fluidly connect cooler 314_M to IT container 306_1. Each of fluid connections 316, 318, 320, and 322 is shown in the figure as a pair of connections (e.g., 316a and 316b) to illustrate the cooling and bypass flow paths through the cooler (see Figure 1). Figure 2However, in practice, both flow paths can be implemented using a single physical fluid connector. Each cluster 302 includes at least one pair of inter-row fluid connectors, such that in each cluster, the IT container and its corresponding cooler, together with the intra-row and inter-row fluid connectors, form at least one recirculation loop 324 within the cluster. The recirculation loop 324 circulates the cooling fluid entirely within the cluster. Other embodiments of cluster 302 may include more than one pair of inter-row fluid connectors, such that more than one recirculation loop can be formed within the cluster (e.g., see...). Figure 5B The internal loop does not require any other facilities.

[0057] As in Figure 3B The most clearly visible, besides the fluid connections to their respective IT containers and downstream IT containers, are the heat exchangers 210 within each cooler (see Figure 2 Therefore, the internal recirculation loop 324 can be fluidly connected to one or more facility heat exchangers via an external fluid loop. Thus, in the illustrated embodiment, each cooler 308 is fluidly connected to its corresponding facility heat exchanger 326 via supply line 328 and return line 330: cooler 308_1 is connected to facility heat exchanger 326_1 via supply line 328_1 and return line 330_1, cooler 308_2 is connected to facility heat exchanger 326_2 via supply line 328_2 and return line 330_2, and so on. The coolers in row 2 (not visible in this figure because they follow row 1) are similarly connected to facility heat exchangers. In one embodiment, for each cluster, a one-to-one correspondence between coolers and facility heat exchangers may exist, meaning each cooler is fluidly connected to its own facility heat exchanger. However, in other embodiments, a many-to-one correspondence may exist between coolers and facility heat exchangers; for example, a cooler from row 1 and a cooler from row 2 may both be fluidly connected to a single facility heat exchanger and thus share that single facility heat exchanger. Other embodiments may have a one-to-many correspondence between coolers and facility heat exchangers (i.e., each cooler is fluidly connected to multiple facility heat exchangers), or a many-to-many correspondence between coolers and facility heat exchangers (i.e., each cooler is fluidly connected to multiple facility heat exchangers, and each facility heat exchanger is fluidly connected to multiple coolers). It can be seen that only an external loop is needed for the entire facility side. Each independent external cooling unit fills the top of the IT cluster. This differs from previous solutions, where the fluid flowing through the top of the rack was an internal loop.

[0058] In one embodiment, all coolers in a row or cluster and their corresponding facility heat exchangers are of the same type, i.e., single-phase or multi-phase. However, in other embodiments, all coolers in a row or cluster and their corresponding facility heat exchangers do not need to be of the same type. For example, in system 300, a group 332 of one or more coolers 308 and facility heat exchangers 326 may be a multi-phase heat exchanger (e.g., liquid supply and vapor return, as shown in the figures for different pipeline types for supply and return), while the remainder of heat exchangers 326 may be a single-phase heat exchanger (e.g., liquid supply and liquid return, as shown in the figures for the same pipeline type for supply and return). Therefore, within each cluster, there may be sub-clusters that are cooled differently. Typically, the heat exchangers in each cooler (see...) Figure 2 The cooling unit should be compatible with its corresponding facility heat exchanger: if the cooler's heat exchanger is single-phase, then the corresponding facility heat exchanger must also be single-phase, and so on. External cooling units can be configured individually based on fluid requirements. This is achieved by forming an internal loop within the cluster. In this design, different types of external cooling fluids can be used in different parts of the cluster, and the internal loop design described herein simplifies the corresponding infrastructure for implementing this implementation. Similarly, the internal loop runs between the IT housing and the cooling units and is formed within multiple combined IT housings and cooling units. Each cooling unit serves the IT container connected to it and extracts heat to the external loop connected to it. Therefore, individual cooling unit control is crucial for maintaining both the recirculation and operating conditions of the internal loop.

[0059] Figures 4A to 4C Other embodiments of the cooling system are shown. Figure 4A An embodiment of system 400 is shown, wherein the external cooling loops coupled to the coolers in the cluster are a shared central distribution loop. In system 400, each cooler 308 in row 1 of a cluster such as cluster 302 is fluidly coupled to the facility cooling system. As in system 300, in system 400, each cooler 308 has a supply line 328 and a return line 330; however, in system 400, each supply line also includes a control valve 408 to control the flow between the facility line and the cooler. The supply temperature of each cooling unit is used to control the valve (e.g., valve 408) on the external loop, and the valve opening controls the flow rate in the external loop. This is the cooling fluid in the heat exchanger within the cooling unit. Each cooler has a heat exchanger for exchanging heat between the internal and external loops (see [link]). Figure 2 This control design ensures that the temperature supplied by the cooling unit remains constant by controlling the cooling fluid delivered to the cooling unit.

[0060] For each cooler 308, its supply line 328 is fluidly connected to facility supply line 402 via its control valve 408. Each return line 330 is fluidly connected to facility return line 404. Facility supply line 402 and facility return line 404 are in turn fluidly connected to facility heat exchanger 406. The coolers in row 2 (not visible in this figure because they are after row 1) may be similarly connected to the facility supply and return lines. In one embodiment, the coolers in row 2 are fluidly connected to the same facility supply and return lines as the coolers in row 1, but in other embodiments, the coolers in row 2 may be fluidly connected to different facility supply and return lines than the coolers in row 1. In another embodiment, facility supply line 402 and facility return line 404 may be integrated into cluster 302 such that fluid connections 328 and 330 are both present within the cluster, and the cluster only needs to be connected to facility heat exchanger 406, thus requiring only one facility connection.

[0061] Figure 4B Another embodiment of cooling system 425 is shown. Cooling system 425 illustrates how a cooling system can be partitioned, modularized, and integrated. As in systems 300 and 400, the coolers in cluster 302 are fluidly connected to distribution module 426 via supply and return pipelines. Distribution module 426 is in turn connected to facility cooling unit 428. Through this modularity, system integration requires only two facility-level connections to prepare the entire system for operation: a connection 432 between each facility cooling unit 428 and distribution module 426, and a set of connections 430 between distribution module 426 and the individual coolers in each cluster. Due to this arrangement, the module also includes partially internal loops and fully external loops, or fully internal loops and partially external loops. The internal loops can be dynamically adjusted with limited impact on IT or the external loops.

[0062] Figure 4CAnother embodiment of cooling system 450 is shown. Cooling system 450 illustrates how a cooling system can be partitioned and modularized. As in systems 400 and 425, coolers 308 in cluster 302 are fluidly coupled to distribution module 452 via supply and return pipelines. Distribution module 452 is in turn coupled to a pair of facility cooling units 454 and 456. Through this modularity, system integration requires only four facility-level connections. The first two connections are connection 458 between facility cooling unit 454 and distribution module 426, and connection 460 between facility cooling unit 456 and distribution module 452. The next connection is a set of connections 462 between distribution module 452 and coolers 308 using cooling unit 454, and the last connection is a set of connections 464 between distribution module 452 and coolers that can use one or both of cooling units 454 and 456. In one embodiment, cooling units 454 and 456 use different cooling fluids, but in other embodiments, they can use the same cooling fluid. In other embodiments, cooling units 454 and 456 may use cooling fluids of different phases; for example, cooling unit 454 may be a liquid-phase cooling unit, while cooling unit 456 may be a vapor-phase cooling unit (e.g., a condenser). Due to this arrangement, the module also includes partially internal loops and fully external loops, or fully internal loops and partially external loops. The internal loops can be dynamically adjusted with limited impact on the IT or external loops. It can be seen that this design enables a very compact architecture for implementing multiple heat transfer loops in a data center, eliminating any facility-side dependencies while allowing for modular design.

[0063] For systems 400, 425, and 450, by adjusting the individual pump 212 within each cooler (see...) Figure 2 The flow rate within the internal loop remains the same. An efficient method involves regulating the pumps to the same constant speed within each cluster. Figure 8 The flowchart shown controls the supply temperature to ensure that the appropriate fluid temperature is maintained in the internal loop and to deliver the fluid to each IT enclosure according to design and operational requirements.

[0064] Figures 5A to 5BAnother embodiment of the cooling system 500 is shown. The cooling system 500 includes a facility 501 in which at least one cluster 302 is positioned between rows 502 and 504 of IT units. In one embodiment, rows 502 and 504 may be separate rows of IT units, but in other embodiments, they may be part of a cluster, the other half of which is not shown in the figure (but see, for example, Figure 6); for example, in one embodiment, row 502 may be the first row of a second cluster similar to cluster 302, while row 504 may be the second row of a third cluster similar to cluster 302. The IT units in cluster 302 and rows 502 and 504 may be as described above. Figure 2 As described, each IT unit includes an IT container and a cooler. Cluster 302 includes a pair of inter-row fluid connectors, forming a single recirculation loop 324 within the cluster.

[0065] Cooling system 500 includes a pair of cooling units (e.g., heat exchangers (HX)) 506 and 512 located outside facility 501 to transfer heat from cooling fluid circulating through cluster 302 and rows 502 and 504 to the atmosphere. Heat exchanger 506 is fluidly coupled to supply loop 508 and return loop 510. Supply loop 508 and return loop 510 are located between row 502 and one row of cluster 302; supply loop 508 is fluidly coupled via control valves to one or more coolers in row 502 and one or more coolers in one row of cluster 302, and the return loop is fluidly coupled to one or more coolers in row 502 and one or more coolers in one row of cluster 302. Similarly, heat exchanger 512 is fluidly coupled to supply loop 514 and return loop 516. Supply loop 514 and return loop 516 are located between row 504 and another row of cluster 302; supply loop 514 is fluidly connected via control valves to one or more coolers in row 504 and one or more coolers in another row of cluster 302, and return loop 516 is fluidly connected to one or more coolers in row 504 and one or more coolers in another row of cluster 302. Cluster 302 is formed by two IT rows, as circled by dashed lines. External loops and service areas are arranged on the opposite side of this row. In this embodiment, the cooling capacity provided by the two cooling units can be distributed by two external recirculation loops. Cluster 302 receives two different external cooling fluid sources, which can be beneficial in cases such as cooling unit or external loop failure. In the illustrated embodiment, system 500 can be considered to be divided into two modules. First module 518 includes heat exchanger 506, supply line 508 and return line 510, and the upper rows of row 502 and cluster 302. The second module 520 includes a heat exchanger 512, a supply line 514 and a return line 516, a row 504 and a downstream of cluster 302. In the illustrated embodiment, modules 518 and 520 are substantially the same.

[0066] Figure 5B This shows cluster 302 in different configurations. Figure 5A System 500. In Figure 5A In this cluster 302, a pair of inter-row fluid connectors are included, forming a single recirculation loop 324 within the cluster. However, the number of inter-row fluid connector pairs within cluster 302 can be varied to form multiple sub-clusters, each with its own recirculation loop. Therefore, in Figure 5BIn the configuration, cluster 302 includes three pairs of inter-row fluid connections, forming three sub-clusters 302a, 302b, and 302c within cluster 302. Sub-clusters 302a and 302c are formed by four IT units, and sub-cluster 302b is formed by two IT units. Of course, other embodiments may use a different number of inter-row connection pairs to divide cluster 302 into a different number of sub-clusters than shown. As described above, the coolers in each row of cluster 302 are connected to their respective facility supply and return loops; however, in the new configuration with inter-row fluid connections within the cluster, fluid flows in, flows out, and flows within each sub-cluster, separate from the other sub-clusters. It can be seen that by using the method proposed in this invention, different types of internal loops can be formed as needed. These three clusters can be assigned to different customers or different software applications. System 500 thus not only eliminates the need for internal loop facility design but also allows for the configuration and management of different types of internal loops between IT clusters. Similarly, these three clusters are shown as examples, deployed for three different types of IT containers / hardware, different types of workloads and services, different service level arrangements, different external cooling loops, different customers, etc.

[0067] Figure 6A Another embodiment of the cooling system 600 is shown. The cooling system 600 includes a facility 601 within which three IT clusters are located: a first cluster 602, a second cluster 604, and a third cluster 606. In one embodiment, clusters 602, 604, and 606 are configured similarly to cluster 302, wherein IT units are configured with alternating IT containers and coolers. Each of clusters 602, 604, and 606 may have one or more internal recirculation loops, such as... Figures 3A to 3B and Figure 5B As shown.

[0068] Cooling system 600 includes three cooling units (e.g., heat exchangers (HX)) 608, 610, and 612 located outside facility 601 to transfer heat from cooling fluid circulating through clusters 602, 604, and 606 to the atmosphere. Each heat exchanger is coupled to a facility supply loop and a facility return loop to supply cold fluid to the clusters and receive hot fluid returning from the clusters. In the illustrated embodiment, each heat exchanger provides facility supply and return lines to a pair of clusters. Facility supply line 614 is fluidly coupled to heat exchangers 608 and 610 to supply cooling fluid to clusters 602 and 604, and facility return line 616 is coupled to heat exchangers 610 and 608 to return fluid from clusters 604 and 602. Similarly, facility supply line 618 is fluidly connected to heat exchangers 610 and 612 to supply cooling fluid to clusters 604 and 606, and facility return line 620 is fluidly connected to heat exchangers 612 and 610 to return fluid from clusters 604 and 606. Facility supply line 615 and return line 613 are similarly fluidly connected to heat exchangers to supply fluid to and return from cluster 602 and an additional cluster (not shown) above cluster 602, and facility supply line 622 and return line 624 are fluidly connected to heat exchangers to supply fluid to and return from cluster 606 and an additional cluster (not shown) below cluster 602. Each cooler is fluidly connected to its corresponding supply line via a control valve.

[0069] Figures 6B to 6C The system 600 demonstrates different cooling module configurations achievable through its modular data center / IT cluster concept. Specific module designs depend solely on how the system can be prefabricated and deployed, and the actual system configuration can be adjusted post-deployment.

[0070] Figure 6B A first module configuration of system 600 is shown, featuring a complete external loop and partial, independent internal loops. In this configuration, the system is divided into four modules B. Within each module B, the external loop (i.e., the facility loop outside the cluster) is formed entirely within the module, and the internal loop (i.e., the internal recirculation loop of the cluster) is partially formed by one row of each cluster within the module. In this design, each module B contains a complete external loop. Two different approaches may be suitable for different use cases, and this design provides a system partitioning method for designing it as modules. In this embodiment, the external cooling loop is completed before module deployment and may need to be connected to the cooling module. In this case, the cooling unit is a separate module.

[0071] Figure 6CThe third module configuration of system 600 is shown, featuring a complete internal loop and a separate external loop. In this configuration, the system is divided into three modules C. Each module C includes a complete cluster and its external loop. At least one complete internal recirculation loop is formed within each cluster, and the external cooling loop (i.e., the facility cooling loop) is independent. The external cooling loop is formed after the IT units are filled into the cluster, while the internal loop is pre-formed within the cluster according to design requirements.

[0072] Figure 7 Another embodiment of the cooling system 700 is shown. The cooling system 700 includes a facility 701 within which two rows of IT units 702 and 704 are located. In one embodiment, rows 702 and 704 may be separate rows of IT units, but in other embodiments, one or both may be part of a cluster, the other half of which is not shown in the figure (but see, for example, Figure 6). For example, in one embodiment, row 702 may be the first row of a cluster similar to cluster 302, while row 704 may be the second row of another cluster similar to cluster 302. The IT units in rows 702 and 704 may be as described above. Figure 2 As described in the document, each IT unit includes an IT container and a cooler.

[0073] Cooling system 700 includes a facility heat exchanger (HX) 706 located outside facility 701 to transfer heat from cooling fluid circulating through rows 702 and 704 to the atmosphere. Heat exchanger 706 is fluidly coupled to a supply loop 708 and a return loop 710. Supply loop 708 and return loop 710 are located between rows 702 and 704. Supply loop 708 is fluidly coupled to one or more coolers in rows 702 and 704 via control valves, and return loop 710 is fluidly coupled to one or more coolers in rows 702 and 704.

[0074] The main difference between cooling system 700 and the other systems described above is the order of the IT containers and coolers in the rows. In previous embodiments, the IT containers and coolers alternated in each row, such that no IT container was adjacent to another IT container, and no cooler was adjacent to another cooler. In other words, from the upstream end of the row to the downstream end, the order of the IT containers and coolers in that row is:

[0075] Container → Cooler → Container → Cooler → etc.

[0076] In contrast, in cooling system 700, the order of the IT containers and coolers in rows 702 and 704 is as follows:

[0077] Container → Cooler → Cooler → Container → Container → Cooler → etc.

[0078] Orienting the IT units in this way ensures that each cooler is adjacent to another cooler, and each IT container is adjacent to another IT container. Alternatively, each IT container is adjacent to both a cooler and another IT container, and each cooler is adjacent to both an IT container and another cooler. By orienting the IT units in a row in this way and connecting the outlet of each cooler to the inlet of each cooler it abuts, each pair of IT units can form a sub-cluster with its own recirculation loop. Then, in row 702, arranging the IT units and connecting their coolers in this way results in sub-cluster 702a having its own recirculation loop 712a, and sub-cluster 702b having its own recirculation loop 712b. Although only sub-clusters 702a and 702b are shown in the figure, in some embodiments, each pair of IT units can be used to form a sub-cluster. This provides very high flexibility in configuring the system and managing the cooling fluid loops.

[0079] Figure 8 It shows how to control such as Figure 2 The cooler shown and, for example, such as Figure 4A An embodiment of process 800, illustrating the interaction between the facility's cooling systems, is shown. This process begins at box 802. At box 804, the process collects temperature T. 入口 The temperature T 入口 It is the outlet temperature of the upstream cooling unit and the inlet temperature of the downstream unit (e.g., the temperature sensor T in each cooler) (see...) Figure 2 ).

[0080] In box 806, the process determines T. 入口 Does it exceed the design temperature? If it's in box 806, T... 入口 If the design temperature is exceeded, the process moves to block 808, where the process attempts to reduce T by increasing the opening of the control valve (that is, by increasing the amount of fluid allowed to pass through control valve 408). 入口 After the opening of control valve 408 is increased in box 808, the process moves to box 810, where the process collects T again. 入口 Then return to box 806 to check T. 入口 Does it exceed the design temperature? However, if it's in box 806, the process determines T. 入口 If the temperature does not exceed the design temperature, the process moves to box 812 to determine T. 入口 Is it lower than the design temperature? If it's in box 812, T... 入口 If the temperature is below the design temperature, the process moves to box 814, where it attempts to increase T by reducing the opening of the control valve. 入口 Then return to box 810, where the process checks T again. 入口 The value of .

[0081] In addition to the embodiments described above, other embodiments of the cooling system arrangement are also possible. For example:

[0082] • IT containers or IT racks (including their loop designs) may differ.

[0083] The internal layout can be configured in different ways.

[0084] • The modular design of the entire system (internal loop, external loop, and external cooling unit) may differ from the diagram.

[0085] The above description of the embodiments is not intended to be exhaustive or to limit the invention to the forms described. For illustrative purposes, specific embodiments and examples of the invention have been described herein, but various modifications are possible.

Claims

1. A cooling device for information technology equipment, comprising: Multiple IT units arranged in an IT cluster, each IT unit including an IT container, the IT container being paired with and fluidly connected to a corresponding cooler adjacent to the IT container, each IT container being upstream of its corresponding cooler, the IT cluster comprising: The first line has an upstream end and a downstream end, and includes one or more IT units positioned adjacent to and abutting each other. The second row has an upstream end and a downstream end, is adjacent to the first row, and includes one or more IT units positioned adjacent to and abutting each other. Wherein, the cooler of each IT unit in the first row is fluidly connected to the IT container of the next downstream IT unit in the first row via an intra-row fluid connector, or is fluidly connected to the IT container of one of the one or more IT units in the second row via an inter-row fluid connector. Each cooler includes: The fluid connection is to the cooler inlet of the corresponding IT container, the fluid connection is to the open loop outlet of the intra-row fluid connection or inter-row fluid connection, and the closed loop outlet. A fluid bypass line having a first three-way valve and a second three-way valve connected therein, wherein the fluid bypass line is fluidly connected to a unit inlet and fluidly connected to the open-loop outlet and the closed-loop outlet via the second three-way valve; and The main cooling loop is fluidly connected to the fluid bypass line.

2. The cooling device for information technology equipment according to claim 1, wherein: The cooler of each IT unit in the second row is fluidly connected to the IT container of the next downstream IT unit in the second row via an intra-row fluid connector, or fluidly connected to the IT container of one of the one or more IT units in the first row via an inter-row fluid connector. The IT cluster includes at least one pair of inter-row fluid connectors, wherein the at least one pair of inter-row fluid connectors, the intra-row fluid connectors in the first row, and the intra-row fluid connectors in the second row form at least one fluid loop within the IT cluster.

3. The cooling device for information technology equipment according to claim 2, wherein, The first row and the second row are parallel to each other, and wherein: The upstream end of the first row is adjacent to the downstream end of the second row; and The downstream end of the first row is adjacent to the upstream end of the second row.

4. The cooling device for information technology equipment according to claim 3, wherein, The at least one pair of inter-row fluid connectors includes a first inter-row fluid connector that fluidly connects the upstream end of the first row to the downstream end of the second row and a second inter-row fluid connector that fluidly connects the downstream end of the first row to the upstream end of the second row, thereby forming a single fluid loop in the IT cluster.

5. The cooling device for information technology equipment according to claim 2, wherein, The at least one pair of inter-row fluid connectors includes two or more pairs of inter-row fluid connectors, wherein the two or more pairs of inter-row fluid connectors, the intra-row fluid connectors in the first row, and the intra-row fluid connectors in the second row form multiple fluid loops within the IT cluster.

6. The cooling device for information technology equipment according to claim 2, wherein, The main cooling loop includes: A heat exchanger having an inlet and an outlet, the inlet of the heat exchanger being fluidly connected to a fluid bypass line at or near the unit inlet via a supply line, and the outlet of the heat exchanger being connected to the fluid bypass line between the first three-way valve and the second three-way valve via a return line; The pump, whose fluid connection is in the return line; and A bidirectional pipeline, fluidly connected to the first three-way valve and upstream of the pump, is connected to the return pipeline.

7. The cooling device for information technology equipment according to claim 6, wherein, The heat exchanger of at least one cooler also includes an external inlet adapted to be connected to the facility supply line via a control valve and an external outlet adapted to be fluidly connected to the facility return line.

8. The cooling device for information technology equipment according to claim 6, wherein, At least one of the coolers has a heat exchanger that is a multiphase heat exchanger, the multiphase heat exchanger further comprising an external inlet adapted for connection to a facility supply line and an external outlet adapted for fluid connection to a facility steam return line.

9. A facility for cooling a data center, comprising: One or more IT clusters, each IT cluster comprising multiple IT units, each IT unit comprising an IT container, the IT container being paired with and fluidly connected to a corresponding cooler adjacent to the IT container, each IT container being upstream of its corresponding cooler, and each IT cluster comprising: The first line has an upstream end and a downstream end, and includes one or more IT units positioned adjacent to and abutting each other. The second row has an upstream end and a downstream end, is adjacent to the first row, and includes one or more IT units positioned adjacent to and abutting each other. Wherein, the cooler of each IT unit in the first row is fluidly connected to the IT container of the next downstream IT unit in the first row via an intra-row fluid connector, or fluidly connected to the IT container of one or more IT units in the second row via an inter-row fluid connector; and One or more data center cooling systems, each data center cooling system including a facility heat exchanger connected to at least one facility loop, the facility loop including facility supply lines and facility return lines, wherein each cooler in the one or more IT clusters is connected to the facility supply lines and facility return lines of one of the one or more data center cooling systems; The at least one cooler includes: a cooler inlet fluidly connected to the corresponding IT container, an open-loop outlet fluidly connected to an in-row or inter-row fluid connector, and a closed-loop outlet. A fluid bypass line having a first three-way valve and a second three-way valve connected therein, wherein the fluid bypass line is fluidly connected to a unit inlet and fluidly connected to the open-loop outlet and the closed-loop outlet via the second three-way valve.

10. The facility for cooling a data center according to claim 9, wherein: The cooler of each IT unit in the second row is fluidly connected to the IT container of the next downstream IT unit in the second row via an intra-row fluid connector, or fluidly connected to the IT container of one of the one or more IT units in the first row via an inter-row fluid connector. The IT cluster includes at least one pair of inter-row fluid connectors, wherein the at least one pair of inter-row fluid connectors, the intra-row fluid connectors in the first row, and the intra-row fluid connectors in the second row form at least one fluid loop within the IT cluster.

11. The facility for cooling a data center according to claim 10, wherein, Pairs of coolers in each cluster share one of the one or more data center cooling systems, such that pairs of coolers, including one cooler from the first row and one cooler from the second row, are connected to the facility supply line and the facility return line of the same facility loop.

12. The facility for cooling a data center according to claim 10, wherein, The one or more data center cooling systems include a first data center cooling system and a second data center cooling system, wherein the first data center cooling system uses a first cooling fluid and is fluidly connected to a subset of coolers in the cluster, and the second data center cooling system uses a second cooling fluid and is connected to another subset of coolers in the cluster.

13. The facility for cooling a data center according to claim 10, wherein, The paired clusters share one of the one or more data center cooling systems, such that the coolers in the first row of the first cluster and the coolers in the second row of the second cluster are fluidly connected to a single facility loop.

14. The facility for cooling a data center according to claim 10, wherein, Each cluster includes an integrated distribution module having supply and return lines adapted for fluid connection to at least one facility heat exchanger.

15. The facility for cooling a data center according to claim 10, wherein, Each cooler is fluidly connected to a single facility heat exchanger via supply and return lines.

16. The facility for cooling a data center according to claim 10, wherein, Each cooler's supply line is fluidly connected to the facility's supply line via a control valve.

17. The facility for cooling a data center according to claim 16, wherein, Each cooler includes a temperature sensor to sense the outlet temperature of the cooler, and the control valve is set based on the sensed outlet temperature.

18. The facility for cooling a data center according to claim 10, wherein, At least one cluster includes two or more pairs of inter-row fluid connections, wherein the two or more pairs of inter-row fluid connections, the intra-row fluid connections in the first row, and the intra-row fluid connections in the second row form multiple fluid loops within the IT cluster.

19. The facility for cooling a data center according to claim 10, wherein, At least one cooler also includes: The main cooling loop, which is fluidly connected to the fluid bypass line, includes: A heat exchanger having an inlet and an outlet, the inlet of the heat exchanger being fluidly connected to a fluid bypass line at or near the unit inlet via a supply line, and the outlet of the heat exchanger being connected to the fluid bypass line between the first three-way valve and the second three-way valve via a return line; The pump, whose fluid connection is in the return line; and A bidirectional pipeline, fluidly connected to the first three-way valve and upstream of the pump, is connected to the return pipeline.

Images