Multi-layer data center cooling infrastructure

Abstract

An embodiment of a cooling system for a data center is disclosed. The cooling system includes an IT area, a ceiling area, and a cooling air plenum sandwiched between the IT area and the ceiling area. The IT area includes one or more IT plenums coupled to the cooling air plenums for supplying cool air to the IT area, the IT area may have a plurality of IT racks, the IT racks are clustered around the IT plenums, and are suitable for accommodating one or more liquid-cooled or hybrid-cooled information technology (IT) equipment pieces. The ceiling area includes one or more ceiling plenums, and one or more sets of heat exchangers, wherein each heat exchanger is cooled by cooling air delivered to the ceiling plenums. The cooling air plenums are fluidly coupled to the one or more IT plenums by flow control, and are fluidly coupled to the one or more ceiling plenums or to the volume of the ceiling area between the ceiling plenums by flow control.

Description

Technical Field

[0001] Embodiments of the present disclosure relate generally to liquid cooling systems for temperature control of electronic equipment, and particularly, but not exclusively, to multi-layer data center cooling systems for temperature control in data center equipment. Background Art

[0002] Many modern information technology (IT) devices, such as servers, blade servers, routers, edge servers, etc., generate a large amount of heat during operation. The heat generated by individual components, especially high-power components such as processors, makes many of these individual components impossible or difficult to cool efficiently with air cooling systems. Therefore, many modern IT devices require liquid cooling or liquid-air hybrid cooling.

[0003] In liquid-to-air hybrid cooling, a liquid cooling loop is formed between a heat generating component in an IT equipment piece and a liquid-to-air heat exchanger. The operation of a liquid cooling loop has several steps. First, heat from the heat generating component is transferred to a working liquid through a fluid connection in the loop. Then, the heated working liquid is transferred to a liquid-to-air heat exchanger, where the heat from the working liquid is transferred to the air (usually the atmosphere), in some cases through multiple heat transfer loops. The working liquid, now cooled after the heat has been transferred to the air, is returned to the heat generating component to be cooled again; the heated air is then exhausted to the atmosphere or used for other purposes.

[0004] Since essentially all heat extracted from components is ultimately transferred directly or indirectly to the airflow through the liquid cooling loop, efficient airflow management is required for hybrid cooling, but existing airflow management solutions are insufficient to efficiently provide airflow to IT equipment clusters, especially when the IT equipment is hybrid cooled, which means that cooling air cannot be efficiently or accurately distributed or managed to the heat load. Summary of the invention

[0005] In one aspect, the present disclosure provides a cooling system for a data center and a data center.

[0006] A cooling system for a data center comprises: an IT area including: one or more IT plenums for supplying cool air to the IT area, a plurality of information technology (IT) racks, each IT rack having a front side through which cool air enters the rack and a rear side through which hot air exits the rack, wherein each IT rack is adapted to accommodate one or more liquid-cooled or hybrid-cooled pieces of information technology (IT) equipment, and wherein clusters of IT racks are grouped around at least one of the one or more IT plenums, each cluster including at least one IT rack, wherein the front side of the IT rack is fluidly coupled to the IT plenum; and a ceiling area located above the IT area , the ceiling area includes: one or more ceiling booster chambers, and one or more groups of heat exchangers, wherein each heat exchanger in each group is fluidly connected to the ceiling booster chamber, and wherein each heat exchanger is a liquid-to-air heat exchanger suitable for being fluidly connected to at least one liquid-cooled or hybrid-cooled IT equipment piece in one of the IT rack clusters; and a cooling air booster chamber sandwiched between the IT area and the ceiling area, wherein the cooling air booster chamber is fluidly connected to the one or more IT booster chambers by flow control, and is fluidly connected to the one or more ceiling booster chambers by flow control or to the volume of the ceiling area between the ceiling booster chambers.

[0007] A data center comprises: a data center facility; a cooling system located in the data center facility. The cooling system comprises: an IT area, including: one or more IT plenums for supplying cold air to the IT area, a plurality of information technology (IT) racks, each IT rack having a front side through which cold air enters the rack and a rear side through which hot air leaves the rack, wherein each IT rack is adapted to accommodate one or more liquid-cooled or hybrid-cooled information technology (IT) equipment pieces, and wherein IT rack clusters are grouped around at least one of the one or more IT plenums, each cluster comprising at least one IT rack, wherein the front side of the IT rack is fluidly coupled to the IT plenum; and a ceiling area located in the IT area Above, the ceiling area includes: one or more ceiling plenums, and each heat exchanger in each group is fluidly connected to the ceiling plenum, and wherein each heat exchanger is a liquid-to-air heat exchanger suitable for being fluidly connected to at least one liquid-cooled or hybrid-cooled IT equipment piece in one of the IT rack clusters; and a cooling air plenum sandwiched between the IT area and the ceiling area, wherein the cooling air plenum is fluidly connected to the one or more IT plenums by flow control and is fluidly connected to the one or more ceiling plenums by flow control or to the volume of the ceiling area between the ceiling plenums. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Non-limiting and non-exhaustive embodiments of the present invention are described with reference to the following figures, wherein like reference numerals refer to like parts throughout the various figures unless otherwise specified.

[0009] Figure 1 is a block diagram of an embodiment of an electronics rack having electronic equipment and a cooling system housed therein.

[0010] Figure 2A and 2B is a cross-sectional side view of an embodiment of a data center cooling system. Figure 2A shows the cooling system viewed from a row of IT racks downward, and Figure 2B The cooling system is shown viewed from the front or back of an IT rack.

[0011] Figure 3 is a side view of another embodiment of a data center cooling system viewed from the end of a row of IT racks.

[0012] Figure 4 is a side view of another embodiment of a data center cooling system viewed from the end of an IT rack.

[0013] Figure 5 is a side view of another embodiment of a data center cooling system viewed from the end of an IT rack.

[0014] Figure 6 is a side view of another embodiment of a data center cooling system viewed from the end of an IT rack.

[0015] Figure 7 is a side view of another embodiment of a data center cooling system viewed from the end of an IT rack. DETAILED DESCRIPTION

[0016] Embodiments of a multi-layer data center cooling system are described. Specific details are described to provide an understanding of the embodiments, but one skilled in the relevant art will recognize that the invention can be implemented without one or more of the described details, or with other methods, components, materials, etc. In some cases, well-known structures, materials, or operations are not shown or described in detail, but are still included within the scope of the invention.

[0017] References to "one embodiment" or "an embodiment" in this specification mean that the described features, structures, or characteristics may be included in the description of at least one embodiment, so that the appearances of "in one embodiment" or "in an embodiment" do not necessarily all refer to the same embodiment. In addition, in one or more embodiments, the particular features, structures, or characteristics may be combined in any suitable manner.

[0018] Airflow management improves the operational efficiency of data centers and IT clusters. As data centers are increasingly used, their energy consumption becomes larger and larger, and can account for a large portion of the total energy consumption of an entire city. Therefore, there is an endless effort to improve the energy efficiency of data centers. Modern data center design needs to accommodate the versatility and capabilities of different generations of IT systems, but always with cost and efficiency in mind. This requires engineers and scientists to develop and invent new cooling architectures.

[0019] The described embodiment provides an innovative airflow distribution system using a multi-layer airflow distribution system to improve airflow management efficiency as well as system versatility and flexibility. The described embodiment enables accurate and efficient airflow distribution and management of heat loads (heat loads for direct air cooling and heat loads for liquid cooling). Since the air cooling part and the liquid cooling part are different, this not only separates the different cooling airflows, but also enables the airflow to be delivered where needed. The current architecture is also compatible with different cooling solutions and can be easily implemented in Brownfield and Greenfield data centers. In addition, it is a very efficient architecture concept for modular and volumetric design.

[0020] An embodiment of an air flow distribution system using a multi-layer design is described. The cooling air flow is distributed from the middle portion of the system through its dedicated cooling air plenum. The air flow is distributed to the ceiling area and the IT area to directly cool the liquid cooling unit and the IT equipment. The ceiling area is designed to have an elevated air flow distribution, and the IT area is designed to have cooling air blown downward from the top. The cooling air plenum is the only part that receives air from the cold air source, and manages the cold air so that it can be distributed to different areas. The design also includes using different cooling air inlet modes (e.g., an economical inlet mode and an enhanced inlet mode) of the cooling air plenum to achieve more efficient system operation. An embodiment of the system includes several modules; the modular design enables the system to be accommodated in a hyperscale data center design or a volumetric design.

[0021] Figure 11 is a block diagram showing a side view of an embodiment of an electronic rack. In one embodiment, the electronic rack 100 includes a CDU 101, a rack management unit (RMU) 102, and one or more blade servers 103A to 103D (collectively referred to as blade servers 103). The blade servers 103 can be inserted into the server slot array from the front end 104 of the electronic rack 100, respectively. Note that although only four blade servers 103A to 103D are shown, more or fewer blade servers can be maintained within the electronic rack 100. Also note that the specific locations of the CDU 101, CMU 102, and blade servers 103 are shown for illustration purposes only; other arrangements or configurations of the CDU 101, CMU 102, and blade servers 103 can also be implemented. In addition, the front door provided on the front end 104 and the rear door provided on the rear end 105 are optional. In some embodiments, there are no doors on the front end 104 and / or the rear end 105.

[0022] In one embodiment, the CDU 101 includes a heat exchanger 111, a liquid pump 112, and a pump controller 110. The heat exchanger 111 may be a liquid-to-liquid heat exchanger. The heat exchanger 111 includes a first conduit having a first pair of liquid connectors coupled to external liquid supply / return lines 131 / 132 to form a main loop, wherein the connectors coupled to the external liquid supply / return lines 131 / 132 may be disposed or mounted on the rear end 105 of the electronic rack 100. In addition, the heat exchanger 111 also includes a second conduit having a second pair of liquid connectors connected to a liquid manifold 125, which may include a supply manifold that supplies cooling liquid to the blade servers 103, and a return manifold that returns warmer liquid to the CDU 101. The processor may be mounted on a cooling plate, wherein the cooling plate includes a liquid distribution channel embedded therein to receive cooling liquid from the liquid manifold 125 and return cooling liquid carrying heat exchanged from the processor to the liquid manifold 125. Supply / return lines 131 and 132 may be fluidly coupled to a liquid to air heat exchanger in the ceiling area (see Figure 2A wait).

[0023] Each blade server 103 may include one or more IT components (e.g., CPU, GPU, memory, and / or storage device). Each IT component may perform data processing tasks, and the IT component may include software installed in a storage device, loaded into a memory, and executed by one or more processors to perform data processing tasks. The blade server 103 may include a host server (referred to as a host node) connected to one or more computing servers (also referred to as computing nodes). The host server (having one or more CPUs) is typically connected to a client via a network (e.g., the Internet) to receive requests for specific services such as storage services (e.g., cloud-based storage services such as backup and / or restore), execute applications to perform certain operations (e.g., image processing, deep data learning algorithms or modeling as part of a software as a service or SaaS platform, etc.). In response to the request, the host server assigns the task to one or more computing servers (having one or more GPUs) managed by the host server. The computing server performs the actual task, and heat may be generated during operation.

[0024] The electronics rack 100 also includes an RMU 102 configured to provide and manage power supplied to the blade servers 103 and the CDU 101. The RMU 102 may be coupled to a power supply unit (not shown) to manage power consumption of the power supply unit and other thermal management of the power supply unit (e.g., cooling fans). The power supply unit may include necessary circuitry (e.g., alternating current (AC) to direct current (DC) or DC to DC power converters, transformers or regulators, etc.) to provide power to the remaining components of the electronics rack 100.

[0025] In one embodiment, the RMU 102 includes an optimal control logic 121 and a rack management controller (RMC) 122. The optimal control logic 111 is connected to at least some of the blade servers 103 to receive the operating status of each blade server 103, such as the processor temperature of the processor, the current pump speed of the liquid pump 112, and the liquid temperature of the coolant. Based on this information, the optimal control logic 111 determines the optimal pump speed of the liquid pump 112 by optimizing a predetermined objective function so that the output of the objective function is maximized when a set of predetermined constraints are met. Based on the optimal pump speed, the RMC 122 is configured to send a signal to the pump controller 110 to control the pump speed of the liquid pump 112 based on the optimal pump speed.

[0026] Figure 2A and 2B Together, an embodiment of a multi-story data center cooling system 200 is shown. Figure 2A is a view from the end of a rack or row of racks, and Figure 2B2 is a view of the same rack from the side (the front side of the rack viewed from the cold aisle, or the back side of the rack viewed from the hot aisle). The cooling system 200 is located in a data center 201 and includes three layers: an information technology (IT) layer 202, in which IT equipment that needs cooling is placed; a ceiling layer 204, located above the IT layer; and a cooling plenum layer 206, sandwiched between the IT layer and the ceiling layer.

[0027] The IT layer 202 is a layer where IT equipment that needs cooling is placed. One or more pieces of IT equipment, such as servers, blade servers, routers, etc., are located in one or more IT racks 208. The racks 208 may also include liquid cooling elements that are fluidly coupled to the liquid cooling elements of the IT equipment contained in the racks (e.g., see Figure 1 ), and fluidly coupled to a liquid cooling element of a cooling system of a data center, as discussed further below. Examples of liquid cooling elements that may be placed in rack 208 include heat exchangers, pumps, pump controllers and associated logic, manifolds, and fluidly coupled to provide fluid coupling between elements (see Figure 1 ). Each rack 208 may also include a device to cool the IT equipment using the cool air flowing through the rack. Each rack 208 has a front side through which cool air can enter the rack and a back side through which hot air can leave the rack. Some embodiments of racks 208 or some embodiments of IT equipment within the racks may use air moving elements such as fans to draw cool air through the front side and exhaust hot air out the back side.

[0028] In the IT layer 202, one or more racks 208 are arranged in rows and clustered (i.e., grouped into clusters) around an IT plenum 210. In the illustrated embodiment, the IT plenum 210 is vertical, but in other embodiments, it may have a different orientation. In each cluster, each rack 208 has a front side that is fluidly coupled to a corresponding IT plenum 210 so that cold air can flow from each IT plenum 210 into the front ends of all racks 208 that the cluster surrounds. The rear side of each rack 208 is fluidly coupled to the interior volume 211 of the IT layer so that the racks are combined upward into a hot aisle / cold aisle arrangement, with the IT plenum 210 forming the cold aisle and the volume 211 forming the hot aisle. In the illustrated embodiment, the IT layer 202 also includes an exhaust port 212 that fluidly couples the interior volume 211 of the IT layer to the exterior of the data center. Hot air leaving the rear side of each rack can be exhausted or discharged from the interior volume 211 to the exterior of the data center 201 through the exhaust port 212.

[0029] The ceiling layer 204 is located above the IT layer 202 and includes a volume 214 in which one or more liquid to air heat exchangers 216 are located. In other words, in this design, heat in the heat exchanger is transferred from the liquid to the air, or the air in the heat exchanger is used to cool the liquid. Each heat exchanger 216 is fluidly coupled to the supply and return connections of at least one rack by a pair of fluid couplings 218 (see Figure 1 , elements 132 and 131). Each pair of fluid connections 218 includes a supply connection and a return connection, as indicated by the arrows. The illustrated embodiment of the cooling system 200 has a one-to-one correspondence between the racks 208 and the heat exchangers 216, i.e., each rack is fluidly connected to a single corresponding heat exchanger. However, in other embodiments, there may be a one-to-many correspondence between the racks and the heat exchangers (each rack is fluidly connected to more than one heat exchanger), a many-to-one correspondence between the racks and the heat exchangers (more than one rack is fluidly connected to each heat exchanger), or a many-to-many correspondence between the racks and the heat exchangers (more than one rack is connected to more than one heat exchanger). A ceiling booster chamber 220 is also provided within the volume 214 of the ceiling layer 204. In the ceiling layer 204, one or more liquid-to-air heat exchangers 216 are clustered (i.e., grouped into clusters) around each ceiling booster chamber 220. In each cluster, each heat exchanger 216 is fluidly coupled to its corresponding ceiling plenum 220 (which may be understood as a hot air enclosure) so that hot air may flow from each heat exchanger into its corresponding ceiling plenum; in this embodiment, the ceiling plenum 220 then acts as an exhaust plenum, meaning that the hot air is exhausted directly into the surrounding environment. In other embodiments, the ceiling plenum 220 may be implemented in a different location or on the side of the ceiling layer 204.

[0030] The cooling plenum 206 is sandwiched between the IT layer 202 and the ceiling layer 204 to provide cooling air to both layers. The cooling plenum 206 includes at least one air inlet 226 through which cooling air can enter the layer. In the illustrated embodiment, a pair of air inlets 226 are located in the wall of the data center so that outdoor cold air can be brought into the cooling air plenum from the external data center 201. The cooling plenum 206 also includes a plurality of outlets through which cooling air from the cooling plenum can be directed and distributed into the IT layer and the ceiling layer. In the illustrated embodiment, the cooling plenum 206 is fluidly coupled to the IT plenum 210 via a flow control 222, and is also fluidly coupled to the volume 214 of the ceiling layer 204 via a flow control 224. The flow controls 222 and 224 can be used to regulate the flow into their respective plenums, and the flow control range can be from fully open to fully closed and any position in between. In one embodiment, flow controls 222 and 224 may be adjustable louvers that control the amount of airflow therethrough, but in other embodiments, flow controls 222 and 224 may be another type of flow control. For example, in other embodiments, the flow controls may be openings with holes or any similar structure. Although not shown in this figure, cooling plenum 206 may also include air moving elements such as fans to assist or optimize airflow through the cooling plenum (e.g., see Figure 4 ).

[0031] In operation of the system 200, cold air flows into the cooling plenum 206 through the inlet 226. The airflow path through the system is represented by the dashed arrows in the figure. The cold air flowing into and through the cooling plenum is then supplied to the IT layer and the ceiling layer. In the IT layer, the cold air from the cooling plenum flows through the flow control 222 into the IT plenum 210. The front side of the rack 208 is fluidly connected to the IT plenum so that the cold air flowing from the cooling air plenum into the IT plenum is guided into the rack 208. The cold air flowing into the front side of each rack 208 flows through the IT components within the rack, absorbs heat from the IT components, and leaves the back side of each rack. The hot air discharged from the back of each rack 208 into the IT layer volume 211 is then discharged to the outside of the data center through the exhaust port 212.

[0032] In the ceiling level 204, the liquid cooling loop is in operation, with hot and cold liquids flowing through the supply and return lines 218: the hot liquid from the racks 208 enters the heat exchanger 216 and is cooled, and then the cold fluid flows back to the racks. Although not shown in the drawings, in some embodiments there may be a pumping system to recirculate the fluid flowing through the supply and return lines 218. In other embodiments, however, a pump may not be required for the supply and return lines 218 (such as for a self-driven thermosyphon system). At the same time, cold air from the cooling plenum 206 flows through the flow control 224 into the ceiling volume 214. Once in the ceiling volume 214, the cold air flows through the heat exchanger 216 and into the ceiling plenum 220, which in this embodiment serves as an exhaust plenum. The air flowing through the heat exchanger extracts heat from the liquid as it flows through the heat exchanger. The now hot air that has flowed through the heat exchanger 216 flows into the ceiling plenum 220 and out of the data center.

[0033] Figure 3 Another embodiment of a data center cooling system 300 is shown. System 300 includes three layers: an IT layer 302, a ceiling layer 304 located above the IT layer, and a cooling plenum layer 306 sandwiched between the IT layer and the ceiling layer. IT layer 302 and cooling plenum layer 306 are substantially similar to IT layer 202 and cooling plenum layer 206 in system 200, and substantially include the same elements arranged in substantially the same manner. The main difference between cooling system 200 and cooling system 300 is in ceiling layer 304.

[0034] The ceiling level 304 includes a volume 308 within which one or more liquid to air heat exchangers 310 are located. Each heat exchanger 310 is fluidly coupled to the supply and return connections of the rack via a pair of fluid couplings 218 (see Figure 3 ). Each pair of fluid connections 218 includes a supply connection and a return connection, as indicated by the arrows. As in system 200, in cooling system 300, there can be a one-to-one correspondence between racks 208 and heat exchangers 216 (each rack is fluidly coupled to a single corresponding heat exchanger), a one-to-many correspondence between racks and heat exchangers (each rack is fluidly coupled to more than one heat exchanger), a many-to-one correspondence between racks and heat exchangers (more than one rack is fluidly coupled to each heat exchanger), or a many-to-many correspondence between racks and heat exchangers (more than one rack is coupled to more than one heat exchanger).

[0035] Also disposed within the volume 314 of the ceiling layer 304 is a ceiling plenum 312 that is fluidly coupled to the cooling plenum layer 306 via a flow control 314. In the illustrated embodiment, the ceiling plenum 312 is in a vertical orientation, but in other embodiments, it may be oriented differently. In the ceiling layer 304, one or more liquid-to-air heat exchangers 310 are clustered (i.e., grouped into clusters) around each ceiling plenum 312. In each cluster, each heat exchanger 310 is fluidly coupled to its corresponding ceiling pressure plenum 312 so that cold air can flow from each ceiling pressure plenum into its corresponding heat exchanger 310. Then, in the system 300, the ceiling layer 304 has a cold aisle / hot aisle arrangement, with the ceiling plenum 312 forming a cold aisle and the volume 308 forming a hot aisle, and it can be understood that the ceiling layer forms a hot aisle due to the inclusion of the cold aisle as a plenum 312.

[0036] In operation of system 300, IT layer 302 and cooling plenum layer 306 operate in a manner similar to IT layer 202 and cooling plenum layer 202 of system 200, but ceiling layer 304 operates in a different manner. In ceiling layer 304, the liquid cooling loop is in operation, and hot liquid and cold liquid flow through supply and return lines 218. At the same time, cold air from cooling plenum layer 306 flows through flow control 314 into ceiling plenum 312. Cold air flows from ceiling plenum 312 and passes through heat exchanger 216 into exhaust volume 308, extracting heat from the liquid as it flows through the liquid to air heat exchanger. The now hot air flows from volume 308 through exhaust port 316 to the outside of data center 201.

[0037] Figure 4 Another embodiment of a data center cooling system 400 is shown. System 400 is similar to system 300 in most respects: it includes an IT layer 302, a ceiling layer 304 located above the IT layer, and a cooling plenum layer 306 sandwiched between the IT layer and the ceiling layer. All three layers are essentially combined as above, except that the ceiling layer 304 is modified as described below. Figure 3 described.

[0038] The primary difference between cooling system 300 and cooling system 400 is the inclusion of external cooling module 402 in system 400. System 400 provides a more complete solution that combines system 300 and a cooling module to supply airflow to cooling plenum 306. In some cases, the quality of air outside of data center 201 (e.g., as measured by temperature, humidity, and the presence of chemicals or particulates, or other amounts) may not allow ambient air to be used for cooling without further conditioning; in such cases, cooling module 402 may provide conditioning. In the illustrated embodiment, cooling module 402 includes: a cooling unit 404, which typically changes the temperature of the outside air by cooling it; one or more filter units 406, which provide chemical or mechanical filtration to remove chemicals and particulates; and a fan unit 408, which drives the cooled and / or filtered air into cooling plenum 306. In one embodiment, cooling unit 404 may be identical to cooling module 402. Other embodiments of cooling module 402 may include more or fewer units than shown. Other components (temperature sensors, controllers, etc.) used to implement open-loop or closed-loop control of the cooling module 402 are not shown in the figure, but may still be used in some embodiments.

[0039] The cooling module 402 is fluidly coupled to the cooling plenum 306 via a flow control 410, such that the flow control can be used to regulate the amount of air entering the cooling plenum. The use of cooling module 408 allows more cubic feet per minute (CFM) of cool air to circulate through the data center, thereby providing greater heat transfer capacity and allowing the data center to operate at higher power (i.e., more kilowatts (kW)). The single inlet design of the cooling plenum 306 provides a convenient and accessible location for additional cooling modules 402, making the overall cooling system 400 more flexible.

[0040] In operation of the system 400, the IT layer 302 and the cooling plenum 306 operate substantially as described above for the system 300, except that the cold air flowing into the cooling plenum 306 is now pre-processed (heated / cooled, humidified / dehumidified, and filtered) and driven through the plenum cooling layer by the fan unit 408. In addition to the fan unit 408, the flow control 410 can help regulate or optimize the amount of cold air flowing into the cooling plenum. As described above for the system 300, hot air is exhausted from the volume 308 of the ceiling layer 304, but in the ceiling layer, the exhaust ports 316 are removed at the location where the cooling modules 402 are added, so that the hot air is exhausted from the volume 308 through a different number of exhaust ports 316 than in the system 300.

[0041] Figure 5Another embodiment of a data center cooling system 500 is shown. System 500 is similar in most respects to systems 300 and 400: it includes an IT layer 302, a ceiling layer 304 located above the IT layer, and a cooling plenum layer 306 sandwiched between the IT layer and the ceiling layer. All three layers are essentially combined as above, except that the ceiling layer 304 is slightly modified as described below. Figure 3 and Figure 4 described.

[0042] The main difference between cooling system 500 and cooling system 300 and cooling system 400 is the inclusion of a recirculation unit 502 in system 500. In some cases, the quality of air outside of data center 201 (e.g., as measured by temperature, humidity, and the presence of chemicals or particulates, or other quantities) may not allow ambient air to be used to cool the data center without further conditioning. One way to avoid using outside air is to recirculate air within the data center, rather than drawing ambient air from the outside. Recirculation module 502 can provide this recirculation.

[0043] The recirculation module 502 is located outside the data center 201 and includes a volume 504. An external cooling unit 506 is located within the volume 504. The volume 308 of the ceiling layer 304 is fluidly connected to the volume 504 via the exhaust port 316, and the volume 211 of the IT layer 302 is connected to the volume 504 through the exhaust port 212. The external cooling unit 506 is also fluidly connected to the outside directly or via the volume 504, so that the external air can flow through the external cooling unit. The external cooling unit 506 is also fluidly connected to the cooling plenum layer 306 through the fan unit 508 and the flow control 510. In various embodiments, the external cooling unit can be a cooling water system, an evaporative cooling unit, or any other type of heat exchanger for extracting heat from the internal air. And, although shown as a single unit, in other embodiments, the cooling can be performed by multiple units. Other components (temperature sensors, controllers, etc.) for implementing open-loop or closed-loop control of the recirculation unit 502 are not shown in the figure, but may still exist in some embodiments.

[0044] In operation of the system 500, the IT layer 302 and the ceiling layer 304 operate substantially as previously described: cold air flows from the cooling plenum layer 306 into the IT layer 302 through the IT plenum 210, and also flows into the ceiling layer 304 through the ceiling plenum 312. Hot air from the ceiling layer and the IT layer is introduced into the recirculation unit 502 through the exhaust ports 316 and 212, respectively. After entering the recirculation unit, the hot air from the IT layer 302 and the ceiling layer 304 is directed into the external cooling unit 506, where the heat from the hot air inside is transferred to the outside air. In some embodiments, air ducts may be used in the recirculation unit 502 to accommodate airflow between the IT layer and the ceiling layer and the external cooling unit 506, so that the outside air is separated from the inside air. After transferring the heat of the internal hot air to the external air, the now cooled air originating from the IT layer 302 and the ceiling layer 304 is recirculated into the cooling plenum layer 306 through the fan unit 508 and the flow control 510, and then the fan unit 508 and the flow control 510 deliver the cooled air to the IT layer 302 and the ceiling layer 304 again.

[0045] Figure 6 Another embodiment of a data center cooling system 600 is shown. System 600 is similar to system 300 in most respects: it includes an IT layer 602, a ceiling layer 604 located above the IT layer, and a cooling plenum layer 606 sandwiched between the IT layer and the ceiling layer. The IT layer 602 and the ceiling layer 604 are combined substantially as described above, except that the ceiling layer 604 is slightly modified as described below. Figures 3 to 5 The main difference between the cooling system 600 and the cooling system 300 is that in the system 600, the cooling plenum layer 606 has multiple inlets with different functions. In the embodiment shown, the cooling plenum layer has two air inlets: the enhancement inlet 608 and the energy saving inlet 610.

[0046] In some cases, the air quality outside the data center 201 (e.g., measured by temperature, humidity, and the presence of chemicals or particulates, or other quantities) may not allow the ambient air to be used to cool the data center without further conditioning. In this case, it may be necessary to enhance the ambient air from the outside. The enhanced air inlet 608, as the name implies, provides an enhancement function. In the illustrated embodiment, the enhanced air inlet 608 includes a cooling system having an evaporator 612, a condenser 614, and a fan 616. The evaporator 612 is located at or near the entrance of the cooling plenum 606, so that the air entering the cooling plenum through the enhanced inlet 608 flows through the evaporator and is cooled before entering the rest of the system. The condenser 614 is located outside the data center 201 and is fluidly coupled to the evaporator 612 by a refrigerant circuit 615. The refrigerant circuit 615 includes a return line, and a supply line, through which vapor or a liquid / vapor mixture can travel from the evaporator to the condenser, and a liquid can travel from the condenser to the evaporator. Although not shown in this figure, the refrigerant circuit may also include a compressor. A fan 616 may be coupled to the condenser 614 to help it condense the vapor into liquid more quickly. In addition to the cooling system, the enhanced inlet 608 may include a flow control 621 that may fully close the enhanced inlet 608, fully open the inlet, or adjust the inlet to any position between fully closed and fully open. The enhanced inlet 608 may also include one or more mechanical or chemical filters 618.

[0047] The energy saving inlet 610 may include a fan 622 to increase airflow to the cooling plenum 606 and a flow control 624 to adjust the amount of air flowing into the plenum. The flow control may completely close the energy saving inlet 610, may completely open the inlet, or may adjust the inlet to any position between completely closed and completely open. The energy saving inlet 610 may also include one or more mechanical or chemical filters 625.

[0048] In operation, the enhanced inlet 608 and the economizer inlet 610 can be operated independently. At any given time, one can be open and the other can be closed, or both can be open at the same time. When open at the same time, the enhanced inlet 608 and the economizer inlet 610 can have the same or different flow rates. Once the cold air flow is established using one or both inlets, the cold air passes through the cooling plenum layer 606 to the IT layer 602 and the ceiling layer 604, as discussed above for the system 300. The system 600 provides flexibility to adapt to different heat load requirements and different environmental conditions.

[0049] Figure 7Another embodiment of a data center cooling system 700 is shown. System 700 is similar in most respects to system 600: it includes an IT layer 702, a ceiling layer 704 located above the IT layer, and a cooling plenum layer 706 sandwiched between the IT layer and the ceiling layer. Ceiling layer 704 is substantially the same as ceiling layer 604 (described above with respect to FIG. Figure 6 ), but IT layer 702 and cooling plenum layer 706 are respectively different from their respective counterparts in system 600 (IT layer 602 and cooling plenum layer 606).

[0050] The cooling plenum layer 706 shares some similarities with the cooling plenum layer 606 in that it has multiple air inlets with different functions. In the embodiment shown, the cooling plenum layer has an enhanced inlet 608 and an energy-saving inlet 610, both of which can have the above Figure 6 706i and 706u. The main difference between cooling plenum 606 and cooling plenum 706 is that cooling plenum 706 is divided into two parts, a lower cooling plenum 706i and an upper cooling plenum 706u. The lower cooling plenum 706i is fluidly connected to the enhanced inlet 608 and the IT plenum 210 so that it only supplies cool air to the IT layer 702. The upper cooling plenum 706u is fluidly connected to the energy-saving inlet 610 and the ceiling plenum 312 so that it only supplies ambient air to the ceiling layer 704. For example, system 700 may be useful when the ceiling layer and the IT layer have different cooling requirements. And even though there are two plenums 706i and 706u, in other embodiments, they may be combined to function as a single channel.

[0051] IT layer 702 also shares some similarities with IT layer 602 in that it houses racks 208 clustered around an IT plenum 210. However, IT layer 702 also includes a return plenum 708. The illustrated embodiment has one return plenum, but other embodiments may have more than one return plenum. Return plenum 708 returns hot air from IT layer 702 to boost inlet 608, which may be cooled, filtered and directed back to lower plenum 7061. IT layer 702 also differs from IT layer 602 in that its IT plenum 210 is fluidly coupled only to the lower cooling plenum 7061 via flow control, rather than to the entire cooling plenum layer. In one embodiment, if directed outside air is brought into pressure plenum 7061 and then supplied to IT plenum 210, exhaust port 212 is used to exhaust the hot air.

[0052] In operation, the enhanced inlet 608 and the energy-saving inlet 610 can be operated independently. At any given time, one can be open and the other can be closed, or both can be open at the same time. When open at the same time, the enhanced inlet 608 and the energy-saving inlet 610 can have the same or different flow rates. Once the cold air flow is established using one or both inlets, the cold air passes through the cooling plenum layer 7061 to the IT layer 702 and through the cooling plenum layer 706u to the ceiling layer 704, and then the operation of the system is basically as discussed above for system 300. System 700 provides flexibility to adapt to different heat load requirements and different environmental conditions.

[0053] In addition to the above embodiments, other embodiments of the cooling system are also possible. For example:

[0054] The current architecture can be connected to different types of cooling solutions or cooling loops;

[0055] A multi-layer air distribution system can be expanded to include more layers for airflow distribution; or

[0056] This architecture can accommodate different IT room cold aisle containment and hot aisle containment.

[0057] The above description of embodiments is not intended to be exhaustive or to limit the invention to the form described. Specific embodiments and examples of the invention are described herein for purposes of illustration, but various modifications are possible.

Claims

1. A cooling system for a data center, the cooling system comprising: Information Technology area, including: one or more IT plenums for supplying cool air to the IT areas, a plurality of IT racks, each IT rack having a front side through which cool air enters the rack and a rear side through which hot air exits the rack, wherein each IT rack is adapted to house one or more pieces of liquid-cooled or hybrid-cooled IT equipment, and wherein clusters of IT racks are grouped around at least one of the one or more IT plenums, each cluster comprising at least one IT rack, wherein the front side of the IT rack is fluidly coupled to the IT plenum; and A ceiling area, located above the information technology area, the ceiling area comprising: One or more ceiling plenums, and one or more sets of heat exchangers, wherein each heat exchanger in each set is fluidly coupled to the ceiling plenum, and wherein each heat exchanger is a liquid-to-air heat exchanger adapted to be fluidly coupled to at least one liquid-cooled or hybrid-cooled piece of information technology equipment in one of the information technology rack clusters; and a cooling air plenum sandwiched between the information technology area and the ceiling area, wherein the cooling air plenum is fluidly coupled to the one or more information technology plenums and is fluidly coupled to the one or more ceiling plenums or to the volume of the ceiling area between the ceiling plenums.

2. The cooling system according to claim 1, wherein: The one or more ceiling plenums are fluidly coupled to the cooling air plenum, and wherein the ceiling area includes at least one hot air exhaust outlet.

3. The cooling system according to claim 1, wherein: The one or more ceiling plenums are exhaust plenums fluidly coupled to the volume of the ceiling area or to an exterior of the data center.

4. The cooling system according to claim 1, wherein: The cooling air plenum is fluidly coupled to an exterior of the data center through at least one fresh air inlet.

5. The cooling system of claim 1, further comprising an external cooling module located outside the data center and fluidly coupled to the cooling air plenum, wherein: The external cooling module includes at least one of the following: a fan to pump airflow into the cooling air plenum; an air filter to filter air before it flows into the cooling air plenum; or A heat exchanger cools outside air entering the cooling air plenum.

6. The cooling system of claim 1 further comprising a recirculation module comprising a heat exchanger fluidly coupled to the information technology area, the ceiling area, and the cooling air plenum, wherein Hot air from the ceiling area and from the information technology area is exhausted to the recirculation module so that the heat exchanger can transfer heat from the hot air entering the recirculation module to outside air and return cooled air to the cooled air plenum.

7. The cooling system according to claim 1, wherein: The cooling air plenum includes an economizer inlet for drawing in uncooled air and a booster inlet for drawing in cooling air.

8. The cooling system of claim 7, further comprising a cooling air plenum module located at or near the enhanced inlet, the cooling air plenum module comprising: a fan to pump airflow into the cooling air plenum; A heat exchanger cools the air before it is pumped into the cooling air plenum.

9. The cooling system according to claim 8, wherein: The heat exchanger includes an evaporator fluidly coupled to a condenser and a compressor to form a refrigerant circuit.

10. The cooling system according to claim 7, wherein: The cooling air plenum includes a first plenum coupled to the economizer inlet and a second plenum coupled to the boost inlet.

11. A data center, comprising: Data center facilities; A cooling system located in the data center facility, the cooling system comprising: Information Technology area, including: one or more IT plenums for supplying cool air to the IT areas, a plurality of IT (information technology) racks, each IT rack having a front side through which cool air enters the rack and a rear side through which hot air exits the rack, wherein each IT rack houses one or more pieces of liquid-cooled or hybrid-cooled IT (information technology) equipment, and wherein clusters of IT racks are grouped around at least one of the one or more IT plenums, each cluster comprising at least one IT rack, wherein the front side of the IT rack is fluidly coupled to the IT plenum; and A ceiling area, located above the information technology area, the ceiling area comprising: One or more ceiling plenums, and one or more sets of heat exchangers, wherein each heat exchanger in each set is fluidly coupled to the ceiling plenum, and wherein each heat exchanger is a liquid-to-air heat exchanger fluidly coupled to at least one liquid-cooled or hybrid-cooled piece of information technology equipment in one of the information technology rack clusters; and a cooling air plenum sandwiched between the information technology area and the ceiling area, wherein the cooling air plenum is fluidly coupled to the one or more information technology plenums and is fluidly coupled to the one or more ceiling plenums or to the volume of the ceiling area between the ceiling plenums.

12. The data center according to claim 11, wherein: The one or more ceiling plenums are fluidly coupled to the cooling air plenum, and wherein the ceiling area includes at least one hot air exhaust outlet.

13. The data center according to claim 11, wherein: The one or more ceiling plenums are exhaust plenums fluidly coupled to the volume of the ceiling area or to an exterior of the data center.

14. The data center according to claim 11, wherein: The cooling air plenum is fluidly coupled to an exterior of the data center through at least one fresh air inlet.

15. The data center of claim 11, further comprising an external cooling module located external to the data center and fluidly coupled to the cooling air plenum, wherein: The external cooling module includes at least one of the following: a fan to pump airflow into the cooling air plenum; an air filter to filter air before it flows into the cooling air plenum; or A heat exchanger cools outside air entering the cooling air plenum.

16. The data center of claim 11, further comprising a recirculation module including a heat exchanger fluidly coupled to the information technology area, the ceiling area, and the cooling air plenum, wherein Hot air from the ceiling area and from the information technology area is exhausted to the recirculation module so that the heat exchanger can transfer heat from the hot air entering the recirculation module to outside air and return cool air to the cooling air plenum.

17. The data center according to claim 11, wherein: The cooling air plenum includes an economizer inlet for drawing in uncooled air and a booster inlet for drawing in cooling air.

18. The data center of claim 17, further comprising a cooling air plenum module located at or near the enhanced inlet, the cooling air plenum module comprising: a fan to pump airflow into the cooling air plenum; A heat exchanger cools the air before it is pumped into the cooling air plenum.

19. The data center according to claim 18, wherein: The heat exchanger includes an evaporator fluidly coupled to a condenser and a compressor to form a refrigerant circuit.

20. The data center of claim 17, wherein: The cooling air plenum includes a first plenum coupled to the economizer inlet and a second plenum coupled to the boost inlet.

Images