Electronic rack and it equipment cooling system

Abstract

An electronic rack includes one or more servers, where each server is contained within a respective server pod and each server is at least partially immersed in a two-phase liquid coolant, where when the servers generate heat, the heat is transferred to the two-phase liquid coolant, causing some of the two-phase liquid coolant to become vapor. The electronic rack includes a condenser pod and condenser coils mounted at a top of the electronic rack to condense the vapor into the two-phase liquid coolant. The electronic rack includes a vapor manifold along a length of the electronic rack that couples the condenser pod to the respective servers, where the vapor manifold carries the vapor from the servers to the condenser coils. The electronic rack includes a first return line coupled to the condenser pod to return the two-phase liquid coolant to a coolant unit.

Description

TECHNICAL FIELD

[0001] Embodiments of the present invention generally relate to server and data center cooling. More specifically, embodiments of the present invention relate to a high availability heterogeneous electronic rack solution. BACKGROUND

[0002] Thermal management for data centers that include several active electronic racks is critical to ensure proper performance of servers and other IT equipment (e.g., performing IT services) running in the racks. However, without proper thermal management, the thermal environment (e.g., temperature) within the racks can exceed thermal operating thresholds, which can result in adverse consequences (e.g., server failure, etc.). One method of managing the thermal environment is to use cooling air to cool the IT equipment. The cooling air is recirculated through a cooling unit. Heat generated by the IT equipment is captured by the cooling air and extracted by the cooling unit. One common cooling unit is a computer room air conditioning (CRAC) unit, which is a device that draws in hot exhaust air and supplies cooled air to maintain the thermal environment of the data center.

[0003] Recently, data centers have deployed electronic racks with higher power densities, where higher densities of chips are packed more closely together to provide greater computing power. Cooling these high density racks by maintaining a proper thermal environment can be problematic for existing cooling systems, such as CRAC units. For example, while a CRAC unit can maintain the thermal environment in the case of more conventional (or lower density) racks, the unit can not be able to effectively cool high power density racks, as the high power density racks can generate heat loads at a higher rate due to the higher density of electronics. Or it can be necessary to upgrade the CRAC system substantially to meet the cooling requirements of high density deployments. Another challenge to air cooling high density racks is moving a large volume of air flow sufficient to cool the racks.

[0004] On the other hand, immersion cooling, which involves at least partially submerging electronic devices in a dielectric solution, is a viable solution for high-density electronic devices. However, implementing immersion cooling in existing data centers presents challenges. For example, the cooling infrastructure of a data center can need to be modified to be able to support operating an immersion cooling system, as existing data centers are designed for air cooling or other types of liquid cooling. In addition, immersion cooling is a more complex cooling solution than existing air / liquid solutions. For example, single-phase immersion cooling requires complex hardware design for electronic components, mechanical pumps that can fail / leak, and significant space modifications for deployment in a data center. As another example, traditional two-phase immersion cooling systems include condensers that are packaged with (e.g., located above) the submerged electronic devices in an immersion tank. When maintenance is performed (e.g., when a server needs to be replaced), a live data center operator must remove the condensers from the tank, thereby breaking the existing cooling loop, which can result in loss of dielectric solution. In addition, to prevent overheating when maintenance is performed, the electronic devices within the tank can be shut down, which results in service interruption.

[0005] Existing solutions for immersion systems do not effectively address the challenges of a highly heterogeneous populated electronic rack, where the design and operation of each individual server populated on the electronic rack can vary from server to server. SUMMARY

[0006] Aspects of the present disclosure relate to electronic rack designs that implement two-phase cooling for servers. The servers are designed to be at least partially submerged in a two-phase coolant for cooling. The electronic rack includes a vapor manifold that carries vapor from the individual servers to a rack-mounted condenser near the top portion of the electronic rack. The condenser includes a liquid container, and the liquid contained in the container can be delivered to the servers. Independent coolant units (or coolant units) can manage the two-phase coolant populated to the servers at the electronic rack. The overall design allows multiple paths for the two-phase coolant to be provided to the servers to accommodate different server power conditions as well as different types of servers in a highly heterogeneous populated electronic rack.

[0007] According to one aspect, an electronic rack includes one or more servers, where each of the one or more servers is contained within a respective server container, and each of the one or more servers is at least partially immersed within a two-phase liquid coolant, where the one or more servers generate heat that is transferred to the two-phase liquid coolant while the one or more servers provide IT services, thereby causing at least some of the two-phase liquid coolant to transition into vapor. The electronic rack includes a condenser container mounted at a top portion of the electronic rack and a cooling condenser (or condensing coil) above the condenser container to condense the vapor into the two-phase liquid coolant. The electronic rack includes a vapor manifold disposed along a length of the electronic rack that couples the condenser container to the respective server containers of the one or more servers, where the vapor manifold directs the vapor from the one or more servers to the cooling condenser. The electronic rack includes a first return line coupled to the condenser container, where the first return line returns the two-phase liquid coolant contained in the condenser container to a coolant unit.

[0008] In one embodiment, the electronic rack further includes a pump along a length of the first return line to actively return the two-phase liquid coolant from the condenser container to the coolant unit. In one embodiment, the electronic rack further includes a liquid manifold disposed along a length of the electronic rack that is coupled to the condenser container. The electronic rack includes a second return line coupled between the liquid manifold and the coolant unit, where the first return line and / or the second return line return the two-phase liquid coolant from the condenser container to the coolant unit.

[0009] In one embodiment, the electronic rack includes a third return line coupled between the first return line and the second return line, and a valve along a length of the third return line to turn on / off a connection path for the two-phase liquid coolant to flow from the first return line to the second return line. In one embodiment, the liquid manifold is coupled between the first return line and the condenser container.

[0010] In one embodiment, the condenser container mounted at the top portion of the electronic rack includes an inlet port and an outlet port, where the pump circulates the cooling liquid through the coil of the cooling condenser. In one embodiment, the cooling liquid is a different cooling medium than the two-phase liquid coolant. In one embodiment, the inlet port and the outlet port are located at a side of the condenser container, and the inlet port and the outlet port circulate the cooling liquid through an interior of the coolant unit.

[0011] In one embodiment, the electronic rack further includes a supply line between the coolant unit and the one or more servers to supply the two-phase liquid coolant from the coolant unit to the one or more servers. In one embodiment, the coolant unit is located proximate to the electronic rack.

[0012] According to a second aspect, an information technology (IT) equipment cooling system includes a coolant unit to supply a two-phase liquid coolant and an electronics rack coupled to the coolant unit. The electronics rack includes one or more servers, where each of the one or more servers is contained within a respective server container and each of the one or more servers is at least partially immersed within the two-phase liquid coolant, where the one or more servers generate heat that is transferred to the two-phase liquid coolant while the one or more servers provide IT services, thereby causing at least some of the two-phase liquid coolant to transition into a vapor. The electronics rack includes a condenser container mounted to a top portion of the electronics rack and a cooling condenser housed in the condenser container to condense the vapor into the two-phase liquid coolant. The electronics rack includes a vapor manifold disposed along a length of the electronics rack that couples the condenser container to the respective server containers of the one or more servers, where the vapor manifold directs the vapor from the one or more servers to the cooling condenser. The electronics rack includes a first return line coupled to the condenser container, where the first return line returns the two-phase liquid coolant contained in the condenser container to the coolant unit. BRIEF DESCRIPTION OF DRAWINGS

[0013] Embodiments of the application are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like reference numerals refer to similar elements.

[0014] Figure 1 is a block diagram illustrating an example of a cooling system architecture according to one embodiment.

[0015] Figure 2 is a block diagram illustrating a cooling system with an additional fluid path from the condenser container to the coolant container according to one embodiment.

[0016] Figure 3 is a block diagram illustrating a cooling system with fluid connections at the fluid release manifold according to one embodiment.

[0017] Figure 4 is a block diagram illustrating a cooling system with fluid connections according to one embodiment.

[0018] Figure 5 is a block diagram illustrating a cooling system with fluid ports at adjacent coolant units according to one embodiment.

[0019] Figure 6 is a block diagram illustrating a cooling system with a coolant path according to one embodiment.

[0020] Figure 7 is a block diagram illustrating a cooling system with an additional coolant path according to one embodiment.

[0021] Figure 8 is a block diagram illustrating a facility loop connected with one or more cooling systems, according to one embodiment. DETAILED DESCRIPTION

[0022] Various embodiments and aspects of the application will be described in detail with reference to the drawings and the following detailed description. The drawings are not intended to limit the application, but to illustrate the various embodiments and aspects of the application. Numerous specific details are described to provide a thorough understanding of various embodiments and aspects of the application. However, in certain instances, well-known or conventional details are not described in order to provide a concise discussion of embodiments of the application.

[0023] Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

[0024] Figure 1 is a block diagram illustrating an example of a cooling system architecture 100, according to one embodiment. The cooling system architecture 100 can include an electronics rack 103 and a coolant unit 105. A condensing unit 115 can be mounted on top of the electronics rack 103, and the condensing unit 115 can be packaged with a condenser vessel 116. The electronics rack 103 can include one or more servers 117-1, 117-2, 117-3 populated on the electronics rack 103, where the servers 117-1, 117-2, 117-3 are housed in respective server vessels or immersion tanks, and the server vessels are connected to a vapor manifold 122. Vapor generated within the servers 117-1, 117-2, 117-3 can travel through ports 121-1, 121-2, 121-3 via the vapor manifold 122 to the condensing unit 115. Each of the servers 117 can include one or more electronic devices, such as processors, memory, storage devices, etc., that generate heat when in operation.

[0025] The electronics rack 103 can include a set of liquid distribution manifolds, where the set of liquid distribution manifolds includes a fluid supply manifold 123 and a fluid release manifold 125. The server fluid supply manifold 123 and the server fluid release manifold 125 can supply / release two-phase liquid coolant to / from the servers 117-1, 117-2, 117-3, respectively. Although only three servers are shown, the cooling system architecture 100 can include any number of servers.

[0026] The coolant unit 105 can include a coolant reservoir 111 to store the two-phase liquid coolant. In one embodiment, the coolant reservoir 111 is connected to the condenser reservoir 116 of the condensing unit 115. In one embodiment, the coolant reservoir 111 is connected to the fluid supply manifold 123 and the fluid release manifold 125. In one embodiment, the prime pump 107 is coupled between the fluid supply manifold 123 and the coolant reservoir 111, and the prime pump 107 can prime the coolant from the coolant reservoir 111 to the fluid supply manifold 123. The drain pump 109 can be coupled between the fluid release manifold 125 and the coolant reservoir 111, and the drain pump 109 can release the coolant from the fluid release manifold 125 to the coolant reservoir 111. In one embodiment, the coolant unit 105 includes a fluid level sensor 127, where a signal of the fluid level sensor 127 is used to regulate the fluid level at the coolant unit 105. For example, the signal can be used by a facility level coolant supply and return to regulate the two-phase liquid coolant level in the coolant reservoir 111.

[0027] In one embodiment, the servers 117-1, 117-2, 117-3 are designed with immersion cooling, where each server 117-1, 117-2, 117-3 is contained within a respective server container. Each of the one or more servers 117-1, 117-2, 117-3 is at least partially submerged within the two-phase liquid coolant within the respective container, where the one or more servers 117-1, 117-2, 117-3 generate heat that is transferred to the two-phase liquid coolant as the one or more servers 117-1, 117-2, 117-3 provide IT services, causing at least some of the two-phase liquid coolant to change to a vapor state.

[0028] The vapor within the respective containers of the servers 117-1, 117-2, 117-3 can travel to the condensing unit 115, and the condensing coils 113 of the condensing unit 115 condense the vapor back to a liquid phase coolant, where the liquid phase coolant can collect in the condenser reservoir 116. Here, a cooling liquid can be supplied to the condensing coils 113 from a liquid cooling source via the recirculation pump 119, where the cooling liquid will circulate through the condensing coils 113. The cooling liquid then carries the heat extracted from the vapor through the condensing coils 113 and exits the condensing unit 115. In one embodiment, the two-phase liquid coolant is directed from the condenser reservoir 116 to the coolant reservoir 111 of the coolant unit 105 via the line 129 by gravity.

[0029] In one embodiment, the cooling liquid circulated from the liquid cooling source is different from the two-phase liquid coolant. In one embodiment, the cooling liquid is a single-phase cooling fluid such as chilled water and the two-phase liquid coolant is a dielectric solution.

[0030] The two-phase liquid coolant can be pumped to the servers 117-1, 117-2, 117-3. In one embodiment, the two-phase liquid coolant is pumped from the coolant reservoir 111 to the servers 117-1, 117-2, 117-3 continuously or at predetermined time intervals, thereby allowing the two-phase liquid coolant to continuously extract heat generated by the IT electronics housed in the servers 117-1, 117-2, 117-3. The two-phase liquid coolant will undergo a change of state from a liquid state to a vapor state, with the vapor exiting the servers 117-1, 117-2, 117-3 via the vapor manifold 122. Here, the liquid-filled pump 107 can periodically pump the two-phase liquid coolant in a liquid phase to the servers 117-1, 117-2, 117-3 at regular intervals or based on the sensed temperature of the servers. In one embodiment, the drain pump 109 is operated when the individual servers 117-1, 117-2, 117-3 need to be relieved of the two-phase liquid coolant, for example, for repair and maintenance. It is noted that the servers 117-1, 117-2, 117-3 can include control valves (not shown) at respective ports 129-1, 129-2, 129-3, 130-1, 130-2, 130-3 coupled to the fluid supply manifold 123 and / or the fluid relief manifold 125, respectively, to control when fluid enters / leaves the respective servers 117-1, 117-2, 117-3. In summary, Figure 1 It is shown that the liquid coolant can flow between the condenser reservoir 116 and the coolant reservoir 111 through the line 129 or the fluid relief manifold 125.

[0031] Figure 2 is a block diagram showing a cooling system 200 having an additional fluid path from the condenser reservoir 116 to the coolant reservoir 111, according to one embodiment. The cooling system 200 can represent Figure 1 the cooling system architecture 100 of FIG. 1. In one embodiment, the cooling system 200 can include a direct connection line 129 and a secondary connection line 201 to release liquid from the condenser reservoir 116 to the coolant reservoir 111, where the secondary connection line 201 extends from the direct connection line 129 to the drain pump 109. This embodiment allows the drain pump 109 to assist in the flow of the two-phase fluid from the condenser reservoir 116 to the coolant reservoir 111. In one embodiment, the secondary connection line 201 includes a valve 203 that is controllable to open or close the secondary connection line 201. In one embodiment, the fluid relief manifold 125 can or can not be connected to the condensing unit 115.Figure 2 As can be seen, in this design variation, the condensing unit 115 is not connected to the fluid release manifold 125.

[0032] Figure 3 is a block diagram illustrating a cooling system 300 having fluid connections at the fluid release manifold, according to one embodiment. The cooling system 300 can represent the cooling system architecture 100 of Figure 1 In one embodiment, the cooling system 300 can include a connection line 129, where the connection line 129 is coupled between the fluid release manifold 125 and the coolant reservoir 111. The condensing unit 115 is connected to the fluid release manifold 125. This design variation can eliminate the exit port at the condenser reservoir 116, which would otherwise be required if the connection line 129 were to be coupled to the condenser reservoir 116.

[0033] Figure 4 is a block diagram illustrating a cooling system 400 having fluid connections, according to one embodiment. The cooling system 400 can represent the cooling system architecture 100 of Figure 1 In one embodiment, the cooling system 400 includes only one connection line 129 between the fluid release manifold 125 and the coolant reservoir 111. The connection line 129 can include the drain pump 109, where the connection line 129 serves as the fluid release for the servers 117-1, 117-2, 117-3, and the condensing unit 115.

[0034] Figure 5 is a block diagram illustrating a cooling system 500 having a cooling liquid port at the coolant unit 105, according to one embodiment. The cooling system 500 can represent the cooling system architecture 100 of Figure 1 The cooling system 500 illustrates a design variation where the condensing unit 115 receives cooling liquid through the coolant unit 105, and the coolant unit 105 includes ports 501, 503 to direct the coolant liquid loop through the coolant unit 105. Here, the condensing unit 115 receives cooling liquid from a liquid cooling source through the coolant unit 105. In one embodiment, the condensing unit 115 is an air-cooled condensing unit and includes an air-cooled condenser. Thus, this design variation allows the coolant unit 105 to house all the fluid ports (two-phase liquid coolant and cooling liquid) charged / discharged from the electronics rack 103.

[0035] Figure 6 is a block diagram illustrating a cooling system of the coolant path, according to one embodiment. Figure 6 illustrates a two-phase coolant path for a cooling system 600. The cooling system 600 can represent the cooling system architecture 100 of Figure 1 In one embodiment, the cooling system 600 includes a connection line 629, where the connection line 629 is coupled between the fluid release manifold 625 and the coolant reservoir 611. The condensing unit 615 is connected to the fluid release manifold 625. The connection line 629 can include the drain pump 609, where the connection line 629 serves as the fluid release for the servers 617-1, 617-2, 617-3, and the condensing unit 615. Figure 6dashed path 601 illustrates the vapor path rising to the condensing unit 115. Path 603 illustrates the recirculation pump 119 supplying the cooling fluid to the condensing unit 115 condensing coils 113 for condensing the vapor into a liquid phase by the condensing coils 113. Path 605 illustrates that the directed fluid flow from the condenser vessel 116 to the coolant vessel 111 by the line 129 can be driven by gravity or a pump. Path 607 illustrates that the charge pump 107 can pump the two-phase liquid coolant from the coolant vessel 111 to the servers 117-1, 117-2, and 117-3.

[0036] Figure 7 is a block diagram illustrating a cooling system with additional coolant paths according to one embodiment. The cooling system 700 can represent the cooling system 600 of Figure 6 . In addition to the paths illustrated in Figure 6 , Figure 7 an additional two-phase coolant path 609 from the condenser vessel 116 to the coolant vessel 111 is illustrated. The path 609 allows the cooling system 700 to quickly drain the two-phase liquid coolant from the condenser vessel 116 to the coolant vessel 111. These variations in the design for the additional path 609 can remove the heat load from the servers 117-1, 117-2, and 117-3 at different rates. For example, a cooling system with a server with a high heat capacity can accommodate the additional drain path 609 for the cooling system 700. In one embodiment, the paths 605, 607, and 609 can be independently controlled.

[0037] Figure 8 is a plan view illustrating a block diagram of a facility loop 801 connecting one or more cooling systems 100A-100F according to one embodiment. The facility system 800 includes a facility loop 801 connecting to the respective coolant units 105A-105F of the cooling systems 100A-100F. In one embodiment, the coolant units 105A-105F are filled and commissioned prior to the deployment of the corresponding electronic racks. As can be seen, such a facility system 800 allows a fast and efficient solution for deploying electronic racks and servers since the coolant units 105A-105F can be made to pass both the cooling fluid and the two-phase cooling coolant through the coolant units 105A-105F.

[0038] In the foregoing specification, specific embodiments of the application have been described in some detail. It will be apparent to those skilled in the art that various modifications can be made to the described embodiments without departing from the broader spirit and scope of the application as set forth in the appended claims. Thus, the specification and drawings are to be regarded as illustrative rather than restrictive.

Claims

1. Electronic rack, including: One or more servers, wherein each of the one or more servers is contained within a respective server container, and each of the one or more servers is at least partially immersed in a two-phase liquid coolant, wherein when the one or more servers provide information technology services, the one or more servers generate heat transferred to the two-phase liquid coolant, thereby causing at least a portion of the two-phase liquid coolant to turn into vapor; A condenser container is mounted on the top portion of the electronic rack; A cooling condenser, housed in the condenser container, for condensing the vapor into the two-phase liquid coolant; A steam manifold, arranged along the length of the electronic rack, connects the condenser container to a corresponding server container of the one or more servers, wherein the steam manifold directs steam from the one or more servers to the cooling condenser. A fluid supply manifold is provided along the length of the electronic rack, and the fluid supply manifold supplies two-phase liquid coolant to each of the servers; A fluid release manifold, arranged along the length of the electronics rack, is connected to the condenser container and releases two-phase liquid coolant from each of the servers; and A first return line is connected to the condenser container, wherein the first return line returns the two-phase liquid coolant contained in the condenser container to the coolant unit; A second return line is connected between the fluid release manifold and the coolant unit, wherein the first return line and / or the second return line returns two-phase liquid coolant from the condenser container to the coolant unit; A third return pipeline is connected between the first return pipeline and the second return pipeline; The coolant unit has a cooling liquid port; the condenser container includes an inlet port and an outlet port, which are located on one side of the condenser container, and the inlet port and the outlet port allow the cooling liquid to circulate through the interior of the coolant unit.

2. The electronic rack according to claim 1, further comprising: A pump, along the length of the first return line, actively returns the two-phase liquid coolant from the condenser container to the coolant unit.

3. The electronic rack according to claim 1, further comprising: A valve, along the length of the third return line, to connect / disconnect the connection path for the two-phase liquid coolant to flow from the first return line to the second return line.

4. The electronic rack according to claim 1, wherein, The fluid release manifold is connected between the first return line and the condenser container.

5. The electronic rack according to claim 1, wherein, The pump circulates the cooling liquid through the coils of the cooling condenser.

6. The electronic rack according to claim 5, wherein, The cooling condenser includes an air-cooled condenser unit.

7. The electronic rack of claim 1, further comprising a supply line located between the coolant unit and the one or more servers for supplying a two-phase liquid coolant from the coolant unit to the one or more servers.

8. The electronic rack according to claim 1, wherein, The coolant unit is located near the electronic rack.

9. A cooling system for information technology equipment, including: The coolant unit is used to supply two-phase liquid coolant; as well as An electronic rack, connected to the coolant unit, the electronic rack comprising: One or more servers, wherein each of the one or more servers is contained within a respective server container, and the servers are at least partially immersed in a two-phase liquid coolant, wherein when the one or more servers provide IT services, the one or more servers generate heat that is transferred to the two-phase liquid coolant, thereby causing at least a portion of the two-phase liquid coolant to turn into vapor; A condenser container is mounted on the top portion of the electronic rack; A cooling condenser, housed in the condenser container, for condensing the vapor into the two-phase liquid coolant; A steam manifold, arranged along the length of the electronic rack, connects the cooling condenser to a corresponding server container of the one or more servers, wherein the steam manifold directs steam from the one or more servers to the cooling condenser; A fluid supply manifold is provided along the length of the electronic rack, and the fluid supply manifold supplies two-phase liquid coolant to each of the servers; A fluid release manifold, arranged along the length of the electronics rack, is connected to the condenser container and releases two-phase liquid coolant from each of the servers; and A first return line is connected to the condenser container, wherein the first return line returns the two-phase liquid coolant contained in the condenser container to the coolant unit; A second return line is connected between the fluid release manifold and the coolant unit, wherein the first return line and / or the second return line returns two-phase liquid coolant from the condenser container to the coolant unit; A third return pipeline is connected between the first return pipeline and the second return pipeline; The coolant unit has a cooling liquid port; the condenser container includes an inlet port and an outlet port, which are located on one side of the condenser container, and the inlet port and the outlet port allow the cooling liquid to circulate through the interior of the coolant unit.

10. The information technology equipment cooling system according to claim 9, further comprising: A pump, along the length of the first return line, actively returns the two-phase liquid coolant from the condenser container to the coolant unit.

11. The information technology equipment cooling system according to claim 9, further comprising: A valve, along the length of the third return line, to connect / disconnect the connection path for the two-phase liquid coolant to flow from the first return line to the second return line.

12. The information technology equipment cooling system according to claim 9, wherein, The fluid release manifold is connected between the first return line and the condenser container.

13. The information technology equipment cooling system according to claim 9, wherein, The pump circulates the cooling liquid through the coils of the cooling condenser.

14. The information technology equipment cooling system according to claim 13, wherein, The cooling condenser includes an air-cooled condenser unit.

15. The information technology equipment cooling system of claim 9, further comprising a supply line located between the coolant unit and the one or more servers to supply a two-phase liquid coolant from the coolant unit to the one or more servers.

16. The information technology equipment cooling system according to claim 9, wherein, The coolant unit is located near the electronic rack.

Images