Cooling system for single-phase immersion servers

Abstract

In one embodiment, an immersion cooling system includes a vessel containing a first coolant received from a first cooling source and server chassis at least partially immersed in the first coolant. Each server chassis includes electronic devices and a cooling plate attached thereto to extract at least a portion of heat generated by the electronic devices. The cooling plate includes an inlet to receive a second coolant from a second cooling source, coolant channels to distribute the second coolant, and an outlet to return the second coolant to the second cooling source. The cooling system further includes a return manifold coupled to the second cooling source, the return manifold having one or more manifold return connectors respectively coupled to the server chassis for receiving the second coolant from the server chassis and returning the second coolant to the second cooling source.

Description

TECHNICAL FIELD

[0001] Embodiments of the present invention relate generally to data centers. More specifically, embodiments of the present invention relate to a system for single-phase immersion servers. BACKGROUND

[0002] Thermal management of data centers that include several active electronic racks is critical to ensure proper performance of servers and other IT equipment operating in the racks. Without proper thermal management, the thermal environment within the racks can exceed operational thresholds, resulting in operational consequences (e.g., server failure, etc.). One method of managing the thermal environment is to use cooling air to cool the IT equipment. Most existing data centers are air-cooled. Recently, data centers have been deploying more high-power density electronic racks, where more high-density chips are packed more closely together to provide more processing power. Cooling these high-density racks by maintaining a proper thermal environment can be a challenge for existing cooling systems, especially as the power density of the racks continues to increase.

[0003] Existing solutions for single-phase immersion systems only consider rack-level fluid recirculation without any local cooling acceleration. As a result, existing solutions do not account for different power density applications and configurations in the racks. Existing solutions also provide non-targeted and limited cooling capabilities by immersing in fluid outside the server enclosure. SUMMARY

[0004] An immersion cooling system includes a container to house a first coolant received from a first cooling source, one or more server enclosures immersed at least partially in the first coolant, wherein each of the server enclosures includes an electronic device and a cold plate attached to the electronic device to extract at least a portion of heat generated by the electronic device, wherein the cold plate includes an inlet to receive a second coolant from a second cooling source, a coolant channel to distribute the second coolant, and an outlet to return the second coolant to the second cooling source, and a return manifold coupled to the second cooling source, the return manifold having one or more manifold return connectors coupled respectively to the server enclosures, and the return manifold to receive the second coolant from the server enclosures and return the second coolant to the second cooling source.

[0005] In some embodiments, each of the manifold return connectors is configured to engage and connect with the outlet of the cold plate of the corresponding server enclosure when the corresponding server enclosure is immersed in the first coolant and down onto the return manifold.

[0006] In some embodiments, each outlet of each cooling plate includes a plate return connector, and wherein each plate return connector and each manifold return connector is a blind mate connector.

[0007] In some embodiments, the container further comprises: a main inlet coupled to the first cooling source by a coolant supply line to receive the first coolant from the first cooling source; and a main outlet coupled to the first cooling source by a coolant return line to circulate the first coolant back to the first cooling source.

[0008] In some embodiments, the return manifold is disposed to be submerged in the first coolant and located near a bottom of the container and below the server chassis.

[0009] In some embodiments, the system further comprises a supply manifold disposed above the server chassis to receive a second coolant from a second cooling source, wherein the supply manifold comprises one or more manifold supply connectors each coupled to an inlet of the cooling plate of the server chassis to supply the second coolant to the server chassis.

[0010] In some embodiments, the system further comprises a first fluid pump coupled between the supply manifold and the second cooling source to pump the second coolant into the supply manifold.

[0011] In some embodiments, the first fluid pump is configured to receive and pump the second coolant from the first coolant contained within the container.

[0012] In some embodiments, the system further comprises a second fluid pump disposed between at least one cooling plate and the supply manifold to pump the second coolant from the supply manifold to the corresponding server chassis.

[0013] In some embodiments, the system further comprises a plate supply connector coupled to an inlet of each cooling plate by a flexible hose, wherein the plate supply connector is connectable with one of the manifold supply connectors of the supply manifold.

[0014] In some embodiments, the first cooling source and the second cooling source are the same cooling source external to the container.

[0015] In some embodiments, an inlet of each cooling plate is submerged in the first coolant contained within the container to receive a portion of the first coolant as the second coolant.

[0016] A server chassis for immersion cooling includes a server frame, a printed circuit board mounted on the server frame, the printed circuit board having one or more electronic devices mounted thereon, and one or more cooling plates attached to the one or more electronic devices, respectively, to extract heat generated from the electronic devices, wherein the server frame, the printed circuit board with electronic devices, and the cooling plates are to be immersed in a first coolant received from a first cooling source and housed within an immersion tank. Each of the cooling plates includes an inlet to receive a second coolant from a second cooling source, a coolant channel to distribute the second coolant, and an outlet to return the second coolant to the second cooling source, wherein the outlet is to be coupled to a return manifold within the immersion tank, the return manifold coupled to the second cooling source, the return manifold having a manifold return connector coupled to the outlet to receive the second coolant from the cooling plate and return the second coolant to the second cooling source.

[0017] In some embodiments, the manifold return connector is configured to engage and connect with the outlet of the cooling plate when the server chassis is immersed in the first coolant and down onto the return manifold.

[0018] In some embodiments, the outlet includes a plate return connector, and wherein the plate return connector and the manifold return connector are blind-mating connectors that are engageable and connectable with each other.

[0019] In some embodiments, the inlet of the cooling plate is coupled to a plate supply connector by a flexible hose, and wherein the plate supply connector is connectable with a supply manifold to receive the second coolant.

[0020] A cooling plate for immersion cooling includes a substrate attached to an electronic device of a server chassis, wherein the server chassis with the electronic device is to be immersed in a first coolant received from a first cooling source and housed within an immersion vessel, an inlet to receive a second coolant from a second cooling source, a coolant channel to distribute the second coolant, and an outlet to return the second coolant to the second cooling source, wherein the outlet is coupled to a return manifold within the immersion vessel, the return manifold coupled to the second cooling source, the return manifold having a manifold return connector coupled to the outlet to receive the second coolant from the cooling plate and return the second coolant to the second cooling source.

[0021] In some embodiments, the manifold return connector is configured to engage and connect with the outlet when the server chassis is immersed in the first coolant and down onto the return manifold.

[0022] In some embodiments, the outlet includes a panel return connector, and wherein the panel return connector and the manifold return connector are blind mate connectors that are engageable and connectable with each other.

[0023] In some embodiments, the inlet is coupled to a panel supply connector by a flexible hose, and wherein the panel supply connector is connectable with a supply manifold to receive the second coolant. BRIEF DESCRIPTION OF DRAWINGS

[0024] Embodiments of the present invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like reference numerals refer to similar elements.

[0025] Figure 1 is a system diagram illustrating a cooling design of an immersion tank with a supply manifold for a server cooling loop according to one embodiment.

[0026] Figure 2 is a system diagram illustrating a cooling design of an immersion tank with a supply manifold for a separate liquid cooling loop according to one embodiment.

[0027] Figure 3A shows a server design including an internal cooling module in a complete cooling loop according to one embodiment.

[0028] Figure 3B is a server design including a local pump for an internal cooling module of a server within an immersion cooling system according to another embodiment.

[0029] Figure 4 shows a server chassis design according to one embodiment.

[0030] Figure 5A is an immersion cooling system with a server having an internal cooling module with a direct access opening for a local cooling loop according to another embodiment.

[0031] Figure 5B is an immersion cooling system with a server having an internal cooling module with a direct access opening for a local cooling loop according to one embodiment.

[0032] Figure 6is a server design with a cooling module having a direct liquid inlet and liquid outlet for a local cooling loop in an immersion cooling system according to another embodiment. DETAILED DESCRIPTION

[0033] Various embodiments and aspects of the application will be described with reference to details discussed below, and the accompanying drawings will illustrate various embodiments. The following description and drawings are illustrative of the application and are not to be construed as limiting the application. Numerous specific details are described to provide a thorough understanding of various embodiments of the present application. However, in certain instances, well-known or conventional details are not described in order to provide a concise discussion of embodiments of the present applications.

[0034] Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

[0035] Embodiments of the present disclosure provide local cooling loops for servers within a single phase immersion cooling system. The design of the immersion system can be implemented for electronic racks and clusters. The cooling architecture can include an immersion tank based computing cluster. The design includes a built-in manifold (e.g., an immersion tank) in an IT enclosure. The manifold can include connectors (e.g., blind mate connectors) to connect servers located in the enclosure. The servers can include a cooling module with internal cooling channels to provide local cooling within the servers located in the enclosure. In one embodiment, the servers can be connected from the top of the servers to a second manifold (e.g., a supply manifold) to receive coolant. In another embodiment, the servers can include an opening to draw in coolant directly from the enclosure (e.g., from the fluid in which the servers are immersed). In one embodiment, the second manifold is connected to a second coolant source (e.g., an external coolant source). In another embodiment, the second manifold is designed with an inlet to draw directly from the enclosure (e.g., from the fluid in the immersion tank). The servers are designed with a fluid inlet to receive fluid from the second manifold or directly from the coolant in the immersion tank.

[0036] Accordingly, embodiments of the present disclosure can enable the management of immersion cooling for different servers with different thermal loads in an immersion system. Accordingly, the system can provide targeted cooling for hot spots and high density chips in an immersion cooling system. In addition, the system provides flexibility to use one single phase coolant for all cooling loops in the system and the immersion tank or to use separate coolants for the server cooling loops and the immersion tank.

[0037] According to one aspect, an immersion cooling system includes a vessel to house a first coolant received from a first cooling source and one or more server chassis at least partially immersed in the first coolant. Each server chassis includes electronic devices and a cooling plate attached to the electronic devices to extract at least a portion of heat generated by the electronic devices. The cooling plate includes an inlet to receive a second coolant from a second cooling source, a coolant channel to distribute the second coolant, and an outlet to return the second coolant to the second cooling source. The cooling system further includes a return manifold coupled to the second cooling source, the return manifold having one or more manifold return connectors coupled to the server chassis, respectively, and the return manifold to receive the second coolant from the server chassis and return the second coolant to the second cooling source.

[0038] In one embodiment, each manifold return connector is configured to engage and connect with the outlet of the cooling plate of the corresponding server chassis when the corresponding server chassis is immersed in the first coolant and down onto the return manifold. Each outlet of each cooling plate includes a plate return connector, each plate return connector and each manifold return connector being a blind mate connector.

[0039] In one embodiment, the vessel further includes a main inlet coupled to the first cooling source by a coolant supply line to receive the first coolant from the first cooling source and a main outlet coupled to the first cooling source by a coolant return line to circulate the first coolant back to the first cooling source. The return manifold is disposed immersed in the first coolant and proximate to a bottom of the vessel and below the server chassis.

[0040] In one embodiment, the system further includes a supply manifold disposed above the server chassis to receive the second coolant from the second cooling source. The supply manifold includes one or more manifold supply connectors, each coupled to the inlet of the cooling plate of the server chassis to supply the second coolant to the server chassis. The system further includes a first fluid pump coupled between the supply manifold and the second cooling source to pump the second coolant into the supply manifold. The first fluid pump is configured to receive and pump the second coolant from the first coolant housed within the vessel.

[0041] In one embodiment, the system further includes a second fluid pump disposed between the at least one cooling plate and the supply manifold to pump the second coolant from the supply manifold to the corresponding server chassis. The system can further include a plate supply connector coupled to the inlet of each cooling plate by a flexible hose. The plate supply connector is connectable with one of the manifold supply connectors of the supply manifold. The first and second cooling sources are the same cooling source, located outside the container. The inlet of each cooling plate is immersed in the first coolant contained within the container to receive a portion of the first coolant as the second coolant.

[0042] According to another aspect, a server chassis for immersion cooling includes a server frame; a printed circuit board (PCB) mounted on the server frame, the PCB having one or more electronic devices mounted thereon; and one or more cooling plates attached to the one or more electronic devices, respectively, to extract heat generated by the electronic devices. The server frame, the PCB with the electronic devices, and the cooling plates are to be immersed in a first coolant received from a first cooling source and contained within an immersion tank. Each cooling plate includes an inlet to receive a second coolant from a second cooling source; a coolant channel to distribute the second coolant; and an outlet to return the second coolant to the second cooling source. The outlet is to be coupled to a return manifold within the immersion tank, the return manifold coupled to the second cooling source. The return manifold includes a manifold return connector to couple with the outlet and to receive the second coolant from the cooling plate and return the second coolant to the second cooling source.

[0043] According to one embodiment, the manifold return connector is configured to engage and connect with the outlet of the cooling plate when the server chassis is immersed in the first coolant and down onto the return manifold. The outlet includes a plate return connector, and wherein the plate return connector and the manifold return connector are blind-mating connectors that can engage and connect with each other. The inlet of the cooling plate is coupled to a plate supply connector by a flexible hose, and the plate supply connector is connectable with a supply manifold to receive the second coolant.

[0044] According to another aspect, a cooling plate for immersion cooling includes a substrate attached to an electronic device of a server chassis, wherein the server chassis with the electronic device is to be immersed in a first coolant received from a first cooling source and contained within an immersion container; an inlet to receive a second coolant from a second cooling source; a coolant channel to distribute the second coolant; and an outlet to return the second coolant to the second cooling source. The outlet is to be coupled to a return manifold within the immersion container, the return manifold coupled to the second cooling source. The return manifold includes a manifold return connector to couple with the outlet and to receive the second coolant from the cooling plate and return the second coolant to the second cooling source.

[0045] In one embodiment, the manifold return connector is configured to engage and connect with the outlet when the server chassis is submerged in the first coolant and down onto the return manifold. The outlet includes a board return connector, and both the board return connector and the manifold return connector are blind mate connectors that can engage and connect with each other. The inlet is coupled to the board supply connector by a flexible hose, and wherein the board supply connector can connect with the supply manifold to receive the second coolant.

[0046] Figure 1 A cooling system design is shown, including a submersion tank with a fluid supply manifold designed with a direct intake circuit from coolant in a submersion tank reservoir. The cooling system can use a submersion fluid, where the entire server is submerged in the submersion coolant. Each server can connect with a return manifold through one or more blind mate connectors. The return manifold can be disposed near the bottom of the submersion tank, which includes or is coupled to a pump to move fluid through the circuit. In Figure 1 In the depicted example, each server is connected to a supply manifold, which has a direct intake circuit for accelerating fluid flow from the submersion tank reservoir (also referred to as a submersion vessel or coolant vessel) to internal fluid passages of the individual servers as part of local cooling of the servers. The servers are integrated with cooling modules (also referred to as cooling plates or cold plates) with internal fluid passages. Fluid is accelerated through the cooling modules and internal fluid passages (e.g., by a pump). Thus, fluid flows through the submersion tank reservoir as part of global submersion cooling and through the internal fluid passages and cooling modules of the servers as part of local cooling.

[0047] In one embodiment, referring to Figure 1 The cooling system 100 includes a submersion vessel 102 filled with a submersion coolant 104 for cooling one or more servers 110A-C located in the submersion cooling tank 102 by vertically inserting each server into a particular server slot from the top of the submersion vessel 102. It is noted that the submersion vessel 102 can also be referred to as a submersion tank. The terms submersion vessel and submersion tank are interchangeable terms in this application. The submersion coolant 104 can be a single-phase coolant. The system 100 can also include a supply manifold 105 disposed near the top of the submersion vessel 102 and a return manifold 115 disposed near the bottom of the submersion vessel 102.

[0048] Servers 110A-C located in the immersion cooling tank 102 can be coupled to the return manifold 115 through blind mate connectors. The servers 110A-C can also be coupled to the supply manifold through manual connectors, blind mate connectors, or any other fluidic connection such as a manual fit connector. As depicted, the supply manifold 105, servers 110A-C, and return manifold 115 can constitute one or more cooling loops through which coolant can flow within cooling plates (not shown) attached to electronic devices of the servers to extract heat from the electronic devices of the servers, while the immersion coolant 104 in which the servers are immersed circulates around the outer surfaces of the components of the servers such as the outer surfaces of the electronic devices and cooling plates.

[0049] According to one embodiment, the return manifold 115 includes an array of manifold return connectors, such as manifold return connector 120B, spaced along the manifold and facing upward. Each manifold return connector defines a server slot in which a server chassis having a server (such as servers 110A-110C) therein can be inserted downward from the top of the immersion vessel 102 to couple with the corresponding manifold return connector. It is noted that while three servers are shown, more or fewer servers can also be applied here.

[0050] In one embodiment, each of the servers 110A-110C includes a server return connector coupled to an outlet of a cooling plate attached to an electronic device (e.g., processor) of the server. Details of the server structure and cooling plate will be further described below. According to one embodiment, the server return connector and the manifold return connector are corresponding blind mate connectors. When the server is placed downward in the server slot, the server return connector and the corresponding manifold return connector of the return manifold 115 will automatically engage and connect to establish a fluidic connection between the cooling plate and the return manifold 115.

[0051] For example, the server chassis 110C includes a cooling plate attached to an electronic device within the server chassis 110C (not shown). The server return connector 120A (e.g., a blind mate connector) is disposed and coupled to an outlet of the cooling plate. When the server chassis 110C is placed downward into the server slot corresponding to the manifold return connector 120B, the server return connector 120A and the manifold return connector 120B automatically engage and connect when they come into contact with each other. Other server chassis such as server chassis 110A-110B can have the same or similar structure or configuration as the server chassis 110C.

[0052] Further, according to one embodiment, the supply manifold 105 includes an array of manifold supply connectors, such as manifold supply connector 121B, facing downward along the supply manifold. Each manifold supply connector corresponds to one of the server slots. Each server chassis includes a server supply connector, such as server supply connector 121A, coupled to an inlet of a cooling plate attached to the electronic devices within the server chassis. The server supply connector can be coupled to an outlet of the cooling plate by a flexible hose. The server supply connector can be connected to the manifold supply connector corresponding to the server slot. The supply manifold 105 can be mounted as part of the lid of the immersion vessel 102.

[0053] In one embodiment, a fluid pump 125 can be coupled to the return manifold 115 to accelerate the coolant that passes through the supply manifold 105, the servers 110A-C, the return manifold 115, and then to the external cooling source or cooling system 150 via the return line 113. In some embodiments, the supply manifold 105 can include a pump 108 to draw some of the coolant 104 directly into the supply manifold 102. In embodiments, the accelerated coolant flowing through the server chassis 110A-110C is 100% of the immersion coolant 104 pumped by the pump 108. Thus, the immersion coolant 104 can be used for both external immersion cooling of the servers 110A-110C as well as direct and local cooling through and into the cooling modules within the servers 110A-110C.

[0054] Further, the immersion vessel 102 also includes a main inlet coupled to the main supply line 131 to receive immersion coolant 104 from the external cooling source or cooling system 150. The immersion vessel 102 can also include a main outlet coupled to the main return line 132 to circulate at least some of the immersion coolant 104 back to the cooling source or cooling system 150 to form a main immersion cooling loop or global immersion cooling loop. The cooling loop from the immersion coolant 104 to the supply manifold 105, through any one of the servers, to the return manifold 115, to the external cooling source 150 via path 133, and then to the immersion coolant 104 via path 131 is referred to as a local cooling loop. Thus, each server chassis is associated with its own local cooling loop.

[0055] It is noted that the pump speeds of the pumps 108 and 125 can be independently controlled, for example, by a controller, while the fluid flow rates on the supply line 131 and the return line 132 can also be controlled differently. Additional pumps can be provided on the supply line 131 and / or the return line 132. The fluid flow rate of the local loop can be controlled based on the local temperature within the cooling plate of the server chassis. The fluid flow rate of the global immersion cooling loop can be determined based on the global temperature within the immersion coolant 104 as well as the fluid / coolant level.

[0056] Figure 2 A system design is shown with a supply manifold connected to a separate fluid circuit from the immersion tank. The depicted design provides for different fluids and different flow rates through the internal fluid passageways of the servers and the immersion tank. The immersion tank coolant is used for external cooling of the servers by immersing the servers in the coolant, and the individual servers packaged with the cooling modules are connected with the supply and return manifolds for thermal management by a second fluid pumped through the internal fluid passageways of the servers.

[0057] In one embodiment, with reference to Figure 2 , the cooling system 200 differs from the cooling system 100 of Figure 1 in that the supply manifold 105 can include a full circuit single phase input, rather than a direct input of immersion coolant from the immersion vessel 102. For example, the supply manifold 105 can receive coolant from an external source (in this example, from the cooling source 150 via the supply line 134). It is noted that in this example, the first coolant 104 in the global cooling circuit via paths 131-132 and the second coolant in the local cooling circuit via paths 133-134 can be drawn from the same cooling source 150.

[0058] In another embodiment, the second coolant for local cooling can be provided by a different cooling source than the cooling source 150. For example, the second coolant can be a single phase coolant and the first coolant can be a two phase coolant. When the two phase coolant warms

[0059] up, it can change from a liquid form to a vapor. When the temperature drops, the vapor can change back to the liquid form. In the configuration where the first coolant 104 is a two phase coolant, the main outlet and return lines 132 can be positioned above the coolant level so that vapor can be collected. A local condenser can be implemented within the immersion vessel 102 and above the coolant level to locally condense the vapor back to the liquid form.

[0060] 0 Figure 3A A server design and fluid flow shown in the complete fluid circuit of the design is shown in accordance with one embodiment

[0061] The server includes one or more cooling modules attached to their respective electronic devices to extract heat generated by the electronic devices. The cooling modules include fluid connectors for connection with a supply manifold and a return manifold disposed within an immersion vessel. The cooling modules can include blind mate connectors and / or manual mate connectors. The server includes a separate fluid flow pumped through the return manifold.

[0062] 5In one embodiment, with reference to Figure 3AThe cooling system 300 may include a server 310, and the server 310 includes an electrical...

[0063] The electronic board 312 (e.g., a printed circuit board or PCB) and liquid cooling modules 322A-332B (e.g., cooling plates) may be included. The electronic board 312 may include electronic devices such as one or more processors (e.g., a central processing unit, a graphics processing unit, etc.). In some examples, the electronic devices of the electronic board 312 may include high-power-density processors that generate a relatively large amount of heat.

[0064] The liquid cooling module 322A-B can be a cooling device that is placed in thermal contact with the electronic device to extract heat from the electronic device through a liquid coolant.

[0065] The cooling system 300 may also include at least one built-in return manifold 115 for connecting the liquid cooling modules 322A-332B of the server 310 (e.g., via blind-mating connectors) to the cooling loop. In some examples, the liquid cooling modules 322A-332B may be connected to the supply manifold 105 via manual connectors. In alternative embodiments, the liquid cooling modules...

[0066] 322A-332B includes a direct inlet opening for receiving liquid coolant directly from the immersion tank reservoir. Cooling system 300 may also include a complete cooling circuit having a liquid cooling unit 150 and a liquid pump 125 to allow coolant to pass through...

[0067] The cooling loop of liquid cooling modules 322A-332B is circulated. Cooling system 300 can be a liquid-to-liquid cooling unit, a liquid-to-air cooling unit, or any other single-phase cooling unit.

[0068] Figure 3B A server with a local fluid pump in each loop of the internal fluid channel, according to some embodiments, is shown.

[0069] Design: Local pumps in each loop provide different cooling capacities for different channels, cooling modules, and servers to provide cooling for applications with different power densities.

[0070] Similar to Figure 3A , Figure 3BThe cooling system 350 includes a server 410, electronic board 312, liquid cooling modules 322A-322B, and a return manifold 115. Further, the cooling system 350 includes a local liquid pump associated with each of the liquid cooling modules 322A-322B, such as pumps 326A-326B. Accordingly, depending on the power density of the electronic devices cooled by the liquid cooling modules 322A-322B, a local liquid pump can be used to provide targeted cooling capacity for each of the liquid cooling devices 322A-322B.

[0071] For example, the liquid cooling device 322A can be in contact with a very high power density processor, while the liquid cooling device 322B can be in contact with a lower power density electronic device. Accordingly, the local liquid pump 326A can be controlled to provide more coolant through the liquid cooling device 322A than the liquid cooling module 322B. Thus, the flow rate of coolant for local cooling can be controlled separately depending on the specific operating conditions, such as the local temperature of the individual cooling module or electronic device. The liquid pump 125 can also be coupled to the return manifold 115 to accelerate the coolant from the liquid cooling modules 322A-322B through the return manifold 115 and back to the liquid cooling unit or cooling source 150. For example, based on the temperature of the coolant, cooling plate, and / or electronic device, each of the individual pumps 326A-326B can be adjusted to provide more localized cooling fluid acceleration. Figure 3B The supply manifold is shown connected with a supply line 134 coupled to an external cooling source / cooling system 150. In embodiments, the supply manifold 105 can be connected to coolant contained in an immersion tank, such as Figure 1 The immersion coolant 104 of the immersion tank 102 is shown with or without a pump 108.

[0072] Figure 4 A server and internal server cooling module design is depicted in accordance with some embodiments. As depicted, the server cooling module can include a liquid outlet with a blind mate connector disposed at the bottom and used to connect the server and internal fluid passages of the server with a return manifold. Further, the cooling module includes a liquid inlet that is a directly open port to receive coolant, for example, from a supply manifold.

[0073] Reference is made to Figure 4 The server chassis 400 can represent any of the servers described above, such as, for example Figure 1of servers 110A-110C. In one embodiment, the server chassis 400 can include a server frame 405 having an electronic board 412 (e.g., PCB, motherboard, etc.) mounted thereon. The electronic board 412 can include one or more electronic devices 414 (e.g., processors) mounted thereon that generate heat during operation. The server chassis 400 further includes a cooling module 422 (e.g., cold plate or cold plate) in thermal contact with the one or more electronic devices 414.

[0074] The cooling module 422 can include or be coupled with a liquid inlet 410 (also referred to as a board inlet) through which coolant can flow to the cooling module 422. The inlet 410 of the cooling module 422 can be connected to an external circuit (e.g., a supply manifold) by a flexible hose. One end of the flexible hose can be connected to the inlet 410 and the other end of the flexible hose can be connected with a manual mating connector that can engage and connect with a manifold supply connector of a supply manifold as described above.

[0075] Further, the cooling module 422 also includes a board outlet, in this example, in the form of a blind mating connector 420. The blind mating connector 420 can automatically engage and connect with a corresponding blind mating connector of a return manifold as described above when the server chassis 400 is moved downward against the return manifold. It is noted that the server frame 405 is configured as an open frame for the purpose of providing a support structure to mount the electronic board 412, the electronic devices 414, and the cooling module 422. The server frame 405

[0076] does not block or interfere with the immersion coolant (such as the immersion coolant 104) from contacting or circulating around the outer surfaces of the mounted electronic board 412, the electronic devices 414, and the cooling module 422. The blind mating connector 420, as well as the cooling module 422, can need to be secured or assembled to the server chassis (such as the server frame 405) to achieve a robust structural design for the blind mating.

[0077] If the blind mating connector on the cooling module is mounted to the electronic devices 414 only, the electronic devices 414 can be damaged during the blind mating engagement.

[0078] Figure 5A A cooling system design is shown in which only the return manifold is used to form the cooling circuit and each server is immersed in a single-phase coolant. Thus, once connected to the return manifold, each server can draw in fluid directly from the immersion tank. For example, the return manifold can include or be coupled with a fluid pump to draw in fluid through the connected

[0079] servers. Thus, once connected to the return manifold, each server can draw in fluid directly from the immersion tank. For example, the return manifold can include or be coupled with a fluid pump to draw in fluid through the connected

[0080] The return manifold acts as a fluid acceleration device to pull fluid from the immersion tank into the direct inlets of the servers and through the internal passageways of the servers.

[0081] In one embodiment, with reference to Figure 5A , the cooling system 500 can include an immersion

[0082] tank 102 filled with an immersion coolant 104. The immersion tank 104 can have servers 510A-510B (collectively, servers 510) disposed therein that are coupled with the return manifold 115 through one or more blind mate connectors as described above. Each of the servers 510 includes a direct inlet 510 through which a portion of the coolant 104 can be drawn to a cooling module for cooling electronic devices of the server 510 when the server 510 is fully immersed in the coolant 104. For example, a liquid pump 125 can pull a portion of the immersion coolant 104 into the direct inlet 512 of each of the servers 510. Thereafter, the coolant can be drawn through the cooling module of the server 510 to extract heat from the electronic devices of the server 510 to the coolant. The coolant then travels to the return manifold 115 and back to an external source or cooling unit. As depicted, the servers 510 can be fully immersed within the cooling liquid 104 (e.g., the coolant level exceeds the top of the servers 510) such that the coolant can be pulled into the direct inlet 510 at the top of the servers 510.

[0083]

[0084] In this configuration, a single coolant supply is used to provide coolant to the IT enclosure (e.g., immersion cooling tank).

[0085] The coolant provides an immersion flow within the immersion tank that has a main outlet to a central return loop. The system includes a return manifold on the IT enclosure 5 and server-level advanced cooling modules to form separate cooling loop fluid flows with different flow rates as shown. The return manifold can be operated by a fluid pump. Both the central return loop from the immersion tank and the return manifold loop return coolant to an external cooling unit that can cool and recirculate the coolant back to the immersion tank. The local supply design of the servers allows for direct fluid acceleration ports independent of the IT enclosure supply. By having a separate pump built into each cooling module, fluid flow can be accelerated to different flow rates depending on the different cooling needs of the servers.

[0086]

[0087] Figure 5B The coolant provides an immersion flow within the immersion tank that has a main outlet to a central return loop. The system includes a return manifold on the IT enclosure 5 and server-level advanced cooling modules to form separate cooling loop fluid flows with different flow rates as shown. The return manifold can be operated by a fluid pump. Both the central return loop from the immersion tank and the return manifold loop return coolant to an external cooling unit that can cool and recirculate the coolant back to the immersion tank. The local supply design of the servers allows for direct fluid acceleration ports independent of the IT enclosure supply. By having a separate pump built into each cooling module, fluid flow can be accelerated to different flow rates depending on the different cooling needs of the servers.

[0088] 0​​Figure 6 A server and internal server cooling module design is depicted in accordance with some embodiments. This server chassis design can represent Figure 5A-5B any server 510. As depicted, the server cooling module can include a liquid outlet with a blind mate connector for connecting the server and the internal fluid passageway of the server with the return manifold. Further, the cooling module includes a liquid inlet that is a direct open port to the immersion coolant of the immersion tank that has no connectors on the side. The liquid inlet can be fully immersed in the immersion fluid of the immersion tank to draw liquid into the port, through the cooling module, and to the return manifold. Figure 4 and Figure 6 This design can require a server chassis and cooling module to be co-designed for immersion coolant management and speed optimization.

[0089] In one embodiment, reference is made to Figure 6 and similar to Figure 4 The inlet 430 can be an opening between the internal fluid passageway and the coolant in which the server 600 is immersed. Thus, the coolant in which the server 600 is immersed can travel through the inlet 430 to the cooling module 422, extract heat from the electronic devices 414, and then exit the fluid passageway to the return manifold 115 to carry the extracted heat away.

[0090] In the foregoing specification, embodiments of the application have been described with reference to specific exemplary embodiments thereof. It will be evident that various modifications can be made thereto without departing from the broader spirit and scope of the application as set forth in the following claims. The specification and drawings are, accordingly, to be regarded in an illustrative sense rather than a restrictive sense.

Claims

1. An immersion cooling system comprising: a vessel for containing a first coolant received from a first cooling source therein; one or more server enclosures at least partially immersed in the first coolant, wherein each of the server enclosures comprises: an electronic device, and a cooling plate attached to the electronic device to extract at least a portion of heat generated by the electronic device, wherein the cooling plate comprises an inlet for receiving a second coolant from a second cooling source, a coolant channel for distributing the second coolant, and an outlet for returning the second coolant to the second cooling source; and a return manifold coupled to the second cooling source, the return manifold having one or more manifold return connectors coupled to the server enclosures, respectively, and the return manifold for receiving the second coolant from the server enclosures and returning the second coolant to the second cooling source; wherein each of the manifold return connectors is configured to engage and connect with the outlet of the cooling plate of the corresponding server enclosure when the corresponding server enclosure is immersed in the first coolant and down onto the return manifold; the inlet is connected to a separate fluid circuit from the immersion tank through a supply manifold; the supply manifold is disposed above the server enclosures to receive the second coolant from the second cooling source, wherein the supply manifold comprises a plurality of manifold supply connectors each coupled to the inlet of the cooling plate of the server enclosures to supply the second coolant to the server enclosures, each of the manifold supply connectors and inlet of the cooling plate is provided with a local liquid pump therebetween.

2. The system of claim 1, wherein each outlet of each cooling plate comprises a plate return connector, and wherein each plate return connector and each manifold return connector is a blind mate connector.

3. The system of claim 1, wherein the vessel further comprises: a main inlet coupled to the first cooling source through a coolant supply line to receive the first coolant from the first cooling source; and a main outlet coupled to the first cooling source through a coolant return line to circulate the first coolant back to the first cooling source.

4. The system of claim 1, wherein the return manifold is disposed to be immersed in the first coolant and located near a bottom of the vessel and below the server enclosures.

5. The system of claim 1, further comprising a first fluid pump coupled between the supply manifold and the second cooling source to pump the second coolant into the supply manifold.

6. The system of claim 5, wherein the first fluid pump is configured to receive and pump the second coolant from the first coolant contained within the vessel.

7. The system of claim 4, further comprising a second fluid pump disposed between at least one cooling plate and the supply manifold to pump the second coolant from the supply manifold to a corresponding server chassis.

8. The system of claim 1, further comprising a plate supply connector coupled to an inlet of each cooling plate by a flexible hose, wherein the plate supply connector is connectable with one of the manifold supply connectors of the supply manifold.

9. The system of claim 1, wherein the first cooling source and the second cooling source are the same cooling source external to the vessel.

10. The system of claim 1, wherein an inlet of each cooling plate is submerged in the first coolant contained within the vessel to receive a portion of the first coolant as the second coolant.

11. A server chassis for immersion cooling, comprising: a server frame; a printed circuit board mounted on the server frame, the printed circuit board having one or more electronic devices mounted thereon; and one or more cooling plates attached to the one or more electronic devices to extract heat generated from the electronic devices, wherein the server frame, the printed circuit board with electronic devices, and the cooling plates are to be submerged in a first coolant received from a first cooling source and contained within an immersion tank, wherein each of the cooling plates comprises: an inlet to receive a second coolant from a second cooling source, a coolant channel to distribute the second coolant, and an outlet to return the second coolant to the second cooling source, wherein the outlet is to be coupled to a return manifold within the immersion tank, the return manifold coupled to the second cooling source, the return manifold having a manifold return connector coupled with the outlet for receiving the second coolant from the cooling plate and returning the second coolant to the second cooling source; wherein the manifold return connector is configured to engage and connect with the outlet of the cooling plate when the server chassis is submerged in the first coolant and down onto the return manifold; the inlet is connected to a separate fluid circuit from the immersion tank through a supply manifold; wherein the inlet of the cooling plate is coupled to a plate supply connector through a flexible hose, and wherein the plate supply connector is connectable with a supply manifold to receive the second coolant, a local liquid pump is disposed between each supply connector and inlet of a cooling plate.

12. The server chassis of claim 11, wherein the outlet comprises a plate return connector, and wherein the plate return connector and the manifold return connector are blind-mating connectors that are engageable and connectable with each other.

13. A cooling plate for immersion cooling, comprising: a base plate attached to an electronic device of a server chassis, wherein the server chassis with the electronic device is to be submerged in a first coolant, the first coolant received from a first cooling source and contained within an immersion vessel; an inlet for receiving a second coolant from a second cooling source; a coolant passage for distributing the second coolant; and an outlet for returning the second coolant to the second cooling source, wherein the outlet is coupled to a return manifold within the immersion vessel, the return manifold being coupled to the second cooling source, the return manifold having a manifold return connector coupled to the outlet for receiving the second coolant from the cooling panel and returning the second coolant to the second cooling source; wherein the manifold return connector is configured to engage and connect with the outlet when the server enclosure is immersed in the first coolant and down onto the return manifold; the inlet is connected by a supply manifold to a separate fluid circuit from the immersion tank; wherein the inlet is coupled by a flexible hose to a panel supply connector, and wherein the panel supply connector is connectable with a supply manifold to receive the second coolant, a local liquid pump being provided between each supply connector and the inlet of a cooling panel.

14. The cooling panel of claim 13, wherein the outlet comprises a panel return connector, and wherein the panel return connector and the manifold return connector are blind mate connectors that are engageable and connectable with each other.

Images