Liquid immersion cooling of electronic systems and devices

Abstract

Liquid immersion cooling devices and systems are described that use a cooling fluid, such as a dielectric coolant, to immerse and cool individual electronic devices or arrays of electronic devices. In one embodiment, the electronic device includes an atmospheric device housing defining an interior space, wherein the pressure in the interior space is equal to or only slightly different than the pressure outside the atmospheric device housing.

Description

Technical Field

[0001] The present disclosure relates to liquid immersion cooling of electronic array systems and devices. Background Art

[0002] Liquid immersion cooling of electronic systems and devices is well known. One example of a liquid immersion cooling electronic device array is a liquid submerged server (LSS) array arranged in a rack system. Examples of LSS arrays in rack systems are disclosed in U.S. Patents 7,905,106, 7,911,793, and 8,089,764. Another example of a liquid immersion cooling electronic device array is disclosed in U.S. Patent 9,451,726. Summary of the invention

[0003] Liquid immersion cooling devices and systems are described that use a cooling fluid, such as a dielectric coolant, to immerse and cool individual electronic devices or arrays of electronic devices. In one embodiment, the electronic device includes an atmospheric (or "zero" pressure) device housing defining an interior space, wherein the pressure in the interior space is equal to or only slightly greater than the pressure outside the atmospheric housing.

[0004] In one embodiment, a liquid immersion cooling electronic device may include a normal pressure device housing defining an internal space, wherein the pressure in the internal space is equal to the pressure outside the normal pressure device housing. One or more heat generating electronic components, such as multiple heat generating electronic components, are arranged in the internal space of the device housing, and a dielectric coolant is located in the internal space, wherein the dielectric coolant partially or completely immerses the one or more heat generating electronic components and is in direct contact with the one or more heat generating electronic components. The pump has a pump inlet and a pump outlet in fluid communication with the internal space. The heat exchanger, such as a liquid-liquid heat exchanger, has a heat exchanger inlet and a heat exchanger outlet, and the heat exchanger inlet is in fluid communication with the pump outlet. The liquid distribution manifold is located in the internal space, wherein the liquid distribution manifold has a manifold inlet and multiple manifold outlets, and the manifold inlet is in fluid communication with the heat exchanger outlet. The tube has an inlet end connected to one of the liquid manifold outlets, and its outlet end is close to one of the heat generating electronic components to direct the return flow of the dielectric coolant directly to one of the heat generating electronic components. The liquid immersion cooling electronic system may include multiple liquid immersion cooling electronic devices.

[0005] One example application of the liquid immersion cooling apparatus and system described herein is for use in an LSS array arranged in a rack system. However, the concepts described herein may be used in other applications where arrays of electronic devices are cooled by liquid immersion, including but not limited to blade servers, disk arrays / storage systems, solid-state storage devices, storage area networks, network attached storage, storage communication systems, routers, telecommunication infrastructure / switches, wired, optical and wireless communication equipment, cell processor devices, printers, power supplies, etc.

[0006] The liquid immersion cooling devices and systems described herein can be used in any field that can benefit from the advantages of liquid immersion cooling. In one example, the liquid immersion cooling devices and systems can be used in blockchain computing (cryptocurrency) applications, such as in ASIC or GPU computer mining configurations. Liquid immersion cooling devices and systems can also be used in deep learning applications, such as in multi-GPU configurations that support direct memory access (DMA) of maximum bandwidth and high-performance GPUs. Liquid immersion cooling devices and systems can also be used in artificial intelligence and high-performance computing (HPC) clusters with multiple coprocessor configurations (e.g., multi-GPU configurations that support DMA functions of GPU coprocessors). Many other applications and uses of the liquid immersion cooling devices and systems described herein are possible and can be expected.

[0007] The liquid immersion cooling devices and systems described herein do not require a fully sealed electronic device housing, which helps reduce costs and simplifies access to repair and modification of the electronic devices. Liquid immersion cooling also has excellent cooling efficiency compared to air cooling, thereby reducing power requirements and associated operating costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a perspective view of yet another example of a liquid immersion cooled electronic device as described herein.

[0009] Figure 2 yes Figure 1 A partial perspective view of a liquid immersion cooled electronic device is shown with the cover removed.

[0010] Figure 3 yes Figure 1 A perspective view of a liquid immersion cooled electronic device is shown with the housing removed.

[0011] Figure 4 is a perspective view of another example of a liquid immersion cooling electronic device described herein, which is similar to Figure 1-3 but wherein the heat exchanger is in an alternative position.

[0012] Figure 5 is a perspective view of another example of a liquid immersion cooling electronic device described herein, which is similar to Figure 1-3 An apparatus as in, but wherein the heat exchanger is in another alternative position.

[0013] Figure 6 is a perspective view of another example of a liquid immersion cooling electronic device described herein, which is similar to Figure 1-3 The device in, but with a cooling distribution unit.

[0014] Figure 7 is a rear view of a vertical array of liquid immersion cooled electronic devices as described herein arranged in a rack. DETAILED DESCRIPTION

[0015] The cooling liquid used to cool the electronic components in the electronic devices described herein can be, but is not limited to, a dielectric liquid. The cooling liquid is preferably a single-phase dielectric cooling liquid. Preferably, the single-phase dielectric cooling liquid has a sufficiently high heat transfer capability and heat capacity to handle the heat generated by the immersed heat-generating electronic components, so that the cooling liquid does not change from a liquid state to a gaseous state during the heat absorption process. Immersion cooling of heat-generating electronic components means that there is sufficient cooling liquid so that one or more heat-generating electronic components are partially or completely immersed in the dielectric cooling liquid and are in direct and intimate contact with the dielectric cooling liquid.

[0016] The one or more heat-generating electronic components to be immersed in the coolant can be any electronic component that generates heat and may wish to be cooled by partially or completely immersing the electronic component in the coolant. For example, the electronic component may include one or more processors (e.g., CPUs and / or GPUs), one or more power supplies, one or more switches, one or more data storage drives, one or more memory modules, and other electronic components. The electronic systems formed by the electronic components include, but are not limited to, blade servers, disk arrays / storage systems, solid-state storage devices, storage area networks, network attached storage, storage communication systems, routers, telecommunications infrastructure / switches, wired, optical and wireless communication equipment, cell processor devices, printers, power supplies, etc.

[0017] Figure 1-3 An example of a liquid immersion cooling electronic device 600 is shown. The device 600 includes a device housing 602 formed by a bottom liquid-impermeable tray 604 defining an interior space 605 and a cover 606 removably mounted on the tray 604 to prevent contaminants from falling into the cooling liquid contained in the interior space 605 of the tray 604. The tray 604 has side walls and a bottom wall defining the interior space 605 and a top that is at least partially open. In the example shown, the entire top of the tray 604 is shown as open. However, in other embodiments, only a portion of the top of the tray 604 may be open. The cover 606 is removably disposed on the open portion of the top of the tray 604.

[0018] The housing 602 can be considered to be atmospheric (or "zero" pressure) or minimally pressurized so that the pressure (or vacuum) in the interior space 605 is equal to or only slightly greater than / less than the pressure outside the device housing. For example, the pressure in the interior space 605 can be equal to ambient pressure. In another embodiment, the pressure in the interior space can be a small pressure, whose value can be small and difficult to measure, such as at most about 0.1 pounds per square inch. Therefore, as used herein, an atmospheric device housing is intended to include an interior space 605 having zero pressure (i.e., the pressure in the interior space is equal to ambient pressure) and containing a small pressure / vacuum, for example, a pressure of at most about 0.1 pounds per square inch greater than ambient pressure. This is in contrast to some device enclosures for liquid immersion cooled electronic devices which may be referred to as pressurized or sealed enclosures which typically operate at a positive measurable pressure level greater than ambient air pressure due to being connected to other similar liquid immersion cooled electronic devices containing the same fluid circuit and being subjected to pressure generated by a centralized or remote pump which creates fluid circulation by generating a positive pressure on the outlet of the pump and a corresponding negative or low pressure on the inlet of the pump.

[0019] Minimizing the pressure between the interior space 605 and the surrounding environment can be achieved in any suitable manner. For example, in one embodiment, Figure 1 As shown, a pressure relief / balancing mechanism 608 such as a vent or a check valve or other pressure relief / balancing mechanism may be provided in the cover 606 to provide air communication between the interior space and the environment. In another embodiment, pressure minimization may be achieved simply because the cover 606 does not fit tightly or seal with the tray 604. Since it is not intended to pressurize the housing 602, it is not necessary to seal and pressurize the device 600. However, the tray 604 does need to be sealed or leakproof to prevent the coolant disposed in the interior space 605 from leaking therefrom. Since the dielectric coolant is recirculated within the tray 604 and the heat exchanger (described below), and the device 600 is not required to operate at a pressure different from the pressure of the surrounding environment, pressure minimization in the device 600 is allowed.

[0020] like Figure 2 and 3As best shown, various heat generating electronic components 610 are arranged within the interior space 605. The various electronic components 610 may vary based on the type of electronic system to be formed by the apparatus 600. Examples of electronic components 610 that may be used include, but are not limited to, one or more processors (e.g., CPUs and / or GPUs), one or more power supplies, one or more switches, one or more data storage drives, one or more memory modules, and other electronic components. The electronic systems formed by the electronic components may include, but are not limited to, blade servers, disk arrays / storage systems, solid-state storage devices, storage area networks, network attached storage, storage communication systems, routers, telecommunication infrastructure / switches, wired, optical, and wireless communication devices, cell processor devices, printers, power supplies, and the like.

[0021] The dielectric coolant is disposed in the interior space 605, wherein the dielectric coolant partially or completely immerses and is in direct contact with at least some of the heat generating electronic components 610. The level of the dielectric liquid in the tray 604 will be sufficient to partially or completely immerse the electronic components for which immersion cooling is desired.

[0022] A coolant distribution circuit is provided for distributing the coolant within the device 600. Figure 2 and 3 In the illustrated embodiment, the distribution circuit includes one or more pumps 612 within the interior space 605 and having a pump inlet and a pump outlet, the pump inlet being in fluid communication with a large amount of coolant contained in the interior space 605. The illustrated example shows two of the pumps 612, one pump 612 serving as a primary pump and the other pump 612 serving as a backup pump in the event of a failure of the primary pump. A control valve 614 may be provided, which is fluidly connected to the outlet of each pump 612 and may be controlled by a suitable pump controller based on the monitored performance of the pump 612 to select which pump outlet will be used. Each pump 612 may be partially or completely immersed in the coolant, or the pump 612 may not be immersed in the coolant but have an inlet in the coolant.

[0023] A heat exchanger 616 is disposed within the interior space 605 and has an inlet and an outlet, the inlet being in fluid communication with the pump outlet via the control valve 614. The heat exchanger 616 can have any configuration suitable for reducing the temperature of the returning coolant. In the example shown, the heat exchanger 616 is configured as a liquid-liquid heat exchanger connected to an external cooling fluid loop 618 that provides secondary coolant to the heat exchanger 616. However, the heat exchanger 616 can be a liquid-air heat exchanger or any other configuration that can reduce the temperature of the returning coolant.

[0024] The heat exchanger 616 may be mounted at any suitable location within the housing 602. In the example shown, the heat exchanger 616 is shown mounted on an interior-facing surface of an end wall 620 of the tray 604. The heat exchanger 616 may or may not be partially or completely submerged in the coolant disposed within the interior space 605.

[0025] Reference Figure 2 and Figure 3 , a liquid distribution manifold 622 is disposed in the internal space 605 and has an inlet 624 in fluid communication with an outlet of the heat exchanger 616 via a supply line 626 and a plurality of manifold outlets 628 leading from the manifold 622. The liquid distribution manifold 622 distributes the cooling liquid to target electronic components among the electronic components 610, and then the cooling liquid enters the bulk cooling liquid in the internal space 605.

[0026] The return coolant can be directed from the manifold 622 directly to some electronic components 610, such as the CPU, GPU, power supply, switch, etc. For example, Figure 2 and Figure 3 As shown, one or more open-top trays 630 are disposed within the interior space 605, and a number of electronic components 610 are disposed in the trays 630. A supply tube 632 extends from the outlet 628 to each tray 630 to direct the returning cooling liquid into the trays 630. The trays 630 retain the cooling liquid around the electronic components 610 located within the trays 630. One or more liquid outlets or weirs 634 are formed in the sidewalls of each tray 630, and the dielectric cooling liquid exits the space defined by the trays 630 from the one or more liquid outlets or weirs 634. In use, each tray 630 is designed to be filled with cooling liquid to a level sufficient to immerse the liquid in the cooling tray 630 electronic components 610. The cooling liquid then overflows from the weirs 634 and flows by gravity into the bulk of the cooling liquid in the remainder of the interior space 605, where it can then be pumped by the pump 612 to the heat exchanger 616 for cooling. Some electronic components within the interior space 605 but not within one of the trays 630 may also be partially or fully immersed in the volume of cooling liquid contained within the interior space 605 .

[0027] The fluid distribution manifold 622 can be configured to assist in proportioning the flow of coolant to each outlet 628 to optimally manage the flow from each outlet 628 to the tray 630. For example, the outlets 628 can be sized differently, the supply tubes 632 can be sized differently, adjustable valves can be provided in the outlets 628 or tubes 632, etc. Flow management is useful in order to direct the proper amount and / or correct pressure of the returning coolant.

[0028] The weir 634 is disposed at the maximum dielectric coolant level of the tray 630, wherein the weir 634 establishes the level of the dielectric coolant within the tray 630 and establishes the volume flow rate of the dielectric coolant within the tray 630 required to cool the heat generating electronic components within the tray 630. As used throughout the specification and claims, a weir is an outlet for the coolant from which the coolant leaves by gravity without the use of a pump pressure connected to the weir. A weir is distinct from an outlet that is intended to be connected to a pump during use so that the pump pressure causes the coolant to flow out through the outlet, such as the outlets 52b, 112 described in U.S. Pat. No. 7,905,106.

[0029] In this embodiment, the pump 612 and heat exchanger 616 are disposed at a first end of the device housing 602, and the liquid distribution manifold 622 is disposed at a second end of the device housing 602 opposite the first end. However, other arrangements are possible.

[0030] Figure 4 Another example of a liquid immersion cooling electronic device 640 is shown, which is similar to the device 600 and like elements are indicated using like reference numerals. In this embodiment, a heat exchanger 616 is disposed outside the interior space 605 of the device housing so that the cooling liquid exits the device housing and is cooled. For example, the heat exchanger 616 may be mounted on the outer surface of the end wall 620 of the tray 604. Although not shown, the device 640 may include a heat exchanger similar to the heat exchanger 616 of FIG. Figure 1 The cover of cover 606 in the.

[0031] Figure 5 Another example of a liquid immersion cooled electronic device 650 is shown, which is similar to the device 600 and like elements are indicated using like reference numerals. In this embodiment, a heat exchanger 616 is disposed outside the interior space 605 of the device housing so that the cooling liquid exits the device housing and is cooled. In this embodiment, the heat exchanger 616 is not mounted on the end wall 620 of the tray 604, but is mounted at an external location separate from the device 650 so that the heat exchanger 616 is not mounted on the device housing. Although not shown, the device 650 will include a heat exchanger similar to Figure 1 The cover of cover 606 in the.

[0032] Figure 6Another example of a liquid immersion cooled electronic device 660 is shown which is similar to the device 600 and like elements are indicated using like reference numerals. In this embodiment, both the heat exchanger and the pump are disposed outside of the interior space 605 of the device housing. Additionally, one or more pumps and heat exchangers are combined into a common unit referred to as a cooling distribution unit 662. The cooling distribution unit 662 is shown not to be mounted on the end wall 620 of the tray 604, but rather at an external location separate from the device 660. However, the cooling distribution unit 662 could be mounted on an outer surface of the end wall 620. Although not shown, the device 660 would include a similar Figure 1 The cover of cover 606 in the.

[0033] Reference Figure 7 , Figure 1-3 The plurality of devices 600 shown are shown arranged together in a vertical array 670, for example, arranged on a rack 672. Alternatively, the devices 600 can be used individually and separately from each other. In an exemplary embodiment, the plurality of electronic devices 600 can be arranged on a rack 672 into a plurality of vertically spaced rows. Figure 7 A vertical manifold 674 is shown mounted on the rack 672, which is part of the external cooling fluid loop 618 and is used to bring secondary cooling liquid into the heat exchanger 616. A similar vertical array may be implemented on the rack 672 for the electronic devices 640, 650, 660.

[0034] The pumps described herein can be adaptively controlled by a pump controller based on the desired performance of one or more electronic devices. For example, the pumps can be controlled to operate in series or in parallel. In addition, the pumps can be operated in a redundant mode in which a second pump acts as a backup pump for the first pump if the first pump fails.

[0035] The electronic device and vertical array design described herein is extremely versatile for a variety of system configurations and is easily serviceable since each electronic device has its own independent fluid cooling circuit. Further, the device housing 602 does not need to be designed with a reinforcing structure between the bottom of the tray 604 and the cover 606 within the interior space 605 to compensate for the deflection forces that would occur if the device 600 was sealed and operated at a net positive gauge pressure, which allows the entire device housing 602 to be designed as an open space within which many electronic component configurations can be installed. In addition, the electronic device and vertical array are also more efficient in terms of the volume of coolant required to achieve immersion cooling compared to sealed / pressurized filled systems or reservoirs, which is beneficial from both a weight and cost perspective. The concepts described allow for high flow immersion cooling, directional flow immersion cooling of high power density components, gravity return immersion cooling (e.g., use of storage trays for holding electronic devices as described in application Ser. No. 16 / 137,015, filed Sept. 20, 2018, and entitled “LIQUID SUBMERSION COOLED ELECTRONIC SYSTEMS AND DEVICES”), and gravity assisted cooling of electronic boards that may protrude above the level of a fluid pool within a device housing, each of which can be accomplished simultaneously within the same device housing.

[0036] In all respects, the examples disclosed in this application should be considered as illustrative rather than restrictive. The scope of the invention is indicated by the appended claims rather than the preceding description; and all changes within the meaning and range of equivalency of the claims are intended to be embraced therein.

Claims

1. A liquid immersion cooling electronic device, comprising: a liquid-tight tray having a horizontal bottom wall and vertical side walls defining an interior space, an at least partially open top opposite the horizontal bottom wall, and a horizontal cover removably attached to the liquid-tight tray and disposed over the at least partially open top, the horizontal cover not being sealed from the liquid-tight tray; a vent or pressure relief / equalization mechanism within the horizontal cover and providing air flow between the interior space and the surrounding environment, wherein the pressure within the interior space is equal to the ambient pressure; at least one heat-generating electronic component, the at least one heat-generating electronic component being disposed within the interior space of the liquid-tight tray; a single-phase dielectric coolant in the interior space, the single-phase dielectric coolant partially or completely immersing the at least one heat-generating electronic component and in direct contact with the at least one heat-generating electronic component; a pump having a pump outlet and a pump inlet in fluid communication with the interior space; as well as a heat exchanger disposed within the interior space or mounted outside the interior space at a location separate from the liquid-tight tray, the heat exchanger having a heat exchanger inlet in fluid communication with the pump outlet and having a heat exchanger outlet in fluid communication with the interior space; and The heat exchanger is configured to be in fluid communication with a secondary cooling fluid circuit that supplies secondary cooling fluid to the heat exchanger.

2. The liquid immersion cooling electronic device according to claim 1, characterized in that: Also includes: a liquid distribution manifold within the interior space, the liquid distribution manifold having a plurality of manifold outlets and a manifold inlet in fluid communication with the heat exchanger outlet; A tube having an inlet end connected to one of the manifold outlets and an outlet end of the tube proximate to the at least one heat-generating electronic component to direct the return flow of the single-phase dielectric coolant to the at least one heat-generating electronic component.

3. The liquid immersion cooling electronic device according to claim 1, characterized in that: The pump is disposed within the interior space, and the pump inlet is immersed in the single-phase dielectric coolant.

4. The liquid immersion cooling electronic device according to claim 2, characterized in that: The pump and the heat exchanger are disposed at a first end of a liquid-tight tray, and the liquid distribution manifold is disposed at a second end of the liquid-tight tray opposite the first end.

5. The liquid immersion cooling electronic device according to claim 2, characterized in that: Also included is an open-top tray disposed within the interior space, the at least one heat-generating electronic component being disposed within the open-top tray, the outlet end of the tube being connected to the open-top tray to direct a return flow of the single-phase dielectric coolant into the space defined by the open-top tray, and a side wall of the open-top tray comprising a single-phase dielectric coolant outlet weir, the dielectric coolant exiting the space defined by the open-top tray from the outlet weir.

6. The liquid immersion cooling electronic device according to claim 1, characterized in that: A plurality of the heat-generating electronic components are included within the interior space, and wherein the heat-generating electronic components include a plurality of data storage devices, a plurality of power supplies, a plurality of processors, or a plurality of switches.

7. A liquid immersion cooling electronic system, comprising: A plurality of liquid immersion cooling electronic devices according to claim 1 are arranged in a vertical array on a rack.

Images