Maintainable liquid immersion cooling

Abstract

Method and apparatus for the thermal management of IT hardware mounting heat generating electronic components. The present thermal management system comprises a plurality of immersion cooling tank assemblies interconnected via a common manifold. Coolant distribution units (CDUs) are connected fluidically to the immersion tanks via common manifold. The apparatus configuration and dimensioning enables switching between a normal state (where all CDUs are active) to a servicing configuration (where at least one CDU is off-line), whilst maintaining the nominal cooling capacity and avoiding temperature fluctuation / variations within the immersion tanks.

Description

[0001] Maintainable Liquid Immersion Cooling

[0002] Field of invention

[0003] The present concept relates to liquid cooling apparatus and in particular, although not exclusively, to liquid cooling apparatus for an electronic device having a heat-generating electronic component such as an IT component immersed within a dielectric coolant liquid.

[0004] Background

[0005] The cooling of electronics, specifically IT components, servers, data storage devices and computational electronic devices having graphics and central processing units (GPUs and CPUs) has become a major technical challenge due to the ongoing development of smaller, faster, higher density and higher power capacity electronics.

[0006] Computing devices produce heat as a by-product of operational processing. In datacentres, where thousands of such devices are located, the amount of heat generated can be extremely large. As the need for access to greater processing and data storage continues to expand, the density of server systems continues to increase, and the resulting thermal challenges present a significant practical obstacle.

[0007] Conventional fan-based cooling systems require large amounts of power. Accordingly, the power demand to drive such systems increases significantly with the increased server densities. Immersion cooling of IT components is a relatively recent development. The operational hot electronics are submerged in direct contact with a dielectric (electrically insulating) coolant liquid that is circulated and cooled through the use of heat exchangers and the likes. Cooling of electronics enhances their performance efficiency enabling higher processing speeds (for example the overclocking of CPUs) whilst reducing power consumption by eliminating fans and enabling more efficient heat rejection systems. The heat generated by the circuit is removed quickly and efficiently by the dielectric liquid directly at the heat source.

[0008] Existing immersion cooling systems typically utilise a coolant distribution unit (CDU) that is effective to circulate the dielectric coolant liquid though a tank having an immersion chamber containing the electronic devices to be cooled. Typically, each CDU includes a heat exchanger and a pump fluidically connected as part of a dielectric liquid closed-circuit network. These components are effective to cool the dielectric liquid and to return a cooled supply to the tanks for effective and efficient cooling of the electronic components. However, maintenance and repair of fluid flow components such as heat exchangers, pumps and fluid flow valves etc, typically require system downtime and often a need to completely drain the dielectric liquid from large parts of the system. Accordingly, there is a need for an immersion cooling system enabling more convenient servicing and repair and / or minimised risk of system downtime due to component part wear and / or failure.

[0009] Summary of the Invention

[0010] It is an objective of the present concept to provide a one phase liquid immersion cooling system with coolant distribution unit redundancy. It is a further specific objective to provide a liquid immersion cooling system offering tank independent control and / or passive tank level control. It is a further objective to provide an immersion cooling arrangement according to a modular construction enabling component interchangeability, servicing or repair of components and / or assemblies.

[0011] Accordingly, the inventors provide a modular constructed immersion cooling system configured with suitable control utilities / sy stems having at least two coolant distribution units connected to a common manifold such that in the first mode of operation all CDUs may be operational to circulate dielectric coolant liquid and according to a second mode at least one CDUs may be non-operational whilst at least one CDU is maintained operational to service dielectric coolant liquid cooling and circulation at all immersion tanks. Specifically, the inventors provide a modular immersion cooling system having dielectric liquid cooling and circulation redundancy such that one or a plurality of CDUs and / or other components of the system may be hydraulically isolated and serviced, repaired or replaced without effecting cooling performance of the system. Such an arrangement and configuration provides redundancy to maintain high operational capacity of the heat generating components mounted at the IT hardware that is being thermally managed / cooled.

[0012] The present system provides a high energy density and energy efficient cooling system with servicing / maintenance redundancy via a balancing of hydraulic liquid head pressure at a common manifold. This is created by at least a pair of immersion tanks that are equal or approximately equal with respect to any one or a combination of internal chamber volume size, geometric construction, the number and positioning of component parts, the elevation or positioning at the system relative to other tanks and / or relative to a mounting surface, mounting structure, mounting assembly, the floor or elevation relative to sea level.

[0013] According to a first aspect of the present concept there is provided liquid cooling apparatus for an electronic device comprising: at least two immersion tanks, each tank having a liquid inlet and a liquid outlet, and a chamber to accommodate at least one electronic device having at least one heat-generating electronic component, and a dielectric coolant liquid capable of flowing through the chamber via the inlet and outlet; at least two coolant distribution units having a liquid inlet, a liquid outlet and at least one cooling component in fluidic communication with each of the chambers via the respective inlets and outlets to enable a flow of the dielectric coolant liquid between each of the chambers and the coolant distribution units; a manifold fluidically connecting the inlets and outlets of the chambers and the coolant distribution units, the manifold having a first flow conduit extending between the outlets of the at least two of the tanks, and a second flow conduit extending between the inlets of the at least two tanks; each inlet of each coolant distribution unit fluidically connected to the first flow conduit and each outlet of each coolant distribution unit fluidically connected to the second flow conduit.

[0014] Optionally, the tanks may be identical or nearly identical in size and / or an internal volume of the respective chambers to contain respectively an equal or approximately equal volume of dielectric coolant liquid such that a liquid head pressure created by the tanks at a region within the first and / or second flow conduit is equal or approximately equal.

[0015] Optionally, the tanks may be located at the same or approximately the same elevation relative to one another such that a liquid head pressure created by the tanks at a region within the first and / or second flow conduit is equal or approximately equal.

[0016] Optionally, a respective static flow condition is created at the first and / or second flow conduit resultant from a balancing of the liquid head pressure created by the tanks. Optionally, the static flow condition is located between respective connection points by which the at least two coolant distribution units are fluidically connected to the first and / or second flow conduits.

[0017] Optionally, each of the tanks comprises a buffer reservoir fluidically connected to each respective chamber to receive an overflow of the dielectric coolant liquid, each buffer reservoir connected in fluidic communication between the chamber and the outlet of the respective tanks.

[0018] Optionally, each tank comprises at least one temperature sensor positioned relative to the chamber to monitor a temperature of the dielectric coolant liquid within the chamber. Optionally, the temperature sensor is located at or towards a liquid-air interface at the chamber to monitor a temperature of the dielectric coolant liquid at or towards the liquid- air interface of the coolant dielectric liquid within the chamber. Optionally the apparatus may comprise a pressure sensor, a flow sensor, a filter, a particulate filter.

[0019] Optionally, the apparatus further comprises at least one valve located between each inlet and each outlet of each coolant distribution unit and the first and second main flow conduit to allow control of a flow of the coolant dielectric liquid between each coolant distribution unit and the manifold to enable each coolant distribution unit to be fluidically isolated from the manifold. Optionally, the apparatus further comprises at least one valve located between each inlet and each outlet of each tank and a respective central section of the first and / or second fluid flow conduit to enable each of the tanks to be fluidically isolated from the central sections of the first and / or second flow conduits and / or the coolant distribution units.

[0020] Optionally, the coolant distribution unit comprises any one or a combination of: a chiller; a refrigeration unit; a heat exchanger; a dry-cooler, an adiabatic cooler, a heat pump.

[0021] Optionally, each coolant distribution unit comprises a heat exchanger, a first side of each heat exchanger fluidically connected to the respective inlet and the outlet of the coolant distribution unit and a second side of each heat exchanger fluidically connected to a working fluid network to allow heat transfer between the dielectric coolant liquid and the working fluid. Optionally, the apparatus comprises a fluid flow control valve located at the second side of the heat exchanger to regulate a flow rate of the working fluid to and / or from the heat exchanger. Optionally, the fluid flow control valve is any one or a combination of: an electromagnetic valve; an electronic valve; an electromechanical valve, a 2-way valve, a 3 -way valve, a pressure independent control valve (PICV). Optionally, each coolant distribution unit comprises any one or a combination of: a liquid flow pump; a filter or a particulate filter; a flow sensor; a temperature sensor; a pressure sensor; at least one valve.

[0022] Optionally, each coolant distribution unit comprises a split flow section having two parallel flow lines, each of the lines connected fluidically between the inlet and the coolant component and each parallel flow line having a pump and at least one of the flow lines having a filter and a respective valve positioned at each fluid flow side of the filter to enable the filter to be fluidically isolated from the coolant distribution unit.

[0023] Optionally, the at least two tanks may comprise a first and a second set of tanks, each of the tanks of the first set of tanks fluidically connected at or towards a first end of the first and / or second flow conduit and each of the tanks of the second sets of tanks fluidically connected at or towards a second end of the first and second flow conduit. Optionally, the first and second sets of tanks are separated by respective central sections of the first and second main flow conduits.

[0024] Optionally, each of the coolant distribution units may be connected to the first and second fluid flow conduits at either side of the respective central sections such that the central sections separate the fluid flow junctions between which the coolant distribution units are coupled fluidically to the first and second flow conduits.

[0025] Optionally, the apparatus may comprise a control unit configured for the automated, semiautomated and / or remote control of at least one operational parameter of the apparatus, wherein the operational parameter comprises any one or a combination of a flow of the dielectric coolant liquid within the apparatus; a driving of flow of the dielectric coolant liquid within the apparatus; a flow rate of the dielectric coolant liquid within the apparatus; a pressure of the dielectric coolant liquid within the apparatus; a temperature of the dielectric coolant liquid within the apparatus; a fluidic connection and / or isolation of any one of the tanks and / or the coolant distribution units from the manifold.

[0026] Optionally, the control unit is configured for the automated, semi-automated and / or remote control of at least one operational parameter comprising any one or a combination of a flow of the working fluid to and / or from the heat exchanger; a driving of a flow of the working fluid to and / or from the heat exchanger; a flow rate of the working fluid to and / or from the heat exchanger; a pressure of the working fluid within the apparatus; a temperature of the working fluid within the apparatus. Controlling the flow of the working fluid at the second side of the heat exchanger (not within the first side and within the dielectric network) is advantageous to minimis / eliminate the use of control valves and / or similar components within the dielectric network. This avoids downtime of the present system in the event of value / component failure or maintenance.

[0027] The control unit may comprise anyone or a combination of: a motherboard, a processor, a data storage utility, RAM, communication ports, communication protocols enabling wired or wireless communication to an external device or network, software installed on a hard drive (located locally or remotely relative to the present apparatus / immersion tanks), a user interface such as a keyboard and screen, a touchscreen or other input means. Optionally, the apparatus may comprise the same or a similar control unit having the same or similar components and function to that described and associated with the control of the flow of the dielectric coolant liquid being applied to the working fluid at the second side of the heat exchanger. Optionally, the apparatus may comprise at least two control units or may comprise a single control unit operable to control the fluid at each of the first and second sides of the heat exchanger.

[0028] According to a further aspect of the present concept there is provided a method of cooling an electronic device comprising: providing at least two immersion tanks, each tank having a liquid inlet and a liquid outlet, and a chamber to accommodate at least one electronic device having at least one heat-generating electronic component, and a dielectric coolant liquid capable of flowing through the chamber via the inlet and outlet; providing at least two coolant distribution units having a liquid inlet, a liquid outlet and at least one cooling component in fluidic communication with each of the chambers via the respective the inlets and outlets to enable a flow of the dielectric coolant liquid between each of the chambers and the coolant distribution units; providing a manifold fluidically connecting the inlets and outlets of the chambers and the coolant distribution units, the manifold having a first flow conduit extending between the outlets of the at least two of the tanks, and a second flow conduit extending between the inlets of the at least two tanks, wherein each inlet of each coolant distribution unit is fluidically connected to the first flow conduit and each outlet of each coolant distribution unit is fluidically connected to the second flow conduit; and circulating a dielectric coolant liquid through the chambers, the manifold and the coolant distribution units to cool at least one electronic device immersed within at least one of the chambers. Optionally, the method further comprises creating an equal or approximately equal liquid head pressure at the first flow conduit and the second flow conduit by providing the tanks with an identical or near identical size and / or internal volume of the respective chambers to contain respectively an equal or approximately equal volume of dielectric coolant liquid within the respective chambers. Optionally, the method comprises creating an equal or approximately equal liquid head pressure at a respective first and a second side of each of the first flow conduit and the second flow conduit by locating the tanks at the same or approximately the same elevation relative to one another.

[0029] Optionally, the method comprises creating a respective static flow condition at a central or approximately central region of each of the first flow conduit and the second flow conduit; and wherein the step of circulating the dielectric coolant liquid comprises circulating the dielectric coolant liquid at either side of each of the static flow conditions in that the dielectric coolant liquid does not flow within the region of each of static flow condition.

[0030] In particular, the present concept is configured to switch between two modes of operation i.e., a first mode in which all CDUs of the system are working / operational and a second mode in which at least one of the CDUs is off-line / not operational. Importantly, the present apparatus and method is configured via the components and function as described herein to achieve this switch without affecting the processing (i.e., cooling of IT hardware notes within the tank(s)) and without the provision of any ‘active’ components (such as valves etc) outside / external to each of the CDUs. Specifically, the configuration and dimensioning of the present apparatus enables switching between a normal state (where all CDUs are active) to a servicing configuration (where at least one CDU is off-line), whilst keeping the nominal, prescribed, set cooling capacity / temperature within the immersion tanks.

[0031] Brief description of drawings

[0032] A specific implementation of the present invention will now be described, by way of example only, and with reference to the accompanying drawings in which: Figure 1 A is a perspective view of a liquid immersion cooling tank part of the present apparatus for the controlled thermal regulation of IT hardware having heat generating electronic devices according to a specific implementation of the present concept;

[0033] Figure IB is a perspective view of a pair of interconnected liquid immersion cooling tanks and coolant distribution units (CDUs) for the controlled thermal regulation of IT hardware according to a specific implementation of the present concept;

[0034] Figure 2 is a schematic illustration of a fluid circuit comprising a plurality of liquid immersion cooling tanks and coolant distribution units (CDUs) interconnected via a common manifold comprising fluid flow conduits according to a specific implementation of the present concept;

[0035] Figure 3 is a schematic illustration of the coolant circuit of figure 2 configured for operation in a first mode in which all CDUs are operational to cool and circulate dielectric coolant liquid to immersion cooling tanks;

[0036] Figure 4 is a schematic illustration of the coolant circuit of figure 2 configured for operation in a second mode in which one CDU is non-operational whilst a remaining CDU is operational to provide cooling and circulation of the dielectric coolant liquid to immersion cooling tanks;

[0037] Figure 5 is a schematic illustration of a thermal management system for the control of operational conditions and parameters of the fluid circuit of figures 1 to 4;

[0038] Figure 6 is a schematic illustration of a fluid circuit according to a further embodiment comprising two sets of immersion cooling tanks interconnected via a common manifold and comprising a pair of CDUs configurable for a first mode and a second mode of operation. Detailed description of preferred embodiment of the invention

[0039] According to the present embodiments, liquid cooling apparatus and a system is provided for the immersion cooling of electronic devices including in particular servers and IT hardware nodes that mount an array of heat generating electronic components including for example microprocessors, GPUs, CPUs, RAM, motherboards etc. The present apparatus is focused to an immersion cooling system configured for the thermal management of heat generating components mounted at such IT hardware nodes.

[0040] Referring to figure 1 A, an immersion cooling tank (alternatively termed immersion bath) 10 comprises an internal chamber 14 to accommodate a plurality of IT electronic devices

[0041] 13, (alternatively termed IT hardware nodes) such as servers, electronic boards and the like. Each electronic device 13 mounts an array of heat generating components (HGCs) 15 in the form of on-board electronic components that may typically comprise relatively low heat generating components (eg RAM, the motherboard and the like) and relatively high heat generating components (eg microprocessors, CPUs, GPUs). Such components may themselves have different maximum operating temperatures. A dielectric cooling liquid 11 is housed within chamber 14 as defined by container 12, alternatively termed an immersion cooling tank.

[0042] Referring to figure IB, the liquid cooling apparatus comprises a pair (first and second) of immersion cooling tank assemblies 20a, 20b, each having a tank 10 with internal chamber

[0043] 14. A pair (first and second) of coolant distribution units (CDUs) 22a, 22b are connected in fluidic communication with each of the tank assemblies 20a, 20b (including tanks 10 and chambers 14) via a common manifold (figure 2) to provide a self-contained and / or mobile immersion cooling system.

[0044] Referring to figure 2, the present fluid network may be divided into at least five fundamental units / sub-assemblies including in particular a first and second immersion cooling tank assembly 20a, 20b; a manifold 21 and a first and second coolant distribution unit (CDU) 22a, 22b. Each of the immersion tank assemblies 20a, 20b are connected hydraulically in fluidic communication with each of the CDUs 22a, 22b via manifold 21. Each immersion tank assembly 20a, 20b comprises an immersion cooling tank 10 having chamber 14 to contain the dielectric coolant liquid and within which the IT hardware 13 is immersed. An overflow buffer reservoir 16 is connected in fluidic communication with chamber 14 to receive excess dielectric coolant liquid via an overflow weir. Each assembly 20a, 20b comprises a plurality of sensors in particular chamber 14 comprises a temperature sensor 17a located at or proximate to an air-liquid interface i.e., at the upper region of chamber 14. Each buffer reservoir 16 also comprises a corresponding level sensor 17b. A liquid flow inlet 31 is provided at a lower region of chamber 14 and a liquid flow outlet 32 is provided at buffer reservoir 16 to allow coolant in-flow at chamber 14 and coolant out-flow from reservoir 16.

[0045] Manifold 21 comprises a first flow conduit 23 and a second flow conduit 24. A flow valve 33, 34 is provided respectively at each lengthwise end of the first and second flow conduits 23, 24. Valves 33, 34, are connected respectively to each outlet 32 and inlet 31 of each immersion tank 10 via connection conduits 19, 18. Accordingly, an inlet flow of coolant liquid is configured to flow from second flow conduit 24 to each respective tank inlet 31 via connection conduits 18 and valves 34. Similarly, an outflow of coolant liquid is configured to flow into main flow conduit 23 from each outlet 32 via each respective connection conduit 19.

[0046] Each coolant distribution assembly 22a, 22b is connected in fluidic communication with manifold 21 and in particular the first and second flow conduits 23, 24. Specifically, an inlet 27 of CDU 22a is connected via connection conduit 25 to the first flow conduit 23 at connection point 50a and an outlet 28 of CDU 22a is connected via connection conduit 26 to second flow conduit 24 at connection point 51a. Similarly, an inlet 27 of CDU 22b is connected to first flow conduit 23 via connection point 50b and an outlet 28 of CDU 22b is connected to second flow conduit 24 via connection point 51b. Each CDU 22a, 22b comprises a heat exchanger 42 having a first side 42a and second side 42b. First side 42a is connected in fluidic communication to CDU inlet 27 and outlet 28 (via connection conduit 46). Second side 42b is connected in fluidic communication to a Facility Water System or suitable working fluid network. Accordingly, heat exchanger 42 is configured to transfer heat energy between the dielectric coolant liquid flowing though the first side 42a and the working fluid of the facility water system flowing within second side 42b. In particular, the working fluid network comprises inlet 29 connected to heat exchanger 42 by connection conduit 45. A electromechanical control valve 43 is connected to heat exchanger second side 42b via connection conduit 44 at a first side and is connected to working fluid circuit outlet 30 via connection conduit 49. The working fluid circuit / Facility Water System (FWS) 54 accordingly comprises heat exchanger second half 42b, inlet 29, outlet 30, at least control valve 43 and connection conduits 45, 44, 49.

[0047] Each CDU 22a, 22b comprising a plurality of sensors including in particular pressure sensors 40, temperature sensors 41 and further condition sensors 56, 55 which together with the other sensors of the fluid circuit form part of a control unit / system configured to control a plurality of operational conditions / parameters of the present system.

[0048] Each CDU 22a, 22b further comprises a split flow section 37 comprising a pair of parallel fluid flow lines extending between respective connection conduits 47 and 48 that are coupled respectively to heat exchanger first side 42a and CDU inlet 27. Each of the split flow lines comprises a pump 39. A particulate filter / trap 38 is connected in fluidic communication to one of the split flow lines (at respective lengthwise ends thereof) via isolation valves 52, 53. Accordingly, filter / trap 38 may be fluidically isolated from each CDU 22a, 22b via valves 52, 53.

[0049] Referring to figures 2 and 3, each of the immersion cooling tanks 10 and in particular tank assemblies 20a, 20b is geometrically and hydraulically matched. In particular, each chamber 14 comprises the same internal volume. Additionally, each chamber 14 is positioned at the same elevation or approximately the same elevation relative to a floor, mounting surface or region that supports the present immersion cooling circuit and system. Additionally, reference to the same elevation may be a relative sea level elevation. The geometric and hydraulic matching of the two tank assemblies 20a, 20b at manifold 21 provides an equal and opposed hydraulic head pressure at each respective first and second flow conduit 23, 24. With dielectric coolant within chambers 14 and the various conduits of the present system, a static flow / stagnation region 60, 61 is created at each respective first and second flow conduit 23, 24. In particular, static flow / stagnation region 60 is created between each connection point 50a, 50b by which each respective CDU inlet 27 is connected to conduit 23. Similarly, static flow / stagnation region 61 is created between the connection points 51a, 51b by which each of the outlets 28 of each CDU 22a, 22b is connected to second flow conduit 24. According to this configuration, each tank assembly 20a, 20b is positioned to each respective lengthwise end of manifold 21 and in particular each respective lengthwise end of each respective static flow / stagnation region 60, 61.

[0050] According to a normal mode of operation, each CDU 22a, 22b is operational and fluidically / hydraulically coupled to manifolds 21 via respective inlets 27 and outlets 28. With each respective control valve 43 held open, working fluid flows within heat exchanger second half 22b to transfer heat energy with the dielectric coolant flowing within heat exchanger first side 42a. As such, dielectric liquid flows from tank 14 of manifolds 21 via connection point 50a and into CDU 22a via inlet 27. Coolant liquid also flows from tank assembly 20b into CDU 22b via first flow conduit 23, connection point 50b and inlet 27. The dielectric liquid, cooled by each heat exchanger 42 is then returned to each respective chamber 14 of each tank assembly 20a, 20b via each CDU outlet 28, respective connection points 51a, 51b and second flow conduit 24.

[0051] Referring to figure 4, the present apparatus / system comprises a modular construction and specifically the assembly of components and units 20a, 20b, 21, 22a and 22b. This facilitates servicing, repair or modification of any one of the components and units 20a, 20b, 21, 22a and 22b. In particular, each of the valves 33, 34 provide fluidic isolation of each of the respective tank assemblies 20a, 20b. Similarly, each of the valves 35, 36 that provide fluidic communication of each CDU 22a, 22b to manifold 21 enable independent fluidic isolation of each CDU 22a, 22b relative to manifold 21 and tank assemblies 20a, 20b.

[0052] According to the configuration of figure 4, the present system is operational in a non- symmetric or redundancy mode in which only one CDU is operational (specifically CDU 22b) whilst one CDU is non-operational (specifically CDU 22a). Accordingly, CDU 22a is fluidically isolated via valves 35, 36 whilst the corresponding valves 35, 36 at CDU 22b are open and provide fluidic communication to each of the respective first and second flow conduits 23, 24.

[0053] In the normal operational mode according to figure 3, each control valve 43 is controlled to provide a reduced flow rate of the working fluid through heat exchanger second side 42b. In addition, pumps 39 are controlled to operate at reduced power so as to provide a balanced and approximately equal flow rate of dielectric coolant through each CDU 22a, 22b. Accordingly, each CDU 22a, 22b is configured to service equally, in a shared operational capacity, the cooling and distribution / circulation of the dielectric coolant liquid at each chamber 14 of each tank assembly 20a, 20b.

[0054] According to the second redundancy or servicing mode, control valve 43 is energised to provide an approximate double flow rate of working fluid through heat exchanger second side 42b (relative to the mode and configuration of figure 3). Similarly, pumps 39 of CDU 22b are powered to provide approximately double the flow rate of the dielectric coolant circulated through heat exchanger first side 42a. According to this mode, CDU 22b is operational to cool the dielectric liquid circulated through both tank assemblies 20a, 20b.

[0055] Referring to figure 5, the present system comprises a control unit preferably implemented via computer hardware and software. In particular, the present system may be provided as a distributed control system (DCS) and optionally implemented as a programmable logic controller (PLC) to control the various operational components of the present system such as valves 43, pumps 39 and optionally any one or a combination of valves 33, 34, 35, 36, 52, 53. The control system (software utility) may be implemented on auxiliary or other independent electronics (eg a user’s DCS, PLC or computer entity) for control of the various operational components of the present system. The present control system is preferably implemented as software incorporating or capable of utilising / interrogating a Smith Predictor Model (SPM) 70. The present control system (either directly or indirectly when hosted or accessed remotely) is coupled electronically to each of the sensors 17a, 17b, 40, 41, 55, 56 so as to provide responsive control of the valves 43, pumps 39 and optionally valves 33, 34, 35, 36, 52, 53 to thereby control the operational conditions and parameters of the present system. Such operational conditions and parameters include any one or a combination of: a flow of the dielectric coolant liquid within the apparatus; a driving of flow of the dielectric coolant liquid within the apparatus; a flow rate of the dielectric coolant liquid within the apparatus; a pressure of the dielectric coolant liquid within the apparatus; a temperature of the dielectric coolant liquid within the apparatus; a fluidic connection and / or isolation of any one of the tanks 10 and / or the CDUs 22a, 22b from the manifold 21.

[0056] The control system 70 receives the surface temperature via sensors 17a, at stage 72. The system uses proportional integral derivative (PID) controllers / functionality to provide a control loop feedback process for the control of the various operational variables of the system such as the valves 43 and pumps 39. The PID tank surface temperature is processed by an initial PID tank surface utility 73. The output of the PID tank surface utility 73 is input to a PID TCS cold utility 74 that forms part of a Temp cold control subprocess entity 71 (that further comprises a disturbance FWS / TCS hot utility 75 and a HX / valve process utility 76). The output 84 of the PID TCS cold utility is fed to a HX / valve process utility 76. An output 84 of the HX / valve process utility is fed to a tank process utility 78. In addition, a distribution (FWS, TCS hot) utility 78 provides parallel feed and output 85 to a tank process utility 78. An output of the tank surface temperature is generated by the tank process utility 78. A parallel distribution / input power utility 77 provides a corresponding output 86 of the surface temperature at the final surface temperature output 87. Optionally, the surface temperature output 87 may be fed to the initial PID tank surface utility 73 via pathways 88, 89 and 82. In addition to the feeding of the output from utility 73 to utility 74, the output may be fed also to a tank model minus current utility 78 and optionally a tank model minus feature utility 80 via respective pathways 91, 81. The output 90 of utility 78 and output 92 of utility 80 may be returned to the utility 73 via pathway 82. Accordingly, the present control system via the SPM 70 is configured to control the various operational conditions, optionally including dielectric coolant flow rate and / or temperature within the system via remote, local, automated or semi-automated electronic control of at least valves 43 and pumps 39.

[0057] The present system provides an arrangement for liquid immersion cooling including independent passive and active control of tanks 10 to enhance thermal efficiency and power density. The present system utilises a dielectric cooling system loop or circuit is combined with a facility water system (FWS) / working fluid network to provide a driven circulating flow of dielectric liquid through the chambers 14 to extract heat from IT hardware 13 and in particular heat generating components 15. The heated / warmed dielectric liquid is delivered and circulated through one or a plurality of coolant distribution units 22a, 22b where the heat load is transferred to the FWS / working fluid via heat exchangers 42. Each FWS dissipates / uses the heat energy optionally via a heat rejection mechanism connected to it (such as a cooling unit, heat reuse components etc). This provides a fully automated or semi-automated system for the thermal management of high heat generating electronic components.

[0058] According to one implementation, the present system comprises two separate sets of coolant distribution units and immersion tanks hydraulically connected in such a way that any one of the coolant distribution units 22a, 22b may be taken off-line for servicing, repair, modification or upgrade. Importantly, the extraction of thermal power from the dielectric coolant liquid is undertaken at the FWS / working fluid side 42b of each heat exchanger 42 by means of control valve 43. Accordingly, the first side 42a of each CDU 22a, 22b is configured with a common constant flow. This, combined with a geometric and / or hydraulic symmetry of the system (with respect to the equal relative size, positioning and / or elevation of each tank assembly 20a, 20b) generates a symmetrical liquid head pressure at manifold 21 that in turn provides the static flow / stagnation regions 60, 61. This enables both sets of coolant distribution units 22a, 22b and tank units 20a, 20b to operate at different temperatures (when in the normal mode of figure 3) despite being hydraulically coupled via the common manifold 21. In such a configuration, the present system is operable in a passive mode. However, when required, such as due to servicing, maintenance or upgrade of any one of the CDUs 22a, 22b, the system may be switched to active mode enabling the hydraulic isolation the desired CDU. According to this active mode (illustrated in figure 4), a single CDU is operational to manage the thermal load (cooling and circulation) of the dielectric liquid received from and delivered to each of the tank assemblies 20a, 20b. Advantageously, the present system, via the hydraulic balancing of the tank assemblies 20a, 20b and / or the specific hydraulic coupling of at least two CDUs to a common manifold 21, provides a low maintenance arrangement with redundancy against system downtime (or minimised risk thereof). That is, the desired mode of operation and control of circulation of the dielectric liquid within manifold 21 is achieved via the creation of the static flow / stagnation region 60, 61. Specifically, this avoids the need for electromagnetic or other controllable valves forming part of the manifold 21 that otherwise increase the architectural / constructional complexity of the system that would otherwise increase the likelihood of component failure that often contributes to system downtime. Accordingly, the present system provides a simplified construction with minimum component parts and in particular electromagnetic, operational control valve components coupling the various assemblies, units and components of the present system.

[0059] A further embodiment of the present concept is illustrated in figure 6. The components and functionality of the embodiment of figure 6 correspond to that of the embodiment and functionality of figures 2 to 5. However, according to the further embodiment, the system comprises a first set of tank assemblies 20a and a second set of tank assemblies 20b. In particular, first set 20a comprises a first tank unit 20a'; a second tank unit 20a" and a third tank unit 20a'". Similarly, the second set of tanks 20b comprises a first tank unit 20b'; a second tank unit 20b'" and a third tank unit 20b'". Each of these sets 20a, 20b are connected at each respective lengthwise end of flow conduits 23, 24 as described and illustrated referring to figures 2 to 4 via connection conduits 95 that are connected fluidically to each of the CDU connection points 50a, 50b, 51a, 51b (at or towards each lengthwise end of first and second flow conduits 23, 24). Accordingly, each CDU 22a, 22b is configured for the circulation of the dielectric coolant liquid within each of the chambers 14 of each set of tanks 20a, 20b. In the normal mode of operation, the dielectric liquid within the chambers 14 of first set 20a is routed through CDU 22a whilst the dielectric liquid within chambers 14 of set 20b is routed through CDU 22b. In the active second mode where for example CDU 22a is hydraulically decoupled from the system, CDU 22b services dielectric coolant circulation to both tank sets 20a, 20b according to the enhanced flow rate through CDU 22b relative to that of the normal (passive) mode as illustrated and described referring to the first embodiment of figures 2 to 5. Example

[0060] Glossary o f terms

[0061] ATS: Automatic Transfer Switch; BMS: Building Monitoring System; BAS: Building Automation System; CDU: Cooling Distribution Unit; DCIM : Data Center Infrastructure Management; FWS: Facility Water System; HMI: Human-Machine Interface; Overflow: Secondary tank in the immersion cooling racks with the function to collect the hottest coolant and keep the fluid level in the main immersion tank; PDU: Power Distribution Unit; PDUM: Power Distribution Unit Module; PHE: Plate Heat Exchanger; TCS: Technology Cooling System (immersion fluid); UI: User Interface; UPS: Uninterruptible Power System; VFD: Variable Frequency Drive for electric motor control.

[0062] A specific operation / implementation is described referring to the apparatus of figures 1 to 5. The system is fluid-agnostic, being capable to operation with different immersion fluids. The Facility Water System (FWS) fluid employed comprises a water (65%) / ethylene glycol (35%) mixture.

[0063] General operational conditions

[0064] 1. All the CDU 22a, 22b are connected in parallel to the FWS, to satisfy the full desired capacity and to minimize the pressure drop of the water / glycol loop.

[0065] 2. A 2N redundancy is given in the frame of the functional block of two tanks 10 and two CDUs 22a, 22b. If a CDU 22a, 22b is out of service, the other CDU of the same functional block will ramp-up to ensure that both tanks 10 are operate at or below the selected set points.

[0066] 3. When the tanks 10 have IT load, the flow rate through each tank 10 is maintained constant. The cooling regulation is performed by varying the flow rate through heat exchanger 42 to reach the desired setpoint temperature in the top of the tank 10 (liquid surface). This control is achieved with regulation control valve 43 controlled by the CDU. Pumps 39, located in the CDU, work at the same speed unless the system changes between the first (nominal) and second (drift) modes. In this case, a Variable Frequency Drive for electric motor control (VFD) will change the pumps operating speed to keep the constant flow condition when feeding one or two tanks 10.

[0067] 4. All immersion tanks 10 and CDUs 22a, 22b are provided with the sensors and controllers that ensure all necessary functions for operation.

[0068] 5. The present system may be fully integrated to with existing (user specific) systems / cooling units. This integration may be via digital and analogue signals or industrial communication bus (preferred) to adjust the supply water temperature as necessary.

[0069] 6. A Rack Power Distribution Unit (PDU) may also be communicated with each CDU 22a, 22b, for more and better control of temperature and tank load.

[0070] Sequence o f operation

[0071] 1. Manual (depending on role-based access rights), Automatic mode (by default) and drift mode (contingency mode) are available.

[0072] 2. System On: The system can be started locally (HMI) or remotely (BMS / BAS / DCIM). The tanks 10 must be already populated with the IT hardware and full of fluid up to the indicated level. Local switch-on will base the operation in the logic programmed in the control unit via the HMI, so the control of the FWS flow rate will depend on the setpoint temperature set with the HMI. In case of remote control, temperature control is set up through communication protocols such as API RESTful or BACnet via the customer BAS / BMS.

[0073] 3. System Off: The system can be turned off when no load is being generated. The process is similar to turning-on a unit.

[0074] 4. Non-conductive fluid temperature control: a. The system is provided with three temperature sensors in the TCS side: tank inlet / CDU outlet (TCScold), tank outlet / CDU inlet (TCShot), and top of the tank (Tsurf). The latter is intended to control the FWS regulation valve. Typically, Tsurf measurement will be around 3°C (5.4°F) higher than TCShot, as it is measured in the hottest point of the tank. This difference may slightly vary as a function of system temperatures, the employed TCS fluid and the heat load. b. Based on the supply water temperature (FWScold), the heat load of the tanks and the desired setpoint fluid temperature (tank surface temperature or Tsurf), thermal balance will be reached at a determined FWS flow rate, and therefore at a certain control valve opening. TCS fluid pump will run continuously. c. When the difference between the temperature sensor at the fluid surface of the tank and the setpoint (i.e. 53°C or 127.4°F) increases, the control valve will gradually open or close depending on the needed cooling capacity. d. The FWS pump will adjust its speed to maintain FWS pressure at the new hydraulic impedance of the CDU. The control system will reach the new required flow rate to dissipate the IT generated heat load. e. If one CDU fails, the TCS pumps of the paired one will speed up to increase the TCS flow rate through the heat exchanger and equilibrate the flow of both tanks. Simultaneously, the FWS control valve will adjust the position as described in (c) to reach thermal balance for the new heat load of the CDU (generated by two tanks) not to exceed the setpoint of any tank.

[0075] Alerts

[0076] System status and any notification of possible operational errors are available both through the HMI and the Customer management software. Alerts are triggered by different severities and events such as information, warning, error and critical (i.e pump motor failure, tank overheating, motor overheating, low TCS flow rate, communication error, fluid leakages in the CDU and the TCS manifold, electrical issues, etc). Depending on how the event affects the process, the CDU system will make decisions, with decisions requiring manual or automatic intervention of a user. The alerts will be shared with the BAS / BMS management system, as well as with the operator via the local HMI.

[0077] Processes under power loss

[0078] • The present system provides redundancy in a situation where there no backup generators. In the event of a regular power outage, each CDU can be connected to a different power source to ensure that at least one CDU is available. Both CDUs connected to a redundant power supply is also a solution. The process in which one CDU ramps-up to take the duty of the entire functional block of tanks 10 after a tank 10 shuts down or has a brief delay of approximately five seconds. The continuity of the operation during this time is ensured by the thermal inertia of the system.

[0079] • The IT hardware being cooled and located within the immersion tanks 10 receive power from an Uninterruptible Power System (UPS).

[0080] • The present controller unit could be connected to a UPS, eliminating the need for a system reboot. If there is no UPS, the controller will oversee the shutdown and subsequent reactivation of the CDU once it becomes possible. This process may take approximately 40 seconds in total, and the continuity of operation may be maintained by the thermal inertia of the system, depending on the operating conditions.

[0081] The present dielectric coolant apparatus and method is advantageous to regulate extraction of thermal power from IT hardware 13 and in particular heat generating electronic components 15 via immersion cooling within a hydraulically balanced arrangement having a minimised numbers of controllable valves and other hydraulic circuit components. The present geometrically and hydraulically balanced system (via at least two CDUs and two immersion cooling tanks) is scalable with regard to both tank size and tank number (for example referring to the sets of coupled or ganged tanks of the embodiment of figure 6. The present system is operable via two modes of operation (passive and active) in which both or all CDUs are operational or a mode in which at least one CDU is inoperative whilst the single or other remaining CDUs are operated at a higher power draw condition to achieve higher dielectric coolant throughput and cooling rate.

[0082] The present arrangement comprising buffer reservoirs 16 is advantageous to compensate for asymmetrical liquid head loss at manifold 21 (required to create the static flow / stagnation regions 60, 61). The present system is further advantageous via the utilisation of mechanical cut-off valves (such as valves 33, 34) to enable fluidic isolation / disconnection of one or more tank assemblies 20a, 20b and optionally CDUs 22a, 22b (via valves 35, 36). The control system of the present concept comprising the various sensors 17a, 17b, 40, 41, 55, 56, control valves 43 and pumps 39, together with the control model software and computer control apparatus (eg PC, handheld device, PLC etc) provides an automated or semi-automated thermal management system. In particular, thermal power extraction is achieved by the circulation of the dielectric coolant liquid that may be controlled using a semi -predictive model. This is used to regulate the temperature and / or flow rate of the dielectric coolant liquid output at each respective CDU outlet 28 (via control of pumps 39 and / or control valves 43). This control, according to the present system is in direct response to the temperature of the dielectric liquid as determined by sensors 17a. The present control system (including sensors 17a, 17b etc) controls a transition between the different passive and active modes (normal or servicing / maintenance modes) as required. The present system is further advantageous to function via minimised power input for energy efficiency and maximised power density.

Claims

Claims1. Liquid cooling apparatus for an electronic device comprising: at least two immersion tanks, each tank having a liquid inlet and a liquid outlet, and a chamber to accommodate at least one electronic device having at least one heatgenerating electronic component, and a dielectric coolant liquid capable of flowing through the chamber via the inlet and outlet; at least two coolant distribution units having a liquid inlet, a liquid outlet and at least one cooling component in fluidic communication with each of the chambers via the respective inlets and outlets to enable a flow of the dielectric coolant liquid between each of the chambers and the coolant distribution units; a manifold fluidically connecting the inlets and outlets of the chambers and the coolant distribution units, the manifold having a first flow conduit extending between the outlets of the at least two of the tanks, and a second flow conduit extending between the inlets of the at least two tanks; each inlet of each coolant distribution unit fluidically connected to the first flow conduit and each outlet of each coolant distribution unit fluidically connected to the second flow conduit.

2. The apparatus as claimed in claim 1 wherein the tanks are identical or nearly identical in size and / or an internal volume of the respective chambers to contain respectively an equal or approximately equal volume of dielectric coolant liquid such that a liquid head pressure created by the tanks at a region within the first and / or second flow conduit is equal or approximately equal.

3. The apparatus as claimed in claims 1 or 2 wherein the tanks are located at the same or approximately the same elevation relative to one another such that a liquid head pressure created by the tanks at a region within the first and / or second flow conduit is equal or approximately equal.

4. The apparatus as claimed in any preceding claim wherein a respective static flow condition is created at the first and / or second flow conduit resultant from a balancing of the liquid head pressure created by the tanks.

5. The apparatus as claimed in claim 4 wherein the static flow condition is located between respective connection points by which the at least two coolant distribution units are fluidically connected to the first and / or second flow conduits.

6. The apparatus as claimed in any preceding claim wherein each of the tanks comprises a buffer reservoir fluidically connected to each respective chamber to receive an overflow of the dielectric coolant liquid, each buffer reservoir connected in fluidic communication between the chamber and the outlet of the respective tanks.

7. The apparatus as claimed in any preceding claim wherein each tank comprises at least one temperature sensor positioned relative to the chamber to monitor a temperature of the dielectric coolant liquid within the chamber.

8. The apparatus as claimed in claim 7 wherein the temperature sensor is located at or towards a liquid-air interface at the chamber to monitor a temperature of the dielectric coolant liquid at or towards the liquid-air interface of the coolant dielectric liquid within the chamber.

9. The apparatus as claimed in any preceding claim comprising at least one valve located between each inlet and each outlet of each coolant distribution unit and the first and second main flow conduit to allow control of a flow of the coolant dielectric liquid between each coolant distribution unit and the manifold to enable each coolant distribution unit to be fluidically isolated from the manifold.

10. The apparatus as claimed in any preceding claim comprising at least one valve located between each inlet and each outlet of each tank and a respective central section of the first and / or second fluid flow conduit to enable each of the tanks to be fluidicallyisolated from the central sections of the first and / or second flow conduits and / or the coolant distribution units.

11. The apparatus as claimed in any preceding claim wherein the coolant distribution unit comprises any one or a combination of:• a chiller;• a refrigeration unit;• a heat exchanger;• a dry-cooler;• an adiabatic cooler;• a heat pump.

12. The apparatus as claimed in any one of claims 1 to 10 wherein each coolant distribution unit comprises a heat exchanger, a first side of each heat exchanger fluidically connected to the respective inlet and the outlet of the coolant distribution unit and a second side of each heat exchanger fluidically connected to a working fluid network to allow heat transfer between the dielectric coolant liquid and the working fluid.

13. The apparatus as claimed in claim 12 comprising a fluid flow control valve located at the second side of the heat exchanger to regulate a flow rate of the working fluid to and / or from the heat exchanger.

14. The apparatus as claimed in claim 13 wherein the fluid flow control valve is any one or a combination of:• an electromagnetic valve;• an electronic valve;• an electromechanical valve;• a 2-way valve;• a 3 -way valve; and• a pressure independent control valve (PICV).

15. The apparatus as claimed in any preceding claim wherein each coolant distribution unit comprises any one or a combination of:• a liquid flow pump;• a filter or a particulate filter;• a flow sensor;• a temperature sensor;• a pressure sensor;• at least one valve.

16. The apparatus as claimed in any preceding claim wherein each coolant distribution unit comprises a split flow section having two parallel flow lines, each of the lines connected fluidically between the inlet and the coolant component and each parallel flow line having a pump and at least one of the flow lines having a filter and a respective valve positioned at each fluid flow side of the filter to enable the filter to be fluidically isolated from the coolant distribution unit.

17. The apparatus as claimed in any preceding claim wherein the at least two tanks comprise a first and a second set of tanks, each of the tanks of the first set of tanks fluidically connected at or towards a first end of the first and / or second flow conduit and each of the tanks of the second sets of tanks fluidically connected at or towards a second end of the first and second flow conduit.

18. The apparatus as claimed in claim 17 wherein the first and second sets of tanks are separated by respective central sections of the first and second main flow conduits.

19. The apparatus as claimed in claim 18 wherein each of the coolant distribution units are connected to the first and second fluid flow conduits at either side of the respective central sections such that the central sections separate the fluid flow junctions between which the coolant distribution units are coupled fluidically to the first and second flow conduits.

20. The apparatus as claimed in any preceding claim comprising a control unit configured for the automated, semi-automated and / or remote control of at least one operational parameter of the apparatus, wherein the operational parameter comprises any one or a combination of• a flow of the dielectric coolant liquid within the apparatus;• a driving of flow of the dielectric coolant liquid within the apparatus;• a flow rate of the dielectric coolant liquid within the apparatus;• a pressure of the dielectric coolant liquid within the apparatus;• a temperature of the dielectric coolant liquid within the apparatus;• a fluidic connection and / or isolation of any one of the tanks and / or the coolant distribution units from the manifold.

21. The apparatus as claimed in claim 20 when dependant on claim 12 wherein the operational parameter comprises any one or a combination of• a flow of the working fluid to and / or from the heat exchanger;• a driving of a flow of the working fluid to and / or from the heat exchanger;• a flow rate of the working fluid to and / or from the heat exchanger;• a pressure of the working fluid within the apparatus;• a temperature of the working fluid within the apparatus.

22. A method of cooling an electronic device comprising: providing at least two immersion tanks, each tank having a liquid inlet and a liquid outlet, and a chamber to accommodate at least one electronic device having at least one heat-generating electronic component, and a dielectric coolant liquid capable of flowing through the chamber via the inlet and outlet; providing at least two coolant distribution units having a liquid inlet, a liquid outlet and at least one cooling component in fluidic communication with each of the chambers via the respective the inlets and outlets to enable a flow of the dielectric coolant liquid between each of the chambers and the coolant distribution units; providing a manifold fluidically connecting the inlets and outlets of the chambers and the coolant distribution units, the manifold having a first flow conduit extending between the outlets of the at least two of the tanks, and a second flow conduit extendingbetween the inlets of the at least two tanks, wherein each inlet of each coolant distribution unit is fluidically connected to the first flow conduit and each outlet of each coolant distribution unit is fluidically connected to the second flow conduit; and circulating a dielectric coolant liquid through the chambers, the manifold and the coolant distribution units to cool at least one electronic device immersed within at least one of the chambers.

23. The method as claimed in claim 22 comprising creating an equal or approximately equal liquid head pressure at the first flow conduit and the second flow conduit by providing the tanks with an identical or near identical size and / or internal volume of the respective chambers to contain respectively an equal or approximately equal volume of dielectric coolant liquid within the respective chambers.

24. The method as claimed in claims 22 or 23 comprising creating an equal or approximately equal liquid head pressure at a respective first and a second side of each of the first flow conduit and the second flow conduit by locating the tanks at the same or approximately the same elevation relative to one another.

25. The method as claimed in any one of claims 22 to 24 further comprising: creating a respective static flow condition at a central or approximately central region of each of the first flow conduit and the second flow conduit; and wherein the step of circulating the dielectric coolant liquid comprises circulating the dielectric coolant liquid at either side of each of the static flow conditions in that the dielectric coolant liquid does not flow within the region of each of static flow condition.

Images