Modular Liquid Cooling Architecture for Liquid Cooling

Abstract

The storage and cooling system for computer hardware includes an infrastructure module and a payload module. The infrastructure module is configured to store the computer hardware and includes either or both of a convection air cooling system and a metal plate arrangement connected by one or more conduits for carrying liquid for cooling the computer equipment stored by the infrastructure module. The infrastructure module also stores a programmable logic controller ("PEC"). The payload module includes an immersion cooling system managed by a PLC and is located outside the infrastructure module.

Description

[Background technology]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. Patent Application No. 17 / 532,031, filed November 22, 2021, the disclosure of which is incorporated herein by reference.

[0002] background Computer hardware generates heat during operation and tends to operate better and fail at a lower rate when cooled. For this reason, cooling systems for computer hardware have been developed. A variety of cooling systems can be found in individual data centers, which often house multiple types of computer hardware with different cooling needs. Individual server racks or other cabinet-type housings for the hardware may have convection air cooling systems, such as liquid-to-air heat exchangers, or cold plate systems, in which a conductive element with liquid conduits extending therethrough is placed in contact with the components to be cooled. Elsewhere in the center, immersion cooling systems may include a tank in which the computer hardware may be immersed in an evaporable liquid, and a condenser located above the tank. These solutions may be inconvenient for customers who use the data center for hardware with completely different cooling needs, due to the additional effort involved in accessing hardware distributed in different locations in the center. Further problems may arise if direct communication between hardware with different cooling needs is desired. For example, components that produce relatively little heat may be placed in a tank with components that must be immersion cooled when direct communication between the components is not required, resulting in unnecessary expense and tank congestion. Because some components cannot be immersion cooled in any case, placing such components in communication with immersion cooled components can be extremely difficult. Summary of the Invention

[0003] Quick Overview Aspects of the present disclosure relate to a modular liquid cooling system that can provide immersion and condenser cooling modules in cooperation with lower cooling capacity modules, such as liquid-air exchangers or cold plate devices. A rack can be decomposed into one or more infrastructure modules and one or more payload modules for greater flexibility and utilization. Infrastructure modules containing low power air-cooled information technology ("IT") or other computing equipment may rely on low impedance rear door type heat exchangers or cold plates where heat is transferred locally to the facility water. Payload modules containing high power IT or other computing equipment may rely on two-phase immersion cooling to transfer heat immediately to the facility water.

[0004] The infrastructure module may house a programmable logic controller ("PLC") that manages the cooling functions of each module. The PLC may thereby manage aspects of the infrastructure module's heat exchangers and each payload module's immersion cooling system. Power for some or all functions of the payload modules and computing equipment housed by the payload modules may be transmitted through the infrastructure module. Cooling fluid may be supplied to all modules in series or parallel, or first to the infrastructure module and then to each payload module in parallel.

[0005] In another aspect, a housing and cooling system for computer hardware may include an infrastructure module configured to store computer hardware. The infrastructure module may include either or both of a convection air cooling system and an arrangement of metal plates connected by one or more conduits for carrying liquid for cooling computer equipment stored by the infrastructure module. The infrastructure module may store a PLC. The system may also include a payload module including an immersion cooling system managed by the PLC and located outside the infrastructure module. Management of the immersion cooling system of the payload module may simplify maintenance and operation of the immersion cooling system. Access to the immersion cooling system may be difficult and disruptive to operation and cooling of hardware in the immersion cooling system, since such systems are usually closed to prevent steam escape. Thus, it may be relatively easy to access the PLC in an infrastructure module, whose housing may be a simple server cabinet or similar.

[0006] In another arrangement according to any of the above, the infrastructure module may include one or more racks of computing hardware having a first power consumption level.

[0007] In another arrangement according to any of the above, the payload module may include computing hardware having a second power consumption level, the second power consumption level being greater than the first power consumption level.

[0008] In another arrangement according to any of the above, the payload module may have the dimensions of one EIA-310 standard server rack, or two or three adjacent EIA-310 standard server racks.

[0009] In another arrangement according to any of the above, the convection air cooling system includes a liquid-to-air heat exchanger.

[0010] In another arrangement according to any of the above, a heat exchanger or conduit of the infrastructure module may be fluidly connected to a condenser of the payload module.

[0011] In another arrangement according to any of the above, the flow path of the cooling fluid through the housing and cooling system may pass through a cooling device of the infrastructure module before reaching the condenser.

[0012] In another arrangement according to any of the above, the payload module may be a first payload module and comprises a second payload module including an immersion cooling system, and the immersion cooling systems of the first and second payload modules are managed by the PLC through a central fluid management system housed within the infrastructure module.

[0013] In another arrangement according to any of the above, the infrastructure module may include an infrastructure bus bar, and the payload module includes a payload bus bar electrically connected to the infrastructure bus bar.

[0014] In another arrangement according to any of the above, the infrastructure module and the payload module may each include a respective bus bar, the bus bar of the payload module being connected to the bus bar of the infrastructure module, the bus bar of the infrastructure module being provided with a connector for receiving power from an external power source.

[0015] In another arrangement according to any of the above, the payload module may be a first payload module. The immersion cooling system of the first payload module may include a first condenser. The housing and cooling system may also include a second payload module, which may include an immersion cooling system managed by the PLC and including a second condenser. The first condenser and the second condenser may be connected in series along the flow path of the cooling fluid. For example, the flow path from the coolant supply to the coolant return may instead run in series first through the infrastructure module, then through the condenser system in the second payload module, and finally through the condenser system of the first payload module. Thus, the first payload module receives only the coolant already used for cooling by the second payload module. Thus, the hardware to be cooled in each module may be selected taking into account the flow path of the coolant to optimize efficiency and energy utilization in the coolant. For example, expenditures per unit volume of coolant, per unit heat imparted to the coolant used, and per unit power required to operate the system and the hardware cooled by the system may be considered in finding the most economical distribution of hardware among the different modules.

[0016] In another example according to any of the above, the housing and cooling system may also include a distribution system that circulates the cooling fluid through the infrastructure and payload systems. The housing and cooling system may also include a dry cooler through which the distribution system passes, the dry cooler configured to exchange heat from the cooling fluid with ambient air.

[0017] In another arrangement according to any of the above, the infrastructure and payload modules may be connected in series in the distribution system.

[0018] In another aspect, a method for storing and cooling computer hardware may include storing a first computer hardware in an infrastructure module that also stores a power line processor ("PLC"). The method may also include submerging a second computer hardware in an evaporable liquid in a two-phase submersion cooling system stored by the payload module. The payload module may be located outside of the infrastructure module and the submersion cooling system managed by the PLC.

[0019] In another arrangement according to any of the above, the method may include cooling the first computer hardware with a cooling device included by an infrastructure module, the cooling device of the infrastructure module being either or both of a liquid-to-air heat exchanger and a metal plate device connected by one or more conduits for transporting liquid.

[0020] In another aspect according to any of the foregoing, the method may include connecting the cooling devices of the infrastructure modules and the condensers of the two-phase immersion cooling system in parallel to a building liquid supply. Because the modules receive cooling fluid in parallel, heat generated by and removed from hardware in any one module has little or no effect on the cooling of the hardware in any other module. Thus, overall system efficiency is less dependent on the selection of which payload module stores and cools a particular hardware than is the efficiency of a system in which components are connected in series.

[0021] In another aspect according to any of the above, the method may include connecting a chiller and a condenser of the infrastructure module in series to a building liquid supply.

[0022] In another aspect according to any of the above, the method may include connecting the cooling device and condenser of the infrastructure module to a building liquid supply such that liquid is supplied by the building liquid supply to the cooling device of the infrastructure module and returned from the condenser to the building liquid supply.

[0023] In another aspect according to any of the above, the method may include providing power to the payload module and the second computer hardware through the infrastructure module.

[0024] In another aspect according to any of the foregoing, the payload module may be a first payload module and may include immersing the third computer hardware in an evaporable liquid in a tank contained by a second payload module including a two-phase submersion cooling system, the two-phase submersion cooling system of the second payload module may be managed by the PLC. [Brief description of the drawings]

[0025] [Figure 1] FIG. 2 is a schematic diagram of a modular cooling system according to a first arrangement. [Diagram 2] FIG. 13 is a schematic diagram of a modular cooling system according to a second arrangement. [Diagram 3] FIG. 13 is a schematic diagram of a modular cooling system according to a third arrangement. [Figure 4] FIG. 13 is a schematic diagram of a modular cooling system according to a fourth arrangement. [Diagram 5] FIG. 13 is a schematic diagram of a modular cooling system according to a fifth arrangement. [Figure 6] 6 is a flow chart of a method for storing and cooling an electronic device comprising any of the modular cooling systems shown in FIGS. 1-5. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] Detailed Description 1 shows a modular cooling system 100 that may be used for information technology ("IT") equipment, other computer hardware, or other items that need to be cooled. System 100 includes an infrastructure module 110 and a first payload module 120. System 100 in the illustrated arrangement also includes a second payload module 130, although system 100 may have only one payload module or any number of payload modules in other arrangements.

[0027] Infrastructure module 110 includes a housing for information technology ("IT") equipment or other electronic hardware. Infrastructure module 110 houses infrastructure hardware 113, which may be, for example, one or more racks of computer components, servers, other types of IT equipment or other electronic equipment, and is housed by the housing of infrastructure module 110. The infrastructure includes and optionally houses programmable logic controllers ("PLCs") 112 and infrastructure bus bars 114 for providing power to components of infrastructure module 110 within or connected to infrastructure module 110. PLCs 112 themselves may be optionally contained by infrastructure hardware 113.

[0028] The infrastructure module 110 may also optionally include a heat exchanger 111 as shown in the illustrated example. The heat exchanger 111 may be, for example, a liquid-to-air heat exchanger that utilizes a supply of cooling fluid to cool air for convection cooling of either or both of the infrastructure hardware 113 and the PLC 112. Air heated by either or both of the infrastructure hardware 113 and the PLC 112 may be propelled to the heat exchanger 111 for cooling before being exhausted from the infrastructure module 110 or recirculated within the infrastructure module. In other examples, air received into the heat exchanger 111 from outside or within the infrastructure module 110 may first be cooled by the heat exchanger 111 before being propelled across either or both of the infrastructure hardware 113 and the PLC 112. Such a liquid-to-air heat exchanger may be mounted to any wall or exterior surface of the housing of the infrastructure module 110, such as, for example, a rear door or a front door.

[0029] In other examples, the heat exchanger 111 may be a contact cold plate system including one or more plates of conductive material, such as aluminum, copper, other metals, or certain ceramics or polymers, or any combination thereof. Thus, in some embodiments, the cold plates may correspond to metal plates. Each plate has at least one conduit extending along or through the plate to direct a cooling fluid to remove heat from the cold plate. Thus, either or both of the infrastructure hardware 113 and the PLC 112 may be cooled by being placed in contact with one of the cold plates.

[0030] Although a liquid-to-air heat exchanger and a cold plate system are presented as examples of heat exchanger 111, heat exchanger 111 can be another type of heat exchanger having similar capabilities and operational needs.

[0031] The system 100 also includes a first payload module 120. The first payload module 120 includes a housing for electronic hardware, a payload bus bar 124, and a bulk cooling system. The payload bus bar 124 is configured to provide power to the first payload module 120 and to components within or connected to the first payload module 120. The bulk cooling system may be, for example, a two-phase immersion cooling system for electronics or other hardware. Such an immersion cooling system includes a tank for an evaporable liquid and a condenser system 121 positioned above the tank or at other location where vapor from the tank travels to the condenser system 121 and then returns to the tank after condensing to a liquid. Examples of suitable evaporable liquids include any of several commercially available dielectric fluids designed specifically for immersion cooling of computer hardware, although water may be used in some applications, particularly where electrical conductivity of the evaporable fluid is not an issue. Condenser system 121 may be any known type of condenser, but in some examples, condenser system 121 includes one or more tubes carrying a cooling fluid that reduces the temperature of the tubes to a suitable temperature for condensing vapor rising from the evaporable liquid in the tank. The immersion cooling system also includes a humidity control system 122 for monitoring and regulating humidity within infrastructure module 120 via condenser system 121 or otherwise, and a fluid management system 125 for controlling the level of evaporable liquid in the tank. Fluid management system 125 may include, for example, any one or any combination of pumps, filters, and fluid level monitors for the evaporable fluid in the tank of the immersion cooling system of first payload module 120.

[0032] The payload hardware 123 may be one or more racks of electronic equipment submerged in a tank of evaporable liquid in the immersion cooling system of the first payload module 120. Because the payload module 120 has a high-capacity cooling system, the payload hardware 123 stored and cooled by the payload module 120 may be hardware that has greater cooling needs and in some cases greater power consumption levels than the infrastructure hardware 113. In some examples, the payload hardware 123 may also be larger in volume than the infrastructure hardware 113. For example, if the infrastructure hardware 113 occupies 12 rack units ("RU"), then the payload hardware 123 may occupy 24 RU, although any other volumes and ratios between the volumes of the infrastructure hardware 113 and the payload hardware 123 are possible.

[0033] The infrastructure module 110 is operatively connected to the first payload module 120 at multiple points in the illustrated example. The coolant supply 146 is routed through the infrastructure module 110 before reaching the condenser system 121 and, after passing through the condenser system 121, exits the system 100 through a coolant return 149 that directs the coolant for the coolant supply 146 to recycle or otherwise discard the coolant. The cooling fluid of the coolant supply 146 can be a liquid, such as, for example, water, glycol and water solutions, or a dielectric fluid, such as a fluorocarbon, or other liquid, or a gas, such as, for example, air, carbon dioxide, or other gas. In an example where the infrastructure module 110 includes a heat exchanger 111, the coolant supply 146 may pass through and act as a coolant for the heat exchanger 111 before reaching and acting as a coolant for the condenser system 121. Both the coolant supply 146 and the coolant return 149 are part of a fluid coolant distribution system that circulates coolant through the internal cooling systems of each of the modules 110, 120, 130 and the system 100 as a whole. The distribution system may be provided by a tap system, another type of building liquid supply system, or any other source of liquid or gas suitable for use as a coolant. The coolant supply 146 is at a relatively low temperature and the coolant return 149 is at a relatively high temperature because the modules 110, 120, 130 transfer heat to the coolant as it flows through the flow paths of the distribution system. Downstream of the coolant return 149 and upstream of the coolant supply 146, the distribution system either cools the coolant or discards the heated coolant to obtain fresh coolant at a lower temperature.

[0034] The PLC 112 is in electronic communication with the condenser system 121, the humidity control system 122, and the fluid management system 125. This electronic connection between the PLC 112, the humidity control system 122, and the fluid management system 125 allows the PLC 112 to control the humidity control system 122 and the fluid management system 125, which in turn manages the immersion cooling system of the first payload module 120. In various examples, the PLC 112 may be able to turn the power of the immersion cooling system on or off, adjust the temperature maintained by the immersion cooling system, adjust the flow rate within the immersion cooling system, or any combination of the above. The management of the immersion cooling system of the first payload module 120 by the PLC 112 of the infrastructure module 110 can simplify the maintenance and operation of the immersion cooling system. Access to the immersion cooling system can be difficult and disruptive to the operation and cooling of the hardware in the immersion cooling system, as such systems are typically closed to prevent the escape of steam. It may therefore be relatively easy to access the PLC 112 in the infrastructure module 110, whose housing may be a simple server cabinet or the like.

[0035] PLC 112 may be in electronic communication with a management system 147, such as a building management system or a data network. A connection between PLC 112 and management system 147 allows PLC 112 and the immersion cooling system of first payload module 120 to be controlled by a remote operator or in conjunction with other systems. In the illustrated example, infrastructure hardware 113 and payload hardware 123 are also in electronic communication with each other, although in other examples, infrastructure hardware 113 and payload hardware 123 may not be in communication with each other.

[0036] The power and network connection 148 may also pass through the infrastructure module 110 before reaching the first payload module 120. The power aspect of the power and network connection 148 may be, for example, a building power supply, a grid power supply, or a battery. The network aspect of the power and network connection 148 may be, for example, the Internet, a telephone, a local area network, or other electronic communication network. In the illustrated example, power is communicated between the infrastructure hardware 113 and the infrastructure bus bar 114, which is connected to the payload bus bar 124. Thus, the payload module 120 may receive some or all of its operating power for itself and for the payload hardware 123 from its connection to the infrastructure module 110. However, the payload module 120 and the payload hardware 123 may obtain power from other sources, and in other examples, either or both of the payload module 120 and the payload hardware 123 may obtain all of their required power from sources other than the infrastructure module 110 and the infrastructure hardware 113.

[0037] There may be a system including an infrastructure module 110 and only one payload module having the characteristics of the first payload module 120. However, the illustrated example system 100 also includes a second payload module 130, which is identical to the first payload module 120 with respect to all the details of the first payload module 120 described above. All connections between the infrastructure module 110 and the first payload module 120 also run in parallel between the infrastructure module 110 and the second payload module 130, except for the coolant flow path from the coolant supply 146 and the coolant return 149. The flow path from the coolant supply 146 to the coolant return 149 instead passes in series first through the infrastructure module 110, then through the condenser system in the second payload module 130, and finally through the condenser system 121 of the first payload module 120. Thus, the first payload module 120 receives only the coolant that has already been used for cooling by the second payload module 130. Thus, the cooled hardware in each module 110, 120, 130 may be selected with consideration of the coolant flow path to optimize the efficiency of coolant and energy utilization. For example, the cost per unit volume of coolant, the cost per unit heat imparted to the coolant used, and the cost per unit power required to operate the system 100 and the hardware cooled by the system 100 may be considered in finding the most economical distribution of hardware within the modules 110, 120, 130.

[0038] Infrastructure module 110 may optionally be of a relatively small form factor, such as that of a typical server rack or cabinet. For example, infrastructure module 110 may have the shape and dimensions of an EIA-310 standard server rack, or at least similar dimensions sufficient to facilitate installation of infrastructure module 110 within such a rack.

[0039] The payload modules 120, 130 may optionally be adapted for installation in a typical server rack or cabinet. For example, the payload modules 120, 130 may have the same or approximately the same shape and dimensions as one, two or more adjacent EIA-310 standard server racks, allowing the payload modules 120, 130 to be installed in such racks. The payload modules 120, 130 so configured may be installed adjacent to or near the infrastructure module 110 in an area of ​​the data center set up to store server racks. Thus, the system 100 allows types of hardware that might otherwise be distributed across a data center or among multiple different facilities due to their different cooling needs to be stored and operated in one location. Furthermore, the system 100 allows low temperature operating hardware that can be effectively cooled by the heat exchanger 111 to be installed near hardware that must be immersion cooled without the need for immersion cooling to be installed near hardware that must be immersion cooled without the low temperature operating hardware taking up space and wasting resources in the immersion cooling system. Locating the payload modules 120, 130 near the infrastructure module 110 also facilitates access to or operation of the modules 110, 120, 130 by their owners or technicians. Locating the payload modules 120, 130 near the infrastructure module 110 also simplifies the electronic connections between the infrastructure hardware 113, the payload hardware 123, and the hardware housed by the second payload module 130.

[0040] The presence of only the first payload module 120 and the second payload module 130 in the system 100 is shown by way of example, and systems having other numbers of payload modules may be created. In some examples, the second payload module 130 may be omitted entirely, such that the coolant flow path extends directly from the infrastructure module 110 to only the first payload module 120 before exiting the system 100 through the coolant return 149. In other examples, the system 100 may include any number of payload modules connected in series along the coolant flow path between the infrastructure module 110 and the first payload module 120. All such additional modules may be connected to the infrastructure module 110 for electronic communication and power transfer in the same manner as the first payload module 120 and the second payload module 130.

[0041] 2 illustrates system 200 in which elements are generally similar to like-numbered elements of system 100. For example, except where specifically illustrated or described, infrastructure module 210 is the same as infrastructure module 110, first payload module 220 is the same as first payload module 120, second payload module 230 is the same as second payload module 130, etc. For this reason, certain numbers in FIG. 2 may not be specifically referenced herein. Moreover, all of the above-mentioned possible variations on elements of system 100 are equally possible for elements of system 200, except to the extent that they are inconsistent with features of system 200 that differ from features of system 100.

[0042] System 200 differs from system 100 in that coolant is supplied to each module 210, 220, 230 in parallel. Thus, coolant from the first coolant supply 246a flows through a heat exchanger 211 before exiting the system 200 through a first coolant return 249a, coolant from the second coolant supply 246b flows through a condenser system 221 of the first payload module 220 before exiting the system 200 through a second coolant return 249b, and coolant from the third coolant supply 246c flows through a condenser system of the second payload module 230 before exiting the system 200 through a third coolant return 249c. In other examples in which the infrastructure module 210 lacks a heat exchanger 211, the infrastructure module 210 may not have any coolant supplies or coolant returns.

[0043] The presence of only the first payload module 220 and the second payload module 230 in the illustrated example is one configuration of the system 200, although in other examples the system 200 may omit the second payload module 230, and in still other examples the system 200 may include any number of payload modules having independent parallel coolant supply and return connections.

[0044] Similar to system 100, system 200 allows an infrastructure module 210 installed between one or more payload modules 220, 230 to provide easy access to management of immersion cooling in the PLC 212 and to store and operate hardware of different cooling needs in one location. However, because the modules 210, 220, 230 receive cooling fluid in parallel, heat generated by and removed from hardware in any one module 210, 220, 230 has little or no effect on the cooling of the hardware in any other module 210, 220, 230. Thus, the overall efficiency of system 200 is less dependent on the selection of which payload module 220, 230 stores and cools a particular hardware than is the efficiency of system 100.

[0045] 3 illustrates a system 300 in which elements are generally the same as the like-numbered elements of systems 100 and 200. For example, except where specifically illustrated or described, infrastructure module 310 is the same as infrastructure modules 110 and 210, first payload module 320 is the same as first payload modules 120 and 220, second payload module 330 is the same as second payload modules 130 and 230, etc. For this reason, certain numbers in FIG. 3 may not be specifically referred to herein. Moreover, all of the above-mentioned possible variations on elements of systems 100 and 200 are equally possible for elements of system 300, except to the extent that they are inconsistent with features of system 300 that differ from features of systems 100 or 200.

[0046] In the system 300, coolant is supplied in parallel to each payload module 320, 330 through the infrastructure module 310. Thus, coolant from the coolant supply 346 flows through a heat exchanger 311 before splitting into separate flow paths. One of the flow paths extends through the condenser system 321 of the first payload module 320 before exiting the system 300 through a first coolant return 349a, while the other flow path extends through the condenser system of the second payload module 330 before exiting the system 300 through a second coolant return 349b.

[0047] While the presence of only the first payload module 320 and the second payload module 330 in the illustrated example is one configuration of the system 300, in other examples, the system 300 may omit the second payload module 330, and in still other examples, the system 300 may include any number of payload modules that receive coolant in parallel with each other after the coolant passes through the heat exchanger 311 and before the coolant exits the system 300 through their respective coolant returns.

[0048] System 300 has all the capabilities that system 200 shares with system 100. Like system 200, system 300 is configured to produce heating and cooling within any one payload module with little, if any, effect on the temperature of the coolant entering any other payload module. Passing all the coolant through infrastructure module 310 first may simplify installation of system 300 in some cases, while connecting all modules in parallel as shown in system 200 may be simpler in other situations. Although infrastructure hardware 313 in some instances exhibits a relatively low heat load, the cooling of payload modules 320, 330 may not be significantly affected by their placement downstream of infrastructure module 310. However, if the heat exchanger 311 splits the coolant into separate channels that are expected to have different heat loads within the heat exchanger 311, and these separate channels flow to different payload modules 320, 330, then the different heat loads within the heat exchanger 311 may be taken into account when selecting payload modules 320, 330 for a particular hardware to optimize the cooling efficiency of the system 300. In other respects, as with system 200, the selection of payload modules 320, 330 for a particular hardware in system 300 has a relatively small impact on the cooling performance of the system 300.

[0049] 4 illustrates system 400 in which elements are generally similar to like-numbered elements of system 300. For example, infrastructure module 410 is the same as infrastructure module 310, first payload module 420 is the same as first payload module 320, second payload module 430 is the same as second payload module 330, etc., except where specifically illustrated or described differently. For this reason, certain numbers in FIG. 4 may not be specifically referred to herein. Furthermore, all of the above-mentioned possible variations in the elements of systems 100, 200 and 300 are equally possible for the elements of system 400, except to the extent that they are inconsistent with features of system 400 that differ from features of systems 100, 200 or 300.

[0050] In system 400, coolant is supplied in parallel to each payload module 420, 430 through infrastructure module 410 in the same manner that system 300 distributes coolant in parallel to payload modules 320, 330 through infrastructure module 310. Thus, coolant from coolant supply 446 flows through heat exchanger 411 before being split into separate flow paths, one of which extends through the condenser system 421 of the first payload module 420 before exiting the system 400 through a first coolant return 449a, while another flow path extends through the condenser system of the second payload module 430 before exiting the system 400 through a second coolant return 449b.

[0051] System 400 has all of the capabilities that system 300 shares with systems 100 and 200, as well as most of the capabilities that differentiate system 300 from systems 100 and 200. However, system 400 differs from system 300 by the location of a central fluid management system 415 within the infrastructure module 410. The central fluid management system 415 manages the liquid levels in the tanks of the submersion cooling systems in each of the payload modules 420, 430, such that each individual payload module 420, 430 does not need to include dedicated circuitry or control devices to manage its own liquid levels. The central fluid management system 415 is in electronic communication with the PLC 412 and may be managed by the PLC 412 in a manner similar to the management of the distributed fluid management systems in systems 100, 200, 300. The placement of the central fluid management system 415 within the infrastructure module 410 provides the same ease of access to the central fluid management system 415 as is provided for the PLC 412 without taking up space in the payload modules 420, 430.

[0052] While the presence of only the first payload module 420 and the second payload module 430 in the illustrated example is one configuration of the system 400, in other examples, the system 400 may omit the second payload module 430, and in still other examples, the system 400 may include any number of payload modules having an immersion cooling system managed by the central fluid management system 415.

[0053] 5 illustrates a system 500 in which elements are generally similar to like-numbered elements of systems 100, 200, and 300. For example, infrastructure module 510 is the same as infrastructure modules 110, 210, and 310, payload module 520 is the same as payload modules 120, 220, and 320, etc., except as specifically illustrated or described. For this reason, certain numbers in FIG. 5 may not be specifically referred to herein. Furthermore, all of the above-mentioned possible variations in the elements of systems 100, 200, 300, and 400 are equally possible for the elements of system 500, except to the extent that they are inconsistent with features of system 500 that differ from features of systems 100, 200, 300, and 400.

[0054] The system 500 includes a stand-alone chiller 550 that supplies coolant serially to the infrastructure module 510 and then to the payload module 520. Coolant exiting the payload module 520 at coolant return 549 returns to the stand-alone chiller 550 to be brought back to the supply temperature and resupplied to the infrastructure module 510 via the coolant supply 546. The stand-alone chiller 550 is thus connected in series along the flow path of the coolant distribution system that extends through the system 500 at a location downstream of the condenser 524 and the payload module 520, but upstream of the heat exchanger 511 and the infrastructure module 510.

[0055] Rather than relying on a building or municipal supply of fluid that may be used as a coolant, the stand-alone chiller 550 is a self-contained cooling system for the coolant used by the system 500. The stand-alone chiller 550 may be, for example, a low-energy or passive chiller, such as a dry cooling tower or another type of dry chiller, and refers to a chiller that may not require a separate cooling liquid to extract heat from the coolant used for the system 500, such as a heat exchanger that dissipates heat from the coolant to the surrounding air. Such passive or low-energy type stand-alone chillers 550 are particularly useful for implementing the system 500 in remote locations where power and other utilities are limited. The infrastructure module 310.

[0056] Although the modular systems 100, 200, 300, 400, 500 each show different variations on a shared concept, other example modular systems may combine these different variations in any manner. For example, the standalone chiller 550 may be used to provide and cool coolant for a system having any number of payload modules and any flow paths between modules, including any of the flow paths of the modular systems 100, 200, 300, 400. The central fluid management system 415 may be replaced with a separate fluid management system of any of the modular systems 100, 200, 300, 500, or any variation of these modular systems. Other example modular systems may include any combination of payload modules connected in series with each other as shown in modular system 100, payload modules that receive coolant that has not passed through an infrastructure module as shown in modular system 200, and payload modules that receive coolant that has passed through an infrastructure module in parallel with other payload modules as shown in modular systems 300 and 400.

[0057] 6 , a process 1000 for storing and cooling electronic equipment comprising any of the above-described systems 100, 200, 300, 400, 500 includes a setup phase 1010, a storage phase 1020, and an operation phase 1030. The process 1000 is described below with reference to elements of system 100 by way of example, however, each reference below to any element of system 100 is equally applicable to the corresponding element of any of systems 200, 300, 400, 500 or any possible variations thereon.

[0058] The setup phase 1010 includes coolant connection 1011 and module interconnection 1012, either of which may be performed before, between or after the other. During coolant connection 1011, the modules 110, 120, 130 are connected to an available distribution system for the fluid coolant. The modules 110, 120, 130 may be connected in parallel or series and in any order. During module interconnection 1012, the modules are connected to each other electrically, electronically or both. For example, the interconnection 1012 may include establishing an electrical connection between bus bars 114, 124, connecting the PLC 112 to the humidity control system 122 and the fluid management system 125, and extending electronic connectors such as wires or cables between the modules 110, 120, 130 through which the computer hardware 113, 123 can electronically communicate with the computer hardware 113, 123 in any other module. The establishment of fluid connections between the modules 110 , 120 , 130 may be considered part of either or both of the coolant connections 1011 and the module interconnects 1012 .

[0059] The setup phase 1011 is followed by a storage phase 1020, which includes infrastructure installation 1021 and soaking 1022. Either infrastructure installation 1021 or soaking 1022 may be performed before, during or after the other. Infrastructure installation 1021 includes placement of infrastructure hardware 113 within infrastructure modules 110 such that infrastructure hardware 113 may operate with power from busbars 114. Soaking 1022 includes soaking payload hardware 123 in an evaporable liquid in the submersion cooling system of payload modules 120, 130. At the end of the storage phase 1020, infrastructure hardware 113 and payload hardware 123 are stored and stored within their respective modules 110, 120, 130, ready to operate and properly cooled.

[0060] The operation phase 1030 follows the storage 1020 and includes cooling 1031 and computing 1032, which may be performed simultaneously. Cooling 1031 includes operating the heat exchanger 1011 to cool the infrastructure hardware 113 and operating the immersion cooling system of one, some or all of the payload modules 120, 130 to cool the payload hardware 123. Cooling 1031 may also include circulating coolant between the modules 110, 120, 130. The computing step 1032 includes operating the infrastructure hardware 113 and the payload hardware 123. Computing 1032 may also include operating any other computing components in any of the modules 110, 120, 130. Thus, operating the PLC 112 to manage the immersion cooling system, such as by the humidity control system 122 and the fluid management system 125, may be considered part of cooling 1031, computing 1032, or both.

[0061] Additions to and variations in process 1000 are possible. For example, the setup phase 1010 may overlap with or follow the installation phase 1020. Also, additional infrastructure installation 1021 and immersion 1022 may occur during or after the operation phase 1030 as hardware 113, 123 is added or replaced in the system 100.

[0062] Although the concepts herein have been described with reference to particular examples, it is to be understood that these examples are merely illustrative of the principles and applications of the inventive concepts. It is thus to be understood that many changes may be made to the illustrative examples and other arrangements may be devised without departing from the spirit and scope of the inventive concepts as defined by the appended claims.

Claims

1. 1. A containment and cooling system for computer hardware, comprising: an infrastructure module configured to store computer hardware and to store a programmable logic controller ("PLC"), the infrastructure module including either or both of a convection air cooling system and a metal plate arrangement connected by one or more conduits for carrying liquid for cooling the computer equipment stored by the infrastructure module; a payload module including an immersion cooling system managed by the PLC and located outside the infrastructure module; The containment and cooling system, wherein the infrastructure module includes an infrastructure bus bar and the payload module includes a payload bus bar electrically connected to the infrastructure bus bar.

2. The system of claim 1 , wherein the infrastructure module includes one or more racks of computing hardware having a first power consumption level.

3. The system of claim 2 , wherein the payload module includes computing hardware having a second power consumption level, the second power consumption level being greater than the first power consumption level.

4. 2. The system of claim 1, wherein the payload module has a dimension of one EIA-310 standard server rack or two or three adjacent EIA-310 standard server racks.

5. The system of claim 1 , wherein the convection air cooling system includes a liquid-to-air heat exchanger.

6. A containment and cooling system for computer hardware, comprising: an infrastructure module configured to store computer hardware and to store a programmable logic controller ("PLC"), the infrastructure module including either or both of a convection air cooling system and a metal plate arrangement connected by one or more conduits for carrying liquid for cooling the computer equipment stored by the infrastructure module; a payload module including an immersion cooling system managed by the PLC and located outside the infrastructure module; the convection air cooling system includes a liquid-to-air heat exchanger; A containment and cooling system, wherein the heat exchanger or conduit of the infrastructure module is fluidly connected to a condenser of the payload module.

7. The system of claim 6 , wherein a flow path of cooling fluid through the containment and cooling system passes through a cooling device of the infrastructure module before reaching the condenser.

8. 8. The system of claim 1, wherein the payload module is a first payload module and the system includes a second payload module including an immersion cooling system, and the immersion cooling systems of the first payload module and the second payload module are managed by the PLC via a central fluid management system housed within the infrastructure module.

9. A containment and cooling system for computer hardware, comprising: an infrastructure module configured to store computer hardware and to store a programmable logic controller ("PLC"), the infrastructure module including either or both of a convection air cooling system and a metal plate arrangement connected by one or more conduits for carrying liquid for cooling the computer equipment stored by the infrastructure module; a payload module including an immersion cooling system managed by the PLC and located outside the infrastructure module; a containment and cooling system, wherein the infrastructure module and the payload module each include a respective bus bar, the bus bar of the payload module connected to the bus bar of the infrastructure module, the bus bar of the infrastructure module being provided with a connector for receiving power from an external power source.

10. A containment and cooling system for computer hardware, comprising: an infrastructure module configured to store computer hardware and to store a programmable logic controller ("PLC"), the infrastructure module including either or both of a convection air cooling system and a metal plate arrangement connected by one or more conduits for carrying liquid for cooling the computer equipment stored by the infrastructure module; a payload module including an immersion cooling system managed by the PLC and located outside the infrastructure module; A storage and cooling system, wherein the payload module is a first payload module, the immersion cooling system of the first payload module includes a first condenser, and the system includes a second payload module, the second payload module includes an immersion cooling system managed by the PLC and includes a second condenser, the first condenser and the second condenser are connected in series along a cooling fluid flow path.

11. The system further comprises: a distribution system for circulating a cooling fluid through the infrastructure modules and the payload modules; 8. The system of claim 1 , further comprising: a dry cooler through which the distribution system passes, the dry cooler configured to exchange heat from the cooling liquid with ambient air.

12. The system of claim 11 , wherein the infrastructure module and the payload module are connected in series to the distribution system.

13. 1. A method of storing and cooling computer hardware, comprising: storing the first computer hardware in an infrastructure module that also stores a PLC; and submerging a second computer hardware in an evaporable liquid in a two-phase submersion cooling system contained by a payload module, the payload module being located outside the infrastructure module, the submersion cooling system being managed by the PLC; 11. A method of storing and cooling computer hardware, wherein the infrastructure module includes an infrastructure bus bar and the payload module includes a payload bus bar electrically connected to the infrastructure bus bar.

14. 14. The method of claim 13, wherein the method includes a step of cooling the first computer hardware by a cooling device included by the infrastructure module, the cooling device of the infrastructure module including at least one of a liquid-to-air heat exchanger and a metal plate device connected by one or more conduits for transporting a liquid.

15. 15. The method of claim 14, wherein the method includes connecting the cooling device of the infrastructure module and a condenser of the two-phase immersion cooling system in parallel to a liquid supply.

16. The method of claim 14 , the method including connecting the cooling device and condenser of the infrastructure module in series to a liquid supply.

17. 17. The method of claim 16, comprising connecting the cooling device and the condenser of the infrastructure module to a liquid supply such that liquid is supplied by the liquid supply to the cooling device of the infrastructure module and from the condenser back to the liquid supply.

18. 18. The method of claim 13, further comprising providing power to the payload module and the second computer hardware through the infrastructure module.

19. 18. The method of any one of claims 13 to 17, wherein the payload module is a first payload module, the method including the step of immersing third computer hardware in an evaporable liquid in a tank stored by a second payload module including a two-phase submersion cooling system, the two-phase submersion cooling system of the second payload module being managed by the PLC.

Images