Server architectures for improved computational density

Abstract

Arrangements of graphics processing unit (GPU) servers to take advantage of advanced cooling systems are described herein. GPUs are disposed with a high density of four or more GPUs per rack unit including eight, 16, 32, and 64 GPUs per rack unit. These GPUs may operate at high powers of over 600 W per GPU and may generate thousands or tens of thousands of watts of waste heat. Techniques for disposing of this generated waste heat using immersion cooling are disclosed.

Description

SERVER ARCHITECTURES FOR IMPROVED COMPUTATIONAL DENSITYCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This Application claims the benefit of U.S. Provisional Patent Application 63 / 588,540, filed October 6, 2023, and entitled “Server Architectures for Improved Computational Density,” which is hereby incorporated by reference in its entirety.BACKGROUND

[0002] As feature sizes and transistor sizes have decreased for integrated circuits (ICs) including chips and semiconductor dies, the amount of heat generated by a single chip, such as a microprocessor, has increased. Chips that once were air cooled have evolved to chips needing more heat dissipation than can be provided by air alone. In some cases, immersion cooling of chips in a tank containing a cooling fluid is employed to maintain IC chips at appropriate operating temperatures.

[0003] One type of immersion cooling is two-phase immersion cooling, in which heat from a semiconductor die boils the cooling fluid. The boiling creates a cooling-fluid vapor in the tank, which is condensed by cooling coils back to liquid form. Heat from the semiconductor dies can then be sunk into the liquid-to-gas and gas-to-liquid phase transitions of the cooling fluid. This allows chips to be run at a faster rate and allows for more computationally intensive processes than were previously possible. State-of-the-art graphics processing units (GPUs) have architectures and structures designed for air cooling; however, these designs do not take advantage of the added efficiencies provided by immersion cooling.SUMMARY

[0004] Modem high-efficiency GPU servers may be built with up to eight GPUs tightly interconnected within 2-3 rack units (RU or U). The tight interconnection may enable functions such as Tensor Parallelism Artificial Intelligence (Al) training algorithms for eight GPUs, also referred to as TP-8. A second algorithm may then utilize a large number of additional GPUs as needed, for example up to 256 GPUs.

[0005] However, each of these GPUs can consume hundreds of watts of power in operation, and almost the entirety of this power is converted to waste heat. To prevent this waste heat from raising the temperature of the GPUs or other hardware components above safe operating temperatures, cooling is employed. Modem systems may utilize air cooling, in which ambient (orchilled) air is forced over the computing hardware, removing waste heat through convection. Additionally or alternatively, self-contained two-phase cooling loops may be used in which a heat-generating component such as a central processing unit (CPU) or GPU is in physical contact with a hollow heat sink such as a copper or aluminum. The heat sink may contain a heat transfer fluid such as water, which then boils and turns to vapor from the heat generated by the heatgenerating component. The vapor rises, contacts and transfers heat to a cooler portion of the heat sink (such as a fin), and recondenses, dissipating the excess heat and cooling the heat-generating component. These systems have a drawback of requiring significant amounts of extra space for heat sink fins and other cooling structures, which in turn reduces the available space for computational hardware and reduces the overall computational density of the system.

[0006] The present disclosure is directed towards systems for enabling computing systems for a variety of computational processes including up to 64-GPU Tensor Parallelism, or TP-64, within 1-2U of rack space. Disclosed systems may enable increasing GPU groupings from 8 GPUs to 16, 32, and 48 GPUs in addition to 64 GPUs. The resulting increase in GPU density may further require higher power consumption and waste heat generation that air cooling systems are not able to handle. Immersion cooling systems may provide a solution to traditional heat generation drawbacks and enable higher powers and greater computational performance.

[0007] Additionally, the present disclosure enables multi-vendor chassis by taking advantage of proprietary switches included with small-GPU groups for volumetrically tight Tensor Parallelism networking. Increasing a number of GPUs in a low-latency domain from eight GPUs to 64 GPUs would provide a marked improvement to performance. The present disclosure may eventually enable even larger GPU networks within a single immersion cooling tank, such as 512 GPUs.

[0008] In some aspects, the techniques described herein relate to a computational system, the computational system including an immersion cooling container containing immersion cooling fluid, at least six graphics processing units (GPUs) disposed within a volume including a height equal to or less than 48 mm. The at least six GPUs are further disposed within the immersion cooling fluid. Each of the six GPUs are configured to operate at a power greater than 600 W.

[0009] The volume may further include a width equal to or less than 800 mm and a depth equal to or less than 900 mm.

[0010] In some aspects, the techniques described herein relate to a method for operating a computational system, the method including operating at least six graphics processing units (GPUs) disposed within a volume including a height equal to or less than 48 mm, each of the sixGPUs operating at a power greater than 600 W for a period of time greater than ten minutes. The method further includes removing waste heat generated by the GPUs during the step of operating using an immersion cooling fluid in which the GPUs are immersed.

[0011] The volume may further include a width equal to or less than 800 mm and a depth equal to or less than 900 mm.

[0012] In some aspects, the techniques described herein relate to a method for operating a computational system, the method including operating at least six graphics processing units (GPUs) disposed within a 1U server rack, each of the six GPUs operating at a power greater than 600 W for a period of time greater than ten minutes. The method further includes removing waste heat generated by the GPUs during the step of operating using an immersion cooling fluid in which the GPUs are immersed.

[0013] All combinations of the foregoing concepts and additional concepts discussed in greater detail below (provided such concepts are not mutually inconsistent) are part of the inventive subject matter disclosed herein. In particular, all combinations of claimed subject matter appearing at the end of this disclosure are part of the inventive subject matter disclosed herein. The terminology used herein that also may appear in any disclosure incorporated by reference should be accorded a meaning most consistent with the particular concepts disclosed herein.BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The skilled artisan will understand that the drawings primarily are for illustrative purposes and are not intended to limit the scope of the inventive subject matter described herein. The drawings are not necessarily to scale; in some instances, various aspects of the inventive subject matter disclosed herein may be shown exaggerated or enlarged in the drawings to facilitate an understanding of different features. In the drawings, like reference characters generally refer to like features (e.g., functionally similar and / or structurally similar elements).

[0015] FIG. 1 illustrates a computational system including at least one GPU disposed within a server rack.

[0016] FIG. 2A illustrates a top-down view of a computational system in accordance with the present technology.

[0017] FIG. 2B illustrates a bottom-up view of computing system illustrating a plurality of backplanes.

[0018] FIG. 3 illustrates additional server configurations and communications arrangements in accordance with the present technology.

[0019] FIG. 4 illustrates additional server configurations spanning various heights in accordance with the present technology.

[0020] FIG. 5 illustrates additional GPU and CPU server configurations in accordance with the present technology.

[0021] FIG. 6 illustrates disaggregated server arrangements in accordance with the present technology.

[0022] FIG. 7 depicts aspects of an immersion-cooling system for dissipating heat from one or more semiconductor die packages via immersion cooling.DETAILED DESCRIPTION

[0023] FIG. 1 illustrates a computational system 100 including at least one GPU disposed within a server rack. The system may include one or more GPUs 1 lOa-n, a printed circuit board (PCB) PCB 120 or similar integrated circuit (IC) board on which one or more GPUs 1 lOa-n are disposed, a backplane 130, or a server rack 140. The system may further include and / or be disposed within an immersion cooling container. For example, GPUs HOa-n, PCB 120, backplane 130, and server rack 140 may be disposed within an immersion cooling tank. The immersion cooling container may be at least partially filled with immersion cooling fluid. One or more of GPUs 1 lOa-n, PCB 120, backplane 130, or server rack 140 may be disposed at least partially within immersion cooling fluid contained within the immersion cooling container. An exemplary immersion cooling system that may be used in accordance with the disclosure is described in further detail with respect to FIG. 7.

[0024] The computational system 100 may include one or more GPUs such as GPUs 1 lOa-n where n may be determined in accordance with performance and cooling parameters. Computational system 100 may include one or more GPUs, two or more GPUs, four or more GPUs, six or more GPUs, eight or more GPUs, 16 or more GPUs, 32 or more GPUs, 64 or more GPUs, up to n GPUs, or any suitable number of GPUs. Each GPU of the plurality of GPUs may operate at a predetermined power or within a predetermined power range. For example, each GPU may operate at about 700 W, between about 600 W and about 800 W, between about 500 W and about 1000 W, between about 300 W and about 1500 W, between about 100 W and about 800 W, greater than about 600 W, greater than about 700 W, less than about 700 W, less than about 250W, or any suitable combination, range, or value therein.

[0025] GPUs 1 lOa-n may be disposed on an integrated circuit board, motherboard, or similar printed circuit board such as PCB 120. PCB 120 may include computing or electronic hardware including one or more GPUs, one or more central processing units (CPUs) or similar processors, one or more memory modules, one or more resistors, one or more capacitors, one or more inductors, one or more diodes, one or more traces for connecting one or more components of PCB 120 including GPUs 1 lOa-n, one or more metal-oxide semiconductor field effect transistors (MOSFETs), one or more power components such as buck converters, or any appropriate components.

[0026] Computing system 100 may include backplane 130, which may be a group of electrical connectors for forming a bus such as a communications bus. Backplane 130 may communicatively couple each of GPUs 1 lOa-n to one another and allow for low-latency, high- bandwidth communication between servers and other components of computing system 100. The communicative coupling may be in series, in parallel, or a combination of both series and parallel. Backplane 130 may further connect GPUs 1 lOa-n to additional GPUs in a separate system analogous to computing system 100. For example, an inverted version of computing system 100 may be coupled with computing system 100 in a “back-to-back” manner where a backplane 130 of computing system 100 connects directly to a backplane (or similar switch hardware) of the inverted version of computing system 100, thus providing low-latency, high-bandwidth communication between 12 or more GPUs in two adjacent computing systems. A single backplane 130 may communicatively couple a plurality of computing systems 100. Such a single backplane 130 may provide communicative coupling from each GPU of a plurality of GPUs across plural computing systems 100 to each other GPU of the plurality of GPUs across the plural computing systems 100. Backplane 130 may additionally provide power, local memory, input / output (VO) functionality, or card plug in slots.

[0027] Server rack 140 may have outer dimensions in accordance with one or more standards; for example, the server rack outer dimensions may accord with the Electronic Industries Alliance (EIA) EIA-310-D standard, which defines at least a rack unit (RU or U) as 44.45 mm (1.75”). Alternatively, a rack unit may be defined in accordance with the Open Compute Open Rack standard such that one U is 48 mm (1.89”). A rack unit in accordance with the Open Rack standard may be abbreviated as OU. A U, OU, and / or RU may define a height of a server rack 140. Server rack 140 may additionally have depth and width dimensions.

[0028] A depth dimension of server rack 140 may be about 800 mm, about 1000 mm, about 1200mm, about 1500mm, between about 750 mm and about 950 mm, between about 600 mm and about 1000 mm, between about 500 mm and about 1500 mm, between about 100 mm and about 1000 mm, between about 50 mm and about 2000 mm, greater than about 100 mm, greater than about 250 mm, greater than about 500 mm, greater than about 750 mm, greater than about 800 mm, greater than about 900 mm, greater than about 1000 mm, greater than about 1500 mm, or any suitable depth value or range of depth values.

[0029] A width dimension of server rack 140 may be about 450 mm, about 500 mm, about 550 mm, about 900mm, between about 400 mm and about 600 mm, between about 850 mm and 950 mm, between about 700 mm and about 1100 mm, between about 500 mm and about 1500 mm, between about 200 mm and about 800 mm, between about 100 mm and about 1000 mm, between about 50 mm and about 2000 mm, between about 400 mm and about 700 mm, greater than about 100 mm, greater than about 250 mm, greater than about 400 mm, greater than about 445 mm, greater than about 500 mm, greater than about 550 mm, greater than about 600 mm, greater than about 650 mm, greater than about 750 mm, greater than about 800 mm, greater than about 900 mm, greater than about 1000 mm, greater than about 1500 mm or any suitable width value or range of width values.

[0030] All GPUs of computing system 100 may be disposed entirely within a volume defined by the depth, width, and height of server rack 140. For example, server rack 140 may have a volume defined by a height of about 1U, a depth of about 800 mm, and a width of about 450 mm, GPUs 1 lOa-n may be disposed entirely within this volume. Computing system 100 may include additional GPUs disposed within this volume; for example, eight GPUs, ten GPUs, 16 GPUs, 32 GPUs, or 64 GPUs disposed entirely within this volume. Alternatively, server rack 140 may have a volume defined by a height of about 2U, a depth of about 800 mm, and a width of about 900 mm, and GPUs 1 lOa-n may be disposed entirely within this volume. Computing system 100 may include additional GPUs disposed within this volume; for example, eight GPUs, ten GPUs, 16 GPUs, 32 GPUs, or 64 GPUs disposed entirely within this volume. All GPUs in the foregoing examples may operate substantially simultaneously at powers listed above, such as 700 W or above.

[0031] Power introduced to computing system 100 during operation of GPUs 1 lOa-n may scale linearly with a number of GPUs. For example, each of six GPUs 1 lOa-n may operate at about 700 W within server rack 140 and produce waste heat at a rate of 4,200 W. By comparison, a computing system 100 including 16 GPUs disposed within server rack 140 may produce waste heat at a rate of 10,200 W. 10,200 W may be more waste heat than ambient air alone can dissipatefrom a volume defined by dimensions recited herein while keeping GPUs 1 lOa-n at or under safe operating temperatures. For example, 16 GPUs operating at 700 W within a server rack having a volume defined by a 1U height, 800 mm depth, and 900 mm width may melt or otherwise be catastrophically damaged using traditional server cooling methods such as forced air cooling or two-phase self-contained conductive cooling. Two-phase immersion cooling provides a uniquely effective and efficient mechanism for removing waste heat from high-performance computing systems.

[0032] Computing system 100 may further include head node 150. Head node 150 may perform one or more management functions for computing system 100 including managing task and computing operations sent to GPUs 1 lOa-n, running operating system software for computing system 100 and / or GPUs 1 lOa-n, controlling at least one parameter of one or more of GPUs 110a- n including power, voltage, current, frequency, processing speed, GPU core allocation, GPU processing thread allocation, managing memory allocation or distribution, and similar operations.

[0033] Computing system 100 may further include one or more memory modules such as memory 160. Memory 160 may be communicatively coupled to one or more of GPUs 1 lOa-n, PCB 120, backplane 130, server rack 140, and / or head node 150. Memory 160 may include disaggregated memory modules that may be disposed outside of a volume defined by the boundaries of server rack 140. For example, memory 160 may be disposed within an immersion cooling container, but outside of server rack 140. Alternatively, memory 160 may be disposed within a volume defined by the boundaries of server rack 140. Disposing memory 160 proximal to GPUs 1 lOa-n, PCB 120, backplane 130, and head node 150 may have the advantage of reducing communication cable depths between memory 160 and GPUs 1 lOa-n. This reduction of communication cable depths may improve GPU performance by reducing latency, which may enable very high throughput algorithms such as those used for training large artificial intelligence (Al) models such as large language models or other transformer models.

[0034] Memory 160 may be disposed within an immersion cooling container above a reservoir of immersion cooling liquid, for example in a headspace of an immersion cooling tank. Memory 160 may be communicatively coupled to one or more components of computing system 100 using a cache-coherent interconnect protocol such as the Compute Express Link™ (CXL™) protocol. Memory 160 may be communicatively coupled to one or more components of computing system 100 using a peripheral component interconnect express (PCIe) protocol or one or more fiber optic transmission systems.

[0035] FIG. 2A illustrates a top-down view of a computational system in accordance with thepresent technology. Computing system 200 may include an immersion cooling container 210 such as an immersion cooling tank at least partially filled with immersion cooling fluid 212. Immersion cooling container 210 may be analogous to tank 720 in FIG. 7. Computing system 200 may include one or more GPU servers 220 which may be analogous to computing system 100. For example, each of the one or more GPU servers 220 may include one or more GPUs (for example n GPUs, where n may be an integer number between 1 and 64), one or more PCBs, one or more server racks, one or more CPU head nodes, one or more memory modules, or one or more backplanes. Additionally or alternatively, a backplane 230 may communicatively couple a plurality of GPU servers 220.

[0036] Computing system 200 may include any suitable number of GPU servers 220. Computing system 200 may include 32 GPU servers, 48 GPU servers, 64 GPU servers, more than 20 GPU servers, more than 50 GPU servers, between 24 and 48 GPU servers, between 32 and 64 GPU servers, between 48 and 96 GPU servers, more than 64 GPU servers, or any suitable number of GPU servers 220.

[0037] Switch cards 232 may communicatively couple one or more GPU servers within computing system 200, one or more GPUs within a GPU server, one or more first GPUs to one or more other GPUs or GPU servers outside of the server in which the first GPU is disposed, one or more switch cards 232 to one or more additional switch cards 232, or any suitable combination of GPUs, GPU servers, and / or switch cards.

[0038] Backplane 230 may be analogous to backplane 130 in computing system 100. Backplane 230 may provide a low-latency, high-bandwidth communications bus between one or more GPU servers 220 and / or one or more GPU domains. One or more CPU head nodes disposed within one or more GPU servers 220 may divide tasks between one or more GPU domains, one or more GPU servers 220, and / or one or more GPUs disposed on one or more GPU servers 220. One or more CPU head nodes may utilize backplane 230 to distribute tasks, allocate memory, and control operation of one or more GPUs within computing system 200.

[0039] In an embodiment, one or more one or more GPU servers 220 may have dimensions similar to or the same as server rack 140 in FIG. 1. For example, one or more GPU servers 220 may have outer dimensions in accordance with one or more standards; for example, GPU server outer dimensions may accord with the Electronic Industries Alliance (EIA) EIA-310-D standard, which defines at least a rack unit (RU or U) as 44.45 mm (1.75”). Alternatively, a rack unit may be defined in accordance with the Open Compute Open Rack standard such that one U is 48 mm (1.89”). A rack unit in accordance with the Open Rack standard may be abbreviated as OU. A U,OU, and / or RU may define a height of GPU servers 220. One or more GPU servers 220 may additionally have depth and width dimensions.

[0040] A depth dimension of GPU servers 220 may be about 800 mm, about 1000 mm, about 1200 mm, about 1500mm, between about 750 mm and about 950 mm, between about 600 mm and about 1000 mm, between about 500 mm and about 1500 mm, between about 100 mm and about 1000 mm, between about 50 mm and about 2000 mm, greater than about 100 mm, greater than about 250 mm, greater than about 500 mm, greater than about 750 mm, greater than about 800 mm, greater than about 900 mm, greater than about 1000 mm, greater than about 1500 mm, or any suitable depth value or range of depth values.

[0041] A width dimension of GPU servers 220 may be about 450 mm, about 500 mm, about 550 mm, about 900mm, between about 400 mm and about 600 mm, between about 850 mm and 950 mm, between about 700 mm and about 1100 mm, between about 500 mm and about 1500 mm, between about 200 mm and about 800 mm, between about 100 mm and about 1000 mm, between about 50 mm and about 2000 mm, between about 400 mm and about 700 mm, greater than about 100 mm, greater than about 250 mm, greater than about 400 mm, greater than about 445 mm, greater than about 500 mm, greater than about 550 mm, greater than about 600 mm, greater than about 650 mm, greater than about 750 mm, greater than about 800 mm, greater than about 900 mm, greater than about 1000 mm, greater than about 1500 mm or any suitable width value or range of width values.

[0042] GPUs in computing system 200 may operate at any suitable power level including those recited in reference to GPUs 1 lOa-n in FIG. 1. For example, each GPU in computing system 200 may operate at about 700 W, above about 600 W, between about 500 and 1000 W, or at any suitable operating power. For a given number n of GPUs within each GPU server, a total waste heat generated during operation may equal a power level of each GPU multiplied by the number of n GPUs within each one or more GPU servers 220 multiplied by the number of GPU servers 220. For example, for n=8 GPUs per one or more GPU servers 220, a total waste heat generated during operation of all of the GPUs within computing system 200 may be equal to about 700 W x 8 GPUs x 40 GPU servers for a total of 224 kW of waste heat that would need to be removed from GPU servers 220 during steady-state operation. In an embodiment, computing system 200 may be capable of removing in excess of 600 kW of waste heat from the system during steadystate operation.

[0043] Computing system 200, immersion-cooling system 700 or other immersion cooling systems in accordance with the present disclosure may enable the operation of at least six GPUsdisposed within a volume comprising a height equal to or less than 48 mm, each of the six GPUs operating at a power greater than 600 W for a period of time greater than ten minutes. The volume may further comprise a width equal to or less than 800 mm and a depth equal to or less than 900 mm. In an embodiment, computing system 200, immersion-cooling system 700 or other immersion cooling systems in accordance with the present disclosure may enable the operation of a number of GPUs disposed within a volume comprising a height equal to or less than 48 mm, each of the six GPUs operating at a power greater than 600 W for a period of time greater than ten minutes. The volume may further comprise a width equal to or less than 800 mm and a depth equal to or less than 900 mm. The number of GPUs may include four GPUs, eight GPUs, 16 GPUs, 32 GPUs, 64 GPUs, or a suitable number of GPUs.

[0044] FIG. 2B illustrates a bottom-up view of computing system 200 illustrating a plurality of backplanes 230, each backplane 230 communicatively coupling a plurality of GPU servers 220 (also referred to as GPU domains) in parallel, in series, or a combination of both. Backplane 230 may utilize a cache-coherent interconnect protocol such as the Compute Express Link™ (CXL™) protocol. Backplane 230 may be communicatively coupled to one or more components of computing system 200 using a peripheral component interconnect express (PCIe) protocol or one or more fiber optic transmission systems. Backplane 230 may additionally or alternatively include one or more high-speed PCBs, one or more electrical cables, or one or more optical cables.

[0045] FIG. 3 illustrates additional server configurations in accordance with the present technology. Server arrangement 310 may include a plurality of GPUs, a plurality of switches to communicatively couple the plurality of GPUs, and a head node a plurality of CPUs and network interface card (NIC) for connecting the GPUs to one or more components outside of the server. Server arrangement 320 may be analogous to server arrangement 310; however, the switches may function to communicatively couple the GPUs to components outside of the server instead of functioning primarily to communicatively couple the GPUs to each other. Server arrangement 330 may include a plurality of GPUs, a head node including a plurality of CPUs, and NIC for connecting the GPUs to one or more components outside of the server; unlike server arrangement 310 and server arrangement 320, however, server arrangement 330 may exclude switches and instead directly communicatively couple the plurality of GPUs to each other. An example of a server arrangement analogous to server arrangement 330 may include the Open Compute Project Accelerator Module (OCPAM). Server arrangement 340 may include a plurality of GPUs (for example, 16 GPUs) connected using a plurality of switches (for example, eight switches) disposed on a single board. In an embodiment, server arrangement 340 may be split onto two boards, withan equal number of GPUs and switches disposed on each board.

[0046] FIG. 4 illustrates additional server configurations in accordance with the present technology. Server arrangement 410 may include two servers including a plurality of switches, a plurality of GPUs, a CPU head node including one or more CPUs, and a NIC for communicatively coupling one or more components of the server with one or more external components. In server arrangement 410, two servers may span a total height of 2U or 2OU; however, each may overlap partially with the other. Each of the plurality of switches in each of the two servers may be communicatively coupled with the plurality of switches in the other server. For each server, the plurality of GPUs and switches may be disposed on a PCB or similar integrated circuit board, while a CPU head node (which may include a PCB), including NIC and one or more CPUs, may be disposed above or below a plane passing through the PCB. The two servers may then fit together with connectors or similar attachment mechanisms securing the two servers together. Each of the plurality of GPUs between the two servers may be communicatively coupled with all of the other GPUs in server arrangement 410 through the plurality of switches.

[0047] In server arrangement 420, two servers analogous to one another may include a plurality of switches, a plurality of GPUs, a CPU head node including one or more CPUs, and a NIC for communicatively coupling one or more components of the server with one or more external components. The servers may be substantially identical and connect to one another using a backplane. Connecting identical servers may require the use of one or more re-timers if using a backplane because of the asymmetrical signal path between analogous components. Alternatively, two servers may be mirrored and connected back to back, which has the advantage of not requiring re-timers, but may require two different server designs and manufacturing considerations.

[0048] FIG. 5 illustrates additional server configurations in accordance with the present technology. In server arrangement 510, a GPU server including a plurality of switches, a plurality of GPUs, a CPU head node including one or more CPUs, and a NIC for communicatively coupling one or more components of the server with one or more external components. Server arrangement 510 may include a plurality of GPUs and switches disposed within a height of about 1U or less, about 1OU or less, or a similar value. For each server, the plurality of GPUs and switches may be disposed on a PCB or similar integrated circuit board, while a CPU head node, including NIC and one or more CPUs, may be disposed above or below a plane passing through the PCB. The CPU head node may be disposed within a volume having a height of about 1U or less, about 1OU or less, or a similar value.

[0049] Server arrangement 520 may include a plurality of servers disposed within a volume having a height of about 3U or less, 3 OU or less, or a similar height. Server arrangement 520 may include a plurality of GPU servers analogous to GPU servers in server arrangement 510 and disposed within a volume having a height of about 2U or less, 2OU or less, or a similar height. The plurality of GPU servers may be communicatively coupled using a backplane, direct board- to-board communication, cabling, protocols such as PCIe or CXL, or one or more optical communication mechanisms. Server arrangement 520 may include 16 or more GPUs, with each of the plurality of servers having an equal number of GPUs. GPU servers in server arrangement 520 may include a plurality of switches, for example four switches per GPU server. Server arrangement 520 may include a plurality of CPU head nodes having one or more CPUs and one or more NICs.

[0050] FIG. 6 illustrates disaggregated server arrangements in accordance with the present technology. Server arrangement 610 may include a plurality of servers having at least one CPU and a plurality of GPUs (for example, four GPUs). Each server of the plurality of servers may be communicatively coupled with the other servers in server arrangement 610 by one or more switches disposed in a middle section of the plurality of servers. Dimensions of each server may be in accordance with dimensions recited above. For example, each server of the plurality of servers may have a volume defined by about a 1U height, about a 900 mm depth, and about a 450 mm width.

[0051] Server arrangement 620 may include a plurality of GPU servers (for example, eight GPU servers, each GPU server including eight GPUs) disposed centrally within the server arrangement, a plurality of switches (for example, 32 total switches with 16 switches on two sides of the server arrangement), and a plurality of CPU head nodes (for example, eight CPU head nodes with four CPU head nodes each disposed on two sides of the server arrangement). In an embodiment, the CPU head nodes may be arranged on each side in a 4 by 1 array of CPU head nodes, a 2 by 2 array of GPU head nodes, or any suitable arrangement of CPU head nodes. Dimensions of each server may be in accordance with dimensions recited above. For example, each GPU server of the plurality of servers may have a volume defined by about a 1U height, about a 900 mm depth, and about an 800 mm width. Each CPU head node may be analogous to one or more CPU head nodes recited above with respect to FIGS. 1 and 3-5. Each CPU head node of the plurality of CPU head nodes may have a volume defined by about a 2U height, about a 900 mm depth, and about a 450 mm width.

[0052] FIG. 7 depicts aspects of an immersion-cooling system 700 for dissipating heat from oneor more semiconductor die packages 705 via immersion cooling. Each package 705 can include one or more semiconductor dies that produce heat when the system is in operation. The immersion-cooling system 700 in the illustrated example of FIG. 7 is a two-phase immersioncooling system, though the invention may also be implemented in a single-phase immersioncooling system.

[0053] The immersion-cooling system 700 includes a container such as tank 720 filled, at least in part, with immersion cooling liquid 764. The immersion-cooling system 700 can further include at least one chiller 780 that flows a heat-transfer fluid through at least one condenser coil 770 that is disposed in the tank 720 and headspace 708. Condenser coil 770 and chiller 780 may be part of a heat exchanger. The packages 705 can be mounted on one or more printed circuit boards (PCBs) 757 that are immersed, at least in part, in the immersion cooling liquid 764. Immersion-cooling system 700 may further include a filter 775 disposed adjacent to the tank 720.

[0054] Filter 775 may include a filtration media, a housing, and a pump configured to force immersion cooling liquid 764 through filter 775 to remove contaminants, particulates, or other impurities that may be added to immersion cooling liquid 764 during use. Filter 775 may be housed outside of tank 720 while being in fluidic communication with immersion cooling liquid 764 in tank 720. Alternatively, filter 775 may be submerged within immersion cooling liquid 764 inside of tank 720.

[0055] Immersion cooling liquid 764 may be a hydrocarbon, a fluoroketone, an oil, or a similar dielectric liquid that will act as an insulator while simultaneously transferring heat from package 705 more efficiently than air. An example of immersion cooling liquid 764 is Novec™ 649 produced by 3M™. An exemplary immersion cooling liquid 764 used in accordance with embodiments of the present invention may have a dielectric constant baseline value of about 1.8-2 at a frequency of about 1 kHz.

[0056] In an embodiment of the invention, immersion cooling liquid 764 may be considered unacceptably contaminated if the dielectric constant and / or dielectric loss tangent of immersion cooling fluid being used in an immersion-cooling system 700 differs by a threshold amount as compared to unused or pure immersion cooling liquid 764. For example, immersion cooling liquid 764 may be considered unacceptably contaminated or degraded if the dielectric constant and / or dielectric loss tangent differs by a threshold of 10% or more as compared to unused or pure immersion cooling liquid 764. In an embodiment, a dielectric constant and / or dielectric loss tangent variation threshold may be 20%, 15%, 5%, 3%, 1%, or any suitable threshold.

[0057] Contamination of the immersion cooling liquid 764 and resulting changes to dielectricconstant and / or dielectric loss tangent may alter or negatively impact operation of components within immersion cooling liquid 764 including semiconductor die(s) 750. An altered dielectric constant and / or dielectric loss tangent may result in undesirable cross-talk between components on a PCB, additional noise or reduction in signal strength transmitted along exposed wires of a PCB or semiconductor die(s) 750 submerged in immersion fluid, and / or signal dissipation through the immersion cooling liquid 764. Signal loss may be severe enough that two elements may be effectively represented as being separated by an open circuit despite being physically connected. In an embodiment, a dielectric constant and / or dielectric loss tangent variation threshold may be selected based on an observed or inferred effect on one or more submerged semiconductor die(s) 750. For example, an increase in PCIe bit error rate above an error rate baseline may be correlated with an increase in dielectric constant and / or dielectric loss tangent above a dielectric constant and / or dielectric loss tangent baseline. Accordingly, operation of semiconductor die(s) 750 may be throttled or suspended when a dielectric constant and / or dielectric loss tangent of immersion cooling liquid 764 exceeds a predetermined threshold.

[0058] Changes to dielectric constant and / or dielectric loss tangent may be caused by contaminants within immersion cooling liquid 764. In some cases, changes to dielectric constant and / or dielectric loss tangent may be reversed by filtering the contaminants from immersion cooling liquid 764. In some embodiments, upon detecting an increase in dielectric constant and / or dielectric loss tangent of immersion cooling liquid 764, controller 702 may instruct filter 775 to increase filtration throughput or notify a user that an immersion cooling liquid 764 filtration media may need to be replaced. If a dielectric constant and / or dielectric loss tangent exceeds a predetermined threshold, controller 702 may throttle or shut down one or more semiconductor die(s) 750, generate a notification that immersion cooling liquid 764 should be replaced, trigger an alarm, etc.

[0059] The illustrated example of FIG. 7 is not intended to be to scale. The immersion-cooling system 700 may house and provide immersion cooling liquid 764 to tens, hundreds, or even thousands of packages 705. In some cases, the immersion-cooling system 700 can be small (e.g., the size of a floor unit air conditioner, approximately 1 meter high, 0.5 meter width, 0.5 meter depth or depth). In some implementations, the immersion-cooling system can be large (e.g., the size of a van or larger, approximately 2.5 meters high, 2.5 meters width, 4 meters depth or depth).

[0060] The immersion-cooling system 700 can also include a controller 702 (e.g., a microcontroller, programmable logic controller (PLC), microprocessor, field-programmable gate array, logic circuitry, memory, or some combination thereof) to manage system operation.Controller 702 can perform various system functions such as monitoring temperatures of system components, cooling fluid level, tank access, chiller operation etc. The controller 702 can further issue commands to control system operation such as executing a start-up sequence, executing a shut-down sequence, assigning workloads among the packages, changing cooling fluid level, changing the temperature of the heat-transfer fluid circulated by the chiller 780, etc. In some implementations, the controller 702 can include (or itself be) a baseboard management controller (BMC) 704. That is, the BMC 704 may monitor and control all aspects of system operation for the immersion-cooling system 700 in addition to monitoring and controlling workloads of the semiconductor dies 750 in the packages 705 cooled by the system. The immersion-cooling system 700 can also include a network interface controller (NIC) 703 to allow the system to communicate over a network, such as a local area network or wide area network. The immersioncooling system 700 can further include a fluid sensor array 790 having a plurality of fluid sensors 710. Fluid sensors 710 may include one or more leak detection sensors at least partially submerged in immersion cooling liquid 764.

[0061] The semiconductor die(s) 750 and can be mounted on and attached to a printed circuit board (PCB) 755 (sometimes referred to as a substrate) in device package 705. The package 705 can be made commercially available as an off-the-shelf (OTS) product. The package 705 can be used for single-phase or two-phase immersion cooling of at least one semiconductor die 750, such as a microprocessor (e.g., a central processing unit (CPU) and / or graphics processing unit (GPU)), voltage regulator (VR), high bandwidth memory (HBM), a digital signal processing (DSP) die, an application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), and / or other densely patterned semiconductor die.

[0062] In the two-phase immersion-cooling system 700 of FIG. 7, heat flows from the semiconductor die 750 where it is generated into the heat spreader 752. The heat spreader 752 is in thermal contact with an immersion cooling liquid 764 that can flow over and extract heat from the heat spreader 752. The amount of heat delivered by the heat spreader 752 to the immersion cooling liquid 764 is enough to boil the immersion cooling liquid 764 that contacts the heat spreader 752 (creating bubbles 765 and potentially creating froth 767 when bubbles 765 reach the surface of immersion cooling liquid 764). The vapor 766 from the boiled immersion cooling liquid 764 can be cooled and condensed back to liquid droplets 768, for example, by the condenser coil 770.

[0063] The heat-transfer fluid, such as chilled water, from the chiller 780 can be circulated through the condenser coil 770 to lower the temperature of the condenser coil 770 below thecondensation point in the headspace 708 of the tank 720. As a result, vapor 766 condenses on exterior surfaces of the condenser coil 770 and liquid droplets 768 from the condensed vapor can drip and / or flow back to the immersion cooling liquid 764. Although a single condenser coil 770 is depicted in FIG. 7, there can be a plurality of condenser coils 770 in tank 720 to condense the vapor 766 into droplets. Some or all of the condenser coils 770 may or may not be located directly over the PCBs 757. Instead, the condenser coil(s) 770 can be located near one or more walls of the tank 720, such that the condenser coil(s) 770 are not directly over the PCBs 757 on which the packages 705 are mounted.

[0064] To improve thermal performance in two-phase immersion-cooling system 700, the heat spreader 752 can include a boiling enhancement coating (BEC) on at least one surface. The BEC can be formed from copper or a copper alloy and can be porous, for example, though BECs can take various forms. In some cases, the BEC is a micro porous copper coating having a thickness from approximately or exactly 50 microns to 700 microns thick (which may be produced by electroplating and / or etching). In some implementations, the BEC comprises a mesh copper layer bonded (e.g., via resistance heating) to at least an outer surface of the heat spreader 752. In some cases, the BEC is applied as particulates to at least one smooth surface of the heat spreader 752 and then subsequently sintered to adhere to one another and to the heat spreader 752. The BEC provides an improved surface area to contact the immersion cooling liquid 764 and can increase the heat transfer coefficient from the heat spreader 752 to the immersion cooling liquid 764 by up to a factor of 15 versus a smooth surface on the heat spreader 752. Accordingly, BECs can increase thermal conductivity to, and accelerate the boiling of, the immersion cooling liquid 764.Conclusion

[0065] While various inventive embodiments have been described and illustrated herein, those of ordinary skill in the art will readily envision a variety of other means and / or structures for performing the function and / or obtaining the results and / or one or more of the advantages described herein, and each of such variations and / or modifications is deemed to be within the scope of the inventive embodiments described herein. More generally, those skilled in the art will readily appreciate that all parameters, dimensions, materials, and configurations described herein are meant to be exemplary and that the actual parameters, dimensions, materials, and / or configurations will depend upon the specific application or applications for which the inventive teachings is / are used. Those skilled in the art will recognize or be able to ascertain using no more than routine experimentation, many equivalents to the specific inventive embodiments described herein. It is, therefore, to be understood that the foregoing embodiments are presented by way ofexample only and that, within the scope of the appended claims and equivalents thereto, inventive embodiments may be practiced otherwise than as specifically described and claimed. Inventive embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and / or method described herein. In addition, any combination of two or more such features, systems, articles, materials, kits, and / or methods, if such features, systems, articles, materials, kits, and / or methods are not mutually inconsistent, is included within the inventive scope of the present disclosure.

[0066] Also, various inventive concepts may be embodied as one or more methods, of which an example has been provided. The acts performed as part of the method may be ordered in any suitable way. Accordingly, embodiments may be constructed in which acts are performed in an order different than illustrated, which may include performing some acts simultaneously, even though shown as sequential acts in illustrative embodiments.

[0067] All definitions, as defined and used herein, should be understood to control over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.

[0068] The indefinite articles “a” and “an,” as used herein in the specification and in the claims, unless clearly indicated to the contrary, should be understood to mean “at least one.”

[0069] The phrase “and / or,” as used herein in the specification and in the claims, should be understood to mean “either or both” of the elements so conjoined, i.e., elements that are conjunctively present in some cases and disjunctively present in other cases. Multiple elements listed with “and / or” should be construed in the same fashion, i.e., “one or more” of the elements so conjoined. Other elements may optionally be present other than the elements specifically identified by the “and / or” clause, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, a reference to “A and / or B”, when used in conjunction with open-ended language such as “comprising” can refer, in one embodiment, to A only (optionally including elements other than B); in another embodiment, to B only (optionally including elements other than A); in yet another embodiment, to both A and B (optionally including other elements); etc.

[0070] As used herein in the specification and in the claims, “or” should be understood to have the same meaning as “and / or” as defined above. For example, when separating items in a list, “or” or “and / or” shall be interpreted as being inclusive, i.e., the inclusion of at least one, but also including more than one, of a number or list of elements, and, optionally, additional unlisteditems. Only terms clearly indicated to the contrary, such as “only one of’ or “exactly one of,” or, when used in the claims, “consisting of,” will refer to the inclusion of exactly one element of a number or list of elements. In general, the term “or” as used herein shall only be interpreted as indicating exclusive alternatives (i.e. “one or the other but not both”) when preceded by terms of exclusivity, such as “either,” “one of,” “only one of,” or “exactly one of.” “Consisting essentially of,” when used in the claims, shall have its ordinary meaning as used in the field of patent law.

[0071] As used herein in the specification and in the claims, the phrase “at least one,” in reference to a list of one or more elements, should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but not necessarily including at least one of each and every element specifically listed within the list of elements and not excluding any combinations of elements in the list of elements. This definition also allows that elements may optionally be present other than the elements specifically identified within the list of elements to which the phrase “at least one” refers, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, “at least one of A and B” (or, equivalently, “at least one of A or B,” or, equivalently “at least one of A and / or B”) can refer, in one embodiment, to at least one, optionally including more than one, A, with no B present (and optionally including elements other than B); in another embodiment, to at least one, optionally including more than one, B, with no A present (and optionally including elements other than A); in yet another embodiment, to at least one, optionally including more than one, A, and at least one, optionally including more than one, B (and optionally including other elements); etc.

[0072] In the claims, as well as in the specification above, all transitional phrases such as “comprising,” “including,” “carrying,” “having,” “containing,” “involving,” “holding,” “composed of,” and the like are to be understood to be open-ended, i.e., to mean including but not limited to. Only the transitional phrases “consisting of’ and “consisting essentially of’ shall be closed or semi-closed transitional phrases, respectively, as set forth in the United States Patent Office Manual of Patent Examining Procedures, Section 2111.03.

Claims

CLAIMS1. A computational system, the computational system comprising: an immersion cooling container containing immersion cooling fluid; and a plurality of graphics processing units (GPUs) disposed within a volume having a height equal to or less than 48 mm; wherein: the plurality of GPUs comprises at least six graphics processing units; the plurality of GPUs is disposed within the immersion cooling fluid; and each graphics processing unit within the plurality of GPUs is configured to operate at a power greater than 600 W, such that the plurality of GPUs is configured to operate at a total power greater than 3,600 W.

2. The system of claim 1, further comprising: a backplane disposed within the immersion cooling fluid and communicatively coupled to the plurality of GPUs; and at least one central processing unit (CPU) head node disposed within the immersion cooling container and communicatively coupled to the plurality of GPUs.

3. The system of claim 1, wherein the volume has a width equal to or less than 800 mm and a depth equal to or less than 900 mm.

4. The system of claim 1, further comprising at least one disaggregated memory module disposed within the immersion cooling container and communicatively coupled to a backplane using a cache-coherent interconnect.

5. The system of claim 1, wherein the plurality of GPUs comprises eight GPUs.

6. The system of claim 1, wherein the plurality of GPUs comprises 16 GPUs.

7. The system of claim 1, wherein the plurality of GPUs comprises 32 GPUs.

8. The system of claim 1, wherein the plurality of GPUs comprises 64 GPUs.

9. A method for operating a computational system, the method comprising: operating a plurality of graphics processing units (GPUs) disposed within a volume having a height equal to or less than 48 mm, each graphics processing unit within the plurality of GPUs operating at a power greater than 600 W for a period of time greater than ten minutes, such that the plurality of GPUs operates at a total power greater than 3,600 W for the period of time; and removing, using an immersion cooling fluid in which the GPUs are immersed, waste heat generated by the GPUs during the step of operating; wherein the plurality of GPUs comprises at least six graphics processing units.

10. The method of claim 9, further comprising controlling, by a processor, at least one parameter of the plurality of GPUs using a central processing unit (CPU) head node; wherein the at least one parameter comprises power, voltage, current, frequency, processing speed, GPU core allocation, GPU processing thread allocation, or managing memory allocation or distribution.

11. The method of claim 9, wherein the volume has a width equal to or less than 800 mm and a depth equal to or less than 900 mm.

12. The method of claim 9, wherein the plurality of GPUs comprises eight GPUs.

13. The method of claim 9, wherein the plurality of GPUs comprises 16 GPUs.

14. The method of claim 9, wherein the plurality of GPUs comprises 32 GPUs.

15. The method of claim 9, wherein the plurality of GPUs comprises 64 GPUs.

16. A method for operating a computational system, the method comprising: operating a plurality of graphics processing units (GPUs) disposed within a 1U server rack, each graphics processing unit within the plurality of GPUs operating at a power greater than 600 W for a period of time greater than ten minutes, such that the plurality of GPUs operates at a total power greater than 3,600 W for the period of time; and removing, using an immersion cooling fluid in which the GPUs are immersed, waste heat generated by the GPUs during the step of operating.

17. The method of claim 16, further comprising controlling, by a processor, at least one parameter of the plurality of GPUs using a central processing unit (CPU) head node; wherein the at least one parameter comprises power, voltage, current, frequency, processing speed, GPU core allocation, GPU processing thread allocation, or managing memory allocation or distribution.

18. The method of claim 16, wherein the plurality of GPUs comprises eight GPUs.

19. The method of claim 16, wherein the plurality of GPUs comprises 16 GPUs.

20. The method of claim 16, wherein the plurality of GPUs comprises 32 GPUs.

21. The method of claim 16, wherein the plurality of GPUs comprises 64 GPUs.

Images