Computing cabinets and associated computing systems

Abstract

Aspects of the present disclosure relate to a computing cabinet for a computing system and a computing system having one or more computing cabinets. The computing cabinet may include a power plane for power conversion, a computation plane, and a host plane. The computing cabinet disclosed herein may be modular and scalable. The computing cabinet may be configured such that its components are hot-swappable.

Description

[Technical Field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 377,942, filed September 30, 2022, entitled "SYSTEM TRAY AND CABINET FOR COMPUTING SYSTEM," the disclosure of which is incorporated herein by reference in its entirety for all purposes.

[0002] The present disclosure relates generally to computing systems, and more particularly to computing systems having one or more computing cabinets. [Background technology]

[0003] Certain computing systems may be used in and / or be specifically configured for high-performance computing and / or computationally intensive applications, such as neural network training, neural network inference, machine learning, artificial intelligence, complex simulations, etc. In some applications, the computing system may be used to perform neural network training. For example, such neural network training may generate data for a vehicle's (e.g., automobile) autopilot system, other autonomous vehicle functions, or advanced driver assistance system (ADAS) functions.

[0004] In high performance computing systems, high speed connections, good power performance, and high density integration are generally desirable. High performance computing systems may have a large number of components and connections between components. There are technical challenges associated with scaling certain high performance computing systems. Summary of the Invention [Means for solving the problem]

[0005] Each claimed innovation has several aspects, no single one of which is solely responsible for its desirable attributes. Without limiting the scope of the claims, some prominent features of this disclosure will now be discussed briefly.

[0006] In one aspect, the technology described herein relates to a computing cabinet. The computing cabinet may include a first section including a first power plane (layer), a first computation plane configured to receive power from the first power plane and perform computations, and a first host plane in communication with the first computation plane. The computing cabinet may include a second section including a second power plane, a second computation plane configured to receive power from the second power plane and perform computations, and a second host plane in communication with the second computation plane. The second section can operate independently of the first section. The computing cabinet may include a cabinet frame. The first section and the second section are positioned within the cabinet frame.

[0007] In one embodiment, the first section is stacked vertically with the second section.

[0008] In one embodiment, the first compute plane, the first host plane, the second power plane, and the second host plane are positioned between the first power plane and the second power plane.

[0009] In one embodiment, at least a portion of the first power plane is hot-swappable while the first compute plane is operational.

[0010] In one embodiment, the first compute plane is hot-swappable while the second compute plane is operational.

[0011] In one embodiment, the first host plane is hot-swappable while the second compute plane is operational.

[0012] In one embodiment, the computing cabinet may include one or more redundant connections between the first power plane and the first compute plane.

[0013] In one embodiment, the computing cabinet may include an interface including a connection to an external power source and a coolant inlet. The connection to the external power source may be electrically connected to the first power plane and the second power plane. The coolant inlet is in fluid communication with the first compute plane and the second compute plane.

[0014] In one embodiment, the first compute plane includes a compute tray and a plurality of computing tiles positioned on the compute tray, each computing tile of the plurality of computing tiles including a plurality of dies and a cooling solution integrated with the plurality of dies.

[0015] In one embodiment, the first power plane includes a plurality of power trays configured to convert external power into power for the first compute plane.

[0016] In one embodiment, individual power trays of the plurality of power trays are hot-swappable while other power trays of the plurality of power trays are operational.

[0017] In one embodiment, the computing cabinet may include a connector extending from a first computing plane and configured to connect with a computing plane of an adjacent cabinet.

[0018] In one embodiment, the computing cabinet may include a blind cooling connector on a side of the cabinet frame and a blind power connector on a side of the cabinet frame, and the first computing plane is connected to the blind cooling connector and the blind power connector upon insertion into the computing cabinet.

[0019] In one aspect, the technology described herein relates to a computing system including: a first computing cabinet including a first power plane, a first computation plane configured to receive power from the first power plane and perform computations, and a first host plane in communication with the first computation plane; a second computing cabinet including a second power plane, a second computation plane configured to receive power from the second power plane and perform computations, and a second host plane in communication with the second computation plane; the first computation plane can be connected to the second computation plane by a connector extending through a side of the first computing cabinet and a side of the second computing cabinet; and a position of the first computation plane can be aligned with a position of the second computation plane.

[0020] In one embodiment, the connector connecting the first and second computational planes is a blind connector.

[0021] In one embodiment, the first computing cabinet may further include a third computing plane. The second computing cabinet may further include a fourth computing plane. The third computing plane may be connected to the fourth computing plane by a second connector extending through a side of the first computing cabinet and a side of the second computing cabinet. The position of the third computing plane may be aligned with the position of the fourth computing plane.

[0022] In one embodiment, the first computing cabinet further includes a third power plane configured to provide power to the third computing plane and a third host plane in communication with the third computing plane, and the second computing cabinet further includes a fourth power plane configured to provide power to the fourth computing plane and a fourth host plane in communication with the fourth computing plane.

[0023] In one embodiment, the first power plane includes a plurality of power trays, and individual power trays of the plurality of power trays are hot-swappable.

[0024] In one embodiment, the first computing cabinet further includes one or more redundant connections between the first power plane and the first compute plane.

[0025] In one embodiment, at least one of the first power plane and the first host plane is hot-swappable.

[0026] In one embodiment, the first computing cabinet and the second computing cabinet are each independently operable.

[0027] In one embodiment, the computing system further includes a third computing cabinet including a third power plane, a third compute plane configured to receive power from the third power plane and perform computations, and a third host plane in communication with the third compute plane. The first compute plane is connected to the third compute plane by a third connector extending through a second side of the first computing cabinet and a side of the third computing cabinet. The first compute plane, the second compute plane, and the third compute plane are connected.

[0028] In one embodiment, the computing system further includes a third computing cabinet connected to the second computing cabinet only by a connector extending from the second side of the second computing cabinet, the third computing cabinet including a third connector extending from the second side of the third computing cabinet for connecting with a fourth computing cabinet.

[0029] For purposes of summarizing the disclosure, certain aspects, advantages, and novel features of the innovations have been described herein. It should be understood that not all such advantages may necessarily be achieved in accordance with any particular embodiment. Thus, the innovations may be embodied or implemented to achieve or optimize one advantage or group of advantages as taught herein without necessarily achieving other advantages as may be taught or suggested herein. [Brief explanation of the drawings]

[0030] Specific implementations will now be described with reference to the following drawings, which are provided by way of example and not limitation.

[0031] [Figure 1A] FIG. 1 illustrates a computing cabinet for use in a computing system according to one embodiment. [Figure 1B] FIG. 1 illustrates a computing cabinet for use in a computing system according to one embodiment. [Figure 1C] FIG. 1 illustrates a computing cabinet for use in a computing system according to one embodiment.

[0032] [Figure 2A] FIG. 1 is a front view of a connection between two computing cabinets using blind connectors according to one embodiment. [Figure 2B]FIG. 1 is a front view of a connection between two computing cabinets using blind connectors according to one embodiment. [Figure 2C] FIG. 1 is a front view of a connection between two computing cabinets using blind connectors according to one embodiment.

[0033] [Figure 3] 1 is a perspective view of a computing cabinet according to one embodiment.

[0034] [Figure 4] 1 is a perspective view of two computing cabinets of a computing system according to one embodiment.

[0035] [Figure 5A] 1 illustrates a connection between a first compute tray of a first computing cabinet and a second compute tray of a second computing cabinet using a blind connector according to one embodiment. [Figure 5B] 1 illustrates a connection between a first compute tray of a first computing cabinet and a second compute tray of a second computing cabinet using a blind connector according to one embodiment.

[0036] [Figure 6] FIG. 1 illustrates a power supply tray according to one embodiment.

[0037] [Figure 7] FIG. 1 illustrates multiple power supply trays tethered together for active cooling according to one embodiment.

[0038] [Figure 8A] FIG. 1 illustrates an array of power supply trays according to one embodiment. [Figure 8B] FIG. 1 illustrates two arrays of power supply trays positioned within a computing cabinet according to one embodiment. [Figure 8C]FIG. 1 illustrates two arrays of power supply trays positioned within a computing cabinet according to one embodiment.

[0039] [Figure 9] FIG. 1 illustrates a power bus integrated into a cabinet frame according to one embodiment.

[0040] [Figure 10A] FIG. 1 illustrates a computation tray according to one embodiment. [Figure 10B] FIG. 1 illustrates a computation tray according to one embodiment.

[0041] [Figure 11] FIG. 1 illustrates a computing tray positioned vertically relative to a host, according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0042] In the following detailed description of some embodiments, various descriptions of specific embodiments are presented. However, the innovations described herein can be embodied in many different ways, for example, as defined and encompassed by the claims. This description refers to the drawings, in which like reference numbers and / or terminology may indicate identical or functionally similar elements. It will be understood that the elements depicted in the figures are not necessarily drawn to scale. Furthermore, it will be understood that some embodiments may include more elements than shown in the drawings and / or a subset of the elements depicted in the drawings. Furthermore, some embodiments may incorporate any suitable combination of features from two or more drawings.

[0043] As described above, certain computing systems may be used in and / or be specifically configured for high-performance computing and / or computationally intensive applications, such as neural network training, neural network inference, machine learning, artificial intelligence, complex simulations, etc. In some applications, the computing system may be used to perform neural network training. For example, such neural network training may generate data for a vehicle's (e.g., automobile) autopilot system, other autonomous vehicle functions, or advanced driver assistance system (ADAS) functions.

[0044] A particular computing system may include various levels of hierarchy for performing computing tasks. For example, a computing system may include chips, computing tiles each including multiple chips packaged together and integrated with a cooling solution, compute trays including an array of connected computing tiles, power supplies for powering the various components, and computing cabinets each including one or more compute trays and one or more power supplies.

[0045] The present disclosure relates to a new computing system. The computing system described herein may be configured for high-performance computing applications. The computing system described herein may include hot-swappable components that can be removed and / or inserted into the computing system while the computing system is actively powered and operating. For example, the hot-swappable components may include a compute tray and a power supply tray that can be removed and / or replaced from the computing system without powering down the computing system. Each computing cabinet may have a hot-swappable power supply. The computing cabinet may include two half-cabinet sections that operate independently of each other. The compute, power, and host of one half can be hot-swapped without affecting the other half. The cabinet may include blind connections to enable such hot-swapping of components without interacting with power connections to ensure safety during the hot-swap. Blind connections, as described herein, may refer to connections that can be coupled to components without direct user access. The computing systems described herein can continue to operate when various components fail.

[0046] A computing system is disclosed using self-contained computing cabinets. Each computing cabinet may have its own power conversion, host, and compute plane. Each computing cabinet can function fully by itself when it receives power and cooling.

[0047] The computing cabinets disclosed herein are modular and scalable. Each computing cabinet can be connected to an adjacent cabinet without external components. A blind connection is added to one side of each computing cabinet. The opposite side of a computing cabinet can be mated with another computing cabinet to continuously scale the computing system. A computing cabinet can be used alone or joined to one or more other cabinets to scale to form a computational plane as large as needed for the computing task (e.g., training different models).

[0048] The computing cabinets disclosed herein may have power redundancy. Each computing cabinet may incorporate power redundancy in case there is a failed component, which may not affect the functionality of other components of the computing cabinet.

[0049] 1A, 1B, and 1C illustrate a computing cabinet 100 for use in a computing system according to one embodiment. The computing cabinet 100 can operate as a fully functional computing system or can be connected to one or more other computing cabinets to increase the computing power of the computing system. The computing cabinet 100 can be fully functional on its own once it is powered and cooled. The computing cabinet 100 receives power and cooling. The computing cabinet 100 distributes the power and cooling to the various components housed within the structure of the computing cabinet 100.

[0050] 1A, computing cabinet 100 can be divided into first section 101a and second section 101b, each including power plane 104, compute plane 102, and host plane 106. In some embodiments, first section 101a and second section 101b can operate independently of each other. In this manner, components of first section 101a can be hot-swapped without affecting the operation of second section 101b.

[0051] Each power plane 104 receives high-voltage input power and converts that input power to meet the specifications of various other computing cabinet 100 components (e.g., compute trays 112). Each power plane 104 may include an array of power trays 114. Each power tray 114 can operate in coordination with other power trays 114 in the power plane 104 to meet the specifications of a section of the computing cabinet 100, such as the first section 101a or the second section 101b. Thus, if a first power tray 114 fails, the other power trays 114 in the power plane 104 can compensate for the failure before the failed first power tray 114 is replaced. A power tray 114 can be removed and replaced from the power plane 104 without shutting down the power plane 104 and / or disconnecting the power plane 104 from the high-voltage input power.

[0052] Each compute plane 102 includes a compute tray 112 with one or more compute tiles. Each compute tile includes multiple chips or dies packaged together and integrated with one or more cooling solutions. In certain applications, the compute tiles may include a system on a wafer including an array of dies. In some such applications, a cold plate may be integrated with the system on the wafer. The compute plane 102 may be used in and / or specifically configured for high-performance computing and / or computationally intensive applications, such as neural network training, neural network inference, machine learning, artificial intelligence, complex simulations, etc. In some applications, the compute plane 102 may be used to perform neural network training. For example, such neural network training may generate data for a vehicle's (e.g., automobile) autopilot system, other autonomous vehicle functions, or advanced driver assistance system (ADAS) functions. The compute plane 102 may operate independently and / or be connected to compute planes 102 in one or more other computing cabinets 100 to increase the computing power of the computing system. For example, multiple compute planes 102 can be connected to provide a computing system of greater capacity, scaling the compute planes as needed, where the computing system can (1) perform more complex computing tasks (e.g., training more demanding models) and / or (2) perform a greater number of computing tasks in parallel with each other.

[0053] Each host plane 106 includes a host tray 116. The host tray 116 can perform ingest processing for the compute planes 102 in the same section of the computing cabinet, such as the first section 101a or the second section 101b. The host tray 116 can include a Peripheral Component Interconnect Express (PCIe) connection to an interface processor. The host tray 116 can provide video decoder support. In some embodiments, the host tray 116 can operate in an x86 Linux environment.

[0054] 1A, computing cabinet 100 includes a first compute tray 112 positioned vertically above a first host tray 116 and a second compute tray 112 positioned vertically above a second host tray 116. The first host tray 116 may be positioned between the first and second compute trays 112 as shown. Power supply trays 114 are included at the top and bottom of computing cabinet 100 and may be positioned vertically relative to the compute trays 112 and host trays 116.

[0055] FIG. 1B illustrates the computing cabinet 100 of FIG. 1A from a front view, and FIG. 1C illustrates the computing cabinet 100 from a rear view. As shown in FIGS. 1B and 1C, the computing cabinet 100 may include a blind connector 120, a coolant interface 122, a power interface 121, a coolant distribution system 124, and a power bus 126. The power interface 121 may include a connection to external power. The external power source may be a high-voltage power supply with a sufficient power rating to power the computing cabinet 100. The power supply tray 114 may receive power from the power interface 121. The coolant interface 122 may include a connection to an external cooling source. The external cooling source may be a connection to a source of fluid, such as a coolant, that can be distributed throughout the computing cabinet 100 to help maintain a sufficiently low operating temperature.

[0056] The coolant distribution system 124 can carry coolant to and from the interface 122 and distribute the coolant to components of the computing cabinet 100, such as the power tray 114, the compute tray 112, and the host tray 116. The coolant distribution system 124 can also output the coolant from the computing cabinet 100. The coolant distribution system 124 can include one or more hoses, one or more manifolds, one or more coolant connections, the like, or any suitable combination thereof to facilitate the flow of coolant within the computing cabinet 100.

[0057] The power bus 126 may include electrical conductors and / or electrical connection points. The power bus 126 can carry converted power from the power supply trays 114 to other cabinet components, such as the compute trays 112 and host trays 116. The power bus 126 may include redundant electrical connections. For example, a compute tray 112 can connect to a power supply tray 114 via multiple electrical connections. Thus, if a power supply tray 114 fails or is removed, the compute tray 112 can receive power from one or more other power supply trays 114.

[0058] 1C , the coolant distribution system 124 and the power bus 126 can be positioned at the back of the computing cabinet. As described below, various components of the computing cabinet 100 can include blind connectors that allow a user to connect the components to the coolant distribution system 124 and the power bus 126 without making manual connections. For example, when the compute tray 112 is fully inserted into the computing cabinet 100, a blind cooling connector can be inserted into the coolant distribution system 124 and a blind power connector can be inserted into the power bus 126 at the back of the computing cabinet 100.

[0059] The computing cabinet 100 includes a cabinet frame 108. The cabinet frame 108 provides structural support for various components, such as support rails for the compute trays 112, power supply trays 114, and host trays 116. The coolant distribution system 124, power bus 126, and interface 122 may be integrated into the cabinet frame 108. Thus, physical support for the components can be established in a single operation, along with connections to the coolant distribution system 124 and power bus 126. For example, when the compute tray 112 is fully inserted into the computing cabinet 100, the compute tray 112 may be connected to the coolant distribution system 124 and power bus 126 as well as physically coupled to the cabinet frame 108.

[0060] A blind connector 120 can connect a first computing tray 112 of a first computing cabinet 100 to a second computing tray 112 of a second computing cabinet 100, enabling the computing planes 102 of the two computing cabinets 100. In some instances, the length of the connection between the computing trays 112 can contribute to a loss of computing power. In this manner, connected computing trays 112 of adjacent computing cabinets 100 can have increased computing power the closer the connected computing trays 112 are placed. To facilitate closer connections, the blind connector 120 can enable the computing trays 112 of adjacent computing cabinets 100 to be connected blindly and / or without physical access to the blind connector 120. The blind connector 120 is described in more detail below.

[0061] 2A, 2B, and 2C illustrate front views of a connection between two computing cabinets 100 using a blind connector 120 according to one embodiment. FIGS. 2A and 2B show the blind connector 120 spanning the first and second cabinet frames 108a, 108b before a computing tray is positioned within the cabinet frames 108a, 108b. As shown in FIGS. 2A and 2B, there may be minimal space between the first and second cabinet frames 108a, 108b. In some embodiments, there may be no space between the first and second cabinet frames 108a, 108b. Therefore, it may be difficult or impossible to access the blind connector 120 from outside the first or second cabinet frames 108a, 108b. 2A or 2B, the first cabinet frame 108a and the second cabinet frame 108b may include components such as a compute tray 112, a power supply tray 114, and a host tray 116, which occupy most of the space within the cabinet frames 108a and 108b. Therefore, it may be difficult to access the blind connectors 120 inside the first cabinet frame 108a or the second cabinet frame 108b.

[0062] FIG. 2B shows cabinet frames 108a and 108b, each having a computing tray 112 positioned therein. Blind connectors 120 can connect the computing trays 112 between adjacent computing cabinets without direct access to the blind connectors 120, as shown, for example, in FIG. 2C. While FIG. 2C shows one computing tray 112 in each cabinet frame 108a, 108b, two or more computing trays 112 can be included in a single computing cabinet. Examples of computing trays 112 connected using blind connectors 120 are shown in FIGS. 5A and 5B and are described in more detail below.

[0063] FIG. 3 illustrates a perspective view of a single computing cabinet according to one embodiment. FIG. 3 illustrates a perspective view of the second cabinet frame 108b of FIG. 2A, with blind connectors 120 extending from the second cabinet frame 108b. As illustrated in FIG. 3, the blind connectors 120 may include multiple groups of connectors. For example, FIG. 3 illustrates six groups of connectors for the blind connector 120 extending from the cabinet frame 108b. There are two sets of three groups of compensators extending from the cabinet frame 108b in FIG. 3. Each group of connectors can connect to a respective computing tile positioned on the computing tray 112. The blind connector 120 illustrated in FIG. 3 can connect to three computing tiles on the first computing tray 112 and three computing tiles on the second computing tray 112. The number of groups of connectors in the blind connector 120 can vary based on the number of computing tiles connected by the blind connector 120.

[0064] FIG. 4 illustrates a perspective view of two computing cabinets according to one embodiment. FIG. 4 illustrates a perspective view of the first and second cabinet frames 108a, 108b of FIG. 2A, with a blind connector 120 extending from one side of the first cabinet frame 108a into the second cabinet frame 108b and extending from the opposite side of the cabinet frame 108a. Although not shown in FIG. 4, a third cabinet frame 108 may be positioned on the opposite side of the first cabinet frame 108a from the second cabinet frame 108b and connected by the blind connector 120. Any suitable number of computing cabinets 100 may be connected in this manner. Such connections may enable scalable computing systems, along with modular, freestanding cabinets.

[0065] 5A and 5B illustrate connection of a first computing tray 112a and a second computing tray 112b using a blind connector 120 according to one embodiment. Referring to FIG. 5A, the blind connector 120 is shown extending into the first cabinet frame 108a and the second cabinet frame 108b. The blind connector 120 has a first connection interface 502a and a second connection interface 502b. The first connection interface 502a and the second connection interface 502b may each include multiple connectors that can electrically couple to corresponding connectors on the computing tray 112, such as the inter-tray tile connector 1008 in FIG. 10B. The first connection interface 502a and the second connection interface 502b may include an actuator that allows the first connection interface 502a and the second connection interface 502b to toggle between a connected position and a disconnected position.

[0066] When the first connection interface 502a and the second connection interface 502b are in the disconnected position, the blind connector 120 can be positioned outside the installation path of the computing tray 112 in the first cabinet frame 108a and the second cabinet frame 108b so that the computing tray 112 can be inserted into the first cabinet frame 108a and the second cabinet frame 108b. When the first connection interface 502a and the second connection interface 502b are in the connected position, the blind connector 120 can be positioned in the installation path of the computing tray 112 and / or coupled to the computing tray 112 in the first cabinet frame 108a and the second cabinet frame 108b.

[0067] Referring to FIG. 5B, blind connectors 120 are shown extending into the first and second cabinet frames 108a and 108b and coupled to the first and second computing trays 112a and 112b. Certain hardware of the first and second computing trays 112a and 112b, such as the inter-tray tile connectors 1008 of FIG. 10B, is omitted from FIG. 5B. When the first computing tray 112a is initially inserted into the first cabinet frame 108a, the first connection interface 502a may be actuated downward to a disconnected position. When the first computing tray 112a is fully inserted into the first cabinet frame 108a, the first connection interface 502a may be actuated upward to a connected position, coupling the first connection interface 502a to the first computing tray 112a.

[0068] FIG. 6 shows a diagram of a power supply tray 114 according to one embodiment. The power supply tray 114 of FIG. 6 can, for example, implement the power supply tray 114 of FIG. 1A. The power supply tray 114 can include a coolant inlet 602, a coolant outlet 604, and a power supply handle 606 disposed at a first end of the power supply tray 114. The power supply tray 114 can include a blind power connector 610 disposed at a second end of the power supply tray 114, the second end being opposite the first end. The first end can be a front end, and the second end can be a rear end.

[0069] The coolant inlet 602 can connect one or more internal coolant manifolds of the power supply tray 114 to a coolant source, such as the coolant distribution system 124 of FIG. 1C. The internal coolant manifolds can distribute the coolant throughout the power supply tray 114 to actively cool the power supply tray 114. The coolant outlet 604 can be connected to one or more internal coolant manifolds of the power supply tray 114 and discharge the coolant to a coolant destination, such as the coolant distribution system 124 of FIG. 1C.

[0070] The blind power connector 610 can be electrically and physically coupled to a power source and / or can supply power to a power destination. For example, the blind power connector 610 can receive high-voltage input power and provide the input power to the power supply tray 114. The power supply tray 114 can perform power conversion and provide the converted power to the power bus 126 to be used by components of the computing cabinet 100, such as the compute tray 112 and / or the host tray 116. The power supply tray 114 can include various internal electrical components, such as power converters, capacitors, resistors, inductors, transistors, the like, or any suitable combination thereof. In some embodiments, the internal electrical components allow the power supply tray 114 to be inserted into an actively powered computing cabinet 100 without damaging the power supply tray 114 or other components of the computing cabinet 100.

[0071] FIG. 7 illustrates multiple power supply trays 114 connected together for active cooling according to one embodiment. As shown in FIG. 7, coolant can be delivered from the coolant distribution system 124 of FIG. 1C (not shown in FIG. 7) to coolant connectors 702 positioned between the power supply trays 114. In some embodiments, a coolant inlet manifold can be positioned between the power supply trays 114 to deliver coolant from the coolant distribution system 124 positioned behind the two power supply trays 114. In some embodiments, a coolant outlet manifold can be positioned between the power supply trays 114 to discharge coolant to the coolant distribution system 124 positioned behind the two power supply trays 114.

[0072] 7, the coolant connector 702 includes a coolant input connector and a coolant output connector. From the coolant input connector, the coolant may be carried by a coolant inlet hose 704 to the coolant inlet 602 (not shown in FIG. 7) of the power supply tray 114. The exhausted coolant may be carried by a coolant outlet hose 706 from the coolant outlet 604 (not shown in FIG. 7) to the coolant output connector.

[0073] 7 , each of the power supply trays 114 may have a blind power connector 610 positioned on the back of the power supply tray 114. In some embodiments, the blind power connector 610 may be connected to the power bus 126, and in the same action, the coolant connector 702 may be connected to the coolant distribution system 124. For example, when the power supply tray 114 is inserted into the cabinet frame 108, the blind power connector 610 may be connected to the power bus 126, and simultaneously, the coolant connector 702 may be connected to the coolant distribution system 124.

[0074] 8A, 8B, and 8C illustrate a power tray array 800 according to one embodiment. The power tray array 800 can include multiple power trays 114. The power tray array 800 can implement, for example, the power plane 104 of FIG. 1A. As shown in FIG. 8B, the power tray array 800 can be inserted into the cabinet frame 108 for use in the computing cabinet 100. Each power tray 114 in the power tray array 800 can operate independently. For example, a single power tray 114 can fail or be removed from the power tray array 800 without causing the power tray array 800 to fail.

[0075] The array of power trays 800 may include redundant power trays 114. For example, the power specifications of the computing cabinet 100 may be met by operating a subset of the power trays 114 in the array of power trays 800. As shown in FIG. 8C , the cabinet frame 108 may accommodate multiple arrays of power trays 800. For example, a first array of power trays 800 may power a first section of the computing cabinet 100, and a second array of power trays 800 may power a second section of the computing cabinet 100.

[0076] FIG. 9 illustrates a power bus 126 integrated into the cabinet frame 108 according to one embodiment. As described above, the power bus 126 may include electrical conductors and electrical connection points. The power bus 126 may carry alternatively converted power from the power supply trays 114 to other cabinet components, such as the compute trays 112 and host trays 116. The power bus 126 may include one or more redundant electrical connections. For example, a compute tray 112 may be connected to a power supply tray 114 via multiple electrical connections. Thus, if an individual power supply tray 114 (or a group of power supply trays 114) fails or is removed, the compute tray 112 can receive power from the other power supply trays 114.

[0077] The power bus 126 may include multiple connection points that may be configured to mate with power connections from various components, such as the blind power connector 610 of the power tray 114 (as illustrated in FIG. 6) and the blind computing connector 1010 of the computing tray 112 (as illustrated in FIG. 10B).

[0078] 10A and 10B illustrate an exemplary compute tray 112 according to one embodiment. The compute tray 112 may include an array of compute tiles 1002 connected to each other and supported by the compute tray 112. In a particular embodiment, each compute tile 1002 includes a system-on-wafer including an array of dies integrated with a cooling solution (e.g., a cold plate). For example, the system-on-wafer may include 16, 25, 36, or 49 dies arranged to perform computing functions in various applications. Depending on the particular application, the system-on-wafer may be positioned between a cold plate and another cooling component to dissipate heat, remove heat, or reduce the temperature of the components of the compute tile 1002 during operation. The compute tiles 1002 may be referred to as training tiles in neural network training applications. Each compute tile 1002 of the compute tray 112 can operate independently. Thus, if a compute tile 1002 of a compute tray 112 fails and / or is removed from the compute tray 112, the compute tray 112 can continue to operate. Any suitable number of computing tiles 1002 may be connected to each other on a compute tray 112. For example, Figure 10A shows six computing tiles 1002 connected to each other.

[0079] The compute tray 112 may have a high computational capacity. For example, the compute tray 112 may be capable of performing over 50 petaflops (PFLOPS). In certain applications, the compute tray may perform in the range of 50 PFLOPS to 200 PFLOPS.

[0080] The compute tray 112 may include intra-tray signal delivery cables 1004 to facilitate communication between each computing tile 1002 and / or the connectors 1008 of the computing tiles 1002. The intra-tray signal delivery cables 1004 may include one or more redundant connections. For example, the computing tiles 1002 may be connected to each other via multiple intra-tray signal delivery cables 1004. Thus, if an intra-tray signal delivery cable 1004 fails and / or is removed and / or a computing tile 1002 fails and / or is removed, the compute tray 112 can continue to operate.

[0081] In a computing tray 112, adjacent computing tiles 1002 are connected to each other by intra-tray signal delivery cables 1004. If a computing tile 1002 fails, the other computing tiles 1002 on the computing tray 112 can still function. For example, the adjacent computing tiles 1002 can route signals around the failed computing tile 1002 to a functioning computing tile to perform computational tasks and / or to route signals around the failed computing tile 1002.

[0082] 10B , computing tiles 1002 may include connectors 1008 around their edges. The connectors 1008 of the computing tiles 1002 may be connected to blind connectors to connect the computing tiles 1002 of two compute trays 112 to each other. For example, the connectors 1008 may be connected to connection interfaces, such as the first connection interface 502 a and / or the second connection interface 502 b of FIG. 5B , to connect a computing tile 1002 in a first computing cabinet 100 and a second computing tile 1002 in a second computing cabinet 100.

[0083] The compute tray 112 may include compute cooling connectors 1006. Certain compute cooling connectors 1006 may receive and provide coolant to the compute tray 112 to cool components of the compute tray 112, such as the computing tiles 1002. Other compute cooling connectors 1006 may exhaust coolant from the compute tray 112. The compute cooling connectors 1006 may be connected to, for example, the coolant distribution system 124 of FIG. 1C . For example, certain compute cooling connectors 1006 may receive coolant from the coolant distribution system 124, while other compute cooling connectors 1006 exhaust coolant to the coolant distribution system 124. The compute cooling connectors 1006 may be positioned on the back of the compute tray 112 such that the cooling connectors 1006 connect to the coolant distribution system 124 when the compute tray 112 is fully inserted into the computing cabinet 100.

[0084] The compute tray 112 may include a blind compute connector 1010 configured to connect the compute tray 112 to a power source. For example, the blind compute connector 1010 may be inserted into the power bus 126 of FIG. 1C and receive power from the power tray 114. The blind compute connector 1010 may be positioned on the back of the compute tray 112. Thus, the compute tray 112 may be safely inserted into an active power source, such as the power bus 126, when the compute tray 112 is fully inserted into the computing cabinet 100.

[0085] 10B, the compute tray 112 may include a capacitor board 1012. The capacitor board 1012 may enable the compute tray 112 to be safely hot-swapped from a powered computing cabinet 100. For example, the capacitor board 1012 may enable the compute tray 112 to be removed from and / or inserted into an active power source without damaging components of the compute tray 112, such as the computing tiles 1002.

[0086] FIG. 11 illustrates a compute tray 112 positioned above a host tray 116, according to one embodiment. As described above, the host tray 116, alone or in combination with additional compute trays 112 in the compute plane 102, can perform ingest processing for the compute trays 112. The host tray 116 may include a Peripheral Component Interconnect Express (PCIe) connection to an interface processor. The host tray 116 can provide support for a video decoder. In some embodiments, the host tray 116 can operate in an x86 Linux environment. Vertical stacking of the compute trays 112 and host trays 116 can efficiently utilize space, enabling high-density integration with minimal connection lengths between compute trays in adjacent cabinets.

[0087] In certain applications, the modular design of the compute cabinet may allow a first section of the compute cabinet to continue operating while a host tray 116 in a second section of the compute cabinet fails, is repaired, or is otherwise offline. The inclusion of paired compute trays 112 and host trays 116 in a modular compute cabinet design may enable such a feature.

[0088] In some embodiments, multiple host trays 116 can perform ingest processing for the compute plane 102. Thus, a host tray 116 can fail and / or be removed from the computing system and the compute plane 102 can continue to operate. Additionally, the compute plane 102 can be partitioned into multiple operations, with one or more host trays 116 performing ingest processing for each partitioned operation.

[0089] Unless the context clearly dictates otherwise, throughout the specification and claims, words such as "comprise," "comprising," "include," "including," and the like, are to be construed in an inclusive sense, i.e., "including, but not limited to," as opposed to an exclusive or exhaustive sense. The term "coupled," as generally used herein, refers to two or more elements that are directly connected or may be connected by one or more intermediate elements. Similarly, the term "connected," as generally used herein, refers to two or more elements that are directly connected or may be connected by one or more intermediate elements. Furthermore, the words "herein," "above," "hereinafter," and words of similar import, when used in this application, shall refer to this application as a whole and not to particular portions of this application. Where the context permits, words in the above detailed description using the singular or plural may also include the plural or singular, respectively. The word "or" in connection with a list of two or more items covers all of the following interpretations of that word: any of the items in the list, all of the items in the list, and any combination of the items in the list.

[0090] Additionally, conditional language used herein, particularly "can," "could," "might," "may," "eg," "for example," "such as," and the like, unless expressly stated otherwise or understood otherwise within the context in which it is used, is generally intended to convey that certain embodiments include certain features, elements, and / or conditions, while other embodiments do not. Thus, such conditional language is generally not intended to imply that features, elements, and / or conditions are in any way required for one or more embodiments.

[0091] The above description has been given with reference to specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the invention to the precise form described. Many modifications and variations are possible in light of the above teachings. This will enable those skilled in the art to best utilize the techniques and various embodiments with various modifications suitable for various applications.

[0092] Although the present disclosure and examples have been described with reference to the accompanying drawings, various changes and modifications will become apparent to those skilled in the art, and such changes and modifications should be understood to be included within the scope of the present disclosure.

Claims

1. a first section including a first power plane, a first compute plane configured to receive power from the first power plane and perform computations, and a first host plane in communication with the first compute plane; a second section including a second power plane, a second compute plane configured to receive power from the second power plane and perform computations, and a second host plane in communication with the second compute plane, the second section operable independently of the first section; a cabinet frame, the first section and the second section being positioned within the cabinet frame; A computing cabinet comprising:

2. The computing cabinet of claim 1 , wherein the first section is vertically stacked with the second section.

3. 3. The computing cabinet of claim 2, wherein the first compute plane, the first host plane, the second power plane, and the second host plane are positioned between the first power plane and the second power plane.

4. The computing cabinet of claim 1 , wherein at least a portion of the first power plane is hot-swappable while the first compute plane is operational.

5. The computing cabinet of claim 1 , wherein the first computing plane is hot-swappable while the second computing plane is operational.

6. The computing cabinet of claim 1 , wherein the first host plane is hot-swappable while the second compute plane is operational.

7. The computing cabinet of claim 1 , further comprising one or more redundant connections between the first power plane and the first compute plane.

8. 10. The computing cabinet of claim 1, further comprising an interface including a connection to an external power source and a coolant inlet, wherein the connection to the external power source is electrically connected to the first power plane and the second power plane, and the coolant inlet is in fluid communication with the first compute plane and the second compute plane.

9. 10. The computing cabinet of claim 1, wherein the first compute plane includes a compute tray and a plurality of computing tiles positioned on the compute tray, each computing tile of the plurality of computing tiles including a plurality of dies and a cooling solution integrated with the plurality of dies.

10. The computing cabinet of claim 1 , wherein the first power plane includes a plurality of power trays configured to convert external power into power for the first compute plane.

11. The computing cabinet of claim 10 , wherein individual power supply trays of the plurality of power supply trays are hot-swappable while other power supply trays of the plurality of power supply trays are operational.

12. The computing cabinet of claim 1 , further comprising a connector extending from the first computing plane and configured to connect with a computing plane of an adjacent cabinet.

13. 10. The computing cabinet of claim 1, further comprising: a blind cooling connector on a side of the cabinet frame; and a blind power connector on a side of the cabinet frame, the first computing plane connecting to the blind cooling connector and the blind power connector upon insertion into the computing cabinet.

14. a first computing cabinet including a first power plane, a first compute plane configured to receive power from the first power plane and perform computations, and a first host plane in communication with the first compute plane; a second computing cabinet including a second power plane, a second compute plane configured to receive power from the second power plane and perform computations, and a second host plane in communication with the second compute plane; Equipped with a first computing plane connected to the second computing plane by a connector extending through a side of the first computing cabinet and a side of the second computing cabinet, and a position of the first computing plane aligned with a position of the second computing plane.

15. 15. The computing system of claim 14, wherein the connector connecting the first and second computational planes is a blind connector.

16. the first computing cabinet further includes a third computing plane; the second computing cabinet further includes a fourth computing plane; 15. The computing system of claim 14, wherein the third computing plane is connected to the fourth computing plane by a second connector that extends through a side of the first computing cabinet and a side of the second computing cabinet, and wherein a position of the third computing plane is aligned with a position of the fourth computing plane.

17. the first computing cabinet further includes a third power plane configured to provide power to the third computing plane and a third host plane in communication with the third computing plane; 17. The computing system of claim 16, wherein the second computing cabinet further includes a fourth power plane configured to provide power to the fourth compute plane and a fourth host plane in communication with the fourth compute plane.

18. 15. The computing system of claim 14, wherein the first power plane includes a plurality of power trays, and wherein individual power trays of the plurality of power trays are hot-swappable.

19. 15. The computing system of claim 14, wherein the first computing cabinet further comprises one or more redundant connections between the first power plane and the first compute plane.

20. 15. The computing system of claim 14, wherein at least one of the first power plane and the first host plane is hot-swappable.

21. 15. The computing system of claim 14, wherein the first computing cabinet and the second computing cabinet are each independently operable.

22. a third computing cabinet including a third power plane, a third compute plane configured to receive power from the third power plane and perform computations, and a third host plane in communication with the third compute plane; Furthermore, the first computing plane is connected to the third computing plane by a third connector extending through a second side of the first computing cabinet and a side of the third computing cabinet; The computing system of claim 14 , wherein the first computational plane, the second computational plane, and the third computational plane are connected.

23. 15. The computing system of claim 14, further comprising a third computing cabinet connected to the second computing cabinet only by a connector extending from a second side of the second computing cabinet, the third computing cabinet comprising a third connector extending from the second side of the third computing cabinet for connecting with a fourth computing cabinet.

Images