Universal Pluggable Data Center Cooling System

Abstract

The invention discloses a universal pluggable data center cooling system, and in particular discloses a data center cooling system, the system comprising a first cooling circuit having a heat exchanger for exchanging heat with a second cooling circuit. The second cooling circuit comprises a cooling distribution unit (CDU) for exchanging heat between the second cooling circuit and a main cooling circuit.

Description

Technical Field

[0001] According to various novel technologies described herein, at least one embodiment relates to a cooling system for a data center. In at least one embodiment, the cooling system includes a first cooling circuit having a heat exchanger for exchanging heat with a second cooling circuit, and the second cooling circuit includes a cooling distribution unit (CDU) that exchanges heat between the second cooling circuit and the main cooling circuit. Background Art

[0002] Data center cooling systems typically use fans to circulate air through server components. Some supercomputers or other high-capacity computers may use water or other cooling systems in addition to air cooling systems to draw heat from server components or racks in the data center to an area outside the data center. The cooling system may include a chiller within the data center area. The area outside the data center may be a cooling tower that receives heated coolant from the data center and recirculates the cooled coolant back to the data center after dissipating the heat into the environment (or external cooling medium) through forced ventilation or other means. In one example, the chiller and cooling tower together form a cooling facility with a pump. Air cooling systems cannot absorb enough heat to support effective or efficient cooling of the data center, while liquid cooling systems can seriously damage server components or racks due to electrical shorts, flooding, or other problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] Various embodiments according to the present disclosure will be described with reference to the accompanying drawings, in which:

[0004] Figure 1 is a block diagram of an example data center having a cooling system that is subject to the improvements described in at least one embodiment;

[0005] Figure 2 is a block diagram of another example data center having an improved cooling system that incorporates a first cooling circuit in addition to the primary cooling circuit and the second cooling circuit described in at least one embodiment;

[0006] Figure 3A , Figure 3B is a combination of Figure 2 Different views of an exemplary rack of at least a portion of the first cooling circuit referenced in;

[0007] Figure 3C , Figure 3D is a detailed view of a manifold and a coupling between the manifold and a tray or plate that can form a Figure 2 at least a portion of the first cooling circuit cited in;

[0008] Figure 4 is a feature diagram illustrating a portion of an internal rack component of an example server tray or an example cooling plate and an example computing component for cooling using an example cooling system according to at least one embodiment;

[0009] Figure 5 According to at least one embodiment, it is possible to use or manufacture Figure 2-Figure 4 A process flow of the steps of the method of cooling the system; and

[0010] Figure 6 An exemplary data center is shown in which data from Figure 2-Figure 5 at least one embodiment of . DETAILED DESCRIPTION

[0011] Due to the high heat demand caused by today's computing components, air cooling efficiency of high-density servers is low and ineffective. Thus, the present disclosure seeks the prospects of liquid coolants (or working cooling fluids) and related systems for cooling computing components such as graphics processing units (GPUs), central processing units (CPUs) or switch components. These computing components are used in servers assembled in server trays on racks in data centers. As technological advances miniaturize computing components, server trays and racks accommodate more and more computing components, so more heat generated by each component needs to be dissipated compared to existing systems. One problem solved in the present disclosure is the lack of versatility that can be applied to requirements in liquid cooling systems, such as determining the cooling fluid or coolant to be used, determining the wetted materials (such as valves, pipes, manifolds, etc.) to achieve liquid cooling, and determining the required pipe locations for liquid cooling systems in servers, racks, and data centers. These requirements complicate the design, maintainability, and reliability of liquid-cooled computing components and electrical components in data centers.

[0012] The present disclosure solves the above and other problems in a fully internal liquid cooling system for electrical and computing components of a liquid-cooled data center. In at least one embodiment, the present disclosure implements a universally integrated pluggable mezzanine module that can be securely built to the desired rack or server tray specifications and can be a single module that can contain a universal coolant. The present disclosure provides examples of connections, manifolds, piping, and heat transfer to implement a universally integrated pluggable mezzanine module that operates between at least one existing cooling circuit with an additional cooling circuit, maintaining the additional cooling circuit even when computing or electrical components in a server tray or rack are changed. In one example, the additional cooling circuit is a universal cooling circuit with a universal coolant that is suitable for operating under a combination of different temperature dissipation requirements (e.g., requirements specified by a component manufacturer), and the additional cooling circuit incorporates a heat exchanger to transfer heat to at least one existing cooling circuit, which in turn utilizes a cooling distribution unit (CDU) to transfer heat to a circuit that exits the data center to a cooling tower. Single-phase or multi-phase cold plates can further be used in conjunction with couplers, or pluggable mezzanine modules implemented by couplers on the modules can be used or replaced. The couplers enable the pluggable cold plate headers to transfer cooling liquid or coolant to heat transfer elements attached to liquid-cooled electronic components, for example. This additional cooling loop is connected to the rack through an indirect contact heat exchanger, or can be located within the rack or server tray.

[0013] In at least one embodiment, the present disclosure implements a cooling system in which there is no visible liquid cooling tube. Instead, a flexible tube can be integrated into a pluggable liquid cooling unit. A snap-in connection is provided by a rack cooling manifold to couple with a row or room manifold. In addition, a similar snap-in connection can be provided on a server tray to couple to a rack cooling manifold and provide coolant to computing components such as GPUs, CPUs, switches or other heat slugs. The server tray and / or rack cooling manifold can be implemented as part of an additional cooling loop (or a common cooling loop) containing a common coolant. The server tray can be formed as a mezzanine module, to which various liquid cooling heat dissipation modules are attached to further dissipate heat from computing components and electronic components. Since the coolant is located inside a pluggable module (one or more of the server tray and the rack cooling manifold), and the pluggable module is part of a common cooling loop (closed loop), heat exchange from the common cooling loop to a second cooling loop can be achieved by an indirect contact heat exchanger. Such a system eliminates the need for system specific wetted materials.

[0014] Figure 11 is a block diagram of an example data center 100 having a cooling system that has undergone the described improvements in at least one embodiment. The data center 100 can be one or more computer rooms 102 having racks 110 and auxiliary equipment to accommodate one or more servers on one or more server trays. The data center 100 is supported by a cooling tower 104 located outside the data center 100. The cooling tower 104 dissipates heat from within the data center 100 by acting on a primary cooling loop 106. In addition, a cooling distribution unit (CDU) 112 is used between the primary cooling loop 106 and the secondary cooling loop 108 to enable heat to be extracted from the secondary cooling loop 108 to the primary cooling loop 106. In one aspect, the secondary cooling loop 108 can connect various pipes to the server tray as needed. The loops 106, 108 are shown as line diagrams, but ordinary technicians will recognize that one or more pipe features can be used. In one case, flexible polyvinyl chloride (PVC) pipes can be used with associated pipes to move fluids along them in each loop 106, 108. Other flexible hose materials may include at least one or more of polytetrafluoroethylene (PTFE), ethylene propylene diene monomer (EPDM), nitrile rubber (NBR), fluorinated ethylene propylene (FEP), fluoroelastomer (FKM), polyurethane, and nylon. For example, one or more coolant pumps may be used to maintain a pressure differential within the circuits 106, 108 to enable coolant movement.

[0015] In at least one embodiment, the coolant in the primary cooling loop 106 and the secondary cooling loop 108 can be one or more of the following: a dielectric fluid alone (e.g., without water for the purposes of this disclosure) or water combined with an additive containing at least one dielectric fluid, such as one or more of deionized water, ethylene glycol, and propylene glycol. In at least one embodiment, in operation, each of the primary cooling loop and the secondary cooling loop has its own coolant. In one aspect, the coolant in the secondary cooling loop can be specific to the components in the server tray or rack 110. Therefore, when the component is switched out of the rack 110, the coolant in the secondary cooling loop 108 may need to be changed. The CDU 112 is capable of precise control of the coolant in the loops 106, 108 independently or simultaneously. For example, the CDU may be adapted to control the flow rate so as to appropriately distribute the coolant to extract the heat generated within the rack 110. In addition, a more flexible tube 114 is provided from the secondary cooling loop 108 to enter each server tray and provide coolant to the electrical components and / or computing components. In the present disclosure, electrical and / or computing components are used interchangeably to refer to heat generating components that benefit from the present data center cooling system. The tube 118 that forms part of the second cooling loop 108 can be referred to as a room manifold. Separately, the tube 116 extending from the tube 118 can also be part of the second cooling loop 108, but it can be referred to as a row manifold. The tube 114 enters the rack as part of the second cooling loop 108, but it can be referred to as a rack cooling manifold. In addition, the row manifold 116 extends to all racks along the rows in the data center 100. The piping of the second cooling loop 108 including the manifolds 118, 116 and 114 can be improved by at least one embodiment of the present disclosure. An optional chiller 120 can be provided in the main cooling loop within the data center 100 to support cooling before the cooling tower. To the extent that there are additional loops in the main control loop, a person of ordinary skill in the art will recognize upon reading the present disclosure that the additional loop provides cooling outside the rack and outside the second cooling loop; and can be used with the main cooling loop of the present disclosure.

[0016] In at least one embodiment, in operation, heat generated within the server trays of the racks 110 can be transferred through the flexible tubes of the discharge manifold 114 of the second cooling loop 108 to the coolant exiting the racks 110. The second coolant from the CDU 112 (in the second cooling loop 108) used to cool the racks 110 moves toward the racks 110. The second coolant from the CDU 112 flows from one side of the room manifold with tubes 118 to one side of the racks 110 via the discharge manifold 116 and passes through one side of the server trays through the tubes 114. The used second coolant (or the outgoing second coolant carrying heat from the computing components) flows out of the other side of the server trays (e.g., entering the left side of the rack and flowing out of the right side of the rack of the server trays after circulating through the server trays or components on the server trays). The used second coolant exiting the server trays or racks 110 comes out of a different side (e.g., the outlet side) of the tubes 114 and moves to the parallel but also outlet side of the discharge manifold 116. From the exhaust manifold 116, the spent second coolant moves in a parallel portion of the room manifold 118, into a direction opposite to the incoming second coolant (which may also be refreshed second coolant), and toward the CDU 112. In the CDU 112, in at least one embodiment, the spent second coolant exchanges its heat with the primary coolant in the primary cooling loop 106. The spent second coolant is refreshed (e.g., relatively cooled compared to the temperature of the spent second coolant stage) and is ready to circulate back through the secondary cooling loop 108 to the computing components. Various flow and temperature control features in the CDU 112 enable control of the heat exchanged from the spent second coolant, as well as control of the flow of refreshed second coolant into and out of the CDU 112. The CDU 112 is also capable of controlling the flow of the primary coolant in the primary cooling loop 106. In this way, the refreshed second coolant may or may not be fully cooled to its default temperature characteristics before being circulated to the racks 110.

[0017] Figure 2200 having an improved cooling system incorporating a first cooling circuit in addition to a main cooling circuit 206 and a second cooling circuit 208, as described in at least one embodiment. Further, the first cooling circuit may have a common coolant that is different from the coolant used in the main cooling circuit and the second cooling circuit. In one example, the coolant in the first cooling circuit may be water or a dielectric fluid that is different from the dielectric fluids of the main cooling circuit and the second cooling circuit. The different reference numerals AC for the first cooling circuits are intended to provide different embodiments that can be combined in any manner understood by a person of ordinary skill in the art reading this disclosure. For example, multiple first cooling circuits may be provided on the second cooling circuit and the main cooling circuit. Alternatively, only one of the three different first cooling circuits may be used at any point in time.

[0018] In at least one embodiment, the first cooling circuit includes a heat exchanger. Heat exchanger 218A; 218B; 218C is a passive heat exchange component that may not be suitable for controlling the flow and temperature of the coolant. Although the heat exchanger may include a pump, it has no intelligent control features. Unless otherwise stated, references to heat exchangers in this disclosure are references to passive heat exchangers. In different embodiments, the heat exchanger may be equipped with limited intelligent control, such as a variable outlet pump; in this case, the alternative heat exchanger is different from a passive heat exchanger and can be applied in certain embodiments where an active heat exchanger is required.

[0019] In at least one embodiment, the first cooling loop includes a heat exchanger for efficient and effective operation, which supports the versatility of the present disclosure. As previously described, in at least one different embodiment, in addition to the heat exchanger (or passive heat exchanger), the first cooling loop may include an active heat exchanger for different operations, not just the versatility provided by the passive heat exchanger. However, application requirements and understanding of the versatility requirements can enable a choice between the two. Compared to the heat exchanger, whether active or passive, the CDU 212 is capable of controlling and monitoring the flow and temperature of the coolant between the second cooling loop 208 and the main cooling loop 206. For example, whether active or passive, the heat exchanger can be distinguished from the CDU by the fact that the heat exchanger cannot be used to perform all the functions of the CDU in the cooling loop other than the first cooling loop. In addition, the first cooling loop can be located at the computer room level (e.g., via the computer room manifold 216A), at the row level (e.g., the row manifold 216B), or at the rack or server level via the tube 214 or the manifold 216C.

[0020] In at least one embodiment, as in Figure 1In the case of the data center 100 of FIG. 1 , the data center 200 may be one or more computer rooms 202 having racks 210 and auxiliary equipment to accommodate one or more servers on one or more server trays. The data center 200 is supported by a cooling tower 204 (or cooling facility) located outside the data center 200. The cooling tower 204 dissipates heat from within the data center 200 by acting on a primary cooling loop 206. In addition, a cooling distribution unit (CDU) 212 is used between the primary cooling loop 206 and the secondary cooling loop 208 to enable heat to be extracted from the secondary cooling loop 208 to the primary cooling loop 206. The secondary cooling loop 208 is able to access various pipes to interface with at least one heat exchanger 218A; 218B; 218C of the first cooling loop. In one aspect, the first cooling loop accesses the pipe system as needed all the way to the server tray.

[0021] In at least one embodiment, in operation, heat generated within the server trays of the racks 210 can be transferred through the flexible ducts 214 or manifolds 216C to the coolant exiting the racks 210 to the room manifold 216A of the first cooling circuit. Suitably, the coolant from the CDU 212 moves toward the racks 210. The coolant from the CDU 212 is transferred from one side of the room manifold (second cooling circuit) 208 to the room manifold 216A of the first cooling circuit via the heat exchanger 218A. In a different aspect, if the first cooling circuit is at the row level, the coolant from the CDU 212 is transferred from one side of the room manifold 208, 216A to the row manifold 216B of the first cooling circuit via the heat exchanger 218B. On the other hand, if the first cooling loop is at the rack level, the coolant from the CDU 212 is transferred from the room manifold 208, 216A and one side of the row manifold 216B to the manifold 216C of the first cooling loop via the heat exchanger 218C. On the other hand, the heat exchanger 218C at the rack level can be positioned as a mezzanine module or located at other locations within the rack. Possible locations for mounting the heat exchanger 218C within the rack are as follows: Figure 3A -D discussion and diagrams to support this.

[0022] In at least one embodiment, further in operation, the coolant in tube 208 (of the second cooling circuit) exchanges heat at heat exchanger 218A (the first cooling circuit at the room level), at heat exchanger 218B (the first cooling circuit at the row level), or at heat exchanger 218C (the first cooling circuit at the rack level). As previously described, the first coolant or common coolant then circulates to the server tray in the first cooling loop at the room, row, or rack level. In one example, the common coolant of the first cooling circuit moves through manifold 216A at the room level of the first cooling circuit at the room level. The common coolant continues to pass through to rack 210 via row manifold 216B (still assuming the first cooling circuit at the room level). In addition, the common coolant moves from row manifold 216B to one side of the server tray via tube 214.

[0023] In at least one embodiment, the spent common coolant (or the exiting common coolant carrying heat from the computing components) flows out the other side of the server tray (e.g., entering the left side of the rack and exiting the right side of the rack of the server tray after circulating through the server tray or through the components on the server tray). The spent common coolant exiting the server tray or rack 210 exits from a different side (e.g., the outlet side) of the tube 214 and moves to a parallel but still exiting side of the manifold 216C. Although reference is made to the spent common coolant, one or ordinary skill will appreciate that the spent common coolant may still be able to exit from other components as it flows through the rack. The used common coolant from manifold 216C moves in a parallel section of room manifold 216A in the opposite direction to the incoming coolant and moves toward heat exchanger 218A to exchange its heat with the second cooling circuit 208. Once the heat from the used common coolant is transferred to the second coolant of the second cooling circuit 208, the common coolant is refreshed (at least cooled relative to the existing common coolant temperature) and recirculated in the room-level first cooling circuit to the server trays.

[0024] However, in at least one embodiment, the second coolant of the second cooling loop 208 is heated and considered as used second coolant when it moves to the CDU 112. This used second coolant of the second cooling loop 208 exchanges its heat with the main coolant of the main cooling loop 206. Thus, the used second coolant is refreshed (e.g., cooled relative to the temperature at the used second coolant stage) and is ready to circulate back through the second cooling loop 208 to reach the heat exchanger 218A (or the heat exchanger 218B in terms of the first cooling loop at the row level, or the heat exchanger 218C in terms of the first cooling loop at the rack level). Various flow and temperature control features in the CDU 212 enable control of the amount of heat exchanged from the used second coolant or the rate at which heat is exchanged (e.g., by controlling the flow rate of the used second coolant relative to the flow rate of the main coolant used to cool the used second coolant). In addition, the CDU 212 can also control the movement of the refreshed second coolant in and out of the CDU 212 to dissipate heat more slowly or more quickly from the second coolant. Alternatively or concurrently, in at least one embodiment, it is possible to control the movement and temperature of the secondary coolant by controlling the flow of the primary coolant of the primary cooling loop 206. For example, faster circulation of the primary coolant extracts more heat from the spent secondary coolant of the secondary cooling loop 208. In this way, the refreshed secondary coolant may or may not be fully cooled to its default temperature characteristics before being circulated to the racks 210.

[0025] Figure 3A , Figure 3B are different views of exemplary racks 300A, 300B incorporating Figure 2 At least a portion of the first cooling loop referenced in . In at least one embodiment, in the case of a first cooling loop at a room level, a row level, or a rack level, the first coolant may circulate all the way to the server tray or rack before existing as a used first coolant. However, because the first cooling loop utilizes a common coolant, it may be necessary to determine the various computing components of each rack at a broad level to determine the appropriate topology of the cooling system to be applied. For example, if a data center room has similar components in all racks, a room-level first cooling system may be appropriate. When a data center room has similar components through a row of racks, a rack-level first cooling loop may be appropriate. When a data center room has similar components in at least a rack, a rack-level first cooling loop may be appropriate.

[0026] In one example, Figure 3A Rack 300A and Figure 3BThe rack 300A and rack 300B may be front and rear views of the same rack, respectively. However, to illustrate that different types of racks are expected to work with the present disclosure, racks 300A, 300B may not show all of the same features in the rear and front views. For example, mezzanine module 302 is not shown in the rear view rack 300B. The mezzanine module may include a heat exchanger (e.g., heat exchanger 218C). Figure 3A A front door 306 for rack 300A is shown, but the rack may not require a door. Alternatively, in order to maintain a constant or appropriate reference temperature, aspects of the present disclosure may use doors on both the front and rear ends of the rack to maintain the cooling effect. For example, the racks 300A, 300B may be completely enclosed, or may be a frame structure that is stable enough to support various inserted trays or boards. In one example, a track or guide 316 is provided to receive the tray or board. The first cooling circuit may include a circuit within the rack 300A, 300B. In one example, a first main coupler 308 is disposed on the first rack cooling manifold 304A at the rear of the rack 300A to receive a first coolant; and a second main coupler 310 is disposed on the second rack cooling manifold 304B to return the first coolant after the first coolant has been used (e.g., after absorbing heat from the computing components of the server trays in the racks 300A, 300B).

[0027] In at least one embodiment, Figure 2 As shown, even if the rack cooling manifold is located externally (e.g., shown as having a plurality of tubes 214 with an inlet for refreshed or cooled coolant and an outlet for spent coolant), the rack cooling manifold can be internal to the rack. Example racks 300A, 300B show such internal rack cooling manifolds. The present disclosure is applicable to flexible tubes 214 as well as Figure 3A , Figure 3B The first coolant travels through the rack cooling manifolds 304A (side rack cooling manifold), 304D (lower rack cooling manifold), and 304B (side rack cooling manifold). The first coolant may exit the rack via a second main coupler 310 at the rear of the racks 300A, 300B. Additionally, other paths may be provided, including an upper rack cooling manifold 304C.

[0028] In at least one embodiment, a mezzanine module 302 can be provided to slide within a rail or guide 316 at the topmost or any other height of the rack 300A, 300B. The mezzanine module can include a cold plate, or can include other couplers to distribute the first coolant within the rack 300A, 300B. In at least one embodiment, the mezzanine module is the only module in the rack that receives the first coolant. In such an embodiment, the mezzanine module and its cold plate can be adapted to extract heat from the entire rack and transfer the heat out of the rack via the first coolant. In addition, the first coupler 312 shown along the side rack cooling manifold 304A serves as a coolant inlet valve for the server tray coupled thereto. The second coupler 314 shown along the side rack cooling manifold 304B, once coupled to the valve, serves as an outlet valve for the server tray. Thus, in operation as part of a first cooling loop, a first or common coolant enters the racks 300A, 300B via the first main coupler 308 , passes through the connected server trays, bottom rack cooling manifold 304D and / or mezzanine modules 302 , and exits via the second main coupler 310 .

[0029] Figure 3C , Figure 3D According to at least one embodiment, a manifold 300C is formed Figure 2 Detail of the coupling 300D between the manifold and the tray or plate of at least a portion of the first cooling circuit referenced in FIG. The manifold 300C may be in Figure 3A , 3B The side rack, lower rack or upper rack cooling manifolds shown in the exemplary racks 300A, 300B of FIG. Each of the manifolds 304A, 304B shown may include a distribution (as an inlet or outlet) coupler 312, 314. The main couplers 308A, 310A also refer to Figure 3A , Figure 3B The couplers 308, 310 of the exemplary racks 300A, 300B are shown. The main couplers 308A, 310A are shown as threaded couplers, but may be press-fit couplers, similar to Figure 3D The distribution couplers 312, 314 can be press-fitted with mating couplers of a server tray or board. In at least one embodiment, the manifolds 304A, 304B are secured to Figure 3A , Figure 3B The rear of the exemplary rack 300A, 300B. Figure 3A , Figure 3BAs shown in the exemplary racks 300A and 300B, the manifold is also adapted to remain in place when a tray or plate is inserted along a track or guide. When the matching coupler of the tray or plate is pressed against the distribution couplers 312 and 314, a leak-proof press fit is obtained. Then, the first cooling circuit enables the passage of the first or universal coolant from the main coupler inlet to reach the distribution coupler inlet 312, pass through the server or tray that is press-fitted with a corresponding one of the distribution coupler inlets 312, come out of the distribution coupler outlet 314, and come out of the main coupler outlet. In addition, the main coupler outlet is shown as having a side coupler, which can be used to provide a connection to an adjacent rack or to provide a connection to the main coupler inlet via an upper rack cooling manifold. For example, an additional rack cooling manifold can be used to prevent pressure from forming in the first cooling circuit.

[0030] Figure 3D A coupling 300D between a manifold and a tray or plate is shown, which may form a Figure 2 at least a portion of the first cooling circuit referenced in . Couplers 308A, 310A, 312, 314, 318 may include built-in check features. This enables one-way flow of fluid so that no leakage occurs, for example, when coupling 300D is removed. This also enables coupling 300D to allow the first or common coolant to flow immediately from the row manifold into the rack cooling manifold and further into the server tray, cooling plate, or mezzanine module. Alternatively, the coupler includes a shut-off valve that can be actuated to shut off the flow before the tray or plate is removed from the rack. In at least one embodiment, Figure 3D Manifolds 304B, 304B are shown with a main inlet coupler 308B and a main outlet coupler 310B. However, the main couplers 308B, 310B differ from the main couplers 308A, 310A in that they have slightly different requirements than a press fit. For example, the main couplers 308B, 310B may require a twist fit because these couplers receive coolant from an existing piping system. In this way, the present disclosure is able to modify existing racks to function within an existing data center architecture.

[0031] Figure 3DAlso shown is a matching coupler 318 for the tray, cooling plate or mezzanine module 302. The tray, cooling plate or mezzanine module 302 may include a narrow enclosed channel 317 to convey fluid from the inlet manifold 304A in a loop, from the inlet matching coupler 318A to the outlet matching coupler 318B, and finally out through the outlet manifold 304B. The enclosed channel achieves the drip-free and quick-connect function disclosed in the present disclosure through a press-fit connection. When the server tray is ready for use with the rack of the present disclosure, the server tray is aligned in the previously referenced rails or guides of the rack and pushed into the rack. The matching couplers 318A, 318B are also aligned with the inlet and outlet distribution couplers 312, 314 of the rack cooling manifolds 304A, 304B. Internal rubber or silicone seals and ribs (shown as dashed lines in mating couplers 318A, 318B) can lock behind notches on the press-fit inlet and outlet distribution couplers 312, 314. The check valves associated with each of the distribution coupler and mating coupler are opened and coolant flows into the server tray.

[0032] Figure 4 4 is a feature diagram showing a portion of an internal rack assembly 400 according to at least one embodiment, the internal rack assembly 400 including an example server tray or an example cooling plate and an example computing component for cooling the computing component using an example cooling system. In at least one embodiment, the computing component can be a circuit board or assembly with a connector 426 or a single computing component 416, such as a GPU. The circuit board 412 may include an area 428 for additional computing components such as one or more GPUs or CPUs 416. In addition, the example server tray or example cooling plate includes one or more of a bottom 402 and a top 432. When the circuit components are selected to be liquid cooled, inlet and outlet couplers 422, 424 may be present on the additional computing component 416. The inlet and outlet couplers 422, 424 enable the distribution of coolant from the first cooling circuit via the inlet and outlet matching couplers 408, 406 of the server tray or cooling plate. The additional computing component can be inserted and secured to the circuit board or assembly in the provided area 428. The circuit board or assembly itself may include an inlet coupler 420 and an outlet coupler 418 for receiving coolant and returning coolant from the first cooling loop. Thus, an additional computing component 416 may be coupled to a matching coupler 408, 406, or may be coupled to an inlet or outlet coupler 420, 418 of the circuit board or assembly.

[0033] In at least one embodiment, a server tray or cooling plate at the bottom portion 402 or top portion 432 may include enclosed channels 404, 436 (in the Figure 3D436 is coupled to the rack cooling manifold via server tray couplers (or matching couplers) 442, 434. In addition, the closed channel 436 can be a tube in the server tray coupled to the rack cooling manifold and coupled to the computing component couplers 422, 424 via server couplers 438, 440. The closed channel can be only on the top portion 432 and can be supported by the fan 430 on the top portion instead of the bottom portion 402, but can also exist on both portions. The closed channel in any part of the server tray or the cold plate can provide cooling without further providing ducts to the computing component or additional computing component via a coupler on the top portion directly coupled to the coupler on the computing component or additional computing component. However, both the closed channel and the further coupling to the computing and additional computing components can exist at the same time. When the tray is a cold plate, a fan 430 can be present. The fan 430 can provide additional forced air circulation for the coolant. The tray includes a spacer 410 to accommodate the connector 426.

[0034] Figure 5 According to at least one embodiment, it is possible to use or manufacture Figure 2-Figure 4 The process 500 of the steps of the method of the cooling system of the data center. In at least one embodiment of the process 500, the subprocess 502 provides a cooling system having a main cooling loop that partially extends outside the data center. The subprocess 504 provides a second cooling loop including a CDU. The CDU is used to exchange heat between the second cooling loop and the main cooling loop. It can be determined via the subprocess 506 that the rack of the server tray needs to be cooled using the cooling system of the main cooling loop and the second cooling loop. For example, when it is determined that the computing component has changed and the new computing component requires a dedicated coolant, the present process can alternatively provide a first cooling loop with a heat exchanger via the subprocess 508. The heat exchanger is used to extract heat from the rack of the data center and to transfer the heat to the main cooling loop. By at least determining that a change has occurred in the subprocess 506, the process 500 can be used with any rack or server tray with a dedicated computing component, or if no change occurs in the component, the existing cooling system of the subprocess 504 can be maintained. Alternatively, the process 500 can be applied to any data center to adapt the existing cooling system to universality.

[0035] Data Center

[0036] Figure 6 An exemplary data center 600 is shown, in which the Figure 2-Figure 5In at least one embodiment, the data center 600 includes a data center infrastructure layer 610, a framework layer 620, a software layer 630, and an application layer 640. In at least one embodiment, such as Figure 2 As described, features in components 204-218C may be performed within or in conjunction with example data center 600. In at least one embodiment, infrastructure layer 610, framework layer 620, software layer 630, and application layer 640 may be provided in part or in full via computing components on server trays located in racks 210 of data center 200. This enables the cooling system of the present disclosure to cool computing components in an efficient and effective manner, reduces the chance of leaks, and enables replacement of computing components without downtime due to the use of a common coolant in the first cooling loop.

[0037] In at least one embodiment, Figure 6 As shown, the data center infrastructure layer 610 may include a resource coordinator 612, grouped computing resources 614, and node computing resources ("node CRs") 616(1)-616(N), where "N" represents any integral positive integer. In at least one embodiment, the node CRs 616(1)-616(N) may include, but are not limited to, any number of central processing units ("CPUs") or other processors (including accelerators, field programmable gate arrays (FPGAs), graphics processors, etc.), memory devices (e.g., dynamic read-only memories), storage devices (e.g., solid-state drives or disk drives), network input / output ("NW I / O") devices, network switches, virtual machines ("VMs"), power modules, and cooling modules, etc. In at least one embodiment, one or more of the node CRs 616(1)-616(N) may be a server having one or more of the computing resources mentioned above. As such, these components 616(1)-616(N) may represent additional computing components that may receive a common coolant as part of a first cooling loop.

[0038] In at least one embodiment, the computing resources 614 of the grouping can include a separate grouping of node CRs housed in one or more racks (not shown), or a separate grouping of many racks (also not shown) housed in various geographical locations in a data center. The separate grouping of node CRs in the computing resources 614 of the grouping can include computing, network, memory or storage resources that can be configured or allocated to support the grouping of one or more workloads. In at least one embodiment, several node CRs including a CPU or a processor can be grouped in one or more racks to provide computing resources to support one or more workloads. In at least one embodiment, one or more racks can also include any number of power modules, cooling modules and network switches in any combination.

[0039] In at least one embodiment, resource coordinator 612 may configure or otherwise control one or more nodes CR 616(1)-616(N) and / or grouped computing resources 614. In at least one embodiment, resource coordinator 612 may include a software design infrastructure ("SDI") management entity for data center 600. In at least one embodiment, resource coordinator may include hardware, software, or some combination thereof.

[0040] In at least one embodiment, Figure 6 As shown, the framework layer 620 includes a job scheduler 622, a configuration manager 624, a resource manager 626, and a distributed file system 628. In at least one embodiment, the framework layer 620 may include a framework for supporting software 632 of the software layer 630 and / or one or more applications 642 of the application layer 640. In at least one embodiment, the software 632 or the application 642 may include, for example, web-based service software or applications provided by Amazon Web Services, Google Cloud, and Microsoft Azure, respectively. In at least one embodiment, the framework layer 620 may be, but is not limited to, a type of free and open source software network application framework, such as Apache Spark, which may utilize the distributed file system 628 for large-scale data processing (e.g., "big data"). TM(hereinafter referred to as "Spark"). In at least one embodiment, the job scheduler 622 may include a Spark driver to facilitate the scheduling of workloads supported by various layers of the data center 600. In at least one embodiment, the configuration manager 624 may be able to configure different layers, such as the software layer 630 and the framework layer 620 including Spark and a distributed file system 628 for supporting large-scale data processing. In at least one embodiment, the resource manager 626 can manage the computing resources mapped to or allocated to the cluster or groupings used to support the distributed file system 628 and the job scheduler 622. In at least one embodiment, the computing resources of the cluster or groupings may include the computing resources 614 of the groupings at the data center infrastructure layer 610. In at least one embodiment, the resource manager 626 may coordinate with the resource coordinator 612 to manage these mapped or allocated computing resources.

[0041] In at least one embodiment, software 632 included in software layer 630 may include software used by node CRs 616(1)-616(N), grouped computing resources 614, and / or at least a portion of distributed file system 628 of framework layer 620. One or more types of software may include, but are not limited to, Internet web page search software, email virus scanning software, database software, and streaming video content software.

[0042] In at least one embodiment, the applications 642 included in the application layer 640 may include one or more types of applications used by at least a portion of the node CRs 616(1)-616(N), the grouped computing resources 614, and / or the distributed file system 628 of the framework layer 620. The one or more types of applications may include, but are not limited to, any number of genomics applications, cognitive computing, and machine learning applications, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), or other machine learning applications used in conjunction with one or more embodiments.

[0043] In at least one embodiment, the data center 600 can use a CPU, an application-specific integrated circuit (ASIC), a GPU, an FPGA, or other hardware to use the above resources to perform training and / or reasoning. In addition, one or more of the above software and / or hardware resources can be configured to allow users to train or perform information reasoning services, such as image recognition, speech recognition, or other artificial intelligence services.

[0044] Other variations are within the spirit of the present disclosure. Thus, while the disclosed technology is susceptible to various modifications and alternative constructions, certain illustrated embodiments thereof are shown in the drawings and have been described in detail above. However, it should be understood that there is no intention to limit the disclosure to one or more specific forms disclosed, but on the contrary, it is intended to cover all modifications, alternative constructions, and equivalents that fall within the spirit and scope of the present disclosure as defined by the appended claims.

[0045] Unless otherwise noted or clearly contradictory to the context, in the context of describing the disclosed embodiments (particularly in the context of the appended claims), the use of the terms "one" and "an" and "the" and similar references should be interpreted as covering the singular and plural, rather than as definitions of terms. Unless otherwise noted, the terms "include", "have", "include" and "contain" should be interpreted as open terms (meaning "including but not limited to") unless otherwise noted. The term "connected" (which refers to a physical connection when unmodified) should be interpreted as partially or completely included, attached to or connected together, even if there are some interventions. Unless otherwise noted herein, references to numerical ranges herein are intended only to be used as a shorthand method of referring to each individual value falling within the range, respectively, and each individual value is incorporated into the specification as if it were individually described herein. Unless otherwise noted or contradictory to the context, the use of the term "set" (e.g., "item set") or "subset" should be interpreted as a non-empty set including one or more members. Furthermore, unless otherwise indicated or contradicted by context, the term "subset" of a corresponding set does not necessarily mean a proper subset of the corresponding set, but rather a subset and a corresponding set may be equivalent.

[0046] Unless otherwise expressly indicated or clearly contradicted by context, conjunctions such as phrases of the form "at least one of A, B, and C" or "at least one of A, B and C" are understood in context to be generally used to indicate an item, clause, etc., which may be A or B or C, or any non-empty subset of the set of A and B and C. For example, in the illustrative example of a set with three members, the conjunction phrases "at least one of A, B, and C" and "at least one of A, B and C" refer to any of the following sets: {A}, {B}, {C}, {A, B}, {A, C}, {B, C}, {A, B, C}. Thus, such conjunction language is generally not intended to imply that certain embodiments require the presence of at least one of A, at least one of B, and at least one of C. In addition, unless otherwise indicated or contradicted by context, a plurality refers to a plural state (e.g., a plurality of items refers to a plurality of items). The number of items in a plurality of items is at least two, but may be more if expressly indicated or indicated by context. Furthermore, unless otherwise indicated or clear from context, based on means based at least in part rather than solely based on.

[0047] The use of any and all examples or exemplary language (e.g., "such as") provided herein is intended only to better illustrate the embodiments of the present disclosure and does not limit the scope of the disclosure unless otherwise required. No language in the specification should be construed to indicate that any non-claimed element is essential to the implementation of the disclosure.

[0048] All references, including publications, patent applications, and patents, cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference.

[0049] In the description and claims, the terms "coupled" and "connected," as well as their derivatives, may be used. It should be understood that these terms may not be intended as synonyms for each other. On the contrary, in specific examples, "connected" or "coupled" may be used to indicate that two or more elements are in direct or indirect physical or electrical contact with each other. "Coupled" may also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other.

[0050] Unless expressly stated otherwise, it is understood that throughout the specification, references to processing, computing, calculating, determining, etc. refer to the action and / or process of a computer or computing system or similar electronic computing device to process and / or convert data represented as physical quantities (e.g., electronic) in registers and / or memories of the computing system into other data similarly represented as physical quantities in the memories, registers or other such information storage, transmission or display devices of the computing system.

[0051] In a similar manner, a processor may refer to any device or part of a memory that processes electronic data from registers and / or memory and converts the electronic data into other electronic data that can be stored in registers and / or memory. As a non-limiting example, a "processor" may be a CPU or a GPU. A "computing platform" may include one or more processors. As used herein, a "software" process may include, for example, software and / or hardware entities that perform work over time, such as tasks, threads, and intelligent agents. Similarly, each process may refer to multiple processes to execute instructions sequentially or in parallel, continuously or intermittently. The terms "system" and "method" may be used interchangeably herein, as long as a system may embody one or more methods, and a method may be considered a system.

[0052] Although the above discussion sets forth example implementations of the described techniques, other architectures may be used to implement the described functionality and are intended to fall within the scope of the present disclosure. In addition, although specific responsibilities are defined above for discussion purposes, various functions and responsibilities may be allocated and divided in different ways depending on the circumstances.

[0053] Furthermore, although the subject matter has been described in language specific to structural features and / or methodological acts, it should be understood that the subject matter claimed in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claims.

Claims

1. A data center cooling system, comprising: A first cooling loop comprising a heat exchanger and a cooling manifold to be coupled to a plurality of servers, the heat exchanger being used to exchange heat between the first cooling loop and a second cooling loop, the second cooling loop comprising a cooling distribution unit (CDU) being used to exchange heat between the second cooling loop and a main cooling loop, the main cooling loop being used to leave the data center to dissipate heat to an external cooling medium.

2. The data center cooling system according to claim 1, further comprising: A first coolant in the first cooling circuit, the first coolant being independent of a second coolant in the second cooling circuit and a third coolant in the main cooling circuit.

3. The data center cooling system according to claim 1, further comprising: A module comprising the first cooling circuit for mounting in a rack of the data center and comprising a fluid coupler for coupling with the second cooling circuit and for coupling with one or more server trays of the rack.

4. The data center cooling system according to claim 3, further comprising: The fluid coupler is adapted to receive at least one mating coupler from the one or more server trays or from the second cooling circuit via a snap-in connection.

5. The data center cooling system of claim 1 , wherein the first coolant in the first cooling loop is water, and wherein the second coolant in the second cooling loop and the main coolant in the main cooling loop are both dielectric fluids only or dielectric fluids with a water component.

6. The data center cooling system of claim 1, further comprising: A cooling plate is configured to be inserted into a rack of the data center and to be coupled to the first cooling circuit, the cooling plate being configured to receive a coolant and provide a path for the coolant to exit back into the first cooling circuit.

7. The data center cooling system of claim 6, further comprising: An area associated with the cold plate is adapted to include computing components of the data center or to include at least one server tray.

8. The data center cooling system of claim 1, further comprising: Auxiliary couplers of one or more server trays or cold plates located in racks of the data center for receiving coolant from the first cooling loop, for passing the coolant through the cold plates associated with one or more computing components, and for returning the coolant to the first cooling loop.

9. The data center cooling system of claim 1, wherein the heat exchanger is located outside or inside a rack of the data center, and wherein the first cooling circuit is adapted to draw heat from inside the rack to the second cooling circuit outside the rack.

10. The data center cooling system of claim 1, wherein the primary cooling circuit is coupled to a cooling tower that enables heat transferred from the first cooling circuit to the second cooling circuit to be dissipated.

11. A method of cooling a data center, comprising: providing a primary cooling loop extending in part outside of said data center; providing a secondary cooling circuit comprising a cooling distribution unit (CDU) for exchanging heat between the secondary cooling circuit and a primary cooling circuit for exiting the data center to dissipate heat to an external cooling medium; and A first cooling loop is provided, comprising a heat exchanger and a cooling manifold to be coupled to a plurality of servers, the heat exchanger being used to exchange heat between the first cooling loop and the second cooling loop.

12. The method according to claim 11, further comprising: A first coolant in the first cooling circuit is provided, the first coolant being independent of a second coolant in the second cooling circuit and a third coolant in the main cooling circuit.

13. The method according to claim 11, further comprising: assembling a module including the first cooling circuit in a rack of the data center; as well as A fluid coupler in the first cooling loop is coupled to the second cooling loop and to one or more server trays of the rack.

14. The method according to claim 13, further comprising: The fluid coupler is adapted to receive at least one mating coupler from the one or more server trays or the second cooling circuit via a snap-on connection.

15. The method according to claim 11, further comprising: Wherein the first coolant in the first cooling circuit is water, and wherein the second coolant in the second cooling circuit and the main coolant in the main cooling circuit are both dielectric fluid only or dielectric fluid with a water component.

16. The method according to claim 11, further comprising: inserting a cooling panel into a rack of the data center; as well as The cooling plate is coupled to the first cooling circuit, the cooling plate being adapted to receive coolant and provide a path for the coolant to exit back into the first cooling circuit.

17. The method according to claim 16, further comprising: An area associated with the cold plate is provided for including computing components of the data center or for including at least one server tray.

18. The method according to claim 11, further comprising: enabling auxiliary couplers of one or more server trays or cold plates located within a rack of the data center to receive coolant from the first cooling loop; enabling the coolant to pass through the cooling plate; as well as The coolant is returned to the first cooling circuit.

19. The method according to claim 11, further comprising: Arranging the heat exchanger outside or inside a rack of the data center; as well as The first cooling circuit is adapted to draw heat from within the rack to the second cooling circuit external to the rack.

20. The method according to claim 11, further comprising: The main cooling circuit is enabled to engage with a cooling tower which enables the heat transferred from the first cooling circuit to be dissipated to the second cooling circuit.

Images