Cooling design for PCIE-mounted server peripheral electronics

Abstract

The present application discloses a fluid cooling module (or system) for peripheral component interconnect express (PCIE) electronic devices housed in a chassis. The system includes: a mounting rail attached to the chassis; a cold plate / cooling plate having one or more mounting bolts attached to the side of the cold plate, the one or more mounting bolts locking the cold plate to a fixing channel of the mounting rail to position the surface of the cold plate against the surface of the PCIE electronic device, the locking teeth of the fixing channel ensuring that the cold plate is horizontally loaded with appropriate pressure on the PCIE electronic device; and a fluid circuit along the length of the mounting rail to circulate fluid to the cold plate to cool the PCIE electronic device.

Description

Technical Field

[0001] Embodiments of the present invention generally relate to electronic device cooling. More specifically, embodiments of the present invention relate to a method for designing and assembling liquid cooling hardware for use in cooling designs of peripheral electronic devices mounted in a peripheral component interconnect express (PCIE). Background Art

[0002] Edge computing is an important technology in the era of artificial intelligence (AI) and 5G. Edge computing devices will allow some level of local data processing and storage. Edge computing requires the deployment of IT equipment / electronic devices closer to end users, such as smart phones, self-driving vehicles, smart homes, etc. Therefore, the deployment environment may vary depending on the situation. In a data center, the deployment environment is very well controlled so that the servers operate under the desired thermal and humidity conditions. However, edge devices may be deployed in an external environment, which directly exposes the computing devices to the outside air, i.e., conditions that are not suitable for running these devices. In some other applications, electronic devices may be deployed in harsh environments. This may affect the long-term reliability and / or normal operation of these electronic devices.

[0003] Previous attempts have used traditional air cooling for edge computing devices / electronics, similar to air cooling solutions used for IT equipment. Some disadvantages of air cooling include limited cooling, and exposing the electronics to the surrounding environment which may affect the operation and reliability of the electronics. Most current edge computing devices are designed to use natural conduction cooling. However, this is only feasible for low power design devices. For higher power density devices, the surface temperature may exceed the design limit, or the device may not work properly. Summary of the invention

[0004] According to one aspect of the present application, a fluid cooling system for peripheral electronic devices housed in a chassis is provided, comprising: a mounting rail attached to a first side panel and a second side panel of the chassis, the first side panel and the second side panel being opposite side panels; a cold plate having one or more mounting bolts attached to a side of the cold plate, the one or more mounting bolts locking the cold plate to the mounting rail to position a surface of the cold plate against a surface of the peripheral electronic devices mounted on a motherboard in the chassis; a fluid supply line disposed on the mounting rail to receive a cooling fluid from an external fluid source and to supply the cooling fluid to the cold plate; and a fluid return line disposed on the mounting rail to return the cooling fluid from the cold plate to the external fluid source, wherein the fluid supply line and the fluid return line form a fluid loop to circulate the cooling fluid to the cold plate to cool the peripheral electronic devices.

[0005] According to another aspect of the present application, there is provided a data processing system housed in a chassis, comprising: a motherboard; a central processing unit (CPU) mounted on the motherboard; peripheral electronic devices mounted on the motherboard; a mounting rail attached to a first side panel and a second side panel of the chassis, the first side panel and the second side panel being opposite side panels; a cold plate having one or more mounting bolts attached to a side of the cold plate, the one or more mounting bolts locking the cold plate to the mounting rail to position a surface of the cold plate against a surface of the peripheral electronic devices; a fluid supply line disposed on the mounting rail to receive a cooling fluid from an external fluid source and supply the cooling fluid to the cold plate; and a fluid return line disposed on the mounting rail to return the cooling fluid from the cold plate to the external fluid source, wherein the fluid supply line and the fluid return line form a fluid loop to circulate the cooling fluid to the cold plate to cool the peripheral electronic devices.

[0006] According to another aspect of the present application, an electronic rack of a data center is provided, comprising: an array of server blades arranged in a stack, each server blade comprising one or more servers housed in a chassis, each chassis comprising: a motherboard; a central processing unit (CPU), mounted on the motherboard; peripheral electronic devices, mounted on the motherboard; a mounting rail, attached to a first side panel and a second side panel of the chassis, the first side panel and the second side panel being opposite side panels; a cold plate, having one or more mounting bolts attached to a side of the cold plate, the one or more mounting bolts locking the cold plate to the mounting rail to position the surface of the cold plate against the surface of the peripheral electronic devices; a fluid supply line, disposed on the mounting rail to receive a cooling fluid from an external fluid source and supply the cooling fluid to the cold plate; and a fluid return line, disposed on the mounting rail to return the cooling fluid from the cold plate to the external fluid source, wherein the fluid supply line and the fluid return line form a fluid loop to circulate the cooling fluid to the cold plate to cool the peripheral electronic devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Embodiments of the invention are illustrated by way of example, and not by way of limitation, in the figures of the drawings in which like references indicate similar elements.

[0008] Figure 1 is a block diagram illustrating an example of a data center facility according to one embodiment.

[0009] Figure 2 is a block diagram illustrating an example of an electronics rack according to one embodiment.

[0010] Figure 3 is a block diagram illustrating a top view of a fluid cooled enclosure / chassis for enclosing an electronic package (EEP) according to one embodiment.

[0011] Figure 4 is a block diagram illustrating a side view of a fluid cooling system for cooling a PCIE device of an EEP according to one embodiment.

[0012] Figure 5 is a block diagram illustrating a fluid cold plate / cold plate for cooling a PCIE device of an EEP according to one embodiment.

[0013] Figure 6 is a block diagram illustrating a top view of a mounting rail for an EEP according to one embodiment.

[0014] Figure 7 is a block diagram illustrating a top view of a mounting rail for an EEP according to another embodiment.

[0015] Figure 8 is a block diagram illustrating a top view of a mounting rail for an EEP according to another embodiment.

[0016] Fig. 9 is a block diagram illustrating a side view of a fluid mounting system for an EEP according to another embodiment.

[0017] Fig.10 is a block diagram illustrating a top view of a fluid mounting system for cooling a PCIE device of an EEP according to one embodiment. DETAILED DESCRIPTION

[0018] Various embodiments and aspects of the present invention will be described with reference to the details discussed below, and the accompanying drawings will illustrate the various embodiments. The following description and accompanying drawings are explanations of the present invention and should not be construed as limiting the present invention. Many specific details are described to provide a comprehensive understanding of various embodiments of the present invention. However, in some cases, well-known or conventional details are not described to provide a concise discussion of embodiments of the present invention.

[0019] Reference to "one embodiment" or "an embodiment" in this specification means that a particular feature, structure or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The phrase "in one embodiment" appearing in various places in this specification does not necessarily all refer to the same embodiment.

[0020] According to one aspect, a fluid cooling system includes: a mounting rail attached to a first side panel and a second side panel of a chassis, the first side panel and the second side panel being opposite side panels; a cold plate having one or more mounting bolts attached to a side of the cold plate, the one or more mounting bolts locking the cold plate to the mounting rail to position a surface of the cold plate against a surface of a peripheral electronic device mounted on a motherboard within the chassis; a fluid supply line disposed on the mounting rail to receive cooling fluid from an external fluid source and supply the cooling fluid to the cold plate; and a fluid return line disposed on the mounting rail to return the cooling fluid from the cold plate to the external fluid source, wherein the fluid supply line and the fluid return line form a fluid loop to circulate the cooling fluid to the cold plate, thereby cooling the peripheral electronic devices.

[0021] In one embodiment, the mounting rail includes a mounting frame, the mounting frame is attached to the chassis, and the mounting rail is attached to a length of the mounting frame, wherein one or more mounting bolts of the cold plate slide along the fixed channels of the mounting rail. In one embodiment, the mounting frame is attached to two side panels of the chassis or the bottom panel of the chassis.

[0022] In one embodiment, the mounting rails are pre-assembled and fixedly attached to the chassis, or after the chassis is assembled and the PCIE electronics are assembled to the chassis, the mounting rails are attached to the chassis based on a locking mechanism. In one embodiment, the PCIE electronics do not need to have mounting holes to mount the cold plate to the PCIE electronics, wherein the cold plate is pressed against the PCIE electronics, and the pressure (e.g., a threshold of loading pressure) is supported by the cold plate attached to the mounting rails.

[0023] In one embodiment, two opposing surfaces of the PCIE electronic device are sandwiched between a cold plate and another cold plate to cool the opposing surfaces of the PCIE electronic device. In one embodiment, the opposing surfaces of the PCIE electronic device are sandwiched between a cold plate and a mounting plate, wherein the mounting plate includes one or more mounting bolts, the one or more mounting bolts are attached to the side of the mounting plate, and the one or more mounting bolts of the mounting plate are locked to the mounting rail to support the PCIE electronic device pressed against the surface of the cold plate, thereby cooling the PCIE electronic device.

[0024] In one embodiment, the fixing channel of the mounting rail is an elongated fixing channel along the length of the mounting rail. In another embodiment, each of the one or more mounting bolts of the cold plate includes an oval head that rotates at least 90 degrees to activate or deactivate the first locking mechanism, wherein if deactivated, the oval head is insertable through the fixing channel of the mounting rail, wherein if activated and the oval head is inserted through the fixing channel, the oval head locks the mounting bolt of the cold plate in the fixing channel such that a neck of the mounting bolt slides along the fixing channel.

[0025] In one embodiment, the fixing channel of the mounting rail has a plurality of teeth or serrated edges along one side along the length of the fixing channel. In one embodiment, each of the serrated edges is oriented so that the neck of the mounting bolt of the cold plate is free to slide in only one direction along the fixing channel of the mounting rail. In one embodiment, the serrated edges for the fixing channel and the portions of the mounting rail may be modular. In one embodiment, the mounting rail may include two modular portions, each of the serrated edges being in a first direction for a first portion of the mounting rail, and each of the serrated edges being in a second direction for a second portion of the mounting rail.

[0026] The serrated edge can lock the cold plate in a non-sliding direction, while sliding the cold plate along the fixed channel in a slidable direction can guide the cold plate to a specific position of a chip, GPU, printed circuit board (PCB) or any other component of a PCIE electronic device. For example, for a dual module mounting track, an operator can slide a first plate in a first direction along a first fixed channel (e.g., the plate base surface slides toward the surface of the PCIE electronic device), and slide a second plate (a cooling plate or a mounting plate, which can have a physical structure similar to the cold plate, but for the purpose of mounting support, for example, no fluid flows through the mounting plate) along a second fixed channel in a second direction (e.g., the plate base surface of the second plate slides toward the opposite surface of the PCIE electronic device).

[0027] The neck of the board is directionally locked to one or more fixed channels of the mounting track by sliding along the directional serrated edge of the fixed channel, and the mounting pressure is applied from the board to the surface of the PCIE electronic device. The required mounting pressure of one or more boards (cold plates or mounting plates) applied to the surface of the PCIE electronic device is provided by the board tightly locked to the mounting track. Once the mounting pressure is loaded to the cold plate and the PCIE electronic device, the serrated edge of the fixed channel provides and maintains appropriate pressure against the neck portion of the board. In another embodiment, the mounting pressure is loaded to the cold plate and the PCIE electronic device through the head of the mounting bolt, which is locked on the fixed channel of the mounting track.

[0028] In one embodiment, the mounting channel of the mounting track has opposing serrated edges facing opposite sides of the mounting channel. In one embodiment, the mounting track further includes an unlocking mechanism to release the serrated edges so that the mounting bolts of the cold plate can be removed from the mounting channel. With the serrated edges released, an operator can adjust the mounting bolts to remove the mounting bolts from the mounting channel and remove the cold plate from the mounting track. In another embodiment, when adjusted, the mounting bolts can be removed through the mounting channel with the serrated edges without releasing the serrated edges. In either case, the cold plate can be removed from a motherboard, device, chip, etc.

[0029] In one embodiment, the cold plate is fixed to the PCIE electronic device by applying force to the cold plate to slide one or more mounting bolts of the cold plate along the direction of the serrations (serrated edges) of the cold plate to be fixed to the PCIE electronic device. The fixing mechanism also provides appropriate mounting pressure to load the cold plate to the chip / processor. The purpose is to ensure that the base surface of the cold plate is in full contact with the processor (reducing thermal resistance) to ensure proper thermal function. In one embodiment, one or more mounting bolts of the corresponding cold plate are pre-assembled to the cold plate.

[0030] In one embodiment, one or more mounting structures are attached to the chassis for securing the mounting rails to the mounting structures. In one embodiment, the chassis forms a completely closed environment such that no outside air is exchanged with air inside the closed environment.

[0031] In one embodiment, the system further comprises a fluid distribution unit coupled to the chassis, wherein the fluid distribution unit comprises a fluid inlet, a fluid outlet, and a fluid distribution channel coupled to the fluid inlet and the fluid outlet, the fluid distribution channel distributing the fluid from the fluid inlet to the fluid outlet. In one embodiment, the fluid circuit is coupled to the fluid distribution channel of the fluid distribution unit.

[0032] The disclosed system can be used for electronic packaging / edge computing devices and / or IT equipment in a data center. In some cases, the electronic packaging can be mobile. Note that although PCIE devices are used as examples of electronic devices in this application, the cooling technology can also be applied to other electronic devices.

[0033] Figure 1 is a block diagram illustrating an example of a data center or data center unit according to one embodiment. In this example, Figure 1 A top view of at least a portion of a data center is shown. Figure 1According to one embodiment, a data center system 100 includes rows of electronic racks of information technology (IT) components, equipment, or instruments 101 to 102, such as computer servers or computing nodes that provide data services to various clients via a network (e.g., the Internet). In this embodiment, each row includes an array of electronic racks such as electronic racks 110A to 110N. However, more or fewer rows of electronic racks may be implemented. Typically, the rows 101 to 102 are aligned in parallel, with their front ends facing each other and their rear ends facing away from each other, thereby forming aisles 103 between the rows to allow management personnel to walk therein. However, other configurations or arrangements may also be applied.

[0034] In one embodiment, each of the electronic racks (e.g., electronic racks 110A to 110N) includes a housing to accommodate multiple electronic racks of IT components operating therein. The electronic racks may include a heat removal liquid manifold, multiple server slots, and multiple server blades that can be inserted into and removed from the server blades or server slots. Each server blade represents a computing node having one or more processors, memory, and / or permanent storage devices (e.g., hard drives). At least one of the processors is attached to a liquid cold plate (also referred to as a cold plate assembly) to receive cooling liquid. Note that the heat removal system 120 can be coupled to multiple data center systems, such as data center system 400.

[0035] In one embodiment, the heat rejection system 120 includes an external liquid loop connected to a cooling tower or dry cooler outside the building / containment vessel. The heat rejection system 120 may include, but is not limited to, evaporative cooling designs, free air designs, rejection designs for large thermal masses, and waste heat recovery designs. The heat rejection system 120 may include or be coupled to a cooling liquid source that provides cooling liquid.

[0036] In one embodiment, each server blade is modularly coupled to a heat exhaust liquid manifold so that the server blade can be removed from the electronics rack without affecting the operation of the remaining server blades on the electronics rack and the heat exhaust liquid manifold. In another embodiment, each server blade is coupled to a heat exhaust liquid manifold (also known as a cooling liquid manifold) by a quick release coupling assembly having a first liquid intake connector and a first liquid outlet connector coupled to a flexible hose to distribute heat exhaust liquid to the processor. The first liquid intake connector receives heat exhaust liquid from the heat exhaust liquid manifold mounted at the rear end of the electronics rack via a second liquid intake connector. The first liquid outlet liquid connector dissipates warmer or hotter liquid carrying heat exchanged from the processor to the heat exhaust liquid manifold via a second liquid outlet connector and then returns to a coolant distribution unit (CDU) within the electronics rack.

[0037] In one embodiment, a heat rejection liquid manifold disposed on the rear end of each electronic rack is coupled to a liquid supply line 132 to receive heat rejection liquid (also referred to as cooling liquid) from the heat rejection system 120. The heat rejection liquid is distributed / circulated / recirculated by a liquid distribution loop attached to a cold plate assembly on which a processor is mounted to reject heat from the processor. The cold plate is configured similarly to a radiator with a liquid distribution tube attached or embedded therein. The resulting warmer or hotter liquid carrying heat exchanged from the processor is transmitted back to the heat rejection system 120 via a liquid return line 131. Liquid supply / return lines 131 to 132 are referred to as data center or room liquid supply / return lines (e.g., global liquid supply / return lines), which supply heat rejection liquid to all electronic racks in rows 101 to 102. Liquid supply line 132 and liquid return line 131 are coupled to the heat exchanger of the CDU located in each of the electronic racks, thereby forming a main loop. A second loop of the heat exchanger is coupled to each of the server blades in the electronics rack to deliver cooling fluid to the cold plates of the processors.

[0038] According to one embodiment, each of the electronic racks includes an optional rack management unit (RMU) coupled to the CDU and each of the computing nodes of the electronic rack (not shown). The RMC periodically or continuously monitors the operating status of the CDU and the computing nodes. The operating data of the operating status may include the operating temperature of each processor, coolant, etc. measured in real time. Based on the operating data received from the various components, the RMU performs optimization using an optimization function to determine the optimal pump speed of the liquid pump of the CDU so that the power consumption of the liquid is minimized while the liquid pumps operate appropriately according to their respective specifications (e.g., the speed of the liquid pumps is within their respective predetermined ranges).

[0039] That is, the optimization is performed at a global level by optimizing all components involved simultaneously so that: 1) the temperature of the processors is below their respective reference temperatures; 2) the total power consumption of the liquid pumps is minimized; and 3) each of the liquid pumps operates within their respective specifications. The liquid pumps are then configured with the optimal pump speed. As a result, the total power consumption of the liquid pumps is minimized when the processors of the computing node operate appropriately.

[0040] Figure 2 2 is a block diagram showing an electronic rack according to one embodiment. The electronic rack 200 may be represented as follows: Figure 1 Any of the electronic racks shown, for example, electronic racks 110A to 110N. Figure 2According to one embodiment, the electronic rack 200 includes, but is not limited to, a CDU 201, an optional RMU 202, and one or more server blades 203A to 203E (collectively referred to as server blades 203). The server blades 203 can be inserted into the array of server slots from the front end 204 or the back end 205 of the electronic rack 200, respectively. Note that although five server blades 203A to 203E are shown here, more or fewer server blades can be maintained within the electronic rack 200. It should also be noted that the specific locations of the CDU 201, RMU 202, and server blades 203 are shown for illustrative purposes only; other arrangements or configurations of the CDU 201, RMU 202, and server blades 203 can also be implemented. In one embodiment, the electronic rack 200 can be open to the environment or partially contained by a rack container. In another embodiment, the electronic rack 200 can be fully contained and not exposed to the environment.

[0041] In one embodiment, the CDU 201 mainly includes a heat exchanger 211, a liquid pump 212 and a pump controller (not shown), as well as some other components such as a liquid container, a power supply, a monitoring sensor, etc. The heat exchanger 211 can be a liquid-to-liquid heat exchanger. The heat exchanger 211 includes a first loop having an inlet port and an outlet port, and the inlet port and the outlet port have a first pair of liquid connectors connected to external liquid supply / return lines 131 to 132 to form a main loop. The connector connected to the external liquid supply / return lines 131 to 132 can be set or installed on the rear end 205 of the electronic rack 200. The liquid supply / return lines 131 to 132 are also called room liquid supply / return lines, and are connected to the heat removal system 120 as above. In addition, the heat exchanger 211 also includes a second loop having two ports, wherein the two ports have a second pair of liquid connectors coupled to the liquid manifold 225 to form a second loop, wherein the second loop may include a supply manifold (also referred to as a rack liquid supply line) and a return manifold (also referred to as a rack liquid return line), wherein the supply manifold supplies cooling liquid to the server blades 203 and the return manifold returns warmer liquid to the CDU 201. Note that the CDU 201 may be any kind of CDU that is commercially available or custom made. Thus, the details of the CDU 201 will not be described herein.

[0042] Each of the server blades 203 may include one or more IT components (e.g., a central processing unit or CPU, a graphics processing unit (GPU), memory, and / or storage devices). Each IT component may perform data processing tasks, wherein the IT component may include software installed in a storage device, loaded into the memory, and executed by one or more processors to perform the data processing tasks. The server blades 203 may include a master server (referred to as a master node) connected to one or more computing servers (also referred to as computing nodes, such as CPU servers and GPU servers). The master server (having one or more CPUs) typically interfaces with a client over a network (e.g., the Internet) to receive requests for specific services such as storage services (e.g., cloud-based storage services such as backup and / or recovery), execute applications to perform certain operations (e.g., image processing, deep data learning algorithms or modeling, etc., as part of a software as a service or SaaS platform).

[0043] In response to the request, the master server assigns the task to one or more of the performance computing nodes or computing servers (with one or more GPUs) managed by the master server. The performance computing server performs the actual task, which may generate heat during operation. One or more computing servers may be edge computing servers having a motherboard and electronic devices such as CPU / GPU electronic devices mounted horizontally on the motherboard and vertically mounted components, such as components mounted vertically on the PCIE expansion bus of the motherboard. These components may be memory modules, accelerators, or any other electronic components.

[0044] The electronics rack 200 also includes an optional RMU 202 configured to provide and manage power supplied to the servers 203 and the CDU 201. The RMU 202 can be coupled to a power supply unit (not shown) to manage the power consumption of the power supply unit. The power supply unit can include necessary circuitry (e.g., an alternating current (AC) to direct current (DC) or DC to DC power converter, a battery, a transformer or a voltage regulator, etc.) to provide power to the remaining components of the electronics rack 200.

[0045] In one embodiment, the RMU 202 includes an optimization module 221 and a rack management controller (RMC) 222. The RMC 222 may include a monitor to monitor the operating status of various components within the electronic rack 200, such as the computing nodes 203 and the CDU 201. Specifically, the monitor receives operating data from various sensors representing the operating environment of the electronic rack 200. For example, the monitor may receive operating data representing the temperature of the processor, coolant, and airflow, which may be captured and collected via various temperature sensors. The monitor may also receive data representing the pump power generated by the liquid pump 212, which may be proportional to the speed of each of the liquid pumps 212. These operating data are referred to as real-time operating data. Note that the monitor may be implemented as a separate module within the RMU 202.

[0046] Based on the operating data, the optimization module 221 performs optimization using a predetermined optimization function or optimization model to derive an optimal pump speed for the liquid pump 212 so that the total power consumption of the liquid pump 212 is minimized while the operating data associated with the liquid pump 212 is within their respective design specifications. Once the optimal pump speed is determined, the RMC 222 configures the liquid pump 212 based on the optimal pump speed.

[0047] Figure 3 is a block diagram showing a top view of a fluid cooled enclosure / chassis for enclosing an electronic package (EEP) according to one embodiment. Figure 3 , a fluid cooling system 300 for an electronic package 301 held in a chassis 303 is disclosed. The electronic package may be closed / hermetically sealed within the chassis. The cooling system 300 includes a plurality of cooling devices / modules 305 to 309 for different types of electronic devices, both for vertically mounted electronic devices 313 and horizontally mounted electronic devices 311. These cooling devices 305, 307, 309 are used to cool heat generating electronic devices 311 to 315, including low power density electronic devices 315 and high power density electronic devices 311 to 313.

[0048] In one embodiment, the cooling device is a cooling plate or a mounting plate. For low power density electronic devices 315, their heat can be cooled by natural convection and / or a plate cooling device (e.g., a plate cooling plate) 305. The cooling system 300 may include a fluid distribution manifold 321 and / or a fluid distribution / pipeline 323. The fluid distribution manifold 321 is used to distribute the fluid to the cooling device. The fluid distribution manifold 321 is connected to a fluid inlet 325 and a fluid outlet 327 so that the fluid flows into and out of the fluid distribution manifold 321, respectively. The cooling system 300 can provide a fluid cooling design for the entire housing at the same time, providing a high separation of the fluid portion from the electronic device. The vertically mounted electronic device can be a PCIE electronic device installed on a PCIE expansion bus.

[0049] Currently, there are no solutions or mature solutions for chassis-level fluid cooling of vertically mounted electronics, such as those mounted to a PCI-E expansion bus. In addition, most components mounted in this manner, such as GPU cards, graphics adapter cards, network interface cards (NICs), storage accelerator devices, solid-state drives (SSDs), and other high-performance peripherals, can have different form factors. This means that the cooling solution should be independent of the component form factor.

[0050] The present application discloses a fluid cooling system for cooling vertically mounted PCIE electronic devices, such as PCIE hardware components and / or GPU cards. In one embodiment, the system includes a mounting track for mounting and fixing a cold plate. The system may include one or more cold plates pressed against one or more surfaces of the PCIE electronic device. Each cold plate includes one or more mounting bolts to fix the cold plate to the mounting track. In one embodiment, the mounting track includes one or more channels (e.g., fixed channels) having locking teeth for fixing the position of the cold plate. Moreover, the one or more channels can load appropriate mounting pressure to the vertically mounted electronic device. The mounting track also includes one or more fluid channels for a fluid circuit to supply fluid to and from the cold plate.

[0051] The mounting rails can be pre-assembled with the chassis, or mounted on the mounting frame of the chassis after the motherboard and PCIE components of the electronic package are assembled. Thus, the mounting rails are structurally supported by the chassis.

[0052] Figure 4 is a block diagram illustrating a side view of a fluid cooling system for cooling PCIE electronics of an EEP according to one embodiment. Figure 4 A cooling system 400 for PCIE electronic devices mounted on a PCIE expansion bus 410 is shown. The cooling system 400 may be Figure 3 Part of a cooling system 300 (eg, a cooling system for a vertically mounted electronic device). Figure 4 , system 400 includes mounting rails 412. Mounting rails 412 can be assembled and secured to one or more side panels or bottom panels of chassis 303. Cold plate 409 and / or mounting plate 403 are used on one or both sides of PCIE electronic devices 313 to cool PCIE electronic devices 313. If one side of a cold plate is used, the other side may include a mounting plate for pressure loading purposes. The cold plate can then be mounted and secured to mounting rails 412 and attached to the electronic components with appropriate loading pressure. Fluid circuit 405 can then be assembled on top of mounting rails 412 or in a fluid channel.

[0053] In this embodiment, no design changes are required to the PCIE components or chassis hardware or the motherboard of the electronic package. The mounting rails provide a reliable structure for assembling the cold plate to the PCIE components.

[0054] Figure 5 is a block diagram illustrating a fluid cold plate (top and side views) for cooling a PCIE device of an EEP according to one embodiment. Figure 5 The cold plate 409 is shown to include a base body 502, a fluid inlet and outlet (not shown), and one or more mounting bolts 504. In one embodiment, the cold plate includes four mounting bolts 504 to mount to a mounting track having two fixed channels. The mounting bolts 504 can be pre-assembled to the cold plate base body 502. In one embodiment, the position of these bolts 504 on the cold plate base body 502 can be adjusted, for example, by threads (not shown) on the bolts and the base body. In one embodiment, the head of the bolt is oval. In one embodiment, the head of the bolt can be rotated 90 degrees in a clockwise or counterclockwise direction. The bolt design is to facilitate mounting the cold plate to the fixed channels of the mounting track. It should be noted that other types of designs can also be used to achieve similar results. The cold plate base body 502 is connected to the fluid supply / return circuit 405 to circulate the fluid through the base body 502.

[0055] Figure 6 is a block diagram illustrating a top view of a mounting rail for an EEP according to one embodiment. Figure 6 The mounting rail 412 is shown with the cold plate 409 assembled. Figure 6 Two fixing channels 602 for mounting rails 412 are shown. Four mounting bolts 504 are used as a locking mechanism to fix the cold plate 409 to the mounting rails 412. In one embodiment, for each fixing channel 602, a plurality of serrated edges / locking teeth 604 are assembled in one or both edges of the fixing channel. The function of the locking teeth 604 is to allow the mounting bolts 504 to move or slide in one direction in the fixing channel 602, rather than in the opposite direction. The directional movement allows the operator to slide the mounting bolts in one direction during installation, thereby moving the cold plate 409 corresponding to the mounting bolts 504 toward the surface of the PCIE component (or electronic device). In this way, the locking teeth 604 prevent the cold plate and the mounting bolts from moving away from the PCIE component. Since there are many locking teeth 604 in the fixing channel 602, the position of the cold plate 409 can be finely adjusted.

[0056] When the operator assembles the cold plate 409 (or mounting plate) on both sides of the PCIE component, by pushing the two sides closer to each other, appropriate loading pressure can be applied to the PCIE electronic component, and the pressure can be maintained by these locking teeth 604. In one embodiment, the mounting rail 412 includes one or more fluid channels 606. In one embodiment, the fluid circuit 405 passes through the opening 608 of the mounting rail 412 and is assembled in the fluid channel 606 of the mounting rail 412. It should be mentioned that although in Figure 6 4 shows a mounting rail 412 having two fixing channels 602 fixed to a cold plate 409 with four mounting bolts 504, but the number of fixing channels and mounting rails can vary. Four mounting bolts are used as an example, however, the number of mounting bolts can vary.

[0057] In one embodiment, the head of the mounting bolt is oval shaped so that the head can be rotated and aligned with the fixing channel so that an operator can insert the oval head through the fixing channel so that the neck of the mounting bolt engages with the fixing channel. In one embodiment, the head of the mounting bolt can be rotated 90 degrees to fix the head of the mounting bolt on the fixing channel. In another embodiment, once the cold plate and / or mounting plate are moved to a certain position, the head of the mounting bolt can be rotated 90 degrees to fully secure the mounting bolt and the cold plate to the mounting rail. In one embodiment, the head of the mounting bolt can be designed to use multiple parts for different functions, such as a first part designed to be oval for mounting purposes and a second part designed to be a screw for tightening purposes.

[0058] Figure 7 is a block diagram illustrating a top view of a mounting rail for an EEP according to another embodiment. Figure 7 A top view of a mounting rail 412 is shown with two cold plates 409 assembled thereto, wherein the mounting rail 412 has two openings 608. It can be seen that each of the openings 608 can be used as a fluid passage for the fluid circuit 405 of a particular cold plate.

[0059] Figure 8 is a block diagram illustrating a top view of a mounting rail for an EEP according to another embodiment. Figure 8A mounting track 412 having two module portions 412A-412B is shown. The first module portion 412A of the mounting track is assembled with the first cold plate 409, and the first module portion has locking teeth 604A in a first direction. The second module portion 412B of the mounting track is assembled with the second cold plate 409, and the second module portion has locking teeth 604B in a second direction. In this way, the operator can press the two cold plates 409 together to sandwich the PCIE electronic device 313 in the middle. Although only two module portions 412A-412B are shown, the mounting track 412 can include any number of module portions. The mounting track 412 can be pre-assembled with the chassis or mounted to the mounting frame 431.

[0060] Fig. 9 is a block diagram showing a side view of a fluid mounting system for an EEP according to another embodiment. In one embodiment, the opening of the mounting track may include one or more fluid channels (ie, layered). Fig. 9 A mounting rail 412 is shown with two layers of openings for two fluid channels 606A-606B. This is useful because when there are multiple cold plates 409 mounted on the mounting rail 412, the operator can place the fluid circuits 405 for one or more cold plates 409 in different layers of openings. In one embodiment, one cold plate 409 can use two layers of openings, one for the supply circuit of the cold plate and one for the return circuit of the cold plate.

[0061] Fig.10 is a block diagram illustrating a top view of a fluid mounting system for cooling a PCIE device of an EEP according to one embodiment. Fig.10 A fluid cooling system 300 is shown for an electronic package 301 (having a motherboard 401) within a chassis 303, the electronic package having a plurality of vertically mounted electronic devices 313, such as PCIE mounted electronic devices. Fig.10 Two mounting rails 412 are shown to be used within the chassis 303. One or more cold plates / mounting support plates are fixed to the locking teeth of the mounting rails 412. The cold plate / mounting support plate is fixed in a position where horizontal pressure is applied to sandwich one or more PCIE electronic devices 313. The pressure is applied through the locking teeth and the fixed channels. It should be mentioned that the fluid circuit of the cold plate is connected to the fluid distribution unit of the fluid distribution unit 323 to circulate the cooling fluid to the cold plate and keep the hot fluid away from the cold plate. The electronic package can be airtight and closed with a fluid inlet and outlet to circulate the fluid.

[0062] However, although PCIE devices are again used as examples of electronic devices in this application, the cooling technology can also be applied to other electronic devices, such as a microprocessor (e.g., a central processing unit or CPU), a general-purpose processing unit (GPU), an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA), etc.

[0063] In the above description, embodiments of the present invention have been described with reference to specific exemplary embodiments of the present invention. It will be apparent that various modifications may be made to the present invention without departing from the broader spirit and scope of the present invention as set forth in the appended claims. Therefore, the present description and the accompanying drawings should be understood in an illustrative rather than a restrictive sense.

Claims

1. A fluid cooling system for peripheral electronic devices contained in a chassis, the fluid cooling system comprising: a mounting rail attached to a first side panel and a second side panel of the chassis, the first side panel and the second side panel being opposing side panels; a cold plate having one or more mounting bolts attached to a side of the cold plate, the one or more mounting bolts locking the cold plate to the mounting rails to position a surface of the cold plate against a surface of the peripheral electronic device mounted on a motherboard within the chassis, each of the one or more mounting bolts including an oval head; a fluid supply line, disposed on the mounting rail to receive cooling fluid from an external fluid source and supply the cooling fluid to the cold plate; as well as A fluid return line is disposed on the mounting rail to return the cooling fluid from the cold plate to the external fluid source, wherein the fluid supply line and the fluid return line form a fluid loop to circulate the cooling fluid to the cold plate to cool the peripheral electronic devices.

2. The system according to claim 1, wherein: The mounting rail includes a mounting frame attached to the chassis, wherein the mounting rail is attached to a length of the mounting frame, wherein the one or more mounting bolts of the cold plate slide along a securing channel of the mounting rail.

3. The system according to claim 2, wherein: The mounting frame is attached to the first and second side panels of the chassis or a bottom panel of the chassis.

4. The system according to claim 1, wherein: The mounting rails are pre-assembled and fixedly attached to the chassis, or after assembling the chassis and assembling the peripheral electronics to the chassis, the mounting rails are attached to the chassis based on a locking mechanism.

5. The system according to claim 1, wherein: The peripheral electronics need not have mounting holes for mounting the cold plate to the peripheral electronics, wherein the cold plate is pressed against the peripheral electronics and the pressure is supported by the cold plate being attached to the mounting rail.

6. The system according to claim 1, wherein: Two opposing surfaces of the peripheral electronic device are sandwiched between the cold plate and another cold plate, so that the cold plate and the another cold plate absorb heat from the opposing surfaces of the peripheral electronic device.

7. The system according to claim 1, wherein: The opposing surfaces of the peripheral electronic device are sandwiched between the cold plate and a mounting plate, wherein the mounting plate includes one or more mounting bolts attached to a side of the mounting plate, and the one or more mounting bolts of the mounting plate are locked to the mounting rails to horizontally lock a force to support the peripheral electronic device pressed against the surface of the cold plate to cool the peripheral electronic device.

8. The system according to claim 2, wherein: The fixing channel of the mounting rail is an elongated fixing channel along the length of the mounting rail.

9. The system according to claim 2, wherein: The oval head is rotated at least 90 degrees to activate or deactivate a first locking mechanism, wherein if the first locking mechanism is deactivated, the oval head can be inserted through the fixing channel of the mounting rail, wherein if the first locking mechanism is activated and the oval head is inserted through the fixing channel, the oval head locks the mounting bolt of the cold plate in the fixing channel so that a neck of the mounting bolt slides along the fixing channel.

10. The system according to claim 9, wherein: The securing channel of the mounting rail includes a plurality of serrated edges along one side along a length of the securing channel.

11. The system according to claim 10, wherein: Each of the serrated edges is oriented so that the neck of a mounting bolt of a cold plate is free to slide along the securing channel of the mounting rail in one direction only.

12. The system according to claim 10, wherein: Each of the serrated edges is in a first direction for a first portion of the mounting track, and each of the serrated edges is in a second direction for a second portion of the mounting track.

13. The system according to claim 11, wherein: The securing channel of the mounting rail includes opposing serrated edges facing opposite sides of the securing channel.

14. The system according to claim 13, wherein: The mounting rail also includes an unlocking mechanism for unlocking the serrated edge to enable removal of the mounting bolts of the cold plate from the securing channel.

15. The system of claim 14, wherein: The cold plate is secured to the peripheral electronic device by applying a force to the cold plate to slide the one or more mounting bolts of the cold plate in the direction of the serrations of the cold plate to be secured to the peripheral electronic device.

16. The system of claim 1, wherein: The one or more mounting bolts of each cold plate are pre-assembled to the cold plate.

17. The system of claim 1, further comprising one or more mounting structures attached to the chassis for securing the mounting rail to the mounting structures.

18. The system of claim 1, wherein: The mounting track further includes one or more fluid channels that guide a fluid circuit along the mounting track, and one or more openings that provide access to the fluid channels and / or the fluid circuit.

19. A data processing system contained in a chassis, comprising: Motherboard; a central processing unit (CPU), mounted on the mainboard; Peripheral electronic components, mounted on the main board; a mounting rail attached to a first side panel and a second side panel of the chassis, the first side panel and the second side panel being opposing side panels; a cold plate having one or more mounting bolts attached to a side of the cold plate, the one or more mounting bolts locking the cold plate to the mounting rail to position a surface of the cold plate against a surface of the peripheral electronics, each of the one or more mounting bolts including an oval head; a fluid supply line, disposed on the mounting rail to receive cooling fluid from an external fluid source and supply the cooling fluid to the cold plate; as well as A fluid return line is disposed on the mounting rail to return the cooling fluid from the cold plate to the external fluid source, wherein the fluid supply line and the fluid return line form a fluid loop to circulate the cooling fluid to the cold plate to cool the peripheral electronic devices.

20. An electronic rack for a data center, comprising: An array of server blades arranged in stacks, each server blade comprising one or more servers housed in a chassis, each chassis comprising: Motherboard; a central processing unit (CPU), mounted on the mainboard; Peripheral electronic components, mounted on the main board; a mounting rail attached to a first side panel and a second side panel of the chassis, the first side panel and the second side panel being opposing side panels; a cold plate having one or more mounting bolts attached to a side of the cold plate, the one or more mounting bolts locking the cold plate to the mounting rail to position a surface of the cold plate against a surface of the peripheral electronics, each of the one or more mounting bolts including an oval head; a fluid supply line disposed on the mounting rail to receive cooling fluid from an external fluid source and supply the cooling fluid to the cold plate; and A fluid return line is disposed on the mounting rail to return the cooling fluid from the cold plate to the external fluid source, wherein the fluid supply line and the fluid return line form a fluid loop to circulate the cooling fluid to the cold plate to cool the peripheral electronic devices.

Images