Cooling method for computer system, and cooling system

Abstract

[Problem] To provide an improved cooling method and cooling system for liquid immersion cooling a three-dimensionally complex computer system. [Solution] A computer system including at least two heat-generating bodies that is divided into a first segment including at least one first heat-generating body, and a second segment including at least one second heat-generating body, the first segment being subjected to a waterproofing treatment by being covered with a waterproof coating film or a waterproof bag. The first segment that has been subjected to the waterproofing treatment is cooled by being immersed in a liquid immersion tank in which a coolant is circulated, and the coolant in the liquid immersion tank is used to forcibly cool, in a localized manner, the first heat-generating body by means of a first cold plate to which a pump is fluid-connected. In the second segment that has not been subjected to a waterproofing treatment, some of the coolant caused to branch away from a circulation path is used to forcibly cool the second heat-generating body by means of a second cold plate to which another pump is fluid-connected.

Description

Cooling method and cooling system for computer system

[0001] The present invention relates to a cooling method and a cooling system for a computer system, and in particular to a cooling method and a cooling system for immersing a computer system that generates a large amount of heat and that requires high performance, stable operation, or low power consumption, such as a supercomputer, a data center, an artificial intelligence (AI) processing system, a quantum computing system, a cryptographic processing system, or a blockchain processing system, in a refrigerant.

[0002] In recent years, one of the biggest challenges determining the performance limits of computer-related equipment is power consumption. The importance of research into power efficiency, particularly in supercomputers, has already been widely recognized. Specifically, speed performance per watt (Flops / W) has become one of the metrics for evaluating supercomputers. Furthermore, it is estimated that more than 30% of the total power consumed by data centers is spent on cooling, creating a growing demand for reducing power consumption through improved cooling efficiency. Furthermore, recent global warming and extreme heat caused by extreme weather have rendered conventional cooling methods insufficient to cool data centers during the summer, creating an urgent need to improve the cooling capacity of existing systems. Furthermore, the rapid growth of artificial intelligence processing, the increasing need for encryption processing, the rapid increase in cryptocurrency mining processing—a key example of blockchain processing—and the development of the metaverse, which is expected to see rapid growth, all of these factors require an exponential increase in the processing power of data centers and computer systems, creating a significant demand for increased cooling capacity.

[0003] Air cooling and liquid cooling have traditionally been used to cool supercomputers and data centers. Liquid cooling uses a liquid that has significantly better heat transfer properties than air, and is therefore generally considered to have good cooling efficiency. Liquid cooling also includes immersion cooling and non-immersion cooling. As an example of immersion cooling, "TSUBAME-KFC," built by the Tokyo Institute of Technology, achieved 4.50 GFlops / W using an immersion cooling system using synthetic oil, and was ranked first in the "Supercomputer Green500 List" announced in November 2013 and June 2014.

[0004] On the other hand, immersion cooling systems using fluorocarbon-based coolants instead of synthetic or mineral oils have been proposed, specifically, examples using fluorocarbon-based coolants (hydrofluoroether (HFE) compounds known as 3M's trade names "Novec 7100" (a trademark of 3M; the same applies hereinafter)," "Novec 7200," and "Novec 7300") (e.g., Patent Documents 1 and 2).

[0005] In addition to these, since 2014, the present inventor has been developing a new liquid immersion cooling system and a series of related technologies that directly cools electronic devices by circulating a cooling liquid that is difficult to evaporate and contains a fully fluorinated substance as its main component within the open space of a cooling tank (for example, Patent Document 3).

[0006] However, PFAS (Perfluoroalkyl Substances and Polyfluoroalkyl Substances), which include all of these fluorocarbon-based coolants, have long been known to have adverse effects on the human body, agricultural crops, and the natural environment, and in December 2022, the world's largest manufacturer announced that it would completely cease production by the end of 2025, and in Europe, both production and use are expected to be banned by law within a few years. Therefore, there is a need for a new immersion cooling method that does not use any harmful compounds such as PFAS as a refrigerant, has superior cooling capacity compared to immersion cooling methods that use PFAS or other refrigerants, and is more inexpensive and can be used widely worldwide.

[0007] A typical example of a cooling method that does not use any harmful compounds such as PFAS as a refrigerant is a cooling method that can use ordinary water (tap water or industrial water) that is not pure water. Specifically, a cooling method has been proposed in which a semiconductor chip mounted on a circuit board is housed in a flexible bag, and the bag is immersed in a container containing water as a cooling liquid, and the bag is filled with water around the bag or the inside of the bag is depressurized, causing the bag to shrink and deform due to the pressure difference between the inside and outside of the bag, and the bag is tightly attached to the semiconductor device (for example, Patent Document 4).

[0008] Another cooling method that can utilize water has been proposed: natural water-cooled computers, which use river, lake, ocean, or tap water as a cooling source to directly cool the computer. Specifically, this cooling method involves immersing a computer whose board surface is coated with parylene resin in water (see, for example, Non-Patent Document 1).

[0009] Furthermore, a cooling method has been proposed in which the entire substrate is completely covered with an ultra-nano hydrophobic coating thin film made of silicon compound nanoparticles to create an electronic device with excellent waterproof and moisture-resistant properties, and the device is then immersed in water (e.g., Patent Document 5).

[0010] One example of non-immersion cooling is a liquid cooling system that liquid-cools a computer system in which multiple electronic boards (servers) are housed in each of multiple racks, in which a cold plate is thermally connected to each semiconductor device on each electronic board in each rack, and a refrigerant is circulated through piping that connects each cold plate to a refrigerant distribution unit (CDU) that includes a heat exchanger and a pump. In this non-immersion liquid cooling system, the semiconductor devices, which are the main heat generators, can be liquid-cooled, but other electronic components, etc. must be air-cooled.

[0011] Japanese Patent No. 2804640, U.S. Patent No. 10717881, Kazuki Fujiwara et al., Information Processing Society of Japan Research Report High Performance Computing (HPC), "The First Step Towards a Direct Natural Water-Cooled Computer," 2017-HPC-158(5), pp.1-5 (March 1, 2017), URL: http: / / research.nii.ac.jp / ~koibuchi / pdf / ikki-sighpc158.pdf

[0012] The cooling system disclosed in Non-Patent Document 1 completely covers the entire electronic components and electronic board with a thin film of parylene, which does not conduct electricity and is impermeable to water, and then the entire electronic components and electronic board coated with the parylene film are immersed in water or seawater, and the low temperature of the water or seawater that comes into contact with the entire parylene film is used to cool the equipment.

[0013] Similarly, the cooling system disclosed in Patent Document 5 also completely covers the entire electronic components and electronic board with an ultra-nano hydrophobic coating film made of silicon compound nanoparticles that are non-conductive and impermeable to water, and then the entire electronic components and electronic board coated with the hydrophobic coating film are immersed in water, and the low temperature of the water that comes into contact with the entire hydrophobic coating film is used to cool the equipment.

[0014] On the other hand, the water-cooling method disclosed in Patent Document 4 involves storing a semiconductor device in a flexible bag and immersing it in a coolant such as water to achieve cooling.

[0015] Computer systems used in supercomputers, data centers, and the like generally include an electronic circuit board, various electronic components (such as a CPU (Central Processing Unit) and / or a GPU (Graphics Processing Unit, general-purpose GPU (GPGPU: General Purpose Graphics Processing Unit)), high-speed memory, a chipset, power supply-related components such as FETs (Field Effect Transistors), electrolytic capacitors, a network unit, a bus switch unit, and SSDs (Solid State Drives)) mounted on the electronic board, and a PSU (Power Supply Unit) for supplying power to the electronic components.

[0016] However, recent computer systems have electronic boards with various electronic components and PSUs mounted on them that have complex three-dimensional structures, and it is expected that establishing a technology to completely coat such a computer system with a defect-free thin film will be technically challenging.

[0017] In addition, in the method of cooling a computer system by encasing it in a flexible bag and immersing it in a coolant such as water, it is desirable to closely fit the outer surface of the computer system to the inner surface of the bag as tightly as possible, thereby increasing the heat transfer from the computer system to the surrounding coolant. However, as the three-dimensional structure of computer systems becomes more complex, it is expected that meeting this requirement will become more technically difficult.

[0018] A non-immersion liquid cooling system requires piping to distribute the coolant from the coolant distribution unit to each rack, piping to distribute to each electronic board within the rack, and piping to distribute to each individual heat-generating element (semiconductor device such as a CPU or GPU) on each electronic board. However, due to the repeated distribution, restrictions on the pipe diameter and increased pressure loss limit the flow rate of the coolant flowing through each cold plate, which limits the cooling capacity of the heat-generating elements.

[0019] Therefore, an object of the present invention is to solve the above-mentioned problems of the conventional technology and to provide an improved cooling method and cooling system for immersion cooling of computer systems that are three-dimensionally complex.

[0020] Another object of the present invention is to provide a cooling method and system for a computer system that can solve the problems of reduced refrigerant flow rate per heat generating element and limited cooling capacity when distributing refrigerant through piping.

[0021] In order to solve the above-mentioned problems, according to one aspect of the present invention, there is provided a method for cooling a computer system, the method comprising: dividing a computer system including two or more heat generating elements into a first segment including one or more first heat generating elements and a second segment including one or more second heat generating elements; waterproofing the first segment by covering the first segment with a waterproof coating or a waterproof bag; and electrically connecting the waterproofed first segment and the non-waterproofed second segment; wherein the waterproofed first segment includes one or more first cold plates thermally connected to the one or more first heat generating elements. The present invention provides a method for cooling a computer system, wherein the non-waterproof second segment includes one or more second cold plates thermally connected to one or more second heat generating elements, a coolant is circulated through a circulation flow path connecting the heat exchanger, the first pump, and the immersion tank with a main pipe, and a second pump is connected to the circulation path with a secondary pipe so that a portion of the coolant is branched off and circulated within the second cold plate, the waterproof first segment is immersed in the coolant within the immersion tank, and a third pump fluidly connected to one or more first cold plates circulates a portion of the coolant within the immersion tank within the first cold plate.

[0022] In a preferred embodiment of the cooling method, the first segment may include an electronic board on which various electronic components are mounted, the one or more first heat generating elements may include one or more GPUs, such as a CPU or a GPGPU, the second segment may include a power supply unit (PSU), and the second heat generating elements may include one or more voltage converters or power semiconductors.

[0023] In a preferred embodiment of the cooling method, the coolant is water, and the third pump may be a submersible pump.

[0024] Furthermore, in a preferred embodiment of the above cooling method, the power supply terminal of the first segment that has been waterproofed and the power supply terminal of the second segment that has not been waterproofed may be electrically connected via a conductive path, which may be a bus bar or an electrical cable.

[0025] In a preferred embodiment of the cooling method, the immersion tank is formed by a bottom wall, one or more side walls, and one or more partition walls that are shorter in height than the one or more side walls, and the coolant that passes over the partition walls is allowed to flow outside the immersion tank. One or more partition walls may be disposed in the immersion tank to form a drain section.

[0026] Furthermore, in a preferred embodiment of the above cooling method, the immersion tank is divided into two or more immersion sections by one or more partition walls arranged within the immersion tank, and the coolant is allowed to flow into the immersion tank from a coolant inlet provided in each of the two or more immersion sections, and the coolant that has passed over the partition walls is allowed to flow out of the immersion tank from a coolant outlet provided in a drain section.

[0027] In a preferred embodiment of the cooling method, the second pump may adjust the flow rate or volume of the coolant through the secondary pipe in response to the temperature of the second heating element, and the third pump may adjust the flow rate or volume of the coolant through the first cold plate in response to the temperatures of one or more first heating elements.

[0028] In a preferred embodiment of the cooling method, a part or the whole of the first cold plate may be disposed inside the waterproof coating or the waterproof bag, or outside the waterproof coating or the waterproof bag.

[0029] In a preferred embodiment of the cooling method, when an opening having an area larger than the area of ​​the first heating element is formed in the waterproof coating or the waterproof bag, and a part or all of the first cold plate is disposed outside the waterproof coating or the waterproof bag, a peripheral area of ​​the opening of the waterproof coating or the waterproof bag and at least one surface of the first cold plate are watertightly connected, and the surface of the first heating element and at least one surface of the first cold plate may be thermally connected through the opening of the waterproof coating or the waterproof bag.

[0030] According to another aspect of the present invention, there is provided a method for cooling a computer system, the method comprising: dividing a computer system including two or more heat-generating elements into a first segment including one or more first heat-generating elements and a second segment including one or more second heat-generating elements; waterproofing the first segment by covering it with a waterproof coating or a waterproof bag; immersing the waterproofed first segment in an immersion bath through which a coolant flows through a circulating path to cool the first segment; and locally forcibly cooling the first or more first heat-generating elements using the coolant in the immersion bath by one or more first cold plates fluidly connected to a pump; and forcibly cooling the second segment, which is not waterproofed, by one or more second cold plates fluidly connected to another pump, using a portion of the coolant branched off from the circulating path to forcibly cool the second or more second heat-generating elements.

[0031] According to yet another aspect of the present invention, there is provided a cooling system including a computer system, an immersion tank, a heat exchanger, first, second, and third pumps, a main pipe constituting a coolant circulation path connecting the heat exchanger, the first pump, and the immersion tank, and a secondary pipe constituting a coolant branch path branching from the circulation path and returning to the circulation path. The computer system may include: a first segment waterproofed by being covered with a waterproof coating or a waterproof bag, the first segment including one or more first heating elements and one or more first cold plates thermally connected to the one or more first heating elements; a second segment not waterproofed, the second segment including one or more second heating elements and one or more second cold plates thermally connected to the one or more second heating elements; and a conductive path electrically connecting the first segment and the second segment. The first segment may be immersed in a coolant in the immersion bath, and a third pump may be fluidly connected to one or more first cold plates to circulate a portion of the coolant in the immersion bath through the first cold plates. A second pump may be fluidly connected to the secondary pipe to circulate the coolant through the branch path through the second cold plate.

[0032] A computer system including two or more heat-generating elements is divided into a first segment including one or more first heat-generating elements and a second segment including one or more second heat-generating elements, the first segment being waterproofed and immersed in an immersion bath through which a coolant (preferably water) flows, and the first heat-generating elements are locally forcedly cooled by a first cold plate fluidly connected to a third pump using the coolant in the immersion bath. The second segment is not waterproofed, and the second heat-generating elements are forcedly cooled by a second cold plate fluidly connected to a second pump using a portion of the coolant branched from the circulation path. In other words, a computer system with improved cooling capacity can be constructed by applying different cooling requirements (immersion cooling and local forced cooling for the former, and non-immersion forced cooling for the latter) to the first and second segments while using a single coolant. For example, a computer system can be divided into a segment that includes a first heat-generating element with a relatively large heat generation amount and is relatively easy to waterproof, and a segment that includes a second heat-generating element with a relatively small heat generation amount and is relatively difficult to waterproof, and these segments can be cooled. In this way, even if the computer system becomes more three-dimensionally complex, the computer system can be divided into, for example, a portion that has a relatively flat structure and is easy to stably waterproof, and a portion that has a relatively three-dimensional structure and is difficult to stably waterproof, and different cooling requirements can be applied to these segments while using a single coolant. Furthermore, as described above, this solves the problem of conventional non-immersion liquid cooling systems, namely, the restriction of the piping diameter and increased pressure loss in the piping paths from the coolant distribution unit to each cold plate, which limits the flow rate of the coolant flowing through each cold plate and limits the cooling capacity of the heat-generating elements. In particular, since the abundant coolant circulating in the immersion bath is used to flow the coolant to each cold plate using a pump, sufficient coolant flow rate and cooling capacity can be ensured to cool each heat-generating element.

[0033] The above and other objects and advantages of the present invention will be more clearly understood through the following description of the embodiments, although the embodiments described below are merely examples and the present invention is not limited thereto.

[0034] 1 is an explanatory diagram showing the overall configuration of a cooling system; 2 is a perspective view showing an example of a cooling system; 3 is a partial perspective view showing an example of a cooling system; 4 is a plan view showing an example of a cooling system; 5 is a partial perspective view showing an example of a cooling system; 6 is a schematic view showing an example of a second segment that is not waterproofed; 7 is a view showing an example of a first segment that has been waterproofed; 8 is a partial cross-sectional view showing an example of a first segment that has been waterproofed; 9 is a partial cross-sectional view showing another example of a first segment that has been waterproofed; 10 is a view showing another example of a first segment that has been waterproofed; 11 is a partial cross-sectional view showing another example of a first segment that has been waterproofed; 12 is a partial cross-sectional view showing yet another example of a first segment that has been waterproofed; 13 is a view showing yet another example of a first segment that has been waterproofed;

[0035] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of a cooling method and a cooling system for a computer system according to the present invention will be described in detail below with reference to the accompanying drawings.

[0036] 1 to 7, the cooling system 1 includes a computer system 30, an immersion tank 11, a heat exchanger 15, a first pump 13, a second pump 23, and a third pump, a main pipe PP that forms a refrigerant circulation path connecting the heat exchanger 15, the first pump 13, and the immersion tank 11, and a sub-pipe SP that forms a branch path for the refrigerant that branches off from the circulation path and returns to the circulation path.

[0037] The computer system 30 includes a first segment 100 that is waterproofed by covering it with a waterproof coating or a waterproof bag 41, a second segment 33 that is not waterproofed, and a conductive path 35 that electrically connects the first segment 100 and the second segment 33.

[0038] The first segment 100 includes one or more first heating elements 312 and one or more first cold plates 25 thermally connected to each of the first heating elements 312. The second segment 33 includes one or more second heating elements 331 and one or more second cold plates 21 thermally connected to the second heating elements 331.

[0039] 1 and 3, the coolant circulation path formed by the main pipe PP specifically includes a path starting from the inlet 15a of the coolant to be cooled of the heat exchanger 15, running from the main pipe P1-the suction port 13a of the pump 13-the outlet 13b-the main pipe P2-the T-pipe T1-the main pipes P3 and P4-the coolant inlets 11a and 11a of the immersion tank 11-the coolant outlets 11b and 11b-the main pipes P5 and P6-the T-pipe T2-the coolant outlet 15b of the heat exchanger 15. For convenience of illustration, arrows indicating the direction of coolant flow are added next to the circulation path in FIG. 1, and hollow arrows indicating the flow of coolant within the immersion tank 11 are added. The main pipe and the T-pipe, together with the pump 13 and the heat exchanger 15, may be disposed below the rack 17.

[0040] 1 and 5, the branching paths of the coolant constituted by the sub-pipe SP specifically include a path starting from the open end of the sub-pipe S1 (inside the immersion tank 11) through the sub-pipe S1-suction port 23a-exhaust port 23b-coolant inlet 21a-coolant outlet 21b of the cold plate 21-sub-pipe S2-open end of the sub-pipe S2 (inside the immersion tank 11). For ease of illustration, arrows indicating the direction of coolant flow are added next to the branching paths in FIG. 1. The sub-pipes may be arranged above the rack 17 together with the immersion tank 11, the pump 23, and the second component 33.

[0041] The first segment 100 is immersed in the coolant 12 in the immersion tank 11. A third pump 28 disposed in the first segment 100 is fluidly connected to the cold plate 25. The third pump 28 may be a submersible pump, which draws in a portion of the coolant 12 in the immersion tank 11 and circulates it within the cold plate 25. Outside the immersion tank 11, a second pump 23 is fluidly connected to the sub-pipe SP, which draws in a portion of the coolant from within the immersion tank 11 and circulates the coolant 12 passing through the branch path within the cold plate 25.

[0042] Generally, a computer system can be divided into an electronic board portion, which includes one or more CPUs or GPUs and various other electronic components mounted on the board, and a PSU portion, which includes one or more voltage converters or power semiconductors. The electronic board corresponds to a first segment separated from the computer system, and the PSU corresponds to a second segment separated from the computer system. The CPU, GPU, voltage converter, and power semiconductors are merely examples, but the CPU or GPU corresponds to a first heat-generating element, and the PSU board on which the voltage converter or power semiconductor is mounted within the PSU housing corresponds to a second heat-generating element.

[0043] 7, one embodiment of the electronic board 100 corresponding to the first segment has four CPUs or GPUs arranged in a row mounted on one surface of the board (PCB) 311, and the backside of the cold plate is thermally connected to each of the CPUs or GPUs. The top of the electronic board 100 is provided with DC voltage input terminals 32a and 32b to which a conductive path 35 is electrically connected, and a connector terminal 36 to which a communication network cable 37 is connected.

[0044] The electronic board 100 is preferably waterproofed except for the DC voltage input terminals 32a, 32b and the connector terminal 36. Waterproofing will be described in detail later. The illustrated electronic board 100 has a shape in which heating elements of the same shape are aligned on the board, and therefore has a relatively flat and simple structure, making it easy to perform stable waterproofing. Because the electronic board 100 is waterproofed, it can be cooled by immersing it in an immersion tank 11 filled with ordinary water. In the illustrated example, a total of 16 electronic boards 100, eight on each side, can be immersed in the immersion tank 11 for cooling.

[0045] As shown in FIG. 6 , the PSU 33, which corresponds to the second segment, includes a PSU board 331 mounted with one or more voltage converters or power semiconductors (not shown) inside a rectangular parallelepiped housing, and a cold plate 21 thermally connected to one surface of the PSU board. A fluid flow path (microchannel) is formed within the body of the cold plate 21. The flow path may be formed to increase the fluid contact area, for example, by forming mesh-like holes, zigzag holes, or spaces with multiple fins or pins arranged at predetermined intervals within the body. One surface of the housing is provided with the refrigerant inlet 21a and refrigerant outlet 21b of the cold plate 21 and a power plug 21c, while the other surface of the housing is provided with DC voltage output terminals 34a and 34b. In the illustrated example, two PSU boards 331 are thermally connected to the front and back surfaces of the cold plate 21, respectively. The PSU board 331 corresponds to a second heating element. The internal structure of the PSU 33 housing is more three-dimensional and complex than that of the electronic board 100. For this reason, it is expected that waterproofing the PSU board 331, which is mounted with various electronic components including voltage converters and power semiconductors, will be extremely difficult. Therefore, the non-waterproof PSU board 331 is thermally connected to a cold plate 21, through which a portion of the coolant flows, to achieve non-immersion cooling. In the illustrated example, one PSU is assigned to four electronic boards immersed in the coolant in the immersion bath 11, and power is supplied to 16 electronic boards 100 from four PSUs 33 arranged around the immersion bath 11. Note that sub-pipes SP and pumps 23 may be provided individually for each PSU 33.

[0046] The DC voltage input terminal 32a of the electronic board 331 and the DC voltage output terminal 34a of the PSU 33, and the DC voltage input terminal 32b and the DC voltage output terminal 34b are electrically connected via bus bars 35. An electric cable may be used instead of the bus bar. The bus bar or the electric cable corresponds to the conductive path.

[0047] The heat exchanger 15 for cooling the refrigerant circulating through the circulation path may be, for example, a plate-type heat exchanger. A cooled external refrigerant is supplied to the cooling refrigerant inlet 15c of the heat exchanger 15 from a chiller (not shown) or from groundwater, well water, river water, industrial water, etc., and heat exchange occurs within the heat exchanger 15 with the refrigerant 41 to be cooled. The warmed refrigerant is discharged from the cooling refrigerant outlet 15d and returned to the chiller (not shown). The cooling refrigerant path can be configured using piping. For ease of illustration, the cooling refrigerant path is omitted from FIG. 1 , and arrows indicating the flow direction of the cooling refrigerant are added to the cooling refrigerant inlet 15c and cooling refrigerant outlet 15d of the heat exchanger 15.

[0048] 1 , the immersion tank 11 may be formed by a bottom wall 111, one or more side walls 112, and one or more partition walls 113 that are shorter in height than the one or more side walls, and the coolant 12 that passes over the partition walls 113 may be allowed to flow outside the immersion tank 11. One or more partition walls 113 may be arranged inside the immersion tank 11 to form a drain section. For example, the immersion tank 11 may be divided into two immersion sections by two partition walls 113 arranged inside the immersion tank, and the coolant 12 flowing through the main pipe PP may be caused to flow into the immersion tank from coolant inlets 11a provided in each of the two immersion sections, and the coolant that passes over the partition walls 113 may be caused to flow out of the main pipe PP to the outside of the immersion tank from coolant outlets 11b provided in the drain section. The appropriate performance of the pump 13 and heat exchanger 15 and the pipe diameter of the main pipe PP may be determined in consideration of the flow rate or amount of the coolant flowing near the surface of the waterproofed first segment.

[0049] 1, one open end (intake side) and the other open end (discharge side) of the sub-pipe SP placed in the immersion tank 11 may be located in the immersion section and the drain section, respectively, but may be arbitrarily changed to any other location as long as they are located within the circulation path. The PSU 33 may be communicably connected to an external controller (not shown) via a connector terminal 36 of the PSU 33 and a network communication cable. For example, the temperature of the PSU board 331 may be detected by a temperature sensor (not shown), and the pump 23 may adjust the flow rate or amount of the coolant passing through the sub-pipe SP in accordance with a command from the controller corresponding to the temperature of the PSU board.

[0050] Next, with reference to FIGS. 7 and 14, one example and another example of the waterproofed first segment, that is, the waterproofed electronic board, will be described in more detail.

[0051] 7 and 8 , a waterproof electronic board 100 shown as an example includes a waterproof bag 41 covering an electronic board including a printed circuit board (PCB) 311 and a plurality of heating elements 312 mounted on the board 311, a plurality of cold plates 25, and a submersible pump 28. When the cold plate 25 is attached to the main body, the cold plate 25 includes a fluid inlet 27 and a fluid outlet 29. A flow path (microchannel) through which a fluid passes is formed within the main body of the cold plate 25. The flow path may be formed to increase the area in contact with the fluid, and may be, for example, a mesh-like hole, a zigzag hole, or a space with a number of fins or pins arranged at predetermined intervals within the main body.

[0052] In the illustrated example, all of the multiple cold plates 25 may be disposed outside the waterproof bag 41, and at least one surface (e.g., the back surface) of the body of each of the multiple cold plates 25 may be thermally connected to each of the multiple heating elements 312. In this case, an opening 45 having an area larger than the area of ​​the heating element 312 may be formed in the waterproof bag 41, and the surrounding area of ​​the opening 45 and the back surface of the body of the cold plate 25 may be watertightly connected via a bonding layer 51.

[0053] The joining layer 51 is formed by using a dissimilar material joining film, double-sided adhesive tape, or watertight packing to connect the back surface of the body of the cold plate 25 to the outer surface of the waterproof bag. An opening having a shape similar to that of the opening 45 may be formed in the dissimilar material joining film, double-sided adhesive tape, or watertight packing that forms the joining layer 51.

[0054] An example of a dissimilar material bonding film for forming the bonding layer 51 is the "Metaseal" (product name of Fujimori Kogyou Co., Ltd.) series. Because this dissimilar material bonding film is formed into a film of uniform thickness, it can be sandwiched between the back surface of the cold plate 25 and the outer surface of the waterproof bag 41 and heat-pressed to form a bonding layer bonding the cold plate 25 and the waterproof bag 41. A heat press or an iron-type heater can be used for heat-pressing, and the bonding process between the cold plate 25 and the waterproof bag 41 can be completed easily and in a short time (several seconds or less). The method for forming the bonding layer 51 is not limited to heat-pressing, and various methods, such as pressure, ultrasonic waves, electromagnetic waves, and light irradiation, can also be used.

[0055] By forming the bonding layer 51 from a dissimilar material bonding film, it is possible to achieve a surface bond between the back surface of the cold plate 25 and the outer surface of the waterproof bag 41 with a uniform film thickness and no variation in adhesive strength.

[0056] Another example of a dissimilar material bonding film that forms a bonding layer is "WelQuick" (a Resonac product name) manufactured by Resonac Corporation. This dissimilar material bonding film utilizes the solid-liquid phase change of the film material to complete the bonding process in a short time (a few seconds). It also allows for reheating after bonding to allow for peeling and re-adhesion. This allows for easy recovery of waterproofed electronic boards from cooling systems after a certain period of use, and for easy removal of the cold plate 25 from the waterproof bag 41, resulting in high resource reusability.

[0057] Here, the dissimilar material bonding film can be preferably a pre-formed sheet or film that can be cut and used. However, this is not limited thereto. For example, if a certain environment is established in which various conditions, including the film thickness and shape, can be appropriately controlled, a bonding layer 51 formed from the dissimilar material bonding film can be obtained starting with a liquid or gel adhesive material. Specifically, as an example, a mold is first placed on the surface of the cold plate and filled with the liquid or gel adhesive material, thereby forming a coating of the adhesive material of the desired shape and volume on the surface of the cold plate. Next, with the coating of the adhesive material in contact with the back surface of the cold plate and the outer surface of the waterproof bag, the coating of the adhesive material can be solidified by methods such as heat and pressure bonding, pressure, ultrasound, electromagnetic waves, or light irradiation. In this way, a bonding layer formed from the dissimilar material bonding film can be obtained starting from the liquid or gel adhesive material.

[0058] Alternatively, the bonding layer may be formed using double-sided adhesive tape. For example, a high-strength acrylic foam tape (e.g., 3M VHB Tape) can be cut to the desired size and shape to prepare the double-sided adhesive tape for the bonding layer. The bonding layer for bonding the cold plate and the waterproof bag can be formed by pressing one side of the double-sided adhesive tape to the back surface of the cold plate 25 or the outer surface of the waterproof bag, and then pressing the outer surface of the waterproof bag or the back surface of the cold plate to the other side of the double-sided adhesive tape. The use of double-sided adhesive tape has the advantage that it does not require a heat treatment and therefore is easy to form the bonding layer. When using double-sided adhesive tape, it is preferable to arrange the cut tape so that the bonding layer forms a continuous, closed band surrounding the opening 45 of the bag 41.

[0059] Alternatively, the bonding layer may be formed by a watertight packing. For example, various rubber packings (typically O-rings) in a continuous, closed strip or line shape may be used. The watertight packing is placed between the back surface of the cold plate and the outer surface of the waterproof bag so as to surround the periphery of the opening 45 of the bag 41. The cold plate is then fixed to the substrate by screwing or the like. In this fixed state, the watertight packing elastically deforms, applying an appropriate surface pressure in a strip or line shape to the area surrounding the opening 45 of the bag 41, thereby maintaining a watertight seal on both sides of the surrounding area.

[0060] In the illustrated example, a manifold 26 may be disposed outside the waterproof bag 41. The inlet of the manifold 26 may be fluidly connected to the outlet of the submersible pump 28, and each outlet of the manifold 26 may be fluidly connected to each fluid inlet 27 of the multiple cold plates 25.

[0061] The submersible pump 28 may suck in a portion of the coolant in the immersion tank 11 and forcibly send the coolant toward the manifold 26. The coolant distributed by the manifold 26 may pass through a fluid inlet 27, flow through a flow path formed in the body of the cold plate 25, and exit from a fluid outlet 29 into the immersion tank 11.

[0062] The waterproof bag 41 may be made of a film of synthetic resin (e.g., polyethylene, polypropylene, polyester, etc.) that is water-resistant and resistant to relatively low temperatures (e.g., 100°C or higher) and can maintain watertightness. In the illustrated example, the upper part of the waterproof bag 41 does not need to be airtight (airtight / watertight), as long as the airtightness of at least the portion immersed in the refrigerant is ensured. In this case, when the cooling system 1 is in operation, the relatively flexible waterproof bag 41 is pressed by the liquid pressure of the surrounding refrigerant, so that the inner surface of the waterproof bag 41 can be brought into relatively close contact with both sides of the substrate 311 and the various electronic components 38 mounted on the substrate.

[0063] The immersion tank 11 contains a sufficient amount of coolant to immerse the waterproofed electronic substrate 100. The coolant may be ordinary water (tap water, industrial water, seawater, etc.). The main pipe PP connected to the immersion tank 11 provides a path for discharging the coolant heated in the immersion tank 11 from the immersion tank 11 and for returning the coolant cooled by the heat exchanger 15 to the immersion tank 11. When the waterproofed electronic substrate 100 is immersed in the coolant, the coolant flowing over its surface cools the entire waterproofed electronic device 100, and the coolant forced to pass through the flow path formed in the body of the cold plate 25 locally and powerfully removes heat from the heat-generating element 312.

[0064] FIG. 9 shows another example of a waterproof electronic device. This waterproof electronic board differs from the waterproof electronic board 100 shown in FIGS. 7 and 8 in that a submersible pump 28 is connected to each of the fluid inlets 27 of multiple cold plates 25 in the waterproof electronic board 110 without using a manifold. This example has many advantages over connecting multiple submersible pumps 28 to each of the multiple cold plates 25. For example, the temperature of each heating element can be detected and the flow rate or volume of the coolant drawn in and discharged from each submersible pump can be adjusted based on the detected temperature. In other words, the flow rate or volume of the fluid passing through the cold plate can be individually controlled. This enables appropriate temperature management in an electronic board containing multiple heating elements according to the heat generation of the heating elements.

[0065] 10 and 11 show yet another example of a waterproofed electronic board. This waterproofed electronic board differs from the waterproofed electronic board 100 shown in FIGS. 7 and 8 in that, in the waterproofed electronic device 200, all of the cold plates 25 are disposed inside the waterproof bag 41, the waterproof bag 41 does not have any openings for thermal connection, and the openings of the fluid outlets 29 of the cold plates 25 in the waterproofed electronic board 200 are disposed outside the waterproof bag 41. Note that, in order to dispose the openings of the fluid outlets 29 of the cold plates 25 outside the waterproof bag 41, the fluid outlets 29, which may be, for example, pipes, must pass through a penetration formed in the waterproof bag 41. In this case, it is advisable to provide an appropriate treatment, such as providing a sealant at the penetration, to maintain watertightness.

[0066] Figure 12 shows yet another example of a waterproofed electronic board, which differs from the waterproofed electronic board shown in Figure 11 in that in the waterproofed electronic board 210, each of a plurality of submersible pumps 28 is fluidly connected to a fluid inlet 27 of each of a plurality of cold plates 25 without the use of a manifold.

[0067] Figure 13 shows yet another example of a waterproofed electronic board. This waterproofed electronic board differs from the waterproofed electronic device 200 shown in Figure 10 in that, in a waterproofed electronic device 300, the fluid outlet 29 of one of the adjacent cold plates 25 (four cold plates in the illustrated example) is fluidly connected to the fluid inlet 27 of the other cold plate via connecting piping 24, and one submersible pump 28 is fluidly connected to the fluid inlet 27 of the lowest cold plate without using a manifold. In other words, the multiple cold plates 25 are connected via connecting piping 24, and the submersible pump 28 forces the coolant to flow through them.

[0068] Fig. 14 shows yet another example of a waterproofed electronic board. This waterproofed electronic board differs from the waterproofed electronic device 300 shown in Fig. 13 in that in the waterproofed electronic device 310, at least one surface of the main body of one cold plate 25 is thermally connected to multiple heating elements (four heating elements in the illustrated example). In other words, the multiple cold plates in the example of Fig. 13 may be integrated to form a single cold plate.

[0069] Although an example of a cooling system and a plurality of waterproofed electronic circuit boards have been described above with reference to the drawings, further modifications may be made to the details of the components. For example, while the example of waterproofing an electronic circuit board by covering it with a waterproof bag has been described, other examples of waterproofing an electronic device include covering the electronic device with a parylene thin film, which is electrically non-conductive and water-impermeable (see, for example, Kazuki Fujiwara et al., Information Processing Society of Japan Research Report High Performance Computing (HPC), "The First Step Toward a Direct Natural Water-Cooled Computer," 2017-HPC-158(5), pp. 1-5 (March 1, 2017). URL: http: / / research.nii.ac.jp / ~koibuchi / pdf / ikki-sighpc158.pdf). Another example is covering the electronic device with an ultra-nanohydrophobic coating thin film using silicon compound nanoparticles, which is electrically non-conductive and water-impermeable (see, for example, U.S. Pat. No. 10,717,881). When an electronic device has a relatively flat structure, it is easy to perform such waterproofing treatment.

[0070] In addition, the waterproofed electronic board may include, instead of a network communication cable, a wireless communication unit (not shown) inside the waterproof bag 41 that enables wireless communication between the waterproofed electronic board and the outside.

[0071] The present invention can be widely applied to techniques for cooling a computer system containing two or more heat generating elements by immersing it in a conductive coolant such as ordinary water, tap water, or seawater, or in a non-conductive coolant.

[0072] REFERENCE SIGNS LIST 1 Cooling system 11 Immersion tank 111 Bottom wall 112 Side wall 113 Partition wall 11a Coolant inlet 11b Coolant outlet 12 Coolant 13 Pump (first pump) 13a Intake port 13b Outlet 15 Heat exchanger 15a Cooled coolant inlet 15b Cooled coolant outlet 15c Cooling coolant inlet 15d Cooling coolant outlet 17 Rack PP, P1, P2, P3, P4, P5, P6 Main pipe T1, T2 T-pipe SP, S1, S2, S3 Sub-pipe 21 PSU substrate cold plate (second cold plate) 21a Coolant inlet 21b Coolant outlet 21c Power plug 23 Pump (second pump) 23a Intake port 23b Outlet 24 Connecting pipe 25 Cold plate (first cold plate) 26 Manifold 27 Fluid inlet 28 Submersible pump (third pump) 29 Fluid outlet 30 Computer system 100, 110, 200, 210, 300, 310 Waterproofed electronic board (first segment) 311 Board (PCB) 312 Heating element (first heating element) 313 Waterproof bag 32a, 32b DC voltage input terminal 33 PSU (second segment) 331 PSU board (second heating element) 34a, 34b DC voltage output terminal 35 Bus bar (conductive path) 36 Connector terminal 37 Network communication cable 38 Electronic component 41 Waterproof bag 45 Opening 51 Bonding layer

Claims

1. Dividing a computer system including two or more heat generating elements into a first segment including one or more first heat generating elements and a second segment including one or more second heat generating elements, waterproofing the first segment by covering the first segment with a waterproof coating or a waterproof bag, electrically connecting the waterproofed first segment to the non-waterproofed second segment, the waterproofed first segment including one or more first cold plates thermally connected to the one or more first heat generating elements, and the non-waterproofed second segment including one or more second cold plates thermally connected to the one or more second heat generating elements, circulating a coolant through a circulation flow path connecting a heat exchanger, a first pump, and the immersion tank with a main pipe, connecting a second pump to the circulation path with a sub-pipe, thereby branching a portion of the coolant and circulating it within the second cold plate, a third pump fluidly connected to the one or more first cold plates to circulate a portion of the coolant in the immersion bath through the first cold plate; 2. The cooling method according to claim 1, wherein the first segment includes an electronic board on which various electronic components are mounted, the one or more first heat generating elements include one or more GPUs (Graphics Processing Units) including a CPU (Central Processing Unit) or a GPGPU (General Purpose Graphics Processing Unit), the second segment includes a power supply unit (PSU), and the second heat generating elements include one or more voltage converters or power semiconductors.

3. The cooling method of claim 1, wherein the coolant is water and the third pump is a submersible pump.

4. The cooling method according to claim 1, wherein the waterproofed power terminal of the first segment and the non-waterproofed power terminal of the second segment are electrically connected via a conductive path.

5. The cooling method according to claim 1, wherein the immersion tank is formed by a bottom wall, one or more side walls, and one or more partition walls that are shorter in height than the one or more side walls, and the coolant that passes over the partition walls is caused to flow outside the immersion tank.

6. The cooling method according to claim 5, wherein the one or more partition walls are disposed in the immersion bath to form a drain portion.

7. The cooling method according to claim 6, wherein the immersion tank is divided into two or more immersion sections by the one or more partition walls arranged in the immersion tank, the coolant is caused to flow into the immersion tank from a coolant inlet provided in each of the two or more immersion sections, and the coolant that has passed over the partition walls is caused to flow out of the immersion tank from a coolant outlet provided in the drain section.

8. The cooling method according to claim 1, wherein the second pump adjusts the flow rate or volume of the refrigerant passing through the secondary pipe in accordance with the temperature of the second heating element.

9. The cooling method of claim 1, wherein the third pump adjusts the flow rate or volume of the coolant through the first cold plate depending on the temperature of the one or more first heating elements.

10. The cooling method of claim 1, wherein a part or all of the first cold plate is disposed inside the waterproof coating or the waterproof bag, or outside the waterproof coating or the waterproof bag.

11. The cooling method according to claim 10, wherein an opening having an area larger than an area of ​​the first heating element is formed in the waterproof coating or the waterproof bag, and when a part or all of the first cold plate is disposed outside the waterproof coating or the waterproof bag, a peripheral area of ​​the opening in the waterproof coating or the waterproof bag and at least one surface of the first cold plate are watertightly connected, and the surface of the first heating element and the at least one surface of the first cold plate are thermally connected through the opening in the waterproof coating or the waterproof bag.

12. A method of cooling a computer system, comprising: dividing a computer system including two or more heat generating elements into a first segment including one or more first heat generating elements and a second segment including one or more second heat generating elements; waterproofing the first segment by covering the first segment with a waterproof coating or a waterproof bag; immersing and cooling the waterproofed first segment in an immersion bath through which a coolant flows through a circulating path, and using the coolant in the immersion bath to locally and forcibly cool the one or more first heat generating elements by one or more first cold plates fluidly connected to a pump; and forcibly cooling the one or more second heat generating elements in the second segment, which is not waterproofed, by one or more second cold plates fluidly connected to another pump, using a portion of the coolant branched off from the circulating path.

13. A computer system comprising: an immersion tank; a heat exchanger; first, second, and third pumps; a main pipe constituting a circulation path for a coolant connecting the heat exchanger, the first pump, and the immersion tank; and a secondary pipe constituting a branch path for the coolant that branches off from the circulation path and returns to the circulation path, wherein the computer system comprises: a first segment that is waterproofed by being covered with a waterproof coating or a waterproof bag, the first segment including one or more first heating elements and one or more first cold plates thermally connected to the one or more first heating elements; a second segment that is not waterproofed, the second segment including one or more second heating elements and one or more second cold plates thermally connected to the one or more second heating elements; and a conductive path that electrically connects the first segment and the second segment, A cooling system for a computer system, wherein the first segment is immersed in the coolant in the immersion bath, the third pump is fluidly connected to the one or more first cold plates and circulates a portion of the coolant in the immersion bath through the first cold plate, and the second pump is fluidly connected to the secondary piping and circulates the coolant through the branch path through the second cold plate.

14. The cooling system of claim 13, wherein the refrigerant is water and the third pump is a submersible pump.

15. The cooling system of claim 13, wherein the first segment includes an electronic board on which various electronic components are mounted, the one or more first heat generating elements include one or more GPUs (Graphics Processing Units) including a CPU (Central Processing Unit) or a GPGPU (General Purpose Graphics Processing Unit), the second segment includes a power supply unit (PSU), and the second heat generating element includes one or more voltage converters or power semiconductors.

16. The cooling system of claim 13, wherein the conductive path is a bus bar or an electrical cable.

17. The cooling system of claim 13, wherein the immersion tank is formed by a bottom wall, one or more side walls, and one or more partition walls that are shorter in height than the one or more side walls, and the coolant that passes over the partition walls is allowed to flow outside the immersion tank.

18. The cooling system of claim 17, wherein the one or more partitions are disposed within the immersion bath to form a drain portion.

19. The cooling system described in claim 18, wherein the immersion tank is divided into two or more immersion sections by the one or more partition walls arranged in the immersion tank, and the coolant is caused to flow into the immersion tank from a coolant inlet provided in each of the two or more immersion sections, and the coolant that has passed over the partition walls is caused to flow out of the immersion tank from a coolant outlet provided in the drain section.

20. The cooling system of claim 13, wherein the second pump adjusts the flow rate or volume of the refrigerant passing through the secondary pipe in accordance with the temperature of the second heating element.

21. The cooling system of claim 13, wherein the third pump adjusts the flow rate or volume of the coolant through the first cold plate depending on the temperature of the one or more first heating elements.

22. The cooling system of claim 13, wherein a part or all of the first cold plate is disposed inside the waterproof coating or the waterproof bag, or outside the waterproof coating or the waterproof bag.

23. The cooling system of claim 22, wherein an opening having an area larger than an area of ​​the first heating element is formed in the waterproof coating or the waterproof bag, and when a part or all of the first cold plate is disposed outside the waterproof coating or the waterproof bag, a peripheral area of ​​the opening in the waterproof coating or the waterproof bag and at least one surface of the first cold plate are watertightly connected, and a surface of the first heating element and the at least one surface of the first cold plate are thermally connected through the opening in the waterproof coating or the waterproof bag.

Images