Cooling structure and cooling system for computer system

Abstract

[Problem] To solve the problems of the lowering of the flow rate of a refrigerant per heating element, and the limitation of cooling capacity thereof, when the refrigerant is distributed to a cold plate by piping. [Solution] This cooling structure for cooling a computer system by immersing the computer system in a refrigerant includes: a waterproof coating or a waterproof bag for covering an electronic substrate including one or more first heating elements; one or more cold plates thermally connected to the one or more first heating elements; a pump for supplying a portion of the refrigerant to the cold plate; and an integrated member for fluidly connecting the cold plate and the pump in a vertical direction or a horizontal direction. A part or all of the cold plate is disposed inside the waterproof coating or the waterproof bag, or outside the waterproof coating or the waterproof bag, and a fluid passage connecting a fluid outlet of the pump and a fluid inlet of the cold plate is formed in the integrated member.

Description

Cooling structures and cooling systems for computer systems

[0001] The present invention relates to a cooling structure and a cooling system for a computer system, and more particularly to a cooling structure and a cooling system for a computer system that requires high-performance operation, stable operation, or low power consumption operation, and generates a large amount of heat itself, such as supercomputers, data centers, artificial intelligence (AI) processing systems, quantum computing systems, cryptographic processing systems, or blockchain processing systems, by immersing it in a coolant.

[0002] One of the biggest challenges determining the performance limits of computer-related equipment in recent years is power consumption, and the importance of research on power efficiency, especially for supercomputers, is already widely recognized. In other words, speed performance per watt (Flops / W) has become one of the indicators for evaluating supercomputers. Furthermore, it is said that more than 30% of the total power consumption of data centers is spent on cooling, and there is a growing demand for reducing power consumption by improving cooling efficiency. In addition, recent global warming and extreme heat waves caused by extreme weather events have made it impossible to cool data centers in the summer using conventional cooling methods, and improving the cooling capacity of existing systems has become an urgent matter. Moreover, the rapid growth of artificial intelligence processing, the increasing need for encryption processing, the surge in cryptocurrency mining processing, a major example of blockchain processing, and the development of the metaverse, which is predicted to grow rapidly in many areas, all necessitate a geometric increase in the processing power of data centers and computer systems, and the demand for increased cooling capacity has become very large.

[0003] For the cooling of supercomputers and data centers, air cooling and liquid cooling have conventionally been used. Liquid cooling generally has good cooling efficiency because it uses a liquid with significantly better heat transfer performance than air. In addition, liquid cooling includes immersion cooling and non-immersion cooling. As an example of immersion cooling, in "TSUBAME-KFC" constructed by the Tokyo Institute of Technology, an immersion cooling system using synthetic oil achieved 4.50 GFlops / W and won the first place in the "Supercomputer Green500 List" announced in November 2013 and June 2014.

[0004] On the other hand, an immersion cooling system using a fluorocarbon-based coolant instead of synthetic oil or mineral oil has been proposed. Specifically, it is an example of using a fluorocarbon-based coolant (hydrofluoroether (HFE) compounds known as 3M's product names "Novec (trademark of 3M. The same applies hereinafter) 7100", "Novec 7200", "Novec 7300") (for example, Patent Document 1, Patent Document 2).

[0005] In addition to these, since 2014, the present inventor has developed a new immersion cooling system that circulates a non-evaporative coolant mainly composed of a perfluoride in the free space of a cooling tank to directly cool electronic devices and a series of technologies around it (for example, Patent Document 3).

[0006] However, regarding PFAS (Perfluoroalkyl Substances and Polyfluoroalkyl Substances) including all of these fluorocarbon-based coolants, adverse effects on the human body, crops, and the natural environment have been pointed out since before. In December 2022, the world's largest manufacturer announced that it will completely stop production by the end of 2025, and in Europe, both production and use are expected to be legally prohibited within a few years. Therefore, there is a need for a new immersion cooling method that does not use any harmful compounds such as PFAS as a refrigerant, is more excellent in cooling capacity compared to immersion cooling methods using PFAS as a refrigerant, and is cheaper and can be widely used around the world.

[0007] A prime example of a cooling method that does not use any harmful compounds such as PFAS as a refrigerant is one that can use ordinary water (tap water or industrial water) rather than pure water. Specifically, a cooling method has been proposed in which semiconductor chips mounted on a circuit board are placed in a flexible bag, the bag is immersed in a container filled with water as a cooling liquid, and the bag is contracted and deformed by the pressure difference between the inside and outside of the bag when the water is filled around the bag or when the pressure inside the bag is reduced, thereby making the bag adhere tightly to the semiconductor device (for example, Patent Document 4).

[0008] Furthermore, as another cooling method that utilizes water, natural water-cooled computers have been proposed, which use river, lake, seawater, or tap water as a cooling source to directly cool the computer. Specifically, this is a cooling method in which a computer with a parylene resin coating on its substrate surface is immersed in water. (For example, Non-Patent Document 1).

[0009] Furthermore, a method has been proposed in which the entire substrate is completely covered with an ultra-nano hydrophobic coating thin film using silicon compound nanoparticles to create an electronic device with excellent waterproof and moisture-resistant properties, and then cooled by immersing it in water (for example, Patent Document 5).

[0010] Another example of non-immersion cooling is a liquid cooling system for a computer system in which multiple electronic circuit boards (servers) are housed in individual racks within multiple racks. In this system, a cold plate is thermally connected to each semiconductor device on each electronic circuit board in each rack, and a coolant is circulated through piping that connects a Coolant Distribution Unit (CDU), which includes a heat exchanger and pump, to each cold plate for cooling. In this non-immersion liquid cooling system, the semiconductor devices, which are the main heat-generating elements, can be liquid-cooled, but other electronic components need to be air-cooled.

[0011] Japanese Patent Publication No. 2013-187251, Japanese Patent Publication No. 2012-527109, Japanese Patent No. 5853072, Japanese Patent No. 2804640, U.S. Patent No. 10717881. Fujiwara, Ikki et al., Information Processing Society of Japan Research Report, High-Performance Computing (HPC) "A First Step Towards a Direct Natural Water Cooled Computer", 2017-HPC-158(5), pp.1-5 (March 1, 2017). URL: http: / / research.nii.ac.jp / ~koibuchi / pdf / ikki-sighpc158.pdf

[0012] The cooling system disclosed in Non-Patent Document 1 employs a method in which the entire electronic component and electronic substrate are completely covered with a parylene thin film that does not conduct electricity and does not allow water to pass through, and the entire electronic component and electronic substrate covered with the parylene thin film are immersed in water or seawater, and the equipment is cooled using the low temperature of the water or seawater that comes into contact with the entire parylene thin film.

[0013] Similarly, the cooling system disclosed in Patent Document 5 also employs a method in which the entire electronic component and electronic substrate are completely covered with an ultra-nano hydrophobic coating thin film made of silicon compound nanoparticles that do not conduct electricity and do not allow water to pass through, and the entire electronic component and electronic substrate covered with the hydrophobic coating thin film is immersed in water, and the equipment is cooled using the low temperature of the water that comes into contact with the entire hydrophobic coating thin film.

[0014] On the other hand, the water-cooling method disclosed in Patent Document 4 involves placing the semiconductor device in a flexible bag and immersing it in a refrigerant such as water to achieve cooling.

[0015] Computer systems used in supercomputers and data centers generally include an electronic circuit board, various electronic components mounted on the circuit board (CPU (Central Processing Unit) and / or GPU (Graphics Processing Unit, General Purpose Graphics Processing Unit (GPGPU)), high-speed memory, chipset, power supply-related components such as FETs (field-effect transistors), electrolytic capacitors, network units, bus switch units, SSDs (Solid State Drives), etc.), and a PSU (Power Supply Unit) for supplying power to the electronic components.

[0016] In a non-immersion liquid cooling system that circulates refrigerant through piping connecting a refrigerant distribution unit and individual cold plates, piping is required to distribute the refrigerant from the distribution unit to each rack, to distribute it to each electronic circuit board within the rack, and further to distribute it to the individual heat-generating components (such as CPUs or GPUs) on each electronic circuit board. However, the repeated distribution of refrigerant limits the flow rate of the refrigerant through each cold plate due to constraints on pipe diameter and increased pressure loss, thereby limiting the cooling capacity of the heat-generating components.

[0017] Therefore, there is a need for a cooling system that can solve the problems of conventional non-immersion liquid cooling systems.

[0018] Therefore, the object of the present invention is to provide a cooling structure and a cooling system for a computer system that can solve the problems of reduced refrigerant flow rate per heat source and limitations in cooling capacity when distributing refrigerant to a cold plate via piping.

[0019] Another object of the present invention is to provide a cooling structure for cooling a computer system by immersion in a refrigerant, wherein a pump for supplying a portion of the refrigerant to a cold plate and the cold plate are integrated, and a cooling system for a computer system including the cooling structure.

[0020] To solve the above problems, according to one aspect of the present invention, a cooling structure is provided for cooling a computer system by immersing it in a coolant, comprising: a waterproof film or waterproof bag for covering an electronic circuit board including one or more first heat-generating elements; one or more cold plates thermally connected to one or more first heat-generating elements; a pump for supplying a portion of the coolant to the cold plates; and an integrated member for fluidly connecting the cold plates and the pump in a vertical or horizontal direction, wherein part or all of the cold plates are disposed inside or outside the waterproof film or waterproof bag, and a fluid passage connecting the fluid outlet of the pump and the fluid inlet of the cold plates is formed within the integrated member.

[0021] In the preferred embodiment of the cooling structure described above, the refrigerant is water, and the pump may be a submersible pump.

[0022] Furthermore, in a preferred embodiment of the cooling structure described above, the pump may be a fan motor pump including a fluid inlet and a fluid outlet.

[0023] Furthermore, in a preferred embodiment of the cooling structure described above, the pump may include a casing with a fluid inlet and a fluid outlet, an impeller that applies pressure to the fluid within the casing, and an ultra-thin motor that drives the impeller.

[0024] Furthermore, in a preferred embodiment of the cooling structure described above, when an opening is formed in a waterproof coating or waterproof bag, and part or all of the cold plate is disposed outside the waterproof coating or waterproof bag, the area surrounding the opening of the waterproof coating or waterproof bag and at least one surface of the cold plate may be watertightly connected, and the surface of one or more heating elements and at least one surface of the cold plate may be thermally connected through the opening of the waterproof coating or waterproof bag.

[0025] Furthermore, in a preferred embodiment of the cooling structure described above, the pump may adjust the flow rate or flow rate of the refrigerant supplied to the cold plate according to the temperature of one or more heat-generating elements.

[0026] In a preferred embodiment of the cooling structure described above, one or more first heat-generating elements may include one or more CPUs or GPUs including a GPGPU.

[0027] In another aspect of the present invention, a cooling system is provided which includes a computer system, an immersion tank, a heat exchanger, a first pump, and piping that constitutes a refrigerant circulation path connecting the heat exchanger, the first pump, and the immersion tank. The computer system may include a first segment which is waterproofed and includes one or more first heating elements, a second segment which is not waterproofed and includes one or more second heating elements, and a conductive path which electrically connects the first segment and the second segment. The first segment is a cooling structure that includes a waterproof coating or waterproof bag covering an electronic substrate including one or more first heating elements, one or more cold plates thermally connected to one or more first heating elements, a second pump fluidly connected to the cold plates to supply a portion of the refrigerant to the cold plates, and an integrated member fluidly connecting the cold plates and the pump in a vertical or horizontal direction, wherein part or all of the cold plates are disposed inside or outside the waterproof coating or waterproof bag, and a fluid passage connecting the fluid outlet of the second pump and the fluid inlet of the cold plates is formed within the integrated member. The first segment may be immersed in a refrigerant in a liquid immersion tank.

[0028] In the preferred embodiment of the cooling system described above, the refrigerant is water, and the second pump may be a submersible pump.

[0029] In a preferred embodiment of the cooling system described above, the second pump may be a fan motor pump including a fluid inlet and a fluid outlet.

[0030] Furthermore, in a preferred embodiment of the cooling system described above, the second pump may include a casing with a fluid inlet and a fluid outlet, an impeller that applies pressure to the fluid within the casing, and an ultra-thin motor that drives the impeller.

[0031] Furthermore, in a preferred embodiment of the cooling system described above, when an opening is formed in a waterproof coating or waterproof bag, and part or all of the cold plate is disposed outside the waterproof coating or waterproof bag, the area surrounding the opening of the waterproof coating or waterproof bag and at least one surface of the cold plate may be watertightly connected, and the surface of the first heating element and at least one surface of the cold plate may be thermally connected through the opening of the waterproof coating or waterproof bag.

[0032] In a preferred embodiment of the cooling system described above, the first segment may include an electronic substrate on which various electronic components are mounted, and one or more first heating elements may include one or more GPUs, including a CPU or GPGPU; the second segment may include a PSU, and one or more second heating elements may include one or more voltage converters or power semiconductors.

[0033] In a preferred embodiment of the above cooling system, the conductive path may be a busbar or an electrical cable.

[0034] In a preferred embodiment of the cooling system described above, the second pump may adjust the flow rate or flow rate of the refrigerant supplied to the first cold plate according to the temperature of one or more first heating elements.

[0035] In a preferred embodiment of the cooling system described above, the second segment, which is not waterproofed, includes a heat dissipation surface for transferring heat from one or more second heat-generating elements to the outside, the heat dissipation surface is thermally connected to a heat-conductive wall of the immersion tank, and the one or more second heat-generating elements may be forcibly cooled using a coolant circulating within the immersion tank.

[0036] The computer system can be configured to use a portion of the refrigerant in which it is immersed to locally force-cool a first heat-generating element via a cold plate to which a pump is fluid-connected. Since the cold plate and the pump are fluid-connected longitudinally or transversely by an integrated component, and a fluid passage connecting the fluid outlet of the pump and the fluid inlet of the cold plate is formed within the integrated component, there is no need to use piping connecting the pump and the cold plate. Therefore, all the problems in conventional non-immersed liquid cooling systems including cold plates, namely the limitations on the flow rate of refrigerant flowing through each cold plate and the resulting limitations on the cooling capacity of the heat-generating element due to the constraints of pipe diameter and increased pressure loss in the piping path from the refrigerant distribution unit to each cold plate, can be solved.

[0037] The above-mentioned objectives and advantages of the present invention, as well as other objectives and advantages, will be more clearly understood through the following description of embodiments. However, the embodiments described below are illustrative and the present invention is not limited thereto.

[0038] This is an explanatory diagram showing an example of a refrigerant circulation path in the overall configuration of a cooling system. This is an explanatory diagram showing an example of a second segment that is not waterproofed in the overall configuration of a cooling system. This is a perspective view showing an example of a cooling system. This is a partial perspective view showing an example of a cooling system. This is a partial perspective view showing an example of a cooling system. This is a partial cross-sectional view showing a second segment that is not waterproofed and an immersion tank. This is a diagram showing an example of a waterproofed first segment. This is a partial cross-sectional view showing a cooling structure in an example of a waterproofed first segment. This is a diagram showing another example of a waterproofed first segment. This is a partial cross-sectional view showing a cooling structure in another example of a waterproofed first segment.

[0039] Hereinafter, several preferred embodiments of the cooling structure and computer system cooling system according to the present invention will be described in detail with reference to the drawings.

[0040] Referring to Figures 1 to 6, the cooling system 1 includes a computer system 30, an immersion tank 11, a heat exchanger 15, a first pump 13, and piping PP that constitutes a refrigerant circulation path connecting the heat exchanger 15, the first pump 13, and the immersion tank 11.

[0041] The computer system 30 includes a waterproof first segment 100, a non-waterproof second segment 33, and a conductive path 35 that electrically connects the first segment 100 and the second segment 33.

[0042] Referring to Figures 7 and 8, the first segment 100 includes one or more first heating elements 312. The first segment 100 includes a cooling structure 10. The cooling structure 10 includes a waterproof coating or waterproof bag 41 covering a substrate (PCB) 311 including one or more first heating elements 312, one or more cold plates 21 thermally connected to one or more first heating elements 312, a second pump 23 fluidly connected to the cold plates 21 to supply a portion of the refrigerant to the cold plates 21, and an integrated member 25 that fluidly connects the cold plates 21 and the pump 23 in the vertical direction. As shown, part or all of the cold plates 21 may be disposed on the outside of the waterproof coating or waterproof bag 41, or part or all of the cold plates 21 may be disposed on the inside of the waterproof coating or waterproof bag 41. The fluid inlet 26 of the second pump 23 may be formed on the upper side of the integrated member 25, the fluid passage 251 connecting the fluid outlet of the second pump 23 and the fluid inlet of the cold plate 21 may be formed inside the integrated member 25, and the fluid outlet 29 of the cold plate 21 may be formed on one side of the integrated member 25. If the first segment 100 includes a plurality of first heating elements, the first segment 100 may include a plurality of cooling structures 10, and each of the plurality of cooling structures 10 may correspond to each of the plurality of first heating elements 312. On the other hand, as shown in Figure 6, the second segment 33 includes one or more second heating elements 332.

[0043] In the cooling structure 10 shown in FIGS. 7 and 8, the second pump 13 may be a submersible pump, for example, a fan motor pump that drives an impeller 232 by a fan motor. The cold plate 21 may include a microgrid 211.

[0044] As shown in FIGS. 1, 2, and 3, the refrigerant circulation path constituted by the pipes PP specifically starts from the cooled refrigerant inlet 15a of the heat exchanger 15, and then passes through the pipe P1 - the suction port 13a of the pump 13 - the discharge port 13b - the pipe P2 - the T-shaped pipe T1 - the pipes P3, P4 - the refrigerant inlets 11a, 11a of the immersion tank 11 - the refrigerant outlets 11b, 11b - the pipes P5, P6 - the T-shaped pipe T2 - the cooled refrigerant outlet 15b of the heat exchanger 15. For the sake of illustration, in FIGS. 1 and 2, it should be noted that arrows indicating the direction of refrigerant flow are added horizontally to the circulation path, and white arrows indicating the flow of refrigerant in the immersion tank 11 are added. The pipes and the T-shaped pipe may be arranged at the lower part of the rack 17 together with the pump 13 and the heat exchanger 15.

[0045] The waterproofed first segment 100 is immersed in the refrigerant 12 in the immersion tank 11. A part of the refrigerant may pass through the inside of the cooling structure 10. As shown in FIGS. 7 and 8, due to the negative pressure generated when the impeller 232 of the second pump 23 rotates, a part of the refrigerant is sucked in from the fluid inlet 26 of the second pump 23. The refrigerant pressurized by the impeller 232 in the second pump forms a radial forced downward flow toward the microgrid 211 through the fluid passage 251 and is discharged from the fluid outlet 29 of the cold plate 21.

[0046] Generally, a computer system can be divided into a part of an electronic substrate on which one or more CPUs or GPUs and various other electronic components are mounted on a substrate, and a part of a PSU that includes one or more voltage converters or power semiconductors. Such an electronic substrate corresponds to the first segment divided from the computer system, and the PSU corresponds to the second segment divided from the computer system. The CPU, GPU, voltage converter, and power semiconductor are only examples. The CPU or GPU corresponds to the first heat-generating body, and the PSU substrate on which the voltage converter or power semiconductor is mounted in the PSU housing corresponds to the second heat-generating body.

[0047] In one embodiment of the electronic substrate 100 corresponding to the first segment, in the example shown in FIGS. 7 and 8, one CPU or GPU is mounted on one surface of the substrate (PCB) 311, and the back surface of the cold plate is thermally connected to the CPU or GPU. On the upper part of the electronic substrate 100, DC voltage input terminals 32a and 32b to which the conductive path 35 is electrically connected, and a connector terminal 36 to which the communication network cable 37 is connected are provided.

[0048] The electronic substrate 100 may be waterproofed except at the locations of the DC voltage input terminals 32a and 32b and the connector terminal 36. A detailed description of the waterproofing treatment will be given later. Since the illustrated electronic substrate 100 has a shape in which a plate-like or tile-like CPU or GPU is arranged on the substrate, it has a relatively planar and simple structure and is a part where stable waterproofing treatment is easy. Since the electronic substrate 100 is waterproofed, the electronic substrate 100 can be immersed in a general liquid immersion tank 11 filled with water for cooling. In the illustrated example, 8 electronic substrates 100 can be immersed and cooled on the left and right in the liquid immersion tank 11, for a total of 16.

[0049] The PSU 33, which corresponds to the second segment, includes a plurality of PSU components 332 inside a rectangular parallelepiped housing, as shown in Figures 3, 5, and 6. One side of the housing is provided with DC voltage output terminals 34a and 34b, a connector terminal 36, and a power plug (not shown). In the illustrated example, the plurality of PSU components 332 inside the housing may include a PSU substrate, capacitors mounted on the PSU substrate, one or more voltage converters or power semiconductors, etc. At least one of these plurality of PSU components 332, typically one or more voltage converters or power semiconductors, corresponds to the second heat source. The internal structure of the housing of the PSU 33 is more three-dimensional and complex than that of the electronic circuit board 100. Therefore, it is expected that waterproofing all of these plurality of PSU components will be considerably difficult. Therefore, a heat sink 335 may be installed on another surface of the housing (the surface facing the side wall 112 of the immersion tank 11), and the heat sink 335 may form a heat dissipation surface for transferring heat from the PSU component 332, which is not waterproofed, to the outside. Then, as shown in Figure 6, the heat sink 335 may be thermally connected to the outer wall surface of the side wall 112 of the immersion tank 11. When the refrigerant 12 (preferably water) flows through the immersion tank 11, the refrigerant 12 absorbs heat from the heat-conductive wall surface of the immersion tank 11. The wall surface (side wall 112) of the immersion tank 11 is made to function as a cold plate for the PSU 33, thereby forcibly cooling the PSU component 332 without immersion. The heat sink 335 may preferably be an aluminum plate or a copper plate having excellent heat conductivity. When thermally connecting the heat sink 335 and the outer wall surface of the side wall 337, a heat transfer layer 337 made of thermal grease may be interposed, or it may not be necessary to interpose the heat transfer layer 337. As will be described later, the side wall 112 of the immersion tank 11 may preferably be formed from an aluminum plate or a copper plate, but when the heat sink 335 of the PSU 33 and the side wall 112 of the immersion tank 11 are made of different metals, it is preferable to interpose a non-metallic spacer (not shown) and a heat transfer layer 337 between the heat sink 335 and the outer wall surface of the side wall 112 to avoid galvanic corrosion.

[0050] Regarding the electrical connection between the electronic circuit boards 100 and the PSUs 33, in the illustrated example, the DC voltage output terminals of the four PSUs 33 are connected in parallel, and the four PSUs 33 are assigned to the four electronic circuit boards 100 that are immersed in the refrigerant in the immersion tank 11. Power is supplied to the 16 electronic circuit boards 100 from the 16 PSUs 33 arranged around the immersion tank 11. The number of electronic circuit boards 100 and the number of PSUs 33 included in the computer system 30 are arbitrary and are not limited to the illustrated example.

[0051] The DC voltage input terminal 32a of the electronic circuit board 331 and the DC voltage output terminal 34a of the PSU 33, and the DC voltage input terminal 32b and the DC voltage output terminal 34b are electrically connected via busbars 35. However, for the sake of illustration, in Figure 6, the busbar 35 is shown for one of the DC voltage output terminals 34a of the PSU 33, but the busbar 35 for the other DC voltage output terminal 34b is omitted from the illustration. Electrical cables may be used instead of busbars. Busbars or electrical cables correspond to conductive paths.

[0052] A heat exchanger 15 for cooling the refrigerant circulating in the circulation path may be, for example, a plate heat exchanger. Cooled external refrigerant is supplied to the cooling refrigerant inlet 15c of the heat exchanger 15 from a chiller (not shown) or from groundwater, well water, river water, industrial water, etc. Heat exchange takes place with the refrigerant 41 to be cooled within the heat exchanger 15, and the heated refrigerant is discharged from the cooling refrigerant outlet 15d and returned to the chiller (not shown). The cooling refrigerant path can be configured using piping. However, for the sake of illustration, the cooling refrigerant path is omitted in Figure 1, and arrows indicating the direction of flow of the cooling refrigerant are added to the cooling refrigerant inlet 15c and cooling refrigerant outlet 15d of the heat exchanger 15.

[0053] As shown in Figure 1, the immersion tank 11 may be formed by a bottom wall 111, one or more side walls 112, and one or more partition walls 113 that are lower in height than one or more side walls, and the refrigerant 12 that exceeds the partition walls 113 may be allowed to flow out of the immersion tank 11. One or more partition walls 113 may be arranged inside the immersion tank 11 to form a drain section. For example, the immersion tank 11 may be divided into two immersion sections by two partition walls 113 arranged inside the immersion tank, and the refrigerant 12 flowing through the piping PP may flow into the immersion tank from a refrigerant inlet 11a provided in each of the two immersion sections, and the refrigerant that exceeds the partition walls 113 may flow out of the piping PP outside the immersion tank from a refrigerant outlet 11b provided in the drain section. The appropriate performance of the pump 13 and heat exchanger 15 and the diameter of the piping PP may be specified considering the flow velocity or flow rate of the refrigerant flowing near the surface of the waterproofed first segment.

[0054] At least the side walls 112 of the immersion tank 11 may be formed from an aluminum plate or a copper plate having high thermal conductivity. For example, if the bottom wall 111 is formed from a stainless steel plate, the side walls 112 can be watertightly fixed to the bottom wall 111 using corner connecting members made of acrylic resin or polycarbonate to avoid galvanic corrosion.

[0055] Next, with reference to Figures 7 to 10, we will describe in more detail an example and other examples of a waterproofed first segment, i.e., a waterproofed electronic circuit board.

[0056] Referring again to Figures 7 and 8, an example of a waterproof electronic circuit board 100 includes a heating element 312 mounted on a substrate (PCB) 311. The electronic circuit board 100 includes a cooling structure 10. The cooling structure 10 includes a waterproof bag 41 covering the substrate (PCB) 311 including the first heating element 312, a cold plate 21 thermally connected to the first heating element 312, a pump 23 fluidly connected to the cold plate 21 to supply a portion of the coolant to the cold plate 21, and an integrated member 25 that fluidly connects the cold plate 21 and the pump 23 in the vertical direction.

[0057] In the illustrated example, all of the cold plates 21 may be arranged on the outside of the waterproof bag 41, and one side (e.g., the back surface) of each cold plate 21 may be thermally connected to the heating element 312. In this case, an opening 45 with an area larger than the area of ​​the heating element 312 may be formed in the waterproof bag 41, and the area surrounding the opening 45 and the back surface of the cold plate 21 may be watertightly connected via a bonding layer 51.

[0058] The bonding layer 51 is formed by using a dissimilar material bonding film, double-sided adhesive tape, or watertight packing to connect the back surface of the cold plate 21 to the outer surface of the waterproof bag. The dissimilar material bonding film, double-sided adhesive tape, or watertight packing forming the bonding layer 51 may have an opening similar in shape to the opening 45.

[0059] As an example of a dissimilar material bonding film that forms the bonding layer 51, the "MetaSeal" series (Fujimori Kogyou's product name) manufactured by Fujimori Kogyou Co., Ltd. can be used. Since this dissimilar material bonding film is formed into a film of uniform thickness, it can be sandwiched between the back surface of the cold plate 21 and the outer surface of the waterproof bag 41 and heat-pressed to form a bonding layer that joins the cold plate 21 and the waterproof bag 41. A heat press or an iron-type heater can be used for heat-pressing, and the bonding process between the cold plate 21 and the waterproof bag 41 can be completed easily and quickly (in a few seconds or less). The method for forming the bonding layer 51 is not limited to heat-pressing, and various methods such as pressurization, ultrasound, electromagnetic waves, and light irradiation may be used.

[0060] By forming the bonding layer 51 from a dissimilar material bonding film, it is possible to achieve surface bonding between the back surface of the cold plate 21 and the outer surface of the waterproof bag 41 with a uniform film thickness and no variation in adhesive strength.

[0061] Another example of a dissimilar material bonding film that forms the bonding layer is "WelQuick" (Resonac Corporation's product name), manufactured by Resonac Corporation. This dissimilar material bonding film utilizes the solid-liquid phase change of the film material, allowing the bonding process to be completed in a short time (a few seconds), and also enabling peeling and re-bonding by reheating after bonding. Therefore, it is easy to recover waterproof electronic circuit boards from cooling systems after a certain period of use and peel the cold plates 21 from the waterproof bags 41, resulting in high resource recyclability.

[0062] Here, the dissimilar material bonding film can preferably be a sheet or film that has been pre-formed into shape and then cut. However, it is not limited to this, and for example, if a certain environment is in place that allows for appropriate control of various conditions, including the film thickness and shape, the bonding layer 51 formed from the dissimilar material bonding film can be obtained starting from a liquid or gel-like adhesive material. Specifically, as an example, first, a mold is placed on the surface of a cold plate and filled with a liquid or gel-like adhesive material to form a coating of adhesive material of a desired shape and volume on the surface of the cold plate. Next, with the coating of adhesive material in contact with the back surface of the cold plate and the outer surface of the waterproof bag, the coating of adhesive material can be solidified by methods such as heating and pressing, pressurizing, ultrasonic waves, electromagnetic waves, or light irradiation. In this way, a bonding layer formed from a dissimilar material bonding film can be obtained starting from a liquid or gel-like adhesive material.

[0063] Alternatively, the bonding layer may be formed from double-sided adhesive tape. For example, a high-strength acrylic foam tape (for example, 3M's product name "3M VHB Tape") can be cut to the desired size and shape to prepare double-sided adhesive tape for the bonding layer. A bonding layer can be formed to join the cold plate and the waterproof bag by pressing one side of the double-sided adhesive tape against the back surface of the cold plate 21 or the outer surface of the waterproof bag, and then pressing the other side of the double-sided adhesive tape against the outer surface of the waterproof bag or the back surface of the cold plate. The use of double-sided adhesive tape has the advantage that it does not require heat treatment, making it easy to form the bonding layer. When using double-sided adhesive tape, it is preferable to arrange the cut tape so that the bonding layer forms a continuous and closed band surrounding the opening 45 of the bag 41.

[0064] Alternatively, the bonding layer may be formed by a watertight packing. As an example, various types of rubber packings, in the form of a continuous and closed strip or line (typically an O-ring), can be used. The watertight packing is placed between the back surface of the cold plate and the outer surface of the waterproof bag so as to surround the opening 45 of the bag 41. The cold plate is then fixed to the substrate by screwing or the like. In this fixed state, the watertight packing elastically deforms, applying appropriate surface pressure in a strip or line to the area surrounding the opening 45 of the bag 41, and can maintain a watertight seal on both sides of the surrounding area.

[0065] In the cooling structure 10, a portion of the refrigerant is drawn in through the fluid inlet 26 of the pump 23 by the negative pressure generated when the impeller 232 of the pump 23 rotates. The refrigerant, pressurized by the impeller 232 within the pump 23, flows through the fluid passage 251 and into the microgrid 211, forming a radial forced downward flow, which is then discharged through the inside of the microgrid 211 and out of the fluid outlet 29 of the cold plate 21.

[0066] The waterproof bag 41 may be made of a film of a synthetic resin (e.g., polyethylene, polypropylene, polyester, etc.) that has water resistance and heat resistance to relatively low temperatures (e.g., 100°C or higher) and can maintain watertightness. In the illustrated example, the top of the waterproof bag 41 does not need to be airtight (airtight / watertight), and it is sufficient that at least the portion immersed in the refrigerant is airtight. In this case, when the cooling system 1 is in operation, the relatively flexible waterproof bag 41 is pressed by the liquid pressure of the surrounding refrigerant, allowing the inner surface of the waterproof bag 41 to come into contact with both sides of the substrate 311 and the various electronic components 38 mounted on the substrate to a certain extent.

[0067] The immersion tank 11 contains a sufficient amount of refrigerant to immerse the waterproofed electronic circuit board 100. The refrigerant can be ordinary water (tap water, industrial water, or seawater, etc.). The piping PP connected to the immersion tank 11 provides a passage for discharging the refrigerant heated in the immersion tank 11 and returning the refrigerant cooled by the heat exchanger 15 to the immersion tank 11. When the waterproofed electronic circuit board 100 is immersed in the refrigerant, the refrigerant flowing over its surface cools the entire waterproofed electronic device 100. In the cooling structure 10, the refrigerant, drawn in from the fluid inlet 26 of the pump 23 and forced to pass through the microgrid 211 of the cold plate 21, locally and powerfully removes heat from the heat-generating element 312. In this configuration, the cold plate 21 and the pump 23 are fluidly connected in the vertical direction, and a fluid passage 251 connecting the fluid outlet of the pump 23 and the fluid inlet of the cold plate 21 is formed within the integrated member 25. Therefore, there is no need to use piping to connect the pump and the cold plate. Consequently, all the problems in conventional non-immersed liquid cooling systems including cold plates, namely the limitations on the pipe diameter and increased pressure loss in the piping path from the refrigerant distribution unit to each cold plate, which restricts the flow rate of refrigerant through each cold plate and limits the cooling capacity of the heat-generating element, can be solved.

[0068] Furthermore, by connecting the pump 23 and the cold plate 21 vertically with an integrated member, a cooling structure 10 for local forced cooling without piping is realized. Consequently, the cooling structure 10 has excellent volumetric efficiency, which contributes to reducing the weight of the waterproofed electronic circuit board 100. In addition, using a fan motor pump for the pump 23 is advantageous as it allows for the realization of a thin (low-profile) cooling structure 10. These advantages of the cooling structure 10 are particularly useful when configuring a cooling system in which multiple waterproofed electronic circuit boards 100 are densely immersed in a liquid immersion tank 11.

[0069] The waterproof electronic circuit board 100 may detect the temperature of the heating element 312, and the pump 23 of the cooling structure 10 may adjust the flow rate or flow rate of the refrigerant supplied to the cold plate 21 according to the detected temperature. This makes it possible to adjust the cooling capacity of the cooling structure 10 according to the amount of heat generated by the heating element 312 and to perform appropriate temperature control.

[0070] Figures 9 and 10 show other examples of waterproof electronic circuit boards. The waterproof electronic circuit board 200 includes a cooling structure 20. The cooling structure 20 includes a waterproof bag 41 covering a circuit board (PCB) 311 including a first heating element 312, a cold plate 21 thermally connected to the first heating element 312, a pump 23 fluidly connected to the cold plate 21 to supply a portion of the coolant to the cold plate 21, and an integrated member 25 that fluidly connects the cold plate 21 and the pump 23 laterally. As shown, part or all of the cold plate 21 may be disposed outside the waterproof coating or waterproof bag 41, or part or all of the cold plate 21 may be disposed inside the waterproof coating or waterproof bag 41. The fluid inlet 26 of the pump 23 may be formed on the upper side of the integrated member 25, the fluid passage 251 connecting the fluid outlet 27 of the pump 23 and the fluid inlet 28 of the cold plate 21 may be formed inside the integrated member 25, and the fluid outlet 29 of the cold plate 21 may be formed on one side of the integrated member 25. If the first segment 200 includes a plurality of first heating elements, the first segment 200 may include a plurality of cooling structures 20, and each of the plurality of cooling structures 20 may correspond to each of the plurality of first heating elements 312.

[0071] In the cooling structure 20 shown in Figures 9 and 10, the pump 23 may include a casing 231 with a fluid inlet 26 and a fluid outlet 27, an impeller 232 that applies pressure to the fluid within the casing 231, and an ultra-thin motor 233 that drives the impeller 232.

[0072] In the cooling structure 20, a portion of the refrigerant is drawn in through the fluid inlet 26 of the pump 23 by the negative pressure generated when the impeller 232 rotates within the casing 231. The refrigerant, pressurized by the impeller 232 within the pump 23, is forced to flow through the fluid passage 251 toward one side of the microgrid 211, and is discharged through the inside of the microgrid 211 from the fluid outlet 29 of the cold plate 21.

[0073] In the cooling structure 20, the refrigerant, which is drawn in from the fluid inlet 26 of the pump 23 and forced to pass through the microgrid 211 of the cold plate 21, locally and powerfully removes heat from the heat-generating element 312. At this time, the cold plate 21 and the pump 23 are fluidly connected laterally, and a fluid passage 251 connecting the fluid outlet 27 of the pump 23 and the fluid inlet 28 of the cold plate 21 is formed within the integrated member 25, so there is no need to use piping to connect the pump and the cold plate. Therefore, similar to the cooling structure 10, the cooling structure 20 can also solve the aforementioned problems in conventional non-immersion liquid cooling systems that include a cold plate.

[0074] Furthermore, by connecting the pump 23 and the cold plate 21 laterally with an integrated member, a cooling structure 20 for local forced cooling without piping is realized. Consequently, the cooling structure 20 has excellent volumetric efficiency, which contributes to the weight reduction of the waterproofed electronic circuit board 200. In addition, using a pump 23 that includes a casing 231, an impeller 232, and an ultra-thin motor 233 is advantageous because it enables the realization of a thin (low-profile) cooling structure 20. These advantages of the cooling structure 20 are particularly useful when configuring a cooling system in which multiple waterproofed electronic circuit boards 200 are densely immersed in a liquid immersion tank 11.

[0075] The above describes an example of a cooling system, a cooling structure, and several examples of waterproofed electronic circuit boards with reference to the drawings, but further modifications to the details of the components are possible. For example, as a specific example of waterproofing an electronic circuit board, an example of covering it with a waterproof bag was shown, but other specific examples of waterproofing an electronic device include, firstly, covering the electronic device with a parylene thin film that does not conduct electricity and does not allow water to pass through (see, for example, Ikki Fujiwara et al., Information Processing Society of Japan Research Report High Performance Computing (HPC) "A First Step Towards a Direct Natural Water Cooled Computer" 2017-HPC-158(5), pp.1-5 (March 1, 2017). URL: http: / / research.nii.ac.jp / ~koibuchi / pdf / ikki-sighpc158.pdf). Secondly, covering the electronic device with an ultra-nanohydrophobic coating thin film using silicon compound nanoparticles that does not conduct electricity and does not allow water to pass through (see, for example, U.S. Patent No. 1,0717881). Electronic devices with a relatively flat structure are easier to waterproof.

[0076] Furthermore, instead of a network communication cable, a waterproof electronic circuit board may contain a wireless communication unit (not shown) within a waterproof bag 41 that enables wireless communication between the waterproof electronic circuit board and the outside.

[0077] On the other hand, the PSU does not have to include a heat sink. In this case, at least one surface of the PSU component itself may form a heat dissipation surface for transferring heat from the PSU component to the outside. Furthermore, the PSU may further include a cooling fan, which may additionally cool part or all of the PSU component inside the enclosure.

[0078] Furthermore, the PSU is not limited to being installed on the outer wall side of the immersion tank's side wall. For example, an upper open compartment for installing the PSU may be formed on the inner wall side of the immersion tank's side wall. The upper open compartment may be surrounded at its bottom and perimeter by an aluminum or copper plate and may be formed to be watertight to prevent refrigerant from entering the interior of the upper open compartment. The PSU may be installed within the upper open compartment such that its heat dissipation surface is thermally connected to the inner wall surface of the upper open compartment. Even with this configuration, the inner wall surface of the upper open compartment functions as a heat-conducting wall surface of the immersion tank, and the PSU component can be forcibly cooled using the refrigerant circulating within the immersion tank.

[0079] This invention can be widely applied to techniques for cooling computer systems that generate a large amount of heat by immersing them in conductive or non-conductive refrigerants such as ordinary water, tap water, or seawater.

[0080] 1 Cooling System 10, 20 Cooling Structure 11 Immersion Tank 111 Bottom Wall 112 Side Wall 113 Partition 11a Refrigerant Inlet 11b Refrigerant Outlet 12 Refrigerant 13 Pump (First Pump) 13a Inlet 13b Outlet 15 Heat Exchanger 15a Refrigerant Inlet to be Cooled 15b Refrigerant Outlet to be Cooled 15c Cooling Refrigerant Inlet 15d Cooling Refrigerant Outlet 17 Rack PP, P1, P2, P3, P4, P5, P6 Piping T1, T2 T-pipe 21 Cold Plate 211 Microgrid 23 Submersible Pump (Second Pump) 231 Casing 232 Impeller 233 Ultra-thin Motor 25 Integrated Component 251 Fluid Passage 26 Fluid Inlet 27 Fluid Outlet 28 Fluid inlet 29 Fluid outlet 30 Computer system 100, 200 Waterproof electronic circuit board (first segment) 311 Circuit board (PCB) 312 Heating element (first heating element) 32a, 32b DC voltage input terminal 33 PSU (second segment) 332 PSU component (second heating element) 335 Heat sink 337 Heat transfer layer 34a, 34b DC voltage output terminal 35 Busbar (conductive path) 36 Connector terminal 37 Network communication cable 38 Electronic component 41 Waterproof bag 45 Opening 51 Bonding layer

Claims

1. A cooling structure for cooling a computer system by immersion in a refrigerant, comprising: a waterproof film or waterproof bag for covering an electronic circuit board including one or more first heat-generating elements; one or more cold plates thermally connected to the one or more first heat-generating elements; a pump for supplying a portion of the refrigerant to the cold plates; and an integrated member for fluidly connecting the cold plates and the pump in a vertical or horizontal direction, wherein part or all of the cold plates are disposed inside or outside the waterproof film or waterproof bag, and a fluid passage is formed within the integrated member connecting the fluid outlet of the pump and the fluid inlet of the cold plates.

2. The cooling complex according to claim 1, wherein the refrigerant is water and the pump is a submersible pump.

3. The cooling structure according to claim 2, wherein the pump is a fan motor pump including a fluid inlet and a fluid outlet.

4. The cooling structure according to claim 2, wherein the pump includes a casing with a fluid inlet and a fluid outlet, an impeller that applies pressure to the fluid within the casing, and an ultra-thin motor that drives the impeller.

5. The cooling structure according to claim 1, wherein when an opening is formed in the waterproof coating or the waterproof bag, and part or all of the cold plate is disposed outside the waterproof coating or the waterproof bag, the area surrounding the opening of the waterproof coating or the waterproof bag is watertightly connected to at least one surface of the cold plate, and the surface of one or more heating elements and at least one surface of the cold plate are thermally connected through the opening of the waterproof coating or the waterproof bag.

6. The cooling structure according to claim 1, wherein the pump adjusts the flow rate or flow rate of the refrigerant supplied to the cold plate according to the temperature of the one or more heating elements.

7. The cooling structure according to claim 1, wherein the one or more heat-generating elements include one or more GPUs (Graphics Processing Units) that include a CPU (Central Processing Unit) or a GPGPU (General Purpose Graphics Processing Unit).

8. A computer system, an immersion tank, a heat exchanger, a first pump, and piping constituting a refrigerant circulation path connecting the heat exchanger, the first pump, and the immersion tank, wherein the computer system includes a first segment which is waterproofed and includes one or more first heating elements, a second segment which is not waterproofed and includes one or more second heating elements, and a conductive path which electrically connects the first segment and the second segment, wherein the first segment is a cooling structure which includes a waterproof coating or waterproof bag that covers an electronic circuit board which includes one or more first heating elements, one or more cold plates which are thermally connected to the one or more first heating elements, a second pump which is fluidly connected to the cold plates and supplies a portion of the refrigerant to the cold plates, and an integrated member which fluidly connects the cold plates and the pump in the vertical or horizontal direction, A cooling system for a computer system, comprising a cooling structure in which part or all of the cold plate is disposed inside or outside the waterproof coating or waterproof bag, and a fluid passage connecting the fluid outlet of the second pump and the fluid inlet of the cold plate is formed within the integrated member, the first segment is immersed in the refrigerant in the immersion tank.

9. The cooling system according to claim 8, wherein the refrigerant is water and the second pump is a submersible pump.

10. The cooling system according to claim 9, wherein the second pump is a fan motor pump including a fluid inlet and a fluid outlet.

11. The cooling system according to claim 9, wherein the second pump comprises a casing including a fluid inlet and a fluid outlet, an impeller that applies pressure to the fluid within the casing, and an ultrathin motor that drives the impeller.

12. The cooling system according to claim 8, wherein when an opening is formed in the waterproof coating or the waterproof bag, and part or all of the cold plate is disposed outside the waterproof coating or the waterproof bag, there is a watertight connection between the area surrounding the opening of the waterproof coating or the waterproof bag and at least one surface of the cold plate, and the surface of the first heating element and the at least one surface of the cold plate are thermally connected through the opening of the waterproof coating or the waterproof bag.

13. The cooling system according to claim 8, wherein the first segment includes an electronic substrate on which various electronic components are mounted, and the one or more first heating elements include one or more GPUs (Graphics Processing Units) including a CPU (Central Processing Unit) or a GPGPU (General Purpose Graphics Processing Unit), and the second segment includes a Power Supply Unit (PSU), and the one or more second heating elements include one or more voltage converters or power semiconductors.

14. The cooling system according to claim 8, wherein the conductive path is a busbar or an electrical cable.

15. The cooling system according to claim 8, wherein the second pump adjusts the flow rate or flow rate of the refrigerant supplied to the first cold plate according to the temperature of the one or more first heating elements.

16. The cooling system according to claim 8, wherein the unwaterproofed second segment includes a heat dissipation surface for transferring heat from the one or more second heating elements to the outside, the heat dissipation surface is thermally connected to a heat-conductive wall of the immersion tank, and the one or more second heating elements are forcibly cooled using the refrigerant flowing through the immersion tank.

Images