Immersion cooling system and data center equipped with it

Abstract

By adopting a circulating system that uses natural water such as groundwater (river water, seawater, dam water, etc.), we aim to reduce overall costs and improve the cooling efficiency, safety of operation, and lifespan of the equipment to be cooled. [Solution] The present invention is an immersion cooling system comprising an immersion tank 2 that houses the equipment to be cooled and is filled with a highly insulating liquid agent, water tanks 4A and 4B that store water, and a heat exchanger 3 that exchanges heat between the water in the water tank and the solvent.The water tank stores water supplied from a natural water source, and the water flows out into the heat exchanger via an injection mechanism, and receives heated water from the heat exchanger via a discharge mechanism and discharges it back into the natural water source.

Description

[Technical Field]

[0001] The present invention relates to a technology relating to liquid immersion cooling, which improves heat dissipation and reduces power consumption by immersing server equipment or the like in an insulating liquid. [Background technology]

[0002] The spread of digital technology has transformed global connectivity since the birth of the internet in 1995. Furthermore, the widespread adoption of communication technologies such as broadband, 4G, and 5G has transformed a wide range of industries, including communications, commerce, and entertainment.

[0003] However, this digital revolution has also come hand in hand with environmental concerns. Since the early 2010s, efforts to achieve carbon neutrality have gained momentum worldwide, and efforts have been made to improve energy efficiency, but these advances have tended to be offset by the dramatic growth of high-performance computing (HPC).

[0004] Furthermore, the evolution of generative AI, such as ChatGPT, which appeared in the second half of 2022, is expected to accelerate by 2024, further expanding the demand for computing resources. For example, the construction of new data centers and the increase in power consumption are becoming major sustainability challenges.

[0005] In this situation, conventional cooling methods such as air cooling and water cooling are finding it difficult to meet the thermal management and energy efficiency requirements of data centers where GPUs play a major role. Against this background, immersion cooling technology has attracted attention, and various proposals have been made to improve heat dissipation, reduce power consumption, and make system designs more compact by submerging server equipment in an insulating liquid.

[0006] For example, Patent Document 1 discloses a data center having an immersion tank that holds information processing devices in a cooling liquid, a sealed water-spray and air-blowing cooling type cooling device that cools the piping exposed to the outside air taken in through an outside air intake by spraying water onto the piping as the cooling liquid from the immersion tank flows through the piping, a first temperature measuring unit that measures a first temperature, which is the temperature of the cooling liquid from the cooling device, and a cooling device control unit that controls the operation of the water-spray cooling unit and the air-blowing cooling unit by controlling an ON / OFF control signal of the cooling device based on the temperature measured by the first temperature measuring unit. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Patent No. 6980969 Summary of the Invention [Problem to be solved by the invention]

[0008] However, Patent Document 1 does not disclose anything about a circulation type configuration that uses natural water such as groundwater.

[0009] The present invention has been made in consideration of these problems, and its purpose is to provide a technology that reduces overall costs by using a circulation type configuration that uses water from natural water sources such as groundwater (groundwater, river water, seawater, dam water, etc.), while improving the cooling efficiency, operational safety, and lifespan of the equipment to be cooled. [Means for solving the problem]

[0011] In order to solve the above problems, one aspect of the present invention is In the immersion cooling system according to the embodiment, the immersion tank contains a device to be cooled and is filled with a highly insulating liquid; a first water tank stores water from a natural water source; and a second water tank stores water from the water tank and the liquidThe system is equipped with a heat exchanger that exchanges heat with a natural water source, and a second water tank that stores water used in the heat exchanger, and the first water tank discharges water supplied from the natural water source into the heat exchanger, and the second water tank receives heated water from the heat exchanger, discharges some of the water into the first water tank, and discharges the rest into the natural water source. [Effects of the Invention]

[0012] According to the present invention, by adopting a circulation type configuration using water from natural water sources such as groundwater (groundwater, river water, seawater, dam water, etc.), it is possible to reduce overall costs and provide technology that improves the cooling efficiency, operational safety, and lifespan of the equipment to be cooled. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a configuration diagram of an immersion cooling system according to an embodiment of the present invention. [Figure 2] FIG. 2 is an external view of the system. [Figure 3] FIG. 3 is a diagram showing the state of a data center that employs this system. [Figure 4] FIG. 4 is a diagram showing the configuration of a control system in a data center that employs this system. [Figure 5] 5(a) and 5(b) are diagrams showing the experimental results. DETAILED DESCRIPTION OF THE INVENTION

[0014] <Features of this system> The rapid expansion of AI and IoT in recent years has dramatically increased the load on datacenter infrastructure. Hyperscale (large-scale, centralized) datacenters, which have been the mainstream until now, require a long construction time of 3 to 5 years and face the issue of concentrated power supply in one location. Furthermore, concerns about longer service response times and serious environmental impacts have emerged. Therefore, small-scale, distributed facilities known as edge datacenters have been gaining attention in recent years. Edge datacenters are particularly useful for applications requiring low response times, such as autonomous vehicles, industrial IoT, and smart cities. Their modular, container-based architecture, which allows for easy deployment and expansion, enhances operational flexibility and robustness. This invention further develops a container-based edge datacenter by distributing small-scale datacenters across multiple locations and combining distributed storage technology with data encryption to enhance overall security.

[0015] The container-type edge data center has the following characteristics:

[0016] (1) Adaptability to various installation configurations Installation on small land: Compact 10ft / 20ft / 40ft containers can be installed even in limited areas such as urban areas or remote areas. Expansion of hyperscale facilities: Adding modules to large-scale data centers enables the expansion of computing resources while minimizing environmental impact. On-premise deployment: For companies that require confidentiality and privacy, we offer a system that allows them to operate their own servers.

[0017] (2) Sustainable cooling methods centered on immersion cooling Excellent thermal performance: High heat dissipation is achieved by immersing electronic components in insulating liquid. Improved energy efficiency: Significant power savings have been achieved by reducing the use of large air conditioning units. Compact design: Simplified cooling equipment reduces installation space

[0018] (3) Scalability and adaptability Up to 0.5MW power supply capacity: Capable of supporting high-performance GPU workloads. Modular expansion: Compared to traditional data centers, this system achieves high scalability while shortening construction time.

[0019] (4) Integration with renewable energy Combination with solar and wind power: To further enhance sustainability, the design makes it easy to connect renewable energy sources.

[0020] In this way, the container-type edge data center according to an embodiment of the present invention can be said to be an approach to building a next-generation digital infrastructure that combines high performance, flexibility, and sustainability. Not only is it easy to install regardless of installation location or scale, but its combination with advanced cooling technology enables a wide range of entities, including businesses, public organizations, and hyperscalers, to quickly deploy energy-efficient data centers. The multipurpose use of small-scale edge data centers could be a path to opening up a new era of distributed computing.

[0021] <Embodiment> Hereinafter, embodiments of the present invention will be described in detail.

[0022] FIG. 1 shows a conceptual diagram of an immersion cooling system according to one embodiment of the present invention, FIG. 2 shows the external configuration of the system, and FIG. 3 shows the appearance of a data center that employs the immersion cooling system, which will be described below.

[0023] As shown in these figures, an immersion cooling system 1 according to an embodiment of the present invention comprises an immersion tank 2, a heat exchanger 3, a first water tank 4A, a second water tank 4B, and water supply pumps 6A and 6B. The immersion tank 2 and heat exchanger 3 are housed in a rack 10, and the immersion tank 2 houses server equipment, network equipment, or GPU or ASIC boards or externally connectable equipment containing such boards. The immersion tank 2 is filled with a highly insulating liquid. Note that when a server device or the like is housed in the immersion tank 2, the fan mechanism may be removed.

[0024] In this example, as shown in FIG. 3, 12 racks 10 are housed in containers 200A, 200B, etc., and multiple containers 200A, 200B, etc. are installed in a building 201. A first water tank 4A and a second water tank 4B are installed outdoors. In areas with heavy snowfall, a simple main house / warehouse can be used, and in areas with normal snowfall, a simple roof can be used. Furthermore, in this example, each container 200A, 200B, etc. is connected to a utility conduit 202 for the wet system (water) and a utility conduit 203 for the dry system (network, electricity, surveillance cameras, security, etc.). If necessary, a UPS or storage battery may be installed separately.

[0025] The first water tank 4A receives water from a natural water source (e.g., groundwater, river water, seawater, dam water, etc.) pumped up by the water supply pump 6A or tap water (approximately 14°C, 480 L / min per container) via an inlet line and constantly stores the water. This first water tank 4A also receives a portion (30% in this example) of the hot water (approximately 30°C) from the second water tank 4B. The water in the first water tank 4A (approximately 14-20°C, 480 L / min per container) is then supplied to the heat exchanger 3 by the water supply pump 6B. The immersion cooling system 1 requires 40 L of water per minute to cool each rack. In this embodiment, it is assumed that one container will accommodate 12 racks, so the first water tank 4A stores 480 L / min of water.

[0026] The immersion tank 2 is connected to a heat exchange system 14 including a heat exchanger 3 via an upper circulation section 12 or an intermediate circulation section 12. The liquid heated by heat radiation from the devices contained in the immersion tank 2 is circulated between the immersion tank 2 and the heat exchanger 3 by the action of a circulation pump 15. In the heat exchanger 3, the liquid is cooled by cooling water (14-20 degrees, 40 L / min) and returned to the immersion tank 2. When the immersion tank 2 is filled to capacity with the liquid, the liquid is circulated via the upper circulation section 12; when the immersion tank 2 is filled to about half capacity, the liquid is circulated via the middle circulation section 12. Cooling water flows into the heat exchanger 3 via an inlet path 13a, and heated water is discharged via an outlet path 13b. A filter may be installed in the circulation path between the immersion tank 2 and the heat exchanger 3 to remove dust, dirt, deteriorated liquid, and the like that may be mixed into the liquid. Due to this mechanism, the rack can be said to be a closed-circulation, single-phase (single-phase) proprietary immersion system.

[0027] The hot water (approximately 30 degrees, 480 L / min per container) used for heat exchange in heat exchanger 3 flows out into second water tank 4B via discharge path 13b. Of the approximately 30 degree water in second water tank 4B, 30% (144 L / min per container) flows out into first water tank 4A, and 70% (336 L / min per container) is discharged into natural water source 6 via discharge path 13b. In this way, by discharging 30% of the used hot water into first water tank 4A, the temperature of the water flowing in from natural water source 5 is raised to approximately 14-20 degrees, thereby improving the efficiency of heat exchange by heat exchanger 3.

[0028] This circulation cools the various devices housed in the immersion tank 2 inside the rack 10. Furthermore, 70% of the heated 30°C water discharged into the second water tank 5 is cooled to a certain extent before being discharged into the natural water source 6 via the discharge path 13b, thereby realizing environmentally friendly circulatory cooling.

[0029] That is, with the above configuration, by using water from natural water sources such as groundwater as the water used in the heat exchanger 3, it is possible to significantly reduce operating costs. Furthermore, by returning the extracted groundwater to the natural water source 5, the impact on the environment can be reduced. In addition to water from the natural water source 5, water from industrial waste treatment plants and the like can also be used as the water used in the heat exchanger 3, in which case recycling is possible. Alternatively, tap water can also be used as the water used in the heat exchanger 3, and in this case, the water heated by the heat exchanger 3 can be used as a heat source for another purpose. In addition to water, it is also possible to use coolants such as alternative chlorofluorocarbons or coolant liquids, and in this case, a higher cooling effect and reduced operating costs can be expected compared to conventional air-cooled systems.

[0030] In the immersion cooling system according to the embodiment of the present invention, the operational stability and the lifespan of the equipment can be improved by immersing the equipment in liquid. This is achieved by improving the following three points, which are factors that shorten the operational stability and the lifespan of the equipment. Immersion stabilizes the operating temperature of the entire device. The operating temperature of the entire device is always kept constant, even when the device is under heavy load. By using liquid immersion, the number of moving parts (fans, etc.) in the equipment is minimized, which is expected to reduce malfunctions in moving parts. Immersion minimizes exposure to and accumulation of dust and dirt normally found in the air, reducing short circuits caused by dust and dirt on the board and unstable operation caused by board contamination.

[0031] As described above, the immersion cooling system according to the first embodiment of the present invention can reduce overall costs by adopting a circulation type configuration that uses natural water such as groundwater (river water, seawater, dam water, etc.), and can also improve cooling efficiency, improve operational safety regardless of the load condition of the equipment, and extend the life of the equipment.

[0032] FIG. 4 shows the configuration of a data center in which an immersion cooling system according to one embodiment of the present invention is installed, and will be described.

[0033] As shown in the figure, a data center 100 is equipped with a plurality of immersion cooling systems 1-1, 1-2, ..., 1-x, which can be managed in an integrated manner by a control device 20. The control device 20 is connected to a server device 30 via a communication network 40 such as the Internet. The control device 20 transmits information such as the operating status of each immersion cooling system 1-1, 1-2, ..., 1-x periodically or in real time. The server device 30 can receive information such as the operating status and can also monitor the operating status remotely.

[0034] <Example> The results of the demonstration experiment of this system are explained below.

[0035] This experiment was carried out by installing a liquid immersion cooling system in a server room in an operating data center. The data center is located near a nuclear power plant, and is in an environment where a stable power supply and a relatively constant outside air temperature can be expected.

[0036] The main hardware and monitoring methods are as follows: This system is designed to accommodate up to eight 2U-sized GPU servers (e.g., NVIDIA H100) in a standard rack. However, in this evaluation, a total of six ASIC machines (Antminer L9) for blockchain mining were connected to measure the cooling efficiency under "continuous operation" with high loads 24 hours a day, 365 days a year. This mining workload generates high heat throughout the year, allowing for a detailed understanding of the cooling performance and thermal characteristics.

[0037] More specifically, it is as follows. A. Experimental environment a1) Computing Hardware (ASIC Miners): Antminer L9 (Scrypt algorithm) x 6 units Firmware: Antminer-L9-CV-release-202408211920.bmu Hash rate: 16 GH / s Power consumption: 3360W (±5%) per unit

[0038] a2) Immersion cooling system: Unique single-phase system using ENEOS IX Type J oil Coolant properties at 40°C: Viscosity 0.025 Pa·s Thermal conductivity 0.12 W / (m K) Specific heat 2.0 J / (g K) Density 0.85 g / cm 3 (25°C) Flash point 220°C

[0039] a3) Immersion tank: Made of SUS304 (internal dimensions: 609 mm x 1039 mm x 902 mm) Oil volume: 502 liters

[0040] a4) Cooling water: Originally designed water-cooled heat exchange system using groundwater at approximately 14°C Flow rate 40 L / min

[0041] a5) Monitoring: 24-hour continuous monitoring measures the following: Power consumption: ASIC and overall system power consumption measured with a power meter (3300 watt meter) Temperature: Immersion oil temperature, ASIC surface temperature, cooling water inlet / outlet temperature, etc. System stability: Logging hashrate, error rate, etc. every minute

[0042] B. Monitoring and Data Collection Through the above-mentioned monitoring system, performance indicators in actual operation were constantly measured and recorded.

[0043] C. Hardware Configuration Modular container unit (20ft container, internal dimensions 2.33m x 5.867m x 2.55m) - Up to 12 immersion cooling systems can be installed per unit (only one was used in the experiment) Maximum power capacity of 0.5MW (supplied at 6.6 kV) Heat exchange method using groundwater at 14°C

[0044] D. Purpose Thermal properties and energy efficiency evaluation Measures heat dissipation, temperature uniformity within the tank, thermal resistance of the coolant, PUE of the entire system, COP of the cooling system, etc. Evaluation of ASIC miner operational stability Monitors hash rate stability, error rate, and other behavior during continuous operation - Knowledge of implementation optimization gained from actual operation Presents points to note and suggestions for improvement in design and operation In this way, the value of this experiment lies in the fact that it measured and analyzed the effects of immersion cooling under actual operating conditions, which differ from laboratory conditions, and provided specific implementation guidelines based on the results.

[0045] E. Results To streamline data center operation and management, a monitoring system was developed to track the data processing volume, power consumption, oil temperature, etc. of the immersion cooling system. The system's data processing volume was six ASICs (16 GH / s each) running stably at a total of 100 GH / s. Each system consumed approximately 3,300 W and performed according to specifications, operating continuously 24 hours a day.

[0046] A major feature is that it does not require large-scale air conditioning equipment like air-cooled systems, and additional consumption other than the electricity consumed by the immersion cooling system itself is reduced. Because the experiment was conducted in winter (in December, with an outside temperature of 2-10°C), there was almost no need to use additional cooling equipment. However, additional measures may be necessary in midsummer.

[0047] Of particular note is the extremely low PuE (Power Usage Effectiveness) of the system, at 1.03. Generally, the PuE of air-cooled data centers is around 1.5 to 2.0, confirming the significant energy-saving effects of the immersion cooling method. Thermal monitoring: The oil temperature inside the tank remains stable within the range of 33°C to 50°C (see Figure 5(a)). Under heavy load, the temperature generally reaches a maximum of around 50°C, but is cooled to around 33°C by heat exchange with groundwater at 14°C. This suggests that more than enough cooling capacity can be achieved even when eight ASICs are operating at full capacity (see Figure 5(b)). Power monitoring: Records ASIC and overall system power consumption simultaneously Data logging: Automatically records hash rate, power consumption, temperature, etc. every minute

[0048] Figure 5(a) shows the change in oil temperature within the immersion cooling system, and Figure 5(b) shows the data throughput of the immersion cooling system. As shown in these figures, the oil temperature was maintained well within the allowable range even under full load for long periods of time, confirming the continuous operation of the system.

[0049] A temperature sensor installed in the immersion tank acquires real-time oil temperature, which is stably controlled at approximately 38°C. The operating temperature range for general commercial components is 0 to 70°C, and for industrial use it is around -40 to 85°C, so it can be seen that there is ample margin from both standards. By maintaining the oil temperature at 38°C, the semiconductor surface temperature also falls well within the allowable range. This system uses groundwater at around 14°C for heat exchange.

[0050] A feature of the single-phase cooling method is that groundwater circulating with a pump is sent to heat exchange pipes, where it exchanges heat efficiently with the oil flowing through them. The groundwater that has absorbed the heat is returned to the ground, so in Japan's environment, where groundwater is abundant, it can circulate almost permanently. This leads to the realization of data centers that use renewable energy.

[0051] <Consideration> The four-hour service outage of OpenAI in early 2024 once again highlighted the fact that AI has become an infrastructure for society. Unlike traditional Internet outages, when an AI service stops, it has a widespread impact on many operations and services. This is due to the 1:N architecture (one large-scale model used by many users), which can be seen as a weakness of centralized AI infrastructure. In this regard, decentralization to small-scale edge data centers has emerged as a means of mitigating single-point failures.

[0052] The container-type immersion cooling system of the present invention is a powerful means for enabling distributed deployment. It is particularly notable for its space efficiency, as it can be installed in a 20-ft container (approximately 2.33 m x 5.867 m x 2.55 m, volume approximately 34.9 m). 3 ), whereas an equivalent air-cooled system would require a large server room (at least 89.1 m) with a ceiling height of 4.5 m and a width of 8.5 m. 3 ), plus outdoor cooling tower space (180 m 3 (or more) may also be required.

[0053] Although a simple comparison requires assumptions, it is possible to expect a nearly fivefold reduction in overall space, which in turn should lead to shorter construction costs and deployment times. When combined with the high energy-saving performance of a PUE of 1.03 (significantly lower than the typical 1.5-2.0 for air-cooled systems) and improved thermal efficiency, this system offers significant advantages for edge data centers. Furthermore, because it is a container-type system, construction can be completed in just a few months, significantly faster than the three to five years required for conventional systems.

[0054] Finally, having a decentralized AI infrastructure at the edge brings the following benefits: Improving local government autonomy: By operating AI models tailored to local characteristics, disaster response and public services can be strengthened. Improved resilience on a national scale: Reliance on centralized systems will be reduced, allowing infrastructure to be built that is more resistant to attacks and disruptions. Improved corporate competitiveness: By locating a data center in your own environment, you can reduce your dependency on third parties, which has benefits in terms of both cost and security.

[0055] As dependence on AI and IoT increases, data centers are required to be even more efficient and sustainable. The liquid immersion cooling system according to an embodiment of the present invention demonstrates that container-type liquid immersion cooling technology is a powerful means of meeting these demands. Compared to air-cooled systems, this system offers several advantages, including significant space savings, energy-saving performance (reduced PUE), and reduced construction costs and time. Furthermore, by combining it with a distributed architecture, further benefits such as fault tolerance and regional optimization can be enjoyed.

[0056] Although the embodiments of the present invention have been described above, it goes without saying that the present invention is not limited to these and that various improvements and modifications can be made without departing from the spirit and scope of the present invention.

[0057] For example, refrigerants other than groundwater may be used in conjunction with water from natural sources. [Explanation of symbols]

[0058] 1...immersion cooling system, 2...immersion tank, 3...heat exchanger, 4A...first water tank, 4B...second water tank, 5...natural water source, 6A, 6B...water supply pump, 10...rack, 11...circulation section, 12...circulation section, 13a...inlet path, 13b...outlet path, 14...heat exchange system, 15...circulation pump, 20...control device, 30...server device, 40...communication network, 100...data center

Claims

1. an immersion tank that contains the device to be cooled and is filled with a highly insulating liquid; a first water tank for storing water from a natural water source; a heat exchanger that performs heat exchange between the water in the water tank and the liquid agent; a second water tank for storing water used in the heat exchanger; The first water tank discharges water supplied from the natural water source to the heat exchanger, and the second water tank receives heated water from the heat exchanger, discharges a portion of the water into the first water tank, and discharges the other portion into the natural water source. Immersion cooling system.

2. The natural water source is at least one of groundwater, river water, seawater, and dam water. The immersion cooling system of claim 1 .

3. When a server device is housed in the immersion tank as the device, the fan mechanism is removed before housed. The immersion cooling system of claim 1 .

4. A data center in which the liquid immersion cooling system according to any one of claims 1 to 3 is installed.

Images