Modular power distribution grid for data centers

Abstract

A power distribution grid for a facility, such as a data center, the power distribution grid being located within the facility. The power distribution grid includes a plurality of power delivery elements arranged in a grid pattern and nodes located at intersections of the grid pattern. Electrical loads are supplied with power via corresponding nodes of the power distribution grid. In addition, each node is supplied with power by more than two delivery elements, so that one or more delivery elements can be allowed to fail, while electrical loads connected to a particular node associated with the failed delivery element continue to receive power supplied to the particular node from at least two different delivery elements.

Description

Background Art

[0001] Typically, operators of large computing facilities (such as data centers) design the facility to include a power distribution system with a radial branch distribution design, which includes one or more main branches and lower-level branches fanned out from the main branches. At any given moment, each main branch receives power from a single power source, but if the main power source fails, the main branch can be switched to receive power from a backup source. Such a power distribution system may also include one or more backup branches with associated lower-level branches, which are similarly designed according to the radial branch distribution design and similarly receive power from a single power source at a certain time. For example, a computing facility may include large electrical equipment such as transformers, switchgear, uninterruptible power supplies (UPS), etc., which receive power from a utility power source and feed one or more main branches of a radial branch distribution design. The lower-level branches fanned out from the corresponding main branches of the main branches receive power from the corresponding main branches and supply power to the load. In some computing facilities, one or more additional backup branch circuits may be mirrored with the main branch circuits. In addition, the cooling system for such a facility is typically arranged using a radial branch distribution design. In a similar manner to power systems, a cooling system for a computing facility may include relatively large chillers, such as mechanical chillers, and the large chillers may feed cooling water to a main header that then branches into smaller branches and ultimately branches to the cooling loads. Alternatively or additionally, a computing facility's HVAC system may include a main air duct that branches into smaller air ducts and ultimately reaches the cooling loads.

[0002] Such radially branched power distribution systems and cooling systems may be designed for predetermined loads and predetermined load distributions and may be difficult to modify without significant cost. Moreover, such radially branched power distribution systems and cooling systems may be significantly affected by single point failures. For example, a single failure may cause a service interruption and / or may render the system without backup. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] Figure 1A is a perspective view illustrating a block diagram of a data center including a power distribution grid and a cooling grid to which power supply units and heat rejection units are connected along a perimeter of the data center according to some embodiments.

[0004] Figure 1B are block and line diagrams illustrating components of a node of a power distribution and cooling grid, wherein the node is connected to a plurality of transport elements of the power distribution and cooling grid, according to some embodiments.

[0005] Figure 1Cis a perspective view of a block diagram showing a node at the intersection of transport elements, wherein the node is connected to power and cooling loads in a group of racks, according to some embodiments.

[0006] Figure 1D is a perspective view of a block diagram showing multiple nodes at the intersection of transport elements, wherein the nodes are connected to power and cooling loads in a group of racks, according to some embodiments.

[0007] Figure 2 An exemplary power supply unit including a fuel-based power generation component and a utility power feed-based component is shown according to some embodiments.

[0008] Figure 3 An exemplary three-dimensional power distribution grid and / or cooling grid is shown according to some embodiments.

[0009] Figure 4 An exemplary vertical power distribution grid and / or cooling grid is shown according to some embodiments.

[0010] Figure 5 An exemplary circular power distribution grid and / or cooling grid is shown according to some embodiments.

[0011] Fig. 6A Shown is a top view of an electric power distribution grid and / or cooling grid and various types of power supply units connected to the grid along the perimeter of the grid, according to some embodiments.

[0012] Figure 6B A high-ohmic ground path that may be included in a node of an electric power distribution grid is shown according to some embodiments.

[0013] Figure 7 A perspective view of a shipping container-based modular data center unit coupled to a node of a power distribution grid and / or a cooling grid is shown according to some embodiments.

[0014] Figure 8 A block diagram of an electric power distribution network controller is shown according to some embodiments.

[0015] Fig. 9 A high-level flow chart for providing and adjusting the capacity of an electric power distribution grid at a facility is shown, according to some embodiments.

[0016] Fig.10 A high-level flow diagram for distributing power to loads at a facility via a power distribution grid is shown, according to some embodiments.

[0017] Fig.11 A high-level flow chart for monitoring and / or responding to faulty transmission elements of a power distribution grid and / or cooling grid is shown, according to some embodiments.

[0018] Fig.12 A top view of a cooling grid including various types of heat rejection units coupled to a perimeter of the cooling grid is shown according to some embodiments.

[0019] Fig.13 A liquid-cooled heat-generating component is shown being cooled by a cooling fluid received from and returned to a cooling network via nodes of the cooling network, according to some embodiments.

[0020] Fig.14 An air-cooled heat-generating component is shown being cooled by a cooling fluid received from and returned to the cooling network via nodes of the cooling network, according to some embodiments.

[0021] Fig.15 An exemplary configuration of a cooling grid operating at a pressure below atmospheric pressure is shown according to some embodiments.

[0022] Fig.16 A block diagram of a cooling network controller according to some embodiments is shown.

[0023] Fig.17 A high-level flow chart for providing a cooling grid and adjusting the cooling grid due to varying cooling demands of loads at a facility is shown, according to some embodiments.

[0024] Fig.18 A high-level flow chart for monitoring a cooling grid for leaks is shown, according to some embodiments.

[0025] Fig.19 An exemplary computer system is shown that can implement a power distribution grid controller and / or a cooling grid controller according to some embodiments.

[0026] Although the present invention is susceptible to various modifications and alternative forms, its specific embodiments are shown in the accompanying drawings by way of example and will be described in detail herein. However, it should be understood that the drawings and detailed description of the present invention are not intended to limit the present invention to the disclosed specific form, but on the contrary, are intended to cover all modifications, equivalents and alternatives falling within the spirit and scope of the present invention as defined in the appended claims. The titles used herein are only used for organizational purposes and are not meant to be used to limit the scope of the specification or claims. As used throughout this application, the word "may" is used in an allowed sense (i.e., meaning to have a potential meaning), rather than a mandatory sense (i.e., meaning to be necessary). Similarly, the word "include" ("include", "including" and "includes") means including but not limited to. DETAILED DESCRIPTION

[0027] Various embodiments of an internal power distribution network and / or internal cooling network in a facility such as a data center are described herein. In some embodiments, the system includes a plurality of power supply units and a data center, the data center including electrical loads such as rack-mounted servers, switches, routers, and / or other electronic devices. The data center also includes an internal power distribution network connected to the power supply units and the electrical loads. The internal power distribution network is positioned within the data center and includes modular power delivery elements arranged in a grid pattern and nodes located at the intersections of the grid pattern. The modular power delivery elements and nodes are arranged in the data center so that each node is configured to receive power from more than two of the modular power delivery elements connected to the node. Moreover, the electrical loads are arranged in the data center so that the electrical loads receive power from the network via corresponding nodes in the nodes. In this way, a failure of a given power delivery element connected to a specific node that supplies power to a specific electrical load will not result in the electrical load being without power and backup power. For example, during a transmission element failure, at least two remaining transmission elements remain configured to supply power to a specific electrical load via a specific node. Additionally, as discussed in more detail below, using an internal power distribution grid in a facility, such as a data center, allows for greater flexibility to increase and / or redistribute electrical loads in the facility without having to redesign or replace existing electrical distribution infrastructure.

[0028] In some embodiments, the power distribution system includes modular power delivery elements configured to be arranged in a grid pattern to supply power to computing devices and / or data storage devices in a facility, and nodes located at intersections of the grid pattern. The modular power delivery elements and the nodes are arranged so that a respective one of the nodes is configured to receive power from two or more of the modular power delivery elements connected to the respective node, and to supply the received power to one or more of the computing devices and / or data storage devices in the facility.

[0029] In some embodiments, a method includes providing a power distribution grid for a facility, wherein the power distribution grid includes modular power delivery elements arranged in a grid pattern to supply power to computing devices and / or data storage devices in the facility and nodes located at intersections of the grid pattern. The method also includes distributing power to computing devices and / or data storage devices in the facility via the nodes of the power distribution grid, wherein power from two or more of the modular power delivery elements connected to the respective nodes is supplied to the respective nodes.

[0030] In some embodiments, the system includes a data center, the data center includes heat generating components installed at different locations in the data center and an internal cooling network positioned in the data center, wherein the internal cooling network includes cooling fluid delivery elements arranged in a grid pattern and nodes located at the intersection of the grid pattern. The system also includes a heat removal unit, which is connected to the internal cooling network and is configured to remove heat from the cooling fluid flowing through the internal cooling network. In the system, the corresponding nodes in these nodes are configured to receive cooling fluid from multiple fluid delivery elements in the fluid delivery elements connected to the corresponding nodes, and return the cooling fluid to multiple fluid delivery elements in the fluid delivery elements. Moreover, the heat generating components are arranged in the data center so that the corresponding heat generating components in the heat generating components are cooled by the cooling fluid received from the internal cooling network and returned to the internal cooling network via the corresponding nodes in the nodes. For example, in a similar manner to that discussed above with respect to the internal power distribution network, the internal cooling network allows the failure of a given cooling fluid delivery element connected to a specific node, and the specific node supplies cooling fluid to the specific heat generating component without causing the specific heat generating component to have no cooling fluid source and a backup cooling fluid source. For example, at least two of the remaining cooling fluid transport elements remain configured to supply cooling fluid to specific heat generating components via specific nodes. In addition, as discussed in more detail below, the use of an internal cooling network in a facility (such as a data center) allows for greater flexibility to increase or redistribute heat generating loads in the facility without having to redesign or replace existing cooling infrastructure. For example, if one set of servers is replaced with another set of servers that generates more concentrated waste heat, additional cooling fluid can be supplied to the other set of servers that generates more concentrated waste heat via the internal cooling network without having to replace the piping or ductwork of the facility.

[0031] In some embodiments, the cooling network includes cooling fluid delivery elements arranged in a grid pattern to provide cooling support to computing devices and / or data storage devices in the facility and nodes located at the intersections of the grid pattern. The cooling fluid delivery elements and the nodes are arranged in the facility so that corresponding ones of the nodes are configured to receive cooling fluid from multiple ones of the fluid delivery elements and return cooling fluid to multiple ones of the fluid delivery elements. Moreover, corresponding ones of the nodes are configured to provide the received cooling fluid to a cooling circuit for removing heat from one or more of the computing devices and / or data storage devices, and to receive cooling fluid that has been used to remove heat from one or more of the computing devices and / or data storage devices from the cooling circuit.

[0032] In some embodiments, a method includes providing a cooling network for a facility, wherein the cooling network includes cooling fluid delivery elements arranged in a grid pattern to provide cooling support to computing devices and / or data storage devices in the facility and nodes located at intersections of the grid pattern. The method also includes flowing a cooling fluid through the cooling network to provide cooling to the computing devices and / or data storage devices in the facility, wherein the cooling fluid is supplied to respective pieces of computing devices and / or data storage devices connected to the specific node via a specific node, the cooling fluid being received at the specific node from two or more of the fluid delivery elements connected to the specific node.

[0033] Facilities, such as data centers or other computing facilities, may operate continuously and, in order to operate properly, may require reliable power and / or cooling to various electrical loads / heat generating components in the facility. Many such facilities utilize radial branch distribution systems and may also use backup radial branch distribution systems to supply power and / or cooling to electrical loads in the facility.

[0034] In such systems, the main power supply system supplies power to the electrical load from a centralized source via a main distribution circuit, wherein the lower-level distribution circuit receives power from the main power distribution circuit and the intermediate-level distribution circuit. For example, the main distribution bus can supply power to multiple areas in the data center, wherein the intermediate-level bus in each area supplies the power received from the main distribution bus to an even lower-level bus, which ultimately supplies the received power to one or more electrical loads. Some such power systems may include a redundant backup power system mirrored with the main power system. However, under normal operating conditions, the backup power system may not actively supply power to the electrical load, and may be idle instead, waiting for the main power system to fail. Because the backup power system remains idle for most of the life of the facility, such an approach may result in wasted power distribution capacity. In addition, when there is a fault in the main power system and the electrical load is supplied with power from the backup power system instead, the electrical load is susceptible to a single additional fault in the backup power system, causing the electrical load to lose power. In addition, the higher-level distribution bus may be designed for fixed electrical loads at the time of installation, and may require a large amount of cost and downtime to modify or replace to upgrade to support larger electrical loads. Thus, in an instance where existing servers in a portion of a data center are replaced with other servers that require greater amounts of power, the entire power distribution system, including the lower level buses and all upstream buses (including the main distribution bus), may need to be replaced to support the upgraded servers with the greater power requirements.

[0035] The cooling system for a facility may also be designed to require considerable expense to upgrade. For example, air-cooled heat-generating components are typically cooled by air supplied to the cold aisle via air ducts that connect to larger air ducts that connect back to the air conditioning unit via one or more additional higher-level ducts. In the event that a portion of the heat-generating components are upgraded to another type of heat-generating component that produces more concentrated waste heat, the air conditioning unit and duct system may not be sufficient to provide cooling to remove the concentrated waste heat produced by the upgraded heat-generating components, thereby requiring considerable cost and downtime to upgrade the air conditioning unit and duct system.

[0036] In some embodiments, in order to provide greater reliability and flexibility to the power distribution system and / or cooling system, a network inside a facility (such as a data center) can be used. The internal network can include a delivery element and a node element. Each node element in these node elements can be connected to a plurality of delivery elements at the intersection of the grid pattern. In some embodiments, each delivery element can include a high-side power path and a low-side power path (e.g., +200V DC and -200VDC). In addition, each delivery element can include a coolant supply line and a coolant return line. In some embodiments, the network can be arranged so that four delivery elements converge at the node position and are connected to the node, so that four separate delivery elements provide a path for power and / or cooling fluid to arrive at the node. In some embodiments, the three-dimensional grid pattern can be arranged so that six delivery elements converge at the node position and are connected to the node, so that there are six passages for receiving power and / or cooling fluid at the node. Electrical loads and / or cooling loads can be connected to the node and receive power and / or cooling from the node, wherein the node receives power and / or cooling via a plurality of delivery elements (e.g., four delivery elements, six delivery elements, etc.).

[0037] In some embodiments, each transport element may be configured to be capable of supplying a greater proportion of the power and / or cooling received by the node than the proportion of the number of connections it has to the node. For example, for a node connected to four transport elements, each transport element may be configured to supply more than one-quarter of the power and / or cooling fluid received by the node and used to support the electrical loads and heat generating components served by the node. In this way, the failure of a transport element does not result in a particular node losing power and / or cooling fluid supply, or having no backup source of power and / or cooling fluid. For example, for a node connected to four transport elements, two of the transport elements may fail, for example, due to a leak or a short circuit, and the remaining two transport elements connected to the node may have sufficient capacity to supply power and / or cooling fluid to the node without reducing power or cooling fluid consumption by the electrical loads and heat generating components connected to the node.

[0038] In some embodiments, multiple power supply units and / or heat removal units are connected to the power distribution network and / or cooling network along the perimeter of the network. For example, the network can be inside a data center facility, and the power supply units and / or heat removal units can be placed outside the data center and connected to the perimeter of the network. In some embodiments, the power supply unit can be a modular unit that can be quickly installed, removed and / or repositioned. For example, the power supply unit can be skid-mounted so that the power supply unit can be moved from one location to another with a forklift. In a similar manner, the heat removal unit can be a modular unit that can be quickly installed, removed and / or repositioned. For example, the heat removal unit can be skid-mounted so that the heat removal unit can be moved from one location to another with a forklift.

[0039] In some embodiments, the power supply unit may include a rectifier, a transformer, and one or more additional electrical components, which convert the low voltage electricity received from a utility power source such as three-phase alternating current power (AC power) into direct current (DC) power. In some embodiments, the power supply unit may include a generator, such as a fast-start multi-fuel turbine. In some embodiments, the power supply unit may include both fuel-based power components (such as generators) and utility power components (such as components configured to receive and regulate the power received from the utility power source). In some embodiments, the power supply unit may include components that generate electricity from other types of energy (such as renewable energy). For example, the power supply unit may include a wind turbine, a solar panel, a geothermal power source, a hydroelectric power source, or other types of power sources. In some embodiments, the power supply unit may include one or more batteries, or an independent battery power supply unit may be connected to a power distribution grid.

[0040] In some embodiments, the heat removal unit may include heat removal components such as heat exchangers, fluid moving devices such as pumps, and fluid reservoirs such as tanks. In some embodiments, the heat removal unit may generate negative pressure to pull the returned cooling fluid out of the internal cooling network, and the cooled cooling fluid may be supplied to a supply line connected to the internal cooling network. In some embodiments, the heat removal component of the heat removal unit may include a cooling tower or other types of evaporative coolers that cool the cooling fluid by evaporating the liquid. In some embodiments, the heat removal component of the heat removal unit may include a free cooling heat exchanger that cools the cooling fluid by passing ambient air through the heat exchanger to cool the cooling fluid flowing through the heat exchanger. In some embodiments, the heat removal component of the heat removal unit may include a mechanical refrigerator that compresses and expands a refrigerant. In some embodiments, the heat removal component of the heat removal unit may include an absorption refrigerant unit that utilizes a salt solution and waste heat from another source to produce a refrigeration effect that cools the cooling fluid. In some embodiments, the cooling fluid may be cooling water or another suitable cooling medium.

[0041] In some embodiments, the internal cooling network can be connected to a highly insulated fluid tank. The cooling fluid flowing through the internal cooling network can flow through the insulated tank. In some embodiments, the insulated tank can be used as a large radiator for the system, especially during high heat loads. For example, during the period when the spike in heat is removed by the internal cooling network, some of the heat can be absorbed by the large amount of cooling fluid in the highly insulated tank, thus reducing the immediate load on the heat removal components of the heat removal unit connected to the internal cooling network. During the time period of lower heat load, the total temperature of the large amount of cooling fluid in the highly insulated tank may decrease, thus refilling the capacity of the tank to absorb heat during future time periods of high heat load.

[0042] In some embodiments, the internal cooling network can be operated at a pressure lower than one atmosphere. Therefore, any leak in the cooling fluid supply line or cooling fluid return line of the conveying element may cause air to leak into the supply line and / or return line of the internal cooling network without causing the cooling fluid to leak outside the internal cooling network. In some embodiments, the internal cooling network may include an array of pressure sensors and / or flow sensors to detect leaks in the internal cooling network. In some embodiments, the exhaust line from the internal cooling network can measure the amount of air exhausted from the internal cooling network. If the amount of air exhausted exceeds a threshold, an alarm system may indicate a potential leak in the internal cooling network.

[0043] In some embodiments, a separate power supply unit can provide a relatively small portion of the total power supplied to the power distribution network of the facility. For example, in some embodiments, each power supply unit can supply less than 20% of the total power supplied to the power distribution network. In some embodiments, each power supply unit can provide about 750 kilowatts (KW) or less of power. Since the power distribution network does not rely on any single power supply unit for most of its power, the power distribution network can be more resilient in the event of a power supply unit failure. For example, the power distribution network may include a backup power supply unit, such as a fuel-based power supply unit, which can be quickly activated in the event of a failure of another power supply unit connected to the power distribution network, wherein the loss of the other power supply unit and the activation of the backup power supply unit are completed so that the electrical loads connected to the power distribution network will not run without power. In a similar manner, the internal cooling network can be connected to a plurality of heat rejection units, wherein no single heat rejection unit supplies most of the cooling capacity or cooling fluid flow capacity required for the internal cooling network. For example, each heat rejection unit can reject 20% or less of the total heat rejected from the cooling network from the cooling network. Furthermore, each heat rejection unit may contribute 20% or less of the total flow of cooling fluid through the cooling network.

[0044] In some embodiments, the facility that is powered by a power distribution grid or receives cooling from a cooling grid can be a group of containerized data center modules installed in an open area or within a facility. For example, in some embodiments, electrical equipment such as computing devices, data storage devices, networking equipment, etc. can be installed in portable containers such as ISO shipping containers. In some embodiments, a group of containerized data center modules can be organized into clusters and can be connected to power distribution grids and cooling grids local to the containerized data center modules of the group of clusters. The power distribution grid and cooling grid can supply power and cooling to the containerized data center modules.

[0045] In some embodiments, the power distribution grid and / or the internal cooling grid may be designed to "right size" the power supply units and / or heat removal units to accommodate the current electrical loads and cooling loads. For example, the power distribution grid and / or the internal cooling grid may be commissioned with a certain amount of power supply units and heat removal units that match the initial power and cooling loads connected to the grid. As power consumption and / or cooling demands rise or fall, additional power supply units and / or heat removal units may be connected to the grid to increase capacity to meet the increased demand. Conversely, as power consumption and / or cooling demands fall, one or more power supply units and / or heat removal units may be disconnected from the grid. For example, a disconnected power supply unit and / or heat removal unit may be moved and connected to another grid that requires additional power and / or cooling capacity.

[0046] In some embodiments, the power distribution network and / or the cooling network may be designed to tolerate failures without immediately performing maintenance. For example, in some embodiments, the size of the conveying elements may be set so that when one or more conveying elements fail, the remaining conveying elements may carry the load (power or cooling) previously carried by the failed conveying elements. Therefore, in some embodiments, failures may be allowed to accumulate until a threshold number of failures is reached. At this point, maintenance operations may be scheduled to repair or replace the failed components. This can result in a more efficient maintenance method that is the opposite of arranging maintenance operations for each failure. Moreover, some facilities may not be equipped with on-site maintenance personnel, so allowing failures to accumulate before being scheduled for maintenance operations can reduce the number of trips that maintenance personnel must make to perform maintenance on the facility.

[0047] Figure 1A is a perspective view illustrating a block diagram of a data center including a power distribution grid and a cooling grid to which power supply units and heat rejection units are connected along a perimeter of the data center according to some embodiments.

[0048] The data center 100 includes an internal power distribution and cooling grid 104. It is noted that in some embodiments, the grid 104 may provide power alone, cooling alone, or both. The grid 104 includes conveying elements 102 arranged in a grid pattern and nodes 106 located at the intersections of the grid pattern. For example, except for the nodes 106 at the perimeter of the grid (where three conveying elements 102 meet at the node location), for most nodes of the grid 104, four conveying elements 102 meet at the nodes 106. In some embodiments, the grid 104 is a self-supporting grid, wherein each node 106 has a structural member such as a beam or leg that extends downward from the node to the floor of the facility, and when connected to other self-supporting nodes via multiple conveying elements, a self-supporting grid structure is formed, which is supported by the legs extending downward from the corresponding nodes of the grid. In some embodiments, the grid 104 can be supported by the structure of the facility. For example, the grid can be suspended from the ceiling of the facility via a hanger. For ease of illustration, in Figure 1A A net 104 is shown suspended from the ceiling.

[0049] The data center 100 also includes electrical loads 108, which may include a plurality of computing devices and / or data storage devices mounted in racks. Additionally, the electrical loads 108 may include networking equipment or other types of electrical loads. The electrical loads 108 may be located in different parts of the data center. For example, Figure 1ASeveral electrical loads 108 are shown located in different aisles of racks within the data center 100. The electrical loads 108 may consume power from the grid 104 and may generate waste heat. The waste heat may be exhausted from the data center 100 via a cooling fluid supplied to the electrical loads 108 through the grid 104. For example, each of the nodes 106 may be a connection point that connects one or more electrical loads to power from the grid 104 and supplies cooling fluid to a cooling circuit for the electrical loads 108 and receives return cooling fluid from the cooling circuit of the electrical loads 108.

[0050] For example, Figure 1B 106, each delivery element 102 may include a high-side power path, a low-side power path, a cooling fluid supply line, and a cooling fluid return line. At node 106, four (or six, etc.) high-side power paths may converge at a point that is also connected to electrical load 108. In addition, four (or six, etc.) low-side power paths may converge at an additional point that is also connected to electrical load 108. Therefore, electrical loads may receive high-side power via any one of the four (or six, etc.) delivery elements connected to node 106, and may make low-side power flow back to the grid via any one of the four (or six, etc.) delivery elements connected to node 106. In a similar manner, four (or six, etc.) cooling fluid supply lines may converge at a common manifold in node 106. In addition, four (or six, etc.) cooling fluid return lines may converge at another common manifold in node 106. Thus, cooling fluid may flow from any of the transport elements 102 connected to the node 106 into the supply-side manifold (and into the heat exchanger that cools the heat-generating component being cooled via the node 106). Also, return cooling fluid may flow from the heat exchanger back into the network 104 that cools the heat-generating component being cooled via the node 106, wherein the cooling fluid may in turn flow back into any of the transport elements 102 connected to the node 106 via the return-side manifold of the node 106.

[0051] In some embodiments, multiple power supply units and heat removal units are connected to a power distribution grid and / or cooling grid, such as grid 104, along the perimeter of the grid. In some embodiments, pads and piping and / or wire connections may be provided along the perimeter of the power distribution grid and / or cooling grid to allow additional power supply units and / or heat removal units to be connected to the grid. For example, grid 104 includes power supply unit 118 and heat removal unit 120 coupled to grid 104 on a first side of grid 104, and additionally includes power supply unit 110 and heat removal unit 116 coupled on a second side of grid 104. Moreover, there is a pad and associated connector 122 on the first side of grid 104 for accommodating the installation of additional heat removal units and connecting the additional heat removal units to grid 104. Additionally, there is a pad and associated connector 124 on the second side of grid 104 for accommodating the installation of additional power supply units and connecting the additional power supply units to grid 104.

[0052] In some embodiments, connecting pipe 116 connects heat rejection unit 114 to grid 104, and electrical wiring 112 connects power supply unit 110 to grid 104. In some embodiments, power supply unit 110 can be a power supply unit of a renewable energy type, such as a wind turbine. In some embodiments, heat rejection unit 114 can be a free cooling type heat rejection unit, such as a cooling tower module that evaporates water to reject heat from grid 104. Moreover, heat rejection unit 120 can be connected to grid 104 via pipe connection 116. Heat rejection unit 120 can be a mechanical refrigerator that provides supplemental cooling when heat rejection unit 114 cannot reject the entire amount of waste heat transferred to grid 104. In some embodiments, power supply unit 118 is connected to grid 104 via wiring connection 126. In some embodiments, power supply unit 118 can be a utility power source.

[0053] In some embodiments, each node 106 supplies power to a power panel of the rack or to a power distribution panel that supplies power to multiple racks. In some embodiments, the power portion of the grid 104 includes a direct current (DC) mesh bonding network.

[0054] In some embodiments, each conveying element in the conveying element 102 is a modular element with standard size. Moreover, the node 106 can be a modular component with standard size. For example, each conveying element 102 can include a high-side power path, a low-side power path, a cooling fluid supply line and a cooling fluid return line with equal size, and can also include a standardized connector to connect the corresponding power path and the cooling fluid line to the corresponding standardized connector of the node. Therefore, as the additional space in the data center is filled, additional conveying elements and nodes can be added to the internal power distribution network and / or the cooling network to grow the network. In addition, additional power supply units and heat removal units can be added to the periphery of the network to increase the ability of the network to supply power and heat removal. In some embodiments, the size of the combined power supply unit based on a public power supply unit, a fuel-based power supply unit or a power supply unit including a public power component and a fuel-based power component can be set to provide a relatively small part of the total power consumed by the network (such as the network 104). For example, the power supply unit 118 can be a 750 kilowatt power supply unit. In this way, the network can not rely excessively on any single power supply to supply power to the network.

[0055] Figure 1B is a block diagram and line diagram illustrating components of a node of a power distribution and cooling grid according to some embodiments, wherein the node is connected to a plurality of transmission elements of the power distribution and cooling grid. In some embodiments, Figure 1B The node 130 shown in FIG. 1 may be as follows: Figure 1A Node 106 is shown in FIG. Moreover, conveying elements 132, 134, 136 and 138 may be as shown in FIG. Figure 1A The conveying element 102 is shown.

[0056] like Figure 1B As shown, each of the delivery elements 132, 134, 136, and 138 includes a high-side electrical pathway 140, a low-side electrical pathway 142, a cooling fluid supply line 144, and a cooling fluid return line 146. The electrical pathways (high and low) meet at a respective node 148, with a set of four high-side pathways (one from each delivery element) meeting at node 148, and a set of four low-side pathways (one from each delivery element) meeting at another respective node 148. In a similar manner, four cooling fluid supply lines (one from each respective delivery element) meet at a respective manifold 150, and a set of four return lines meet at another respective manifold 150. As shown in FIG. Figure 1C As shown in more detail in , electrical loads may be connected to high-side and low-side nodes 148 to receive power from the grid. Also, a cooling circuit for cooling heat-generating components of the electrical loads may be connected to both the supply manifold 150 and the return manifold 150 at node 130 .

[0057] In some embodiments, the node 130 further includes an isolating switch 152 and a shutoff valve 154. In some embodiments, the isolating switch 152 or the shutoff valve 154 can be automatically operated to isolate a faulty delivery element. In some embodiments, the isolating switch 152 or the shutoff valve 154 can be a passive protection element, where an imbalance between high-side and low-side currents or voltages, or a difference between supply and return pressures or flows causes the isolating switch or the shutoff valve to automatically isolate the corresponding delivery element.

[0058] Figure 1C is a perspective view of a block diagram showing a node at the intersection of transport elements, wherein the node is connected to power and cooling loads in a group of racks, according to some embodiments.

[0059] As described above, in some embodiments, the electrical load is connected to a node of the power distribution network, and the cooling loop for the electrical load is connected to a node of the internal cooling network. In some embodiments, the power distribution network and the internal cooling network can be combined into a common network, and both the power connection and the cooling connection can be formed to the same node. For example, the node 166 is connected to the power distribution panel 170 that supplies power to the electrical load 168. Moreover, the cooling loop 176 is connected to the node 166. The cooling loop 166 includes a supply line 172 and a return line 172, each of which is connected to a corresponding supply manifold and a corresponding return manifold of the node 166. The cooling loop 176 provides cooling to the heat exchanger, which removes heat from the heat generating components included in the electrical load 168.

[0060] As in Fig.13 and Fig.14 As discussed in , in some embodiments, a liquid heat exchanger or a liquid-to-air heat exchanger may be connected to a cooling circuit, such as cooling circuit 176, and may cool heat-generating components included in an electrical load, such as heat-generating components of electrical load 168. Figure 1CAs shown, node 166 is located at the intersection of the mesh of delivery elements 160, 162, 164, and 178. Thus, power from any one of the delivery elements 160, 162, 164, or 178 may be supplied to distribution panel 170 via node 166. If one or more of the delivery elements 160, 162, 164, or 178 are not available to supply power to distribution panel 170 via node 166, the remaining delivery elements 160, 162, 164, or 178 may supply power to distribution panel 170 via node 166. In a similar manner, cooling fluid from any one of the delivery elements 160, 162, 164, or 178 may be supplied to cooling circuit 176 via node 166, and return cooling fluid from cooling circuit 176 may be returned to any one of the delivery elements 160, 162, 164, or 178 via node 166. If one or more of delivery elements 160 , 162 , 164 , or 178 are not available to supply cooling fluid to or receive cooling fluid from cooling circuit 176 via node 166 , the remaining delivery elements 160 , 162 , 164 , or 178 may supply or receive cooling fluid from cooling circuit 176 via node 166 .

[0061] Figure 1D is a perspective view of a block diagram showing multiple nodes at the intersection of transport elements, wherein the nodes are connected to power and cooling loads in a group of racks, according to some embodiments.

[0062] In some embodiments, nodes of the power distribution grid and / or the internal cooling grid may be connected to Figure 1C A set of racks as shown, or can be connected to Figure 1D For example, each of nodes 182, 184, 186, 188, and 190 is connected to a separate power panel 192 included in a separate one of racks 195, 196, 197, 198, and 199, each rack including electrical loads 168. Also, each of nodes 182, 184, 186, 188, and 190 is connected to a separate cooling circuit 194, wherein each of racks 195, 196, 197, 198, and 199 is cooled by a separate one of cooling circuits 194.

[0063] Figure 2 An exemplary power supply unit including a fuel-based power generation component and a utility power feed-based component is shown according to some embodiments. For example, Figure 1A The power supply unit 118 shown in FIG. 1 may be a Figure 2 A power supply unit similar to the power supply unit 200 shown in FIG.

[0064] In some embodiments, a power supply unit such as power supply unit 200 may be a skid-mounted unit configured to receive a utility power component 202 in a utility tank 206 and configured to receive a fuel-based power component 204 in a fuel tank 208. The power supply unit may provide direct current (DC) power to a power distribution network such as network 104. In some embodiments, utility component 202 may include a rectifier and one or more transformers. Utility component 202 may be configured to receive low voltage alternating current (AC) power from a utility power source and provide DC power to a power distribution network such as network 104. In some embodiments, fuel component 204 may include a diesel generator, or may include a fast-start multi-fuel turbine. For example, a fast-start multi-fuel turbine may convert natural gas or another fuel into electricity. In some embodiments, power supply unit 200 may include circuit protection, and may supply isolated and regulated DC power to a power distribution network such as network 104.

[0065] In some embodiments, utility components 202 and fuel-based components 204 can be configured to be installed in power supply unit 200 manually or with the assistance of a forklift or small crane. In some embodiments, power supply unit 200 can be configured to be moved from one location (e.g., a pad, such as pad 124) to another location (such as a different pad) at the same or different facility (such as a data center).

[0066] In some embodiments, the power distribution grid and / or the internal cooling grid such as grid 104 can be configured according to a variety of geometric configurations. For example, in some embodiments, the grid can be a horizontal grid, such as Figure 1A The mesh 104 shown in FIG. 1 may be a 3-D spatial mesh, such as Figure 3 In some embodiments, the mesh may be as shown. Figure 4 vertical as shown, or as Figure 5 Circular as shown.

[0067] Figure 3 An exemplary three-dimensional power distribution grid and / or cooling grid according to some embodiments is shown. The grid 300 includes delivery elements 302 and nodes 304 organized in a three-dimensional grid pattern. In some embodiments, a corresponding node in the nodes 304 can be located at the intersection of 3, 4, 5, 6 or more delivery elements. Each of these nodes can receive power and / or cooling fluid via any of the delivery elements connected to the corresponding node, and can supply power and cooling fluid to the electrical loads and / or cooling circuits for the heat generating components served by the corresponding node.

[0068] Figure 4An exemplary vertical power distribution grid and / or cooling grid is shown according to some embodiments. Grid 400 is a vertical grid and includes conveying elements 402 arranged in a vertical plane, with nodes 404 located at intersections of conveying elements 402 .

[0069] Figure 5 An exemplary circular power distribution grid and / or cooling grid according to some embodiments is shown. The grid 500 is a circular grid having semi-circular conveying elements 502 and nodes 504. In some embodiments, the circular grid 500 may include conveying elements arranged in concentric circles, with radial conveying elements connecting continuous bands of concentric semi-circular conveying elements.

[0070] Fig. 6A Shown is a top view of an electric power distribution grid and / or cooling grid and various types of power supply units connected to the grid along the perimeter of the grid, according to some embodiments.

[0071] In some embodiments, power supply units are connected to the power distribution grid on multiple sides of the grid. For example, data center 600 includes grid 602, which includes transmission elements 604 and nodes 606. Fuel-based power supply units 608, utility-based power supply units 610, geothermal-based power supply units 614, wind-based power supply units 616, solar-based power supply units 618, and other renewable power supply units 612 are connected to the first side of grid 602. In addition, similar groups of power supply units are also connected to grid 602 on the other three sides of grid 602 (e.g., the top side, bottom side, and left side of grid 602).

[0072] In some embodiments, a power grid controller (such as Figure 8 602) may select power supply units to be activated or deactivated based on the proximity of the power supply units to electrical loads that will consume power from the power supply units. For example, if the electrical load of the grid 602 is unbalanced such that a greater amount of power is consumed via nodes 606 in the upper right region of the grid 602, the power distribution grid controller may activate power supply units on the top side of the grid 602 and / or on the right side of the grid 602. If the electrical load pattern of the grid 602 changes such that a greater amount of power is consumed via nodes 606 in the lower left region of the grid 602, the power distribution grid controller may deactivate one or more power supply units on the top or right side of the grid 602 and instead activate one or more power supply units on the left or bottom side of the grid 602.

[0073] In some embodiments, the combination of power supply units supplying power to an internal network of a data center, such as network 602, can be adjusted in response to one or more failures of a delivery element and / or node of the network. For example, a failed delivery element on a first side of the network may block the power path from the power supply unit to the load, or may unevenly concentrate the distribution of power on the network onto a limited number of delivery elements. In response, a power distribution network controller can adjust the power supply units supplying power to the network so that power is supplied from another side of the network that does not require power to flow through the failed delivery element and / or reduces the concentrated distribution of power on a limited number of delivery elements.

[0074] Figure 6B A high-ohmic ground path that can be included in a node of an electric power distribution grid according to some embodiments is shown. For example, a high-ohmic ground path 620 can be included in any node of the node 606. The high-ohmic ground path 620 includes a ground tap 626 that is electrically coupled to a high-side electrical path via a high-resistance resistor 622 and electrically coupled to a low-side electrical path via a high-resistance resistor 624. In the event of a ground fault in one of the electrical loads, nodes, or transmission elements, the high-ohmic ground path 620 provides an alternative path to ground, thereby reducing the impact of the ground fault and protecting the electrical loads, transmission elements, and nodes from catastrophic failure during the ground fault.

[0075] Figure 7 A perspective view of a shipping container-based modular data center unit coupled to a node of a power distribution grid and / or a cooling grid is shown according to some embodiments.

[0076] In some embodiments, the facility includes a slab or ground area and a cluster of containerized data centers, such as computing equipment and networking equipment installed in ISO shipping containers or other suitable transportable containers. A power distribution grid and / or cooling grid can be constructed around the containerized data center to provide power support and cooling support to the containerized data center. The power distribution grid and / or cooling grid can function in a similar manner as described above with respect to grid 104. However, instead of providing power and cooling support to racks or groups of racks in a data center building, the grid can provide power and cooling support to containerized data center modules in an open area or warehouse building.

[0077] For example, facility 700 includes floor 724 and containerized data center modules 702, 704, 706, and 708 mounted on floor 724. In addition, transport elements 710, 712, 714, and 716 are mounted on floor 724 and connected to node 718, which is also mounted on floor 724. In some embodiments, transport elements 710, 712, 714, and 716 and node 718 can be installed in an elevated position. Each containerized data center module is connected to node 718 to receive power and is also connected to node 718 to receive (and return) cooling fluid. For example, each of containerized data center modules 702, 704, 706, and 708 is connected to node 718 via power connection 722 and cooling supply and return connection 720.

[0078] Figure 8 A block diagram of an electric power distribution network controller is shown according to some embodiments.

[0079] The power grid controller 802 includes a power capacity monitor 804 , a swing power controller 806 , an additional capacity required alert 808 , a power balance monitor / controller 810 , a fault alert module 812 , and a fault detection module 814 .

[0080] In some embodiments, the power capacity monitor 804 monitors the relationship between the total power consumption and the current power capacity of the power supply unit connected to the network (such as any power distribution network described herein). The power capacity monitor 804 can determine whether it is necessary to activate or deactivate one or more power supply units in order to better match the power capacity with the current power consumption. The power capacity monitor 804 can instruct the swing power controller 806 to start or shut down one or more power supply units in order to better match the power consumption and power capacity. In some embodiments, the swing power controller 806 can preferably activate the renewable power supply unit or otherwise configure the renewable power supply unit to provide power to the network before activating the non-renewable power supply unit. Moreover, the swing power controller 806 can give priority to the utility power supply unit or component over the fuel-based power supply unit or component. In some embodiments, the power capacity monitor and the swing power controller can work together to control the power of the baseline amount of power that will be fed from the renewable power supply and / or the utility power supply consumed by the power distribution network, and cause the peak power that will be fed from the fuel-based power supply during the consumption peak.

[0081] The power capacity monitor 804 may also monitor the overall trend of power consumption and, if the overall trend indicates that additional power supply units need to be connected to the power distribution grid in order to better match power capacity to power consumption, cause activation of the additional capacity required alarm 808. For example, if the power trend indicates that the fuel-based power supply modules are providing an increased amount of power because the renewable-based power supply modules do not have sufficient capacity to meet the current demand, the power capacity monitor 804 may determine that additional renewable power supply units need to be connected to the power distribution grid and may alert facility personnel of this situation via the additional capacity required alarm 808.

[0082] The power balance monitor / controller 810 can monitor the power consumption density on the network. For example, the power balance monitor / controller 810 can monitor whether more power is consumed in a specific area of ​​the network compared to other areas of the network. Moreover, the power balance monitor / controller 810 can monitor the current of the corresponding transmission elements and / or nodes through the network. The power balance monitor / controller 810 can activate and / or deactivate the power supply units on different sides of the network to better balance the power flow through the network. In some embodiments, the power balance monitor / controller 810 can work with a swing power controller (such as a swing power controller 806) to activate and / or deactivate the power supply units on different sides of the network. Moreover, the fault detection module 814 can detect faulty transmission elements and report the fault of one or more transmission elements to the power balance monitor / controller 810 in order to better balance the network. In some embodiments, the power balance monitor / controller 810 can report via the fault alarm module 812 that there is not enough power supply units on different sides of the network to balance the unbalanced condition of the network. Moreover, the fault detection module 814 can report the faulty transmission element via the fault alarm module 812.

[0083] In some embodiments, any power distribution grid described herein may include a power grid controller, such as power grid controller 802 .

[0084] Fig. 9 A high-level flow chart for providing and adjusting the capacity of an electric power distribution grid at a facility is shown, according to some embodiments.

[0085] At 902, a power distribution grid, such as grid 104, is provided for use locally at a facility, such as a data center. At 904, electrical loads at the facility, such as computing devices and / or data storage devices, are connected to the power distribution grid. At 906, a quantity of power supply units corresponding to the expected power consumption level at the facility is provided at the facility. At 908, the power supply units are connected to the power distribution grid along the perimeter of the grid. In some embodiments, 902, 904, 906, and 908 may be performed simultaneously.

[0086] At 910, power from a power supply unit connected to an internal power distribution grid in a facility (e.g., a data center) is distributed to electrical loads in the facility connected to the internal power distribution grid in the facility. The internal power distribution grid includes transmission elements and nodes connected at intersections of the transmission elements, wherein each node receives power from two or more transmission elements.

[0087] At 912, it is determined whether the current power consumption level of the electrical load connected to the power distribution network deviates from the expected power consumption level (which is used to determine the amount of power supply units) by more than a threshold amount. If the current power consumption level deviates from the expected power consumption level by no more than the threshold amount, the process returns to 910, and power is distributed to the electrical load from the current amount of power supply units connected to the power distribution network. In some embodiments, 912 can be performed by a power capacity monitor of a power grid controller (such as power capacity monitor 804 of power grid controller 802).

[0088] If the current power consumption level deviates from the expected power consumption level by more than a threshold amount, then at 914, one or more additional power supply units are provided at the facility, and at 916, the additional power supply units are connected to the power distribution grid while continuing to provide power to the electrical loads connected to the power distribution grid. Conversely, if the current power consumption level deviates from the expected power consumption level by more than a threshold amount in the other direction (e.g., the current power consumption level is significantly less than the expected power consumption level), then at 914, one or more of the power supply units are disconnected from the power distribution grid, and at 916, they are removed from the facility, e.g., for reuse at another facility.

[0089] Fig.10 A high-level flow diagram for distributing power to loads at a facility via a power distribution grid is shown, according to some embodiments.

[0090] In some embodiments, the power supply unit is managed so that power is preferably provided from the renewable power supply unit and / or the utility power supply unit when available. For example, at 1002, a baseline level of power consumed by the electrical load is distributed to the electrical load from the renewable power supply unit or the utility power supply unit via a power distribution grid. For example, the baseline amount of power can be a steady-state amount of power consumed by the electrical load under normal operating conditions.

[0091] At 1004, it is determined whether there is a capacity loss from the utility power unit or the renewable power unit. For example, a solar-based power unit may produce less electricity when the sun is not shining or is shaded. As another example, a wind-based power unit may produce less electricity when the wind is not blowing. Additionally or alternatively, a renewable power unit or a utility power unit may fail, resulting in a loss of power capacity. If there is no capacity loss, a baseline amount of power continues to be provided by the renewable power unit and / or the utility power unit. If there is a capacity loss, at 1008, a marginal amount of power to make up for the capacity loss may be provided to the power distribution grid from one or more of the fuel-based power units connected to the power distribution grid.

[0092] At 1006, it is determined whether the power consumed by the electrical loads has increased by more than a threshold amount over the baseline amount. If not, the power distribution grid continues to distribute power received from the renewable power supply units and / or utility power supply units connected to the power distribution grid. If the power consumed by the electrical loads has increased above the baseline amount, a marginal amount of power that satisfies the increased power demand is provided to the power distribution grid from one or more fuel-based power supply units connected to the power distribution grid.

[0093] In some embodiments, there may be no utility power units connected to the electricity distribution grid, and the baseline amount of electricity may be provided solely by the renewable power units, with a marginal amount of power being provided to the electricity distribution grid from the fuel-based power units to compensate for the shortfall in power provided by the renewable power units.

[0094] In some embodiments, such as Fig.10 The distribution of power to loads at a facility via the power distribution grid as described herein may be managed by a power balance monitor / controller of a power grid controller, such as power balance monitor / controller 810 of power grid controller 802 .

[0095] Fig.11 A high-level flow chart for monitoring and / or responding to faulty transmission elements of a power distribution grid and / or cooling grid is shown, according to some embodiments.

[0096] At 1102, a fault at a delivery unit is detected. The fault may be a short circuit in electrical wiring of the delivery element or a leak in one or more of the cooling fluid lines of the delivery element. At 1104, a warning alarm is issued to alert facility personnel of the fault of the delivery element.

[0097] At 1106, another fault is detected at another conveying element of the same web. At 1108, another warning alarm is issued, and at 1110, it is determined whether the total number of faulty conveying elements of the web or a zone of the web exceeds a threshold number of allowable faulty conveying elements. If the threshold has not been reached, at 1114, the system continues to monitor the web for faults.

[0098] If the threshold has been reached, at 1112, the system (e.g., a power distribution controller and / or a fault alarm module) issues a work order to repair or replace the number of failed delivery elements. Since each node receives power from more than two delivery elements, the power distribution network in a data center or other facility can tolerate delivery element failures without the need to immediately repair the failed delivery elements. For example, a node connected to four delivery elements can be connected to two failed delivery elements and still be connected to two other non-faulty delivery elements that provide redundant power support. In networks with more delivery element connections at nodes, such as 3-D networks, even more delivery unit component failures can be tolerated. Moreover, accumulating delivery element failures before performing maintenance can improve maintenance efficiency.

[0099] In some embodiments, such as Fig.11 The monitoring and / or response to faulty transmission elements of the power distribution grid and / or cooling grid described in the foregoing may be performed by a fault detection module and / or a fault alarm module of a power grid controller (such as the fault detection module 814 and the fault alarm module 812 of the power grid controller 802).

[0100] Internal cooling network

[0101] As described above, in some embodiments, an intranet within a facility may include delivery elements and nodes that provide power to connected loads, may include delivery elements and nodes that provide cooling support to connected cooling loops that cool heat-generating elements in electrical loads, or may include delivery elements and nodes that provide both power and cooling support to connected electrical loads. Figures 12 to 18 The description of discusses in more detail the cooling support provided by the intranet in the facility. However, it should be understood that in some embodiments, the following description of Figures 12 to 18 The described embodiments can be combined with the above Figures 2 to 11 In addition, as described above, FIGS. 1 to 2 Figure 4 The grid 104 described in provides both power support and cooling support to the connected electrical loads.

[0102] Fig.12 A top view of a cooling grid including various types of heat rejection units coupled to a perimeter of the cooling grid is shown according to some embodiments.

[0103] In some embodiments, the power distribution grid and / or the internal cooling grid can be connected to heat rejection units on multiple sides of the grid. This can allow the cooling to be balanced so that for parts of the grid that reject more waste heat, more heat rejection units are activated to reject the waste heat.

[0104] In some embodiments, the power distribution grid and / or the internal cooling grid may be connected to an insulated coolant reservoir that acts as a buffer or capacitor for the internal cooling grid, wherein when there is excess cooling capacity, the temperature of the large volume of water in the insulated coolant reservoir is reduced, and when there is a lack of cooling capacity, heat rejected to the cooling grid is absorbed by the large volume of water in the insulated reservoir.

[0105] In some embodiments, the cooling grid 1202 includes conveying elements 1204 arranged in a grid pattern and nodes 1206 connected to the conveying elements 1204 at intersections of the conveying elements 1204. In some embodiments, the cooling grid is internal to a facility such as a data center 1200. For example, the cooling grid 1202 can be located within the wall of the data center building 1200.

[0106] In some embodiments, various types of heat rejection units may be connected to the cooling grid 1202 at the perimeter of the cooling grid 1202. In some embodiments, the heat rejection unit may be located at the data center premises of the data center 1200, but may be located outside the wall of the data center building 1200. In some embodiments, the evaporative cooling module 1214, the free cooling module 1216, and the mechanical cooling module 1218 may be located on a first side (e.g., bottom side) of the cooling grid 1202, and the absorption refrigeration module 1208, the evaporative cooling module 1210, and the free cooling module 1212 may be located on the other side (e.g., left side) of the cooling grid 1202. In some embodiments, an adiabatic coolant reservoir 1220 may be connected to the cooling grid 1202. In some embodiments, the cooling fluid flowing through the cooling grid 1202 may flow through the transport element 1204 and the node 1206 via one or more cooling circuits to a heat exchanger connected to the node 1206. Furthermore, the cooling fluid flowing through the cooling network 1202 may flow through heat rejection units connected to the cooling network, such as absorption refrigeration module 1208, evaporative cooling module 1210, free cooling module 1212, evaporative cooling module 1214, free cooling module 1216, and mechanical cooling module 1218. In some embodiments, the cooling fluid flowing through the cooling network 1202 may flow through an insulated coolant reservoir 1220. For example, when the cooling fluid flows through the insulated coolant reservoir 1220, at least some of the contents of the tank may be turned over so that the temperature of the bulk coolant in the insulated coolant reservoir is adjusted based on the temperature of the cooling fluid flowing through the cooling network 1202. In some embodiments, the insulated coolant reservoir 1220 may maintain a constant liquid level when the cooling fluid flows through the tank.

[0107] In some embodiments, the heat rejection unit may be a modular unit and may be moved by a forklift, and may include a standardized connector configured to connect to a standard connector on the perimeter of the cooling grid 1202 .

[0108] Fig.13 A liquid-cooled heat-generating component is shown being cooled by a cooling fluid received from and returned to a cooling network via nodes of the cooling network, according to some embodiments.

[0109] In some embodiments, a cooling circuit connected to a node of a cooling grid can flow a cooling fluid through one or more direct heat exchangers mounted on a heat generating component of an electrical load. For example, the heat exchanger can be a cold plate heat exchanger, a radiator, an immersion cooling heat exchanger, or other type of heat exchanger that exchanges heat between a flowing cooling fluid and a heat generating component of an electrical load. In some embodiments, any cooling circuit described herein can flow a cooling fluid through a heat generating component such as a heat generating component of an electrical load. Fig.13 The direct heat exchanger.

[0110] In some embodiments, cooling circuit 1314 flows cooling fluid from a supply manifold of node 1304 (which may be connected to four or more transport elements) via supply line 1306 to direct heat exchanger 1312, which removes heat from heat generating components 1310 installed in rack 1302. The cooling fluid that has absorbed the heat removed from heat generating components 1310 via heat exchanger 1312 may flow back to a return manifold of node 1304 (which may be connected to four or more transport elements) via return line 1308.

[0111] In some embodiments, in addition to or in lieu of Fig.12 The described direct heat exchangers, racks or electrical loads may include a liquid-to-air heat exchanger that transfers heat between air and liquid flowing through the liquid-to-air heat exchanger. The cooled air may then be directed through the heat generating components of the electrical load to remove waste heat from the electrical load.

[0112] For example, Fig.14 An air-cooled heat-generating component is shown being cooled by a cooling fluid received from and returned to the cooling network via nodes of the cooling network, according to some embodiments.

[0113] In some embodiments, cooling loop 1416 flows cooling fluid from a supply manifold of node 1404 (which may be connected to four or more transport elements) via supply line 1406 to liquid-air heat exchanger 1412, which removes heat from air 1416, which is directed to heat generating components 1314 mounted in rack 1402 via fan 1310. The cooling fluid that has absorbed the heat removed from air 1416 via liquid-air heat exchanger 1412 may flow back to a return manifold of node 1404 (which may be connected to four or more transport elements) via return line 1408. In some embodiments, where the electrical loads are rack-mounted servers, each server may include a separate liquid-air heat exchanger. Moreover, in some embodiments, a single liquid-air heat exchanger may cool air supplied to multiple electrical loads mounted in a rack.

[0114] Fig.15 An exemplary configuration of a cooling grid operating at a pressure below atmospheric pressure is shown according to some embodiments.

[0115] In some embodiments, cooling network such as any cooling network described herein can be operated under vacuum. For example, the pressure in the cooling supply line and / or cooling return line of the cooling network can be less than the atmospheric pressure at the facility where the cooling network is located. In some embodiments, the pressure can be less than 760mmHg. In some embodiments, operating the cooling network at a pressure lower than one atmosphere can prevent the cooling fluid from leaking from the cooling network into the electrical load. For example, in the case of a leaking conveying element or node, air may leak into the supply line or return line of the cooling network, rather than the cooling fluid leaking from the supply line or return line of the cooling network.

[0116] In some embodiments, an array of pressure and / or flow sensors may be distributed throughout the cooling network to measure the respective pressures of the cooling fluid supply and return lines and also measure the flow through these lines. The pressure and flow monitoring system may determine flow imbalances through the system and / or leaks in the system based on the measured pressures and flows.

[0117] For example, the cooling network 1520 includes delivery elements 1506 arranged in a grid pattern and nodes 1508 at the intersections of the grid pattern. The pressure transmitters 1502 are connected to corresponding ones of the delivery elements and measure corresponding supply pressures and return pressures of the cooling fluid flowing through the corresponding delivery elements. Moreover, the flow transmitters 1504 are connected to corresponding ones of the delivery elements and measure corresponding supply flows and return flows of the cooling fluid flowing through the corresponding delivery elements. In some embodiments, the pressure and flow monitoring system 1500 can receive pressure and flow measurements from the pressure transmitters 1502 and the flow transmitters 1504. In some embodiments, the pressure and flow monitoring system can be part of a fault / leak detection module of a cooling network controller, such as with respect to Fig.16 Furthermore, in some embodiments, the pressure and flow measurements received from the pressure transmitter 1502 and the flow transmitter 1504 may be communicated to a flow / pressure controller of a cooling network controller, such as with respect to Fig.16 Described in more detail.

[0118] In some embodiments, any heat removal unit described herein may be a modular heat removal unit, e.g. Fig.1515. The modular heat removal unit 1510 is shown in FIG. 15. The modular heat removal unit 1510 includes a reservoir 1512 that provides a holding location for cooling fluid to be drawn from the cooling grid 1520 as it is drawn therefrom. The modular heat removal module 1510 also includes a pump 1516 that draws heated cooling fluid from the cooling grid 1520 and causes fresh (or cooled) cooling fluid to be drawn from the reservoir 1512. The heat removal module 1510 also includes a heat removal unit 1514 between the discharge of the pump 1516 and the reservoir 1512. In some embodiments, the pump 1516 can cause the heated cooling fluid drawn from the cooling grid 1520 to flow through the heat removal unit 1516 and into the reservoir 1512. As the cooling fluid flows through the heat removal unit 1514, heat can be removed from the cooling fluid, and the reservoir 1512 can be highly insulated so that the cooling fluid cooled by the heat removal unit 1514 remains cool while in the reservoir 1512. In some embodiments, the heat removal unit 1514 can be an evaporative cooler, a free cooling module, a mechanical chiller, an absorption refrigeration unit, a geothermal cooler, or other type of heat removal device. In some embodiments, the modular heat removal module can optionally include a vacuum pump 1518 to remove air from the reservoir 1512 and ensure that the pressure of the cooling fluid flowing through the cooling network 1520 remains below one atmosphere. In addition, in some embodiments, a flow transmitter (not shown) can be located on the discharge side of the vacuum pump to measure the amount of air removed from the cooling network.

[0119] In some embodiments, the supply lines, return lines, and pumps 1516 can be sized so that the supply pressure and return pressure of the corresponding supply lines and cooling lines remain consistent when the cooling network 120 flows the cooling fluid through the cooling network. For example, the diameters of the supply lines and return lines can be large enough so that pressure losses due to fluid flow are negligible. Moreover, by selecting relatively large diameter manifolds and fittings, pressure losses due to pipe connections such as at nodes and manifolds within nodes can be negligible.

[0120] Fig.16 A block diagram of a cooling network controller according to some embodiments is shown.

[0121] In some embodiments, any cooling grid described herein may include a cooling grid controller, such as cooling grid controller 1602. In some embodiments, cooling grid controller 1602 and power grid controller 802 may be combined into a common controller for the grid, or may be implemented as separate controllers.

[0122] In some embodiments, a cooling network controller (such as cooling network controller 1602 ) includes a cooling capacity monitor 1604 , a flow / pressure controller 1606 , an additional capacity required controller 1608 , a flow balance monitor / controller 1610 , a fault alarm 1612 , and a fault / leak detection module 1614 .

[0123] In some embodiments, the cooling capacity monitor 1604 monitors the relationship between the total cooling load and the current capacity of the heat rejection unit connected to the cooling network (such as any cooling network described herein). The cooling capacity monitor 1604 can determine whether it is necessary to activate or deactivate one or more heat rejection units in order to better match the cooling capacity with the current cooling load. The cooling capacity monitor 1604 can instruct the flow / pressure controller 1606 to start or shut down one or more heat rejection units in order to better match the cooling capacity and the cooling load. In some embodiments, the flow / pressure controller 1606 can preferably activate a lower cost heat rejection unit, such as a free cooling heat rejection unit or an evaporative cooling heat rejection unit, to provide cooling to the network before activating a higher cost heat rejection unit such as a mechanical refrigerator. In some embodiments, the cooling capacity monitor and the flow / pressure controller can jointly control the cooling fluid flow so that a baseline amount of cooling fluid is provided to the cooling network from the low cost heat rejection unit, and can cause the higher cost heat rejection unit to flow additional fluid through the cooling network during the peak of waste heat discharged into the cooling network. Alternatively, the flow rate may be controlled to be constant, and additional heat rejection units may be activated or deactivated to reduce the overall temperature of the cooling fluid flowing through the cooling network.

[0124] The cooling capacity monitor 1604 may also monitor the overall trend of the electrical load and, if the overall trend indicates that additional heat rejection units need to be connected to the cooling grid to better match cooling capacity to cooling load, cause activation of the additional capacity needed alarm 1608. For example, if the cooling trend indicates that an increased amount of waste heat is being removed by a higher cost heat rejection unit because the lower cost heat rejection unit does not have sufficient capacity to meet the current demand, the cooling capacity monitor 1604 may determine that additional lower cost heat rejection units need to be connected to the cooling grid and may alert facility personnel to this situation via the additional capacity needed alarm 1608.

[0125] The flow balance monitor / controller 1610 can monitor the cooling fluid flow on the network, which can indicate the level of waste heat being discharged into the cooling network. For example, the flow balance monitor / controller 1610 can monitor whether more fluid is flowing in a specific area of ​​the network compared to other areas of the network. Moreover, the flow balance monitor / controller 1610 can monitor the current cooling fluid flow through the corresponding delivery elements and / or nodes of the network. The flow balance monitor / controller 1610 can activate and / or deactivate heat removal units located on different sides of the network to better balance the cooling fluid flowing through the network. In some embodiments, the flow balance monitor / controller 1610 can work with a flow / pressure controller (such as flow / pressure controller 1606) to activate and / or deactivate heat removal units on different sides of the network. Moreover, the fault detection module 1614 can detect faulty delivery elements and report the fault of one or more delivery elements to the flow balance monitor / controller 1610 in order to better balance the network. In some embodiments, flow balance monitor / controller 1610 may report an unbalanced condition where there are not enough heat rejection units on different sides of the grid to balance the grid via fault alarm module 1612. Also, fault detection module 1614 may report a faulty transport element via fault alarm module 1612.

[0126] Fig.17 A high-level flow chart for providing a cooling grid and adjusting the cooling grid due to varying cooling demands of loads at a facility is shown, according to some embodiments.

[0127] At 1702, a cooling grid for internal use in a facility such as a data center is provided. The internal cooling grid includes conveying elements arranged in a grid pattern and nodes at intersections of the grid. Each of the nodes is configured to receive cooling fluid from two or more conveying elements and is also configured to return cooling fluid to the two or more conveying elements of the cooling grid.

[0128] At 1704, the cooling grid is connected to a cooling loop that removes waste heat from heat-generating components of the electrical loads at the facility. For example, the cooling loop may flow cooling fluid received from the cooling grid through a direct cooling heat exchanger, a liquid-to-air heat exchanger, or a combination of both. At 1706, a quantity of heat rejection units having a common cooling capacity that matches the expected cooling load at the facility is provided. At 1708, the heat rejection units are connected to the internal cooling grid along the perimeter of the cooling grid.

[0129] At 1710, cooling fluid is supplied to a direct heat exchanger and / or a liquid-to-air heat exchanger that removes waste heat from heat generating components of the electrical load.

[0130] At 1712, it is determined whether the total cooling load on the cooling grid has increased or decreased by more than a threshold amount compared to the expected cooling load at 1708. If the change in the total cooling load does not exceed the threshold amount, the system continues to supply cooling fluid to cool the heat generating components of the electrical loads at the facility. If the total cooling capacity has increased or decreased by more than a threshold amount compared to the original expected cooling load used to determine the number of heat rejection units connected to the cooling grid, additional heat rejection units are connected to the cooling grid (or removed from the cooling grid) at 1716.

[0131] At 1714, it is determined whether the heat rejection density of one or more electrical loads cooled by the cooling network has increased or decreased by more than a threshold amount. If not, the system continues to supply coolant to the air-cooled and / or liquid-cooled electrical loads in the facility. However, if the cooling density changes, for example, more concentrated waste heat is rejected by the heat-generating components of the electrical loads, such as when the server is upgraded to a more powerful processor group, then at 1718, the electrical load is switched between being air-cooled and being liquid-cooled to adjust the heat rejection density from the corresponding electrical load. In some embodiments, a cooling capacity monitor (such as cooling capacity monitor 1604 of cooling network controller 1602) can execute 1712 to determine whether the total cooling network load has increased or decreased. Moreover, in some embodiments, flow / balance alarm 1610 can execute 1714 to determine whether the heat rejection density has changed.

[0132] Fig.18 A high-level flow chart for monitoring a cooling grid for leaks is shown, according to some embodiments.

[0133] At 1802, a pressure and flow monitoring system (such as Fig.15 Pressure and flow monitoring system 1500 described herein or with respect to Fig.16 The fault / leak detection module 1614 of the cooling network controller 1602 depicted monitors the supply pressure and return pressure in the delivery elements of the cooling network. Also at 1804, the pressure and flow monitoring system or the fault / leak detection module 1614 of the cooling network controller 1602 monitors the flow of air exhausted from the cooling network (such as air removed from a reservoir via a vacuum pump). At 1806, it is determined whether the exhaust flow exceeds a threshold amount, and if not, the system continues to monitor the exhaust flow at 1804.

[0134] If the exhaust flow does exceed the threshold amount, then at 1808, the corresponding pressure distribution on the cooling grid measured by the pressure transmitter array is analyzed to identify one or more leaking delivery elements and / or nodes. At 1810, the leaking delivery element (and / or associated node) is automatically isolated.

[0135] At 1812, it is determined whether the number of isolated delivery elements exceeds a threshold number of delivery elements. If not, the system continues to monitor additional leaking or faulty delivery elements. However, if the number of delivery elements exceeds the threshold number, the system issues a work order to replace the faulty or leaking delivery element that has been isolated.

[0136] Exemplary Computer System

[0137] Fig.19 1 to 10. Fig.18 The computer system 1900 may be configured to perform any or all of the above embodiments. In various embodiments, the computer system 1900 may be any of a variety of types of devices, including but not limited to a personal computer system, a desktop computer, a laptop computer, a notebook computer, a mainframe computer system, a network computer, a programmable logic controller PLC, or generally any type of computing or electronic device.

[0138] Various embodiments of the power distribution grid controller and / or cooling grid controller as described herein may be executed in one or more computer systems 1900 that may interact with various other devices. Fig.18 Any component, action or function described may be configured as Fig.19 1900. In the illustrated embodiment, the computer system 1900 includes one or more processors 1910 coupled to a system memory 1920 via an input / output (I / O) interface 1930. The computer system 1900 also includes a network interface 1940 coupled to the I / O interface 1930, and one or more input / output devices 1950, such as a cursor control device 1960, a keyboard 1970, and a display 1980. In some cases, it is contemplated that an embodiment may be implemented using a single instance of the computer system 1900, while in other embodiments, multiple such systems or multiple nodes making up the computer system 1900 may be configured to host different parts or instances of the embodiment. For example, in one embodiment, some elements may be implemented via one or more nodes of the computer system 1900, which are different from those nodes that implement other elements.

[0139] In various embodiments, computer system 1900 may be a uniprocessor system including one processor 1910, or a multiprocessor system including several processors 1910 (e.g., two, four, eight, or another suitable number). Processor 1910 may be any suitable processor capable of executing instructions. For example, in various embodiments, processor 1910 may be a general-purpose or embedded processor that implements any of a variety of instruction set architectures (ISAs), such as an x86, PowerPC, SPARC, or MIPS ISA, or any other suitable ISA. In a multiprocessor system, each of processors 1910 may typically, but not necessarily, implement the same ISA.

[0140] The system memory 1920 may be configured to store program instructions 1922 accessible by the processor 1910. In various embodiments, the system memory 1920 may be implemented using any suitable memory technology such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash type memory, or any other type of memory. In the illustrated embodiment, the program instructions 1922 may be configured to implement a power distribution network controller and / or a cooling network controller having any of the functionality described above. In some embodiments, program instructions and / or data may be received, sent, or stored on different types of computer accessible media or similar media separate from the system memory 1920 or computer system 1900. Although the computer system 1900 is described as implementing the functionality of the functional blocks of the previous figures, any functionality described herein may be implemented via such a computer system.

[0141] In one embodiment, the I / O interface 1930 may be configured to coordinate I / O traffic between the processor 1910, the system memory 1920, and any peripheral devices in the device, including the network interface 1940 or other peripheral interfaces such as the input / output device 1950. In some embodiments, the I / O interface 1930 may perform any necessary protocol, timing, or other data transformations to convert data signals from one component (e.g., the system memory 1920) into a format suitable for use by another component (e.g., the processor 1910). In some embodiments, the I / O interface 1930 may include support for devices attached through various types of peripheral buses such as a variant of the peripheral component interconnect (PCI) bus standard or the universal serial bus (USB) standard. In some embodiments, the functionality of the I / O interface 1930 may be split into two or more independent components, such as a north bridge and a south bridge. Moreover, in some embodiments, some or all of the functionality of the I / O interface 1930 (such as the interface of the system memory 1920) may be directly incorporated into the processor 1910.

[0142] The network interface 1940 may be configured to allow data to be exchanged between the computer system 1900 and other devices attached to the network (e.g., a carrier or proxy device) or between nodes of the computer system 1900. In various embodiments, the network may include one or more networks, including but not limited to a local area network (LAN) (e.g., an Ethernet or an enterprise network), a wide area network (WAN) (e.g., the Internet), a wireless data network, some other electronic data network, or some combination thereof. In various embodiments, the network interface 1940 may support communication via a wired or wireless general data network (such as any type of Ethernet); via a telecommunications / telephone network (such as an analog voice network or a digital fiber optic communication network); via a storage area network (such as a Fiber Channel SAN), or via any other suitable type of network and / or protocol.

[0143] In some embodiments, input / output devices 1950 may include one or more display terminals, keyboards, keypads, touch pads, scanning devices, voice or optical recognition devices, or any other device suitable for entering or retrieving data through one or more computer systems 1900. Multiple input / output devices 1950 may be present in computer system 1900, or may be distributed across various nodes of computer system 1900. In some embodiments, similar input / output devices may be separate from computer system 1900 and may interact with one or more nodes of computer system 1900 through a wired or wireless connection (such as through network interface 1940).

[0144] like Fig.19 As shown, memory 1920 may include program instructions 1922, which may be executable by a processor to implement any of the above elements or actions. In one embodiment, the program instructions may implement the above method. In other embodiments, different elements and data may be included. Note that the data may include any of the above data or information.

[0145] Those skilled in the art will appreciate that computer system 1900 is merely illustrative and is not intended to limit the scope of the embodiments. Specifically, computer systems and devices may include any combination of hardware or software that can perform the indicated functions, including computers, network devices, etc. Computer system 1900 may also be connected to other devices not shown, or may alternatively be operated as an independent system. In addition, in some embodiments, the functions provided by the components shown may be combined in fewer components or distributed in additional components. Similarly, in some embodiments, the functions of some components in the components shown may not be provided, and / or other additional functions may be used.

[0146] Those skilled in the art will also understand that, although various items are described as being stored in memory or on a storage device when in use, for the purpose of memory management and data integrity, these items or a portion of them may be transferred between memory and other storage devices. Alternatively, in other embodiments, some or all of the software components may be executed in a memory on another device and communicated with the computer system shown via inter-computer communication. Some or all of the system components or data structures may also be stored (e.g., as instructions or structured data) on a computer-accessible medium or portable article to be read by an appropriate drive, various examples of which are described above. In some embodiments, instructions stored on a computer-accessible medium separated from the computer system 1900 may be transmitted to the computer system 1900 via a transmission medium or via a signal (such as an electrical signal, an electromagnetic signal, or a digital signal) conveyed via a communication medium (such as a network and / or a wireless link). Various embodiments may also include receiving, sending, or storing instructions and / or data implemented on a computer-accessible medium according to the above description. In general, computer-accessible media may include non-transitory computer-readable storage media or memory media such as magnetic or optical media, e.g., disks or DVD / CD-ROMs, volatile or non-volatile media such as RAM (e.g., SDRAM, DDR, RDRAM, SRAM, etc.), ROM, etc. In some embodiments, computer-accessible media may include transmission media or signals such as electrical, electromagnetic, or digital signals communicated via a communication medium such as a network and / or wireless link.

[0147] Embodiments of the present disclosure may be described according to the following terms: 1. A system comprising:

[0148] a power supply unit; and

[0149] A data center, comprising:

[0150] Electrical loads; and

[0151] an internal power distribution network connected to the power supply unit and the electrical load, wherein the internal power distribution network is located within the data center and comprises:

[0152] modular power delivery elements arranged in a grid pattern; and

[0153] nodes, the nodes being located at intersections of the grid pattern;

[0154] wherein the modular power delivery elements and the nodes are arranged in the data center such that each node is configured to receive power from two or more of the modular power delivery elements connected to the node, and

[0155] Wherein the electrical loads are arranged in the data center such that the electrical loads receive power from the grid via respective ones of the nodes.

[0156] 2. A system as described in clause 1, wherein:

[0157] The power supply unit is configured to supply direct current (DC) power to the internal power distribution grid; and

[0158] The internal power distribution network is configured to distribute the DC power received from the power supply unit to the electrical loads.

[0159] 3. A data center as described in any of clauses 1 to 2, wherein:

[0160] A respective one of the modular delivery elements comprises a high-side voltage path and a low-side voltage path, and

[0161] The internal power distribution network includes a ground connection, which is connected between one or more high-side paths of the high-side paths of the modular power delivery element and one or more low-side paths of the low-side paths of the modular power delivery element via a high-ohmic resistor, wherein the ground connection and the high-ohmic resistor provide ground fault protection for the internal power distribution network of the data center.

[0162] 4. A data center as described in any of clauses 1 to 3, wherein the internal power distribution network is configured to simultaneously supply power to a given electrical load connected to a given node via two or more of the modular power delivery elements connected to the given node.

[0163] 5. An electric power distribution network, comprising:

[0164] Modular power delivery elements configured to be arranged in a grid pattern to supply power to computing devices, networking devices, or data storage devices in a facility; and

[0165] nodes, the nodes being located at intersections of the grid pattern;

[0166] The modular power delivery elements and the nodes are arranged so that a corresponding one of the nodes is configured to receive power from two or more of the modular power delivery elements connected to the corresponding node, and to supply the received power to one or more of the computing devices, the networking devices or the data storage devices in the facility.

[0167] 6. The power distribution grid of clause 5, wherein the power distribution grid is configured to distribute direct current (DC) power.

[0168] 7. An electric power distribution network as claimed in clause 5 or 6, further comprising:

[0169] A power supply unit is located at the facility connected to a perimeter of the power distribution grid.

[0170] 8. The power distribution network of clause 7, wherein each of the power supply units has a capacity of 750 kilowatts (KW) or less.

[0171] 9. An electricity distribution network as claimed in clause 7 or 8, wherein each of the power supply units supplies less than 20% of the total amount of electricity provided to the electricity distribution network.

[0172] 10. An electricity distribution network as claimed in any one of clauses 7 to 9, wherein the power supply unit comprises one or more of the following:

[0173] Solar power supply unit;

[0174] Wind energy based power supply unit;

[0175] A geothermal based power supply unit; or

[0176] Power supply unit based on hydroelectric power.

[0177] 11. An electric power distribution network as claimed in any one of clauses 7 to 10, wherein the power supply unit comprises one or more of the following:

[0178] a utility power component configured to convert alternating current power received from a utility power source into direct current (DC) power; or

[0179] A fuel-based power supply component is configured to generate DC power using one or more types of fuel.

[0180] 12. An electricity distribution network as claimed in any one of clauses 5 to 11, wherein:

[0181] The computing device, the networking device, or the data storage device of the facility is included in a portable shipping container; and

[0182] The power distribution grid is configured to distribute power to the computing device, the networking device, or the data storage device included in the portable shipping container.

[0183] 13. An electricity distribution network as claimed in any one of clauses 5 to 12, wherein:

[0184] The facility is a data center building; and

[0185] The power distribution grid is configured to distribute power to the computing devices, the networking devices, or the data storage devices included in a data center building.

[0186] 14. An electric power distribution network as described in any of clauses 5 to 13, wherein the modular power transmission elements of the electric power distribution network are arranged in a three-dimensional grid pattern, and wherein respective ones of the nodes are located at three-dimensional intersections of the modular transmission elements.

[0187] 15. A method comprising:

[0188] An electric power distribution grid for a facility is provided, wherein the electric power distribution grid comprises:

[0189] modular power delivery elements arranged in a grid pattern to supply power to computing devices, networking devices, or data storage devices in the facility; and

[0190] nodes, the nodes being located at intersections of the grid pattern; and

[0191] Power is distributed to the computing devices, the networking devices, or the data storage devices in the facility via the nodes of the power distribution network, wherein power from two or more of the modular power delivery elements connected to the corresponding nodes is supplied to the corresponding nodes.

[0192] 16. The method of clause 15, further comprising:

[0193] A quantity of power supply units is provided to supply power to the power distribution grid, wherein the quantity of power supply units has a combined power capacity corresponding to current power consumption levels of the computing devices and the data storage devices in the facility.

[0194] 17. The method of clause 16, further comprising:

[0195] responsive to an anticipated increase in current power consumption levels of the computing devices and the data storage devices in the facility, providing additional power supply units; and

[0196] The additional power supply unit is connected to the power distribution network while the quantity of power supply units continues to supply power to the power distribution network.

[0197] 18. The method of clause 16 or 17, wherein the quantity of power supply units comprises utility power supply units or renewable energy based power supply units and fuel based power supply units; and

[0198] Wherein distributing power to the computing device, the networking device, or the data storage device in the facility comprises:

[0199] providing a baseline amount of power consumed by the computing device and the data storage device from the utility power unit or the renewable energy-based power unit; and

[0200] A marginal amount of power is provided to the computing device and the data storage device during spikes in power consumption from the fuel-based power supply unit.

[0201] 19. The method of any one of clauses 16 to 18, further comprising:

[0202] detecting a failure of a modular delivery element on a first side of the power distribution network; and

[0203] Switching from a first power supply unit on the first side of the power distribution network supplying power to the power distribution network to a second power supply unit on the second side of the power distribution network supplying power to the power distribution network.

[0204] 20. The method of any one of clauses 15 to 19, further comprising:

[0205] Detecting failure of modular conveying elements or nodes; and

[0206] Generating a command to repair said faulty modular conveying element or node is inhibited until a threshold number of faulty modular conveying elements or nodes has been reached.

[0207] 21. A system comprising:

[0208] A data center, comprising:

[0209] heat generating components installed at various locations within the data center; and

[0210] an internal cooling network positioned in the data center, wherein the internal cooling network comprises:

[0211] cooling fluid delivery conduits arranged in a grid pattern; and

[0212] nodes, the nodes being located at intersections of the grid pattern; and

[0213] a heat removal unit connected to the internal cooling network and configured to remove heat from a cooling fluid flowing through the internal cooling network,

[0214] wherein a corresponding node among the nodes is configured to receive cooling fluid from a plurality of cooling fluid delivery pipes among the cooling fluid delivery pipes connected to the corresponding node and return the cooling fluid to a plurality of cooling fluid delivery pipes among the cooling fluid delivery pipes, and

[0215] The heat generating components are arranged in the data center such that respective ones of the heat generating components are cooled by cooling fluid received from the internal cooling network via respective ones of the nodes and returned to the internal cooling network.

[0216] 22. The system of clause 21, wherein the heat generating components are included in servers or networking devices that are mounted in racks located at different locations within the data center.

[0217] 23. The system of clause 21 or 22, wherein at least some of the racks further comprise a liquid-to-air heat exchanger, and

[0218] The heat generating components in at least some of the racks are air cooled by air which has been cooled by a portion of the cooling fluid received from the internal cooling network via corresponding nodes among the nodes and returned to the internal cooling network, wherein the portion of the cooling fluid received from the cooling network via the corresponding nodes and returned to the cooling network flows through corresponding liquid-to-air heat exchangers.

[0219] 24. The system of any of clauses 21 to 23, wherein the heat generating components are liquid cooled by respective liquid cooling circuits connected to the internal cooling network via respective ones of the nodes.

[0220] 25. A cooling network comprising:

[0221] cooling fluid delivery elements arranged in a grid pattern to provide cooling support to computing, networking, or data storage devices in a facility; and

[0222] nodes, the nodes being located at intersections of the grid pattern;

[0223] wherein the cooling fluid transport elements and the nodes are arranged in the facility such that respective ones of the nodes are configured to receive cooling fluid from a plurality of the fluid transport elements and return cooling fluid to a plurality of the fluid transport elements, and

[0224] The corresponding nodes in the nodes are configured as follows:

[0225] providing the received cooling fluid to a cooling circuit for removing heat from one or more of the computing device, the networking device, or the data storage device, and

[0226] A cooling fluid is received from the cooling circuit, the cooling fluid having been used to remove heat from one or more of the computing device, the networking device, or the data storage device.

[0227] 26. A cooling network as claimed in clause 25, wherein each node is connected to:

[0228] a cooling fluid supply line included in a respective cooling fluid delivery element of the plurality of cooling fluid delivery elements connected to the node,

[0229] a cooling fluid return line included in the respective cooling fluid delivery element of the plurality of cooling fluid delivery elements connected to the node,

[0230] a supply line from the node to one or more of the computing device, the networking device, or the data storage device that is cooled using cooling fluid from the node; and

[0231] A return line from one or more of the computing device, the networking device, or the data storage device that is cooled using the cooling fluid from the node.

[0232] 27. The cooling network of clause 25 or 26, wherein the respective absolute pressures of the cooling fluid flowing in the cooling fluid supply line and the cooling fluid return line are less than one atmosphere.

[0233] 28. The cooling network of clause 26 or 27, wherein the cooling fluid supply line and the cooling fluid return line are configured to provide cooling fluid to the nodes at a consistent supply pressure and to receive cooling fluid from the nodes at a consistent return pressure.

[0234] 29. The cooling grid according to any one of clauses 25 to 27, further comprising:

[0235] A heat rejection unit is connected to the cooling network and is configured to remove heat from the cooling fluid flowing through the cooling network.

[0236] 30. The cooling network of clause 29, wherein the heat rejection unit is a modular unit configured to be connected to or disconnected from the cooling network as the cooling fluid flows through the cooling network.

[0237] 31. The cooling network of clause 30, wherein the modular heat rejection units comprise evaporative cooling units or free cooling units.

[0238] 32. The cooling network of clause 30 or 31, wherein the modular heat rejection units comprise mechanical cooling units or absorption refrigeration cooling units.

[0239] 33. The cooling grid according to any one of clauses 25 to 32, further comprising:

[0240] an insulated tank configured to store a portion of the cooling fluid,

[0241] wherein during periods of high cooling load, the cooling fluid flowing through the insulated tank increases the temperature of the bulk cooling fluid in the insulated tank, and

[0242] Wherein during periods of lower cooling load, cooling fluid flowing through the insulated tank reduces the temperature of the bulk cooling fluid in the tank.

[0243] 34. The cooling grid according to any one of clauses 25 to 33, further comprising:

[0244] an array of pressure sensors connected to respective ones of the cooling fluid delivery elements or the nodes; and

[0245] A cooling network monitoring system, wherein the cooling network monitoring system is configured to:

[0246] comparing pressure readings received from corresponding pressure sensors in the pressure sensor array; and

[0247] Identify leaking cooling fluid delivery components or nodes.

[0248] 35. A method comprising:

[0249] A cooling network for a facility is provided, wherein the cooling network comprises:

[0250] cooling fluid delivery elements arranged in a grid pattern to provide cooling support to computing, networking, or data storage devices in the facility; and

[0251] nodes, the nodes being located at intersections of the grid pattern; and

[0252] A cooling fluid is caused to flow through the cooling network to provide cooling to the computing devices, the networking devices, or the data storage devices in the facility, wherein the cooling fluid is supplied to corresponding pieces of computing devices, networking devices, or data storage devices connected to the specific node via a specific node, and the cooling fluid is received at the specific node from two or more of the fluid transport elements connected to the specific node.

[0253] 36. The method of clause 35, further comprising:

[0254] One of the nodes is connected to a liquid cooling system to cool each piece of the computing equipment, the networking equipment, or the data storage equipment that generates concentrated waste heat.

[0255] 37. The method of clause 35 or 36, wherein flowing the cooling fluid through the cooling grid to provide cooling to the computing devices, the networking devices, or the data storage devices in the facility comprises:

[0256] flowing the cooling fluid through a liquid-to-air heat exchanger connected to one of the nodes: and

[0257] The cooling fluid is caused to flow through a cooling circuit of a liquid-cooled computing device, the cooling circuit being connected to the one or another node of the cooling network.

[0258] 38. The method of any one of clauses 35 to 37, further comprising:

[0259] monitoring a corresponding cooling fluid pressure in a corresponding one of the cooling fluid delivery elements or nodes; and

[0260] In response to detecting the pressure anomaly, one of the cooling fluid delivery elements is automatically isolated.

[0261] 39. The method of any one of clauses 35 to 38, wherein the respective absolute pressures of the cooling fluid flowing in the cooling fluid supply line and the cooling fluid return line of the cooling fluid transport element are less than one atmosphere, the method further comprising:

[0262] monitoring the flow rate of air removed from the cooling grid;

[0263] responsive to the flow rate of air removed from the cooling network exceeding a threshold amount, analyzing cooling fluid pressure in corresponding ones of the cooling fluid delivery elements or nodes to identify a leaking cooling fluid delivery element; and

[0264] The leaking cooling fluid delivery element is automatically isolated.

[0265] 40. The method of any one of clauses 35 to 39, further comprising:

[0266] detecting a failure of a cooling fluid delivery element; and

[0267] Generating a command to repair the failed cooling fluid delivery element is inhibited until a threshold number of failed cooling fluid delivery elements is reached.

[0268] Although the above embodiments have been described in considerable detail, numerous changes and modifications will become apparent to those skilled in the art once the above disclosure is fully appreciated. It is intended that the following claims be interpreted to encompass all such changes and modifications.

Claims

1. A power distribution network, comprising: Modular power delivery elements configured to be arranged in a grid pattern to supply power to computing devices, networking devices, or data storage devices in a facility; as well as nodes, the nodes being located at intersections of the grid pattern; The modular power delivery elements and the nodes are arranged so that a corresponding one of the nodes is configured to receive power from two or more of the modular power delivery elements connected to the corresponding node, and to supply the received power to one or more of the computing devices, the networking devices or the data storage devices in the facility. 2 . The power distribution grid of claim 1 , wherein the power distribution grid is configured to distribute direct current (DC) power.

3. The power distribution network of claim 1, further comprising: A power supply unit is located at the facility connected to a perimeter of the power distribution grid. 4 . The power distribution network of claim 3 , wherein each of the power supply units has a capacity of 750 kilowatts (KW) or less.

5. The power distribution network of claim 3, wherein each of the power supply units supplies less than 20% of the total power supplied to the power distribution network.

6. The power distribution network of claim 3, wherein the power supply unit comprises one or more of the following: Solar power supply unit; Wind energy based power supply unit; A geothermal based power supply unit; or Power supply unit based on hydroelectric power.

7. The power distribution network of claim 3, wherein the power supply unit comprises one or more of the following: a utility power component configured to convert alternating current power received from a utility power source into direct current (DC) power; or A fuel-based power supply component is configured to generate DC power using one or more types of fuel.

8. The power distribution network of claim 1, wherein: said computing device, said networking device, or said data storage device of said facility is included in a portable shipping container; as well as The power distribution grid is configured to distribute power to the computing device, the networking device, or the data storage device included in the portable shipping container.

9. The power distribution grid of claim 1, wherein the modular power delivery elements of the power distribution grid are arranged in a three-dimensional grid pattern, and wherein respective ones of the nodes are located at three-dimensional intersections of the modular delivery elements.

10. A method for distributing electric power, comprising: An electric power distribution grid for a facility is provided, wherein the electric power distribution grid comprises: modular power delivery elements arranged in a grid pattern to supply power to computing devices, networking devices, or data storage devices in the facility; and nodes, the nodes being located at intersections of the grid pattern; and Power is distributed to the computing devices, the networking devices, or the data storage devices in the facility via the nodes of the power distribution network, wherein power from two or more of the modular power delivery elements connected to the corresponding nodes is supplied to the corresponding nodes.

11. The method of claim 10, further comprising: A quantity of power supply units is provided to supply power to the power distribution grid, wherein the quantity of power supply units has a combined power capacity corresponding to current power consumption levels of the computing devices and the data storage devices in the facility.

12. The method of claim 11, further comprising: providing additional power supply units in response to an anticipated increase in current power consumption levels of said computing devices and said data storage devices in said facility; as well as The additional power supply unit is connected to the power distribution network while the quantity of power supply units continues to supply power to the power distribution network.

13. The method of claim 11, wherein the amount of power supply units includes utility power supply units or renewable energy based power supply units and fuel based power supply units; and Wherein distributing power to the computing device, the networking device, or the data storage device in the facility comprises: providing a baseline amount of power consumed by the computing device and the data storage device from the utility power unit or the renewable energy-based power unit; and A marginal amount of power is provided to the computing device and the data storage device during spikes in power consumption from the fuel-based power supply unit.

14. The method of claim 11, further comprising: detecting a failure of a modular delivery element on a first side of the power distribution network; as well as Switching from a first power supply unit on the first side of the power distribution network supplying power to the power distribution network to a second power supply unit on the second side of the power distribution network supplying power to the power distribution network.

15. The method of claim 10, further comprising: Detecting failure of modular conveyor elements or nodes; as well as Generating commands to repair a faulty modular conveying element or node is inhibited until a threshold number of faulty modular conveying elements or nodes has been reached.

Images