Data center thermal control system

Abstract

A data center thermal control system includes: a local cooler configured to cool a local coolant used for cooling electronic hardware; an external heat exchanger configured to exchange heat from a fluid to outside air; and a fluid circulation system configured to transfer heat from the local cooler to the external heat exchanger by circulating at least one fluid cooling medium, the fluid circulation system including a cold section pointing towards the air cooler. The thermal control system also includes one or more processors and a non-transitory computer-readable medium storing instructions that, when executed by the one or more processors, cause the one or more processors to control the external heat exchanger to cool the fluid in the cold section to a first target temperature during a hot season and to a lower target temperature during a cold season.

Description

Adaptable thermal control for data centers and IT equipment Electronic hardware, such as information technology ("IT") equipment, computer hardware, and servers, generates heat during operation. Such hardware operates more efficiently at lower temperatures and tends to fail more quickly at higher temperatures. For these reasons, a variety of electronic hardware cooling solutions have been developed. Data centers are used to house large amounts of electronic hardware, which is often used in remote applications such as cloud computing or internet hosting. In order to manage the large amounts of heat generated by the hardware they house, data centers typically have complex thermal management systems. These systems may include a chain of multiple heat exchanges between various cooling media to transport heat from the hardware out of the data center. Each link in the chain operates according to a set of parameters, such as cooling medium temperature and flow rate. Typically, these parameters are selected to balance a variety of considerations. For example, the cooling medium temperature and flow rate may be selected to balance the overall desirability of lower operating temperatures for the cooled electronic hardware with the costs associated with transporting heat from those components out of the building. The considerations being balanced can change over time, so the parameters are set to accommodate the most difficult conditions expected. By adapting the operating parameters of a data center's thermal control system as conditions change, efficiency and hardware life can be improved. In some specific instances, parameters can be adapted in anticipation of favorable conditions to take advantage of them. In some aspects of the present invention, the operating parameters of the data center thermal control system can vary with the seasons. Where certain operating parameters have been found to strike an acceptable balance between the desire for greater cooling of the hardware or media and the difficulty of removing heat from the building during the hottest times of the year, those parameters can be varied during times of the year when cooler weather is expected. In some instances, an outdoor heat exchanger (such as a cooling tower) can be used to transfer the cooling medium used by the heat-free data center to the outside air. Because cooler weather will tend to result in a lower temperature of the cooling medium immediately downstream of the tower without additional cost, any heat exchange in the data center that transfers heat to the cooling medium will tend to be more efficient when the weather is cooler. Thus, the thermal control operations of the entire data center (such as any refrigerant circuit, and in some instances, the refrigerant circuit used to cool process water) can be adjusted to take advantage of the temporary increase in cooling efficiency achieved by cooling to a lower temperature. In other aspects of the invention, operating parameters can be adapted to suit conditions within the data center. Areas storing and cooling hardware can be divided into multiple zones, and cooling medium circulating through zones containing less than the full capacity of its hardware can be circulated at a higher rate, lower temperature, or both, with little cost due to the lower cooling burden provided by the underfilled zones. Cooling systems (such as fan systems) for individual hardware components or containers for components can adapt to lower coolant inlet temperatures by reducing the thermal margin for the components or contained components. After the installation of several new components, the thermal margin of those new components can be temporarily increased to slow component failures during the expected early failure phase, thereby reducing the probability of equipment shortages; or temporarily reduced to shorten the expected early failure phase, thereby resulting in a stable operation phase during which few failures are expected to occur prematurely. In some aspects of any of the foregoing, a data center thermal control system may include: a local cooler configured to cool a local coolant used to cool electronic hardware; an external heat exchanger configured to exchange heat from a fluid to external air; and a fluid circulation system configured to transfer heat from the local cooler to the external heat exchanger by circulating at least one fluid cooling medium, the fluid circulation system including a cold section directed to the air cooler. The thermal control system may also include one or more processors and a non-transitory computer-readable medium storing instructions. The instructions, when executed by the one or more processors, may cause the one or more processors to control the external heat exchanger to cool the fluid in the cold section to a first target temperature during hot seasons and to cool the fluid in the cold section to a second target temperature lower than the first target temperature during cold seasons. In some examples according to any of the foregoing, the cold season can include all months in which the annual average temperature in the geographic area where the control system is located is below a threshold temperature. In some examples according to any of the foregoing, the fluid circulation system may include: an internal heat exchanger; an external loop that circulates fluid between the internal heat exchanger and the external heat exchanger; and an internal loop that includes a cold portion and circulates fluid between a local cooler and the internal heat exchanger. In some examples according to any of the foregoing, the local coolant can be circulating air. In some examples according to any of the foregoing, the second target temperature can vary during cold seasons. In some examples according to any of the foregoing, the cold season may include multiple intervals each having an annual mean temperature, and at each transition from an earlier interval to a later interval, if the annual mean temperature of the later interval is lower than the annual mean temperature of the earlier interval, the difference between the first target temperature and the second target temperature increases; and if the annual mean temperature of the later interval is higher than the average temperature of the earlier interval, the difference between the first target temperature and the second target temperature decreases. In some examples according to any of the foregoing, throughout the cold season, the second target temperature can be a non-piecewise function of the difference between the threshold temperature and the annual mean temperature of a current interval in the intervals. In some embodiments according to any of the preceding examples, a data center may have a control system according to any of the preceding examples installed therein. The data center may also include a plurality of cooling zones through which a local coolant circulates, each cooling zone having electronic hardware storage capacity. The instructions, when executed by one or more processors, may cause the processors to control the control system to increase a flow rate of the local coolant in a cooling zone known to contain less than a predetermined proportion of the hardware storage capacity of any cooling zone. In some examples according to any of the foregoing, the predetermined ratio can be equal for each cooling zone. In some instances according to any of the foregoing, for a cooling zone containing less than a predetermined proportion of the hardware storage capacity of each cooling zone, the instructions, when executed by one or more processors, may increase the airflow rate by an amount that decreases as the difference between the hardware storage capacity of the cooling zone and the amount of hardware known to be stored in the cooling zone decreases. In some examples according to any of the foregoing, the control system may include a plurality of cooling zones through which a local coolant circulates, each cooling zone having electronic hardware storage capacity. The instructions, when executed by one or more processors, may cause the processors to control the control system to reduce a temperature of the local coolant circulating in a cooling zone known to contain less than a predetermined proportion of the hardware storage capacity of any cooling zone. In another aspect, a container for electronic hardware may include a cooling system comprising one or more processors and a non-transitory computer-readable medium storing instructions. The instructions, when executed by the one or more processors, may cause the processors to control the cooling system to maintain a thermal margin of an electronic hardware component operating in the container above a minimum value, wherein the minimum value is a piecewise function of a supply temperature of a cooling medium, wherein the thermal margin is the difference between a predetermined temperature and an actual temperature of the component. In some examples according to any of the foregoing, the piecewise function may include a first sub-function applied to a first domain and a second sub-function applied to a second domain, the second domain being on an opposite side of the threshold temperature from the first domain, the first sub-function being a constant, and the second sub-function being a function of the supply temperature. In some examples according to any of the foregoing, the first domain can be above a threshold temperature. In some examples according to any of the foregoing, the second sub-function can establish a positive relationship between the absolute values ​​of the difference between the minimum and threshold temperatures and the supply temperature. In some examples according to any of the foregoing, the piecewise function may include a first sub-function applied to a first domain and a second sub-function applied to a second domain, wherein the second domain is on an opposite side of the threshold temperature from the first domain, the first sub-function and the second sub-function being different functions of the supply temperature. In some examples according to any of the foregoing, the first sub-function and the second sub-function can both establish a positive relationship between the absolute values ​​of the differences between the minimum and threshold temperatures and the supply temperature. In some examples according to any of the preceding items, the container can include a fan. The cooling medium can be air, and the control system can be configured to maintain the margin by varying the operating speed of the fan as needed to keep the margin above a minimum value. In another aspect, a data center thermal control system for cooling a fleet of electronic components may include one or more processors and a non-transitory computer-readable medium storing instructions that, when executed by the one or more processors, cause the processors to control the control system to cool the fleet to a different lower thermal margin limit during at least a portion of an early failure phase as compared to during a stable phase. The early failure phase may be a window after installation of the component during which the expected failure rate of the component decreases by at least a first rate based on historical failure data. The stable phase may be a window after the early failure phase during which the failure rate of the component decreases by a rate less than the first rate and increases by a rate less than the second rate based on historical failure data. The thermal margin may be the difference between a predetermined temperature of the component and an actual operating temperature. In some examples according to any of the foregoing, the instructions, when executed by one or more processors, may cause the processors to control the control system to cool the team to a stable lower thermal margin limit during the stable phase, and to cool the team to an early lower margin limit during the early failure phase, wherein the early lower margin limit is less than the stable lower margin limit. In some examples according to any of the foregoing, the instructions, when executed by one or more processors, may cause the processors to control the control system to cool the fleet to an early margin lower limit from installation of the fleet until an adapted stable transition time, and to cool the fleet to at least the stable margin lower limit starting at the adapted stable transition time, wherein the adapted stable transition time is the earliest time after installation of the fleet when the actual failure rate of components in the fleet is expected to decrease at a rate less than a predetermined rate. In some examples according to any of the foregoing, the historical failure data can be derived from observed failures of electronic devices of the same type as the component before installation. FIG1 shows a thermal control system 100 for a data center. Thermal control system 100 is controlled by controller 110, which is in electronic communication 111 with the other components of thermal control system 100. Controller 110 includes memory 115 in the form of non-transitory computer-readable media that stores data that can be accessed electronically. The media can be, for example, a hard drive, a memory card, read-only memory (ROM), random access memory (RAM), an optical disk, or any other type of writable and read-only memory. Memory 115 stores instructions 117 that, when read by one or more processors 119 in controller 110, cause one or more processors 119 to perform, individually or in any combination, the adaptable thermal control operations described herein by controlling the other hardware in thermal control system 100. Instructions 117 can be instructions that, individually or in any combination, perform any of the thermal control operations described herein. Although controller 110 is depicted as a single unit, controller 110 may instead be multiple devices distributed across thermal control system 100 that have different functions and control different hardware, and in various examples, those distributed devices may or may not be in electronic communication with each other. Thus, in other examples, any element of thermal control system 100 shown in the illustrated example as being in electronic communication 111 with controller 110 may instead be in electronic communication with a discrete controller that does not control other elements of the system. Thermal control system 100 includes multiple cooling circuits. The illustrated example will describe water as the medium circulating in some of these circuits (such as a "plant water" circuit and a "process water" circuit). Thus, water is presented as an example of a cooling medium with which the concepts of the present invention can be implemented, but the same concepts can be applied in the same manner to any other fluid cooling medium. Therefore, any reference to water in this specification should be considered interchangeable with other fluid cooling media. The plant water loop 101 is used to transfer heat collected from throughout the data center out of the building. The plant water loop 101 includes a hot side 112A, which carries water heated by other parts of the thermal control system 100 toward a cooling tower 114, and a cold side 112B, which carries water from the cooling tower 114 back to the other parts of the thermal control system 100. The cooling tower 114 is an external heat exchanger that transfers heat from the hot plant water to the environment outside the data center. In the specific embodiment of the cooling tower 114, heat leaves the plant water through the passage of ambient air 116 across the cooling tower 114. The cooling tower 114 is constructed to conduct heat from the plant water within the cooling tower 114 to the exterior surfaces of the cooling tower 114, so that the flow of ambient air 116 across those exterior surfaces cools the plant water. In various examples, the cooling tower 114 can operate entirely passively, or a fan or other impeller can be used to force ambient air 116 across the cooling tower 114. The cooling tower 114 is one example of an external heat exchanger, and in other examples, other external heat exchangers can be used in place of or in addition to the cooling tower 114 to transfer heat from the plant water loop 101 to the environment outside the data center. The process water loop 102 similarly includes a hot side 122A, which carries water heated by other components in the data center, and a cold side 122B, which carries water that can be used as a coolant. Between the hot side 122A and the cold side 122B of the process water loop 102, heat is transferred from the process water to the plant water. Therefore, heat collected by the process water loop 102 can be transferred to the plant water loop 101 for transport out of the building. Therefore, the plant water loop 101 is an external loop, as it is closer to the point where the heat leaves the building along the heat's path through the data center, while the process water loop 102 is an internal loop, as it is farther from the point where the heat leaves the building along the heat's path through the data center. While only one process water loop 102 and plant water loop 101 are shown in the illustrated example, other examples of the thermal control system 100 may include multiple process water loops 102 that transfer heat to a single plant water loop 101, or multiple plant water loops 101 that collect heat from a single process water loop 102. Heat is transferred from process water loop 102 to plant water loop 101 via internal heat exchanger 118 and chiller 120, both located downstream of hot side 122A and upstream of cold side 122B of process water loop 102, and downstream of cold side 112A and upstream of hot side 112B of plant water loop 101. Heat exchanger 118 can be any type of heat exchanger, such as a shell and tube heat exchanger, a plate heat exchanger, or any other type of structure that allows process water and plant water to flow across opposite sides of a heat transfer barrier. Chiller 120 is located downstream of heat exchanger 118 and can be any device that uses a refrigerant circuit to transfer heat from process water to plant water. Heat exchanger 118 is optional, and therefore, some embodiments other than the illustrated example omit it. The process water loop 102 collects heat from one or more zone coolers 130. Although only one zone cooler 130 is shown, multiple zone coolers 130 may be connected to the process water loop 102 between its hot side 122A and its cold side 122B. Each zone cooler 130 includes a cooling element 134, which can be, for example, a heat exchanger, a refrigerant-based chiller, or both, for transferring heat from the local medium loop 103 to the process water loop 102. The zone cooler 130 of the illustrated example also includes a driver 136 for pressurizing the local cooling medium along the local medium loop 103. Thus, the illustrated zone cooler 130 can be, for example, a cooling fan, in which case the cooling element 134 can be a cooling fan coil and the driver 136 can be an impeller for driving air as the local cooling medium, or a cooling distribution unit ("CDU"), in which case the cooling element 134 can be any device suitable for cooling a fluid as the local cooling medium and the driver 136 can be a pump. In other configurations, the driver 136 can be omitted if the local medium loop 103 is of a type that does not require pressurization, such as in an embodiment where the local medium loop 103 is an evaporative cooling loop and the cooling element 134 is a condenser. In a data center that accommodates multiple district coolers 130 and local media loops 103 , the district coolers 130 and local media loops 103 may be of different types. The local media loop 103 includes a cold side 132B that carries a local cooling medium (such as air, water, dielectric fluid, or any other coolant appropriate for the type of electronic hardware being cooled) at a relatively low temperature to one or more enclosures 140, and a hot side 132A that returns the local cooling medium to the zone cooler 130 after it has been used to cool the electronic hardware. Although only one enclosure 140 is shown, each zone may contain several enclosures. Each housing 140 contains one or more units 141. Each unit 141 can be an individual electronic hardware component, or it can be a container or housing for electronic hardware. In the illustrated example, the units 141 are containers in the form of trays for electronic hardware in the form of servers, but the concepts of the present invention are applicable to any type of heat-generating electronic hardware, and the housing 140 is a server rack. As shown in the illustrated example, each unit 141 may optionally include an onboard cooling system. The illustrated example of an onboard cooling system includes a driver 148 (such as a fan or pump) for driving the local cooling medium across the heat load 150; an inlet thermometer 146B for measuring the temperature of the inlet flow 142B of the local cooling medium; and an outlet thermometer 146A for measuring the temperature of the outlet flow 142A of the local cooling medium. The onboard cooling system may also include or communicate with one or more thermometers that measure the operating temperature of the cooled electronic hardware stored in the unit 141. Load 150 is an object cooled by the local cooling medium flowing through or across unit 141. Load 150 can be an individual electronic hardware component, or, as shown in the illustrated example, can be a thermal electronic component 154, a heat sink 158, and a thermal interface 156 between the thermal electronic component 154 and the heat sink 158. Thermal electronic component 154 can be any piece of electronic hardware that generates heat and would benefit from cooling, such as a processor die. Heat sink 158 can be any structure, such as fins, pins, or a cold plate, that facilitates the transfer of heat 155 to the local cooling medium passing through heat sink 158 through load portion 152. Although only one load 150 is shown in unit 141 of the illustrated example, each unit 141 can contain multiple loads 150. The loaded portion 152 of the local cooling medium leaves unit 141 as outlet stream 142A. Outlet stream 142A is connected to the hot side 132A of the local medium loop 103 and returns to the district cooler 130. The district cooler uses cooling elements 134 to transfer heat from the heated local cooling medium returned from the hot side 132A of the local medium loop 103 to the process water. The process water heated by the cooling elements 134 of the district cooler 130 travels along the hot side 122A of the process water loop 102 to the heat exchanger 118 and chiller 120. Both the heat exchanger 118 and chiller 120 transfer the hot process water received from the hot side 122A of the process water loop 102 to the plant water. Plant water heated by heat exchanger 118 and chiller 120 travels along the hot side 112A of plant water loop 101 to an external heat exchanger, such as cooling tower 114, which transfers heat from the plant water out of the building and into the environment surrounding the data center. Thus, thermal control system 100 utilizes a chain of cooling operations to collect heat from several individual loads 150 and transport it out of the building. This chain ultimately relies on an external heat exchanger (i.e., cooling tower 114 in the example shown) to transport heat out of the building and to establish capacity for transporting heat away from loads 150. Because cooling tower 114 or any other external heat exchanger used to transfer plant water heat out of the building relies on the environment outside the building as the cooling medium, the temperature difference between hot side 112A and cold side 112B of plant water loop 101 will vary with the weather. Therefore, if the heat generated by the data center remains approximately constant, the temperature of cold side 112B of plant water loop 101 will decrease as the weather gets colder. Conversely, if the heat generated by the data center remains approximately constant as cold side 112B of plant water loop 101 cools, heat can be transferred more efficiently from process water loop 102 to plant water loop 101. That is, as cold side 112B of plant water loop 101 cools, more heat is transferred from process water to plant water at heat exchanger 118, with virtually no energy cost. Therefore, as cold side 112B of plant water loop 101 cools, the process water immediately downstream of heat exchanger 118 and upstream of chiller 120 cools. Thus, the temperatures of both the process water and the plant water flowing into the chiller 120 will affect the amount of power required to operate the chiller 120 to reduce the cold side 122B of the process water loop 102 to a given target temperature. As the difference between the temperature of the process water immediately upstream of the chiller 120 and the target temperature decreases, the chiller 120 will require less power to reduce the process water to the target temperature, and as the temperature of the plant water immediately upstream of the chiller 120 becomes cooler, the chiller 120 will operate more efficiently. Due to the aforementioned interaction, when the weather outside the data center is colder, the cold side 122B of the process water loop 102 can be set to a lower temperature with little or no additional energy cost. Lowering the temperature of the cold side 122B of the process water loop 102 can be used to enable the zone cooler 130 to more efficiently lower the cold side 132B of its respective local dielectric loop 103 to its normal temperature, or to lower the cold side 132B of its respective local dielectric loop 103 to below normal temperature, to improve the operating efficiency and life of the cooled hardware represented by the load 150. As shown in Figures 2A and 2B, instructions 117 can be provided to the controller 110 or manually operated to take advantage of cooler weather by adapting the target temperature 210 over time, wherein the controller 110 is configured to operate the chiller 120 as needed to maintain the cold side 122B of the process water loop 102 at or below the target temperature 210. In Figures 2A and 2B, a year is divided into intervals 202, and those intervals 202 are grouped into hot seasons 204 or cold seasons 208, which together constitute a full year. The hot season 204 includes intervals 202 during which the average outside temperature 207 exceeds a threshold temperature 206, while the cold season 208 includes intervals 202 during which the average outside temperature 207 is below the threshold temperature 206. The threshold temperature 206 represents the temperature at which the target temperature 210 can be lowered below the upper limit temperature 216 without unacceptable energy costs. During hot season 204, target temperature 210 is held constant at upper limit temperature 216, which is a temperature that achieves an acceptable compromise between the data center's cooling needs and the cost of cooling process water during the hottest weather expected at the data center's location. Average external temperature 207 can be derived from historical weather data for intervals 202 from previous years. While hot season 208 and cold season 204 are comprised of continuous intervals 202 in the illustrated example, seasons 204 and 208 may include non-contiguous intervals, depending on the region. Intervals 202 can have any length, such as months, days, or the time between samples at the sampling rate of a real-time weather monitoring system. In various embodiments, temperature data 207 and any other temperature measurements of conditions outside the data center can be pure temperature measurements, i.e., dry bulb temperature ("DBT"), DBT measurements, and any or any combination of humidity, wind chill, and cloud cover measurements, wet bulb temperature ("WBT"), or wet bulb globe temperature ("WBGT"). In both FIG2A and FIG2B , target temperature 210 varies during cold season 208 in response to changes in the weather, such as by making target temperature 210 a non-piecewise function of a selected form of temperature measurement of the weather. In the example of FIG2A , target temperature 210 varies over a subinterval 212 of cold season 208 based on historical weather data from a data center location. While subinterval 212 is half the length of interval 202 in the example shown, subinterval 212 in other examples can have any length, including a length greater than the length of interval 202. In the example of FIG2B , target temperature 210 varies continuously throughout cold season 208 in response to real-time temperature measurements or weather forecasts. In any case, when the target temperature 210 is caused to vary in response to current, forecasted, or historical weather, rather than being held constant at the upper limit temperature 216, the target temperature 210 may, in various instances, be set to maintain a constant difference between the outside temperature and the target temperature 210, or to be the lowest temperature of the cold side 122B of the process water loop 102 that can be expected to be maintained without exceeding acceptable energy costs. Instead of predetermining discrete hot seasons 204 and cold seasons 208 as shown in FIG2A and FIG2B , a determination can be made continuously in response to a weather forecast or real-time temperature measurements as to whether the target temperature 210 will remain at the upper limit temperature 216 or vary with the weather. When a determination is made in response to a weather forecast or real-time temperature measurements as to whether the target temperature 210 will remain at the upper limit temperature 216 or vary with the weather, the determination can be made continuously or over any recurring time interval. Thus, without predetermining seasons 204 and 208, the target temperature 210 can be allowed to climb to or fall below the upper limit temperature 216 day by day, or even at different times of the day, in response to changes in the weather that cause the outside temperature to rise above or fall below the threshold temperature 206. Because more heat leaves the building through the cooling tower 114 during colder weather, the cold side 122B of the process water loop 102 can occasionally be made cooler at little or no additional energy cost by lowering the target temperature 210 below the upper limit temperature 216 when colder weather is predicted or detected, as described in any of the examples above. Cooler process water, in turn, makes zone cooling more efficient, so seasonal or instantaneous weather-based adaptation of the target temperature 210 can be used to improve the overall energy efficiency of the thermal control system 100, cool electronic hardware in the data center to lower temperatures during periods of cold weather, or both. Cooling electronic hardware to lower temperatures, even occasionally, will lower the average operating temperature of the hardware throughout the year, which will tend to increase the lifespan and operating efficiency of the hardware. The weather data 207, measurements, and weather forecasts that rely on the adaptation described above with respect to Figures 2A and 2B can have any degree of geographic specificity. For example, the weather data 207, measurements, and weather forecasts can be derived individually or in any combination from measurements taken on-site at a data center, local weather records, or regional weather reports. Alternatively or additionally, instructions 117 may be provided to controller 110 or manually controlled to adjust zone cooling to the fill state of the zones shown in FIG. 3 . A zone can be, for example, any space used to store electronic hardware or groups of hardware containers, such as rows of enclosures 140 in a server hall, where the stored hardware is cooled by a localized cooling medium supplied by a single zone cooler 130 or discrete groups of zone coolers 130. For example, a server hall may be filled with rows of enclosures 140, each in the form of a cabinet or server rack, configured to draw in cooling medium in the form of cool air on one side and exhaust hot air from an opposite side. The enclosures 140 in each row may face the same direction, and the rows may face alternating directions. Thus, with the exception of the rows at the ends of the server hall, each row will have its intake side facing the intake side of an adjacent row across a relatively cool air aisle, and each row will have its exhaust side facing the exhaust side of an adjacent row across a relatively hot air aisle. Aisles with relatively cool air may be referred to as cold aisles, and aisles with relatively hot air may be referred to as hot aisles. In a server hall configured in this manner, the air in each hot aisle will be considered at least part of the hot side 132A of the local dielectric loop 103, and will therefore be drawn from the hall and directed to a zone cooler 130 in the form of one or more fans with cooling coils. The zone cooler 130 in the form of a fan with cooling coils will then create the cold side 132B of the local dielectric loop 103 by cooling the air and blowing the cooled air into one or more cold aisles. A server hall configured in this manner may have multiple zone coolers 130 distributed around the hall, so that zones within the hall can be identified by dividing the hall into groups of rows or groups of portions of rows, which groups primarily receive cool air from a shared zone cooler 130. Cooling in zones that are not fully populated with electronic hardware can be adapted to take advantage of the lower heat load generated by the zone by reducing the temperature of the local cooling medium supplied to the zone, increasing the rate at which the local cooling medium circulates through the zone, or both. Due to the lower heat load, any of these measures can be achieved at relatively little additional energy cost. For example, as shown in FIG3 , when zone 304 of a cold aisle as described above is filled with less than a threshold amount 306 (which can be any amount less than 100% of the maximum amount of electronic hardware the zone is designed to accommodate), the zone cooler 130 serving that zone 304 can be caused to increase its fan speed 310 from its normal set point and reduce the local coolant supply temperature 320 by an amount directly related to the difference between the threshold amount 306 and the actual zone fill level. Supplying the local cooling medium at a higher flow rate and lower temperature allows the electronic hardware to be cooled more efficiently to lower temperatures. The same principles can be applied to the storage and cooling of other types of electronic hardware. For example, in the case where the local cooling medium is a fluid, the rate at which the local medium circuit 103 pumps the fluid through the region may be increased when the region fills to less than a threshold amount 306. Such adaptations may be applied while an initially empty region is gradually being hard-filled, meaning that the flow rate and temperature of the local cooling medium in the region will approach its final set point as the region becomes closer to being filled, i.e., the region 308 closer to the aisle is filled to or beyond the threshold amount 306. FIG4A through FIG4C are graphs of thermal margin TM versus inlet temperature TINLE for each of the piecewise functions shown in FIG5A through FIG5C. FIG4A shows the results for the function of FIG5A, FIG4B shows the results for the function of FIG5B, and FIG4C shows the results for the function of FIG5C. Each of the functions of FIG5A through FIG5C can be provided as instructions for a controller (such as instructions 117 for controller 110) to control the thermal margin TM that a container (such as unit 141) will maintain for electronic hardware (such as component 154) stored therein. In each of the functions in Figures 5A through 5C, thermal margin TM is defined as the difference between the hardware's predetermined maximum tolerable operating temperature and the hardware's actual operating temperature. Thus, as the thermal margin TM of a given hardware component increases, the component's actual operating temperature decreases. The maximum tolerable operating temperature can be set, for example, by the hardware manufacturer, an independent testing organization, or the owner of the specific unit of hardware involved. The inlet temperature, T inlet, is the temperature of the inlet flow of the local cooling medium to the container (such as inlet flow 142B) and can be measured using a thermometer at the container inlet (such as inlet temperature gauge 146B). Each piecewise function is divided into a cold region 405 below a threshold temperature 406 and a hot region 407 above the threshold temperature 406. The threshold temperature 406 may be any temperature determined as the point at which the thermal margin TM is changed. In the piecewise function of FIG5A , the thermal margin TM remains constant at the lower margin in the hot domain 407. The lower margin is a margin that achieves an acceptable balance between favorable operating conditions for the cooled hardware and the energy required to cool the hardware at the maximum inlet temperature T INLET expected in the relevant domain (which may be the thermal domain 407 of the function of FIG5A ). Therefore, the function of FIG5A applies a sub-function with a constant value, the lower margin, in the hot domain 407. In the cold domain 405, the function of FIG5A applies a sub-function that establishes a direct correlation between the thermal margin TM and the absolute value of the difference between the threshold temperature 406 and the inlet temperature T INLET. In the example of FIG5A , the sub-function applied to the cold domain 405 is a geometric function in which the absolute value of the difference between the threshold temperature 406 and the inlet temperature T INLET is multiplied by a constant C, and the resulting product is added to the lower margin. However, piecewise functions according to other examples may keep the thermal margin TM constant at the lower margin of the hot domain 407 and apply any type of function, including non-geometric functions, to establish a direct correlation between the thermal margin TM and the absolute value of the difference between the threshold temperature 406 of the cold domain 405 and the inlet temperature TINlet. When the inlet temperature T INLET exceeds the threshold temperature 406 , the thermal margin TM is held constant at the lower margin, and then the thermal margin TM is increased as the inlet temperature T INLET further falls below the threshold temperature 406 . This allows the cooled hardware to have a lower operating temperature during times when the local cooling medium supply is cold (such as during winter or when the local weather is cold), while still maintaining energy efficiency when the local cooling medium supply is relatively warm. Thus, a step function with this characteristic (such as the step function of FIG. 5A ) can reduce the lifetime average operating temperature of the adaptively cooled hardware at a low energy cost compared to hardware that always maintains the lower margin. FIG5B and FIG5C show additional piecewise functions that establish a direct correlation between thermal margin TM and the absolute value of the difference between the threshold temperature 406 of the hot domain 407 and the inlet temperature TInlet. The piecewise function of FIG5B maintains thermal margin TM at the lower margin in the cold domain 405, while the piecewise function of FIG5C also establishes a direct correlation between thermal margin TM and the absolute value of the difference between the threshold temperature 406 of the cold domain 407 and the inlet temperature TInlet. Therefore, the function of FIG5C can also be expressed as a non-piecewise function of TM = Threshold - TInlet | * C + Lower Limit. Increasing thermal margin in the hot domain 407 can offset the detrimental effects of higher ambient temperatures on the cooled electronic hardware, which tend to occur concurrently with higher inlet temperatures TInlet. The piecewise functions of FIG. 5B and FIG. 5C utilize geometric subfunctions to establish their direct correlation, but in other examples, non-geometric subfunctions may be utilized to establish a direct correlation between the thermal margin TM and the absolute value of the difference between the threshold temperatures 406 on one or both sides of the threshold temperature 406 . Maintaining at least a minimum between the hot and cold sides of a local medium circuit can improve the efficiency of cooling and circulating the local cooling medium. Therefore, in some examples, a controller responsible for the container cooling system can be instructed to execute a function that seeks the highest possible thermal margin TM while maintaining the temperature difference DT at, or at least at, a target difference. According to various examples, the function that maintains the temperature difference DT at, or at least at, the target difference can be used in place of any of the piecewise functions described above, or in addition to the piecewise functions described above, wherein the piecewise function is modified once the temperature difference DT falls below the target difference. Thermal Margin TM is directly related to the product of the temperature difference DT and the rate of local coolant flow through the container. As the temperature difference between the hardware and the medium increases, the rate of heat transfer from the cooled hardware to the local coolant increases. Therefore, as the inlet temperature TInlet decreases, the ratio of the temperature difference DT to the rate of coolant flow through the container increases. Therefore, as the inlet temperature TInlet decreases, the ratio of Thermal Margin TM to the rate of local coolant flow through the container also increases. Therefore, when the inlet temperature TInlet is relatively low, the flow rate of the local coolant through the container can be driven higher without causing the temperature difference DT to fall below the target difference. Therefore, whenever the actual inlet temperature TInlet falls below the inlet threshold temperature TInlet, the fan or other hardware responsible for driving the local coolant through the container can be controlled to increase the flow rate of the local coolant proportionally to the difference between the inlet threshold temperature TInlet and the actual inlet temperature TInlet, thereby maintaining a constant temperature difference DT while establishing a larger Thermal Margin TM. FIG6 shows a container cooling system 400 that can be controlled according to any of the functions described above with respect to FIG4A through FIG5D to establish and maintain a thermal margin TM. Container cooling system 400 is configured to monitor the thermal margin of a plurality of electronic hardware components stored in a container, such as unit 141. Cooling system 400 includes a plurality of margin proportional-integral-derivative controllers (PIDs) 460 that each receive a respective thermal margin TM from a digital controller 410 to target a respective one of the electronic hardware components stored in the container. If container cooling system 400 is integrated into a thermal control system similar to thermal control system 100, digital controller 410 may be controller 110, which is in electronic communication 411 with margin PIDs 460. Margin PIDs 460 receive input from a thermometer 446 that measures at least the operating temperature of the electronic hardware in the container and the inlet temperature, T, of the local cooling medium flowing into the container. Margin PID 460 receives temperatures from thermometers that measure the operating temperatures of various individual electronic hardware components within the container. Margin PID 460 may optionally receive the inlet temperature, TInlet, from a common inlet thermometer (such as inlet thermometer 146) or from different individual inlet thermometers. Margin PID 460 may also optionally receive the outlet temperature of an outlet stream (such as outlet stream 142A) of the container's local cooling medium from one or more outlet thermometers (such as outlet thermometer 146A). Margin PID 460 uses the temperature measurements received from thermometer 446 to determine whether the flow rate of the local cooling medium through the container should be increased or decreased to achieve the respective thermal margin TM sought for the respective electronic hardware components in the container. Output 462 from the determination of Margin PID 460 is sent to decision controller 464. Decision controller 464 considers output 462 and sends a single speed command to driver PID 461. Driver PID 461 controls driver 448 based on the speed command received from decision controller 464. Driver 448 is any mechanism for driving the local cooling medium through the container, such as a fan, pump, or any other type of mechanism capable of driving the cooling medium through a space. Driver 448 can be the same as driver 448. According to various examples, the decision controller 464 can be configured to select either the lowest received output 462 or the highest received output 462 to send as a single speed command to the driver PID 461. In other examples, the decision controller 464 can be configured to find a compromise among the received outputs 462 (e.g., an arithmetic mean of the fan speeds that will satisfy each of the received outputs 462) and send the compromise speed as a command to the driver PID 461. The decision controller 462 can be any device capable of performing any of the aforementioned logical functions for which the decision controller 462 is configured in a given implementation, such as an integrated circuit or a programmable logic controller (PLC). While two margin PIDs 460 are shown in the illustrated example, the container cooling system 400 may be configured with any number of margin PIDs 460 that receive measurements from the temperature meter 446 and commands from the digital controller 410 and send outputs 462 to the decision controller 464. In a given embodiment of the container cooling system 400, the number of margin PIDs 460 may be equal to the number of electronic hardware components in the container that are expected to be independently monitored when determining the speed of the drive 448. Thus, the container cooling system 400 may include any plurality of margin PIDs 460. In other examples, the container cooling system 400 may include only a single margin PID 460. In such examples, the single margin PID 460 may send output 462 directly to the drive PID 461 without routing output 462 through the decision controller 464, or the margin PID 460 and the drive PID 461 may be combined into a single PID. Figures 7A-7C illustrate adaptive thermal margin profiles 517, 527, and 537 for the installation of multiple electronic hardware components. In each of Figures 7A-7C, profiles 517, 527, and 537 are depicted based on a typical infant mortality phase 501, a stable phase 502, and a wear-out phase 503. These phases can be derived from historical data for hardware of the same type as the adaptively cooled hardware, or predicted for hardware for which no such historical data is available. Infant mortality phase 501, stable phase 502, and wear-out phase 503 represent portions of the "bathtub curve" often observed in the failure rates of a large number of manufactured items. Infant mortality phase 501 is the phase after hardware installation, during which failures due to manufacturing defects are expected. The stable phase 502 follows infant mortality phase 501. During the stabilization phase 502, few failures are expected because most defective parts have already failed, but inevitably, usage-related wear-out failures have not yet begun. The wear-out phase 503 follows the stabilization phase 502. Throughout the wear-out phase 503, the failure rate climbs until all parts fail due to usage-related degradation. The failure rate of the hardware will typically decrease during the early failure phase 501, remain relatively stable during the stabilization phase 502, and then increase throughout the wear-out phase 503. Therefore, the transition from the early failure phase 501 to the stabilization phase 502 can be defined as the earliest time after installation that the failure rate of the hardware decreases at a rate less than a predetermined rate. Similarly, the transition from the stabilization phase 502 to the wear-out phase 503 can be defined as the earliest time after installation that the failure rate of the hardware increases at a rate exceeding a predetermined rate. Any values ​​can be used for the predetermined decrease rate and the predetermined increase rate to mark these transitions. Specific examples of predetermined rates of increase or decrease that may mark transitions between phases include 1% per day, 2% per day, 3% per day, 4% per day, 5% per day, 10% per day, 15% per day, 20% per day, and 25% per day. Figures 7A through 7C each present a separate example of how the thermal margin floor for a fleet of components in a single installation can be adapted over time to change the failure rate curve for that fleet. In Figure 7A, the thermal margin floor 517 for the component fleet is lower during a portion of the infantry failure phase 501 than at any point during the stable phase 502. This accelerated infantry failure rate is visible in the relative height of the leftmost portion of the adapted failure rate curve 515. Reducing the thermal margin floor 517 for the hardware fleet causes the expected infantry failure to arrive sooner, resulting in an adapted infantry failure phase 511 that is shorter than infantry failure phase 501, which would be observed if the thermal margin floor 517 remained constant over the lifetime of the hardware in the fleet. Thus, the adapted stable phase 512 following the adapted early failure phase 501 is reached earlier than the typical stable phase 502, and the adapted wear-out phase 513 following the adapted stable phase 512 is reached earlier than the typical wear-out phase 503. Therefore, in applications requiring reliable operation of an entire hardware fleet, reducing the lower thermal margin limit 517 early in the lifetime of the fleet can be useful because defective parts can be identified and replaced early in the lifetime of the fleet. In various examples, the amount of time after installation of the fleet that the lower thermal margin limit 517 may be maintained low may be a predetermined amount of time, or may be an amount of time corresponding to a transition from the typical early failure stage 501 to the typical stable stage 502, or an actual or predicted transition from the adapted early failure stage 511 to the adapted early failure stage 512. As noted above, such transitions may be marked by the earliest time after installation of the fleet that the typical, actual, or predicted failure rate decreases by less than a predetermined amount. As shown in FIG7B , during the adapted infantry failure phase 521, the thermal margin lower limit 527 can be raised to mitigate the failure rate 525 during the adapted infantry failure phase 521, thereby causing the adapted infantry failure phase 521 to last longer than the typical infantry failure phase 501. As a result, the adapted stable phase 522 is reached later than the typical stable phase 502, and the adapted wear-out phase 523 is reached later than the typical wear-out phase 523. Therefore, raising the thermal margin lower limit 527 early in the life of the fleet can help reduce the expected peak demand for hardware replacement during the typical infantry failure phase 501. In various examples, the amount of time that the lower thermal margin limit 527 may be maintained high after installation of the fleet may be a predetermined amount of time, or may be an amount of time corresponding to a transition from the typical early failure stage 501 to the typical stable stage 502, or an actual or predicted transition from the adapted early failure stage 521 to the adapted early failure stage 522. As noted above, such transitions may be marked by the earliest time after installation of the fleet that the typical, actual, or predicted failure rate decreases by less than a predetermined amount. As shown in FIG7C , by increasing the thermal margin lower limit 537 before the typical wear-out stage 503 is expected to be reached, the adapted stable stage 532 can be extended beyond the typical stable stage 502. By maintaining the elevated thermal margin lower limit 537 during the adapted wear-out stage 513, the failure rate 535 during the adapted wear-out stage 533 can be kept relatively low. Thus, raising the thermal margin lower limit 537 before the typical wear-out stage 503 is expected to be reached can postpone the need to replace hardware and slow the rate at which hardware must be replaced after it begins to wear out. The adaptation of FIG7C can be applied in conjunction with the adaptation of FIG7A or FIG7B . According to various examples, after applying a relatively high or relatively low thermal margin lower limit during at least a portion of the typical early failure stage 501, the thermal margin lower limit can be raised before the expected arrival of the typical wear-out stage 503, the early adapted wear-out stage 513, or the late adapted wear-out stage 523. Although the concepts herein are described with reference to specific examples, it should be understood that these examples are merely illustrative of the principles and applications of the present concepts. It should be understood that many modifications may be made to the illustrative examples, and that other configurations may be devised, without departing from the spirit and scope of the present concepts as defined by the appended claims. 100: Thermal Control System 101: Plant Water Loop 102: Process Water Loop 103: Local Media Loop 110: Controller 111: Electronic Communications 112A: Hot Side 112B: Cold Side 114: Cooling Tower 115: Memory 116: Ambient Air 117: Instructions 118: Internal Heat Exchanger 119: Processor 120: Chiller 122A: Hot Side 122B: Cold Side 130: District Cooler 132A: Hot Side 132B: Cold Side 134: Cooling Element 136: Actuator 140: Housing 141: Unit 142A: Outlet Flow 142B: Inlet Flow 146A: Outlet Thermometer 146B: Inlet Thermometer 48: Driver 150: Thermal Load 152: Loaded Section 154: Thermal Electronic Component 156: Thermal Interface 158: Heat Sink 202: Interval 204: Hot Season 206: Threshold Temperature 207: Average External Temperature 208: Cold Season 210: Target Temperature 212: Sub-Interval 216: Upper Limit Temperature 306: Threshold Amount 310: Fan Speed ​​320: Local Coolant Supply Temperature 400: Container Cooling System 405: Cold Zone 406: Threshold Temperature 407: Hot Zone 410: Digital Controller 411: Electronic Communication 446: Thermometer 448: Driver 460: Margin Proportional Integral Derivative Controller (PID) 461: Driver PID 462: Output 464: Decision Controller 501: Early Failure Stage 502: Stable Stage 503: Wear Stage 511: Adapted Early Failure Stage 512: Adapted Stable Stage 513: Adapted Wear Stage 515: Adapted Failure Rate Curve 517: Adjustable Thermal Margin Distribution 521: Adapted Early Failure Stage 522: Adapted Stable Stage 523: Adapted Wear Stage 525: Failure Rate 527: Adjustable Thermal Margin Distribution 532: Adapted Stable Stage 533: Adapted Wear Stage 535: Failure Rate 537: Adjustable Thermal Margin Distribution FIG1 is a schematic diagram of a thermal control system for a data center. 2A and 2B are graphs illustrating seasonal adaptation of a target coolant temperature according to various aspects of the present invention. FIG. 3 is a graph showing possible adaptations of the coolant temperature and flow rate depending on the filling ratio of the area for storing electronic hardware. 4A-4C are graphs illustrating the adaptation of thermal margin for electronic hardware depending on coolant temperature according to various aspects of the present invention. 5A to 5C are functions represented by the graphs of FIG. 4A to 4C . FIG6 is a schematic diagram of a container cooling system. 7A to 7C are graphs showing thermal margin adaptation depending on the service life of electronic hardware. 100: Thermal control system 101: Factory Water Circuit 102: Process water circuit 103: Local medium circuit 110: Controller 111: Electronic Communications 112A: hot side 112B: cold side 114: Cooling Tower 115: Memory 116: Ambient air 117: Instructions 118: Internal heat exchanger 119: Processor 120: Freezer 122A: hot side 122B: Cold side 130: District Cooler 132A: hot side 132B: cold side 134: Cooling element 136:Drive 140: Shell 141: Unit 142A: Export flow 142B: Import flow 146A: Outlet thermometer 146B: Imported thermometer 148:Drive 150: Heat load 152: Load part 154:Thermionic components 156: Thermal Interface 158: Radiator

Claims

1. A data center thermal control system for cooling a fleet of electronic components, the data center thermal control system comprising one or more processors and a non-transitory computer-readable medium storing instructions, the instructions, when executed by the one or more processors, causing the one or more processors to control the data center thermal control system to cool the fleet to a different thermal margin lower limit during at least a portion of an early failure phase, compared to a stable phase, wherein: The early failure phase is a window after the installation of the electronic components, during which the expected failure rate of one of the electronic components decreases by at least a first rate based on historical failure data; the stabilization phase is a window after the early failure phase, during which the failure rate of the electronic components decreases by a rate lower than the first rate and increases by a rate lower than a second rate based on historical failure data; and the thermal margin is the difference between a predetermined temperature and an actual operating temperature of one of the electronic components.

2. The data center thermal control system of claim 1, wherein the instructions, when executed by the one or more processors, will cause the one or more processors to control the data center thermal control system to cool the queue to a stable lower thermal margin limit during the stabilization phase and to cool the queue to an early lower thermal margin limit during the early failure phase, wherein the early lower thermal margin limit is less than the stable lower thermal margin limit.

3. The data center thermal control system of claim 2, wherein the instructions, when executed by the one or more processors, will cause the one or more processors to control the data center thermal control system to cool the team to the early margin lower limit from the installation of the team until an adapted stabilization transition time, and to cool the team to at least the stable thermal margin lower limit at the beginning of the adapted stabilization transition time, wherein the adapted stabilization transition time is the earliest time after the installation of the team when the actual failure rate of one of the electronic components in the team is expected to decrease to less than a predetermined ratio.

4. The data center thermal control system as requested in item 1, wherein the historical fault data originates from observed faults in electronic equipment of the same type as those electronic components prior to installation.

5. The data center thermal control system of claim 1, wherein the different lower limit of thermal margin is lower than any point during at least a portion of the early failure phase than during the steady phase.

6. The data center thermal control system as requested in item 5, wherein the time amount for which the different thermal margin lower limit is kept low is a predetermined time amount, or a time amount corresponding to an actual or predicted transition from a typical early failure phase to a typical stable phase or from an adapted early failure phase to an adapted early stable phase.

7. The data center thermal control system of claim 1, wherein the different lower limit of thermal margin is higher than any point during at least a portion of the early failure phase than during the steady phase.

8. The data center thermal control system as requested in item 7, wherein the time for which the different thermal margin lower limit is maintained to rise is a predetermined time, or a time corresponding to an actual or predicted transition from a typical early failure phase to a typical stable phase or from an adapted early failure phase to an adapted early stable phase.

9. The data center thermal control system of claim 1, wherein the lower limit of the different thermal margin is higher than any point during at least a portion of a wear-out phase than during any point during the steady phase.

10. The data center thermal control system as claimed in claim 3, wherein the predetermined ratio is a predetermined increase ratio or a predetermined decrease ratio used to mark the transition between the early failure phase, the stable phase and a wear-out phase.

11. The data center thermal control system as requested in item 10, wherein the predetermined rate includes 1% per day, 2% per day, 3% per day, 4% per day, 5% per day, 10% per day, 15% per day, 20% per day and 25% per day.

Images