Prediction-based power reservation regulation of data center scale

Abstract

The invention relates to prediction-based power reservation regulation for data center scale. In various examples, systems and methods related to prediction-based power reservation regulation of a data center scale are disclosed. One or more circuits may receive power consumption data for a first time period from a plurality of components of a data center. The one or more circuits may generate predicted power consumption for the plurality of components for a second time period after the first time period using at least one prediction model and based at least on the power consumption data. The one or more circuits may determine a power policy for the plurality of components based at least on the predicted power consumption and a state of the data center, and cause the plurality of components to limit power consumption for a second period of time according to the power policy.

Description

Background Technology

[0001] As computing demands continue to grow, data centers are experiencing ever-increasing energy consumption, which is severely impacting the environment. Traditional power management strategies primarily focus on regulating the power consumption of central processing units (CPUs) within data centers. However, due to the diverse needs of different workloads and the variety of computing components, general power management in modern data centers faces challenges. Summary of the Invention

[0002] Modern data centers are constrained by power supply. However, once built, such data centers typically operate using only a small fraction of their available peak power capacity under normal circumstances. Traditional approaches to power management in data centers focus on managing the power consumption of active devices within the data center, including the central processing unit (CPU), which can lead to inefficient and poor power utilization strategies. Traditional methods often enforce high, fixed safety margins to ensure the data center does not exceed its maximum power allocation. Exceeding the maximum power consumption can result in severe data center outages. These traditional power management techniques also often rely heavily on application configuration data, which may be insufficient in environments running multiple types of computing workloads. Using specific application profiles can lead to suboptimal power allocation for other types of workloads.

[0003] To address these limitations, the technique described in this paper implements a centralized control loop that leverages predictive analytics to optimize power distribution among various components within a data center—e.g., any device in the data center that requires power tracking / management. The system collects telemetry data from active components of the data center cluster, encompassing graphics processing (GPU) devices, CPU devices, storage devices, and network devices such as switches or routers, as well as power management systems. Based on the collected telemetry data, one or more predictive models can be employed to predict the power consumption of each component over subsequent time intervals. The resulting policy (containing dynamically determined power caps for each component) can then be applied via hardware and software interfaces to enforce the caps over the time intervals. This process can be repeated to enforce dynamic policies across multiple time periods during data center operation, thereby improving overall efficiency and reducing operating costs.

[0004] At least one aspect involves one or more processors. The one or more processors may include one or more circuits. The one or more circuits may receive power consumption data for a first time period from multiple components in a data center. The one or more circuits may use at least one predictive model (e.g., a machine learning model, a sliding window predictor, etc.) and, at least based on the power consumption data, generate predicted power consumption for the multiple components in a second time period following the first time period. The one or more circuits may determine a power policy for the multiple components, at least based on the predicted power consumption and the state of the data center. The one or more circuits may cause the multiple components to limit (e.g., control, constrain) their power consumption in the second time period according to the power policy.

[0005] In some implementations, at least one prediction model includes a converter model or a Long Short-Term Memory (LSTM) model. In some implementations, at least one prediction model includes a sliding window prediction function. In some implementations, the plurality of components includes one or more of a graphics processing unit (GPU), a network interface controller (NIC), a network switch, a central processing unit (CPU), a storage device, or a cooling unit. In some implementations, the one or more circuits may further determine a power strategy based on at least one job being performed on the plurality of components in the data center (e.g., current / queued jobs in the data center).

[0006] In some implementations, the one or more circuits may determine the predicted power consumption of at least one processing job based at least on a previous instance of that processing job. In some implementations, the one or more circuits may determine a power strategy (e.g., a Hedge-based power strategy determined per job) based at least on the predicted power consumption of at least one processing job. In some implementations, the one or more circuits may further determine the power strategy based on a data center power budget. In some implementations, the one or more circuits may select at least one hyperparameter for the at least one prediction model (e.g., Hedge-based prediction model optimization) based at least on previously predicted power consumption at previous time steps. In some implementations, the one or more circuits may select a power strategy from multiple power strategies based at least on previously predicted power consumption at previous time steps.

[0007] At least one aspect relates to a system. The system may include one or more processors. The system may select at least one predictive model based at least on first power consumption data of multiple components in a data center during a first time period. The system may use the at least one predictive model to generate predicted power consumption for at least a subset of the multiple components in a second time period following the first time period. The system may, at least based on the predicted power consumption, limit the power consumption of at least said subset of the multiple components in the second time period.

[0008] In some implementations, the system can select at least one prediction model based on a Hedge function. In some implementations, the system can select at least one prediction model by selecting one or more hyperparameters for the at least one prediction model based at least on the first power consumption data corresponding to the first time period. In some implementations, the system can select a power policy type based at least on the first power consumption data and the power budget of the plurality of components. In some implementations, the system can generate a power policy for the data center based at least on the first power consumption data and the power policy type.

[0009] In some implementations, the system can identify a second subset of multiple components assigned to a processing job in a data center. In some implementations, the system can select at least one second prediction model for the second subset of multiple components, based at least on the first power consumption data. In some implementations, the system can generate a power policy for the second subset of multiple components, based at least on the predicted power consumption of the second subset. In some implementations, the system can also generate a power policy based at least on the estimated time to determine the maximum power consumption during the execution of the processing job.

[0010] At least one aspect relates to a method. The method may include acquiring power consumption data of multiple components in a data center during a first time period. The method may include generating predicted power consumption of the multiple components for a second time period following the first time period, using at least one prediction model and based at least on the power consumption data. The method may include determining a power strategy to limit the power consumption of the multiple components during the second time period, based at least on the predicted power consumption and the state of the data center.

[0011] In some implementations, the prediction model includes a transformer model or a Long Short-Term Memory (LSTM) model. In other implementations, the prediction model includes a sliding window prediction function.

[0012] The processors, systems, and / or methods described herein can be implemented by or included in at least one of the following: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing analog operations; a system for performing digital twin operations; a system for performing optical transmission simulation; a system for performing collaborative content creation of 3D assets; a system for performing deep learning operations; a system for performing generative AI operations using a small language model; a system for performing generative AI operations using a large language model; a system for performing generative AI operations using a video language model; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system for generating synthetic data; a system containing one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources. Attached Figure Description

[0013] The following describes in detail, with reference to the accompanying drawings, the system and method for implementing prediction-based power reservation steering at the data center scale, wherein:

[0014] Figure 1 This is a block diagram of an example system for implementing predictive power reservation regulation at the data center scale, according to some embodiments of the present disclosure.

[0015] Figure 2 Example diagrams illustrating example control loops for implementing prediction-based power reservation regulation according to some embodiments of the present disclosure are depicted.

[0016] Figure 3 This is a flowchart illustrating an example of a method for implementing prediction-based power reservation regulation at the data center scale.

[0017] Figure 4 This is a block diagram of an example computing device suitable for implementing some embodiments of the present disclosure; and

[0018] Figure 5 This is a block diagram of an example data center suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0019] This disclosure relates to systems and methods for predictive power reservation regulation for achieving data center scaling. Due to the exponential growth in computing demands, data center energy consumption is increasing, resulting in a significant environmental impact. Available power infrastructure often constrains data center design.

[0020] High-power computing accelerators like GPUs are becoming increasingly common in modern data centers. As a result, the power consumption of large-scale data centers varies significantly between periods of low and high compute load.

[0021] Overloading available power budgets in data centers is extremely costly. A single circuit breaker trip can cause downtime of hours or even days until all systems are stabilized and mitigation measures are implemented to prevent recurrence. Therefore, traditional data center design methods ensure that the total power of all data center components never exceeds the data center's power budget, even if all components are simultaneously consuming 100% of their rated power; this design point is known as no overprovisioning.

[0022] During normal data center operation, it's impossible to ever utilize 100% of available power. In a typical data center, at any given time, some nodes are under maintenance due to hardware or software issues, while others are idle, running different workloads. Furthermore, due to bottlenecks such as memory capacity, memory bandwidth, network latency and bandwidth, or poorly optimized algorithms, nodes running workloads rarely utilize 100% of their power budget. Therefore, most data centers utilize an average of 40%–70% of their available power budget.

[0023] To address these shortcomings, the data center design paradigm must shift from building under-subscribed data centers to building over-subscribed data centers, requiring a novel power management system to efficiently manage such data centers. The technology described in this paper implements a centralized control loop that leverages predictive analytics to optimize power allocation among components within an over-subscribed data center. To this end, the system collects telemetry data from active components of the data center cluster, including but not limited to GPU devices, CPU devices, storage devices, network devices such as switches or routers, and the power management system. This telemetry data includes real-time or near-real-time metrics for each component in the cluster, such as power consumption metrics.

[0024] Based on the collected telemetry data, one or more predictive models are used to predict the power consumption of each component in subsequent time intervals. The time intervals can be, but are not limited to, 1 minute, 30 seconds, 15 seconds, 10 seconds, or 1 second. The prediction period can be the same as or longer than the telemetry data collection period. The predictive models can include any suitable type of machine learning model and can predict the estimated power consumption of each component in subsequent time intervals. In some implementations, the model can receive additional data as input, such as previous time interval predictions for each component or power policy data from previous time intervals.

[0025] The predicted power consumption value for each component is then used to generate a policy for the corresponding time interval. This policy may include power limits for each component in the data center and may be determined based at least on the predicted power consumption, the current data center state (which may include the data center's power budget), job scheduler information, and other inputs. The generated policy may include dynamically determined power caps for each data center component. The generated policy can then be applied via hardware and / or software interfaces to enforce the limits imposed on each component by the policy during the time interval. This process can then be repeated to enforce the dynamic policy across multiple time periods during data center operation.

[0026] refer to Figure 1 , Figure 1 This is an example computing environment according to some embodiments of the present disclosure, which includes a system 100 for implementing prediction-based power reservation regulation at a data center scale. It should be understood that such and other arrangements described herein are merely illustrative examples. Other arrangements and elements (e.g., machines, interfaces, functions, commands, functional groups, etc.) may be used in addition to or in place of the arrangements and elements shown, and certain elements may be omitted entirely. Furthermore, many of the elements described herein are functional entities that may be implemented as discrete or distributed components, or in combination with other components, and may be implemented in any suitable combination and location. The various functions performed by the entities described herein may be performed by hardware, firmware, and / or software. For example, various functions may be performed by a processor executing instructions stored in memory.

[0027] System 100 is shown to include a data processing system 102 and one or more data center components 130A-130N (sometimes referred to as "data center component 130"). The data processing system 102 (and / or its components) can receive power consumption data 104 and processing job data 116 (sometimes referred to as "job data 116") from one or more data center components 130. The data processing system 102 (and / or its components) can provide policy instructions 114 to the data center components 130 to enable real-time or near-real-time power reservation control. The data processing system 102 can execute a data receiver 106 to receive power data 104 from the data center components 130 and execute a prediction generator 108 to generate a predicted power consumption 111 of the data center components 130 over a future time period (e.g., one time step). The prediction generator 108 can use one or more prediction models 110A-110N (sometimes collectively referred to as "prediction model 110 type") to generate predicted power consumption 111. The data processing system 102 can execute the policy generator 112 to generate power policies for the data center and send policy instructions 114 to control the power consumption of the data center components 130.

[0028] Data center component 130 may include any type of device capable of reporting (or providing data to devices capable of reporting) real-time or near-real-time power consumption of the data center. For example, data center component 130 may include any hardware that facilitates data processing, storage, and transmission within the data center. Data center component 130 may include servers, CPUs, GPUs, network interface controllers (NICs), network switches, and other components / devices / systems. In some implementations, a server may comprise multiple data center components 130, each reporting power consumption data 104 independently. In some implementations, data center component 130 may include one or more servers reporting overall power consumption data 104, allowing data processing system 102 to control the future power consumption of one or more servers holistically, rather than limiting / controlling the power consumption of each component individually.

[0029] A central processing unit (CPU) can be included in a node / server / data center processing infrastructure. Each CPU can include hardware or executing software capable of reporting power consumption data 104 for a given time period, as described in further detail herein. Data center components can include any suitable architecture, clock speed, core count, and other attributes. In some implementations, the data center component 130 containing the CPU can be equipped with power management circuitry or systems that can dynamically control the CPU's frequency to control or otherwise limit the CPU's power consumption according to policy instructions 114, as described in further detail herein.

[0030] In some implementations, data center component 130 may include one or more accelerated computing devices, such as graphics processing units (GPUs) or artificial intelligence accelerator circuits / devices. GPUs within the data center may include any device with multiple parallel computing units and shared memory, suitable for performing graphics processing tasks or machine learning (such as tensor operations). GPUs in the data center may be included in one or more processing nodes and may be connected via high-speed interconnects (such as PCIe, NVLINK, InfiniBand, or Ethernet). In some implementations, GPUs may be included as part of a distributed computing cluster within the data center. In some implementations, the GPUs of data center component 130 may report power consumption data 104 to data processing system 102 at different time intervals. GPUs may include hardware sensors or software interfaces that monitor and measure the power consumption of the GPU in real-time or near real-time for configurable time periods, or, in some implementations, derive the power consumption of the GPU in real-time or near real-time. In some implementations, power consumption data 104 may be periodically sampled and aggregated at specified time intervals, such as every 1 millisecond, every second, every 10 seconds, or any other time interval (which may be configurable as described herein). Power consumption data 104 may be transmitted or otherwise retrieved by data processing system 102. Data processing system 102 may execute data receiver 106 to retrieve the power consumption data 104 from the GPU of data processing system 102, which is then used for predictive analytics and policy generation as described herein. In some implementations, the GPU power consumption data 104 may be collected and / or transmitted by the hardware and / or software of the computing device / cluster hosting the GPU.

[0031] In some implementations, data center component 130 may include one or more network switches. Network switches facilitate communication and data transmission between various devices, components, and systems within the data center. For example, switches can provide routing and switching capabilities for various networks implemented within the data center and can support / implement various protocols and standards, including Ethernet, Fibre Channel, and InfiniBand.

[0032] The network switch and / or NIC of data center component 130 can report power consumption data 104 to data processing system 102. The network switch may include power monitoring circuitry or software that measures and / or manages the power usage of different ports and interfaces. The network switch may periodically collect the power consumption data 104 and be able to transmit it to data processing system 102 for reception and processing by data receiver 106, as described in further detail herein. The power consumption data 104 of the network switch may be provided to data processing system 102 at predetermined / configurable time intervals, as described herein.

[0033] The power consumption data 104 provided by / retrieved from data center component 130 may include any suitable metrics for quantifying the power consumption of each data center component 130 within a given time interval. The power consumption data 104 may be a numerical value representing the amount of power used by the respective data center component 130 during that time interval (e.g., in watts, milliwatts, kilowatts, etc.). In one example, the power consumption data 104 may include the respective average power usage of each data center component 130 during the given time interval.

[0034] In some implementations, certain data center components 130 can continuously report power usage data to data receiver 106, and data receiver 106 can calculate the cumulative / average power consumption of the respective data center components 130 over a predetermined time interval. In some implementations, data receiver 106 can periodically poll / query one or more data center components 130 to retrieve power consumption data 104 for the corresponding time interval, instead of receiving power consumption data 104 from one or more data center components 130. Data receiver 106 can store the received / retrieved power consumption data 104 in a data structure for subsequent analysis by prediction generator 108 and policy generator 112.

[0035] In some implementations, data center components 130 can be aggregated into "power domains." Power domains can be associated with components of the data center power distribution network 132, including components such as top-level power circuit breakers, circuit breakers in each server room, circuit breakers in each island, or other circuit breakers. In some implementations, power domains can be hierarchical; for example, they can contain other power domains. The data processing system 102 can store / maintain and use the power domain information for the data center components 130 and the power distribution network 132 to determine the power budget for each domain. Furthermore, some power domains can support telemetry data collection, which can be read by the data receiver 106 and used by the policy generator 112 to track power management compliance and monitor necessary safety margins to prevent any circuit breaker tripping.

[0036] Data receiver 106 may receive job data 116 from one or more job schedulers 117. Job schedulers 117 may include software, hardware, or a combination thereof running within the data center to manage and allocate processing jobs across various data center components 130. Job schedulers 117 may receive and process job requests, which may include detailed specifications and requirements for performing different processing tasks. For example, job schedulers 117 may determine the appropriate pattern to be allocated to a given processing job (e.g., data center component 130) based on the compute load and resource requirements indicated in the job data 116.

[0037] The job scheduler 117 can generate job data 116, which contains any information related to one or more processing jobs scheduled or executed by the data center component 130. Job data 116 may contain information related to the processing requirements, characteristics, or identifiers of one or more processing jobs. For example, job data 116 may indicate the expected processing time required for the processing job, the number of data center components 130 executing the processing job, the status of the processing job, and the identifiers of the data center components 130 assigned to the job. Job data 116 may specify resource allocation among different types of data center components 130 (such as CPUs, GPUs, NICs, and network switches).

[0038] In some implementations, job data 116 may also include information about the type of instructions implemented by the processing job, the type of data to be processed using the processing job, or other information related to one or more processing jobs. For example, job data 116 may specify whether the processing job involves a CPU-intensive task, a GPU-intensive task, or a combination of both. In some implementations, job data 116 may indicate the expected computational load and resource requirements of the processing job. In some implementations, job data 116 may include metadata associated with the processing job, such as the priority or type of the workload (e.g., machine learning, data analytics, web services, etc.), and other metadata.

[0039] In some implementations, job data 116 may include an indication of whether a power cap should be applied to the data center component 130 performing a particular processing job. Certain processing jobs may be marked (e.g., in job data 116) as being subject to power caps or limitations, allowing the processing job to be performed using the maximum processing capacity allocated to the component 130. In such implementations, job data 116 may include a flag indicating that the power of the component 130 to perform the processing job is unrestricted. In some implementations, job data 116 may indicate the minimum power allocation guaranteed for each component 130 allocated to the job. Processing job data 116 (including any associated metadata) may be provided by one or more job schedulers 117 in the data center.

[0040] The prediction generator 108 can access the power consumption data 104 and generate predicted power consumption data 111 for the next time interval. The prediction generator 108 can use one or more prediction models 110 to generate predicted power consumption 111 for some or all of the data center components 130 in subsequent time intervals. Each prediction model 110 can be any type of model that predicts (or is used / performed / applied to prediction) the power consumption of one or more data center components 130 in subsequent time intervals based on previous power consumption data 104. In one example, one or more prediction models 110 can be machine learning models, including but not limited to recurrent neural networks (RNNs), such as long short-term memory (LSTM) models, transformer-based models, or any other type of machine learning model.

[0041] In one example, one or more prediction models 110 may contain LSTM machine learning models. The LSTM models can be trained / updated to predict power consumption based on historical and real-time telemetry data (e.g., using supervised learning techniques). The LSTM models can contain various hidden state sizes, ranging from, for example, 5 to 60 states, and can contain any number of parameters, including up to 50,000 parameters. Similarly, one implementation of the predictive model 110 containing a transformer model can contain any number of transformer layers, and in one example, up to 150,000 parameters. The transformer model can be any suitable transformer-based machine learning model that can be trained / updated to receive multivariate time series data and generate regression / classification outputs.

[0042] In some implementations where the prediction model 110 includes a converter model and / or an LSTM / RNN model, the prediction model 110 may receive / store multiple time steps (e.g., time intervals) as input and generate predicted power consumption data 111 for each data center component 130 (or a subset of data center components 130) within subsequent time intervals. In some implementations, power consumption data 104 from any suitable number of previous time intervals may be used as input to the prediction model 110, including power consumption data 104 ranging from 5 minutes to approximately 60 minutes. In various implementations, the time interval may include intervals such as 15 milliseconds, 1 second, 5 seconds, 15 seconds, 30 seconds, or 1 minute.

[0043] In some implementations, prediction model 110 may include one or more sliding window predictors. Sliding window predictors can be used to calculate statistics such as the mean, standard deviation, and maximum power consumption of data center component 130 (or a subset thereof) within a sliding window of recent power consumption data 104 values. Sliding window predictors can combine statistical calculations to generate predicted power consumption data 111 for data center component 130 (or a subset thereof). In one example, the sliding window predictor can calculate the mean and standard deviation of power consumption data based on any number of previous intervals of power consumption data 104 for data center component 130. The sliding window predictor can then add the mean to twice the standard deviation to generate a predicted value for the next time interval. In another example, the sliding window predictor can generate the predicted power consumption using the sum of the maximum value and standard deviation of any number of previous intervals of power consumption data 104 for data center component 130.

[0044] In some implementations, the prediction generator 108 can run as a Hedge instance that executes multiple prediction models 110 to determine the optimal predicted power consumption 111 for the data center component 130. For example, one or more prediction generators 108 can execute a Hedge algorithm to track the losses of multiple prediction models 110 to generate the optimal power consumption value 111. In such implementations, each prediction model 110 implemented using the prediction generator 108 can contain a corresponding set of hyperparameters and / or model type (e.g., LSTM, sliding window, etc.). The prediction generator 108 can run by evaluating multiple prediction models 110 in parallel and selecting the output of the best prediction model 110 based on the evaluation results.

[0045] To implement the Hedge function, the prediction generator 108 can implement a loss function that helps the prediction generator 108 evaluate the performance of each prediction model 110 (e.g., each set of hyperparameters, each different type of prediction model 110, etc.). In one example, the loss function could be the root mean square error (RMSE) of each prediction model 110. For each prediction model 110, the loss can be calculated over the current time interval and updated in the next time interval when subsequent power consumption data 104 is received. Once the loss for each prediction model 110 has been calculated, the Hedge function can update and select the output of the prediction model 110 with the lowest loss value, which is provided as the predicted power consumption value 111.

[0046] Although only a single prediction generator 108 (e.g., a Hedge instance) is shown here, in some implementations, multiple prediction generators 108 can be implemented as multiple Hedge instances, where the corresponding Hedge instances can be used to generate predicted power consumption values ​​111 for multiple subsets of data center components 130 respectively. In some implementations, a single global Hedge instance can be used such that the predicted power consumption 111 for all data center components 130 is generated using the same prediction generator 108 (and one or more corresponding prediction models 110). In some implementations, multiple corresponding Hedge instances can be implemented for multiple data center components 130 in a cluster or set of clusters of data center components 130, or multiple corresponding Hedge instances can be provided for different processing jobs indicated in job data 116, as described in further detail herein.

[0047] In one example, multiple prediction generators 108 are initialized to implement Hedge instances based on each job (rather than a global prediction). In such implementations, prediction generators 108 can be initialized when a new processing job is detected in job data 116. The initialized prediction generators 108 can implement the Hedge function by iteratively calculating the loss of each of their prediction models 110 based on power consumption data 104 of the components 130 assigned to the processing job, according to the techniques described herein. The output of the prediction model 110 with the lowest loss for predicting the power consumption data 104 of the components 130 can be selected to generate predicted power consumption values ​​111 for these components. After the processing job is completed, the Hedge instance for that job can be released / deallocated. In some implementations, the data processing system 102 can store the loss values ​​of the Hedge instances for use in subsequent processing jobs with similar properties (e.g., similar operations, data, etc.) to be performed by the data center components 130. After such a processing job is detected in job data 116, a new prediction generator 108 can be initialized to implement a Hedge instance, which uses the loss values ​​(a set of prediction models 110) from similar previous jobs to select the output of prediction model 110 to generate a predicted power value 111.

[0048] The predicted power consumption 111 can be provided to the policy generator 112 to determine policy instructions 114 from one or more power policies 113A-113N (sometimes collectively referred to herein as "Power Policy 113" or "Multiple Power Policies 113"), thereby controlling the power limits of one or more data center components 130. Compared to conventional methods, policy instructions 114 can be generated to add additional computing power within a given data center power budget. The policy generator 112 can generate policy instructions 114, which may include instructions for controlling the power limits / caps of different data center components 130. Power policies 113 can be determined, generated, and / or modified for each time interval for calculating the predicted power consumption value 111. Power policies 113 can be determined based on the available power capacity of each component 130 in the data center and the predicted power consumption 111 of one or more components 130 in the data center.

[0049] In some implementations, the power strategy 113 may be further determined based on the state of the data center, which may include, but is not limited to, information such as current processing load, data center power budget, minimum power consumption of each component 130 of the data center, information in the job data 116 indicating one or more current or planned processing jobs, and / or information indicating which data center components 130 are active / online or inactive / offline for processing tasks. The state of the data center may be maintained, determined, or otherwise identified based on communications from one or more data center components 130 or other information sources associated with the data center (e.g., a state database, etc.).

[0050] Power policy 113 can be generated and / or determined using various techniques. In one example, policy generator 112 can determine / enforce power policy 113 using the predicted power consumption 111 of data center component 130, minimum and maximum power limits (e.g., hardware constraints, etc.) for each data center component 130, and the maximum allowable power consumption (e.g., maximum power budget) for all devices within the data center. To determine / calculate the output power limit based on power policy 113, policy generator 112 can assign a data structure to represent the power consumption of each active data center component 130 and add a minimum power consumption (which is the default minimum value when active) to the data structure. At this stage, the remaining power budget for the data center is equal to the maximum power budget minus the minimum power consumption of each active component 130.

[0051] Then, the policy generator 112 can sum the differences between the predicted power value 111 and the corresponding minimum power value for each component 130 to calculate the total additional power required to satisfy all power predictions for each component 130. If the remaining power budget is less than or equal to the additional power required to satisfy all power predictions, an equal proportion of the power budget can be allocated to each component 130 (e.g., by summing and updating the data structure), so that the total allocation equals the available power budget. At each allocation, the policy generator 112 can ensure that the power allocated to each component is within its minimum and maximum power thresholds and adjust it as needed. Since these adjustments can affect the power balance of the data center, the policy generator 112 can iteratively check and adjust the limits of the data center components 130 to execute the data center power budget while meeting the minimum and maximum thresholds for each component 130.

[0052] As described herein, in some implementations, job data 116 may indicate that one or more power components 130 will be uncapped or otherwise operated at their maximum power consumption values. In such implementations, policy generator 112 may determine / compute policies for components 130 that allocate maximum power consumption to these components 130 while limiting the power consumption of other components 130 according to the data center's power budget. In some implementations, policy generator 112 may compute multiple different types of policies for multiple subsets of components 130, including subsets of components 130 assigned to one or more processing jobs or having different power allocation priorities within the data center.

[0053] In some implementations, the policy generator 112 may implement a Hedge algorithm power policy that selects from multiple different power policies 113 to enforce power caps on one or more data center components 130. To implement the Hedge algorithm, the policy generator 112 may implement multiple policies 113 in parallel. Policies 113 may be models (sometimes referred to as "policy experts 113" or "policy model 113") that implement rules for setting power limits / caps on data center components 130 based on specific inputs (e.g., predicted power consumption data 111, other data described herein, etc.) to meet power constraints (such as global or domain power budgets) while ensuring that the power limits / caps for each data center component 130 are within the corresponding hardware limits. Policy models 113 may implement different hyperparameters, including but not limited to the initial job window size (e.g., no caps are enforced on the job's devices before learning the job's behavior), and / or the way policy models 113 allocate the remaining power budget (after prediction). For example, one strategy 113 could divide the power budget equally among the data center components 130, while another strategy could apply methods such as proportional fairness.

[0054] In some implementations, due to global power budget constraints (e.g., the interdependence of restrictions imposed on different components 130), the policy generator 112 may implement a single global policy Hedge instance for each power domain. In some implementations, the entire cluster of the data center may run on a single power domain, meaning that only a single global policy Hedge instance is implemented. In some implementations, the data center may contain multiple power domains, and the data processing system 102 may execute the corresponding policy generator 112 to implement a corresponding Hedge instance for each power domain in the data center.

[0055] The Hedge instance implemented by policy generator 112 can be a function of the power budget of all data center components 130, whose power is affected by policy instructions 114 generated by the Hedge instance. In one example, the loss can be a measure (or a percentage in some implementations) of the total number of data center components 130 whose power budget is under-allocated (e.g., underestimated) relative to the power budget allocated to these components 130 in the previous time interval. The loss can be determined based on power data 104 received from the components 130. Other losses can also be implemented by policy generator 112, including but not limited to losses of the root mean square error type or any other type of loss function that is a function of the underestimation of the power budget for each data center component 130 in the previous time interval.

[0056] In some implementations, policy generator 112 may implement a positive backoff function, for example, when the power allocated to one or more data center components 130 is determined to be insufficient. The positive backoff function can be implemented to ensure that data center components 130 receive sufficient power to operate efficiently without overprovisioning. In one example, the positive backoff function may automatically and exponentially increase the predicted power consumption of those data center components 130 based on the number of consecutive time intervals where the power consumption data 104 of one or more components 130 is equal to or approximately equal to the power allocation (indicating insufficient allocation). This can compensate for the power consumption of these data center components 130 that is limited by power caps. Policy generator 112 uses the positive backoff function to increase the predicted power consumption 111 for the next time interval to ensure sufficient power is allocated to the data center components 130.

[0057] In some implementations, policy generator 112 may use job data 116 to determine / compute one or more different types of power policies for one or more data center components 130. For example, job data 116 may indicate that a specific processing job will be executed in a subsequent time interval. After identifying the job to be executed, policy generator 112 may assign a data structure storing the value of the maximum power consumed during the processing job (or similar job type). The maximum power value may be initialized to an initial value (e.g., zero, a very large number, etc.) and may be updated during an initial time period. The initial time period may be a period during which no job-specific policy changes have been made to the corresponding processing job, and the estimated maximum power consumption of the job (e.g., for a given time interval of power consumption data 104) may be determined.

[0058] The timeframe for observing the estimated maximum power consumption of a processing job can vary and can be selected based on the type of the processing job or any other metadata / configuration information about that job. In some implementations, this timeframe can be predetermined or estimated. In some implementations, the policy generator 112 can estimate the maximum power consumption of the processing job based on its characteristics. Once the maximum power consumption of the processing job is determined, the policy generator 112 can determine / compute a policy that avoids allocating power exceeding the estimated maximum power consumption of the processing job to the component 130 executing the processing job. This type of power policy can be implemented because higher power consumption is unlikely to occur in subsequent time steps during the execution of the job by these data center components 130.

[0059] The output of the power policy 113, determined / computed by the policy generator 112, may include a set of power cap values ​​(e.g., allocated power amounts) for one or more data center components 130. These values ​​can be translated into a set of policy instructions 114, which may be instructions, commands, or actions that cause each active data center component 130 to conform to its respective power cap values ​​during subsequent time intervals. The policy instructions 114 can be generated in a format compatible with each corresponding component 130, each component 130 may contain different architectures, power configurations, operating points, and control / operation configurations. The policy instructions 114 can be transmitted such that the data center components 130 can implement these instructions with minimal latency during subsequent time intervals.

[0060] Data processing system 102 can provide policy instructions 114 to data center components 130 to control the power consumption limits of different devices in a predetermined order. For example, data processing system 102 can provide a first policy instruction 114 to a component 130 in the data center whose power limit is about to be reduced, so as to reduce the total power consumption of the data center before providing a second policy instruction 114 to increase the power limits of other components 130 in the data center. This prevents the power limits of previously set power limits of components 130 from being removed or significantly increased, which could lead to a sudden increase in the total power consumption of the data center before policy instructions 114 can be provided to other components 130 to reduce their power consumption limits / restrictions.

[0061] Once policy instruction 114 is provided, the power consumption limit of component 130 can be updated so that the power policy generated by that power policy can be executed in subsequent time intervals. This may include updating the configuration information of each component 130 to limit the amount of power that the corresponding component 130 can consume. In some implementations, the power consumption limit executed using policy instruction 114 may remain in effect until the corresponding component 130 is reset, reinitialized, or receives another policy instruction 114 that modifies the current power consumption configuration of component 130.

[0062] The data processing system 102 can then receive updated power consumption data 104 for subsequent time intervals and can repeat the techniques described herein to calculate a power policy for each time interval. The time interval for generating the power policy and / or policy instructions 114 can be a hyperparameter stored in the configuration settings of the data processing system 102. The configuration settings can be updated via operator input to the data processing system 102 and / or in response to messages / commands from external computing systems or computing systems within a data center. The data processing system 102 can be located inside a data center, or in some implementations, outside a data center.

[0063] refer to Figure 2 Example Figure 200 depicts an example control loop for implementing prediction-based power reservation regulation according to some embodiments of this disclosure. As shown, the control loop begins with data collection step 202, where each component of the data center (e.g., data center component 130) subject to the power capping / configuration techniques described herein collects / determines its instantaneous power consumption or an average value over a predetermined time period (e.g., a time interval). The average power consumption can be calculated by measuring the energy usage during that time period and dividing that energy usage by the length of that time period. Data collection step 202 can be performed by each component in the data center and can include access to current / voltage / power sensors that report power consumption at fixed sampling intervals.

[0064] The control loop continues to data formatting step 204, where the power management system (e.g., data processing system 102) receives, stores, and, in some implementations, formats the power consumption data generated from each component in step 202 (e.g., power consumption data 104). Data formatting step 204 may also involve cleaning / processing missing telemetry data. The power management system can then execute a prediction model (or multiple prediction models, e.g., implementing a Hedge function) in prediction step 204 to generate a predicted power consumption value for each component (e.g., predicted power consumption value 111).

[0065] The power management system can use the predicted power consumption values ​​in step 208 to calculate (determine, acquire, generate, etc.) a policy. In some implementations, and as shown here, job scheduler 214 (e.g., a provider of job data 116) can provide job information, which the power management system uses to calculate / determine / etc. one or more power policies, as described herein. Once determined, the power management system can broadcast the power policy to the various components of the data center in step 210, which may involve generating instructions (e.g., policy instruction 114) for each applicable component to implement the corresponding power consumption cap indicated in the policy.

[0066] Upon receiving the instruction, the components in the data center can, in step 212, apply the policy to subsequent time intervals by establishing power limits based on the policy broadcast in step 210. In some implementations, the policy for subsequent time intervals can be applied in two phases. During the first phase, all components whose power limits are to be reduced are reduced according to the policy. Once the power limits have been reduced, the second phase raises the power limits of all components whose power limits are to be raised in subsequent time periods according to the policy. The power consumed during that time period while the generated policy is being implemented can then be collected to calculate / determine the policy for subsequent time periods. This process can be repeated to continuously monitor and / or manage the power consumption of one or more components in the data center.

[0067] Now for reference Figure 3 Each block of the method 300 described herein contains a computational process that can be executed using any combination of hardware, firmware, and / or software. For example, various functions can be implemented by one or more processors executing instructions stored in memory. The method can also be embodied as computer-usable instructions stored on a computer storage medium. The method can be provided by any number of circuits, logic devices, applications, services, or managed services (standalone or in combination with another managed service), or plug-ins of other products. Furthermore, relative to… Figure 1 System 100 in the document describes method 300 by way of example. However, this method may also be performed additionally or alternatively by any system or any combination of systems, including but not limited to the system described herein.

[0068] Figure 3 This is a flowchart illustrating a method 300 for implementing prediction-based power reservation regulation at a data center scale according to some embodiments of the present disclosure. Method 300 includes, at block B302, receiving power consumption data (e.g., power consumption data 130) for a first time period from multiple components of the data center (e.g., data center component 130). The power consumption data may be collected and / or acquired at each data center component during the time period and transmitted to a power management system (e.g., data processing system 102, a system performing method 300, etc.). The power consumption value may include the average power consumption during the time period and may be provided in other possible values ​​such as milliwatts, watts, or kilowatts. In some embodiments, the power consumption value may be determined based on sensors or other circuitry in the data center component responsible for measuring the power consumption of the component in real-time or near real-time.

[0069] Method 300 at block B304 includes: using a prediction model and based at least on the power consumption data, generating a predicted power consumption (e.g., predicted power consumption 111) for a data center component in a second time period following the first time period. The prediction model may include an RNN model, an LSTM model, a transformer model, or a sliding window prediction model. In some implementations, multiple prediction models may be executed based on a Hedge function / Hedge instance as described herein. The Hedge function can be used to select the best prediction model for generating the predicted power consumption of the data center component. In some implementations, Hedge instances may be created for one or more processing jobs to be performed on the data center component (e.g., the processing jobs indicated in job data 116).

[0070] Method 300, at block B306, includes determining a power strategy for the components of the data center based at least on the predicted power consumption and the state of the data center. Calculating / determining the power strategy may include performing the steps outlined herein. Figure 1 The strategy generator 112 describes any technique. For example, a power policy can be generated based on the minimum power consumption of each component, the predicted power consumption of each component, and the power budget of the data center. In some implementations, multiple policies can be calculated / determined based on one or more Hedge instances. In one example, a global Hedge instance can be implemented for the components of the data center. The power policy can specify a power cap for each component in its corresponding data center.

[0071] Method 300 includes, at block B308, transmitting instructions (e.g., policy instruction 114) to cause components of the data center to limit power consumption according to the power policy. The policy instruction may include instructions to modify one or more configurations of components to cause the components to limit their respective power consumption during a second time period. The transmission instruction may include transmitting a first instruction to reduce the power limit before transmitting a second instruction to increase the power limit, to avoid potentially exceeding the data center's power capacity. Once the instructions are provided, each component can execute / implement these instructions to reduce, increase, or maintain its corresponding power limit. Method 300 can then return to step 302 to receive the power consumption for the second time period.

[0072] The systems and methods described herein can be used for a variety of purposes, such as, but not limited to, circuit layout definition, machine control, machine motion, machine driving, synthetic data generation, model training, perception, augmented reality, virtual reality, mixed reality, robotics, security and surveillance, simulation and digital twins, autonomous or semi-autonomous machine applications, deep learning, environmental simulation, object or participant simulation and / or digital twins, data center processing, conversational artificial intelligence (AI), optical transmission simulation (e.g., ray tracing, path tracing, etc.), collaborative content creation of three-dimensional (3D) assets, cloud computing, generative AI, and / or any other suitable application.

[0073] The disclosed embodiments can be included in a variety of different systems, such as automotive systems (e.g., control systems for autonomous or semi-autonomous machines, perception systems for autonomous or semi-autonomous machines), systems implemented using robots, aviation systems, medical systems, boating systems, smart area monitoring systems, systems for performing deep learning operations, systems for performing simulation operations, systems for performing digital twin operations, systems implemented using edge devices, systems containing one or more virtual machines (VMs), systems for performing synthetic data generation operations, systems implemented at least partially in a data center, systems for performing conversational AI operations, systems implementing one or more language models (such as one or more large language models (LLMs)), systems for performing optical transmission simulation, systems for performing collaborative content creation of 3D assets, systems implemented at least partially using cloud computing resources, and / or other types of systems.

[0074] Example computing device

[0075] Figure 4This is a block diagram of an example computing device 400 suitable for implementing some embodiments of the present disclosure. The computing device 400 may include an interconnect system 402 directly or indirectly coupled to: a memory 404, one or more central processing units (CPUs) 406, one or more graphics processing units (GPUs) 408, a communication interface 410, input / output (I / O) ports 412, input / output components 414, a power supply 416, one or more presentation components 418 (e.g., one or more displays), and one or more logic units 420. In at least one embodiment, one or more computing devices 400 may include one or more virtual machines (VMs), and / or any component of its components may include virtual components (e.g., virtual hardware components). For a non-limiting example, one or more GPUs 408 may include one or more vGPUs, one or more CPUs 406 may include one or more vCPUs, and / or one or more logic units 420 may include one or more virtual logic units. Accordingly, one or more computing devices 400 may include discrete components (e.g., a full GPU dedicated to computing device 400), virtual components (e.g., a portion of the GPU dedicated to computing device 400), or a combination thereof.

[0076] although Figure 4 The various boxes are shown as being connected to lines via interconnect system 402, but this is not intended to be limiting and is merely for clarity. For example, in some embodiments, presentation component 418 (such as a display device) may be considered I / O component 414 (e.g., if the display is a touchscreen). As another example, CPU 406 and / or GPU 408 may include memory (e.g., memory 404 may represent a storage device other than the memory of GPU 408, CPU 406, and / or other components). In other words, Figure 4 The computing devices described are for illustrative purposes only. No distinction is made between such categories as “workstation,” “server,” “laptop computer,” “desktop computer,” “tablet,” “client device,” “mobile device,” “handheld device,” “game console,” “electronic control unit (ECU),” “virtual reality system,” and / or other device or system types, as all are considered within the scope of this description. Figure 4 Within the scope of computing devices.

[0077] Interconnect system 402 may represent one or more links or buses, such as address buses, data buses, control buses, or combinations thereof. Interconnect system 402 may include one or more bus or link types, such as Industry Standard Architecture (ISA) bus, Extended Industry Standard Architecture (EISA) bus, Video Electronics Standards Association (VESA) bus, Peripheral Component Interconnect (PCI) bus, Peripheral Component Interconnect High Speed ​​(PCIe) bus, and / or another type of bus or link. In some embodiments, there is a direct connection between components. As an example, CPU 406 may be directly connected to memory 404. Further, CPU 406 may be directly connected to GPU 408. In cases where there is a direct or point-to-point connection between components, interconnect system 402 may include a PCIe link to perform the connection. In these examples, a PCI bus is not required to be included in computing device 400.

[0078] Memory 404 may include any of a variety of computer-readable media. Computer-readable media may be any available medium accessible by computing device 400. Computer-readable media may include volatile and non-volatile media, as well as removable and non-removable media. By way of example and not limitation, computer-readable media may include computer storage media and communication media.

[0079] Computer storage media may include volatile and non-volatile media and / or removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules, and / or other data types. For example, memory 404 may store computer-readable instructions (e.g., representing one or more programs and / or one or more program elements, such as an operating system). Computer storage media may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible by computing device 400. As used herein, computer storage media does not include the signal itself.

[0080] Computer storage media may embody computer-readable instructions, data structures, program modules, and / or other data types in modulated data signals, such as carrier waves or other transmission mechanisms, and include any information delivery medium. The term "modulated data signal" may refer to a signal whose one or more characteristics are set or altered in a manner that encodes information in the signal. By way of example and not limitation, computer storage media may include wired media (such as wired networks or direct wired connections) and wireless media (such as acoustic, RF, infrared, and other wireless media). Any combination of the above should also be included within the scope of computer-readable media.

[0081] One or more CPUs 406 may be configured to execute at least some of the computer-readable instructions to control one or more components of computing device 400 to perform one or more of the methods and / or processes described herein. Each of the one or more CPUs 406 may include one or more cores (e.g., one, two, four, eight, twenty-eight, seventy-two, etc.) capable of processing numerous software threads simultaneously. The one or more CPUs 406 may include any type of processor and, depending on the type of computing device 400 implemented, may include different types of processors (e.g., processors with fewer cores for mobile devices and processors with more cores for servers). For example, depending on the type of computing device 400, the processor may be an advanced RISC machine (ARM) processor implemented using Reduced Instruction Set Computing (RISC) or an x86 processor implemented using Complex Instruction Set Computing (CISC). In addition to one or more microprocessors or supplementary coprocessors (such as math coprocessors), computing device 400 may include one or more CPUs 406.

[0082] In addition to or in lieu of one or more CPUs 406, one or more GPUs 408 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 400 to perform one or more of the methods and / or processes described herein. One or more GPUs 408 may be integrated GPUs (e.g., where one or more CPUs 406 and / or one or more GPUs 408 may be discrete GPUs). In embodiments, one or more GPUs 408 may be coprocessors of one or more CPUs 406. One or more GPUs 408 may be used by computing device 400 to render graphics (e.g., 3D graphics) or perform general-purpose computing. For example, one or more GPUs 408 may be used for general-purpose computing on a GPU (GPGPU). One or more GPUs 408 may include hundreds or thousands of cores capable of processing hundreds or thousands of software threads simultaneously. One or more GPUs 408 may generate pixel data for an output image in response to rendering commands (e.g., rendering commands received via a host interface from one or more CPUs 406). One or more GPUs 408 may include graphics memory, such as display memory, for storing pixel data or any other suitable data, such as GPGPU data. Display memory may be included as part of memory 404. One or more GPUs 408 may include two or more GPUs operating in parallel (e.g., via a link). The link may directly connect the GPUs (e.g., using NVLINK) or may connect the GPUs via a switch (e.g., using NVSwitch). When combined, each GPU 408 may generate pixel data or GPGPU data for a different portion of the output or for different outputs (e.g., a first GPU for a first image and a second GPU for a second image). Each GPU may include its own memory or may share memory with other GPUs.

[0083] In addition to or alternatively to one or more CPUs 406 and / or one or more GPUs 408, one or more logic units 420 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 400 to perform one or more of the methods and / or processes described herein. In embodiments, one or more CPUs 406, one or more GPUs 408, and / or one or more logic units 420 may execute any combination of methods, processes, and / or portions thereof discretely or jointly. One or more logic units 420 may be one or more CPUs 406 and / or one or more GPUs 408 and / or integrated into one or more CPUs 406 and / or one or more GPUs 408, and / or one or more logic units 420 may be discrete components or otherwise external to one or more CPUs 406 and / or one or more GPUs 408. In an embodiment, one or more of the logic units 420 may be coprocessors of one or more CPUs 406 and / or one or more GPUs 408.

[0084] Examples of one or more logic units 420 include one or more processing cores and / or components thereof, such as a data processing unit (DPU), a tensor core (TC), a tensor processing unit (TPU), a pixel vision core (PVC), a vision processing unit (VPU), a graphics processing cluster (GPC), a texture processing cluster (TPC), a streaming multiprocessor (SM), a tree traversal unit (TTU), an artificial intelligence accelerator (AIA), a deep learning accelerator (DLA), an arithmetic logic unit (ALU), an application-specific integrated circuit (ASIC), a floating-point unit (FPU), input / output (I / O) elements, peripheral component interconnect (PCI) or fast peripheral component interconnect (PCIe) elements, etc.

[0085] Communication interface 410 may include one or more receivers, transmitters, and / or transceivers that enable computing device 400 to communicate with other computing devices via electronic communication networks, including wired and / or wireless communications. Communication interface 410 may include components and functions for enabling communication over any of a plurality of different networks, such as wireless networks (e.g., Wi-Fi, Z-Wave, Bluetooth, Bluetooth LE, ZigBee, etc.), wired networks (e.g., via Ethernet or wireless bandwidth), low-power wide area networks (e.g., LoRaWAN, SigFox, etc.), and / or the Internet. In one or more embodiments, logic unit 420 and / or communication interface 410 may include one or more data processing units (DPUs) to directly transmit data received via a network and / or via interconnect system 402 to one or more GPUs 408 (e.g., their memory).

[0086] I / O port 412 enables computing device 400 to be logically coupled to other devices including I / O component 414, one or more presentation components 418, and / or other components, some of which may be built into (e.g., integrated into) computing device 400. Illustrative I / O component 414 includes microphones, mice, keyboards, joysticks, game pads, game controllers, satellite dish antennas, scanners, printers, wireless devices, etc. I / O component 414 can provide a natural user interface (NUI) that processes air gestures, voice, or other physiological input generated by the user. In some cases, input may be sent to appropriate network elements for further processing. The NUI can implement any combination of voice recognition, pen recognition, facial recognition, biometric recognition, on-screen and near-screen gesture recognition, air gestures, head and eye tracking, and touch recognition (as described in more detail below) associated with the display of computing device 400. Computing device 400 may include depth cameras, such as stereo camera systems, infrared camera systems, RGB camera systems, touchscreen technology, and combinations thereof, for gesture detection and recognition. Additionally, computing device 400 may include an accelerometer or gyroscope (e.g., as part of an inertial measurement unit (IMU)) that enables motion detection. In some examples, computing device 400 may use the output of the accelerometer or gyroscope to render immersive augmented reality or virtual reality.

[0087] Power supply 416 may include a hard-wired power supply, a battery power supply, or a combination thereof. Power supply 416 may provide power to computing device 400 so that the components of computing device 400 can operate.

[0088] One or more presentation components 418 may include displays (e.g., monitors, touchscreens, television screens, head-up displays (HUDs), other display types, or combinations thereof), speakers, and / or other presentation components. One or more presentation components 418 may receive data from other components (e.g., one or more GPUs 408, one or more CPUs 406, DPUs, etc.) and output said data (e.g., as images, videos, sounds, etc.).

[0089] Example Data Center

[0090] Figure 5 An example data center 500 that can be used in at least one embodiment of this disclosure is shown. The data center 500 may include a data center infrastructure layer 510, a framework layer 520, a software layer 530, and / or an application layer 540.

[0091] like Figure 5 As shown, the data center infrastructure layer 510 may include a resource coordinator 512, grouped computing resources 514 and node computing resources (“nodes CR”) 516(1)-516(N), where “N” represents any whole positive integer. In at least one embodiment, nodes CR516(1)-516(N) may include, but are not limited to, any number of central processing units (“may include”) or other processors (including DPUs, accelerators, field-programmable gate arrays (FPGAs), graphics processors or graphics processing units (GPUs), etc.), memory devices (e.g., dynamic read-only memory), storage devices (e.g., solid-state or disk drives), network input / output (“output” devices, network switches, virtual machines (“devices”), power modules and / or cooling modules (e.g., cooling units), etc. In some embodiments, one or more nodes CR from nodes CR516(1)-516(N) may correspond to servers having one or more of the aforementioned computing resources. Furthermore, in some embodiments, nodes CR516(1)-516(N) may include one or more virtual components, such as vGPUs, vCPUs, etc., and / or one or more nodes CR516(1)-516(N) may correspond to virtual machines (VMs).

[0092] In at least one embodiment, the grouped computing resources 514 may include individual groups of nodes CR516 housed within one or more racks (not shown), or individual groups of many racks housed within a data center in different geographical locations (also not shown). Individual groups of nodes CR516 within the grouped computing resources 514 may include grouped computing, networking, memory, or storage resources that can be configured or allocated to support one or more workloads. In at least one embodiment, several nodes CR516, including CPUs, GPUs, DPUs, and / or other processors, may be grouped within one or more racks to provide computing resources to support one or more workloads. One or more racks may also include any number of power modules, cooling modules, and / or network switches in any combination.

[0093] Resource coordinator 512 may be configured or otherwise control one or more nodes CR516(1)-516(N) and / or grouped computing resources 514. In at least one embodiment, resource coordinator 512 may include a software design infrastructure (“software design”) management entity for data center 500. Resource coordinator 512 may include hardware, software, or some combination thereof.

[0094] In at least one embodiment, such as Figure 5 As shown, framework layer 520 may include job scheduler 528, configuration manager 534, resource manager 536, and / or distributed file system 538. Framework layer 520 may include a framework for software 532 supporting software layer 530 and / or one or more applications 542 of application layer 540. Software 532 or application 542 may respectively include web-based service software or applications, such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure. Framework layer 520 may be, but is not limited to, a free and open-source software web application framework that can use distributed file system 538 for large-scale data processing (e.g., "big data"), such as Apache Spark. TM(Hereinafter referred to as "hereinafter"). In at least one embodiment, job scheduler 528 may include a Spark driver to facilitate the scheduling of workloads supported by various layers of data center 500. Configuration manager 534 may be able to configure different layers, such as software layer 530 and framework layer 520 including Spark and distributed file system 538, to support large-scale data processing. Resource manager 536 may be able to manage computing resources mapped to or allocated to clusters or groups of distributed file system 538 and job scheduler 528 to support clusters or groups of distributed file system 538 and job scheduler 532. In at least one embodiment, clustered or grouped computing resources may include grouped computing resources 514 at data center infrastructure layer 510. Resource manager 536 may coordinate with resource coordinator 512 to manage these mapped or allocated computing resources.

[0095] In at least one embodiment, the software 532 included in the software layer 530 may include software used in at least a portion of the nodes CR516(1)-516(N), the grouped computing resources 514, and / or the distributed file system 538 of the framework layer 520. One or more types of software may include, but are not limited to, internet webpage search software, email virus scanning software, database software, and streaming video content software.

[0096] In at least one embodiment, one or more applications 542 included in application layer 540 may include one or more types of applications used by at least a portion of the nodes CR516(1)-516(N) of framework layer 520, grouped computing resources 514, and / or distributed file system 538. One or more types of applications may include, but are not limited to, any number of genomics applications, cognitive computing, and machine learning applications, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), and / or other machine learning applications used in combination with one or more embodiments. In some embodiments, nodes CRs 516(1)-516(N) may include one or more computing nodes, storage nodes, and / or management nodes in a data center. By performing a combination... Figure 1 The data processing system 102 describes an operation in which one or more nodes CRs 516(1)-516(N) can operate as power reservation control controller entities. In some implementations, one or more management nodes can be designated to execute data center management software (e.g., as head nodes). Job scheduler 528 can be executed on one or more head nodes in the data center.

[0097] In at least one embodiment, any of the configuration manager 534, resource manager 536, and resource coordinator 512 can implement any number and type of self-modification actions based on any amount and type of data acquired in any technically feasible manner. Self-modification actions can free data center operators of data center 500 from making potentially poor configuration decisions and potentially avoid underutilization and / or poor performance of the data center.

[0098] According to one or more embodiments described herein, data center 500 may include tools, services, software, or other resources to train / update one or more machine learning models or to predict or infer information using one or more machine learning models. For example, one or more machine learning models can be trained by calculating weight parameters according to a neural network architecture using the software and / or computing resources described above regarding data center 500. In at least one embodiment, a trained / updated or deployed machine learning model corresponding to one or more neural networks can be used to infer or predict information using the resources described above regarding data center 500 by using weight parameters calculated through one or more training techniques (such as, but not limited to, those described herein).

[0099] In at least one embodiment, the data center 500 may use a CPU, application-specific integrated circuit (ASIC), GPU, FPGA, and / or other hardware (or corresponding virtual computing resources) to perform training and / or inference using the aforementioned resources. Furthermore, one or more of the software and / or hardware resources described above may be configured to allow users to train / update or perform services that infer information, such as image recognition, speech recognition, or other artificial intelligence services.

[0100] Example network environment

[0101] A network environment suitable for implementing embodiments of this disclosure may include one or more client devices, servers, network-attached storage (NAS), other backend devices, and / or other device types. Client devices, servers, and / or other device types (e.g., each device) may be... Figure 4 It is implemented on one or more instances of one or more computing devices 400—for example, each device may include similar components, features, and / or functions of one or more computing devices 400. Furthermore, in the case of implementing backend devices (e.g., servers, NAS, etc.), the backend devices may be included as part of the data center 500, examples of which are described herein. Figure 5 To describe in more detail.

[0102] Components of a network environment can communicate with each other via one or more networks, which can be wired, wireless, or both. A network may include multiple networks or one of multiple networks. For example, a network may include one or more wide area networks (WANs), one or more local area networks (LANs), one or more public networks such as the Internet and / or the Public Switched Telephone Network (PSTN), and / or one or more private networks. In cases where the network includes a wireless telecommunications network, components such as base stations, communication towers, or even access points (and other components) can provide wireless connectivity.

[0103] A compatible network environment may include one or more peer-to-peer network environments (in which case the server may not be included in the network environment) and one or more client-server network environments (in which case one or more servers may be included in the network environment). In a peer-to-peer network environment, the functionality described herein with respect to one or more servers can be implemented on any number of client devices.

[0104] In at least one embodiment, the network environment may include one or more cloud-based network environments, distributed computing environments, combinations thereof, etc. A cloud-based network environment may include a framework layer, a job scheduler, a resource manager, and a distributed file system implemented on one or more servers, which may include one or more core network servers and / or edge servers. The framework layer may include a framework of software supporting the software layer and / or application layers of one or more applications. The software or applications may respectively include web-based service software or applications. In embodiments, one or more client devices may use web-based service software or applications (e.g., by accessing the service software and / or applications via one or more application programming interfaces (APIs)). The framework layer may be, but is not limited to, free and open-source software web application frameworks, such as those used for large-scale data processing (e.g., "big data") using a distributed file system.

[0105] A cloud-based network environment can provide cloud computing and / or cloud storage for any combination of the computing and / or data storage functions (or one or more portions thereof) described herein. Any of these different functions can be distributed across multiple locations from a central or core server (e.g., distributed across one or more data centers at the state, region, country, planet, etc.). If the connection to a user (e.g., a client device) is relatively close to one or more edge servers, one or more core servers can assign at least a portion of the functionality to one or more edge servers. A cloud-based network environment can be private (e.g., limited to a single organization), public (e.g., available to many organizations), and / or a combination thereof (e.g., a hybrid cloud environment).

[0106] One or more client devices may be included in this discussion. Figure 4 The components, features, and functions of one or more example computing devices 400 described herein are at least some of the components, features, and functions described herein. By way of example and not limitation, the client device may be implemented as a personal computer (PC), laptop computer, mobile device, smartphone, tablet computer, smartwatch, wearable computer, personal digital assistant (PDA), MP3 player, virtual reality headset, global positioning system (GPS) or device, video player, camera, surveillance equipment or system, vehicle, ship, spacecraft, virtual machine, drone, robot, handheld communication device, hospital equipment, gaming device or system, entertainment system, vehicle computer system, embedded system controller, remote control, electrical appliance, consumer electronics device, workstation, edge device, any combination of these depicted devices, or any other suitable device.

[0107] This disclosure can be described in the general context of computer code or machine-usable instructions, including computer-executable instructions (such as program modules) that are executed by a computer or other machine (such as a personal data assistant or other handheld device). Typically, a program module, including routines, programs, objects, components, data structures, etc., refers to code that performs a specific task or implements a specific abstract data type. This disclosure can be practiced in a variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, and more specialized computing devices. This disclosure can also be practiced in distributed computing environments, where tasks are performed by remote processing devices linked via a communication network.

[0108] As used herein, the phrase “and / or” relating to two or more elements should be interpreted as referring to only one element or a combination of elements. For example, “element A, element B and / or element C” can include only element A, only element B, only element C, element A and element B, element A and element C, element B and element C, or element A, B and C. Furthermore, “at least one of element A or element B” can include at least one of element A, at least one of element B, or at least one of element A and at least one of element B. Further, “at least one of element A and element B” can include at least one of element A, at least one of element B, or at least one of element A and at least one of element B.

[0109] This document provides a specific description of the subject matter of this disclosure to meet statutory requirements. However, this description itself is not intended to limit the scope of this disclosure. Rather, the inventors have anticipated that the claimed subject matter may be embodied in other ways, in conjunction with other current or future techniques, to include different steps or combinations of steps similar to those described in this document. Furthermore, although the terms “step” and / or “box” may be used herein to indicate different elements of the method employed, these terms should not be construed as indicating any particular order among or between the various steps disclosed herein, unless the order of the steps is explicitly described.

Claims

1. One or more processors, said one or more processors comprising: One or more circuits are used for: Receive power consumption data for the first time period from multiple components in the data center; Using at least one prediction model and based at least on the power consumption data, generate the predicted power consumption of the plurality of components for a second time period following the first time period; The power strategy for the plurality of components is determined at least based on the predicted power consumption and the state of the data center; as well as This causes the multiple components to limit power consumption during the second time period according to the power strategy.

2. The processors according to claim 1, wherein, The at least one prediction model includes a converter model or a long short-term memory (LSTM) model.

3. The processors according to claim 1, wherein, The at least one prediction model includes a sliding window prediction function.

4. The processors according to claim 1, wherein, The plurality of components includes one or more of a graphics processing unit (GPU), a network interface controller (NIC), a network switch, a central processing unit (CPU), a storage device, or a cooling unit.

5. The processors according to claim 1, wherein, The one or more circuits are used for: The power strategy is also determined based on at least one operation performed on the plurality of components in the data center.

6. The processors according to claim 5, wherein, The one or more circuits are used for: The predicted power consumption of the at least one job is determined based at least on previous instances of the at least one job; and The power strategy is determined based at least on the predicted power consumption of the at least one operation.

7. The processors according to claim 1, wherein, The one or more circuits are used for: The power strategy is also determined based on the power budget of the data center.

8. The processors according to claim 1, wherein, The one or more circuits are used for: At least one hyperparameter is selected for the at least one prediction model based on the previously predicted power consumption at the previous time step.

9. The processors according to claim 1, wherein, The one or more circuits are used for: The power strategy is selected from multiple power strategies based at least on the previously predicted power consumption at the previous time step.

10. The processors according to claim 1, wherein, The one or more processors are included in at least one of the following: Control systems for autonomous or semi-autonomous machines; Sensing systems for autonomous or semi-autonomous machines; A system used to perform simulation operations; Systems used to perform digital twin operations; A system for performing optical transmission simulation; A system for collaborative content creation of 3D assets; A system used to perform deep learning operations; Systems implemented using edge devices; Systems implemented using robots; A system for performing conversational AI operations; A system for performing generative AI operations using large language model LLM; A system for performing generative AI operations using a small language model (SLM); A system for performing generative AI operations using the Video Language Model (VLM); A system for performing generative AI operations using multimodal language models; A system for generating synthetic data; A system containing one or more virtual machines (VMs); A system that is at least partially implemented in a data center; or A system that utilizes cloud computing resources at least in part.

11. A system comprising: One or more processors are used for: At least one predictive model is selected based on the first power consumption data of multiple components in the data center during the first time period; The at least one prediction model is used to generate predicted power consumption of at least a subset of the plurality of components in a second time period following the first time period; as well as Based at least on the predicted power consumption, at least a subset of the plurality of components limits the power consumption during the second time period.

12. The system according to claim 11, wherein, The one or more processors are used for: Select at least one prediction model based on the Hedge function.

13. The system according to claim 11, wherein, The one or more processors are used for: The at least one prediction model is selected by selecting one or more hyperparameters for the at least one prediction model based at least on the first power consumption data corresponding to the first time period.

14. The system according to claim 11, wherein, The one or more processors are used for: The power strategy type is selected based at least on the first power consumption data and the power budget of the plurality of components; and A power policy is generated for the data center based at least on the first power consumption data and the power policy type.

15. The system according to claim 11, wherein, The one or more processors are used for: Identify a second subset of the plurality of components that are assigned to the processing jobs in the data center; At least one second prediction model is selected for the second subset of the plurality of components based on the first power consumption data; as well as A power strategy is generated for the second subset of the plurality of components based at least on the predicted power consumption of the second subset.

16. The system according to claim 15, wherein, The one or more processors are used for: The power strategy is also generated at least based on the estimated time to determine the maximum power consumption during the execution of the processing job.

17. The system according to claim 11, wherein, The system is included in at least one of the following: Control systems for autonomous or semi-autonomous machines; Sensing systems for autonomous or semi-autonomous machines; A system used to perform simulation operations; Systems used to perform digital twin operations; A system for performing optical transmission simulation; A system for collaborative content creation of 3D assets; A system used to perform deep learning operations; Systems implemented using edge devices; Systems implemented using robots; A system for performing conversational AI operations; A system for performing generative AI operations using large language model LLM; A system for performing generative AI operations using a small language model (SLM); A system for performing generative AI operations using the Video Language Model (VLM); A system for performing generative AI operations using multimodal language models; A system for generating synthetic data; A system containing one or more virtual machines (VMs); A system that is at least partially implemented in a data center; or A system that utilizes cloud computing resources at least in part.

18. A method, the method comprising: Obtain power consumption data for multiple components in the data center during the first time period; Using at least one prediction model and based at least on the power consumption data, generate the predicted power consumption of the plurality of components in a second time period following the first time period; as well as Based at least on the predicted power consumption and the state of the data center, a power strategy is determined to limit the power consumption of the plurality of components during the second time period.

19. The method according to claim 18, wherein, The at least one prediction model includes a converter model or a long short-term memory (LSTM) model.

20. The method according to claim 18, wherein, The at least one prediction model includes a sliding window prediction function.

Images