Energy consumption regulation method and system of GPU BOX, and computer device

Abstract

The application discloses a GPU BOX energy consumption regulation method and system, and a computer device. The method comprises the following steps: performing feature analysis on a received to-be-processed task, and identifying task characteristic information; acquiring historical performance data of all GPU BOXs of a same node corresponding to a task type, constructing a GPU BOX performance model according to real-time acquired state information of all GPU BOXs, and acquiring a GPU BOX allocation strategy for executing the task in the same node according to the model; determining a target GPU BOX according to the GPU BOX allocation strategy, and executing the to-be-processed task; monitoring running information of the target GPU BOX in real time, acquiring a GPU BOX energy consumption dynamic regulation strategy according to the running information, and executing the GPU BOX energy consumption dynamic regulation strategy. The method can realize dynamic regulation of energy consumption of the GPU BOX under different working modes, realize efficient utilization of energy consumption, prolong the service life of the equipment, and save energy and reduce cost.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method and system for controlling the power consumption of a GPU BOX, and a computer device. Background Technology

[0002] In current technologies, the demand for AI applications, primarily focused on fine-tuning and inference, is experiencing explosive growth. This has led to a significant increase in the frequency of GPU server usage and computational load. During GPU server operation, the main power consumption comes from the individual GPU modules. In processes such as large model pre-training and fine-tuning, if a GPU involved in the task malfunctions, the entire task will fail and need to be restarted. This not only wastes a significant amount of computing resources but also increases unnecessary energy consumption. Even under low load conditions, GPUs maintain a high energy consumption state. Even when the server is handling a small amount of tasks, GPUs still consume a large amount of power, severely impacting overall energy efficiency and resulting in high maintenance costs. Furthermore, GPUs generate a large amount of heat under prolonged high-load operation, which can damage their internal electronic components, reducing their performance and stability. Frequent replacement of aging GPUs not only increases hardware costs but also affects the normal operation of the server.

[0003] Currently, GPU server power management mostly adopts a simple cumulative approach, relying on manual control of the GPU server's power on / off state. This approach cannot dynamically activate and use the GPU based on actual task requirements. Furthermore, the power control interfaces of different GPU manufacturers are not standardized, making the management process complex and inefficient. Manual management is not only prone to operational errors but also cannot adjust the server's power consumption status in a timely manner according to changes in tasks. Summary of the Invention

[0004] In view of this, the present disclosure provides a method and system for regulating the energy consumption of a GPU BOX, as well as a computer device, which can solve the problems of high energy consumption, low energy utilization efficiency, and inability to dynamically adjust the energy consumption of multiple GPU BOX servers under different task states in the prior art.

[0005] This disclosure provides a method for controlling the power consumption of a GPU box, including:

[0006] The received tasks to be processed are subjected to feature analysis to identify task characteristic information, including task type.

[0007] Obtain historical performance data for all GPU boxes on the same node corresponding to the task type;

[0008] A GPU BOX performance model is constructed based on the real-time acquired status information of all GPU BOXes and the historical performance data.

[0009] The task characteristic information is analyzed based on the GPU BOX performance model to obtain the GPU BOX allocation strategy for executing tasks in the same node;

[0010] The target GPU BOX is determined according to the GPU BOX allocation strategy, and the task to be processed is executed based on the target GPU BOX;

[0011] The system monitors the operating information of the target GPU BOX in real time, obtains a dynamic power consumption control strategy for the GPU BOX based on the operating information, and executes the dynamic power consumption control strategy for the GPU BOX.

[0012] Secondly, this application discloses a power consumption control system for a GPU BOX, used to execute the power consumption control method for the GPU BOX, including:

[0013] The server is used to receive tasks to be processed, perform feature analysis on the tasks to be processed, and identify task characteristic information, including task type, computing requirements, memory usage, priority information, and estimated task execution time.

[0014] The historical performance data acquisition module is used to acquire historical performance data of all GPU BOXes on the same node corresponding to the task type;

[0015] The model building module is used to build a GPU BOX performance model based on the real-time acquired status information of all GPU BOXes and the historical performance data.

[0016] The allocation strategy acquisition module is used to analyze the task characteristic information based on the GPU BOX performance model and acquire the GPU BOX allocation strategy for executing tasks in the same node.

[0017] An execution module is used to determine the target GPU BOX according to the GPU BOX allocation strategy, and to execute the task to be processed based on the target GPU BOX;

[0018] The dynamic control module is used to monitor the operating information of the target GPU BOX in real time, obtain the GPU BOX power consumption dynamic control strategy based on the operating information, and execute the GPU BOX power consumption dynamic control strategy.

[0019] Thirdly, this disclosure also provides a computer device, which adopts the following technical solution:

[0020] The computer device includes:

[0021] At least one processor; and,

[0022] A memory communicatively connected to the at least one processor; wherein,

[0023] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform any of the above-described power consumption control methods for the GPU BOX.

[0024] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium storing computer instructions for causing a computer to execute any of the above-described power consumption control methods for a GPU BOX.

[0025] Fifthly, embodiments of this disclosure also provide a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of any of the methods described above.

[0026] The GPU BOX energy consumption control method disclosed in this application enables real-time control of the energy consumption of each GPU. By analyzing task characteristic information and constructing a GPU BOX performance model, tasks can be rationally allocated to suitable GPU BOXes, fully leveraging the performance advantages of each GPU BOX, avoiding resource waste and task backlog, thereby improving overall task execution efficiency. Real-time monitoring of the target GPU BOX's operating information and dynamic adjustment of energy consumption based on operating conditions prevents excessive energy consumption by GPU BOXes under unnecessary circumstances, reducing data center operating costs and meeting energy conservation and emission reduction requirements. Reasonable task allocation and energy consumption control strategies ensure full utilization of all GPU BOXes within the same node, preventing some GPU BOXes from being overloaded while others remain idle, thus improving resource utilization. Real-time monitoring and dynamic control of GPU BOXes allow for timely detection and resolution of problems during GPU BOX operation, such as overheating and overload, ensuring stable system operation. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart illustrating the power consumption control method for a GPU BOX provided in an embodiment of this disclosure.

[0029] Figure 2This is a flowchart illustrating a method for obtaining historical performance data of all GPU BOXes on the same node corresponding to a task type, as provided in an embodiment of this disclosure.

[0030] Figure 3 This is a flowchart illustrating the method for constructing a GPU BOX performance model provided in this embodiment of the disclosure.

[0031] Figure 4 This is a flowchart illustrating the method for obtaining the GPU BOX allocation strategy provided in an embodiment of this disclosure.

[0032] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure. Detailed Implementation

[0033] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0034] Reference Figure 1 The first aspect of this application discloses a method for controlling the power consumption of a GPU BOX, comprising:

[0035] S100 receives tasks to be processed.

[0036] Specifically, the server receives tasks to be processed through a network interface, which can use common network protocols such as HTTP and TCP. For example, in a distributed computing system, the client sends a task request to a specific port of the server. The server uses the corresponding network programming library (such as Python's socket library or the Flask framework) to listen on that port. When it receives the task request, it parses it into the format of the task to be processed.

[0037] S200 performs feature analysis on the task to be processed and identifies task characteristic information.

[0038] The task characteristic information includes task type, computational requirements, memory usage, priority information, and estimated task execution time.

[0039] The priority information includes high priority (such as real-time tasks) and low priority (such as batch processing tasks).

[0040] By identifying task characteristic information, we can gain a deeper understanding of the task's needs and characteristics, providing a basis for selecting a suitable GPU box and formulating energy consumption control strategies. Different types of tasks, different computing needs, and memory usage place different performance requirements on the GPU box, and accurate feature analysis helps improve the efficiency and accuracy of task execution.

[0041] Furthermore, task type identification can be performed by analyzing the task's file extension, description, or execution command. Computational requirements analysis can estimate computational needs based on factors such as the task's code structure and input data size. Memory usage estimation can be performed by analyzing variables and data structures involved in the task's code. Priority information determination can be based on factors such as the task submitter and its urgency. Runtime estimation can be performed by combining historical task runtime data with current task characteristics using machine learning algorithms (such as linear regression) to predict the task's runtime.

[0042] S300 retrieves historical performance data for all GPU boxes on the same node corresponding to the task type.

[0043] During the execution of tasks on each GPU BOX, performance data for each GPU BOX, including computation speed, memory bandwidth, temperature, and power consumption, is recorded and stored in a database. Depending on the task type, historical performance data for all GPU BOXes on the same node can be retrieved from the database using SQL statements or a database query interface.

[0044] Historical performance data can reflect the actual performance of each GPU box when handling specific types of tasks, providing a reference for building performance models. By analyzing historical data, we can understand the strengths and weaknesses of each GPU box, thus making more reasonable decisions when allocating tasks.

[0045] S400 constructs a GPU BOX performance model based on the real-time acquired status information and historical performance data of all GPU BOXes.

[0046] By constructing the model, we can understand the performance of each GPU BOX in advance, avoiding resource waste or low task execution efficiency caused by blindly allocating tasks.

[0047] Specifically, intelligent energy management software can be deployed on the server to synchronize historical data and collect real-time status of energy management agents of each GPU server, store them in time-series database, and synchronize them with the intelligent energy management software.

[0048] The S500 analyzes task characteristic information based on the GPU BOX performance model to obtain the GPU BOX allocation strategy for executing tasks on the same node.

[0049] A reasonable task allocation strategy can fully utilize the performance advantages of each GPU box, improve the overall task execution efficiency, and avoid the situation where some GPU boxes are overloaded while others are idle due to arbitrary task allocation, thereby improving resource utilization.

[0050] S600 determines the target GPU BOX based on the GPU BOX allocation strategy and executes the pending task based on the target GPU BOX.

[0051] This step clarifies the specific device on which the task will be executed, ensuring that the task can be executed smoothly according to the allocation strategy; matching the task with the appropriate GPU BOX improves the accuracy and efficiency of task execution.

[0052] The S700 monitors the operating information of the target GPU BOX in real time, obtains the GPU BOX power consumption dynamic control strategy based on the operating information, and executes the GPU BOX power consumption dynamic control strategy.

[0053] By monitoring and dynamically adjusting in real time, the power consumption can be flexibly adjusted according to the actual operation of the GPU BOX, avoiding the situation where the GPU BOX still consumes a lot of power when under low load, thereby reducing overall power consumption and improving energy utilization efficiency.

[0054] The GPU BOX energy consumption control method disclosed in this application enables real-time control of the energy consumption of each GPU. By analyzing task characteristic information and constructing a GPU BOX performance model, tasks can be rationally allocated to suitable GPU BOXes, fully leveraging the performance advantages of each GPU BOX, avoiding resource waste and task backlog, thereby improving overall task execution efficiency. Real-time monitoring of the target GPU BOX's operating information and dynamic adjustment of energy consumption based on operating conditions prevents excessive energy consumption by GPU BOXes under unnecessary circumstances, reducing data center operating costs and meeting energy conservation and emission reduction requirements. Reasonable task allocation and energy consumption control strategies ensure full utilization of all GPU BOXes within the same node, preventing some GPU BOXes from being overloaded while others remain idle, thus improving resource utilization. Real-time monitoring and dynamic control of GPU BOXes allow for timely detection and resolution of problems during GPU BOX operation, such as overheating and overload, ensuring stable system operation.

[0055] Reference Figure 2 The method for obtaining historical performance data of all GPU boxes on the same node corresponding to a task type in S300 specifically includes:

[0056] S310 acquires performance data from each GPU BOX during the execution of historical tasks and stores it in the target database.

[0057] The performance data includes computing speed, memory bandwidth usage, temperature, power consumption, load requirements, and historical execution time for different tasks.

[0058] For example, in a large data center, multiple GPU boxes are used for different tasks, such as deep learning training, video rendering, and scientific computing. For each GPU box, a corresponding monitoring system collects performance data in real time as it executes historical tasks. For instance, when a GPU box performs a deep learning training task, the monitoring system records its computation speed, i.e., the number of floating-point operations per second (e.g., 10 trillion floating-point operations per second); memory bandwidth usage information, such as the amount of data read from and written to memory per second; temperature information, obtained through temperature sensors to measure the GPU's real-time temperature; power consumption information, measured using a power meter; load requirements information, such as the percentage of GPU cores utilized by the task; and historical execution duration, the time spent from the start to the end of the task. This collected performance data is organized into a specific format and then stored in a target database. The target database can be a relational database, storing data according to fields such as GPU box number, task type, and timestamp, facilitating subsequent querying and management.

[0059] Recording performance data helps to comprehensively understand the actual performance of each GPU box under different tasks. By analyzing this data, we can discover the strengths and weaknesses of a GPU box. For example, a GPU box may have high computation speed but high power consumption when processing a certain type of task, which helps to make more reasonable decisions in subsequent task allocation. Storing data in a target database facilitates long-term data preservation and management. The structured storage method of the database makes it easy to query, statistically analyze, and perform data analysis, laying the foundation for subsequent queries of relevant performance data based on task type.

[0060] S320 retrieves performance data from the target database for all GPUBOXs on the same node that have executed the same task type, based on the task type.

[0061] This step allows for the targeted acquisition of performance data relevant to the current task type, providing accurate references for subsequent task allocation and energy consumption control. By comparing the performance of different GPU boxes on the same node under this task type, the most suitable GPU box for executing the current task can be selected, improving task execution efficiency, effectively avoiding interference from irrelevant data, reducing the workload of data processing, and querying only data relevant to the current task type, thereby improving the efficiency and accuracy of data acquisition.

[0062] Reference Figure 3The method for S400 to "build a GPU BOX performance model based on the real-time acquired status information and historical performance data of all GPU BOXes," specifically the method for building the GPU BOX performance model, includes:

[0063] S410 can obtain the status information of all GPU boxes in real time.

[0064] The status information includes the current load, current temperature, current power consumption, and current available memory for each GPU BOX.

[0065] Specifically, GPU BOX management tools (such as NVIDIA SMI and AMD ROCm SMI) can be used to obtain the status information of all GPU BOXes in real time.

[0066] Furthermore, in a data center containing multiple GPU boxes, real-time status information can be obtained through specialized monitoring software and hardware sensors. For example, NVIDIA's management tools (such as NVIDIA SMI) can be used to obtain the current load of the GPU box; temperature sensors can measure the current temperature; power sensors can monitor the current power consumption in real time; and system software can query the currently available memory. Real-time status information reflects the current operating status of the GPU box, providing the latest and most accurate data foundation for subsequent assessment of its availability and construction of performance models; only based on real-time data can decisions be made that conform to the current actual situation, avoiding unreasonable task allocation due to the use of outdated information.

[0067] S420 determines the availability assessment results for the current load, current temperature, current power consumption, and current available memory.

[0068] Evaluating each key metric individually provides a clear understanding of the GPU Box's availability across various aspects. This helps identify potential problems with the GPU Box, such as excessive load or overheating, providing detailed information for subsequent decisions and preventing tasks from being affected by a metric exceeding a reasonable range.

[0069] S430 determines preset load weights, preset temperature weights, preset power consumption weights, and preset memory weights based on historical performance data from all GPU boxes.

[0070] Determining weights based on historical performance data allows for a more scientific consideration of the impact of various metrics when comprehensively evaluating GPU BOX availability. Different tasks may have different sensitivities to different metrics, and reasonable weight settings can more accurately reflect the actual performance of the GPU BOX, providing a more reasonable basis for task allocation.

[0071] Specifically, the performance of all GPU boxes in historical tasks was analyzed. For example, in a large number of past tasks, it was found that the load had a greater impact on the task execution speed, while the temperature had a relatively smaller impact on the task execution speed. Through statistical analysis and empirical judgment, for example, the preset load weight can be determined to be 0.5, the preset temperature weight to be 0.1, the preset power consumption weight to be 0.2, and the preset memory weight to be 0.2. These weights represent the importance of each indicator in evaluating the performance of the GPU box.

[0072] When all availability assessment results are passed, the S440 calculates a weighted sum of the current load, current temperature, current power consumption, and current available memory based on preset load weight, preset temperature weight, preset power consumption weight, and preset memory weight to obtain the availability score of a single GPU BOX.

[0073] The availability score for a single GPU box is calculated as follows: Preset load weight × Current load + Preset temperature weight × Current temperature + Preset power consumption weight × Current power consumption + Preset memory weight × Currently available memory. This weighted summation of availability scores comprehensively considers the impact of various metrics, transforming multiple performance indicators of a GPU box into a single value. This facilitates comparison and ranking of different GPU boxes. During task allocation, GPU boxes with higher availability scores can be prioritized based on their availability scores, improving task execution efficiency.

[0074] S450 constructs a GPUBOX performance model based on the availability scores of all GPU BOXes and the historical execution time of all GPU BOXes.

[0075] Specifically, the availability ratings of all GPU BOXes are sorted in descending order. The sorted list is the GPU BOX performance model. The higher the GPU BOX's rating, the higher it appears in the list, meaning that its performance is relatively better.

[0076] Alternatively, determine the number of clusters; cluster all GPU BOX availability scores according to preset rules to build a cluster-based GPU BOX performance model; determining the number of clusters includes: dividing GPU BOX into three categories: high performance, medium performance, and low performance.

[0077] Alternatively, machine learning algorithms (such as linear regression) can be used to construct a performance model with availability score as the independent variable and historical execution time as the dependent variable. Assuming that the model is trained on data from multiple GPU boxes, a linear regression equation can be obtained: Historical execution time = a × Availability score + b, where a and b are model parameters. This model can predict the approximate execution time of a GPU box for the current task based on its availability score. The constructed performance model can quantitatively predict the performance of GPU boxes, providing a more scientific basis for task allocation and scheduling. During task allocation, the GPU box with the shortest execution time can be selected based on the model's prediction, improving overall task execution efficiency. Simultaneously, this model can also be used for dynamic adjustments during task execution, optimizing task allocation strategies according to actual conditions.

[0078] Methods for determining the availability assessment results of the current load, current temperature, current power consumption, and current available memory for the S420 include:

[0079] A100: Determine if the current load is lower than the preset load threshold. If yes, determine that the availability assessment result of the current load is passed; otherwise, determine that the availability assessment result of the current load is failed.

[0080] A200 determines whether the current temperature is lower than the preset temperature threshold. If yes, the availability assessment result of the current temperature is determined to be passed; otherwise, the availability assessment result of the current temperature is determined to be failed.

[0081] A300 determines whether the current power consumption is lower than the preset power consumption threshold. If yes, the availability assessment result of the current power consumption is determined to be passed; otherwise, the availability assessment result of the current power consumption is determined to be failed.

[0082] A400 determines whether the current available memory is higher than the preset memory threshold. If yes, the availability assessment result of the current available memory is determined to be passed; otherwise, the availability assessment result of the current available memory is determined to be failed.

[0083] Availability assessments of four key metrics—current load, temperature, power consumption, and available memory—can promptly identify potential issues with GPU boxes, preventing tasks from being assigned to poorly performing GPU boxes and ensuring stable system operation. This reduces task failures due to hardware malfunctions or insufficient resources. Selecting GPU boxes with good performance and sufficient resources for task execution fully utilizes each GPU box's performance and resources, avoiding resource waste and improving overall system resource utilization efficiency. Prioritizing lower-power GPU boxes for tasks, while ensuring system performance, helps reduce energy consumption and cooling costs, thereby lowering data center operating costs. Based on the availability assessment results for each GPU box, more scientific and reasonable task allocation strategies can be developed, accurately assigning tasks to the most suitable GPU boxes, improving task execution efficiency and quality.

[0084] Reference Figure 4 The method for S500 to "analyze task characteristic information based on the GPU BOX performance model and obtain the GPU BOX allocation strategy for executing tasks on the same node" includes the following methods for obtaining the GPU BOX allocation strategy:

[0085] S510, based on the GPU BOX performance model, identifies GPU BOXes whose historical execution time is no higher than the estimated task execution time and whose availability score is no lower than the preset score threshold, and records them as qualified GPU BOXes.

[0086] This step can quickly filter out GPU boxes that meet the basic requirements, narrowing the search scope for subsequent allocation strategies, avoiding assigning tasks to GPU boxes with excessively long historical execution times or low availability scores, and improving the quality and efficiency of task allocation.

[0087] S520 encodes the task allocation for each qualifying GPU BOX as a chromosome.

[0088] Suppose we have three qualified GPU boxes (A, B, and C), each with two task allocation states: assigned a task (represented by 1) and not assigned a task (represented by 0). The possible chromosome encodings are as follows: [1, 0, 1] indicates that the task is assigned to GPU boxes A and C, but not to B; [0, 1, 0] indicates that the task is assigned to GPU box B, but not to A and C. Encoding the task allocation as chromosomes provides a suitable data structure for subsequent optimization using genetic algorithms. Genetic algorithms can easily manipulate chromosomes to simulate biological evolution and find the optimal solution.

[0089] S530, determine the fitness function used to evaluate the merits of each chromosome, the fitness function being the reciprocal of the estimated task execution time.

[0090] For a given chromosome and its corresponding task allocation scheme, if the GPU BOX performance model predicts that the task execution time under this scheme will be 8 hours, then the fitness value of this chromosome is 1 / 8 = 0.125. The fitness function provides the genetic algorithm with a standard for evaluating the merits of each task allocation scheme. The reciprocal of the estimated task execution time is used as the fitness function, ensuring that schemes with shorter execution times have higher fitness. This guides the genetic algorithm to evolve towards finding the shortest execution time, thereby finding the optimal task allocation strategy.

[0091] S540: Based on the value of the fitness function, select chromosomes with fitness greater than the preset fitness threshold for crossover and mutation operations to generate new chromosomes.

[0092] Assuming a preset fitness threshold of 0.1, the current chromosomes [1, 0, 1] have a fitness of 0.125, and [0, 1, 0] have a fitness of 0.08. Chromosome [1, 0, 1] with a fitness greater than 0.1 is selected for crossover and mutation operations. Crossover can involve exchanging some genes between [1, 0, 1] and another chromosome, for example, crossing with [0, 1, 0] to obtain a new chromosome [1, 1, 1]. Mutation can involve randomly changing a gene in a chromosome, such as changing the second gene in [1, 0, 1] from 0 to 1, resulting in [1, 1, 1]. Crossover and mutation operations simulate the genetic and mutation processes in biological evolution. By manipulating chromosomes with high fitness, new chromosomes are generated, increasing the diversity of the search space and helping to find better task allocation schemes.

[0093] S550, based on a new chromosome, repeatedly performs chromosome selection, crossover, and mutation operations until the maximum number of iterations is reached or the fitness value converges, until the optimal chromosome is found.

[0094] The difference between the optimal fitness values ​​of two adjacent generations is calculated. If the difference is less than a preset threshold, the fitness value is considered to have converged.

[0095] Through multiple iterations and evolutions, the task allocation scheme is continuously optimized to increase the probability of finding the optimal solution. The setting of the maximum number of iterations and the convergence condition of the fitness value can prevent the algorithm from falling into an infinite loop and ensure that the algorithm terminates within a reasonable time.

[0096] S560 obtains the GPU BOX allocation strategy for task execution in the same node based on the optimal chromosome.

[0097] In the previous steps, we encoded the task allocation of each qualified GPU BOX as a chromosome. Specifically, a chromosome is a binary list, where each element corresponds to a qualified GPU BOX. A value of 0 indicates that the GPU BOX does not participate in task execution, while a value of 1 indicates that the GPU BOX participates in task execution. When the optimal chromosome is found through a genetic algorithm, this chromosome represents an optimal GPU BOX allocation scheme. We only need to iterate through the optimal chromosome and select the qualified GPU BOXes corresponding to the elements with a value of 1; these GPU BOXes are the optimal GPU BOXes we need.

[0098] Assuming the final optimal chromosome is [1, 0, 1], it represents allocating tasks to GPU boxes A and C, but not to B. This is the GPU box allocation strategy for task execution within the same node. After optimization by the genetic algorithm, the task allocation strategy corresponding to the final optimal chromosome is the solution that minimizes task execution time while meeting basic requirements, effectively improving task execution efficiency and achieving optimal resource allocation.

[0099] In this embodiment, a genetic algorithm is used to optimize the task allocation scheme and find the GPU BOX allocation strategy that minimizes task execution time, which can significantly improve task execution efficiency and reduce the overall task execution time. The optimal allocation scheme is found among many qualified GPU BOXes to make full use of the resources of each GPU BOX, avoid resource waste, and improve resource utilization.

[0100] The method for S700 to "monitor the operating information of the target GPU BOX in real time, obtain the dynamic power consumption control strategy of the GPU BOX based on the operating information, and execute the dynamic power consumption control strategy of the GPU BOX" includes:

[0101] S710 monitors the target GPU BOX's operating information in real time.

[0102] The operational information includes the actual computational load, actual temperature information, and actual power consumption information when executing the task to be processed.

[0103] Real-time acquisition of GPU BOX operating information is the foundation for subsequent energy consumption control. Computational load reflects the current workload of the GPU BOX, while temperature and power consumption are closely related to the performance and energy consumption of the GPU BOX. By monitoring this information in real time, we can understand the status of the GPU BOX in a timely manner and provide accurate data support for subsequent energy consumption control decisions.

[0104] S720: Obtain the GPU BOX corresponding to the actual computing load below the lower load threshold, and denot it as the GPU BOX to be adjusted. Obtain the actual low-load running time of each GPU BOX to be adjusted, which is continuously running at a load below the lower load threshold.

[0105] The lower limit of the load is 15% of the rated load of the corresponding GPU BOX.

[0106] Screening out GPU boxes running under low load can help us focus on devices that may be wasting power; the actual time spent running under low load is an important basis for determining whether power consumption regulation is needed and what regulation strategy to adopt; if a GPU box is in a low-load running state for a long time, then it is necessary to regulate its power consumption to reduce unnecessary power consumption.

[0107] The S730 determines the corresponding GPU BOX power consumption dynamic control strategy based on the actual low-load running time, and performs GPU BOX power consumption control according to the GPU BOX power consumption dynamic control strategy.

[0108] By developing different energy consumption control strategies based on the actual low-load running time, precise energy consumption management can be achieved. Different low-load running times reflect different usage states of the GPU BOX. Targeted control strategies can minimize energy consumption and improve energy utilization efficiency while ensuring the normal operation of the GPU BOX.

[0109] Furthermore, the method for S730 to "determine the corresponding GPU BOX power consumption dynamic adjustment strategy based on the actual low-load running time, and adjust the GPU BOX power consumption according to the GPU BOX power consumption dynamic adjustment strategy" specifically includes:

[0110] S731, determine if the total number of GPU BOXes to be adjusted is greater than N / 2. If so, determine if the average load of all target GPU BOXes is lower than the load lower limit threshold. If so, randomly select... If the target GPU box is put into sleep mode, otherwise, a random selection will be made. The GPU BOX to be adjusted is put into hibernation.

[0111] in, ; N is the total number of all target GPU boxes. This represents the total number of all GPU boxes to be adjusted.

[0112] When there are many GPU boxes to be adjusted, the average load of all target GPU boxes can be judged to gain a more comprehensive understanding of the overall system's operating status. If the average load is also low, it means that the entire system is in a low-load state. Randomly selecting a target GPU box to hibernate can quickly reduce the system's power consumption. If the average load is not low, only randomly selecting the GPU boxes to be adjusted to hibernate can accurately target low-load devices, avoid affecting the normal operation of high-load devices, and reduce power consumption while ensuring system performance.

[0113] S732, if the total number of GPU BOXes to be adjusted is no greater than N / 2, adjust the power consumption of all GPU BOXes to 5%-13% of the rated load of the corresponding GPU BOX.

[0114] Furthermore, it is preferable to adjust the power consumption of all GPU BOXes to be adjusted to 10% of the rated load of the corresponding GPU BOX.

[0115] When there are a few GPU boxes to be adjusted, adjusting the power consumption of these GPU boxes to a low level can effectively reduce the power consumption of these low-load devices without affecting the overall system performance. Choosing the range of 5%-13% can ensure that the GPU boxes maintain a basic operating state so that they can quickly resume high-load operation when needed, while also achieving a significant reduction in power consumption.

[0116] In this embodiment, different energy consumption control strategies are adopted based on the number of GPU boxes to be adjusted and the average system load. This allows for precise energy consumption optimization for different system states, avoiding performance loss or energy waste that may result from a one-size-fits-all approach. By hibernating low-load GPU boxes or reducing their power consumption, unnecessary energy consumption is reduced, improving the overall energy efficiency of the data center and lowering operating costs. The overall system operating status and performance requirements are fully considered during energy consumption control. Randomly selecting hibernation devices or adjusting power consumption will not significantly impact the normal operation of the system, ensuring system stability and reliability. This solution can dynamically adjust energy consumption strategies based on real-time monitoring of GPU box operating information, adapting to changes in system load and ensuring the system always operates in a highly efficient and energy-saving state.

[0117] Furthermore, when all target GPU boxes in the same node are divided into several groups, each group of target GPU boxes is denoted as a subgroup, and N is the total number of all target GPU boxes in each subgroup. The total number of all GPU boxes to be adjusted in each subgroup;

[0118] The operating mode in which the total number of GPU BOXes to be adjusted is greater than N / 2 and the average load of all target GPU BOXes is lower than the lower load threshold is defined as the target mode.

[0119] Identify all subgroups containing the target pattern, and denote them as the target group;

[0120] When at least two target groups exist, the load is migrated to one target group for processing, and all GPU boxes in the target group without tasks are put to sleep.

[0121] Group management allows for more granular management of GPU boxes. Different subgroups may have different working characteristics and load conditions. Grouping allows for more precise energy consumption control and task allocation based on the actual situation of each subgroup, improving management efficiency. Defining target modes helps us quickly identify which subgroups have significant energy consumption optimization potential. When most GPU boxes in a subgroup are under low load, it indicates that the subgroup may be wasting energy; defining it as a target mode facilitates centralized processing later. Clearly defining target groups allows us to focus on those subgroups that require intensive energy consumption control, avoiding unnecessary operations on all subgroups and improving the targeting and efficiency of energy consumption control. Through load migration and hibernation operations, resources can be further integrated, reducing the number of GPU boxes under low load, lowering the overall system energy consumption, and concentrating tasks on a subgroup for processing. This also improves the resource utilization of that subgroup and enhances the overall system performance.

[0122] In this embodiment, by centralizing the processing of tasks in low-load subgroups and putting idle GPU boxes to sleep, unnecessary energy consumption is reduced, significantly lowering the overall energy consumption of the data center and saving operating costs. Load migration concentrates tasks in certain subgroups, improving the resource utilization of GPU boxes within these subgroups, avoiding resource dispersion and waste, and enhancing the overall efficiency of the system. Grouping and managing GPU boxes on the same node and centrally processing them for target groups simplifies the energy consumption control and task management process, reducing management complexity. This solution can dynamically adjust task allocation and energy consumption strategies according to the actual operating status of subgroups, enabling the system to better adapt to different load changes and enhancing the system's flexibility and adaptability.

[0123] This application includes real-time monitoring of the target GPU BOX's operating information, obtaining a dynamic GPU power consumption control strategy based on the operating information, and executing the dynamic GPU power consumption control strategy. It also includes:

[0124] B100 retrieves the historical computing power demand of the same node during off-peak hours and the computing power demand of the same node during peak hours for the same task type.

[0125] In this context, "same node" refers to a single server.

[0126] Suppose we have a server node used for machine learning training tasks. Through the server's logging system, we can obtain the historical computing power requirements of this node during off-peak hours (e.g., from midnight to 6 AM daily) over the past month. Statistical analysis shows that the average computing power requirement during off-peak hours is 1000 GFLOPS (floating-point operations per second). Similarly, by obtaining the computing power requirements of the same node during peak hours (e.g., from 10 AM to 4 PM daily), we find that the average computing power requirement during peak hours is 5000 GFLOPS. Understanding the computing power requirements during off-peak and peak hours provides fundamental data for developing energy consumption control strategies. Different task types have significantly different computing power requirements at different times. By accurately obtaining this data, we can more precisely adjust the GPU BOX's energy consumption based on actual needs, avoiding energy waste during low-demand periods while ensuring sufficient computing power supply during high-demand periods.

[0127] B200 determines the level of historical computing power demand during non-working hours.

[0128] when When the historical computing power demand during non-working hours is determined, it is classified as the first level; when At that time, the historical computing power demand during non-working hours is classified as the second level; among which, This represents the historical computing power demand of the same node during its off-peak hours. This refers to the computing power demand of the same node during peak periods.

[0129] Classifying computing power requirements by level makes the formulation of energy consumption control strategies more intuitive and convenient; different levels correspond to different energy consumption control strategies, so that the appropriate control method can be quickly determined based on the approximate range of computing power requirements, thereby improving decision-making efficiency.

[0130] B300 determines the power consumption control strategy for all GPU boxes in the same node based on their level, and performs dynamic power consumption control for all GPU boxes in the same node based on the power consumption control strategy.

[0131] Developing energy consumption control strategies based on computing power demand levels enables precise energy management. Reducing the power consumption of some GPU boxes during low-demand periods satisfies basic computing power requirements while significantly reducing energy consumption and saving costs. Conversely, ensuring all GPU boxes operate normally during high-demand periods guarantees that the system can provide sufficient computing power to process tasks and ensure normal business operations.

[0132] The methods disclosed in B100-B300 analyze and categorize computing power demands across different time periods, and adjust GPU BOX power consumption accordingly. This avoids energy waste during off-peak or low-demand periods, effectively reducing overall server power consumption and operating costs. Dynamically adjusting GPU BOX operation based on actual computing power needs allows for more rational resource allocation and utilization. It fully utilizes the computing power of all GPU BOXes during peak periods and appropriately reduces power consumption of some GPU BOXes during off-peak periods, improving resource efficiency. While meeting computing power demands across different time periods, it ensures sufficient computing power for task processing during peak hours, preventing disruptions to normal business operations due to energy consumption adjustments, thus guaranteeing business continuity and stability. This solution dynamically adjusts based on historical computing power demands for different task types, enabling the server system to better adapt to different business scenarios and working periods, improving system flexibility and adaptability.

[0133] The B300 method of "determining the energy consumption control strategy for all GPU boxes in the same node based on their level, and dynamically controlling the energy consumption of all GPU boxes in the same node according to the energy consumption control strategy" specifically includes:

[0134] When the level is Level 1, the power consumption control strategy for all GPU BOXes in the same node is to disconnect the power supply of at least 1 / 3 of the GPU BOXes in the same node.

[0135] When the level is Level 2, the power consumption control strategy for all GPU BOXes in the same node is to disconnect the power supply of at least 2 / 3 of the GPU BOXes in the same node, or to randomly select two GPU BOXes to keep the power supply on.

[0136] Based on the power consumption control strategy of all GPU BOXes in the same node, the power supply switch of the corresponding GPU BOX is controlled through the I²C chip.

[0137] Under the first level of computing power requirement, it means that the current computing power requirement is relatively low. Disconnecting at least 1 / 3 of the GPU BOX power supply can directly reduce the number of GPU BOXes in operation, thereby significantly reducing the overall energy consumption of the node. At the same time, keeping some GPU BOXes running can also ensure that the system maintains a certain computing power to cope with the small number of tasks that may occur.

[0138] The second level of computing power requires even less. Further reducing the number of running GPU boxes—disconnecting the power supply to at least two-thirds of the GPU boxes or keeping only two GPU boxes powered—can lower energy consumption to an even lower level, achieving more efficient energy saving. Moreover, both methods offer flexibility, allowing the selection of a more suitable strategy based on actual conditions (such as performance differences between different GPU boxes, anticipated subsequent tasks, etc.). I²C chips are common for inter-device communication. In server nodes, the power supply switch of each GPU box is connected to the I²C bus. Once the energy consumption control strategy is determined, the server's control system can send control signals to the corresponding power supply switches via the I²C protocol. The I²C chip enables precise control of the power supply switch of each GPU box, accurately disconnecting or connecting the power to a specified GPU box according to the energy consumption control strategy, ensuring accurate energy consumption control.

[0139] The scheme disclosed in this embodiment precisely adjusts the operating state of the GPU BOX according to different computing power demand levels, which can minimize energy consumption and save electricity costs in different low computing power demand scenarios. It provides different energy consumption control strategies for different levels of computing power demand, and provides two optional methods in the second level, so that the system can flexibly adjust according to the actual situation and better adapt to various changes in computing power demand. The power supply switch is automatically controlled by the I²C chip, which reduces manual intervention, improves the efficiency and accuracy of energy consumption control, and also reduces the risk of errors that may be caused by human operation.

[0140] Furthermore, the power consumption control method for GPU BOX disclosed in this application also includes: when several target GPU BOXes executing tasks run continuously at no less than 75% of the rated load of the corresponding GPU for more than a preset high load running time threshold, other target GPU BOXes are started to execute the tasks.

[0141] For example, the assigned tasks are executed by the three selected target GPU boxes. When the three target GPU boxes continue to run at high load (75%) for more than a preset time (e.g., 30 seconds), the other two sleeping / low-power GPUs are launched through the VLLM inference framework to join in executing the unprocessed tasks.

[0142] When some GPU boxes run under high load for extended periods, their performance may be limited, and they may even overheat, affecting the overall system's task processing speed. Starting other GPU boxes to join the task execution can alleviate the workload of the high-load GPU boxes, preventing system performance bottlenecks and ensuring tasks are completed efficiently and stably. Increasing the number of running GPU boxes means the system can handle more computing tasks simultaneously, improving parallel computing capabilities. Tasks that were originally handled by a few GPU boxes can now be completed by more GPU boxes, significantly reducing task processing time and improving overall efficiency.

[0143] Prolonged high-load operation accelerates the aging of GPU boxes and increases the risk of hardware failure. By activating other GPU boxes to distribute the workload, the continuous high-load operation time of a single GPU box can be reduced, minimizing hardware wear and tear, extending the lifespan of the GPU boxes, and reducing hardware maintenance and replacement costs. High-load operation generates a significant amount of heat. If not cooled in time, this can lead to overheating and damage to the GPU box. Activating more GPU boxes to distribute the workload reduces the load on each GPU box, thus reducing the heat generated and helping to maintain the operating temperature of the GPU boxes within a safe range, protecting the hardware. In some cases, some GPU boxes may be in a dormant or low-power state, with underutilized resources. When existing GPU boxes are running under high load, activating these idle GPU boxes can fully utilize the system's computing resources and improve the overall system resource utilization.

[0144] This strategy can dynamically adjust the number of GPU boxes participating in task execution based on the actual workload of the task. When the task load increases, more GPU boxes are automatically launched to cope with it; when the task load decreases, some GPU boxes can be adjusted to low power or sleep mode, realizing dynamic optimization of resource allocation and making the system more flexible to adapt to different task requirements.

[0145] The following explanation uses a GPU server built with hot-swappable technology, which includes several GPU boxes. Taking a group of 5 GPU boxes as an example, this explanation will be provided.

[0146] Scenario 1 (Mostly Idle): When the average load of a group of GPU boxes is below 15% and continues to exceed the time threshold, the energy management agent can be invoked to control multiple GPU boxes in that group to hibernate, such as 3 or 4, which can save at least 3 * U²² / R watts of energy in real time. Scenario 2 (Inference Mode with Few Concurrent Sessions): When two GPU boxes in a group have high load, and the load of the other three is below 15% for an extended period, the energy management agent can be invoked to control two GPU boxes in that group to hibernate, which can save 2 * U²² / R watts of energy in real time. Scenario 3 (Multiple GPU Groups Mostly Idle): When Scenario 1 or 2 occurs in multiple GPU boxes on the same server, the load is migrated to one GPU box for processing, and all GPUs in the GPU box group without tasks are hibernated.

[0147] Scenario 4 (Mode with Fewer Inference Tasks): For inference use, normal usage periods will be similar to scenarios 1 and 2, requiring automatic power consumption control. Non-working periods (such as commuting hours, weekends, and holidays) can be set. By using the power management agent to collect data in real-time when the GPU server is idle, the above strategy can be applied to control multiple GPU groups on the server to only 1-2 GPUs active within 2-3 threshold time periods, significantly saving power. Scenario 5 (Fine-tuning and Pre-training Mode): If the server is set for fine-tuning or pre-training, it will generally be in high-occupancy mode. The idle time threshold can be set relatively long, and the idle time will also be relatively long after the task ends. The pre-processing strategy corresponding to scenario 4 can be used, modifying parameters such as the time threshold and the number of GPUs not in sleep mode according to actual needs. Scenario 6 (Low-Power Inference Mode): For usage scenarios with high response requirements, during GPU BOX wake-up and task reloading, if many GPUs are in sleep mode, there will be a slight delay or stuttering. The number of GPUs not in sleep mode can be increased. In this case, the idle GPUs not in sleep mode can be adjusted to low-power mode. Case 7 (Task Increase Mode): When the workload of the working GPU remains high and reaches the high load time threshold, the low power consumption of the same group will be adjusted to normal power consumption first to meet the computing power demand. If this is still not enough, the dormant GPUs will be gradually enabled to normal working mode.

[0148] Furthermore, this application also includes: setting consumption values ​​for actions such as hibernation scheduling, energy-saving scheduling, and wake-up scheduling. The higher the value, the higher the accuracy required for the corresponding action to effectively save energy. For example, hibernation scheduling outside the group involves cross-switching chip task migration, which is more expensive than within the group. Hibernation within the group is more expensive than low-power scheduling, but the higher the value, the better the energy-saving effect. Preprocessing is performed periodically based on the task patterns and energy consumption records of the existing control strategy. Preprocessing statistically analyzes time periods, task volume patterns within time periods, real-time energy consumption, overall time period energy consumption, and scheduling accuracy, generating statistical results and reports, which are then saved. The intelligent algorithm periodically performs statistical analysis based on the statistical results, analyzing the characteristics of historical tasks occupying peak tasks for extended periods, such as the task's corresponding model, the initiating IP, or the username, gradually reducing energy-saving scheduling that continuously occupies peak periods; analyzing the time period characteristics of tasks, analyzing the scheduling success rate and energy-saving effect of tasks with time period characteristics, achieving load prediction, and forming an optimized time-axis-based dynamic control strategy. For relatively regular and stable computing environments, after multiple recursive iterations of the above method, a relatively practical dynamic control strategy can be formed, achieving energy-saving effects far exceeding those achieved through manual intervention.

[0149] In this application, the GPU BOX includes an internal PCIe interconnect module, and a GPU BOX module, a control module, a storage module, and a network module, all of which are connected to the internal PCIe interconnect module via the PCIe protocol. The GPU BOX adopts a modular structure, which facilitates quick replacement and installation of the GPU and ensures physical stability and safety during the insertion and removal process.

[0150] Specifically, the GPU Box includes a housing with a recess for mounting the GPU module. A control motherboard (i.e., circuit board) and a network module are installed within the housing. The control motherboard integrates a control module (preferably an integrated system-on-a-chip), a storage module (preferably an NVMe SSD), and a PCIe interconnect module (preferably a PCIe switch chip). The GPU module, control module, storage module, and network module are all connected to the PCIe interconnect module via the Type I PCIe protocol. A P2P DMA-enabled channel is established between the GPU module and the storage module, allowing direct access between the storage device and the GPU module's video memory, supporting direct read / write access to the disk. Compared to traditional data access methods where data is first transferred from the disk to system memory and then processed and scheduled by the CPU to move the data to the GPU's video memory, the GPU Box disclosed in this embodiment provides a direct access method that significantly reduces data transmission latency and improves data processing efficiency. This solution bypasses the CPU, enabling direct communication between the GPU and storage device, greatly improving data transmission efficiency and speed.

[0151] The control motherboard features a PCIe slot for installing the GPU module, a hard drive interface for installing the storage module, and a backplane connection interface. A BP connector is mounted on the backplane connection interface; the network module connects to the BP connector, which is used to route the network signals from the network module to the OSFP interface on the server CPU control board. The PCIe slot is compatible with the PCIe interfaces of different GPU card models.

[0152] In this application, "all GPU BOXes in the same node" refers to all GPU BOXes belonging to the same CPU server, with each GPU BOX containing one independent GPU card. Each GPU BOX is independently configured, functioning like a standardized component. During server or computing system deployment, it simply needs to be installed onto the corresponding interface, eliminating the need for complex wiring and debugging. This plug-and-play feature significantly shortens system deployment time and improves efficiency. When computing demands increase, independent GPU BOXes can be easily added to enhance system computing power, whether increasing the number of GPUs within a single server or expanding within a cluster of multiple servers, without requiring large-scale modifications to the existing system. Because each GPU BOX is independently configured, a failure in one will not affect the normal operation of others. The system can quickly identify and isolate the faulty unit, continuing to utilize other functioning GPUs to complete computing tasks, thereby improving the overall system reliability and availability and reducing downtime due to hardware failures. Independently configured GPU BOXes also offer greater operability during maintenance. If a unit malfunctions, it can be directly removed from the system for repair or replacement without affecting other units or the normal operation of the entire system. This makes maintenance simpler and more efficient, reducing maintenance costs and complexity. Since each GPU is independent, its performance can be monitored and managed individually. Monitoring software can obtain real-time information such as the working status and performance metrics of each unit, allowing for timely detection of potential problems and adjustments and optimizations to ensure the entire system is always operating at its best.

[0153] For different GPU boxes, each GPU box on the same node is connected to an external PCIe interconnect module via the Type II PCIe protocol; the version of the Type II PCIe protocol is lower than the version of the Type I PCIe protocol. A GPUDirect P2P connection channel is established between GPUs in any pair of GPU boxes on the same node, enabling direct P2P DMA communication between GPUs using the PCIe bus. Specifically, the GPU module integrates a DMA controller connected to the GPU card, used to control data read / write operations on the GPU card and data interaction with the PCIe switch chip. The storage module integrates a DMA controller connected to the storage chip, used to control data read / write operations on the storage chip and data interaction with the PCIe switch chip. The channel supporting P2P DMA is the transmission channel between the DMA controller within the GPU module and the DMA controller within the storage module.

[0154] Each GPU box on different nodes (i.e., different CPU servers) is connected to an RDMA switch module via a Type 3 PCIe protocol. The version of the Type 3 PCIe protocol is consistent with the version of the Type 1 PCIe protocol. GPU modules in any pair of GPU boxes on different nodes establish GPUDirect RDMA (GDR) connection channels, enabling cross-node GPU interconnection through the RDMA switch module. The CPU (i.e., the server) is not on a shared board with the GPU boxes and is connected to them via a Type 4 PCIe protocol, with a version no higher than the Type 2 PCIe protocol. Specifically, the CPU unit manages the GPU modules, network modules, and storage modules via the PCIe bus. This non-shared-board structure eliminates dependence on PCIe interface switching devices and retimers.

[0155] The GPU BOX and CPU are connected via a hot-swappable interface. The GPU BOX is integrated within the server and connected via a hot-swappable interface based on a specific high-speed data transmission protocol. This interface provides high-speed, low-latency data transmission, enabling real-time data interaction and command transfer between the GPU BOX and CPU. Furthermore, the GPU BOX can be plugged in or unplugged at any time during normal server operation without causing electrical damage to the server system or the GPU BOX, ensuring system stability and data integrity. The interface design ensures stable data transmission and prevents signal interference and electrical problems during plugging and unplugging.

[0156] This application implements hot-swapping functionality for the GPU BOX from a technical control layer through structural design and hardware circuitry, supporting dynamic power management adjustments to improve system flexibility and energy efficiency, ensuring GPU replacement or management without shutting down the server. It controls GPU hibernation and wake-up via IIC, designing a corresponding state machine to ensure the GPU automatically enters a low-power mode during prolonged idle periods, thereby reducing energy consumption. Upon task resumption, a save and load mechanism restores the GPU's current state, ensuring seamless task continuation. A persistent storage solution is designed to save the current computational state and important data while the GPU is hibernating, enabling rapid resumption of work upon wake-up, ensuring the security and integrity of the state saving process, and preventing data loss and computational interruption.

[0157] Furthermore, it includes a hot-swap fault monitoring mechanism and a safety protection mechanism. The hot-swap fault monitoring mechanism is used to monitor faults during the hot-swap process, ensuring timely identification and alarms during GPU insertion and removal, such as abnormal temperature or poor connection. Through its self-diagnostic function, it can automatically detect and record fault information during hot-swap for subsequent analysis and improvement. Protection circuits, such as overcurrent protection and short-circuit protection, are incorporated into the circuit design to ensure the safety of the device during hot-swap operations. User access control and operation log recording are designed to ensure that only authorized users can perform hot-swap operations, avoiding system failures caused by misoperation.

[0158] In this application, from the perspective of the energy management agent layer, based on GPU hot-swappable drivers and services, all GPUs on the same server (i.e., the same node) can be managed and controlled. Furthermore, it allows users to set different energy management strategies according to specific application needs, including high-efficiency mode, performance mode, and power-saving mode, flexibly responding to different tasks and scenarios; it can automatically switch GPU working modes based on the current task load and user-defined strategies to achieve optimal energy efficiency; it can periodically evaluate the effectiveness of different energy management strategies, continuously optimizing strategy settings by analyzing energy consumption data and performance feedback, ensuring the system always maintains optimal performance under changing load conditions. An effective hibernation and wake-up mechanism can optimize GPU energy consumption in low-load or idle states, ensuring minimal energy consumption without affecting service quality. When a task is idle for a long time, the GPU can be automatically placed in a low-power state or hibernation state to reduce unnecessary energy consumption. The system determines whether the GPU can enter hibernation based on activity monitoring and task scheduling information. Before hibernation, the software saves the current task state and GPU configuration to memory or persistent storage for quick recovery upon wake-up; this mechanism can be implemented through IIC control to ensure rapid response. When a new task arrives or an existing task needs to continue, the system can automatically wake up the GPU and restore it to its previous working state, ensuring task continuity and user experience. Furthermore, by classifying and managing task priorities, critical tasks can be prioritized, thereby improving the overall efficiency and responsiveness of the system. Specifically, different tasks are assigned priorities based on factors such as urgency, importance, and resource requirements (e.g., high, medium, low priority). During task scheduling, high-priority tasks are given priority and allocated to GPUs with suitable workloads to ensure timely completion of critical tasks and optimize resource utilization. Based on real-time monitoring data and task execution status, the system can dynamically adjust task priorities. For example, when a high-priority task requires resources, its scheduling priority can be temporarily increased.

[0159] This application also provides in-depth insights into the operation of GPUs and the system through comprehensive status monitoring and logging, supporting troubleshooting, performance optimization, and decision support. It can continuously monitor the operating status of each GPU, including load, temperature, power consumption, and fault status, and feed this information back to the management system in real time. It can record all important events and status changes, including task initiation, completion, GPU status changes, and fault events. Logs should include timestamps, event types, and relevant parameters for subsequent analysis. Furthermore, by analyzing the recorded status and log data, it can generate periodic or on-demand reports to help administrators understand system operating efficiency, energy consumption, and potential problems, supporting optimization and adjustments.

[0160] In this application, from the perspective of the global intelligent energy management layer, energy efficiency optimization of each GPU BOX can be achieved through comprehensive scheduling and intelligent algorithms, thereby improving energy utilization efficiency and reducing operating costs. Specifically, this includes: 1) an energy consumption monitoring and analysis mechanism, used to monitor and analyze the energy consumption and status of the GPU cluster in real time, providing data support for decision-making, ensuring efficient system operation and optimizing resource utilization; 2) an intelligent task scheduling mechanism, used to optimize task scheduling on GPUs through intelligent algorithms, ensuring the high efficiency and stability of system operation, while improving resource utilization efficiency and reducing energy consumption; 3) a cross-node resource coordination mechanism, used to coordinate and optimize resources in a multi-node GPU server environment to improve overall energy efficiency and ensure the efficiency of each node. 4) User configuration and custom policy mechanism, which provides flexible user configuration options, enabling users to customize energy management policies according to their needs to meet specific application requirements and performance goals; 5) Fault detection and automatic recovery mechanism, which ensures system stability and reliability by monitoring GPU status in real time, responds quickly to faults and performs automatic recovery; 6) Reporting and visualization mechanism, which generates visualized energy consumption reports and monitoring dashboards to help users understand the system's energy efficiency performance for management decision-making; 7) Feedback mechanism, which evaluates the overall performance of the GPU cluster through comprehensive data and provides optimization suggestions to help users optimize resource utilization and energy efficiency.

[0161] For energy consumption monitoring and analysis mechanisms, the following methods can be adopted to effectively manage GPU energy consumption: First, real-time monitoring is implemented. Dedicated monitoring tools and sensors are used to periodically collect key indicators such as energy consumption, temperature, and load of each GPU, ensuring a sufficiently high data collection frequency, updating every second or minute to reflect the GPU status in real time. A distributed monitoring architecture is designed, using message queues or stream processing frameworks (such as Kafka or Flume) to transmit the collected data to the processing system in real time, ensuring effective data collection and aggregation in large clusters. Next is data storage and management. The collected monitoring data is stored in a time-series database (such as InfluxDB or Prometheus), and a reasonable storage strategy is designed to ensure long-term data preservation and fast access. Simultaneously, the collected data is cleaned and preprocessed to remove noise and outliers. Then, data analysis is conducted to automatically generate energy consumption reports regularly, including energy consumption trends, temperature fluctuations, load conditions, and key performance indicators such as average energy consumption, peak energy consumption, and GPU utilization for each GPU, ensuring the reports are easy to read and practical. Data visualization tools (such as Grafana or Tableau) are used to display the monitoring data, presenting information intuitively through charts and dashboards, and providing an interactive interface for users to analyze the performance of specific time periods or specific GPUs. Next, energy efficiency bottlenecks are identified by analyzing historical energy consumption data using data mining techniques. By comparing energy consumption and performance across different time periods or tasks, unreasonable resource allocation or configuration issues are identified. Improvement opportunities are then identified by analyzing the relationship between energy consumption trends and workload, proposing energy efficiency improvement suggestions under high load conditions, and using machine learning algorithms to predict future energy consumption trends to identify potential problems in advance. Finally, decision support is provided by offering optimization decision suggestions to management based on monitoring and analysis results, such as adjusting task scheduling strategies, reducing unnecessary GPU usage, and optimizing power management. A feedback mechanism is established to continuously monitor the effects of strategy adjustments, ensuring the effectiveness of measures and continuously optimizing energy management based on new data.

[0162] The intelligent task scheduling mechanism comprises four main parts: 1) Feature-based scheduling: First, task characteristics are analyzed. Feature analysis is performed on the tasks to be processed, identifying attributes such as computational requirements, memory usage, priority, and estimated runtime. Tasks are then classified into high-priority, real-time tasks, low-priority, and batch processing tasks. Simultaneously, GPU status is evaluated. Real-time monitoring of each GPU's load, temperature, power consumption, and available memory is conducted, and historical performance data is used to assess its performance in handling specific types of tasks, establishing a GPU performance model. Then, intelligent scheduling algorithms, such as those based on genetic algorithms, ant colony algorithms, or deep learning optimization algorithms, are designed and implemented. These algorithms intelligently select the best GPU for task allocation based on task characteristics and GPU status, while also considering task dependencies and adhering to task execution order and resource contention logic.

[0163] 2) Dynamic Strategy Adjustment: Continuous real-time data analysis tracks GPU load and power consumption, using real-time data stream analysis to assess utilization efficiency and energy efficiency. The system identifies high and low load periods through real-time feedback. Regarding strategy adjustment, GPU power is automatically increased during high load periods to improve performance and meet real-time requirements, while power is automatically reduced or idle GPUs are placed in power-saving mode during low load periods to reduce energy consumption. An interface is also provided for users to set dynamically adjustable strategy parameters, such as load thresholds and power limits, allowing for flexible adjustments to the scheduling strategy.

[0164] 3) Prediction and Planning: Historical data analysis involves collecting and analyzing historical load and energy consumption data. Statistical analysis methods are used to identify load patterns and trends. Time series analysis, regression analysis, and other techniques are applied to build load prediction models to predict future GPU load and energy consumption trends. Machine learning algorithms, such as linear regression, decision trees, or neural networks, are used to train historical data to generate load prediction models. Based on the model's prediction results, resource allocation and scheduling strategies are planned in advance. Based on the prediction results, GPU resources are reserved or adjusted in advance to cope with high load periods, and tasks are scheduled in advance during low load periods to optimize resource utilization.

[0165] 4) Scheduling Performance Evaluation: After each task scheduling, monitor the task execution performance metrics, including task completion time, energy consumption, and GPU load changes, evaluate the scheduling performance, and generate a feedback report. Combine the scheduling performance feedback with the intelligent scheduling algorithm to continuously optimize the scheduling strategy and algorithm, and regularly evaluate and update the scheduling algorithm to adapt to constantly changing workloads and system environments.

[0166] For cross-node resource coordination mechanisms, the following implementation methods can be adopted to achieve efficient coordination of GPU resources across nodes: 1) Global Resource View: Deploy resource monitoring tools to collect information such as GPU status, CPU load, memory usage, and network bandwidth of each node in real time, integrate the data to form a global resource view, and display the real-time status of all nodes through a visual interface. Design and implement a load balancing algorithm to dynamically adjust the task allocation of each node based on the data collected from the global view, and use strategies such as round-robin, minimum connection count, and load weight to ensure even task distribution and avoid node overload or idleness. Set resource usage thresholds for each node. When the node load exceeds the threshold, the system automatically allocates new tasks to low-load nodes, and continuously optimizes the threshold strategy through intelligent algorithms to adapt to different workloads and resource requirements.

[0167] 2) Task Migration: Continuously monitor the resource usage of each node, and combine this with task characteristics. Utilize decision trees or other intelligent algorithms for real-time data analysis to identify tasks requiring migration and assess the necessity and feasibility of the migration. Design effective task migration strategies to ensure data consistency and task integrity during the migration process, minimize the impact on system performance, and provide a bidirectional migration mode. Develop a task migration scheduling system to automatically handle task migration requests, including task pause, data transfer, and resumption of execution on the target node. Accurately record and manage task status, allowing users to track task progress in real time.

[0168] 3) Coordination Strategy Optimization: Collect and analyze historical task allocation data, evaluate task migration effectiveness and resource utilization, utilize machine learning techniques to identify optimal resource allocation and migration patterns under different loads, and optimize future scheduling and migration strategies. Provide a user feedback interface to allow users to evaluate migration strategies and results, and optimize system decision-making logic by analyzing user feedback, making task scheduling and migration more closely aligned with actual needs.

[0169] 4) Performance Monitoring and Evaluation: After each migration task, monitor the performance changes of nodes before and after the migration, such as task completion time, energy consumption, and load changes, and generate a migration effect evaluation report to help the team understand migration performance under different scenarios and further adjust and optimize strategies. Establish a regular evaluation and optimization mechanism to continuously improve cross-node resource coordination strategies and algorithms based on changes in system performance and user needs. Adopt agile development methods to quickly iterate system functions to cope with constantly changing workloads and market demands.

[0170] For user-configured and custom policy mechanisms, energy management can be implemented from two aspects: policy setting and custom policy saving and management. Regarding policy setting, the user interface should be designed first, creating a user-friendly configuration interface with a simple wizard-style setup process, allowing users to intuitively set and adjust energy management policies and quickly get started. Users should also be allowed to set task priorities, providing multi-level priority options to support highly customized task processing strategies and ensure critical tasks receive sufficient resources. For time-based control, users can define policies for different time periods according to their needs, such as increasing GPU performance during peak hours and reducing energy consumption during off-peak hours. They can also use the scheduled task function to set periodically executed energy management policies to automatically adjust resource utilization. Furthermore, a power limit setting allows users to customize the power limit for each GPU to control overall energy consumption, and allows users to set task scheduling rules, such as the number of tasks executed in parallel and task migration conditions, to optimize resource allocation. Regarding custom policy saving and management, users can create and save multiple energy management policy templates, and a policy version management function is also provided, allowing users to view, restore, and manage different versions of policy configurations. It also provides a strategy testing function, allowing users to simulate the effect of the strategy in a non-production environment, verify the rationality and effectiveness of the settings, and adjust the strategy according to the test results to ensure that the strategy finally applied to the production environment is optimized.

[0171] To effectively address GPU failures and ensure stable system operation, the following implementation methods can be adopted for fault detection and automatic recovery mechanisms: 1) Fault Detection: Deploy advanced monitoring tools to monitor the GPU's temperature, load, power consumption, and operating status in real time. Set fault thresholds; once any indicator exceeds the preset threshold, the system automatically triggers the fault detection mechanism. Utilize machine learning algorithms to analyze historical data, establish a normal state baseline, thereby automatically identifying potential fault modes, and set up an alarm mechanism to immediately send alarm notifications to administrators and system administrators when a fault is detected.

[0172] 2) Fault Handling Mechanism: When a GPU failure is detected, the system automatically switches to an available backup GPU, ensuring that the task state is preserved during the switchover process and minimizing the impact on business operations. An intelligent decision-making system is developed to automatically execute fault recovery procedures based on real-time monitoring data and historical fault records, and to record the fault and recovery process in detail for subsequent analysis and improvement.

[0173] 3) Task Rescheduling: After a failure occurs, the system can automatically identify the affected tasks and intelligently select the most suitable healthy GPU for rescheduling based on task priority and resource availability. Simultaneously, it automatically generates a failure recovery report, recording the time, cause, and recovery process of the failure, providing a basis for subsequent failure analysis. A visual interface is also provided to facilitate real-time tracking of the failure and recovery process, enhancing system transparency.

[0174] To effectively manage and monitor GPU energy consumption, status, and performance through reporting and visualization mechanisms, the following methods can be employed: First, utilize visualization tools to design intuitive dashboards that display key indicators such as GPU utilization, temperature, and power consumption. This allows users to monitor system status in real time and supports user customization of dashboard layouts, selected display indicators, and chart types to suit individual needs. Data stream processing technology ensures real-time data updates and includes alarm functions to visually alert users to abnormal situations. Second, provide a periodic reporting function. The system can automatically generate energy consumption and performance reports at user-defined frequencies (e.g., daily, weekly, or monthly). These reports cover key performance indicators, trend analysis, and summaries of abnormal situations, and users can customize the report generation frequency and content. Furthermore, the system features report distribution capabilities, allowing users to send reports to specific team members or managers for collaborative analysis and decision-making. Reports can also be exported in various formats such as PDF and Excel for further analysis and archiving.

[0175] To optimize and improve system performance, the following methods can be used for the feedback mechanism: 1) Performance benchmarking: Establish a regular performance benchmarking plan to evaluate the system's energy efficiency, response time, and task success rate under different loads. Collect and analyze test data to generate detailed performance reports, providing a basis for subsequent optimization. Develop or integrate existing benchmarking tools to ensure that tests cover different types of tasks and loads, comprehensively evaluate system capabilities, and support user-defined test scenarios to simulate different workloads.

[0176] 2) Optimization Suggestion Generation: Based on monitoring data and performance evaluation results, data analysis and mining techniques are used to identify bottlenecks and resource waste issues in the system. Combined with task usage patterns, targeted resource allocation and scheduling optimization suggestions are generated. An intelligent optimization system is designed to automatically analyze historical data and generate suggestions including adjusting resource allocation, modifying energy consumption strategies, and task scheduling. Furthermore, simulation tools are provided to allow users to predict the effects before implementing the suggestions, thereby helping users make more informed decisions.

[0177] The implementation of this technical solution can significantly improve the energy management of GPUs under different task loads. Based on preliminary predictions and test results, the energy management effects after applying this solution are as follows: Actual tests show that after applying the intelligent scheduling algorithm, the overall energy consumption of the GPU is reduced by approximately 20%-30%. In continuous inference mode, energy consumption is expected to decrease further by more than 10%; by optimizing power management and reducing GPU power consumption under low load conditions, the lifespan of the GPU is expected to be extended by approximately 15%-25%; the dynamic plug-and-play method improves system resource utilization by approximately 25%. It ensures performance when high computing power is required, while effectively saving energy under low load conditions. These results demonstrate that this solution not only solves the shortcomings of existing technologies in energy management but also provides an efficient and flexible solution for intelligent GPU management, possessing high practical value and market prospects. Quantitative test analysis and practical application results verify the effectiveness and feasibility of this solution, laying the foundation for further optimization and promotion in the future.

[0178] The energy consumption control method for GPU BOX disclosed in this application comprehensively and systematically solves a series of problems existing in the prior art by addressing multiple aspects from task analysis and GPU allocation to real-time monitoring and control. Specifically, by performing feature analysis on the received tasks to be processed and identifying task characteristic information (including task type), the system can gain a deeper understanding of the nature and requirements of the tasks, avoiding blind allocation of GPU resources. For example, for some simple inference tasks, if GPU resources suitable for large model pre-training are used incorrectly, it will result in wasted resources and increased energy consumption. However, by accurately identifying task characteristics, GPUs can be allocated to execute tasks more rationally.

[0179] The system acquires historical performance data of all GPU boxes on the same node corresponding to the task type, and combines this with real-time status information of all GPU boxes to construct a GPU box performance model. This model comprehensively considers historical performance and current status, providing a more scientific basis for task allocation. During large model pre-training and large model fine-tuning, the system can accurately evaluate the performance and reliability of each GPU box based on the model, and prioritize selecting GPU boxes with stable performance and good status to execute tasks. This reduces the probability of the entire task failing due to GPU problems and needing to be restarted, thereby reducing the waste of computing resources and unnecessary energy consumption.

[0180] Based on the GPU BOX performance model, the system analyzes task characteristic information to determine the GPU BOX allocation strategy for task execution within the same node. Under low load conditions, the system dynamically allocates an appropriate number and performance of GPU BOXes according to the actual needs of the task, preventing all GPUs from maintaining a high-power consumption state. For example, when the server is handling a small number of tasks, the system can activate only some appropriately performing GPU BOXes to execute tasks, while allowing other GPUs to enter low-power mode, thereby effectively reducing overall energy consumption, improving energy efficiency, and lowering operational costs.

[0181] The system monitors the operating information of the target GPU BOX in real time, obtains and executes dynamic power consumption control strategies for the GPU BOX based on the operating information, and enables timely adjustment of the GPU's operating status, such as reducing its computing frequency and task load, when the GPU generates a large amount of heat under long-term high-load operation. This reduces heat generation, protects the internal electronic components of the GPU, improves its performance and stability, reduces the need for frequent replacements due to GPU aging, lowers hardware costs, and ensures the normal operation of the server.

[0182] This application's method breaks away from the existing simple, cumulative, manual control approach, enabling dynamic, on-demand activation and use of GPUs based on actual task requirements. It automatically allocates resources and regulates energy consumption according to task characteristics and GPU status, requiring no manual intervention and avoiding errors prone to occur in manual operations, thus improving management efficiency. Although energy control interfaces from different GPU manufacturers are not standardized, this application's method achieves unified management of GPU boxes from different manufacturers by constructing a universal GPU box performance model and dynamic adjustment strategy. The system focuses on the overall performance and energy consumption of the GPU box, rather than relying on specific manufacturer interfaces, thereby simplifying the management process, improving management flexibility and efficiency, and enabling timely adjustments to server energy consumption status based on task changes.

[0183] In the current AI field, the demand for applications primarily focused on fine-tuning and inference is experiencing explosive growth, leading to a significant increase in the frequency of GPU server usage and computational load. However, existing GPU server energy management suffers from numerous drawbacks, such as task failure due to GPU malfunctions, high energy consumption even under low load, GPU damage from prolonged high-load operation, and outdated energy management methods. These problems result in resource waste and increased costs. The solution disclosed in this application not only optimizes the hardware structure but also possesses intelligent software management capabilities. With this technology, multi-GPU BOX servers can automatically perform dynamic intelligent management and optimization of energy consumption under different task states. The application of this technology can significantly reduce the energy consumption of GPU BOX servers, improve energy efficiency, and thus reduce operating costs. Furthermore, it can effectively extend the lifespan of GPU BOX servers, which is of great significance for promoting the sustainable development of the AI ​​field.

[0184] Secondly, this application discloses a power consumption control system for a GPU BOX, used to execute the power consumption control method for the GPU BOX, including:

[0185] The server is used to receive tasks to be processed, perform feature analysis on the tasks to be processed, and identify task characteristic information, including task type, computing requirements, memory usage, priority information, and estimated task execution time.

[0186] The historical performance data acquisition module is used to acquire historical performance data of all GPU BOXes on the same node corresponding to the task type;

[0187] The model building module is used to build a GPU BOX performance model based on the real-time acquired status information and historical performance data of all GPU BOXes;

[0188] The allocation strategy acquisition module is used to analyze task characteristic information based on the GPU BOX performance model and obtain the GPU BOX allocation strategy used for executing tasks in the same node.

[0189] The execution module is used to determine the target GPU BOX according to the GPU BOX allocation strategy and to execute the task to be processed based on the target GPU BOX;

[0190] The dynamic control module is used to monitor the operating information of the target GPU BOX in real time, obtain the GPU BOX power consumption dynamic control strategy based on the operating information, and execute the GPU BOX power consumption dynamic control strategy.

[0191] A computer device according to embodiments of the present disclosure includes a memory and a processor. The memory is used to store non-transitory computer-readable instructions. Specifically, the memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc.

[0192] The processor may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the computer device to perform desired functions. In one embodiment of this disclosure, the processor is used to execute computer-readable instructions stored in the memory, causing the computer device to perform all or part of the steps of the power consumption control method for the GPU BOX of the foregoing embodiments of this disclosure.

[0193] Those skilled in the art will understand that, in order to solve the technical problem of how to achieve a good user experience, this embodiment may also include well-known structures such as communication buses and interfaces, and these well-known structures should also be included within the protection scope of this disclosure.

[0194] like Figure 5 This is a schematic diagram of a computer device provided for an embodiment of the present disclosure. It illustrates a structural schematic diagram suitable for implementing the computer device in the embodiments of the present disclosure. Figure 5 The computer device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0195] like Figure 5 As shown, a computer device may include a processor (such as a central processing unit, graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) or programs loaded from storage devices into random access memory (RAM). The RAM also stores various programs and data required for the operation of the computer device. The processor, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0196] Typically, the following devices can be connected to the I / O interface: input devices, such as sensors or visual information acquisition devices; output devices, such as displays; storage devices, such as magnetic tapes or hard drives; and communication devices. Communication devices allow the computer device to communicate wirelessly or wiredly with other devices (such as edge computing devices) to exchange data. Although Figure 5A computer apparatus with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or included alternatively.

[0197] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from ROM. When the computer program is executed by a processor, all or part of the steps of the GPU BOX power consumption control method of embodiments of this disclosure are performed.

[0198] A computer-readable storage medium according to embodiments of the present disclosure stores non-transitory computer-readable instructions thereon. When the non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the power consumption control methods for GPU BOX described in the foregoing embodiments of the present disclosure are performed.

[0199] The aforementioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or portable hard drive), media with built-in rewritable non-volatile memory (e.g., memory card), and media with built-in ROM (e.g., ROM cartridge).

[0200] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

Claims

1. A method for regulating energy consumption of a GPU BOX, characterized in that, The method comprises the following steps: characteristic analysis is performed on the received to-be-processed task to identify task characteristic information, wherein the task characteristic information comprises a task type; historical performance data of all GPU BOXs of the same node corresponding to the task type is acquired; a GPU BOX performance model is constructed according to real-time acquired state information of all GPU BOXs and the historical performance data; the task characteristic information is analyzed according to the GPU BOX performance model to acquire a GPU BOX allocation strategy for executing the task in the same node; a target GPU BOX is determined according to the GPU BOX allocation strategy, and the to-be-processed task is executed based on the target GPU BOX; real-time monitoring is performed on running information of the target GPU BOX, a GPU BOX energy consumption dynamic regulation and control strategy is acquired according to the running information, and the GPU BOX energy consumption dynamic regulation and control strategy is executed; the historical performance data of all GPU BOXs of the same node corresponding to the task type is acquired by acquiring performance data of each GPU BOX in a historical task execution process and storing the performance data into a target database; the performance data comprises calculation speed, memory bandwidth usage information, temperature information, power consumption information, load requirement information and historical execution time length corresponding to different tasks; the performance data of all GPU BOXs of the same node executing the task type is queried from the target database according to the task type; the GPU BOX performance model is constructed according to real-time acquired state information of all GPU BOXs and the historical performance data, which comprises the following steps: real-time acquisition is performed on state information of all GPU BOXs, wherein the state information comprises current load, current temperature, current power consumption and current available memory of each GPU BOX; availability evaluation results of the current load, the current temperature, the current power consumption and the current available memory are respectively determined; preset load weight, preset temperature weight, preset power consumption weight and preset memory weight are determined according to the historical performance data of all GPU BOXs; when all the availability evaluation results are passed, the current load, the current temperature, the current power consumption and the current available memory are weighted and summed according to the preset load weight, the preset temperature weight, the preset power consumption weight and the preset memory weight to obtain an availability score of a single GPU BOX; a GPU BOX performance model is constructed according to the availability scores of all GPU BOXs and the historical execution time lengths corresponding to all GPU BOXs.

2. The energy consumption regulating method of the GPU BOX according to claim 1, wherein, the availability evaluation results of the current load, the current temperature, the current power consumption and the current available memory are respectively determined, which comprises the following steps: it is judged whether the current load is lower than a preset load threshold; if yes, it is determined that the availability evaluation result of the current load is passed, and if no, it is determined that the availability evaluation result of the current load is not passed; determining whether the current temperature is lower than a preset temperature threshold, if yes, determining that the availability evaluation result of the current temperature is pass, if no, determining that the availability evaluation result of the current temperature is fail; determining whether the current power consumption is lower than a preset power consumption threshold, if yes, determining that the availability evaluation result of the current power consumption is pass, if no, determining that the availability evaluation result of the current power consumption is fail; determining whether the current available memory is higher than a preset memory threshold, if yes, determining that the availability evaluation result of the current available memory is pass, if no, determining that the availability evaluation result of the current available memory is fail.

3. The energy consumption regulating method of the GPU BOX according to claim 1, wherein, The analysis of the task characteristic information according to the GPU BOX performance model comprises: obtaining, according to the GPU BOX performance model, a GPU BOX whose historical execution duration is not higher than the estimated task execution time and whose availability score is not lower than a preset score threshold, and denoted as a qualified GPU BOX; encoding the task allocation of each qualified GPU BOX as a chromosome; determining a fitness function for evaluating the advantages and disadvantages of each chromosome, wherein the fitness function is the inverse of the estimated task execution time; selecting, according to the value of the fitness function, a chromosome with a fitness greater than a preset fitness threshold for crossover and mutation operations to generate a new chromosome; repeating the selection, crossover and mutation operations of the chromosome based on the new chromosome, and when the fitness value converges, finding an optimal chromosome until the optimal chromosome is found; obtaining the GPU BOX allocation strategy for executing the task in the same node according to the optimal chromosome.

4. The energy consumption regulating method of the GPU BOX according to claim 1, wherein, The real-time monitoring of the running information of the target GPU BOX, the obtaining of the GPU BOX energy consumption dynamic regulation strategy according to the running information, and the execution of the GPU BOX energy consumption dynamic regulation strategy comprise: real-time monitoring of the running information of the target GPU BOX; the running information comprises actual operation load, actual temperature information and actual power consumption information when the task is executed; obtaining a GPU BOX corresponding to the actual operation load lower than a lower limit threshold of the load, denoted as an adjusted GPU BOX, and obtaining the actual low-load running time of each adjusted GPU BOX; wherein the lower limit threshold of the load is 15% of the rated load of the corresponding GPU BOX; determining a corresponding GPU BOX energy consumption dynamic regulation strategy according to the actual low-load running time, and regulating the energy consumption of the GPU BOX according to the GPU BOX energy consumption dynamic regulation strategy.

5. The energy consumption regulating method of the GPU BOX according to claim 4, wherein, The determination of the corresponding GPU BOX energy consumption dynamic regulation strategy according to the actual low-load running time, and the regulation of the energy consumption of the GPU BOX according to the GPU BOX energy consumption dynamic regulation strategy comprise: determining whether the total number of the to-be-adjusted GPU BOXs is greater than N / 2, if yes, determining whether the average load of all the target GPU BOXs is lower than a load lower threshold, if yes, randomly selecting one of the target GPU BOXs to sleep, if no, randomly selecting one of the to-be-adjusted GPU BOXs to sleep one of the to-be-adjusted GPU BOXs to sleep;​ wherein, ; N is the total number of all the target GPU BOXs, is the total number of all the GPU BOXs to be adjusted. if the total number of the adjusted GPU BOXs is not greater than N / 2, adjusting the power consumption of all the adjusted GPU BOXs to 5%-13% of the rated load of the corresponding GPU BOX.

6. The energy consumption regulating method of the GPU BOX according to claim 5, wherein, When all the target GPU BOXs in the same node are divided into several groups, the target GPU BOXs in each group are recorded as a subgroup, N is the total number of all the target GPU BOXs in each subgroup, is the total number of all the target GPU BOXs in each subgroup. The operation mode that the total number of the GPU BOXes to be adjusted is greater than N / 2 and the average load of all the target GPU BOXes is lower than the load lower threshold is recorded as a target mode; All the subgroups in which the target mode exists are determined and recorded as target groups; When there are at least two target groups, load is migrated to one target group for processing, and all GPU BOXes in the target group without tasks are hibernated.

7. The energy consumption regulating method of the GPU BOX according to claim 1, wherein, The real-time monitoring of the operation information of the target GPU BOX, the acquisition of the GPU energy consumption dynamic regulation strategy according to the operation information, and the execution of the GPU energy consumption dynamic regulation strategy further comprise: Acquiring the historical computing power demand quantity of the same node in the non-working time period and the computing power demand quantity of the same node in the peak period corresponding to the task type; Determining the level of the historical computing power demand quantity in the non-working time period; According to the level, determining the energy consumption regulation strategy of all the GPU BOXes in the same node, and performing energy consumption dynamic regulation of all the GPU BOXes in the same node according to the energy consumption regulation strategy.

8. The energy consumption regulating method of the GPU BOX according to claim 7, wherein, The determination of the level of the historical computing power demand quantity in the non-working time period comprises: When the historical computing power demand amount during the non-working time period is determined as a first level. When the historical computing power demand amount during the non-working time period is determined as a second level. wherein, is the historical computing power demand amount when the same node is in the non-working time period, is the computing power demand amount when the same node is in the peak period.

9. The energy consumption regulating method of the GPU BOX according to claim 8, wherein, According to the level, determining the energy consumption regulation strategy of all the GPU BOXes in the same node, and performing energy consumption dynamic regulation of all the GPU BOXes in the same node according to the energy consumption regulation strategy, comprising: When the level is the first level, the energy consumption regulation strategy of all the GPU BOXes in the same node is determined to be at least disconnecting the power supply of 1 / 3 of the GPU BOXes in the same node; When the level is the second level, the energy consumption regulation strategy of all the GPU BOXes in the same node comprises at least disconnecting the power supply of 2 / 3 of the GPU BOXes in the same node, or randomly selecting two GPU BOXes to keep powered on; According to the energy consumption regulation strategy of all the GPU BOXes in the same node, the power supply switch of the corresponding GPU BOX is controlled through an I2C chip. 10.The method of claim 9, wherein, Further comprising: When the several target GPU BOXes executing tasks all continuously run at not less than 75% of the corresponding GPU rated load for more than a preset high-load running time threshold, starting other target GPU BOXes to execute tasks.

11. An energy consumption regulation system of a GPU BOX, configured to perform the energy consumption regulation method of the GPU BOX according to any one of claims 1-10. Comprise: A server is configured to receive a task to be processed, perform feature analysis on the task to be processed, and identify task characteristic information, wherein the task characteristic information comprises task type, computing demand, memory occupation, priority information, and task execution estimated time; A historical performance data acquisition module is configured to acquire historical performance data of all GPU BOXes in the same node corresponding to the task type; A model construction module is configured to construct a GPU BOX performance model according to real-time acquired state information of all GPU BOXes and the historical performance data; An allocation strategy acquisition module is configured to analyze the task characteristic information according to the GPU BOX performance model, and acquire a GPU BOX allocation strategy in the same node for executing tasks; The execution module is configured to determine a target GPU BOX according to the GPU BOX distribution strategy, and execute the to-be-processed task based on the target GPU BOX. The dynamic regulation module is configured to monitor running information of the target GPU BOX in real time, obtain a GPU BOX energy consumption dynamic regulation strategy according to the running information, and execute the GPU BOX energy consumption dynamic regulation strategy. The method further includes: obtaining performance data of each GPU BOX in a historical task execution process, and storing the performance data in a target database; the performance data includes calculation speed, memory bandwidth usage information, temperature information, power consumption information, load requirement information, and historical execution time length corresponding to different tasks; and querying, from the target database, performance data of all GPU BOXs of the same node that have executed the task type according to the task type. The method further includes: obtaining state information of all GPU BOXs in real time, the state information including current load, current temperature, current power consumption, and current available memory of each GPU BOX; determining availability evaluation results of the current load, the current temperature, the current power consumption, and the current available memory respectively; determining preset load weight, preset temperature weight, preset power consumption weight, and preset memory weight according to the historical performance data of all GPU BOXs; when all of the availability evaluation results pass, performing weighted summation on the current load, the current temperature, the current power consumption, and the current available memory according to the preset load weight, the preset temperature weight, the preset power consumption weight, and the preset memory weight to obtain an availability score of a single GPU BOX; and constructing a GPU BOX performance model according to the availability scores of all GPU BOXs and the historical execution time lengths corresponding to all GPU BOXs. The computer device includes: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the GPU BOX energy consumption regulation method in any one of claims 1-10. The computer readable storage medium stores computer instructions for causing a computer to execute the GPU BOX energy consumption regulation method in any one of claims 1-10. The computer instructions are executed by the processor to implement the steps of the method in any one of claims 1-10. ​ ​ 12. A computer apparatus, comprising: ​ ​ ​ ​ 13. A computer-readable storage medium, characterized in that, ​ 14. A computer program product comprising computer instructions, characterized in that, ​

Images