A Method and System for Dynamic Balancing of GPU Computing Power Scheduling

Abstract

This application discloses a dynamic balancing scheduling method and system for GPU computing power, which relates to the field of GPU cluster resource scheduling technology. The disclosed dynamic balancing scheduling method and system for GPU computing power solves the problems of scheduling lag and uneven resource distribution in the prior art by dynamically predicting load changes and actively migrating tasks. It can accurately predict GPU cluster load changes and realize active task scheduling, thereby improving resource utilization and operational stability.

Description

Technical Field

[0001] This application relates to the field of GPU cluster resource scheduling technology, and in particular to a method and system for dynamic balancing of GPU computing power scheduling. Background Technology

[0002] With the rapid development of artificial intelligence and large-scale modeling technologies, the scheduling and utilization efficiency of computing resources in large-scale GPU clusters faces severe challenges. Existing GPU cluster scheduling systems generally employ load balancing strategies based on fixed rules or static thresholds, or rely solely on real-time monitoring metrics for task allocation. These methods struggle to effectively capture complex dynamic load characteristics when dealing with heterogeneous GPU clusters, particularly exhibiting a severe deficiency in predicting short-term load changes. This leads to scheduling decisions lagging behind actual load fluctuations, resulting in significant fluctuations in cluster resource utilization. In scenarios with sudden peak task loads, such as when multiple high-computation-density tasks are submitted simultaneously, existing systems cannot predict load peaks in advance. This causes the utilization of some GPU nodes to surge to saturation, while other nodes remain idle or operate at low load, resulting in significant uneven resource allocation. In mixed-precision computing environments, different precision types of tasks (such as half-precision, single-precision, and double-precision floating-point operations) have significantly different requirements for GPU computing and storage resources. Existing scheduling strategies lack mechanisms to identify task precision types, failing to achieve precise matching between resource requirements and hardware specifications, further exacerbating computing waste. Furthermore, during concurrent execution of multiple tasks, the data dependencies between tasks are complex. Existing methods fail to fully integrate task data dependency graph information, leading to additional network communication overhead during task migration and reducing overall task throughput. Although task migration technology based on remote direct memory access protocols can reduce data transmission latency, it has shortcomings at the coordination level of scheduling decisions and migration execution. The system lacks predictive judgment and dynamic adaptation capabilities for changes in cluster state, cannot proactively intervene before load imbalance occurs, and struggles to achieve fine-grained elastic allocation of computing resources and global optimization goals.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a method and system for dynamic balancing of GPU computing power scheduling, which aims to improve the resource utilization and operational stability of GPU clusters.

[0005] To achieve the above objectives, this application proposes a dynamic GPU computing power balancing scheduling method, which includes:

[0006] Acquire GPU cluster monitoring data, task queue data, and network status data, and merge the GPU cluster monitoring data, task queue data, and network status data to generate a monitoring indicator data set;

[0007] The monitoring indicator data set is processed by feature extraction to obtain time series feature data and statistical feature data, and the time series feature data and statistical feature data are merged to generate a feature data set;

[0008] The time series feature data in the feature data set is processed by a pre-trained time series prediction model to generate time series prediction data, and the statistical feature data in the feature data set is processed by a statistical prediction model to generate statistical prediction data.

[0009] The time-series forecast data and statistical forecast data are dynamically weighted and fused to generate the final load forecast data;

[0010] The final load prediction data is compared with the preset load balancing threshold data. When the final load prediction data exceeds the preset load balancing threshold data, load imbalance judgment result data is generated, and scheduling instruction data is generated based on the load imbalance judgment result data.

[0011] The scheduling instruction data is transmitted to the target GPU node via the Remote Direct Memory Access Protocol, and a task migration operation is performed to achieve dynamic balancing of GPU computing power.

[0012] In one embodiment, the step of acquiring GPU cluster monitoring data, task queue data, and network status data, and merging the GPU cluster monitoring data, task queue data, and network status data to generate a monitoring metric data set includes:

[0013] Periodically collect GPU utilization data, video memory usage data, streaming multiprocessor activity data, temperature data, and power consumption data for each GPU node;

[0014] Measure network latency and bandwidth utilization data between nodes in the GPU cluster;

[0015] Extract task computation density data and task data dependency graph data of the tasks to be scheduled from the task queue;

[0016] The GPU utilization data, memory usage data, streaming multiprocessor activity data, temperature data, power consumption data, network latency data, bandwidth utilization data, task computation density data, and task data dependency graph data are merged to generate the monitoring indicator data set.

[0017] In one embodiment, the step of generating final load forecast data by dynamically weighting and fusing the time-series forecast data and statistical forecast data includes:

[0018] Based on the data feature types in the feature data set, calculate the confidence data of the pre-trained time series prediction model and the confidence data of the statistical prediction model;

[0019] The time series prediction data is weighted and calculated with the confidence data of the pre-trained time series prediction model to generate weighted time series prediction data.

[0020] The statistical prediction data is weighted and calculated with the confidence data of the statistical prediction model to generate weighted statistical prediction data.

[0021] The weighted time-series prediction data and the weighted statistical prediction data are added together to generate the final load prediction data.

[0022] In one embodiment, the steps of comparing the final load prediction data with a preset load balancing threshold data, generating load imbalance judgment result data when the final load prediction data exceeds the preset load balancing threshold data, and generating scheduling instruction data based on the load imbalance judgment result data include:

[0023] The final load prediction data is processed through load balancing calculation to generate load deviation data for each GPU node;

[0024] The load deviation data is compared with a preset load balancing threshold data to identify target GPU node data whose load deviation data exceeds the preset load balancing threshold data;

[0025] Based on the task execution status data in the target GPU node data, the task data to be migrated is determined by a task selection algorithm.

[0026] Based on the task data to be migrated and the target GPU node data, the scheduling instruction data containing target node identification data and task migration parameter data is generated.

[0027] In one embodiment, the steps of transmitting the scheduling instruction data to the target GPU node via a remote direct memory access protocol and performing task migration operations to achieve dynamic GPU computing power balancing include:

[0028] Parse the scheduling instruction data to obtain the target GPU node identifier data and the task identifier data to be migrated;

[0029] Based on the target GPU node identification data, a direct memory access channel is established between the GPU node's video memory and the target GPU node's video memory using the remote direct memory access protocol.

[0030] The execution status data and computation data of the task to be migrated are directly transferred from the GPU node memory to the target GPU node memory through the direct memory access channel.

[0031] The task execution is restored in the target GPU node's memory based on the transmitted execution status data and computation data, thus completing the task migration operation.

[0032] In one embodiment, after transmitting the scheduling instruction data to the target GPU node via a remote direct memory access protocol and performing a task migration operation to achieve dynamic balancing of GPU computing power, the method further includes:

[0033] Obtain actual load balancing status data;

[0034] The actual load balancing status data is compared with the final load prediction data to generate prediction error data.

[0035] The prediction error data is fed back to the dynamic weighted fusion processing to dynamically adjust the calculation weights of the confidence data of the pre-trained time series prediction model and the confidence data of the statistical prediction model, forming a dual-loop control architecture for prediction and correction.

[0036] In one embodiment, the method further includes:

[0037] Identify the precision type data of the task to be scheduled, wherein the precision type data includes half-precision floating-point data, single-precision floating-point data, and double-precision floating-point data;

[0038] The task resource requirement data is calculated based on the precision type data, and the task resource requirement data includes computing resource requirement data and storage resource requirement data.

[0039] The task resource requirement data is matched and calculated with the hardware specification data of the GPU node to generate precision matching scheduling scheme data, and the precision matching scheduling scheme data is used as reference data for generating scheduling instruction data.

[0040] In one embodiment, the method further includes:

[0041] Real-time monitoring of node running status data in the GPU cluster; when a node failure event is detected, obtain the fault node identification data.

[0042] Obtain the data of tasks currently executing on the faulty node based on the faulty node identification data;

[0043] The data of the tasks being executed on the failed node is reallocated to the available node data in the standby GPU node data to generate task recovery scheduling data.

[0044] The task execution state is restored from the available node data in the standby GPU node data based on the task recovery scheduling data.

[0045] In one embodiment, the method further includes:

[0046] The total computing power data of each GPU node is divided into multiple equally divided computing power unit data.

[0047] Based on the task's computational requirements data, GPU computing resources are allocated in units of computing power units to generate computing power allocation scheme data;

[0048] The number of data allocated to computing units is dynamically adjusted according to the changing computational needs of tasks in different scheduling cycles.

[0049] Furthermore, to achieve the above objectives, this application also proposes a GPU computing power dynamic balancing scheduling system, which includes: a memory, a processor, and a GPU computing power dynamic balancing scheduler stored in the memory and executable on the processor. The GPU computing power dynamic balancing scheduler is configured to implement the steps of the GPU computing power dynamic balancing scheduling method.

[0050] The GPU computing power dynamic balancing scheduling method and system proposed in this application solves the problems of scheduling lag and uneven resource distribution in the prior art by dynamically predicting load changes and actively migrating tasks. It can accurately predict GPU cluster load changes and realize active task scheduling, thereby improving resource utilization and operational stability. Attached Figure Description

[0051] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0052] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart illustrating an embodiment of the GPU computing power dynamic balancing scheduling method of this application;

[0054] Figure 2 This is a schematic diagram of a structure provided for an embodiment of the GPU computing power dynamic balancing scheduling system of this application.

[0055] Explanation of icon numbers:

[0056] 10. Memory; 20. Processor.

[0057] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0058] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of this application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0059] It should be understood that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0060] In existing technologies, GPU cluster scheduling systems typically employ load balancing strategies based on fixed rules or static thresholds, or rely solely on real-time monitoring metrics for task allocation. These methods struggle to handle the complex dynamic load characteristics of heterogeneous GPU clusters and cannot effectively predict short-term load changes. This leads to delayed scheduling decisions, significant fluctuations in cluster resource utilization, and uneven resource allocation issues such as overloaded local GPU nodes while other nodes remain idle, ultimately reducing overall task throughput.

[0061] Based on this, embodiments of this application provide a method for dynamic balancing of GPU computing power scheduling, referring to... Figure 1 The GPU computing power dynamic balancing scheduling method includes steps S100 to S600, wherein:

[0062] Step S100: Obtain GPU cluster monitoring data, task queue data, and network status data, and merge the GPU cluster monitoring data, task queue data, and network status data to generate a monitoring indicator data set;

[0063] Step S200: The monitoring indicator data set is processed by feature extraction to obtain time series feature data and statistical feature data, and the time series feature data and statistical feature data are merged to generate a feature data set;

[0064] Step S300: The time series feature data in the feature data set is processed by a pre-trained time series prediction model to generate time series prediction data. At the same time, the statistical feature data in the feature data set is processed by a statistical prediction model to generate statistical prediction data.

[0065] Step S400: The time-series prediction data and statistical prediction data are dynamically weighted and fused to generate the final load prediction data;

[0066] Step S500: Compare the final load prediction data with the preset load balancing threshold data. When the final load prediction data exceeds the preset load balancing threshold data, generate load imbalance judgment result data and generate scheduling instruction data based on the load imbalance judgment result data.

[0067] In step S600, the scheduling instruction data is transmitted to the target GPU node via the remote direct memory access protocol, and a task migration operation is performed to achieve dynamic balancing of GPU computing power.

[0068] In this embodiment, GPU cluster monitoring data refers to real-time operational status information collected from each GPU node in the GPU cluster, such as GPU utilization, memory usage, temperature, and power consumption, reflecting the current workload of the GPU nodes. Task queue data refers to information about tasks currently awaiting scheduling or being executed, such as task computation density, data dependencies, and priority, used to assess the task's demand for GPU resources. Network status data refers to performance metrics of the network connections within the GPU cluster, such as network latency between nodes and bandwidth utilization, used to assess the impact of data transmission efficiency on task execution. The monitoring metric dataset is a comprehensive dataset formed by integrating GPU cluster monitoring data, task queue data, and network status data, used to comprehensively reflect the cluster's operational status and task requirements.

[0069] In this embodiment, feature extraction processing refers to the process of processing the original monitoring indicator data set to identify and extract representative and predictive feature information for subsequent model input. Time series feature data refers to feature data with time-series dependence, such as the historical trend of GPU utilization and the periodic fluctuation of network bandwidth, reflecting the evolution of system state over time. Statistical feature data refers to feature data that does not directly depend on time sequence but reflects the overall distribution or central tendency of the data set, such as the average, variance, and peak value of GPU utilization, describing the statistical characteristics of the system state. The feature data set refers to the dataset formed by merging the time series feature data and statistical feature data after feature extraction processing, serving as the unified input to the prediction model.

[0070] In this embodiment, the pre-trained time-series prediction model refers to a machine learning model trained on historical time-series data, capable of predicting data trends at future points in time, such as a Long Short-Term Memory (LSTM) network or a Gated Recurrent Unit (GRU). Time-series prediction data refers to the prediction results about future load trends output by the pre-trained time-series prediction model based on time-series feature data. A statistical prediction model refers to a model that analyzes statistical feature data and predicts future load states based on statistical principles or machine learning methods, such as linear regression, support vector machines, or decision trees. Statistical prediction data refers to the prediction results about future load states output by a statistical prediction model based on statistical feature data. Dynamic weighted fusion processing refers to the process of dynamically adjusting the weights of multiple prediction results and merging them according to different data features or model confidence levels to generate a more accurate and robust final prediction result. The final load prediction data refers to the comprehensive prediction result of the future load of the GPU cluster obtained after dynamic weighted fusion processing. Preset load balancing threshold data refers to the reference standard that the system pre-sets to determine whether the GPU node load is in a balanced state, such as the upper limit of GPU utilization or the maximum allowable value of load deviation between nodes.

[0071] In this embodiment, the load imbalance judgment result data refers to the conclusion drawn from comparing the final load prediction data with the preset load balancing threshold data regarding whether the cluster experiences a load imbalance. Scheduling instruction data refers to the operation instructions generated based on the load imbalance judgment result data, used to guide the migration or reallocation of tasks between GPU nodes to restore cluster load balance. Remote Direct Memory Access Protocol (RDMA) is a data transfer technology that allows network adapters to directly access remote computer memory without CPU intervention, significantly reducing data transfer latency and CPU overhead. Task migration operation refers to the process of transferring a task being executed or pending execution on one GPU node, along with its execution status and related data, to another GPU node for continued execution. Dynamic GPU computing power balancing refers to maintaining a relatively balanced computing resource load on each node in the GPU cluster through real-time monitoring, prediction, and scheduling, avoiding local overload or idleness, thereby maximizing the overall computing power utilization and task throughput of the cluster.

[0072] In this embodiment, GPU cluster monitoring data, task queue data, and network status data are first acquired and then merged to generate a monitoring metric data set. For example, each GPU node can be manually inspected periodically, and information such as GPU utilization and memory usage can be manually recorded. The number and type of tasks in the task queue can also be manually counted. Simultaneously, network latency can be obtained through a simple network ping command. Subsequently, this data is summarized into a table to form a monitoring metric data set. As an optional implementation, a basic script program can be deployed to periodically read basic metrics such as GPU utilization and memory usage from the GPU driver interface, parse task submission records from the task scheduler log, and perform bandwidth tests using standard network tools (such as iperf). Then, these scattered data are processed through simple text parsing and concatenation operations to generate a monitoring metric data set containing all the information.

[0073] Furthermore, the monitoring indicator dataset undergoes feature extraction processing to obtain time-series and statistical feature data, which are then merged to generate a feature data set. Specifically, for each indicator in the monitoring indicator dataset, its historical values ​​can be directly used as time-series feature data; for example, GPU utilization values ​​from the past N time points can be directly used as a sequence. For statistical feature data, the current average or maximum value of each indicator can be simply calculated. Subsequently, these raw time-series values ​​and simple statistical values ​​are directly concatenated to form the feature data set. As a preferred implementation, preset fixed rules can be used to extract features from the monitoring indicator dataset. For example, continuous GPU utilization data can be directly considered as time-series feature data, while the instantaneous value of memory usage can be used as statistical feature data. These feature data are then directly merged to form a feature data set for subsequent prediction.

[0074] Building upon this foundation, the time-series feature data in the feature dataset is processed by a pre-trained time-series prediction model to generate time-series prediction data. Simultaneously, the statistical feature data in the feature dataset is processed by a statistical prediction model to generate statistical prediction data. For example, a simple linear regression model can be used as the pre-trained time-series prediction model, which performs linear extrapolation based only on the most recent points of the time-series feature data to generate prediction data. Meanwhile, a model based on historical averages is used as the statistical prediction model, directly calculating the historical average of the statistical feature data as the statistical prediction data. Alternatively, a fixed-parameter autoregressive moving average (ARMA) model can be deployed as the time-series prediction model to predict the time-series feature data. For the statistical prediction model, a simple decision tree model can be used to perform classification prediction based on the discrete intervals of the statistical feature data, generating statistical prediction data.

[0075] Subsequently, the time-series forecast data and statistical forecast data are dynamically weighted and fused to generate the final load forecast data. Specifically, a fixed weighting can be used to sum the time-series and statistical forecast data; for example, the time-series forecast data can always be multiplied by 0.7, and the statistical forecast data by 0.3, and then the two are added together to generate the final load forecast data. Alternatively, the fusion can be performed according to preset simple rules; for example, when the system load is low, the time-series forecast data is given a higher weight; when the system load is high, the statistical forecast data is given a higher weight. These weights are pre-set fixed values ​​and are not dynamically adjusted with real-time data.

[0076] Furthermore, the final load prediction data is compared with a preset load balancing threshold. When the final load prediction data exceeds the preset load balancing threshold, a load imbalance judgment result is generated, and scheduling instruction data is generated based on this result. For example, the final load prediction data can be directly compared with a globally unified fixed threshold. Once the prediction value exceeds this threshold, a simple "load imbalance" judgment result is generated. Subsequently, based on this judgment result, a preset general scheduling instruction is generated, for example, always migrating a task from the node with the highest load to the node with the lowest load. Alternatively, a simple binary judgment logic can be used. If the final load prediction data is higher than a certain fixed upper threshold, it is judged as overloaded; if it is lower than a certain fixed lower threshold, it is judged as underloaded. Based on the overload or underload judgment result, a scheduling instruction is generated containing the target node ID and the number of tasks, but the specific task selection and migration parameters are preset.

[0077] Finally, the scheduling instruction data is transmitted to the target GPU node via Remote Direct Memory Access (RDA) and a task migration operation is performed to achieve dynamic balancing of GPU computing power. Specifically, the scheduling instruction data can be encapsulated into a standard network packet and transmitted to the target GPU node via TCP / IP. Upon receiving the instruction, the target GPU node, using its local CPU and GPU drivers, copies the memory data of the task to be migrated from the source GPU's memory to system memory, then transmits it over the network to the target GPU node's system memory, and finally copies it to the target GPU's memory, completing the task migration. Alternatively, the scheduling instruction data can be sent to the target GPU node via a simple UDP protocol. Upon receiving the instruction, the target GPU node triggers a task pause and resume mechanism through a pre-defined API call. The task's execution status and computation data are first serialized and stored on the local disk, then transmitted to the target GPU node's disk via a file transfer protocol (such as NFS), and finally deserialized and resumed on the target GPU node.

[0078] In this embodiment, by fusing GPU cluster monitoring, task queue, and network status data, and performing feature extraction and multi-model prediction, the future load trend of the GPU cluster can be accurately predicted. By dynamically weighting and fusing the prediction results and comparing them with preset thresholds, timely identification and proactive intervention of load imbalances can be achieved. Combined with the Remote Direct Memory Access protocol for efficient task migration, the problem of uneven resource allocation and decreased throughput caused by decision lag in traditional scheduling methods is effectively avoided, significantly improving the overall computing power utilization efficiency and operational stability of the GPU cluster.

[0079] In one feasible implementation, the steps of acquiring GPU cluster monitoring data, task queue data, and network status data, and merging the GPU cluster monitoring data, task queue data, and network status data to generate a monitoring indicator data set include: periodically collecting GPU utilization data, memory usage data, streaming multiprocessor activity data, temperature data, and power consumption data of each GPU node; measuring network latency data and bandwidth utilization data between nodes in the GPU cluster; extracting task computation density data and task data dependency graph data of tasks to be scheduled from the task queue; and merging the GPU utilization data, memory usage data, streaming multiprocessor activity data, temperature data, power consumption data, network latency data, bandwidth utilization data, task computation density data, and task data dependency graph data to generate the monitoring indicator data set.

[0080] In this embodiment, GPU utilization data, memory usage data, streaming multiprocessor activity data, temperature data, and power consumption data of each GPU node are periodically collected to obtain real-time operating status and load information at the GPU hardware level. GPU utilization and memory usage data directly reflect the busyness of GPU core computing units and memory resources, and are key indicators for measuring the current task load. Streaming multiprocessor activity data provides more granular information on the usage of parallel computing units. Temperature and power consumption data are related to the GPU's operational stability and energy efficiency, and are of great significance for long-term stable cluster operation and resource management. This data can be periodically queried and collected through programming interfaces provided by GPU manufacturers (such as NVIDIA SMI, AMD ROCm SMI) or operating system-level performance monitoring tools. The collection frequency can be flexibly configured according to actual monitoring needs and system overhead to ensure the real-time nature and effectiveness of the data.

[0081] Simultaneously, measuring network latency and bandwidth utilization data between nodes in the GPU cluster is crucial for evaluating the efficiency and potential bottlenecks of inter-node communication in a distributed GPU cluster. Network latency directly impacts the time required for task migration and data synchronization, while bandwidth utilization reflects the saturation of network transmission capacity. These network performance metrics are critical for the performance of cross-node task scheduling and data-intensive applications. Network latency can be obtained by periodically sending probe packets (such as ICMP or TCP / UDP probes) between nodes and calculating the round-trip time. Bandwidth utilization can be measured by monitoring network interface traffic statistics or using specialized network performance testing tools.

[0082] Furthermore, extracting task computation density data and task data dependency graph data from the task queue aims to gain a deeper understanding of the task's characteristics and resource requirements. Task computation density data quantifies the intensity of a task's consumption of computing resources, such as the number of floating-point operations and memory access patterns, helping the scheduler allocate tasks to the most suitable GPU nodes. Task data dependency graph data describes the data flow and execution order between tasks, which is instructive for optimizing task scheduling, reducing data transfer overhead, and avoiding deadlocks. This task characteristic data can be provided by the task submitter during task definition, or estimated through static analysis of the task code or dynamic sampling at the beginning of task execution.

[0083] Through the above technical solution, this application can periodically collect GPU utilization data, memory usage data, streaming multiprocessor activity data, temperature data, and power consumption data of GPU nodes, thereby comprehensively understanding the real-time operating status and load of GPU hardware. Simultaneously, it measures network latency and bandwidth utilization data between nodes in the GPU cluster, enabling the scheduling system to assess the efficiency and potential bottlenecks of inter-node communication. Furthermore, it extracts task computation density data and task data dependency graph data of tasks to be scheduled from the task queue, providing crucial information for understanding the resource requirements and interrelationships of the tasks themselves. Merging these multi-dimensional, fine-grained data to generate a monitoring indicator dataset greatly enriches the original data source, allowing subsequent feature extraction and load prediction to be based on more comprehensive and accurate information. This effectively solves the problem that coarse data alone cannot accurately reflect the true load and bottlenecks of the cluster, significantly improving the quality of the monitoring indicator dataset, thus laying a solid foundation for accurate load prediction and efficient dynamic balancing scheduling of GPU computing power, ensuring the scientific nature and effectiveness of scheduling decisions.

[0084] In one feasible implementation, the step of generating final load prediction data by dynamically weighting and fusing the time-series prediction data and statistical prediction data includes: calculating the confidence data of the pre-trained time-series prediction model and the confidence data of the statistical prediction model according to the data feature types in the feature data set; performing a weighted calculation on the time-series prediction data and the confidence data of the pre-trained time-series prediction model to generate weighted time-series prediction data; performing a weighted calculation on the statistical prediction data and the confidence data of the statistical prediction model to generate weighted statistical prediction data; and adding the weighted time-series prediction data and the weighted statistical prediction data to generate the final load prediction data.

[0085] In this embodiment, during the dynamic weighted fusion processing, the confidence scores of the pre-trained time-series prediction model and the statistical prediction model are first calculated based on the data feature types in the feature dataset. Here, "data feature type" can refer to attributes such as volatility, periodicity, trend, and noise level of the input data. There are various methods for calculating confidence scores. For example, they can be evaluated based on historical prediction errors; if a model has historically performed better when handling specific data feature types, it is given a higher confidence score. Alternatively, they can be dynamically determined by analyzing the statistics of current data features (such as variance, autocorrelation coefficient, entropy, etc.). For instance, when the data exhibits strong periodicity, the confidence score of the time-series prediction model can be increased accordingly; when the data exhibits high randomness, the confidence score of the statistical prediction model can be increased. Furthermore, methods such as meta-learning or reinforcement learning can be used to adaptively adjust the confidence score calculation strategy based on real-time data features. These confidence scores quantify the reliability or applicability of each prediction model in the current data context.

[0086] Subsequently, the time-series prediction data is weighted and calculated with the confidence data of the pre-trained time-series prediction model to generate weighted time-series prediction data. This weighting calculation is typically achieved by multiplying the time-series prediction data by its corresponding confidence data. For example, if the confidence data is normalized to between 0 and 1, the weighted time-series prediction data is the product of the time-series prediction data and the normalized confidence data. This step aims to adjust the contribution weight of the time-series prediction model to the final prediction result based on its reliability under the current data characteristics.

[0087] Similarly, the statistical prediction data is weighted and calculated with the confidence data of the statistical prediction model to generate weighted statistical prediction data. This process is the same as the weighting calculation method for time series prediction data, multiplying the statistical prediction data by its corresponding confidence data to reflect the reliability of the statistical prediction model under the current data characteristics.

[0088] Finally, the weighted time-series prediction data and the weighted statistical prediction data are added together to generate the final load prediction data. In this way, the weighted prediction results of the two models are combined to form a comprehensive prediction value. If the confidence data is properly normalized (e.g., the sum of the confidence scores of the two models is 1), this addition operation effectively achieves a weighted average, ensuring that the final load prediction data more accurately reflects the actual load of the current GPU cluster.

[0089] Through the above technical solution, this application can adaptively adjust the weights of the time-series prediction model and the statistical prediction model in the final load prediction based on the dynamic changes in the characteristics of the input data. This dynamic weighted fusion process avoids the limitations that may arise from fixed-weight fusion, allowing the system to fully utilize the advantages of both prediction models: when the data has a significant time dependency, the prediction results of the time-series model are given more weight; while when the data exhibits randomness or burstiness, the prediction results of the statistical model are given more weight. This significantly improves the accuracy and robustness of the final load prediction data, resulting in more accurate load imbalance judgment data and more targeted and effective scheduling instruction data. Ultimately, this helps to achieve more refined and efficient dynamic balancing of GPU computing power, improving the overall resource utilization and task processing performance of the GPU cluster.

[0090] In one feasible implementation, the steps of comparing the final load prediction data with a preset load balancing threshold data, generating load imbalance judgment result data when the final load prediction data exceeds the preset load balancing threshold data, and generating scheduling instruction data based on the load imbalance judgment result data include: processing the final load prediction data through load balancing degree calculation to generate load deviation data for each GPU node; comparing the load deviation data with the preset load balancing threshold data to identify target GPU node data whose load deviation data exceeds the preset load balancing threshold data; determining the task data to be migrated based on the task execution status data in the target GPU node data using a task selection algorithm; and generating the scheduling instruction data containing target node identification data and task migration parameter data based on the task data to be migrated and the target GPU node data.

[0091] In this embodiment, the final load prediction data is processed through load balancing calculation to generate load deviation data for each GPU node. This aims to quantify the difference between the current load state of each GPU node and the ideal balanced state. This processing refines the final load prediction data down to each GPU node, generating load deviation data that reflects the load status of individual nodes. For example, the relative deviation can be calculated by comparing the final load prediction data of each GPU node with the average load prediction data of the entire GPU cluster; or, based on a preset ideal load distribution model, the degree of deviation of each node from the model can be calculated. This calculation can comprehensively consider a weighted combination of multiple factors such as GPU utilization, memory usage, and task queue length.

[0092] In this embodiment, the load skewness data is compared with a preset load balancing threshold to identify target GPU nodes whose load skewness exceeds the threshold. This accurately filters out GPU nodes with significant load imbalances, avoiding unnecessary scheduling operations on nodes with only slight deviations. The preset load balancing threshold can be a fixed value or a dynamically adjusted range based on cluster size, task type, historical data, etc. During the comparison, the system iterates through the load skewness data of all GPU nodes, marking nodes exceeding the threshold as target GPU nodes. These nodes will serve as candidate nodes for subsequent task migration.

[0093] In this embodiment, the task selection algorithm determines the task data to be migrated based on the task execution status data in the target GPU node data. This aims to select the most suitable tasks for migration from the identified target GPU nodes requiring load balancing, minimizing migration costs and impact on task execution. Task execution status data may include task priority, execution progress, remaining computation, data dependencies, memory usage, and CPU usage. The task selection algorithm can be based on various strategies, such as prioritizing tasks with low memory usage and low data dependencies to minimize the amount of data to be migrated; prioritizing tasks with high interruptibility and low real-time requirements to minimize the impact of interruptions; or selecting tasks that significantly reduce the load on the target nodes after migration to maximize the balancing effect. This algorithm can be a heuristic algorithm, a greedy algorithm, or a machine learning-based decision model.

[0094] In this embodiment, the scheduling instruction data, which includes target node identification data and task migration parameter data, is generated based on the task data to be migrated and the target GPU node data. This transforms the results of the preceding analysis and decision-making into executable instructions to guide subsequent task migration operations. The scheduling instruction data needs to contain sufficient information so that the receiver can accurately execute the migration. Specifically, the target node identification data clearly indicates which GPU node is the overloaded source node and which GPU node is the target node receiving the migration task (if the scheduling instruction also includes target receiving node information). The task migration parameter data includes the unique identifier of the task to be migrated, the task's current execution context (such as register status and program counter), the address and size of the data block in video memory, the task's priority, and other configuration information required for resumption of execution, ensuring that the task can seamlessly resume execution on the new node.

[0095] In this embodiment, through the above technical solution, this application can refine the overall load prediction data down to each GPU node, thereby accurately quantifying the load deviation of each node. Furthermore, by comparing these load deviation data with preset load balancing threshold data, the system can accurately identify the target GPU node data that truly needs load balancing, avoiding ineffective intervention on nodes with normal or slight deviations. Based on this, combined with the task execution status data in the target GPU node data, a task selection algorithm can intelligently determine the most suitable task data for migration, effectively reducing migration costs and the impact on business operations. Finally, the generated scheduling instruction data includes clear target node identification data and task migration parameter data, making subsequent task migration operations highly targeted and operable, significantly improving the efficiency and accuracy of dynamic GPU computing power balancing scheduling, and ensuring the optimal utilization of cluster resources.

[0096] In one feasible implementation, the steps of transmitting the scheduling instruction data to the target GPU node via a remote direct memory access protocol and performing a task migration operation to achieve dynamic GPU computing power balancing include: parsing the scheduling instruction data to obtain target GPU node identification data and task identification data to be migrated; establishing a direct memory access channel from the GPU node memory to the target GPU node memory via the remote direct memory access protocol based on the target GPU node identification data; directly transmitting the execution status data and computation data of the task to be migrated from the GPU node memory to the target GPU node memory via the direct memory access channel data; and resuming task execution in the target GPU node memory based on the transmitted execution status data and computation data, thus completing the task migration operation.

[0097] In this embodiment, upon receiving scheduling instruction data, the system first parses the scheduling instruction data. The scheduling instruction data is typically encoded in a structured format (e.g., JSON, Protocol Buffers, or a custom binary format) and contains all the critical information required to perform task migration. The parsing process aims to accurately extract the target GPU node identifier data and the task identifier data to be migrated. The target GPU node identifier data uniquely identifies the target GPU node to which the task will be migrated, while the task identifier data uniquely identifies the specific task that needs to be migrated from the current GPU node.

[0098] Subsequently, based on the acquired target GPU node identification data, the system establishes a direct memory access channel from the source GPU node's memory to the target GPU node's memory via the Remote Direct Memory Access (RDMA) protocol. RDMA allows network adapters to transfer data directly between host memory or GPU memory without CPU intervention. Establishing this channel typically involves using an RDMA network interface card (RNIC) and a corresponding RDMA programming interface (e.g., InfiniBand verbs or RoCE) to negotiate and establish a logical direct data transfer path between the source and target GPU nodes. Once established, this channel enables high-speed, low-latency data transfer between GPU memory.

[0099] After the direct memory access channel data is established, the system will use this channel to directly transfer the execution status data and computation data of the task to be migrated from the source GPU node's memory to the target GPU node's memory. The execution status data includes all necessary context information when the task is paused on the source GPU, such as the GPU core's register state, program counter, stack pointer, and any other transient data used to resume task execution. The computation data includes all input data being processed or generated by the task, intermediate results, and other data structures, which are typically stored in the GPU's memory. Through the remote direct memory access protocol, these data blocks can be copied directly from the source GPU's memory to a pre-allocated memory area in the target GPU's memory with extremely high efficiency, avoiding the overhead of multiple copies between the CPU and main memory in traditional data transfers.

[0100] Finally, in the target GPU node's memory, the system resumes task execution based on the transferred execution state data and computation data, thus completing the entire task migration operation. The target GPU node's scheduler or runtime environment takes over the task, reloads the task's context based on the restored execution state data, and continues execution using the transferred computation data. This ensures that the task can seamlessly resume execution on the target GPU node from the precise point where it paused on the source GPU node, thereby achieving dynamic balancing of GPU computing resources.

[0101] In this embodiment, by employing a remote direct memory access protocol for task migration, this application can significantly reduce the latency of transmitting task execution status data and computation data between different GPU nodes. Since data can be transferred directly between GPU memory, CPU intervention and multiple copies to main memory are avoided, thereby greatly reducing CPU overhead and improving data transmission efficiency. This makes the GPU task migration process faster and seamless, minimizing task interruption time or performance fluctuations caused by load balancing scheduling, and ensuring dynamic balancing and efficient utilization of GPU cluster computing resources.

[0102] In one feasible implementation, after transmitting the scheduling instruction data to the target GPU node via the Remote Direct Memory Access Protocol and performing task migration operations to achieve dynamic balancing of GPU computing power, the method further includes: acquiring actual load balancing status data; comparing and calculating the actual load balancing status data with the final load prediction data to generate prediction error data; and feeding the prediction error data back to dynamic weighted fusion processing to dynamically adjust the calculation weights of the pre-trained time-series prediction model confidence data and the statistical prediction model confidence data, forming a dual-loop control architecture for prediction and correction.

[0103] In this embodiment, obtaining actual load balancing status data refers to the real load situation of each node in the GPU cluster after executing scheduling instructions and performing task migration operations. It reflects the actual impact of scheduling behavior on the system state. This data can be obtained by re-collecting GPU cluster monitoring data (such as GPU utilization, memory usage, streaming multiprocessor activity, temperature data, and power consumption data), task queue data, and network status data. This data should be collected within a certain time interval after the task migration operation is completed to ensure that it reflects the stable state after migration. For example, the same collection mechanism used to generate the monitoring metric data set can be used to trigger data collection again some time after the scheduling instruction is executed (e.g., several seconds or tens of seconds).

[0104] In this embodiment, the actual load balancing state data is compared and calculated with the final load prediction data to generate prediction error data. This error refers to the quantified difference between the system's predicted value (final load prediction data) and the actual observed value (actual load balancing state data). This error is a key indicator for evaluating the accuracy of the prediction model and is also the basis for subsequent adjustments. Various mathematical methods can be used for the comparison calculation, such as calculating the absolute difference, relative difference, mean squared error (MSE), mean absolute error (MAE), or root mean square error (RMSE). The specific error calculation method chosen depends on the system's error sensitivity requirements and the design of the error feedback mechanism.

[0105] In this embodiment, the prediction error data is fed back to the dynamic weighted fusion processing module to dynamically adjust the calculation weights of the confidence data of the pre-trained time-series prediction model and the confidence data of the statistical prediction model. This is the core of achieving adaptive adjustment. By using the prediction error data as input, the dynamic weighted fusion processing module can evaluate the prediction performance of the time-series prediction model and the statistical prediction model respectively, and adjust their weights in the final load prediction data generation process accordingly. If the prediction error of a certain model is large, its weight may be reduced; conversely, if the prediction error is small, its weight may be increased. The dynamic weighted fusion processing can employ an error-based adaptive algorithm. For example, a feedback controller can be designed to dynamically adjust the calculation weights of the confidence data of the time-series prediction model and the confidence data of the statistical prediction model based on the magnitude and direction of the prediction error. This can be a simple proportional adjustment mechanism or a more complex machine learning algorithm, such as reinforcement learning, which optimizes the weight allocation strategy through continuous trial and error and learning. The weight adjustment can be periodic or event-driven, for example, triggered when the prediction error exceeds a certain threshold.

[0106] In this embodiment, the dual-loop control architecture of prediction and correction refers to a system that includes not only a feedforward prediction loop (for scheduling based on predictions) but also a feedback correction loop (for adjusting the prediction mechanism based on actual results). This architecture is the result of the synergistic effect of all the aforementioned steps. The prediction loop consists of acquiring monitoring data, feature extraction, model prediction, and generating scheduling instructions; the correction loop consists of acquiring the actual load, calculating the prediction error, feeding back the error, and adjusting the model weights. These two loops interact to form a closed loop, enabling the system to continuously learn and optimize its prediction and scheduling capabilities.

[0107] In this embodiment, by introducing the acquisition of actual load balancing state data, comparing it with the final load prediction data to generate prediction error data, and feeding this prediction error data back into the dynamic weighted fusion processing, this application forms a dual-loop control architecture for prediction and correction. This architecture effectively solves the prediction bias problem that may exist in purely prediction-based scheduling schemes. When the prediction result differs from the actual situation, the system can promptly detect and quantify this difference, and adaptively adjust the weights of the prediction model through a feedback mechanism, thereby improving the accuracy of subsequent predictions. This enables the GPU computing power dynamic balancing scheduling method not only to schedule based on forward-looking predictions, but also to self-correct and optimize according to the actual running effect, significantly enhancing the robustness and adaptability of the scheduling system, ensuring the long-term stability and efficiency of GPU cluster computing power allocation, avoiding frequent or ineffective scheduling due to inaccurate predictions, and thus improving overall resource utilization and task execution efficiency.

[0108] In one feasible implementation, the method further includes: identifying the precision type data of the task to be scheduled, the precision type data including half-precision floating-point data, single-precision floating-point data, and double-precision floating-point data; calculating task resource requirement data based on the precision type data, the task resource requirement data including computing resource requirement data and storage resource requirement data; matching the task resource requirement data with the hardware specification data of the GPU node to generate precision matching scheduling scheme data, and using the precision matching scheduling scheme data as reference data for generating scheduling instruction data.

[0109] In this embodiment, the step of identifying the precision type data of the task to be scheduled aims to clarify the specific requirements of the task for floating-point computation precision. Different computational tasks, such as deep learning training, scientific simulation, and graphics rendering, have different precision requirements for floating-point numbers. For example, deep learning training often uses half-precision floating-point data (FP16) to accelerate computation and reduce memory usage; general computing or graphics processing usually uses single-precision floating-point data (FP32); while high-precision scientific computing or financial modeling may require double-precision floating-point data (FP64). Precision type data can be identified in various ways. For example, when a task is submitted, the user or application can explicitly declare the required precision type in the task metadata; or, the scheduling system can automatically infer its precision requirements by analyzing the task code, task configuration file, or task tags. In addition, a database mapping task types to default precision can be maintained, automatically matching its precision type when a specific type of task is identified.

[0110] In this embodiment, when calculating task resource requirements based on precision type data, after identifying the precision type of the task, this step further quantifies the specific computing and storage resources required by the task. Different precision types of data have a significant impact on the utilization efficiency of computing units (such as CUDA cores and Tensor cores) and the amount of video memory occupied during processing. For example, half-precision computing usually requires fewer computation cycles and video memory bandwidth than single-precision or double-precision computing, but may require hardware that supports a specific half-precision instruction set. Task resource requirements can be calculated based on precision type data, estimated computational load of the task, and the data size of the task. Computational resource requirements can be expressed as the required number of floating-point operations (FLOPs), the required number of computing cores, or the computation time. Storage resource requirements can be expressed as the required video memory capacity and video memory bandwidth. These data can be estimated using preset empirical formulas, benchmark test results, or machine learning models. For example, for a task with a known computational load, if its precision type is half-precision floating-point data, its computational resource requirements may be lower than those of single-precision or double-precision tasks, and its data storage requirements will also be reduced accordingly.

[0111] In this embodiment, the core of precision-aware scheduling is the process of matching task resource requirements with GPU node hardware specifications to generate precision-matching scheduling scheme data, and then using this precision-matching scheduling scheme data as reference data for generating scheduling instruction data. It compares the task's resource requirements (including precision requirements) with the actual hardware capabilities of the GPU node to find the most suitable GPU node for executing the task. The GPU node's hardware specifications include its supported floating-point precision types (FP16, FP32, FP64), number of computing cores, memory capacity, and memory bandwidth. Through matching calculations, it can be ensured that the task is allocated to a GPU that not only has sufficient resources but also can efficiently process its specific precision requirements. Matching calculations can employ various strategies. One approach is rule-based matching: for example, if the task requires double-precision floating-point data, a GPU node supporting FP64 and with high FP64 performance is prioritized; if the task requires half-precision floating-point data, a GPU node with FP16 acceleration units such as Tensor cores is prioritized. Another approach is score-based matching: a matching score is calculated for each GPU node based on its hardware specifications and the resource requirements of the task, and the node with the highest score is selected. The resulting precision-matching scheduling scheme data can include a recommended list of target GPU nodes, priority ranking, or specific resource allocation suggestions. This scheme data is then used as an important reference for generating scheduling instruction data, guiding the final scheduling decision for the task.

[0112] In this embodiment, based on the above technical solution, this application further introduces the ability to perceive the precision type of tasks, building upon the dynamic balancing scheduling of GPU computing power. By identifying the precision type data of the tasks to be scheduled and calculating the computational and storage resource requirements accordingly, the scheduling system can more accurately assess the actual needs of tasks for GPU hardware. Subsequently, this task resource requirement data is matched with the hardware specifications of the GPU nodes to generate precision-matched scheduling scheme data, which is then used as a reference for generating scheduling instruction data. This precision-aware scheduling strategy ensures that tasks are assigned to GPU nodes best suited to their precision requirements. For example, scheduling deep learning tasks requiring half-precision floating-point data processing to GPUs with Tensor Cores can significantly improve computational efficiency; while scheduling scientific computing tasks requiring double-precision floating-point data processing to GPUs with excellent FP64 performance ensures computational accuracy. This not only avoids the problem of inefficient operation or resource waste on mismatched hardware but also fully leverages the specific acceleration capabilities of GPU hardware, thereby improving the overall resource utilization and task execution efficiency of the GPU cluster and further optimizing the effect of dynamic balancing of GPU computing power.

[0113] In one feasible implementation, the method further includes: real-time monitoring of node running status data in the GPU cluster; when a node failure event is detected, obtaining fault node identification data; obtaining data of tasks being executed on the fault node based on the fault node identification data; reallocating the data of tasks being executed on the fault node to available node data in the backup GPU node data to generate task recovery scheduling data; and restoring the task execution status in the available node data in the backup GPU node data based on the task recovery scheduling data.

[0114] In this embodiment, the system monitors the real-time operating status data of nodes in the GPU cluster. When a node failure event is detected, the system acquires the faulty node identification data, aiming to ensure that the system can promptly detect and respond to hardware or software failures in the GPU cluster. This can be achieved by deploying lightweight monitoring agents on each GPU node. These agents periodically collect the node's heartbeat signals, resource usage (e.g., CPU utilization, memory usage, disk I / O, network throughput), and the GPU's own health status (e.g., driver status, hardware error logs, temperature, power consumption, etc.). The collected data is sent to a central monitoring service for aggregation and analysis. When the monitoring service detects that a node's heartbeat signal has been lost for an extended period, its resource usage pattern is abnormal, or it reports a specific hardware or software error code, the system can determine that the node has failed and generate faulty node identification data containing the node's unique identifier (such as IP address, hostname, or node ID).

[0115] In this embodiment, the system obtains the data of tasks currently executing on the faulty node based on the faulty node identification data. This serves to accurately identify and record all affected tasks after a node failure, providing necessary information for subsequent task recovery operations. Once the faulty node identification data is determined, the scheduling system or task management module immediately queries its internally maintained task status database. This database records in detail the current execution status of each task, the allocated GPU node, the task's input data path, output data path, and any saved checkpoint data. By matching the faulty node identification data, the system can quickly filter out all tasks running on that faulty node and extract detailed information about these tasks, including task ID, task type, current execution stage, and required computing and storage resources.

[0116] In this embodiment, the data of tasks currently executing on the failed node is reallocated to available nodes in the backup GPU node database, generating task recovery scheduling data. This aims to ensure that tasks interrupted due to a failure can be rescheduled and continue execution, thereby significantly improving the fault tolerance and task continuity of the entire system. The scheduling system maintains a dynamically updated backup GPU node database, which contains data of all healthy nodes in the current cluster that have sufficient resources to handle new tasks. After obtaining the data of tasks currently executing on the failed node, the scheduler selects one or more of the most suitable available nodes for task reallocation based on the resource requirements of these tasks (e.g., required GPU computing power, memory size, network bandwidth, etc.) and the available resources in the backup node database, using preset scheduling strategies (e.g., minimizing migration time, maximizing resource utilization, task priority, etc.). The result of the reallocation generates task recovery scheduling data, which includes the task ID, new target node identification data, and specific parameters required for task recovery.

[0117] In this embodiment, the task execution state is restored from the available node data in the backup GPU node data based on the task recovery scheduling data. This process effectively performs the task recovery operation, allowing the task to continue execution from the point of interruption or restart from the beginning, thereby minimizing the impact of failures on the overall computation progress. Upon receiving the task recovery scheduling data, the available node data in the selected target backup GPU node data loads the task's input data and the most recent checkpoint data (if the task supports checkpointing) from a shared storage system (e.g., a distributed file system or object storage) according to the information in the scheduling data. Subsequently, a new task process is started on this available node data, instructing it to resume execution from the checkpoint. If the task does not support checkpointing, or the checkpoint data is unavailable, the task may need to restart from the beginning. This process ensures task continuity, avoids task reruns or data loss due to node failures, and improves the overall availability and efficiency of the GPU cluster.

[0118] In this embodiment, by introducing a real-time monitoring mechanism for the running status of GPU cluster nodes, this application can promptly detect and identify node failure events and obtain data on tasks currently executing on the failed nodes. Based on this, the system can intelligently reassign these affected tasks to available nodes in the backup GPU node data and restore their execution status. This effectively solves the problem of task interruption or loss when nodes in a GPU cluster experience unexpected failures, significantly improving the fault tolerance of the GPU computing power scheduling method and the stability of the system. Compared to solutions that only focus on load balancing, this solution ensures that critical tasks can continue to run even when some nodes fail, thereby guaranteeing the continuity and reliability of computing tasks, avoiding task reruns or data loss due to node failures, and improving the overall availability and efficiency of the GPU cluster.

[0119] In one feasible implementation, the method further includes: dividing the total computing power data of each GPU node into multiple equally divided computing power unit data; allocating GPU computing resources in units of computing power unit data according to the computing requirements data of the task, and generating computing power allocation scheme data; and dynamically adjusting the allocation quantity of computing power unit data according to the changes in the computing requirements data of the task in different scheduling cycles.

[0120] In this embodiment, the computing power unit data can be defined as the smallest allocatable unit of GPU computing resources. For example, it can represent a certain number of stream processors, a specific floating-point operation capability (such as TFLOPS), or a certain proportion of memory bandwidth. This division is implemented in a software-defined manner, enabling GPU resources to be managed and scheduled with finer granularity. Based on the task's computing requirements data, GPU computing resources are allocated in units of computing power unit data, and computing power allocation scheme data is generated. When a new task is submitted or an existing task needs resource adjustments, the system analyzes the task's computing requirements data, which may include the task's expected computational load, required memory size, and latency sensitivity. Based on these requirements, the resource scheduler selects and allocates a corresponding number of computing power unit data from the available computing power unit data pool to the task, thereby generating computing power allocation scheme data that records the computing power unit data allocated to each task and its corresponding GPU node information.

[0121] In this embodiment, to further optimize resource utilization, the allocation of computing unit data is dynamically adjusted based on changes in task computational requirements during different scheduling cycles. Within each scheduling cycle, the system monitors in real-time metrics such as actual GPU utilization, memory usage, and task progress of running tasks to obtain data on changes in task computational requirements. If the computational requirements of a task increase, the system can dynamically allocate additional computing unit data to it; conversely, if the requirements decrease, excess computing unit data can be reclaimed and released back to the common pool for use by other tasks. This dynamic adjustment process can be based on preset strategies or more complex models to ensure that resource allocation always maintains an optimal match with the actual needs of the tasks.

[0122] In this embodiment, the total computing power data of each GPU node is divided into multiple equally divided computing power units, achieving fine-grained management of GPU computing resources. Resource allocation based on task computing requirements, using computing power units as the unit, more accurately matches task needs and avoids resource waste or insufficiency that may result from coarse-grained allocation. Furthermore, the number of allocated computing power units is dynamically adjusted according to changes in task computing requirements during different scheduling cycles, enabling GPU resources to scale elastically and adapt to fluctuations in task load in real time. This fine-grained dynamic resource adjustment mechanism effectively improves GPU resource utilization and overall system throughput, reduces load imbalance caused by improper resource allocation, thereby reducing reliance on large-scale task migration and improving scheduling efficiency and system stability.

[0123] In the embodiments of this application, the GPU computing power dynamic balancing scheduling method solves the problems of scheduling lag and uneven resource distribution in the prior art by dynamically predicting load changes and actively migrating tasks. It can accurately predict GPU cluster load changes and realize active task scheduling, thereby improving resource utilization and operational stability.

[0124] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the GPU computing power dynamic balancing scheduling method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0125] This application also provides a GPU computing power dynamic balancing scheduling system, referenced Figure 2 The GPU computing power dynamic balancing scheduling system includes: a memory 10, a processor 20, and a GPU computing power dynamic balancing scheduler stored in the memory 10 and capable of running on the processor 20. The GPU computing power dynamic balancing scheduler is configured to implement the steps of the GPU computing power dynamic balancing scheduling method.

[0126] The GPU computing power dynamic balancing scheduling system provided in this application, employing the GPU computing power dynamic balancing scheduling method in the above embodiments, can improve the resource utilization and operational stability of GPU clusters. Compared with the prior art, the beneficial effects of the GPU computing power dynamic balancing scheduling system provided in this application are the same as those of the GPU computing power dynamic balancing scheduling method provided in the above embodiments, and other technical features of the GPU computing power dynamic balancing scheduling system are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0127] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0128] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. All equivalent structural transformations made under the technical concept of this application using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included within the scope of patent protection of this application.

Claims

1. A method for dynamic balancing and scheduling of GPU computing power, characterized in that, The method includes: Acquire GPU cluster monitoring data, task queue data, and network status data, and merge the GPU cluster monitoring data, task queue data, and network status data to generate a monitoring indicator data set; The monitoring indicator data set is processed by feature extraction to obtain time series feature data and statistical feature data, and the time series feature data and statistical feature data are merged to generate a feature data set; The time series feature data in the feature data set is processed by a pre-trained time series prediction model to generate time series prediction data, and the statistical feature data in the feature data set is processed by a statistical prediction model to generate statistical prediction data. The time-series forecast data and statistical forecast data are dynamically weighted and fused to generate the final load forecast data; The final load prediction data is compared with the preset load balancing threshold data. When the final load prediction data exceeds the preset load balancing threshold data, load imbalance judgment result data is generated, and scheduling instruction data is generated based on the load imbalance judgment result data. The scheduling instruction data is transmitted to the target GPU node via the Remote Direct Memory Access Protocol, and a task migration operation is performed to achieve dynamic balancing of GPU computing power. The steps for generating the final load forecast data by dynamically weighting and fusing the time-series forecast data and statistical forecast data include: Based on the data feature types in the feature data set, calculate the confidence data of the pre-trained time series prediction model and the confidence data of the statistical prediction model; The time series prediction data is weighted and calculated with the confidence data of the pre-trained time series prediction model to generate weighted time series prediction data. The statistical prediction data is weighted and calculated with the confidence data of the statistical prediction model to generate weighted statistical prediction data. The weighted time-series prediction data and the weighted statistical prediction data are added together to generate the final load prediction data; The steps of transmitting the scheduling instruction data to the target GPU node via the Remote Direct Memory Access protocol and performing task migration operations to achieve dynamic GPU computing power balancing include: Parse the scheduling instruction data to obtain the target GPU node identifier data and the task identifier data to be migrated; Based on the target GPU node identification data, a direct memory access channel is established between the GPU node's video memory and the target GPU node's video memory using the remote direct memory access protocol. The execution status data and computation data of the task to be migrated are directly transferred from the GPU node memory to the target GPU node memory through the direct memory access channel. The task execution is restored in the target GPU node's memory based on the transmitted execution status data and computation data, thus completing the task migration operation.

2. The GPU computing power dynamic balancing scheduling method as described in claim 1, characterized in that, The steps of acquiring GPU cluster monitoring data, task queue data, and network status data, and merging the GPU cluster monitoring data, task queue data, and network status data to generate a monitoring metric data set include: Periodically collect GPU utilization data, video memory usage data, streaming multiprocessor activity data, temperature data, and power consumption data for each GPU node; Measure network latency and bandwidth utilization data between nodes in the GPU cluster; Extract task computation density data and task data dependency graph data of the tasks to be scheduled from the task queue; The GPU utilization data, memory usage data, streaming multiprocessor activity data, temperature data, power consumption data, network latency data, bandwidth utilization data, task computation density data, and task data dependency graph data are merged to generate the monitoring indicator data set.

3. The GPU computing power dynamic balancing scheduling method as described in claim 1, characterized in that, The steps of comparing the final load prediction data with a preset load balancing threshold data, generating load imbalance judgment result data when the final load prediction data exceeds the preset load balancing threshold data, and generating scheduling instruction data based on the load imbalance judgment result data include: The final load prediction data is processed through load balancing calculation to generate load deviation data for each GPU node; The load deviation data is compared with a preset load balancing threshold data to identify target GPU node data whose load deviation data exceeds the preset load balancing threshold data; Based on the task execution status data in the target GPU node data, the task data to be migrated is determined by a task selection algorithm. Based on the task data to be migrated and the target GPU node data, the scheduling instruction data containing target node identification data and task migration parameter data is generated.

4. The GPU computing power dynamic balancing scheduling method as described in claim 1, characterized in that, After transmitting the scheduling instruction data to the target GPU node via the Remote Direct Memory Access protocol and performing task migration operations to achieve dynamic balancing of GPU computing power, the method further includes: Obtain actual load balancing status data; The actual load balancing status data is compared with the final load prediction data to generate prediction error data. The prediction error data is fed back to the dynamic weighted fusion processing to dynamically adjust the calculation weights of the confidence data of the pre-trained time series prediction model and the confidence data of the statistical prediction model, forming a dual-loop control architecture for prediction and correction.

5. The GPU computing power dynamic balancing scheduling method as described in claim 1, characterized in that, The method further includes: Identify the precision type data of the task to be scheduled, wherein the precision type data includes half-precision floating-point data, single-precision floating-point data, and double-precision floating-point data; The task resource requirement data is calculated based on the precision type data, and the task resource requirement data includes computing resource requirement data and storage resource requirement data. The task resource requirement data is matched and calculated with the hardware specification data of the GPU node to generate precision matching scheduling scheme data, and the precision matching scheduling scheme data is used as reference data for generating scheduling instruction data.

6. The GPU computing power dynamic balancing scheduling method as described in claim 1, characterized in that, The method further includes: Real-time monitoring of node running status data in the GPU cluster; when a node failure event is detected, obtain the fault node identification data. Obtain the data of tasks currently executing on the faulty node based on the faulty node identification data; The data of the tasks being executed on the failed node is reallocated to the available node data in the standby GPU node data to generate task recovery scheduling data. The task execution state is restored from the available node data in the standby GPU node data based on the task recovery scheduling data.

7. The GPU computing power dynamic balancing scheduling method as described in claim 1, characterized in that, The method further includes: The total computing power data of each GPU node is divided into multiple equally divided computing power unit data. Based on the task's computational requirements data, GPU computing resources are allocated in units of computing power units to generate computing power allocation scheme data; The number of data allocated to computing units is dynamically adjusted according to the changing computational needs of tasks in different scheduling cycles.

8. A GPU computing power dynamic balancing scheduling system, characterized in that, The GPU computing power dynamic balancing scheduling system includes: a memory, a processor, and a GPU computing power dynamic balancing scheduler stored in the memory and executable on the processor. The GPU computing power dynamic balancing scheduler is configured to implement the steps of the GPU computing power dynamic balancing scheduling method as described in any one of claims 1 to 7.

Images