Method and system for scheduling heterogeneous GPU (Graphics Processing Unit) by cross-cluster management software

Abstract

The invention provides a method and system for dispatching heterogeneous GPUs through cross-cluster management software, and relates to the technical field of GPU resource dispatching.The method comprises the steps that physical topology data, hardware specification data and historical task records of GPU nodes are obtained through the cross-cluster management software, an association degree matrix between GPU units is established, and a topology grouping set is generated; calculating a communication efficiency index based on topology detection analysis, constructing a topological graph model, and determining an optimal communication path of a multi-card task; generating a perception scheduling strategy in combination with a multi-objective optimization algorithm, and selecting an optimal GPU combination from the grouping set; in the task execution process, a scheduling strategy is dynamically adjusted according to the real-time cluster state and task efficiency data, continuous adaptation of a physical topological structure and task requirements is kept, and finally efficient collaborative scheduling of cross-cluster heterogeneous GPU resources is achieved. According to the method, the scheduling efficiency of heterogeneous GPU resources in a distributed computing scene and the overall performance of the system are improved.

Description

Technical Field

[0001] This application relates to the field of graphics processing unit (GPU) resource scheduling technology, and in particular to a method and system for scheduling heterogeneous GPUs across cluster management software. Background Technology

[0002] With the continued growth of artificial intelligence and large-scale computing demands, efficient scheduling technology for heterogeneous GPU resources across clusters is increasingly becoming a key to improving the overall performance of distributed computing. This type of technology can be widely applied in many fields such as cloud gaming, deep learning training, and large-scale scientific computing, showing broad application prospects.

[0003] Some existing technologies employ multi-objective optimization algorithms for heterogeneous GPU scheduling. For example, genetic algorithms or particle swarm optimization methods are used to balance multiple objectives such as computational efficiency and energy consumption, thereby achieving preliminary optimization of resource allocation. Some solutions also combine static topology awareness methods, which use fixed hardware connection topology information to assist in task scheduling decisions.

[0004] However, existing scheduling methods exhibit limitations in adaptability in dynamically changing cluster environments. Their predetermined scheduling strategies are prone to mismatches with the real-time fluctuating cluster operating status and task requirements. These mismatches lead to a decrease in resource utilization efficiency and affect the overall performance of collaborative computing among multiple GPU cards. Therefore, existing technologies generally suffer from a technical problem: they cannot achieve efficient resource scheduling in dynamic environments. Summary of the Invention

[0005] This application provides a method and system for scheduling heterogeneous GPUs across cluster management software, in order to solve the problems of low scheduling efficiency and poor overall system performance of heterogeneous GPU resources in distributed computing scenarios in the prior art.

[0006] To address the aforementioned technical problems, in a first aspect, this application provides a method for scheduling heterogeneous GPUs across cluster management software, comprising:

[0007] The physical topology data of multiple GPU nodes in the distributed computing cluster, the hardware specification data of the GPU units, and historical task execution records are obtained through cross-cluster management software.

[0008] Based on the physical topology data, the hardware specification data, and the historical task execution records, the relationships between GPU units are modeled to generate a GPU group set;

[0009] A topology probing analysis is performed on the GPU group set to construct a topology graph model across the cluster management software, and the optimal communication path for multi-GPU tasks is determined in the topology graph model.

[0010] Based on the optimal communication path and combined with a multi-objective optimization algorithm, a target GPU combination is selected from the GPU group set;

[0011] During the execution of computing tasks by the target GPU combination, the perceptual scheduling strategy is dynamically adjusted by combining cluster running status data and execution performance data to maintain the adaptation relationship between physical topology data and the task requirements of multi-GPU tasks, thereby realizing the collaborative scheduling of heterogeneous GPU resources across clusters.

[0012] Optionally, the step of selecting a target GPU combination from the GPU group set based on the optimal communication path and in conjunction with a multi-objective optimization algorithm includes:

[0013] Based on the baseline performance corresponding to the optimal communication path, a multi-objective optimization problem is established, which includes communication efficiency objective, resource utilization objective, and energy consumption control objective.

[0014] The multi-objective optimization problem is solved by a multi-objective optimization algorithm to generate a perceptual scheduling strategy, which includes an evaluation function and selection rules for GPU combination.

[0015] Based on the evaluation function and the selection rules, each candidate GPU combination in the GPU group set is calculated and filtered to obtain a comprehensive evaluation score for each filtered candidate GPU combination.

[0016] Based on the comprehensive evaluation score, all selected candidate GPU combinations are sorted, and candidate GPU combinations with a comprehensive evaluation score greater than a preset evaluation threshold are selected as target GPU combinations.

[0017] Optionally, the step of solving the multi-objective optimization problem using a multi-objective optimization algorithm to generate a perceptual scheduling strategy includes:

[0018] The multi-objective optimization problem is decomposed into a first optimization sub-problem, a second optimization sub-problem, and a third optimization sub-problem;

[0019] The first, second, and third optimization subproblems are solved collaboratively using a multi-objective optimization algorithm, and a population evolution mechanism is adopted during the solution process to generate a candidate solution set.

[0020] The candidate solution set is sorted to identify the Pareto optimal solution set;

[0021] Based on the Pareto optimal solution set, a preset propagation mechanism is introduced to determine the selection rules;

[0022] Key parameters are extracted from the Pareto optimal solution set, and an evaluation function for GPU combination is constructed based on the key parameters using an adaptive weighted fusion method.

[0023] The evaluation function and the selection rule are combined to form a perceptual scheduling strategy.

[0024] Optionally, the step of dynamically adjusting the perception scheduling strategy using a reinforcement learning model by combining cluster operation status data and execution performance data includes:

[0025] Monitor and collect cluster operation status data and execution performance data;

[0026] The cluster running status data and the execution performance data are input into the reinforcement learning model. Through the policy network of the reinforcement learning model, the current state of the distributed computing cluster is analyzed, and scheduling policy adjustment parameters are output.

[0027] Based on the scheduling strategy, the parameters are adjusted, and the weights of the evaluation function and the thresholds of the selection rules in the perception scheduling strategy are dynamically updated to obtain the updated perception scheduling strategy.

[0028] Optionally, the step of inputting the cluster running status data and the execution performance data into the reinforcement learning model, analyzing the current state of the distributed computing cluster through the policy network of the reinforcement learning model, and outputting scheduling policy adjustment parameters includes:

[0029] Extract core utilization, video memory usage, and inter-node network traffic from the cluster running status data; extract task completion progress and communication latency from the execution performance data.

[0030] The core utilization rate, the video memory usage rate, the inter-node network traffic, the task completion progress, and the communication latency are fused into a multi-dimensional data to construct a state vector that reflects the current state of the cluster.

[0031] The state vector is input into the policy network of the reinforcement learning model. The feature extraction layer of the policy network is used to reduce the dimensionality of the state vector in order to extract key features for operation.

[0032] The scheduling features are obtained by performing a nonlinear transformation on the key operational features through the hidden layer of the policy network.

[0033] The scheduling features are passed to the output layer of the policy network through a multi-layer neural network connection. An adjusted output value is generated based on the attention weight allocation mechanism. Based on the adjusted output value, the scheduling policy adjustment parameters are generated.

[0034] Optionally, the step of modeling the relationships between GPU units based on the physical topology data, the hardware specification data, and the historical task execution records to generate a GPU group set includes:

[0035] The connection relationship information is extracted from the physical topology data, the performance parameter information is extracted from the hardware specification data, and the cooperation mode information is extracted from the historical task execution records;

[0036] The connection relationship information, the performance parameter information, and the cooperation mode information are fused to form the initial node features of each GPU unit;

[0037] Based on the initial node features, calculate the feature correlation degree between GPU units;

[0038] GPU units with feature correlation greater than a preset correlation threshold are grouped into the same group to form a GPU group set.

[0039] Optionally, determining the optimal communication path for multi-card tasks in the topology graph model includes:

[0040] In the topology graph model, for the communication requirements of multi-card tasks, the starting point and ending point of the path are determined, and all possible communication paths connecting the starting point and ending point are generated through path discovery processing.

[0041] A path search algorithm is used to traverse all possible communication paths, calculate the communication efficiency index of each communication path, and select the communication path with the highest communication efficiency index as the optimal communication path.

[0042] Secondly, this application provides a system for scheduling heterogeneous GPUs across cluster management software, comprising:

[0043] The acquisition module is used to acquire physical topology data of multiple GPU nodes in a distributed computing cluster, hardware specification data of GPU units, and historical task execution records through cross-cluster management software.

[0044] The modeling module is used to model the relationships between GPU units based on the physical topology data, the hardware specification data, and the historical task execution records, and generate a GPU group set.

[0045] The computing module is used to perform topology probing analysis on the GPU group set, construct a topology graph model across the cluster management software, and determine the optimal communication path for multi-GPU tasks in the topology graph model.

[0046] The generation module is used to select a target GPU combination from the GPU group set based on the optimal communication path and in conjunction with a multi-objective optimization algorithm;

[0047] The adjustment module is used to dynamically adjust the perceptual scheduling strategy by combining cluster running status data and execution performance data during the execution of computing tasks by the target GPU combination, using a reinforcement learning model, in order to maintain the adaptation relationship between physical topology data and the task requirements of multi-card tasks, and realize the collaborative scheduling of heterogeneous GPU resources across clusters.

[0048] Thirdly, this application provides an electronic device, comprising:

[0049] Memory, used to store computer programs;

[0050] A processor, used to execute the computer program, implements the steps of the method for scheduling heterogeneous GPUs across cluster management software as described in the first aspect above.

[0051] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, enables the implementation of the steps of the method for scheduling heterogeneous GPUs across cluster management software as described in the first aspect above.

[0052] The technical solution provided in this application has the following beneficial effects:

[0053] This application provides solid data support for subsequent intelligent scheduling decisions by establishing complete cluster resource information. Based on this, it identifies the inherent connections between GPU computing units, forming logically reasonable resource groupings. Furthermore, it quantitatively evaluates the communication performance between different GPU units, thereby constructing a topology model that accurately reflects the actual network conditions. Based on this model, it selects the most efficient data transmission path for tasks requiring multi-GPU collaboration. Simultaneously, it achieves a dynamic balance among multiple optimization objectives such as computing efficiency, energy consumption, and resource utilization, forming a scientific resource selection standard. Ultimately, this enables the entire scheduling strategy to adapt to changes in the operating environment, thereby continuously maintaining the rationality of the resource allocation scheme in a dynamic environment.

[0054] Furthermore, this application, while prioritizing communication performance, also considers factors such as resource utilization and energy consumption control. By combining multi-objective optimization methods with a comprehensive scoring mechanism, it can achieve balanced optimization of heterogeneous GPU resource allocation, thereby improving the overall operating efficiency of the distributed computing system.

[0055] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A flowchart illustrating a method for scheduling heterogeneous GPUs across cluster management software, provided in an embodiment of this application;

[0058] Figure 2 This application provides a schematic diagram illustrating a specific implementation of a method for scheduling heterogeneous GPUs across cluster management software.

[0059] Figure 3 This is a schematic diagram of the structure of a system for scheduling heterogeneous GPUs across cluster management software, provided in an embodiment of this application. Detailed Implementation

[0060] Existing cross-cluster GPU scheduling methods struggle to keep pace with the real-time changes in cluster resource status when dealing with dynamically changing computing loads and diverse task requirements. This often results in a matching lag, which directly leads to the underutilization of cluster computing resources and consequently affects the execution efficiency of computing tasks that require multi-GPU collaboration. This phenomenon is particularly pronounced in environments with highly heterogeneous hardware configurations.

[0061] This application proposes a method for scheduling heterogeneous GPUs across cluster management software. This method generates a scheduling strategy that can adapt to the current state of the cluster by constructing a topology graph model that reflects the real-time network conditions and establishing a multi-objective optimization mechanism. During the actual execution of tasks, reinforcement learning algorithms are further used to continuously optimize the internal parameters of the strategy, so that the resource allocation scheme can always dynamically match the underlying physical topology and the ever-changing computing task requirements of the upper layer. This method effectively solves the problem of the scheduling strategy being out of sync with the dynamic operating environment in the prior art, realizes the efficient collaborative utilization of heterogeneous GPU resources across clusters, and ultimately improves the overall performance of the distributed computing system.

[0062] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0063] The core of this application is to provide a method for scheduling heterogeneous GPUs across cluster management software, and a flowchart of one specific implementation is shown below. Figure 1 As shown, the method includes:

[0064] Step 101: Obtain physical topology data of multiple GPU nodes in the distributed computing cluster, hardware specification data of GPU units, and historical task execution records through cross-cluster management software.

[0065] In the above scheme, a GPU node refers to a physical server or computing node that contains one or more GPU units. A GPU unit is a specific GPU processor within the node. A GPU node can contain multiple GPU units, which are connected through interconnect technologies such as NVLink or PCIe.

[0066] Physical topology data refers to the hardware connection relationship information between graphics processor nodes. This physical topology data includes the fast interconnect link connection method and bus connection path information.

[0067] The "hardware specification data of GPU units" reflects the "heterogeneous" characteristics. The hardware specification data includes information on the differences between different GPU units in terms of computing power, storage capacity, communication bandwidth, etc. These differences reflect the heterogeneous characteristics of GPU units in terms of model, architecture and performance.

[0068] Historical task execution records refer to records of the execution time, resource usage, and communication efficiency of past computing tasks.

[0069] In practical applications, a distributed deep learning training platform needs to schedule a total of 32 graphics processing unit (GPU) nodes for model training. The platform collects the connection status information between nodes in real time through an agent program deployed on each node, reads the detailed specifications of each GPU from the device's hardware management interface, and retrieves recent historical training task records from the central task management database. After unified preprocessing, the above data finally forms a complete resource dataset containing information on 320 GPU units.

[0070] Step 102: Based on the physical topology data, the hardware specification data, and the historical task execution records, model the relationship between GPU units and generate a GPU group set.

[0071] In this embodiment, step 102 includes the following process:

[0072] Step 1021: Extract connection relationship information from the physical topology data, extract performance parameter information from the hardware specification data, and extract cooperation mode information from the historical task execution records.

[0073] In the above scheme, the connection relationship information refers to the details of the physical connection path formed between graphics processing units through high-speed interconnect or bus, the performance parameter information includes hardware characteristic parameters such as the computing speed, storage capacity and communication bandwidth of the graphics processing units, and the cooperation mode information refers to the data transmission frequency and collaborative work efficiency records between graphics processing units in historical tasks.

[0074] For example, this application identifies specific fast interconnect relationships between eight graphics processing units, obtains the specific computing power and memory size of each unit, and calculates the data transfer efficiency between each pair of units in historical tasks.

[0075] Step 1022: Perform feature fusion on the connection relationship information, the performance parameter information, and the cooperation mode information to form the initial node features of each GPU unit.

[0076] In the above scheme, the initial node features are feature vectors that represent the comprehensive characteristics of the graphics processing unit, formed by fusing multiple types of information.

[0077] In this embodiment, step 1022 quantizes the connection relationship information into connection strength values, normalizes the performance parameter information into a unified standard score, and converts the collaboration mode information into a collaboration efficiency value. Then, these three types of quantized information are fused to generate the initial node feature vector for each graphics processing unit. For example, a high connection strength value is assigned to fast interconnect connections, a low connection strength value is assigned to bus connections, computing power and memory size are converted to a standard score within [0, 1], and the collaboration efficiency record is quantized into a specific collaboration efficiency value, ultimately forming an 8-dimensional feature vector for each graphics processing unit.

[0078] Step 1023: Calculate the feature correlation degree between GPU units based on the initial node features.

[0079] In the above scheme, feature correlation is a quantitative indicator used to measure the degree of similarity between two graphics processing units in terms of topology and functional characteristics.

[0080] In this embodiment of the application, step 1023 calculates the similarity between the feature vectors of each pair of graphics processor units based on the initial node feature vectors, and uses the similarity value as the feature correlation degree that characterizes the closeness of the relationship between the units; for example, the cosine similarity algorithm is used to calculate the cosine value of the angle between each pair of graphics processor unit feature vectors to obtain the feature correlation degree in the range of 0 to 1.

[0081] Step 1024: Divide GPU units with feature correlation greater than a preset correlation threshold into the same group to form a GPU group set.

[0082] In the above scheme, the preset correlation threshold is a grouping judgment standard value set according to actual application needs.

[0083] In this embodiment of the application, step 1024 compares the feature correlation between all pairs of graphics processor units with a preset correlation threshold, and divides the graphics processor units whose correlation between each other is higher than the threshold into the same group, thereby forming multiple graphics processor group sets; for example, all feature correlations form an 8-row 8-column correlation matrix, the correlation threshold is set to 0.8, and the graphics processor units whose correlation between each other is higher than 0.8 are grouped into the same group, and finally a group set is formed.

[0084] This application realizes intelligent grouping of heterogeneous graphics processor resources by establishing a graphics processor unit association model based on multi-source information fusion, and provides a reasonable resource organization basis for subsequent topology-aware scheduling, thereby effectively improving the scientific nature of resource management and scheduling efficiency.

[0085] Step 103: Perform topology probing analysis on the GPU group set, construct a topology graph model across the cluster management software, and determine the optimal communication path for multi-GPU tasks in the topology graph model.

[0086] In this embodiment, step 103 includes the following process:

[0087] Step 1031: Perform link quality detection analysis on each GPU unit in the GPU group set to generate connection status data.

[0088] In the above scheme, the connection status data includes measurements of the transmission bandwidth and communication delay between graphics processor units.

[0089] In this embodiment of the application, a bidirectional link test is performed on each graphics processor unit in the graphics processor group set in step 1031 to measure the maximum transmission bandwidth and minimum communication delay between each pair of graphics processor units, thereby generating connection status data containing specific bandwidth and delay values; for example, in a test group containing 20 graphics processor units, the bandwidth and delay values ​​between each pair of units are measured by sending test data packets of different sizes.

[0090] Step 1032: Based on the connection status data, calculate the communication performance index between each pair of GPU units using a weighted scoring method.

[0091] In the above scheme, the communication performance index is a quantitative score that comprehensively reflects the communication capability between the two graphics processing units.

[0092] In this embodiment of the application, the communication performance index is calculated by a weighted scoring method based on the bandwidth value and latency value in the connection status data in step 1032. The bandwidth value can be included in the score according to a direct proportional relationship, and the latency value can be included in the score according to an inverse proportional relationship. The two are combined into a single score value by a preset weight coefficient. For example, the bandwidth value is multiplied by the bandwidth weight, and the reciprocal of the latency value is multiplied by the latency weight. The two are added together to obtain the final communication performance index.

[0093] Step 1033: Standardize the communication performance index and construct a topological graph model with GPU units as vertices and standardized communication performance indexes as edge weights.

[0094] In the above scheme, the topology graph model is a weighted graph structure with graphics processing unit units as nodes and standardized communication performance indicators as edge weights.

[0095] In this embodiment of the application, step 1033 normalizes all communication performance indicators to the range of 0 to 1, and constructs a topological graph model with graphics processing units as nodes and standardized communication performance indicators as edge weights; for example, find the maximum value among all communication performance indicators, and then divide all indicators by the maximum value to complete the standardization, thereby constructing a weighted graph model.

[0096] Step 1034: In the topology graph model, for the communication requirements of multi-card tasks, determine the path start point and end point, and generate all possible communication paths connecting the path start point and end point through path discovery processing.

[0097] In the above scheme, the starting point and the ending point of the path are the starting graphics processing unit and the target graphics processing unit specified by the multi-card task. All possible communication paths refer to all connected path combinations that exist between the specified starting point and the ending point in the topology graph model. These paths include various connection methods with different numbers and relay nodes, including both direct single-hop paths and multi-hop paths forwarded through one or more intermediate graphics processing units. Each path combination represents a complete data transmission path selection. During the path search process, it is necessary to traverse all these feasible connection schemes to evaluate their respective communication performance indicators.

[0098] In this embodiment of the application, step 1034 determines the starting and ending nodes of the path in the constructed topology graph model according to the specific communication requirements of the multi-card collaborative task, and finds all possible communication paths connecting the starting and ending points through a depth-first search algorithm; for example, the starting and ending graphics processor units are determined according to the task requirements, and all connected paths are found by traversing the graph structure.

[0099] Step 1035: Use a path search algorithm to traverse all possible communication paths, calculate the communication efficiency index of each communication path, and select the communication path with the highest communication efficiency index as the optimal communication path.

[0100] In the above scheme, the communication efficiency index is the product of the weights of all edges on the communication path, and the optimal communication path is the communication path with the highest communication efficiency index.

[0101] In this embodiment of the application, step 1035 uses a path search algorithm to calculate the product of the weights of all edges on each communication path as the communication performance index of that path, and selects the path with the highest cumulative index as the optimal communication path for the task; for example, the product of all standardized communication performance indices on each path is calculated, and the path with the largest product is selected as the optimal communication path.

[0102] This application achieves intelligent selection of communication paths for multi-card tasks by establishing a precise communication performance evaluation system and topology map model, thereby effectively improving data transmission efficiency in distributed computing scenarios.

[0103] Step 104: Based on the optimal communication path, and combined with a multi-objective optimization algorithm, select a target GPU combination from the GPU group set.

[0104] In this embodiment, step 104 includes the following process, such as... Figure 2 As shown:

[0105] Step 1041: Based on the baseline performance corresponding to the optimal communication path, establish a multi-objective optimization problem that includes communication efficiency objective, resource utilization objective, and energy consumption control objective.

[0106] In the above scheme, the benchmark performance is derived from the process of calculating the communication performance index of each communication path and selecting the path with the highest value. It means that the best communication performance level that the GPU unit sequence represented by the optimal communication path can achieve can be used as a performance benchmark reference for subsequent multi-objective optimization.

[0107] The communication efficiency target is derived from the baseline performance of the optimal communication path and the real-time network status, which means that data transmission efficiency should be maximized while ensuring communication quality. The resource utilization target is derived from the real-time load status and historical usage data of each GPU unit in the GPU group set, which means that the overall utilization level of GPU computing resources and video memory resources should be improved.

[0108] The energy consumption control target is derived from the power monitoring data of the GPU unit and the energy consumption statistics of the cluster. It means that the overall energy consumption of the system should be reduced while meeting the performance requirements.

[0109] In this embodiment of the application, a multi-objective optimization problem is established by step 1041 based on the baseline efficiency value determined by the optimal communication path, which simultaneously considers maximizing communication efficiency, optimizing resource utilization, and minimizing energy consumption. For example, the baseline efficiency is used as a reference value for the communication efficiency objective, and combined with the actual resource status and energy consumption requirements of the current cluster, a mathematical programming model containing the above three optimization objectives is constructed.

[0110] Step 1042: Solve the multi-objective optimization problem using a multi-objective optimization algorithm to generate a perceptual scheduling strategy, which includes an evaluation function and selection rules for GPU combination.

[0111] In the above scheme, the evaluation function is the final application function obtained after solving the multi-objective optimization algorithm. This function integrates multiple optimization objectives into a single scoring function through fixed weight coefficients, which is used to directly calculate the comprehensive score of the candidate combination. The purpose of the multi-objective optimization algorithm is to find the Pareto optimal solution set without pre-setting the weight relationship between the objectives.

[0112] The selection rules are a set of conditions for filtering qualified candidate graphics processor combinations.

[0113] Step 1042 may specifically include the following steps:

[0114] A1: Decompose the multi-objective optimization problem into a first optimization subproblem, a second optimization subproblem, and a third optimization subproblem.

[0115] A2: The first optimization subproblem, the second optimization subproblem, and the third optimization subproblem are solved collaboratively using a multi-objective optimization algorithm. During the solution process, a population evolution mechanism is used to generate a candidate solution set.

[0116] A3: Sort the candidate solution set to identify the Pareto optimal solution set.

[0117] A4: Based on the Pareto optimal solution set, a preset propagation mechanism is introduced to determine the selection rules.

[0118] A5: Extract key parameters from the Pareto optimal solution set, and construct an evaluation function for GPU combination based on the key parameters using an adaptive weighted fusion method.

[0119] In the above scheme, the key parameter refers to the specific value of the balanced solution that can simultaneously optimize the three objectives of communication efficiency, resource utilization and energy consumption control in the Pareto optimal solution set. The specific content of the key parameter includes the weight coefficient of the communication performance index, the threshold parameter of resource utilization, the specific value of energy consumption limit and the trade-off ratio between each optimization objective.

[0120] Furthermore, the embodiments of this application do not specifically limit the content of the propagation mechanism, and can be specifically set according to the actual situation.

[0121] A6: Combine the evaluation function and the selection rule to form a perceptual scheduling strategy.

[0122] For example, a multi-objective optimization algorithm is used to generate multiple candidate solutions, a Pareto optimal solution set is identified through non-dominated sorting, resource usage limits and communication delay requirements are determined based on dynamic constraint propagation, a weighted evaluation function is constructed based on the parameter distribution of the optimal solution set, and then a perceptual scheduling strategy is constructed by combining selection rules.

[0123] Step 1043: Calculate and filter each candidate GPU combination in the GPU group set according to the evaluation function and the selection rules to obtain the comprehensive evaluation score of each selected candidate GPU combination.

[0124] In the above scheme, the comprehensive evaluation score is the calculated score of the candidate graphics processor combination under the evaluation function.

[0125] For example, an evaluation function is used to calculate the score for each candidate combination while checking whether it meets the constraints in the selection rules, and only candidate combinations that meet all conditions and their scores are retained.

[0126] Step 1044: Based on the comprehensive evaluation score, sort all the selected candidate GPU combinations and select the candidate GPU combination with a comprehensive evaluation score greater than the preset evaluation threshold as the target GPU combination.

[0127] In the above scheme, the preset evaluation threshold is the score limit for determining whether the candidate graphics processor combination is qualified. The embodiment of this application does not specifically limit the size of the threshold, but can be set according to the actual situation.

[0128] For example, candidate combinations can be sorted from highest to lowest score, and the best combination with a score above a set threshold can be selected as the final resource combination for task execution.

[0129] This application achieves balanced optimization of communication efficiency, resource utilization, and energy consumption control by establishing a multi-objective optimization scheduling strategy generation mechanism, thereby effectively improving the scheduling quality of heterogeneous graphics processor resources and system energy efficiency.

[0130] Step 105: During the execution of computing tasks by the target GPU combination, the perceptual scheduling strategy is dynamically adjusted by combining cluster running status data and execution performance data to maintain the adaptation relationship between physical topology data and the task requirements of multi-card tasks, thereby realizing the collaborative scheduling of heterogeneous GPU resources across clusters.

[0131] In the above scheme, the task requirements are derived from the resource requirements of the computing task to be executed, which include the required number of GPUs and communication bandwidth requirements.

[0132] In this embodiment, step 105 includes the following process:

[0133] Step 1051: Monitor and collect cluster running status data and execution performance data.

[0134] In the above scheme, cluster operation status data refers to the set of real-time working parameters of the graphics processing unit during the execution of computing tasks, and execution performance data refers to the record of the execution progress and performance of computing tasks on the graphics processing unit.

[0135] In this embodiment of the application, during the execution of computing tasks by the target graphics processor combination in step 1051, the utilization rate of the graphics processor computing core, the utilization rate of the video memory, and the network traffic between nodes are monitored in real time. At the same time, the task completion progress and the communication latency between nodes are recorded, thereby forming cluster operation status data and task execution performance data. For example, the workload data of each graphics processor unit is continuously collected by the deployed monitoring module, and the real-time execution progress information of the task is obtained synchronously through the task manager.

[0136] Step 1052: Input the cluster running status data and the execution performance data into the reinforcement learning model, analyze the current state of the distributed computing cluster through the policy network of the reinforcement learning model, and output the scheduling policy adjustment parameters.

[0137] In the above scheme, the current state refers to the system state vector composed of cluster running status data and execution performance data. This state is obtained by real-time monitoring of the computing core utilization, memory usage, inter-node network traffic, task completion progress and communication latency of the target GPU combination. It is used to reflect the comprehensive state of the resource usage and task execution performance of the entire distributed computing cluster at a specific moment.

[0138] Step 1052 may specifically include the following steps:

[0139] B1: Extract core utilization, video memory usage and inter-node network traffic from the cluster operation status data, and extract task completion progress and communication latency from the execution performance data.

[0140] B2: The core utilization rate, the video memory usage rate, the inter-node network traffic, the task completion progress, and the communication latency are fused into a multi-dimensional data to construct a state vector that reflects the current state of the cluster.

[0141] In the above scheme, the state vector is a unified feature representation formed by fusing multiple operating state data and performance data.

[0142] B3: Input the state vector into the policy network of the reinforcement learning model, and perform dimensionality reduction on the state vector through the feature extraction layer of the policy network to extract key features for operation.

[0143] In the above scheme, the key operational features refer to the compressed feature vectors extracted from the original multidimensional state data that can characterize the core operation status of the cluster. Their physical meanings include the graphics processor computing load concentration feature reflecting the distribution balance of the workload of each computing unit, the data communication bottleneck feature reflecting the congestion of data transmission between nodes, and the task execution efficiency feature representing the optimization space of task progress speed under the current resource configuration. These features remove redundant information from the original state data through dimensionality reduction processing, retain the key operational status indicators that affect scheduling decisions, and provide a refined decision basis for subsequent strategy adjustments.

[0144] Furthermore, this application does not impose specific limitations on the structural design of the reinforcement learning model, policy network, and the layers used in the internal structure of the policy network, and these can be set according to the actual situation.

[0145] B4: Through the hidden layer of the policy network, the key operational features are nonlinearly transformed to obtain scheduling features.

[0146] In the above scheme, the scheduling features are deep feature representations obtained through nonlinear transformation of a neural network.

[0147] B5: The scheduling features are transmitted to the output layer of the policy network through a multi-layer neural network connection. An adjusted output value is generated based on the attention weight allocation mechanism. Based on the adjusted output value, the scheduling policy adjustment parameters are generated.

[0148] In the above scheme, the adjusted output value is the value of the policy adjustment instruction output by the neural network, and the scheduling policy adjustment parameter is a set of values ​​used to modify the specific parameters of the scheduling policy.

[0149] In this embodiment, key operational indicators are extracted from the collected cluster operational status data in step 1052, and performance indicators are extracted from the execution efficiency data. These indicators are then fused to construct a state vector and input into the policy network of the reinforcement learning model. The network performs dimensionality reduction on the state vector through its feature extraction layer to obtain condensed key operational features. These features are then transformed by the nonlinear transformation of the hidden layer to obtain scheduling features. Finally, the output layer generates adjusted output values ​​based on attention weight allocation and converts them into a specific set of scheduling policy adjustment parameters. For example, the state vector is composed of the graphics processor core utilization, memory usage, network traffic, task completion progress, and communication latency. After processing by the above neural network, a set of scheduling policy adjustment parameters is output.

[0150] Step 1053: Adjust the parameters based on the scheduling strategy, dynamically update the weight of the evaluation function and the threshold of the selection rule in the perception scheduling strategy, and obtain the updated perception scheduling strategy.

[0151] In this embodiment of the application, step 1053 adjusts the weight coefficients of the evaluation function and the threshold parameters of the selection rule in the original perception scheduling strategy according to the specific values ​​in the parameter set of the scheduling strategy adjustment, thereby forming an updated perception scheduling strategy; for example, the weight ratios of various indicators such as communication efficiency, resource utilization and energy consumption in the evaluation function are modified according to the adjustment parameters, and the upper limit of resource usage and the performance requirement threshold in the selection rule are updated at the same time.

[0152] This application establishes a dynamic policy adjustment mechanism based on reinforcement learning to achieve real-time adaptation of scheduling policies to the operating environment, thereby effectively improving the adaptive capability of heterogeneous graphics processor resource scheduling and the stability of system performance.

[0153] Figure 3 This application provides a schematic diagram of the structure of a system for scheduling heterogeneous GPUs across cluster management software, as shown in the embodiments of this application. Figure 3 As shown, the detailed implementation section describes:

[0154] The acquisition module 31 is used to acquire physical topology data of multiple GPU nodes in the distributed computing cluster, hardware specification data of GPU units, and historical task execution records through cross-cluster management software.

[0155] Modeling module 32 is used to model the relationship between GPU units based on the physical topology data, the hardware specification data and the historical task execution records, and generate a GPU group set.

[0156] The computing module 33 is used to perform topology probing analysis on the GPU group set, construct a topology graph model across the cluster management software, and determine the optimal communication path for multi-GPU tasks in the topology graph model.

[0157] The generation module 34 is used to select a target GPU combination from the GPU group set based on the optimal communication path and in combination with a multi-objective optimization algorithm.

[0158] The adjustment module 35 is used to dynamically adjust the perception scheduling strategy by combining cluster running status data and execution performance data during the execution of computing tasks by the target GPU combination, using a reinforcement learning model, in order to maintain the adaptation relationship between physical topology data and the task requirements of multi-card tasks, and realize the collaborative scheduling of heterogeneous GPU resources across clusters.

[0159] The system for scheduling heterogeneous GPUs across cluster management software in this application embodiment is used to implement the aforementioned method for scheduling heterogeneous GPUs across cluster management software. Therefore, the specific implementation of the system for scheduling heterogeneous GPUs across cluster management software can be found in the embodiment section of the method for scheduling heterogeneous GPUs across cluster management software above. The specific implementation can be referred to the description of the corresponding embodiments, which will not be repeated here.

[0160] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the method for scheduling heterogeneous GPUs across cluster management software as described above.

[0161] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the methods described above for scheduling heterogeneous GPUs across cluster management software.

[0162] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0163] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described methods for scheduling heterogeneous GPUs across cluster management software.

[0164] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0165] The above provides a detailed description of the method and system for scheduling heterogeneous GPUs across cluster management software, as provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for managing software dispatching of heterogeneous GPUs across a cluster, the method comprising: The method comprises the following steps: obtaining physical topology data of multiple GPU nodes in a distributed computing cluster, hardware specification data of GPU units and historical task execution records through cross-cluster management software; modeling the association relationship between the GPU units based on the physical topology data, the hardware specification data and the historical task execution records, and generating a GPU grouping set; performing topology detection analysis on the GPU grouping set, constructing a topology graph model of the cross-cluster management software, and determining an optimal communication path of a multi-card task in the topology graph model; based on the optimal communication path, combining a multi-objective optimization algorithm and selecting a target GPU combination from the GPU grouping set; during the execution of a computing task by the target GPU combination, combining cluster running state data and execution performance data, dynamically adjusting a perception scheduling strategy by using a reinforcement learning model to maintain an adaptive relationship between the physical topology data and the task requirements of the multi-card task, and realizing collaborative scheduling of heterogeneous GPU resources across clusters.

2. The method of claim 1, wherein, The method of selecting a target GPU combination from the GPU grouping set based on the optimal communication path and combining a multi-objective optimization algorithm comprises the following steps: establishing a multi-objective optimization problem containing a communication efficiency target, a resource utilization target and an energy consumption control target based on the benchmark performance corresponding to the optimal communication path; solving the multi-objective optimization problem by using a multi-objective optimization algorithm to generate a perception scheduling strategy, wherein the perception scheduling strategy contains an evaluation function and a selection rule of the GPU combination; calculating and screening each candidate GPU combination in the GPU grouping set according to the evaluation function and the selection rule, and obtaining a comprehensive evaluation score of each screened candidate GPU combination; based on the comprehensive evaluation score, sorting all the screened candidate GPU combinations and selecting a candidate GPU combination with a comprehensive evaluation score greater than a preset evaluation threshold as the target GPU combination.

3. The method of claim 2, wherein, The method of solving the multi-objective optimization problem by using a multi-objective optimization algorithm to generate a perception scheduling strategy comprises the following steps: decomposing the multi-objective optimization problem into a first optimization sub-problem, a second optimization sub-problem and a third optimization sub-problem; cooperatively solving the first optimization sub-problem, the second optimization sub-problem and the third optimization sub-problem by using a multi-objective optimization algorithm, and generating a candidate solution set in the solving process by using a population evolution mechanism; sorting the candidate solution set to identify a Pareto optimal solution set; based on the Pareto optimal solution set, introducing a preset propagation mechanism to determine a selection rule; extracting key parameters from the Pareto optimal solution set, constructing an evaluation function of the GPU combination by using an adaptive weighted fusion method based on the key parameters; combining the evaluation function and the selection rule to form a perception scheduling strategy.

4. The method of claim 1, wherein, The method of dynamically adjusting a perception scheduling strategy by using a reinforcement learning model based on cluster running state data and execution performance data comprises the following steps: monitoring and collecting cluster running state data and execution performance data; inputting the cluster running state data and the execution performance data into a reinforcement learning model, analyzing a current state of the distributed computing cluster through a policy network of the reinforcement learning model, and outputting a scheduling strategy adjustment parameter; based on the scheduling strategy adjustment parameter, dynamically updating weights of an evaluation function and threshold values of selection rules in the perception scheduling strategy to obtain an updated perception scheduling strategy.

5. The method of claim 4, wherein, The inputting the cluster running state data and the execution performance data into a reinforcement learning model, analyzing a current state of the distributed computing cluster through a policy network of the reinforcement learning model, and outputting a scheduling strategy adjustment parameter comprises: extracting core usage rate, video memory occupancy rate and inter-node network traffic from the cluster running state data, and extracting task completion progress and communication delay time from the execution performance data; performing multi-dimensional data fusion on the core usage rate, the video memory occupancy rate, the inter-node network traffic, the task completion progress and the communication delay time to construct a state vector for reflecting the current state of the cluster; inputting the state vector into the policy network of the reinforcement learning model, performing dimension reduction processing on the state vector through a feature extraction layer of the policy network to extract running key features; performing non-linear transformation on the running key features through a hidden layer of the policy network to obtain scheduling features; passing the scheduling features through a multi-layer neural network connection to the output layer of the policy network, generating an adjustment output value based on an attention weight distribution mechanism, and generating a scheduling strategy adjustment parameter based on the adjustment output value.

6. The method of claim 1, wherein, The modeling of the association relationship between GPU units based on the physical topology data, the hardware specification data and the historical task execution record, and generating a GPU grouping set comprises: extracting connection relationship information from the physical topology data, extracting performance parameter information from the hardware specification data, and extracting cooperation mode information from the historical task execution record; performing feature fusion on the connection relationship information, the performance parameter information and the cooperation mode information to form initial node features of each GPU unit; based on the initial node features, calculating feature association degrees between GPU units; dividing GPU units with feature association degrees greater than a preset association degree threshold to the same group to form a GPU grouping set.

7. The method of claim 1, wherein, The determining of the optimal communication path of the multi-card task in the topology graph model comprises: in the topology graph model, determining a path starting point and a path ending point for the communication demand of the multi-card task, and generating all possible communication paths connecting the path starting point and the path ending point through path discovery processing; adopting a path search algorithm to traverse all possible communication paths, calculating communication performance indicators of each communication path, and selecting a communication path with the highest communication performance indicator as the optimal communication path.

8. A system for managing software dispatching of heterogeneous GPUs across a cluster, the system comprising: comprises: an acquisition module configured to acquire, through cross-cluster management software, physical topology data of a plurality of GPU nodes in a distributed computing cluster, hardware specification data of GPU units, and historical task execution records; The modeling module is configured to model the association relationship between GPU units based on the physical topology data, the hardware specification data, and the historical task execution record, and generate a GPU grouping set; The computing module is configured to perform topology detection analysis on the GPU grouping set, construct a topology graph model of the cross-cluster management software, and determine an optimal communication path of a multi-card task in the topology graph model; The generating module is configured to select a target GPU combination from the GPU grouping set based on the optimal communication path and in combination with a multi-objective optimization algorithm; The adjusting module is configured to dynamically adjust a perception scheduling strategy by using a reinforcement learning model in combination with cluster running state data and execution performance data during execution of the target GPU combination on a computing task, so as to maintain an adaptive relationship between the physical topology data and task requirements of the multi-card task, and realize collaborative scheduling of the heterogeneous GPU resources of the cross-cluster.

9. An electronic device, comprising: Comprise: A memory for storing a computer program; A processor for executing the computer program to implement the steps of the method for scheduling heterogeneous GPUs of the cross-cluster management software according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method for scheduling heterogeneous GPUs of the cross-cluster management software according to any one of claims 1 to 7.

Images