A future key job schedulability protection method for online heterogeneous GPU cluster scheduling

Abstract

The application discloses a future key job schedulability protection method for online heterogeneous GPU cluster scheduling, comprising the following steps: firstly, constructing a cluster resource state model and a future key job demand template based on historical load; then, when scheduling a current job, generating candidate placement schemes and evaluating the damage score of each scheme on the schedulable space of the future key job; meanwhile, identifying and marking key bearing spaces such as complete idle nodes, high memory nodes and compact topology groups, and adaptively adjusting the protection strength according to the queue congestion state; finally, calculating the comprehensive score by comprehensively considering the damage score, the key space occupation cost, the current placement cost and the congestion priority, and selecting the optimal scheme to execute scheduling. The application can guarantee the current scheduling efficiency while improving the successful scheduling probability of future large-scale jobs, high memory jobs and compact topology jobs, and can balance the resource utilization rate, fairness and scheduling overhead, and is suitable for various online scheduling scenes of heterogeneous GPU clusters.

Description

Technical Field

[0001] This invention relates to the field of intelligent computing resource scheduling technology, and in particular to a method for protecting the schedulability of future critical jobs in online heterogeneous GPU cluster scheduling. Background Technology

[0002] With the rapid growth of computationally intensive tasks such as large-scale AI model training and inference, scientific computing, and big data analytics, heterogeneous GPU clusters have become the core infrastructure of intelligent computing centers and AI computing platforms. In real-world production environments, different types of jobs have significantly different requirements for the number of GPUs, single-card memory capacity, runtime, and node topology. For example, lightweight inference or data analysis jobs typically require only a small number of GPUs and low memory resources; while large-scale distributed training jobs often require multiple GPUs working collaboratively within a single node, or tightly coupled computing resources across multiple nodes; and jobs with high memory requirements must be scheduled to GPU nodes with high memory capacity to run normally.

[0003] Currently, most online GPU cluster scheduling methods focus on the efficiency of real-time resource matching in a single scheduling decision. Common scheduling strategies include First-Fit, Random-Fit, and Best-Fit. In low-load or resource-sufficient scenarios, these methods can accomplish basic job scheduling tasks. However, in online scheduling environments with high load, heterogeneous resources, and a mix of different job types, existing methods reveal the following shortcomings.

[0004] First, locally optimal scheduling strategies may disrupt the capacity of future critical jobs. If ordinary jobs are placed in fully idle nodes, high-memory nodes, or compact GPU topologies that could originally accommodate large-scale jobs, subsequent large-scale or high-memory jobs will be forced to wait due to a lack of suitable resource space, potentially increasing the risk of timeouts.

[0005] Second, existing methods lack explicit modeling of future critical job demand patterns. Traditional scheduling strategies typically only determine whether a current job can be placed on a certain node, without assessing the extent to which the current placement decision compresses the available resource space for future critical jobs, and also lack forward-looking protection mechanisms for large jobs, high memory usage jobs, and compact topology jobs.

[0006] Third, heterogeneous GPU environments lack effective differentiation and protection of high-value, scarce resources. In clusters containing ordinary nodes, high-memory nodes, and different GPU topology groups, simply matching resources based on available memory or the number of available GPUs per node makes it difficult to distinguish between ordinary and critical resources. This can easily lead to high-value resources being preempted by low-demand jobs, resulting in resource misallocation.

[0007] Fourth, in online scheduling, the length of the job queue, the average waiting time, the risk of timeout, and the real-time load of the cluster all change dynamically. Traditional static scheduling strategies are difficult to adaptively adjust the resource retention strength and job priority according to the congestion status. This may cause critical jobs to wait for a long time during high load phases, or excessively retain resources during low load phases, thus affecting the overall throughput efficiency.

[0008] Furthermore, in large-scale GPU clusters, if each scheduling requires precise scoring of all nodes, it will result in high decision-making latency, which is not conducive to the real-time operation of the online scheduling system.

[0009] In summary, current technologies lack a unified scheduling method that can simultaneously ensure current job scheduling efficiency, future schedulability of critical jobs, protection of critical resources, fairness of waiting, and control of scheduling overhead in an online heterogeneous GPU cluster environment. Summary of the Invention

[0010] The purpose of this invention is to provide a method for protecting the schedulability of future critical jobs in online heterogeneous GPU cluster scheduling, in order to solve the problem that existing scheduling methods cannot guarantee the schedulability of future critical jobs.

[0011] A technical solution, a method for protecting the schedulability of future critical jobs in online heterogeneous GPU cluster scheduling, includes the following steps:

[0012] Construct cluster resource status data for heterogeneous GPU clusters, and build template data for future critical job requirements based on historical load characteristics and current job queuing status;

[0013] For the current job to be scheduled, several candidate placement scheme data are generated, and the damage of each candidate placement scheme to the schedulable space of the future critical job is evaluated based on the future critical job requirement template data, so as to obtain the schedulability damage score data of each candidate scheme.

[0014] Based on the future critical operation requirement template data and the cluster resource status data, identify and mark the critical bearer spaces in the cluster, and generate critical bearer space marking data.

[0015] Collect current scheduling queue status data, calculate cluster congestion status index data, and adaptively adjust the protection strength of critical bearer space according to the congestion level to generate adaptive scheduling parameter data.

[0016] Based on the schedulability damage score data, critical bearing space marking data, adaptive scheduling parameter data, and current job placement cost, a comprehensive score is calculated for each candidate placement scheme, and the scheme with the best comprehensive score is selected for scheduling.

[0017] Beneficial effects: By proactively assessing the damage to the schedulable space of future critical jobs caused by the placement of current jobs, and combining congestion-aware adaptive protection of critical carrying space, this invention can improve the success rate of scheduling future large-scale jobs, high-memory jobs, and compact topology jobs while ensuring current scheduling efficiency, and taking into account both resource utilization and scheduling fairness. Attached Figure Description

[0018] Figure 1 This is a flowchart of the overall solution of the present invention. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0020] like Figure 1 This embodiment describes in detail the data processing flow of a future-critical job schedulability protection method for online heterogeneous GPU cluster scheduling, specifically including:

[0021] Step S1: Construct cluster resource status data for heterogeneous GPU clusters, and construct template data for future critical job requirements based on historical load characteristics and current job queuing status.

[0022] The technical solution provided in this application first requires establishing an accurate description of the heterogeneous GPU cluster resources and effectively modeling the critical job requirements that may be reached in the future, specifically including:

[0023] S1-1: Collect and aggregate node static configuration data.

[0024] Specifically, through cluster management interfaces or hardware monitoring components, static attribute information determined at the time of manufacture or deployment is collected from each physical node. This includes: node type labels, which mark nodes as "normal nodes" or "high-memory nodes" based on whether their video memory capacity exceeds a preset threshold; the total number of GPUs within the node, such as an 8-card or 16-card configuration; the total video memory capacity of each GPU, recorded in megabytes or gigabytes; and the topological connection relationships between GPUs within the node, such as a connection diagram formed through the NVLink bus or a tree-like topology structure built based on the PCIe switching hierarchy. The collected data undergoes uniform formatting to generate static resource description data for each node. This data is organized into a set of key-value pairs, where the keys are the node identifier and GPU index, and the values ​​are the corresponding static attributes.

[0025] S1-2: Collect and synchronize real-time status data of nodes.

[0026] Furthermore, through periodic polling or event-triggered mechanisms, dynamically changing resource usage information is obtained from each node, including: the current remaining video memory capacity of each GPU, i.e., the available value after subtracting allocated video memory from the total video memory; the identifier of each running job and the set of GPUs and video memory it occupies; and the current load status of the node, such as whether any jobs are being executed or waiting to be released. All dynamic data is accompanied by a collection timestamp, and outliers or sampling points with excessive latency are removed through consistency checks to obtain real-time resource usage data for each node.

[0027] S1-3: Construct a structured cluster resource state model.

[0028] In one optional implementation, the obtained static resource description data of the nodes is associated and aggregated with the obtained real-time resource occupancy data of the nodes. Specifically, the aggregation method is as follows: using nodes as the basic unit, static attributes and dynamic states are merged to form a comprehensive resource view for each node, including the following fields: node type, GPU topology group (i.e., marking a group of GPUs with high-bandwidth interconnects as a group), total and remaining video memory of each GPU, current set of idle GPUs (i.e., a list of GPU indices not occupied by any jobs), and continuous idle GPU space information (e.g., the number of consecutively unoccupied GPUs within the same topology group and their starting index). The comprehensive resource views of all nodes are summarized into a global dictionary structure or relational table to obtain structured cluster resource state model data, which serves as the basic input for all subsequent scheduling decisions.

[0029] S1-4: Collect historical and real-time job load characteristics.

[0030] In this embodiment, to construct a data model that reflects future critical job requirements, data from two sources needs to be collected. The first type is historical job arrival logs recorded by the cluster within a past time window (e.g., the past 24 hours or the past week), which includes the arrival time, resource requirements (number of GPUs, memory requirements), job type label, and actual runtime for each job. The second type is the real-time list of jobs waiting in the current scheduling queue and their resource requirements. In addition, business scenario label data is collected, such as "peak period for large model training," "stable period for inference services," or "mixed load period." The above data is cleaned, deduplicated, and normalized to extract the job arrival time distribution characteristics and resource requirement distribution characteristics, resulting in job load characteristic data.

[0031] S1-5: Identify and quantify the types of key future tasks.

[0032] Based on a further improvement of this embodiment, the obtained job load characteristic data is analyzed and processed. The analysis process includes: counting the frequency of jobs requiring more than a first threshold (e.g., 4 or 8 GPUs) within the historical window and the current queue, and marking them as "large-scale GPU jobs"; counting the proportion of jobs with single GPU memory requirements exceeding a second threshold (e.g., 40GB or 80GB), and marking them as "high memory jobs"; counting the number of jobs requiring resource allocation within consecutive GPUs on the same node or within a specific topology group, and marking them as "compact topology jobs". Based on the statistical results, key job template data is dynamically generated or updated. Each key job template data is a structured record containing: job type enumeration values ​​(e.g., TYPE_LARGE_SCALE, TYPE_HIGH_MEMORY, TYPE_COMPACT_TOPOLOGY), the range of required GPUs (lower and upper limits), the single GPU memory requirement threshold (in gigabytes), and topology compactness requirements (e.g., "requires consecutive GPU indices within the node" or "requires being located in the same NVLink domain").

[0033] S1-6: Assign dynamic weights to critical task templates.

[0034] In a preferred embodiment, a dynamic weight value is further assigned to each generated critical job template based on the current cluster congestion level and business priority. The weight is calculated as follows: First, the number of each type of job waiting in the current scheduling queue is obtained, denoted as N_large, N_highmem, and N_compact, respectively; then, a basic weight value is preset according to the business scenario label, for example, during peak training periods, the basic weight of large-scale jobs is set to 0.6, high-memory jobs to 0.3, and compact topology jobs to 0.1; then, the final weight is obtained through normalization, so that the sum of all weights is 1. The weight value is appended to the corresponding critical job template to obtain future critical job demand model data with weight attributes. This model data expresses the predictive information of "which type of future critical job is most worthy of protection in the next scheduling cycle".

[0035] Step S2: For the current job to be scheduled, generate several candidate placement scheme data, and evaluate the damage of each candidate placement scheme to the schedulable space of the future critical job based on the future critical job requirement template data, and obtain the schedulability damage score data of each candidate scheme.

[0036] After completing resource status modeling and future demand modeling, this application further performs a forward-looking placement assessment on the currently arriving jobs. The specific process is as follows:

[0037] S2-1: Obtain the current job requirements and generate candidate placement schemes.

[0038] For example, when a new job arrives at the scheduler, its resource requirements are first parsed, including the required number of GPUs, the memory requirements per GPU, and whether topology compactness is required. Then, based on the obtained cluster resource state model data, a restricted scanning strategy is used to generate candidate placement schemes. This strategy does not traverse all nodes, but prioritizes checking nodes with high resource fragmentation (e.g., nodes where some GPUs are already occupied but the remaining GPUs can still meet the current job's requirements), as well as non-critical resource spaces (ordinary resource areas defined in subsequent steps). For each node and GPU combination that meets the current job's resource requirements, its placement information is recorded, including the node ID, the specific GPU index list allocated, and the remaining resource state of the node after allocation. This information together constitutes a candidate placement scheme data set. Typically, 3 to 5 candidate schemes are generated to control scheduling overhead.

[0039] S2-2: Simulate placement and generate a snapshot of the future immediate resource state.

[0040] For each generated candidate placement scheme, a simulated placement operation is performed. Specifically, a copy of the current cluster resource state model data is made as a working copy, and the GPU and video memory resources to be occupied by the current job are subtracted from this working copy. Specific operations include: reducing the remaining video memory of the corresponding GPU by the amount required by the current job; removing the corresponding GPU from the set of idle GPUs; and updating the contiguous idle GPU space information if the allocation disrupts previously contiguous GPU space. After these modifications, the working copy becomes the "future instantaneous resource state snapshot" data for that candidate scheme, reflecting the cluster resource distribution at the very next instant after placement if the current placement scheme is adopted.

[0041] S2-3: Assess the snapshot's capacity to support future critical operations.

[0042] In one optional implementation, the obtained future critical job demand model data with weighted attributes is used to perform a schedulability check on each future immediate resource state snapshot. The check process is as follows: For each critical job template, the snapshot is searched for the existence of resource space capable of accommodating the template. For example, for a large-scale GPU job template, the snapshot needs to be searched for a node whose consecutive idle GPUs are greater than or equal to the number of GPUs required by the template; for a high-memory job template, the snapshot needs to be searched for a high-memory node whose remaining memory on its idle GPUs is greater than or equal to the memory threshold required by the template; for a compact topology job template, the snapshot needs to be searched for a complete topology group that has not been split and occupied. If a resource space that meets the conditions exists, the template is considered schedulable under the current snapshot, and the schedulable value is recorded as 1; otherwise, it is recorded as 0.

[0043] S2-4: Calculate the schedulability impairment score.

[0044] Furthermore, the schedulability changes of each critical job template are compared before placement (original cluster resource state model data) and after placement (future immediate resource state snapshot). For each critical job template i, the schedulable value in the original state is denoted as S_i_original, and the schedulable value in the snapshot state is denoted as S_i_snapshot. The damage level D_i is defined as follows: if S_i_original is 1 and S_i_snapshot is 0, then D_i = 1 (indicating that the originally schedulable critical job has become unschedulable); otherwise, D_i = 0. Then, the damage level D_i of each template is multiplied by its dynamic weight w_i, and the result is summed over all templates to obtain the schedulability damage score data of the candidate placement scheme. The calculation formula is: Damage_Score = Σ (w_i × D_i); where w_i represents the weight of the i-th type of critical job template, Σ represents the summation over all critical job template types, and D_i takes the value 0 or 1. The lower the score, the better. A score of 0 indicates that the placement scheme will not disrupt the schedulability of any future critical operations, while a positive value indicates that it has caused varying degrees of damage.

[0045] Step S3: Based on the future critical operation requirement template data and the cluster resource status data, identify and mark the critical bearer space in the cluster, and generate critical bearer space marking data.

[0046] To prevent routine operations from unintentionally occupying resource areas crucial for future critical operations, this application designs a mechanism for identifying and marking critical carrying spaces. The specific implementation is as follows:

[0047] S3-1: Analyze the capacity of the resource space to support the critical operation template.

[0048] According to this embodiment, firstly, cluster resource status model data and future critical job demand model data with weighted attributes are acquired. Each resource space unit in the cluster is traversed; a resource space unit can be a single node, a topology group within a node, or a group of consecutive idle GPUs. For each resource space unit, it is determined whether it can accommodate each critical job template. The determination criteria are as follows: for large-scale GPU job templates, the number of consecutive idle GPUs within the resource space unit must be greater than or equal to the number of GPUs required by the template; for high-memory job templates, the resource space unit must be located on a high-memory node, and the remaining memory of all idle GPUs must be greater than or equal to the memory threshold required by the template; for compact topology job templates, the resource space unit must be a complete topology group and not occupied by any job.

[0049] S3-2: Identify high-value critical carrier spaces.

[0050] In a preferred embodiment, resource space units that meet any of the above-mentioned critical job template carrying conditions and whose template weight is higher than a preset threshold (e.g., greater than 0.3) are identified as "critical carrying spaces". Specifically, the following typical types of critical carrying spaces can be identified: The first type is a complete idle node capable of carrying large-scale GPU jobs, characterized in that all GPUs on the node are not occupied, and the node type can be a normal node or a high-memory node; the second type is a high-memory node capable of carrying high-memory jobs, characterized in that the node type is a high-memory node, and at least one GPU has more remaining memory than the high-memory threshold; the third type is a complete GPU group capable of meeting the requirements of a compact topology, characterized in that all GPUs in the group are in an idle state and are not split for use.

[0051] S3-3: Generate key carrier space marker data.

[0052] Furthermore, structured labeled data is generated for each identified critical bearer space, containing the following fields: a unique space identifier (e.g., node ID plus group index), space type (e.g., "full node type", "high-memory node type", "compact topology group type"), the ID of the node to which it belongs, the set of GPU indexes it contains, and the corresponding protection level (calculated based on the weight of the matched critical job template; the higher the weight, the higher the protection level). All labeled data are aggregated to form a critical bearer space labeled dataset. This dataset will be used in the subsequent comprehensive scoring step to evaluate whether a candidate placement scheme occupies resource space with a high protection level.

[0053] Step S4: Collect current scheduling queue status data, calculate cluster congestion status index data, and adaptively adjust the protection strength of critical bearer space according to the congestion level to generate adaptive scheduling parameter data.

[0054] To avoid excessive resource protection during periods of high load, which could cause current jobs to wait for extended periods, this application introduces a congestion-aware mechanism to dynamically adjust the protection strength. The specific process is as follows:

[0055] S4-1: Collect queue status data.

[0056] For example, before each scheduling decision, the current queue status information is read from the scheduler's waiting queue management module. The collected data includes: the waiting queue length Q_len, i.e., the total number of jobs currently queuing for scheduling; the waiting time WaitTime_j for each job in the queue (the time difference from the job's arrival time to the current time); and the expected timeout Timeout_j for each job (typically set according to the job type and service level agreement, e.g., 30 seconds for inference jobs and 5 minutes for training jobs). This raw data is then organized into a queue status data set.

[0057] S4-2: Calculate congestion status indicators.

[0058] In this embodiment, the collected queue status data is statistically processed to calculate a comprehensive congestion status index. The calculation formula is as follows: Avg_Wait = (1 / N) × Σ (WaitTime_j); Max_Wait_Ratio = max(WaitTime_j / Timeout_j); Queue_Pressure = Q_len / (Historical_Max_Q_len); where N is the total number of jobs in the waiting queue, Σ represents the sum of the waiting times of all jobs, max represents the maximum value, and Historical_Max_Q_len is the maximum queue length observed in the historical records (or a preset reference capacity value). Avg_Wait represents the average waiting time, with the same unit as WaitTime; Max_Wait_Ratio represents the ratio of the maximum waiting time to the timeout threshold, and a value greater than 1 means that a job is about to time out or has already timed out; Queue_Pressure represents the degree of queue backlog, with a value ranging from 0 to a positive number, and a larger value indicates more severe congestion. These three indicators together constitute the cluster congestion status indicator data.

[0059] S4-3: Determine the congestion level.

[0060] According to a further improvement of this embodiment, the obtained congestion status index data is compared with a preset threshold to classify three congestion levels. The specific judgment logic is as follows:

[0061] Low congestion level: Satisfies Avg_Wait < first waiting threshold (e.g., 5 seconds), Max_Wait_Ratio < 0.5, and Queue_Pressure < 0.3. This level indicates that cluster resources are relatively abundant, and the current job basically does not need to wait.

[0062] Medium congestion level: Meets the following conditions: First waiting threshold ≤ Avg_Wait < Second waiting threshold (e.g., 20 seconds) or 0.5 ≤ Max_Wait_Ratio < 0.8 or 0.3 ≤ Queue_Pressure < 0.7. This level indicates a certain degree of resource contention.

[0063] High congestion level: Satisfies Avg_Wait ≥ the second waiting threshold, or Max_Wait_Ratio ≥ 0.8, or Queue_Pressure ≥ 0.7. This level indicates that the cluster load is very high, and some jobs are at risk of timeout.

[0064] S4-4: Adaptively adjusts protection strength and generates parameters.

[0065] In one optional implementation, the protection strength and other scheduling parameters of the critical bearer space are dynamically adjusted based on the determined congestion level. The adjustment rules are as follows:

[0066] When the congestion level is low, the generated adaptive scheduling parameter data includes: protection strength coefficient P = 1.0 (indicating full protection), penalty factor Penalty = 10 (high) for occupying critical bearer space, and no job priority is increased.

[0067] When the congestion level is medium, the generated adaptive scheduling parameter data includes: protection strength coefficient P = 0.6, penalty factor Penalty = 5, and a moderate priority boost coefficient Priority_Boost = 1.2, which is applied to jobs with a waiting time exceeding Avg_Wait.

[0068] When the congestion level is high, the generated adaptive scheduling parameter data includes: protection strength coefficient P = 0.2, penalty factor Penalty = 1 (very low or even negligible), and a priority boost coefficient Priority_Boost = 2.0 is set for jobs whose waiting time exceeds 0.8 times the timeout threshold, so that they can bypass some protection constraints and be scheduled with priority.

[0069] The above parameters are encapsulated into a configuration object, which is then output as adaptive scheduling parameter data to the next step.

[0070] Step S5: Based on the schedulability damage score data, critical bearing space marking data, adaptive scheduling parameter data, and current job placement cost, calculate the comprehensive score of each candidate placement scheme, and select the scheme with the best comprehensive score for scheduling.

[0071] Finally, this application integrates the information generated from all the aforementioned steps to make a final scheduling decision. The detailed process is as follows:

[0072] S5-1: Obtain the various scoring factors.

[0073] For each candidate placement scheme, the following four types of data need to be collected as input factors for the comprehensive score:

[0074] The schedulable damage score is denoted as Factor_Damage. A smaller value indicates better protection for future critical operations.

[0075] Critical bearer space occupancy flag: Examine the generated critical bearer space flag data to determine whether the GPU set allocated by the current candidate scheme belongs to any flagged critical bearer space. If yes, the flag variable Flag_Occupied = 1; otherwise, it is 0.

[0076] Adaptive scheduling parameter data, including protection strength coefficient P, penalty factor Penalty, and priority boost coefficient Priority_Boost.

[0077] Current job placement cost: Calculates the immediate cost incurred when placing the current candidate solution. This can include resource utilization (i.e., the degree to which the allocated resources match the actual needs of the job, such as avoiding the allocation of large nodes that exceed the demand), node load balancing (avoiding the concentration of multiple jobs on the same node), or the degree of fragmentation caused by placement. This cost is quantified as Factor_Cost, with a value ranging from 0 to 1. The smaller the value, the lower the placement cost.

[0078] S5-2: Calculate the overall score.

[0079] In this embodiment, a weighted summation method is used to calculate the comprehensive score of each candidate placement scheme. The calculation formula is: Total_Score = w1 × Factor_Damage + w2 × (Flag_Occupied × Penalty) + w3 × Factor_Cost - w4 × Priority_Boost × Is_Waiting_Job; where w1, w2, w3, and w4 are pre-set basic weight coefficients, for example, w1=0.4, w2=0.3, w3=0.2, and w4=0.1; the first term w1 × Factor_Damage: reflects the penalty for future schedulability damage, the greater the damage, the higher the score (the worse); the second term w2 × (Flag_Occupied × Penalty): reflects the penalty for occupying critical carrying space. When Flag_Occupied is 0, this item is 0; when it is 1, the penalty value is Penalty, and the size of Penalty is dynamically determined by step S4-4 based on the congestion level (the lower the congestion, the higher the penalty); the third item w3 × Factor_Cost: reflects the current placement cost, the higher the cost, the higher the score; the fourth item w4 × Priority_Boost × Is_Waiting_Job: where Is_Waiting_Job is an indicator variable. If the currently scheduled job has been waiting in the queue for a long time (e.g., exceeding Avg_Wait) or its priority is high, then Is_Waiting_Job = 1, otherwise it is 0. This item is negative, meaning that a score bonus is given to jobs that have been waiting for a long time, making them more likely to be selected, thereby achieving fairness correction. The smaller the Total_Score value, the better the overall performance of the candidate solution.

[0080] S5-3: Select the optimal solution and execute the placement.

[0081] In a preferred embodiment, all candidate placement schemes are sorted in ascending order according to the calculated Total_Score, and the scheme with the lowest score (i.e., the optimal one) is selected as the final scheduling decision. Based on the node IDs and GPU index list recorded in this scheme, a resource allocation instruction is generated. This instruction can be in the format of a JSON object or a protocol buffer message, containing the operation type ("allocate"), job ID, node list, and the GPU index allocated to each node. The instruction is issued to the corresponding node for execution through the cluster management interface to complete the placement and startup of the current job.

[0082] S5-4: State update and subsequent scheduling triggering.

[0083] After a job is successfully placed and starts running, the global state of the cluster needs to be updated. Specifically, the cluster resource state model data is persistently updated according to the actual allocation results: the allocated resources are deducted, and the job information is added to the list of running jobs. Then, it checks whether there are any unprocessed jobs in the scheduling queue. If the queue is not empty, it returns to step S2 to begin the same processing flow for the next job to be scheduled; if the queue is empty, it enters an event listening state, waiting for a new job to arrive before restarting execution from step S1 (or at least re-executing steps S1-2 and subsequent steps, as the real-time state of the nodes may have changed).

[0084] Through the complete process of steps S1 to S5 described above, this application provides a method for explicitly protecting the schedulability of future critical jobs during online heterogeneous GPU cluster scheduling. This method, through forward-looking assessment, critical resource space identification, congestion-aware adaptive adjustment, and multi-factor comprehensive scoring, significantly improves the successful scheduling probability of future large jobs, high-memory jobs, and compact topology jobs while ensuring current scheduling efficiency.

[0085] This invention has the following advantages: First, by constructing a template for future critical task requirements and proactively assessing the damage to schedulable space caused by each placement scheme, it effectively avoids the destruction of the carrying capacity of future large tasks, high-memory tasks, and compact topology tasks by current local optimal decisions. This reduces the risk of waiting and timeouts caused by resource fragmentation or the occupation of critical space for critical tasks from the source, improving the scheduling success rate and timeliness of critical tasks under high load scenarios. Second, by identifying and marking critical carrying spaces such as complete idle nodes, high-memory nodes, and compact topology groups, and dynamically adjusting the protection strength in conjunction with the congestion status, it not only firmly preserves high-value resources to cope with sudden critical tasks under low load, but also appropriately relaxes protection and prioritizes long-awaited tasks when under high load or when tasks are about to time out. This achieves an adaptive balance between resource protection and system real-time performance, avoiding resource idleness or resource mismatch caused by over-protection. Furthermore, based on a multi-factor comprehensive scoring mechanism that considers damage scoring, critical space occupancy cost, current placement cost, and congestion-aware priority, future schedulability protection, resource utilization efficiency, and job fairness are integrated into a single decision framework. This ensures that the scheduling strategy maintains excellent overall performance under different load conditions. In addition, the candidate placement scheme employs restricted scanning instead of full node traversal, effectively controlling the computational overhead of scheduling decisions and meeting the real-time requirements of online scheduling systems. In summary, this invention enhances the schedulability protection capability for future critical jobs while ensuring the online scheduling efficiency of heterogeneous GPU clusters, and balances fairness with scheduling overhead, demonstrating good practical value and promising prospects for widespread application.

[0086] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific details of the above embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all fall within the protection scope of the present invention.

Claims

1. A method for protecting the schedulability of future critical jobs in online heterogeneous GPU cluster scheduling, characterized in that, include: Construct cluster resource status data for heterogeneous GPU clusters, and build template data for future critical job requirements based on historical load characteristics and current job queuing status; For the current job to be scheduled, several candidate placement scheme data are generated, and the damage of each candidate placement scheme to the schedulable space of the future critical job is evaluated based on the future critical job requirement template data, so as to obtain the schedulability damage score data of each candidate scheme. Based on the future critical operation requirement template data and the cluster resource status data, identify and mark the critical bearer spaces in the cluster, and generate critical bearer space marking data. Collect current scheduling queue status data, calculate cluster congestion status index data, and adaptively adjust the protection strength of critical bearer space according to the congestion level to generate adaptive scheduling parameter data. Based on the schedulability damage score data, critical bearing space marking data, adaptive scheduling parameter data, and current job placement cost, a comprehensive score is calculated for each candidate placement scheme, and the scheme with the best comprehensive score is selected for scheduling.

2. The method according to claim 1, characterized in that, The cluster resource status data for constructing a heterogeneous GPU cluster includes: Collect static configuration data of each physical node in the cluster, including node type, total number of GPUs, total video memory capacity of each GPU, and topological connection relationship between GPUs, to obtain node static resource description data; Collect real-time status data of each physical node, including the current remaining video memory capacity of each GPU and the resource usage of running jobs, to obtain real-time resource usage data of the node; The static resource description data and real-time resource usage data of the nodes are correlated and aggregated to generate comprehensive resource view data for each node. The comprehensive resource view includes node type, GPU topology grouping, remaining video memory of each GPU, set of idle GPUs, and information on continuous idle GPU space. By aggregating the comprehensive resource view data of all nodes, a structured cluster resource status model data is obtained.

3. The method according to claim 2, characterized in that, Construct template data for future critical operational requirements, including: Collect historical job arrival logs from the cluster, resource demand data of jobs waiting in the current scheduling queue, and business scenario tag data, extract features, and obtain job load feature data. The workload characteristic data is analyzed and processed to identify key job types such as large-scale GPU jobs, high memory jobs, and compact topology jobs. Several key job template data are generated, and each key job template data records the job type, the range of required number of GPUs, the threshold of single GPU memory requirement, and the topology compactness requirement. Based on the current cluster congestion level and business priority, dynamic weight data is assigned to each critical task template data to obtain future critical task demand model data with weight attributes.

4. The method according to claim 3, characterized in that, The impairment of each candidate placement scheme on the schedulable space of future critical operations is evaluated to obtain schedulability impairment score data for each candidate scheme, including: Obtain the resource requirement data of the currently scheduled jobs, and generate several candidate placement schemes that meet the resource requirements based on the cluster resource status model data; For each candidate placement scheme, a simulated placement process is performed. The GPU and video memory resources that the current job will occupy are subtracted from the cluster resource status model data to generate the future immediate resource status snapshot data corresponding to the scheme. Using the future critical task demand model data with weighted attributes, the schedulability of each future immediate resource status snapshot data is determined, and each critical task template is checked one by one to see whether it can still be successfully scheduled under the current resource snapshot. The degree of decrease in the schedulability of critical tasks before and after placement is calculated. The results are then weighted and summed based on the dynamic weights of each critical task template to quantify the schedulability impairment score for each candidate placement scheme.

5. The method according to claim 4, characterized in that, Identify and label key bearer spaces in the cluster, and generate key bearer space labeling data, including: Based on the key operation templates in the future key operation requirement model data, perform spatial carrying capacity analysis on the cluster resource status model data; Resource regions capable of accommodating any high-weight critical job template without conflict are identified as critical carrying spaces. These resource regions include completely idle physical nodes, GPU nodes with remaining video memory above the high video memory threshold, and complete GPU topology groups. For each identified key bearer space, tag data is generated, which includes space type, node to which it belongs, set of GPUs occupied, and corresponding protection level.

6. The method according to claim 5, characterized in that, Calculate cluster congestion status index data, and adaptively adjust the protection strength of critical bearer space according to the congestion level to generate adaptive scheduling parameter data, including: Collect the queue length data of the current scheduling queue, the waiting time data and the expected timeout time data of each job in the queue, perform statistical processing, and obtain the average waiting time, the ratio of the longest waiting time to the timeout threshold, and congestion status index data of the queue backlog. The congestion status index data is compared with a preset first threshold and a second threshold to determine the current congestion level; When the congestion level is below the first threshold, adaptive scheduling parameter data is generated to enhance the protection strength of critical bearer space. The parameters include increasing the penalty coefficient for occupying critical bearer space in the comprehensive score. When the congestion level is higher than the second threshold, adaptive scheduling parameter data is generated to reduce the protection strength of critical bearer space and increase the priority of long-waiting jobs.

7. The method according to claim 6, characterized in that, Calculate the overall score for each candidate placement scheme, and select the scheme with the best overall score for scheduling, including: Acquire schedulability damage score data for each candidate placement scheme, marking data on whether the scheme occupies critical carrying space, current job placement cost data, and adaptive scheduling parameter data; The above data is weighted and fused, and the weight coefficients of the weighted fusion are adjusted according to the protection strength and priority in the adaptive scheduling parameter data to generate the final comprehensive score data for each candidate placement scheme. The comprehensive score data of all candidate placement schemes are sorted, and the scheme with the best comprehensive score is selected as the scheduling decision result. Based on the node and GPU set specified in the scheduling decision, a resource allocation instruction is generated and sent to the cluster to place the current job.

8. The method according to claim 7, characterized in that, After the current job placement is completed, the process also includes status updates and subsequent scheduling triggering steps: Update the cluster resource status model data according to the executed resource allocation instructions, and deduct the GPU and video memory resources occupied by the current job; Check the scheduling waiting queue; if the waiting queue is not empty, trigger the execution of the method steps of any one of claims 1 to 7 for the next job to be scheduled. If the waiting queue is empty, the system enters a listening state to wait for a new job to arrive, and re-executes the method steps of any one of claims 1 to 7 when a new job arrives.

9. The method according to claim 1, characterized in that, The key job types in the future key job requirement template data include large-scale GPU jobs, high memory jobs, and compact topology jobs. Specifically, large-scale GPU jobs are characterized by the number of GPUs required exceeding a first threshold, high memory jobs are characterized by the memory requirement of a single GPU exceeding a second threshold, and compact topology jobs are characterized by the requirement to allocate resources within consecutive GPUs on the same node or within a specific topology group.

Images