A method and system for scheduling optimization of heterogeneous GPU clusters based on resource affinity

Abstract

This invention relates to a method and system for scheduling optimization of heterogeneous GPU clusters based on resource affinity, belonging to the field of computer resource scheduling. It includes: Step 1: Based on a historical database of job affinity, matching historical jobs with high similarity through job features to estimate resource requirements; Step 2: For new jobs, receiving the job and performing probe tests on each type of GPU to acquire and record job performance data; Step 3: Using historical information or test data, establishing an affinity evaluation model for each job to evaluate the job's affinity value under different resource configurations, providing data support for resource allocation decisions; Step 4: Entering a scheduling optimization cycle, constructing an allocation matrix based on the objective function of the optimization problem, and allocating corresponding resources to each job. This invention provides accurate resource matching and data support, ensures dynamic adaptability, and enhances system scalability.

Description

Technical Field

[0001] This invention relates to the field of computer resource scheduling, particularly GPU resource scheduling technology in deep learning model training. Specifically, this invention proposes a method and system for optimizing the scheduling of heterogeneous GPU clusters based on resource affinity. Background Technology

[0002] In recent years, the rapid development of deep learning technology has spurred a huge demand for high-performance computing resources. GPUs (Graphics Processing Units), with their powerful parallel computing capabilities, have become the core hardware for accelerating deep learning model training. However, as the complexity and scale of deep learning training tasks increase, a single type of GPU (homogeneous cluster) is no longer sufficient to meet performance requirements. Heterogeneous GPU clusters, as a computing platform that includes multiple GPU hardware (such as NVIDIA V100, A100, T4, etc.), can provide greater flexibility and computational efficiency for deep learning model training, but also bring challenges in areas such as resource heterogeneity awareness, task allocation, and scheduling efficiency.

[0003] Traditional scheduling algorithms often assume that GPUs in a cluster have similar performance characteristics, failing to adequately consider the heterogeneity between GPUs. This leads to problems such as low resource utilization and inefficient task execution, making it difficult to achieve optimal performance in heterogeneous environments. Specific problems include:

[0004] 1) Insufficient resource awareness: It cannot effectively capture the performance differences between heterogeneous GPUs, resulting in low resource utilization.

[0005] 2) High scheduling overhead: Under high dynamic load, existing scheduling algorithms are unable to respond quickly to changes in resource status.

[0006] 3) Poor load balancing: Training jobs may not be adequately matched to GPU performance, which may cause some GPUs to be overloaded while others are idle.

[0007] Therefore, how to achieve efficient resource awareness and intelligent scheduling in heterogeneous GPU clusters has become a key technical problem that urgently needs to be solved. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this invention provides a method and system for optimizing the scheduling of heterogeneous GPU clusters based on resource affinity;

[0009] The core objective of this invention is to design an innovative resource affinity scheduling method and system for heterogeneous GPU clusters. By deeply analyzing the heterogeneous characteristics of GPUs, carefully constructing a precise and efficient resource affinity scheduling algorithm, ingeniously structuring a hierarchical collaborative scheduling framework, and organically integrating a resource scheduling optimization mechanism, this invention achieves highly intelligent adaptation and efficient utilization of training tasks and GPU resources, significantly improving the overall performance, resource utilization, and energy efficiency of the cluster, and providing strong support for training complex models.

[0010] The technical solution of this invention is as follows:

[0011] A resource affinity-based scheduling optimization method for heterogeneous GPU clusters includes:

[0012] Step 1: Based on the affinity history database of historical tasks, match historical tasks with high similarity by task features to estimate resource requirements;

[0013] Step 2: For new jobs, receive the job and perform probe tests on each type of GPU to obtain and record job performance data;

[0014] Step 3: Using historical information or test data, establish an affinity assessment model for each task, evaluate the affinity value of the task under different resource configurations, and provide data support for resource allocation decisions;

[0015] Step 4: Enter the scheduling optimization cycle, construct the allocation matrix according to the objective function of the optimization problem, and allocate corresponding resources to each job.

[0016] Specifically, a resource affinity-based method for optimizing scheduling in heterogeneous GPU clusters includes:

[0017] Step S101: Record historical jobs; For clusters with accumulated historical jobs, collect historical job information from the cluster over a period of time in advance; including: job characteristic information, execution performance data on different GPUs, and the final resource allocation scheme, to form a historical job library;

[0018] The recorded job feature information is used to match newly submitted jobs by users and is recorded in numerical form; while the execution performance data and resource allocation scheme provide a basis for preliminary resource allocation for similar jobs; at the same time, historical jobs are classified according to job feature information and resource allocation scheme to form an affinity history database.

[0019] Step S102: Receive job; Receive newly submitted jobs from users;

[0020] Step S103: Job feature matching; Upon receiving a newly submitted job from the user, the job feature matching process is immediately initiated to compare the feature information of the newly submitted job with the jobs in the affinity history database.

[0021] Step S104: Preliminary resource allocation for similar jobs; After finding the job with the highest similarity, based on the execution information and resource configuration of the job with the highest similarity, make a preliminary estimate of the resource requirements of the current job; list the resource configuration of the job with the highest similarity as potential adaptable resources and perform preliminary resource allocation for the current job; at the same time, obtain the historical affinity data of the job with the highest similarity;

[0022] Step S105: Start probe testing;

[0023] Step S106: Scheduling optimization loop; After determining the matching relationship between the newly submitted job and the GPU, the job enters the scheduling cycle loop to continuously optimize resource allocation.

[0024] According to a preferred embodiment of the present invention, job feature matching includes:

[0025] Step S1031: Extract job features; Extract feature information for the received new job. Job feature information includes: model features, data features, load characteristics and accuracy requirements. Data features include data type and data size.

[0026] Step S1032: Traverse the historical job database; query the recorded historical jobs in the cluster, and execute step S1033 until the historical job most similar to the received new job is found;

[0027] Step S1033: Similarity matching;

[0028] Based on the feature-based vector representation, the job features extracted in step S1031 are transformed into numerical representations, and each job feature is merged into a vector representation; model features use one-hot encoding to represent model type, data features are numerically represented, load characteristics are classified through category encoding, and accuracy requirements are mapped to numerical values ​​for scalar representation.

[0029] Calculate the similarity between the current assignment and historical assignments; use weighted similarity, weight each feature according to its importance, calculate the weighted similarity score for each assignment, and find the assignment with the highest similarity.

[0030] Step S1034: Determine if the most similar job exists; After calculating the similarity score, set a similarity threshold to determine if the most similar job exists; If the similarity between the current job and historical jobs exceeds the similarity threshold, it is considered that the most similar job exists, otherwise it does not exist;

[0031] If the most similar job exists, load the cache. The process of obtaining the resource allocation scheme of the job with the highest similarity to the current job from the historical job database as a reference for preliminary resource allocation means that the resource allocation scheme of the job with the highest similarity to the current job is directly assigned to the current job.

[0032] If no similar job exists, the job is considered an unknown job, a probe test is initiated and the actual execution performance of the job is recorded to achieve preliminary resource matching and provide data support for affinity matching.

[0033] In a further preferred embodiment, in step S1031, for model features, the L0 model is obtained based on the model definition of the job submission script or framework, and the approximate job category is determined based on the basic model architecture information.

[0034] Based on the analysis of job configuration information, determine its load characteristics;

[0035] The accuracy requirements are determined based on the settings in the model framework.

[0036] A further preferred approach is to use weighted similarity, weighting each feature according to its importance, calculating a weighted similarity score for each job, and identifying the job with the highest similarity; including:

[0037] For model features, calculate cosine similarity. ;

[0038] For data types, calculate cosine similarity. ;

[0039] For data size, calculate Euclidean distance. ;

[0040] For load characteristics, calculate Jaccard similarity. ;

[0041] For accuracy requirements, calculate the Euclidean distance. ;

[0042] Comprehensive similarity calculation: The similarity of each feature is calculated according to a preset weight. The weighted sum is then used to obtain the total similarity score: .

[0043] More preferably, in step S105, the probe test is initiated; including:

[0044] Step S1051: Receive unknown job;

[0045] Step S1052: Perform performance testing;

[0046] Unknown jobs were deployed to different types of GPUs in the cluster for small-scale trial runs. For each GPU type, multiple different combinations of running parameters were set during the test. In the deep learning training job scenario, the parameters included batch size and gradient accumulation steps. Under each parameter combination including batch size and gradient accumulation steps, the execution performance data of the job was strictly monitored and collected. This included job computing power requirements, GPU memory usage, amount of data processed, data transfer time, job running time, actual throughput, GPU resource utilization, and power consumption.

[0047] Step S1053: Record performance data, i.e., execution performance data;

[0048] Step S1054: Determine the initial match; calculate the actual execution efficiency of the job based on the performance data recorded in step S1053;

[0049] Actual execution efficiency The calculation formula is as follows:

[0050] ;

[0051] in, Indicates the throughput of the job. This represents the throughput after job normalization; This indicates the overall utilization rate of the GPU; These are weighting coefficients, used to balance performance indicators;

[0052] The actual execution performance of unknown jobs on different GPUs is sorted from high to low, compared and filtered, and the highest execution performance is taken as the optimal value. The selected optimal GPU resources are used as the initial resource matching scheme for unknown jobs.

[0053] According to a preferred embodiment of the present invention, in step S106, the scheduling optimization loop includes:

[0054] Step S1061: Initial allocation exploration;

[0055] For unknown jobs, in the initial stage of scheduling, it is assumed that the throughput of unknown jobs increases linearly with the amount of resources; that is:

[0056] ;

[0057] in, This represents the throughput of the job with a configuration including n GPUs, t GPU type, t batch size, and t gradient accumulation steps;

[0058] Step S1062: Throughput model estimation;

[0059] Collection jobs on different numbers of GPUs and type The actual throughput data is used to fit the model parameters. Gradient computation time and gradient synchronization time :in, This represents the fixed time overhead during gradient calculation, independent of the model size. This indicates that gradient computation time varies with model size. The proportional coefficient for linear growth; This represents the fixed time overhead during gradient synchronization, independent of the number of GPUs. This indicates that gradient synchronization time varies with the number of GPUs. The growth rate coefficient;

[0060] Gradient computation time: ;

[0061] Gradient synchronization time: ;

[0062] Time required for each iteration The calculations, including the time for 𝑠 gradient calculations, are performed. and a synchronization time ;

[0063] Iteration time : ;

[0064] Build a throughput model for each job;

[0065] The throughput model formula is: ;

[0066] in, Indicates the total batch size;

[0067] The actual utilization rate of modeling resources is calculated based on the ratio of effective computation time to total iteration time. :

[0068] ;

[0069] Step S1063: Affinity model evaluation;

[0070] Define a set of resources as a resource allocation tuple. ,express indivual ;

[0071] First, by quantifying the degree of fit between the GPU's floating-point computing power and the actual requirements of the job, we obtain the computing power matching degree. This includes two sub-items: basic computing power matching and architecture adaptation and correction.

[0072] ;

[0073] in, This represents the number of floating-point operations required for a single iteration of the task. This represents the theoretical peak computing power of the GPU; Indicates the architectural characteristics required for the task; Indicates the actual GPU architecture; This indicates the maximum architectural generation difference; As a sensitivity coefficient, it represents architectural sensitivity;

[0074] For basic computing power matching items ,when hour, ,when hour, ;

[0075] The efficiency of GPU memory resource utilization in quantification tasks is used to obtain memory fit, which is then used to determine the memory fit. It covers two aspects: bandwidth utilization and capacity adaptability.

[0076] ;

[0077] in, Indicates the total amount of video memory operations in a single batch; Indicates the effective bandwidth of the GPU memory; Indicates the processing time for a single batch; Indicates the peak memory requirements of the task; This represents the available video memory after deducting system-reserved memory;

[0078] By calculating the energy consumed per unit of computation, energy utilization efficiency is quantified, thereby obtaining the degree of energy efficiency optimization. :

[0079] ;

[0080] in, (This represents the actual number of operations performed by the GPU in the job). Indicates the effective computation time of the GPU; Indicates the total energy consumption of the task;

[0081] Based on actual operational data and throughput model predictions, the effective capacity under actual workload is quantified, thereby obtaining execution efficiency. :

[0082] ;

[0083] Among them, throughput As a core indicator for measuring operational efficiency, reflecting the amount of work completed per unit time, the resource utilization rate correction factor... The actual utilization rate of resources is measured by the ratio of effective calculation time to total iteration time.

[0084] Taking into account hardware procurement or leasing costs and operating costs Evaluate the cost-effectiveness of different types and quantities of GPUs. :

[0085] ;

[0086] For each job, an affinity evaluation model for the job and GPU resources is built under different configuration scenarios to quantitatively evaluate the compatibility between job tasks and GPU resources:

[0087] ;

[0088] in, Indicate homework With resource allocation scheme Multidimensional affinity value; This is a normalization process based on the Sigmoid function; As a dynamic weighting coefficient;

[0089] Based on the evaluation data from the affinity assessment model, an affinity matrix is ​​constructed. Record different jobs under different resource configuration schemes The affinity of the lower body Indicate homework In configuration The affinity value is below;

[0090] Step S1064: Global scheduling optimization;

[0091] Specifically, during the global resource scheduling process, a search algorithm is run according to a predetermined scheduling cycle to generate a series of... A binary allocation matrix, where each matrix represents a possible resource allocation scheme, is defined as a candidate allocation matrix. , For the number of assignments, The number of resource allocation schemes, the elements in the matrix Used to indicate whether it is a task Select configuration For candidate allocation matrix The following constraints must be met:

[0092] Assignment uniqueness: Each line contains at most one 1, and each job is assigned one configuration; that is: ;

[0093] Resource capacity limit: The total resource allocation requirements of all allocated resources ≤ the total physical resources of the cluster;

[0094] The goal of global scheduling is to achieve globally optimal scheduling, that is, to obtain the optimal affinity allocation matrix. , Indicate homework The basic affinity gain obtained; the objective function of the optimization problem is:

[0095] ;

[0096] in, Indicate homework The basic affinity gains obtained; As a fairness adjustment factor, a generalized power average is used to achieve multi-objective balance; Indicates the redistribution penalty factor;

[0097] Set a redistribution penalty factor The reassignment overhead is increased based on the historical reassignment frequency of jobs, and is designed as an exponentially decaying function. :

[0098] ;

[0099] in, Indicate homework The number of historical redistributions; Indicates the time interval between the two most recent reallocations; Indicate homework The average operating cycle; Indicates the penalty intensity coefficient;

[0100] Global scheduling optimization transforms the scheduling problem into an optimization problem by maximizing the sum of affinity values ​​for jobs in the cluster. The objective function is:

[0101] ;

[0102] in, As a fairness adjustment factor, a generalized power average is used to achieve multi-objective balance: when When it degenerates into linear summation, prioritizing pure efficiency, resources are tilted towards tasks with higher affinity; when Approaching the geometric mean, prioritizing fairness, and emphasizing the fairness of resource allocation; Considering maximum and minimum fairness;

[0103] Step S1065: Job-level scheduling optimization; the optimization goal is to find the optimal batch size. and gradient accumulation steps :

[0104] ;

[0105] Step S1066: Job execution;

[0106] Step S1067: Parameter fitting; data is continuously collected during the operation. Perform parameter fitting on the throughput model;

[0107] Based on the throughput data recorded in step S1065, the model parameters are fitted using the L-BFGS-B optimization algorithm.

[0108] Using root mean square logarithmic error as the loss function :

[0109] ;

[0110] Where n is the number of samples. This is the actual measurement time. It is the predicted iteration time;

[0111] By minimizing this loss function, the model parameters are adjusted. This makes the prediction iteration time It is closer to the actual time;

[0112] Based on the optimized throughput model, solid data support is provided for resource allocation decisions, thus forming a scheduling optimization loop.

[0113] A resource affinity-based heterogeneous GPU cluster scheduling system includes:

[0114] The similar job matching module is configured to: match historical jobs with high similarity based on job features in a history database of historical job affinity, and estimate resource requirements.

[0115] The probe testing module is configured to: receive new jobs and perform probe tests on each type of GPU, and acquire and record job performance data;

[0116] The affinity assessment module is configured to: use historical information or test data to build an affinity assessment model for each job, evaluate the affinity value of the job under different resource configurations, and provide data support for resource allocation decisions;

[0117] The job scheduling module is configured to: enter a scheduling optimization cycle loop, construct an allocation matrix based on the objective function of the optimization problem, and allocate corresponding resources to each job.

[0118] According to a preferred embodiment of the present invention, the heterogeneous GPU cluster scheduling system further includes a performance monitoring module, which is configured to: construct a full-stack monitoring system, collect multi-dimensional indicators during job execution in real time, detect abnormal fluctuations, and generate alarm events.

[0119] Compared with the prior art, the present invention has the following beneficial effects:

[0120] 1) Precise Resource Matching and Data Support: Through an affinity matching model between jobs and GPU resources, the system comprehensively considers job characteristics and GPU performance to achieve the best match between jobs and hardware resources, fully utilizing the heterogeneous characteristics of GPU clusters. The throughput model can predict task performance data under different GPU configurations, providing more scientific and comprehensive data support for resource affinity allocation and effectively avoiding blind and inefficient resource allocation.

[0121] 2) Dynamic Adaptability Guarantee: The periodic scheduling mechanism can adjust the resource allocation of jobs in real time according to the dynamic changes in task load, ensuring that the system maintains optimal performance under various workloads. By introducing a scheduling cycle, the affinity function is evaluated in real time during job execution, and the optimal resource allocation matrix is ​​generated in a timely manner to dynamically adjust resource allocation and ensure optimal overall cluster performance.

[0122] 3) Enhanced System Scalability: The hierarchical collaborative scheduling framework combines global scheduling and job-level scheduling, reducing the complexity of the scheduling process and significantly alleviating the performance bottlenecks and overhead of centralized scheduling. It optimizes resource allocation between jobs at the global level and fine-grainedly adjusts task parameters at the job level, achieving effective decomposition of scheduling responsibilities, improving system scalability and scheduling efficiency, and better adapting to the needs of large-scale heterogeneous GPU clusters. Attached Figure Description

[0123] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0124] Figure 1 This is a flowchart illustrating the overall process of the resource affinity-based heterogeneous GPU cluster scheduling optimization method provided in Embodiment 2 of the present invention.

[0125] Figure 2 This is a schematic diagram of the historical job matching process based on similarity provided in Embodiment 2 of the present invention;

[0126] Figure 3This is a schematic diagram of the actual performance probe test process provided in Embodiment 2 of the present invention;

[0127] Figure 4 This is a flowchart of job cycle scheduling optimization based on a hierarchical architecture provided in Embodiment 2 of the present invention;

[0128] Figure 5 This is an overall architecture diagram of the heterogeneous GPU cluster scheduling system based on resource affinity provided in Embodiment 3 of the present invention. Detailed Implementation

[0129] The present invention will be further defined below with reference to the accompanying drawings and embodiments, but is not limited thereto.

[0130] Example 1

[0131] A resource affinity-based scheduling optimization method for heterogeneous GPU clusters includes:

[0132] Step 1: Based on the affinity history database of historical tasks, match historical tasks with high similarity by task features to estimate resource requirements;

[0133] Step 2: For new jobs, receive the job and perform probe tests on each type of GPU to obtain and record job performance data;

[0134] Step 3: Using historical information or test data, establish an affinity assessment model for each task, evaluate the affinity value of the task under different resource configurations, and provide data support for resource allocation decisions;

[0135] Step 4: Enter the scheduling optimization cycle, construct the allocation matrix according to the objective function of the optimization problem, and allocate corresponding resources to each job.

[0136] Example 2

[0137] The difference between the resource affinity-based heterogeneous GPU cluster scheduling optimization method described in Example 1 and the following is:

[0138] A resource affinity-based scheduling optimization method for heterogeneous GPU clusters, referencing Figure 1 ,include:

[0139] Step S101: Record historical jobs; For clusters with a certain accumulation of historical jobs, collect historical job information of the cluster within a certain period of time in advance; The historical job database records various types of information of past jobs in detail, including: job feature information (job feature information includes model features, data features, load characteristics and accuracy requirements), execution performance data on different GPUs (such as throughput, execution time, resource utilization and power consumption) and the final resource allocation scheme, forming the historical job database;

[0140] Recorded job feature information is used to match newly submitted jobs by users and is recorded in numerical form; while execution performance data and resource allocation schemes provide a basis for preliminary resource allocation for similar jobs; at the same time, historical jobs are classified according to job feature information and resource allocation schemes to form an affinity history database; the classification is based on job feature information; the affinity history database records the resource allocation schemes of jobs so that initial resource allocation can be quickly performed when similar jobs are scheduled in the future.

[0141] Step S102: Receive job; Receive newly submitted jobs from users; This is the beginning of the actual scheduling process.

[0142] Step S103: Job feature matching; Upon receiving a newly submitted job from the user, the job feature matching process is immediately initiated to compare the feature information of the newly submitted job with the jobs in the affinity history database.

[0143] Step S104: Preliminary resource allocation for similar jobs; After finding the job with the highest similarity, make a preliminary estimate of the current job's resource requirements based on the execution information and resource configuration of the job with the highest similarity; list the resource configuration of the job with the highest similarity as potential adaptable resources and make a preliminary resource allocation for the current job; at the same time, obtain the historical affinity data of the job with the highest similarity; as an important reference for subsequent precise resource allocation and optimization.

[0144] Step S105: Initiate probe testing; for new jobs that cannot find enough similar jobs in the affinity history database, conduct a comprehensive actual execution performance test mechanism.

[0145] Step S106: Scheduling optimization loop; After determining the matching relationship between the newly submitted job and the GPU, the job enters the scheduling cycle loop to continuously optimize resource allocation.

[0146] like Figure 2 As shown, job feature matching includes:

[0147] Step S1031: Extract Job Features; Extract feature information from the received new job. Job feature information includes: model features, data features, load characteristics, and accuracy requirements. Data features include data type and data scale (e.g., number of samples, total data size); Model features: Indicate the type of model architecture used by the job. Based on the model features, its L0 model (i.e., the base model) can be determined, thus identifying the approximate category of the job (e.g., image processing, natural language processing); Data features: Indicate the data type and data scale used by the job; Load characteristics: The inherent characteristics of deep learning job load, indicating the job's requirements for computing power, memory resources, and communication capabilities; Accuracy requirements: Indicate the accuracy requirements of the job in numerical computation; This can be obtained by parsing the job configuration file.

[0148] Step S1032: Traverse the historical job database; query the recorded historical jobs in the cluster, and execute step S1033 until the historical job most similar to the received new job is found;

[0149] Step S1033: Similarity matching;

[0150] Based on the feature-based vector representation, the job features extracted in step S1031 are transformed into numerical representations, and each job feature is merged into a vector representation; model features use one-hot encoding to represent model type, data features are numerically represented, load characteristics are classified through category encoding, and accuracy requirements are mapped to numerical values ​​for scalar representation.

[0151] Model features are represented using one-hot encoding to indicate model type. This means that one-hot encoding is used. First, all possible model categories are listed. Then, a unique index is assigned to each model type. Finally, a one-hot vector is created, where the dimension corresponding to the job's model type has a value of 1, and the other dimensions have values ​​of 0. For example, if there are three model types: CNN, RNN, and Transformer, and a job's model type is RNN, then the model feature encoding for that job would be [0, 1, 0].

[0152] The numerical representation of data features refers to: (1) Data type One-Hot encoding: listing all possible data types, assigning a unique index to each data type, and performing One-Hot encoding. (2) Data scale encoding: collecting data scale information of the job (number of samples, total amount of data), performing min-max normalization on the data scale, and mapping the values ​​to the [0,1] interval.

[0153] Load characteristics are categorized by type encoding, which means using multi-tag encoding. Load types are defined (compute-intensive, memory-intensive, communication-intensive, I / O-intensive), each type is assigned a unique index, and multi-tag encoding is performed, with the position corresponding to the load type set to 1, and the rest set to 0.

[0154] The accuracy requirements are mapped to numerical values ​​using scalar representation, meaning that a numerical mapping is employed. A numerical value is assigned to each accuracy level (FP16→0.5, FP32→1.0, FP64→2.0), mapping the accuracy requirements of the task to the corresponding numerical value.

[0155] Calculate the similarity between the current assignment and historical assignments; use weighted similarity, weight each feature according to its importance, calculate the weighted similarity score for each assignment, and find the assignment with the highest similarity.

[0156] Step S1034: Determine if the most similar job exists; After calculating the similarity score, set a similarity threshold. This threshold determines the degree of "similarity" during the matching process. The specific threshold should be adjusted according to actual needs. High-precision matching: threshold set above 0.9; Medium-precision matching: threshold set between 0.7 and 0.9; Low-precision matching: threshold set between 0.5 and 0.7. This is used to determine if the most similar job exists; if the similarity between the current job and historical jobs exceeds this similarity threshold, then the most similar job exists; otherwise, it does not.

[0157] If the most similar job exists, load the cache. The process of obtaining the resource allocation scheme of the job with the highest similarity to the current job from the historical job database as a reference for preliminary resource allocation means that the resource allocation scheme of the job with the highest similarity to the current job is directly assigned to the current job.

[0158] If no similar job exists, the job is considered an unknown job, a probe test is initiated, and the actual execution performance of the job is recorded to achieve preliminary resource matching and provide data support for affinity matching. (See Probe Test for details.) Figure 3 Steps S1051-S1054 involve running tests on unknown jobs under different resource configurations, obtaining and recording actual job running data (all running data are data required for affinity assessment, including: job computing power requirements, GPU memory usage, data processing volume, data transmission time, job running time, actual throughput, GPU resource utilization, and running power consumption).

[0159] In step S1031, for model features, the L0 model is obtained based on the model definition of the job submission script or framework, and the approximate job category is determined based on the basic model architecture information.

[0160] The model feature extraction process is as follows:

[0161] (1) Parse the job submission script to identify the framework (such as TensorFlow, PyTorch) and model definition file used;

[0162] (2) Extract the architecture information of the model based on the model definition file and identify the L0 model (i.e., the basic model architecture).

[0163] (3) Based on the extracted L0 model, map it to the predefined job category.

[0164] The L0 model refers to the basic model architecture used in a task, that is, the lowest-level structure or original architecture of the model. For example, in image processing tasks, L0 models that may be used include ResNet, VGG, and EfficientNet; in natural language processing tasks, L0 models that may be used include BERT, GPT, and Transformer.

[0165] Based on the model characteristics, the approximate category of the task was determined.

[0166] Based on the analysis of job configuration information, determine its load characteristics;

[0167] Load characteristics refer to the computational resource requirements of a deep learning job, including computing power, GPU memory resources, communication capabilities, and I / O capabilities. The method for extracting load characteristics is based on the job's configuration (job type, framework used, model structure), and the specific implementation steps are as follows:

[0168] (1) Parse the job submission script, configuration file or model definition, and extract relevant information;

[0169] (2) Identify the key characteristics of the task, including the type of model used, the data scale, and the training method;

[0170] (3) Map the identified features to the corresponding load types according to predefined rules;

[0171] (4) Since the job may have multiple load characteristics at the same time, a multi-label encoding method is used for numerical representation.

[0172] Detailed explanation:

[0173] (1) Computationally intensive: Uses complex model structures (such as deep neural networks) and a large number of mathematical operations. This is determined by analyzing the depth and complexity of the model structure. Example: The assignment uses the ResNet-152 model for image classification, and the training process mainly involves forward propagation and backpropagation calculations.

[0174] (2) Memory-intensive: Large number of model parameters, high dimensionality of input data, and large batch size. This is determined by analyzing the number of model parameters and the dimensionality of input data. Example: The assignment uses the GPT-3 model for text generation, and the long length of the input sequence leads to high memory requirements.

[0175] (3) Communication-intensive: Employs distributed training methods, frequently performs parameter synchronization and gradient exchange, and is sensitive to network bandwidth and latency. This is determined by analyzing whether the job uses a distributed training framework. Example: The job uses the Horovod framework for multi-node distributed training, requiring frequent AllReduce operations.

[0176] (4) I / O intensive: Frequent reading of large data files, network communication, or database access. This is determined by analyzing the data loading method and data size. Example: The job uses a large dataset, requiring frequent reading of large-scale image datasets from the disk during training, causing I / O to become a bottleneck.

[0177] The accuracy requirements are determined based on the settings in the model framework.

[0178] The required numerical accuracy for extraction tasks is achieved through settings within the analysis model framework. The specific implementation process is as follows:

[0179] (1) Parse the job submission script or model definition file and identify the deep learning framework used (such as TensorFlow or PyTorch).

[0180] (2) Based on the specific API of the framework, identify the job accuracy strategy and determine the numerical calculation accuracy requirements of the job, such as FP16, FP32, FP64;

[0181] (3) Map to numerical representation, assign a numerical value to each precision level (FP16→0.5, FP32→1.0, FP64→2.0), and map the identified precision requirements to numerical representation to facilitate subsequent similarity calculation.

[0182] Weighted similarity is used, where each feature is weighted according to its importance, and a weighted similarity score is calculated for each job to identify the job with the highest similarity. This includes:

[0183] When calculating the similarity between the current task and historical tasks, for each task feature, after converting it into a numerical representation, an appropriate similarity measurement method is used: one-hot encoding uses cosine similarity, multi-label encoding uses Jaccard similarity, numerical feature mapping is normalized, and Euclidean distance is used to measure the difference.

[0184] For model features (one-hot encoded), calculate cosine similarity. A and B are the model feature vectors for the current job and historical jobs, respectively. .

[0185] Example: Assume there are three model types: ResNet, BERT, and Transformer. The current job uses BERT, with One-Hot encoding of [0, 1, 0]; previous jobs used ResNet, with One-Hot encoding of [1, 0, 0]. Cosine similarity:

[0186] ;

[0187] For data types (one-hot encoded), calculate cosine similarity. A and B are data type vectors for the current job and historical jobs, respectively. .

[0188] For data size (numerical characteristics), calculate the Euclidean distance. A and B represent the data size of the current job and historical jobs, respectively, while max and min represent the maximum and minimum data sizes of all jobs. .

[0189] Example: Assume the data size ranges from 10GB to 100GB. The current job's data size is 50GB, and the historical job's data size is 70GB. Calculate the Euclidean distance. :

[0190] ;

[0191] For the payload characteristics (multi-label encoding), calculate the Jaccard similarity. A and B are the load characteristic sets of the current job and the historical job, respectively. .

[0192] Example: The current job load characteristics are {compute-intensive, memory-intensive}, and the historical jobs are {compute-intensive, communication-intensive}. Calculate similarity. :

[0193] ;

[0194] For accuracy requirements (numerical characteristics), calculate the Euclidean distance. A and B represent the accuracy requirements for the current and historical jobs, respectively, while max and min represent the maximum and minimum data sizes for all jobs. .

[0195] Comprehensive similarity calculation: The similarity of each feature is calculated according to a preset weight. The weighted sums of values ​​(e.g., 0.3, 0.2, 0.1, 0.25, 0.15) are used to obtain the total similarity score. .

[0196] In step S105, as Figure 3 As shown, initiate the probe test; including:

[0197] Step S1051: Receive an unknown job; after similarity matching, if no similar job exists, it is considered an unknown job. Receive the unknown job from step S104 and execute subsequent steps.

[0198] Step S1052: Perform performance testing;

[0199] Unknown jobs were deployed to different types of GPUs in the cluster for small-scale trial runs. For each GPU type, multiple different combinations of running parameters were set during the test. In the deep learning training job scenario, the parameters included batch size and gradient accumulation steps. Under each parameter combination including batch size and gradient accumulation steps, the execution performance data of the job was strictly monitored and collected. The test process required recording multi-dimensional performance data of the job's actual operation, including job computing power requirements, GPU memory usage, amount of data processed, data transfer time, job running time, actual throughput, GPU resource utilization, and power consumption.

[0200] The methods for obtaining each indicator are as follows:

[0201] The computational power requirement of a job is defined as the amount of floating-point operations required for a single iteration of the job. It is obtained by using DL framework analysis tools, such as PyTorch Profile.

[0202] Video memory usage, defined as the amount of video memory operations per batch and peak demand, is obtained by using the training framework memory analysis tool during job execution.

[0203] The amount of data processed is defined as the amount of data processed in job performance testing, depending on the job type, such as the number of model iterations or the number of data samples processed.

[0204] Data transfer time is defined as the time for a single batch of data transfer, which is recorded using tools such as gpustat or cudaMemcpy, and the average value is taken after multiple measurements.

[0205] Job execution time is defined as the time taken to complete a job performance test. It is recorded by a timer from job startup to completion of a specific amount of work.

[0206] Actual throughput is defined as the amount of data that can be processed per unit of time, calculated as the ratio of the amount of data processed to the completion time.

[0207] GPU resource utilization is defined as the degree to which the GPU is actually used during the test. It is obtained through GPU performance monitoring tools, and the average value is randomly queried and calculated during the test.

[0208] Operating power consumption is defined as the total energy consumption during job execution, which is recorded by a power consumption sensor.

[0209] Step S1053: Record performance data, i.e. execution performance data; record the performance data obtained through testing in detail to provide accurate data support for establishing affinity evaluation models under different configurations for subsequent jobs, and achieve optimal affinity matching of "job-resource configuration".

[0210] Step S1054: Determine the initial matching; calculate the actual execution efficiency of the job based on the performance data recorded in step S1053; quantify the effective capacity under the actual workload and determine the initial resource matching scheme for the unknown job;

[0211] Actual execution efficiency The calculation formula is as follows:

[0212] ;

[0213] in, Indicates the throughput of the job. This represents the throughput after job normalization; This indicates the overall utilization rate of the GPU; These are weighting coefficients, used to balance performance indicators;

[0214] The actual execution performance of unknown jobs on different GPUs is sorted from high to low, compared and filtered, and the highest execution performance is taken as the optimal value. The selected optimal GPU resources are used as the initial resource matching scheme for unknown jobs.

[0215] This strategy focuses on the performance of jobs during actual operation, temporarily setting aside other limiting factors such as cost and computing power matching. Its core objective is to allocate resources with superior performance to unknown jobs as much as possible, thereby providing a scientific and reliable basis for resource allocation in the initial stage of job operation, in order to improve job execution efficiency and overall performance.

[0216] like Figure 4 As shown, in step S106, the scheduling optimization loop includes:

[0217] Step S1061: Initial allocation exploration;

[0218] For unknown jobs, in the initial scheduling phase, given the lack of sufficient historical data on the actual performance of the jobs, a "perfect scaling" assumption is adopted for resource allocation exploration, that is, it is assumed that the throughput of unknown jobs increases linearly with the amount of resources; that is:

[0219] ;

[0220] in, This represents the throughput of the job with a configuration including n GPUs, t GPU type, t batch size, and t gradient accumulation steps;

[0221] Under this assumption, an increase in default resources will lead to a proportional increase in throughput, providing a reasonable basis for early resource allocation. In practice, in the initial stage, there is a tendency to allocate more GPU resources to jobs, encouraging jobs to start with a larger resource scale. This ensures that each job can fully realize its potential under expanded resources, so as to collect sufficient performance data in a resource-rich environment and provide key data support for subsequent throughput model fitting.

[0222] Step S1062: Throughput model estimation;

[0223] Collection jobs on different GPU numbers and type The actual throughput data is used to fit the model parameters. Gradient computation time and gradient synchronization time The fitted parameter set is used to estimate the gradient computation time. and gradient synchronization time .in, This refers to the fixed time overhead during gradient calculation, which is independent of the model size, such as initialization time. This indicates that gradient computation time varies with model size. The proportional coefficient for linear growth; This represents the fixed time overhead during gradient synchronization, independent of the number of GPUs. This indicates that gradient synchronization time varies with the number of GPUs. The growth rate coefficient;

[0224] Gradient computation time: ;

[0225] Gradient synchronization time: ;

[0226] Time required for each iteration The calculations, including the time required for the nth gradient calculation, are performed. and a synchronization time ;

[0227] Iteration time : ;

[0228] Build a throughput model for each job;

[0229] The throughput model formula is: ;

[0230] in, This indicates the total batch size; it determines the amount of work completed in each iteration.

[0231] The actual utilization rate of modeling resources is calculated based on the ratio of effective computation time to total iteration time. :

[0232] ;

[0233] By modeling multiple parameters, the throughput model can predict the performance of a job under different numbers of GPU configurations and provide accurate data support for resource allocation.

[0234] Step S1063: Affinity model evaluation;

[0235] Based on historical job information or relevant data collected by probe tests, an affinity assessment model is built for each job under different configuration conditions.

[0236] Define a set of resources as a resource allocation tuple. ,express indivual This determines the resource environment in which the job runs. In deep learning model training scenarios, the affinity evaluation process considers multi-dimensional matching to accurately determine the "comprehensive affinity" between the job and different resource configurations, thereby achieving the optimal match between the job and GPU resources.

[0237] First, to prevent resource waste and performance bottlenecks caused by excessive or insufficient computing power, the degree of matching between the GPU's floating-point computing power and the actual job requirements is quantified to obtain the computing power matching degree. This includes two sub-items: basic computing power matching and architecture adaptation and correction.

[0238] ;

[0239] in, This represents the number of floating-point operations required for a single iteration of the task. This represents the theoretical peak computing power of the GPU; Indicates the architectural characteristics required for the task; Indicates the actual GPU architecture; This indicates the maximum architectural generation difference; As a sensitivity coefficient, it represents architectural sensitivity;

[0240] For basic computing power matching items ,when hour, To prevent a situation of excessive computing power; when hour, To prevent insufficient computing power. Architecture adaptation and correction items. By adapting the architecture generational mapping, the correction factor will significantly reduce the score when the architecture differences are large.

[0241] The efficiency of GPU memory resource utilization in quantification tasks is used to obtain memory fit, which is then used to determine the memory fit. It covers two aspects: bandwidth utilization and capacity adaptability.

[0242] ;

[0243] in, Indicates the total amount of video memory operations in a single batch; Indicates the effective bandwidth of the GPU memory; Indicates the processing time for a single batch; Indicates the peak memory requirements of the task; This represents the available video memory after deducting system-reserved memory;

[0244] By calculating the energy consumed per unit of computation, energy utilization efficiency is quantified, thereby obtaining the degree of energy efficiency optimization. :

[0245] ;

[0246] in, This represents the actual number of operations performed by the GPU in the job (number of floating-point operations per second). Indicates the effective computation time of the GPU; Indicates the total energy consumption of the task;

[0247] Based on actual operational data and throughput model predictions, the effective capacity under actual workload is quantified, thereby obtaining execution efficiency. This reflects the overall performance under real workload conditions.

[0248] ;

[0249] Among them, throughput As a core indicator for measuring operational efficiency, it reflects the amount of work completed per unit of time and is a direct manifestation of effectiveness; resource utilization rate correction factor The actual utilization rate of resources is measured by the ratio of effective calculation time to total iteration time.

[0250] Taking into account hardware procurement or leasing costs and operating costs Evaluate the cost-effectiveness of different types and quantities of GPUs. :

[0251] ;

[0252] Hardware costs include hardware purchase and leasing fees and related ancillary costs; operating costs include energy consumption costs. And maintenance costs, etc.

[0253] For each job, the above-mentioned multi-dimensional matching factors are systematically considered to construct an affinity evaluation model between the job and GPU resources under different configuration scenarios, and to quantitatively evaluate the adaptability between the job task and GPU resources:

[0254] ;

[0255] in, Indicate homework With resource allocation scheme Multidimensional affinity value; This is a normalization process based on the Sigmoid function; As a dynamic weighting coefficient, it is adjusted in real time through a Bayesian optimization algorithm to adapt to affinity assessment under different demand scenarios;

[0256] Based on the evaluation data from the affinity assessment model, an affinity matrix is ​​constructed. Record different jobs under different resource configuration schemes The affinity of the lower body Indicate homework In configuration The affinity value is below;

[0257] Step S1064: Global scheduling optimization; After entering the scheduling optimization cycle, the global scheduler optimizes the allocation of resources within the entire cluster by periodically analyzing the overall resource allocation status of the cluster.

[0258] Specifically, during the global resource scheduling process, a search algorithm is run according to a predetermined scheduling cycle to generate a series of... A binary allocation matrix, where each matrix represents a possible resource allocation scheme, is defined as a candidate allocation matrix. , For the number of assignments, The number of resource allocation schemes, the elements in the matrix Used to indicate whether it is a task Select configuration For candidate allocation matrix The following constraints must be met:

[0259] Assignment uniqueness: Each line contains at most one 1, and each job is assigned one configuration; that is: ;

[0260] Resource capacity limit: The total resource allocation requirements of all allocated resources ≤ the total physical resources of the cluster;

[0261] Example: Assuming the cluster has 3 jobs and 3 resource configurations, the global scheduler generates a possible candidate allocation matrix, as shown in Table 1.

[0262] Table 1. Three tasks and three resource configurations;

[0263]

[0264] The goal of global scheduling is to achieve globally optimal scheduling, that is, to obtain the optimal affinity allocation matrix. , Indicate homework The basic affinity gain obtained; the objective function of the optimization problem is:

[0265] ;

[0266] The significance of this formula lies in obtaining the optimal allocation matrix. .in, Indicate homework The basic affinity gains obtained; As a fairness adjustment factor, a generalized power average is used to achieve multi-objective balance; Indicates the redistribution penalty factor;

[0267] To avoid performance degradation caused by frequent resource reallocation, a reallocation penalty mechanism is introduced into the global scheduling policy. When evaluating job affinity benefits, a penalty is imposed on jobs that require resource reallocation. A reallocation penalty factor is set. The reassignment overhead is increased based on the historical reassignment frequency of jobs, and is designed as an exponentially decaying function. :

[0268] ;

[0269] The penalty factor comprehensively considers the impact of historical redistribution frequency and time interval. Among them, Indicate homework The number of historical redistributions; Indicates the time interval between the two most recent reallocations; Indicate homework The average operating cycle; This represents the penalty intensity coefficient. The penalty factor discounts the basic affinity gain of the current job, and rescheduling will only be chosen if not rescheduling would significantly reduce the optimal value of the scheduling objective.

[0270] Example: Suppose a certain job parameter is ; ; ; (Medium penalty intensity). Therefore, the total penalty factor is:

[0271]

[0272] Assuming the basic affinity payoff for the task is 0.85, the discounted payoff is: As shown in Table 2:

[0273] Table 2 Exponential Decay Function Formula table;

[0274]

[0275] Global scheduling optimization transforms the scheduling problem into an optimization problem by maximizing the sum of affinity values ​​for jobs in the cluster. The objective function is:

[0276] ;

[0277] in, As a fairness adjustment factor, a generalized power average is used to achieve multi-objective balance: when When it degenerates into linear summation, prioritizing pure efficiency, resources are tilted towards tasks with higher affinity; when Approaching the geometric mean, prioritizing fairness, and emphasizing the fairness of resource allocation; Considering maximum and minimum fairness;

[0278] Step S1065: Job-level scheduling optimization; Job-level scheduling utilizes a lightweight agent launched with each job to monitor and optimize the performance of individual jobs. Based on existing resource allocation schemes, the throughput function of the job is fitted, and the batch size and gradient accumulation steps are dynamically adjusted to ensure that the job achieves optimal throughput performance under a given GPU resource configuration. The optimization objective is to find the optimal batch size. and gradient accumulation steps :

[0279] ;

[0280] For some users who specify that a fixed batch size should be used to run jobs, the throughput model parameter m of these jobs is fixed, and resources are allocated according to user needs to ensure that users can obtain the expected resource configuration during the training process.

[0281] Step S1066: Job Execution; After a series of resource allocation and scheduling optimizations, the job enters the formal execution state. During execution, the job continuously runs according to the determined resource allocation scheme, batch size, gradient accumulation steps, and other parameters. The time required for each iteration is recorded in real time. We also collected key variables related to this, including the number of GPUs n, batch size m, and gradient accumulation steps s, to provide data support for subsequent model parameter fitting and optimization steps.

[0282] Step S1067: Parameter fitting; The construction of the throughput model and the accuracy of model prediction depend on a set of fitting parameters: Data is continuously collected during the operation. Perform parameter fitting on the throughput model;

[0283] Based on the throughput data recorded in step S1065, the model parameters are fitted using the L-BFGS-B optimization algorithm.

[0284] Using root mean square logarithmic error (RMSLE) as the loss function :

[0285] ;

[0286] Where n is the number of samples. This is the actual measurement time. It is the predicted iteration time;

[0287] By minimizing this loss function, the model parameters are adjusted. This makes the prediction iteration time It is closer to the actual time;

[0288] After multiple iterations and optimizations, the model can accurately reflect the actual performance in the GPU cluster and accurately predict the throughput performance of jobs under different GPU counts, batch sizes, and gradient accumulation step configurations. Based on the optimized throughput model, solid data support is provided for resource allocation decisions, thus forming a scheduling optimization loop.

[0289] Throughput Model It is a core indicator for measuring the actual execution efficiency of a task (the core content of execution efficiency in the affinity assessment dimension). By constructing a throughput model, the actual execution efficiency of a task under different resource configurations can be predicted, thus providing data support for the affinity assessment between tasks and resource configurations. Then, affinity assessment is performed based on the task execution efficiency and several other dimensions. Based on the affinity scores of the task under different configurations, a two-dimensional affinity matrix is ​​compiled and constructed. Record jobs under different resource configurations The affinity of the lower, among which Indicate homework In configuration The affinity value below.

[0290] The final resource allocation scheme is the optimal allocation matrix obtained from the objective function in "global scheduling". ;

[0291] Affinity assessment is conducted across five dimensions. Execution efficiency represents the actual performance of the job under different GPU resource configurations, with throughput as the core factor, requiring separate modeling (because in actual operation, throughput does not increase exponentially with resource increases, i.e., it cannot scale perfectly). The other dimensions do not require separate modeling; relevant data can be obtained from job configuration files and performance tests, and calculations can be performed using the affinity calculation formulas for each dimension.

[0292] Example 3

[0293] A resource affinity-based heterogeneous GPU cluster scheduling system, such as Figure 5 As shown, it includes:

[0294] The similar job matching module is configured to: match historical jobs with high similarity based on job affinity history databases, and estimate resource requirements; as the system's initial processing unit, it is responsible for matching submitted jobs with historical jobs. It extracts multi-dimensional parameters such as job model architecture, computational features, and resource requirements, and performs similarity matching with the historical job database. When the matching degree reaches a preset threshold, the cache is loaded. The system invokes historical resource configuration schemes to perform initial resource allocation for jobs, while simultaneously acquiring historical affinity data for similar jobs as an important reference for subsequent precise resource allocation and optimization. If the job is a new type, it is marked as an unknown job, triggering the probe testing module to start performance testing. This module ensures that the matching strategy adaptively optimizes according to the cluster load status through a dynamic weight adjustment mechanism.

[0295] The probe testing module is configured to: receive new jobs and perform probe tests on each type of GPU, acquiring and recording job performance data; for unknown jobs or jobs with historically failed matches, generate and collect actual performance data of the job through performance testing. In a resource-isolated environment, the module performs multi-parameter combination tests on unknown jobs, covering different GPU types, batch sizes, and gradient accumulation steps, collecting key performance indicators such as throughput, memory usage, and computing power utilization. The test data uses an intelligent sampling algorithm to quickly locate the optimal parameter combination and generate a configuration table containing actual performance data for each GPU type, providing benchmark support for subsequent resource allocation. The probe testing module adopts a lightweight design to ensure minimal impact on cluster resources.

[0296] The affinity assessment module is configured to: use historical information or test data to build an affinity assessment model for each job, evaluate the affinity value of the job under different resource configurations, and provide data support for resource allocation decisions; and construct an affinity assessment system based on historical data or test results to evaluate the affinity value of the job under different resource configurations. The module outputs the assessment results to the job scheduling module, providing data support for the scheduler to generate resource allocation strategies. Simultaneously, this module utilizes feedback data from the performance monitoring module to continuously optimize the model through parameter fitting, improving model accuracy.

[0297] The job scheduling module is configured to: enter a scheduling optimization cycle, construct an allocation matrix based on the objective function of the optimization problem, and allocate corresponding resources to each job. It also generates a scheduling scheme based on the suitability report from the affinity assessment module. The module adopts a layered architecture, comprising two cooperating subsystems: global scheduling and job-level scheduling. The global scheduler operates with a periodic scheduling strategy, using a multi-objective optimization algorithm to generate a cluster-level optimal resource allocation scheme, while introducing a dynamic penalty factor to suppress resource oscillations and improve scheduling stability. The job-level scheduler, as the execution terminal, is responsible for performing fine-grained resource adjustments, translating the global strategy into specific resource configurations. By dynamically adjusting the batch size and gradient accumulation steps of jobs, it performs local optimization on individual jobs to ensure that jobs achieve optimal performance under given GPU resources.

[0298] Example 4

[0299] The difference between the heterogeneous GPU cluster scheduling system based on resource affinity described in Example 3 and the one described in Example 3 is as follows:

[0300] The heterogeneous GPU cluster scheduling system also includes a performance monitoring module, configured to: build a full-stack monitoring system, collect multi-dimensional metrics during job execution in real time, detect abnormal fluctuations, and generate alarm events. Performance monitoring is used to synchronously feed runtime data back to the throughput prediction module to calibrate the prediction model, forming a closed-loop control flow of "execution-monitoring-optimization". Simultaneously, anomaly detection algorithms identify performance deviations, report resource conflict events to the job scheduling module, and trigger resource rescheduling or parameter calibration.

[0301] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A scheduling optimization method for heterogeneous GPU clusters based on resource affinity, characterized in that, include: Step 1: Based on the affinity history database of historical tasks, match historical tasks with high similarity by task features to estimate resource requirements; Step 2: For new jobs, receive the job and perform probe tests on each type of GPU to obtain and record job performance data; Step 3: Using historical information or test data, establish an affinity assessment model for each task, evaluate the affinity value of the task under different resource configurations, and provide data support for resource allocation decisions; Step 4: Enter the scheduling optimization cycle loop, construct the allocation matrix according to the objective function of the optimization problem, and allocate corresponding resources to each job; Specifically include: Step S101: Record historical jobs; For clusters with accumulated historical jobs, collect historical job information from the cluster over a period of time in advance; including: job characteristic information, execution performance data on different GPUs, and the final resource allocation scheme, to form a historical job library; The recorded job feature information is used to match newly submitted jobs by users and is recorded in numerical form; while the execution performance data and resource allocation scheme provide a basis for preliminary resource allocation for similar jobs; at the same time, historical jobs are classified according to job feature information and resource allocation scheme to form an affinity history database. Step S102: Receive job; Receive newly submitted jobs from users; Step S103: Job Feature Matching; Upon receiving a newly submitted job from the user, the job feature matching process is immediately initiated, comparing the feature information of the newly submitted job with the jobs in the affinity history database; If a job with the highest similarity exists, proceed to step S104; otherwise, if no similar job exists, proceed to step S105; A similarity threshold is set to determine if a job with the highest similarity exists; If the similarity between the current job and historical jobs exceeds the similarity threshold, it is considered that a job with the highest similarity exists; otherwise, it does not exist. Step S104: Preliminary resource allocation for similar jobs; After finding the job with the highest similarity, based on the execution performance data and resource allocation scheme corresponding to the job with the highest similarity, make a preliminary estimate of the resource requirements of the current job; list the resource configuration of the job with the highest similarity as potential adaptive resources, and perform preliminary resource allocation for the current job; at the same time, obtain the historical affinity data of the job with the highest similarity; Step S105: Start probe testing; acquire and record job performance data; Step S106: Scheduling optimization loop; After determining the matching relationship between the newly submitted job and the GPU, the job enters the scheduling cycle loop to continuously optimize resource allocation; Based on the evaluation data from the affinity assessment model, an affinity matrix A is constructed to record the affinity of different jobs under different resource configuration schemes (n,t), where (n,t) represents the number of GPUs. t A i,j This represents the affinity value of job i under configuration j; Global scheduling optimization includes; During the global resource scheduling process, a search algorithm is run according to a predetermined scheduling cycle to generate a series of J×C binary allocation matrices. Each matrix represents a possible resource allocation scheme and is defined as a candidate allocation matrix D∈{0,1}. J×C J represents the number of jobs, C represents the number of resource allocation schemes, and D is an element in the matrix. i,j This is used to indicate whether configuration j was selected for job i; the following constraints should be satisfied for the candidate assignment matrix D: Assignment uniqueness: Each line contains at most one 1, and each job is assigned one configuration; that is: Resource capacity limit: The total resource allocation requirements of all allocated resources ≤ the total physical resources of the cluster; The goal of global scheduling is to achieve globally optimal scheduling, i.e., to obtain the optimal affinity allocation matrix, ∑D i,j ×A i,j This represents the basic affinity gain obtained from task i; Set the redistribution penalty factor ρ i The reassignment overhead is increased based on the historical reassignment frequency of jobs, and is designed as an exponentially decaying function ρ. i : Where, r i T represents the number of historical reallocations for job i; recent Indicates the time interval between the two most recent reallocations; T avg λ represents the average running cycle of task i; λ represents the penalty intensity coefficient. Global scheduling optimization transforms the scheduling problem into an optimization problem by maximizing the sum of affinity values ​​for jobs in the cluster. The objective function is: Where γ∈(-∞,1] serves as a fairness adjustment factor, employing a generalized power mean to achieve multi-objective balance: when γ=1, it degenerates into a linear summation, prioritizing pure efficiency and tilting resources towards tasks with higher affinity; when γ→0, it approaches the geometric mean, prioritizing fairness and emphasizing the fairness of resource allocation; when γ→-∞, it considers maximum and minimum fairness; ∑D i,j ×A i,j ρ represents the basic affinity gain obtained from task i; i This represents the redistribution penalty factor.

2. The method for scheduling optimization of heterogeneous GPU clusters based on resource affinity according to claim 1, characterized in that, Job feature matching; including: Step S1031: Extract job features; Extract feature information for the received new job. Job feature information includes: model features, data features, load characteristics and accuracy requirements. Data features include data type and data size. Step S1032: Traverse the historical job database; query the recorded historical jobs in the cluster, and execute step S1033 until the historical job most similar to the received new job is found; Step S1033: Similarity matching; Based on the feature-based vector representation, the job features extracted in step S1031 are transformed into numerical representations, and each job feature is merged into a vector representation; model features use one-hot encoding to represent model type, data features are numerically represented, load characteristics are classified through category encoding, and accuracy requirements are mapped to numerical values ​​for scalar representation. Calculate the similarity between the current assignment and historical assignments; use weighted similarity, weight each feature according to its importance, calculate the weighted similarity score for each assignment, and find the assignment with the highest similarity. Step S1034: Determine if the most similar job exists; After calculating the similarity score, set a similarity threshold to determine if the most similar job exists; If the similarity between the current job and historical jobs exceeds the similarity threshold, it is considered that the most similar job exists, otherwise it does not exist; If the most similar job exists, the cached profile is loaded, and the resource allocation scheme of the job with the highest similarity to the current job is obtained from the historical job library as a reference for preliminary resource allocation. This means that the resource allocation scheme of the job with the highest similarity to the current job is directly assigned to the current job. If no similar job exists, the job is considered an unknown job, a probe test is initiated and the actual execution performance of the job is recorded to achieve preliminary resource matching and provide data support for affinity matching.

3. The method for scheduling optimization of heterogeneous GPU clusters based on resource affinity according to claim 2, characterized in that, In step S1031, for model features, the L0 model is obtained based on the model definition of the job submission script or framework, and the approximate job category is determined based on the basic model architecture information. Based on the analysis of job configuration information, determine its load characteristics; The accuracy requirements are determined based on the settings in the model framework.

4. The method for scheduling optimization of heterogeneous GPU clusters based on resource affinity according to claim 2, characterized in that, Weighted similarity is used, where each feature is weighted according to its importance, and a weighted similarity score is calculated for each job to identify the job with the highest similarity. This includes: For model features, calculate cosine similarity Sim1; For data types, calculate the cosine similarity Sim2; For the data size, calculate the Euclidean distance Sim3; For load characteristics, calculate Jaccard similarity Sim4; For the required accuracy, calculate the Euclidean distance Sim5; Comprehensive similarity calculation: The similarity of each feature is calculated according to a preset weight w. u The weighted sum is then used to obtain the total similarity score:

5. The method for scheduling optimization of heterogeneous GPU clusters based on resource affinity according to claim 2, characterized in that, In step S105, the probe test is started; include: Step S1051: Receive unknown job; Step S1052: Perform performance testing; Unknown jobs were deployed to different types of GPUs in the cluster for small-scale trial runs. For each GPU type, multiple different combinations of running parameters were set during the test. In the deep learning training job scenario, the parameters included batch size and gradient accumulation steps. Under each parameter combination including batch size and gradient accumulation steps, the execution performance data of the job was strictly monitored and collected. This included job computing power requirements, GPU memory usage, amount of data processed, data transfer time, job running time, actual throughput, GPU resource utilization, and running power consumption. Step S1053: Record performance data, i.e., execution performance data; Step S1054: Determine the initial match; calculate the actual execution efficiency of the job based on the performance data recorded in step S1053; The formula for calculating actual performance efficiency is as follows: Efficacy=α×σ(Throughput)+β×Utiliztion GPU ; Where Throughput represents job throughput, and σ(Throughput) represents the normalized throughput of the job; Utilization GPU This represents the overall utilization rate of the GPU; α and β are weighting coefficients, balancing performance indicators. The actual execution performance of unknown jobs on different GPUs is sorted from high to low, compared and filtered, and the highest execution performance is taken as the optimal value. The selected optimal GPU resources are used as the initial resource matching scheme for unknown jobs.

6. A method for optimizing scheduling of heterogeneous GPU clusters based on resource affinity according to any one of claims 2-5, characterized in that, In step S106, the scheduling optimization loop is performed, including: Step S1061: Initial allocation exploration; For unknown jobs, in the initial stage of scheduling, it is assumed that the throughput of unknown jobs increases linearly with the amount of resources; that is: Throughput ideal (n,t,m,s)=n×Throughput(1,t,m,s); Where Throughput(n,t,m,s) represents the throughput of the job under the configuration of n GPUs, t GPUs, m batch size and s gradient accumulation steps; Step S1062: Throughput model estimation; Collect actual throughput data for jobs under different GPU counts n and types t, and fit the model to the parameters iα. frad ,β grad ,α sync ,β sync l Calculation gradient time T grad and gradient synchronization time T sync Where α grad This represents the fixed time overhead during gradient calculation, independent of batch size; β grad α represents the scaling factor that indicates the linear increase in gradient computation time with batch size m; sync This represents the fixed time overhead during gradient synchronization, independent of the number of GPUs; β sync This represents the proportionality of gradient synchronization time as the number of GPUs n increases; Gradient computation time: T grad =α grad +β grad ×m; Gradient synchronization time: The time T required for each iteration iter The calculation is performed on (n, t, m, s), including the s-th gradient calculation time T. grad (n,m) and the synchronization time T sync (n,m); Iteration time T iter (n,t,m,s):T iter (n,t,m,s)=s×T grad (n,m)+T sync (n,m); Build a throughput model for each job; The throughput model formula is: Where TotalBatch represents the total batch size; The actual utilization rate of modeling resources is calculated based on the ratio of effective computation time to total iteration time. RSRC (n,t,m,s): Step S1063: Affinity model evaluation; Define a set of resources as a resource allocation tuple (n, t), representing n GPUs. t ; First, by quantifying the degree of adaptation between the GPU's floating-point computing power and the actual needs of the job, the computing power matching degree S1 is obtained. The computing power matching degree S1 includes two sub-items: basic computing power matching and architecture adaptation correction. Among them, Job FLOPS This represents the number of floating-point operations required for a single iteration of the task; GPU FLOPS Indicates the theoretical peak computing power of a GPU; Arch Job Indicates the architectural characteristics required for the task; Arch GPU Indicates the actual GPU architecture; Arch Max q represents the maximum architectural generation difference; q is used as a sensitivity coefficient to represent architectural sensitivity. For the BaseCCS (Base Computing Power Matching Item), when Job FLOPS ≤GPU FLOPS hour, When Job FLOPS >GPU FLOPS hour, The efficiency of GPU memory resource utilization in quantification tasks is used to obtain memory fit, which encompasses both bandwidth utilization and capacity fit. Among them, Job MemOps Indicates the total amount of memory operations in a single batch; GPU Bandwidth Indicates the effective bandwidth of GPU memory; T batch Indicates the processing time for a single batch; Job MemUsage Indicates peak memory requirements for the task; GPU FreeMem This represents the available video memory after deducting system-reserved memory; By calculating the energy consumed per unit of computation, energy utilization efficiency is quantified, thereby obtaining the energy efficiency optimization degree S3: Among them, GPU Perf T represents the number of measured operations performed by the GPU in the job. active Indicates the effective computation time of the GPU; E toral Indicates the total energy consumption of the task; Based on actual operational data and throughput model predictions, the effective capacity under actual workload is quantified, thereby obtaining the execution efficiency S4: S S =Throughput×Utilization RSRC ; Among them, throughput, as a core indicator for measuring job execution efficiency, reflects the amount of tasks completed per unit of time, while the resource utilization correction factor is used. RSRC The actual utilization rate of resources is measured by the ratio of effective calculation time to total iteration time. Taking into account the cost of hardware procurement or leasing Capex and operating costs Opex Evaluate the cost-effectiveness of different types and quantities of GPUs in S5: S5=Cost Capex +Cost Opex ; For each job, an affinity evaluation model for the job and GPU resources is built under different configuration scenarios to quantitatively evaluate the compatibility between job tasks and GPU resources: Among them, AffScore Job (n,t) represents the multidimensional affinity value between job J and resource allocation scheme (n,t); Norm(·) is a normalization process based on the Sigmoid function; w k As a dynamic weighting coefficient; Step S1064: Global scheduling optimization; The goal of global scheduling is to achieve globally optimal scheduling, that is, to obtain the optimal affinity allocation matrix; Step S1065: Job-level scheduling optimization; the optimization objective is to find the optimal batch size m. * and gradient accumulation steps s * : (m * ,s * )=argmax m,s Throughput(n,m,s); Step S1066: Job execution; Step S1067: Parameter fitting; During the operation, continuously collect data θ sys ={α grad ,β grad ,α syac ,β sync Perform parameter fitting on the throughput model; Based on the throughput data recorded in step S1065, the model parameters are fitted using the L-BFGS-B optimization algorithm. Using the root mean square logarithmic error as the loss function RMSLE: Where N is the number of samples, T actual,p It is the actual measurement time, T iter,p It is the predicted iteration time; By minimizing this loss function, the model parameters θ are adjusted. sys This makes the predicted iteration time T iter,p It is closer to the actual time; Based on the optimized throughput model, solid data support is provided for resource allocation decisions, thus forming a scheduling optimization loop.

7. A resource affinity-based heterogeneous GPU cluster scheduling system, used to execute the resource affinity-based heterogeneous GPU cluster scheduling optimization method according to any one of claims 1-6, characterized in that, include: The similar job matching module is configured to: match historical jobs with high similarity based on job features in a history database of historical job affinity, and estimate resource requirements. The probe testing module is configured to: receive new jobs and perform probe tests on each type of GPU, and acquire and record job performance data; The affinity assessment module is configured to: use historical information or test data to build an affinity assessment model for each job, evaluate the affinity value of the job under different resource configurations, and provide data support for resource allocation decisions; The job scheduling module is configured to: enter a scheduling optimization cycle loop, construct an allocation matrix based on the objective function of the optimization problem, and allocate corresponding resources to each job.

8. A heterogeneous GPU cluster scheduling system based on resource affinity according to claim 7, characterized in that, The heterogeneous GPU cluster scheduling system also includes a performance monitoring module, which is configured to: build a full-stack monitoring system, collect multi-dimensional indicators during job execution in real time, detect abnormal fluctuations and generate alarm events.

Images