Heterogeneous general GPU task optimization scheduling method and system based on adaptation degree matrix

Abstract

The invention discloses a heterogeneous general GPU task optimization scheduling method and system based on an adaptation degree matrix, and belongs to the technical field of computing power resource scheduling in high-performance computation.The method comprises the steps that the adaptation degree matrix between different task types and card types is constructed based on multi-type tasks and task characteristics and GPU hardware capacity in a multi-architecture GPU cluster, and the adaptation degree matrix is used for scheduling the task types and the card types; the matching degree of the task and the GPU is quantified; designing a scheduling optimization problem based on the adaptation degree matrix; and solving the scheduling optimization problem through a genetic algorithm to realize optimal resource scheduling. According to the heterogeneous general GPU task optimization scheduling method and system based on the adaptation degree matrix, optimal scheduling is achieved by matching task characteristics with GPU hardware capacity, and a new scheme is provided for efficient scheduling of heterogeneous computing tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of computing resource scheduling in high-performance computing, and in particular to a method and system for optimizing and scheduling heterogeneous general-purpose GPU tasks based on a fitness matrix. Background Art

[0002] In recent years, the rapid development of fields such as Deep Learning (DL) and High-Performance Computing (HPC) has driven demand for heterogeneous computing resources. Graphics Processing Units (GPUs) have become a core computing power source due to their parallel computing capabilities. However, the diversity of GPU architectures (such as NVIDIA's Tensor Core and AMD's CDNA architecture) means that task execution efficiency is highly dependent on the compatibility of task type and hardware features. For example, tasks optimized with CUDA (Compute Unified Device Architecture, NVIDIA's parallel computing framework) perform significantly more efficiently on the Ampere architecture (such as the A100) than on other architectures, while tasks optimized with HIP (Heterogeneous-Compute Interface for Portability, AMD's GPU programming framework) perform even better on AMD GPUs. At the same time, enterprises are widely adopting hybrid GPU clusters (such as deploying the A100, V100, and AMD MI series simultaneously) to balance cost and performance. However, dynamically scheduling tasks to the appropriate GPU resources remains a key challenge.

[0003] In real-world scenarios, there are a large number of heterogeneous GPU clusters that require consideration of computing power adaptation. Take industrial simulation and deep learning training as examples:

[0004] Scenario 1: A task is expected to use an A100 GPU cluster to train a multimodal model. Due to the lack of consideration for adaptability, the model is optimized to the HIP model, which increases the training time.

[0005] Scenario 2: Scientific computing tasks (such as molecular dynamics simulation) rely on double-precision floating-point performance, but assigning them to a GPU optimized for single precision may result in large errors or even failure.

[0006] Therefore, adaptability modeling is the key to improving the efficiency of heterogeneous clusters and meeting the requirements of software ecosystem construction.

[0007] Current mainstream GPU scheduling technologies focus on resource utilization, JCT (Job Completion Time), and avoiding OOM (Out-Of-Memory) errors. Representative work includes:

[0008] SchedTune (a machine learning-based scheduler): optimizes GPU allocation by predicting task memory requirements and execution time. However, it does not explicitly model the compatibility of tasks with the GPU architecture, resulting in insufficient computing power utilization.

[0009] Hydra (deadline-oriented scheduler): uses a branch-and-bound algorithm to minimize task delays, but assumes that the execution time of tasks on different GPUs is fixed, ignoring the dynamic impact of adaptability differences on execution efficiency.

[0010] However, the tasks running on GPUs are diverse, and these tasks may rely on specific computing libraries (such as CUDA and ROCm) or hardware features (such as Tensor Cores and ray tracing units). Furthermore, GPU hardware is heterogeneous, with significant differences in computing power, memory bandwidth, and dedicated acceleration units between different GPUs. The efficiency of the same task on different GPUs may vary several times, and in extreme cases, may even be completely unsuitable. Therefore, the core problems of existing technologies are as follows:

[0011] Lack of adaptability modeling: The adaptability relationship between tasks and GPUs is not quantified. For example, the computing power loss coefficient of CUDA / HIP tasks on GPUs of different architectures is not considered, and the matching requirements between task types and hardware characteristics are not fully considered.

[0012] Suboptimal scheduling decisions: In a hybrid GPU cluster, because the compatibility between tasks and hardware characteristics is not considered (or statically specified), highly compatible tasks may be assigned to inefficient GPUs, resulting in longer overall completion times or tasks being restricted to running on designated GPUs, making collaboration between multiple tasks impossible.

[0013] Based on the above, current heterogeneous GPU task scheduling technologies mainly focus on the following types:

[0014] First, resource scheduling is performed with different optimization objectives, such as focusing on resource efficiency to minimize the task completion time (JCT), or meeting the deadline of each task.

[0015] Second, different resource scheduling methods, such as through heuristic algorithms, branch and bound or MILP (Mixed-Integer Linear Programming) optimization.

[0016] Third, the execution model assumptions of resources on GPU cards can be based on known numerical models or dynamic modeling measured during actual operation.

[0017] However, computationally intensive tasks, such as those exemplified by industrial software, require full-stack adaptation to achieve maximum efficiency. For example, adaptation of the software's numerical algorithms to the underlying numerical libraries, adaptation of the numerical libraries to the underlying hardware abstraction layer, and adaptation of the underlying hardware abstraction layer to specific GPUs can lead to efficiency compromises and even complete incompatibility at each level (for example, when full-precision computation is required but the underlying hardware cannot provide it). Therefore, from the perspective of the industrial software ecosystem, task-GPU compatibility is crucial. However, there is currently no explicit modeling of the compatibility relationship between task types and GPU architectures, nor is a solution proposed based on this. For example, SchedTune relies on machine learning to predict memory and JCT, but does not incorporate adaptability coefficients. Consequently, existing GPU scheduling technologies lack the ability to quantitatively model and dynamically optimize task-GPU adaptability. This leads to deficiencies in adaptability modeling and the dynamic resource allocation and optimization based on this modeling. This results in low resource utilization. The development of new GPU scheduling technologies is urgently needed. Summary of the Invention

[0018] The present invention provides a heterogeneous general-purpose GPU task optimization scheduling method and system based on a fitness matrix to solve the technical problems of insufficient computing power utilization and low resource utilization caused by the unquantified adaptability of task types and GPU architectures during task scheduling in heterogeneous GPU clusters.

[0019] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0020] In one aspect, the present invention provides a method for optimizing and scheduling heterogeneous general-purpose GPU tasks based on a fitness matrix. The method comprises:

[0021] Based on the task characteristics and GPU hardware capabilities of multi-type tasks and multi-architecture GPU clusters, a compatibility matrix between different task types and card types is constructed to quantify the matching degree between tasks and GPUs.

[0022] Based on the fitness matrix, design a scheduling optimization problem;

[0023] The scheduling optimization problem is solved by genetic algorithm to achieve optimal resource scheduling.

[0024] Furthermore, based on the task characteristics and GPU hardware capabilities of multi-type tasks and multi-architecture GPU clusters, a compatibility matrix between different task types and card types is constructed to quantify the degree of matching between tasks and GPUs, including:

[0025] Assume that there are multiple computing tasks in a multi-type task and multi-architecture GPU cluster, which require a computing power platform based on general-purpose GPUs to complete the calculations; each task is associated with its memory requirement, computing power requirement, and task type; the task type is expressed as a positive integer; the heterogeneous GPU computing platform is composed of multiple computing power card pools; each card pool is associated with the number of cards, the memory of a single card, the computing power of a single card per time slot, and the card type; the card type is expressed as a positive integer;

[0026] For a certain computing card, it can only be used as a whole, that is, it can only be delivered to a certain task at a time;

[0027] For a single task, it is assumed that only a certain number of cards from a certain hashrate pool can be allocated, i.e. the same task does not use multiple types of cards;

[0028] Considering the adaptability between task types and card types, a adaptability matrix between different task types and card types is constructed to quantify the matching degree between tasks and GPUs. The adaptability matrix is ​​expressed as:

[0029]

[0030] Among them, h i,j is the fitness of the i-th task on the j-th card, i = 1, 2, 3, ..., K C , K C is the maximum positive integer value corresponding to the task type; j=1,2,3,…,K T , K T is the maximum positive integer value corresponding to the card type; h i,j The value of h is a real number between 0 and 1. i,j =1 indicates that there is no performance loss in adaptation, h i,j =0 indicates no fit at all.

[0031] Furthermore, based on the fitness matrix, a scheduling optimization problem is designed, including:

[0032] For the i-th task T i , assuming that the jth i Card pools, allocated N i Block card, starting time is S i , end time is E i , where the time is taken as the time unit of periodic scheduling, that is, time slot; then:

[0033] The actual computing power is: Among them, P(C j ) represents the computing power of each time slot of a single card in the jth card pool; Represents task T iThe fitness level in the jth card pool;

[0034] Actual video memory: N i ·M(C j ); where M(C j ) represents the memory of a single card in the j-th card pool;

[0035] End time: Among them, P(T i ) represents the computing power requirement of the i-th task;

[0036] Once a task is deployed, its running environment will not be modified until it is completed. Assume that the earliest running task starts at time slot 0, and the adjustable parameter set X = {A i ,N i ,S i} i＝1..n ; Where n represents the total number of computing tasks;

[0037] The scheduling optimization problem is then expressed as:

[0038]

[0039] Among them, Z(t,j) is the set of tasks running on the jth card in time slot t; M(T i ) represents the memory requirement of the i-th task; Indicates A i Single card memory of a card pool; N(C j ) represents the number of cards in the jth card pool; m represents the total number of computing power card pools.

[0040] Furthermore, the scheduling optimization problem is solved by genetic algorithm to achieve optimal resource scheduling, including:

[0041] Encode each scheduling scheme as a gene individual and randomly generate the initial population;

[0042] Completed the GPU type and quantity fixes and performed time slot compression optimization;

[0043] Calculate the fitness value of individual genes;

[0044] The population is weightedly selected according to the fitness value, and several individuals with the highest fitness value are retained in each generation;

[0045] Randomly select the crossover point, split the two parent genes into two segments, and then splice them together; randomly select the mutation task number, randomly select the feasible GPU type according to the fitness matrix, and randomly select the number of cards of the corresponding type;

[0046] Determine whether the preset termination condition is met. If the termination condition is met, the gene individual with the highest fitness value is output as the optimal scheduling plan; if the termination condition is not met, the current generation is replaced by the offspring and the next round of iteration is entered.

[0047] Furthermore, when encoding each scheduling scheme into a gene individual, each gene individual contains the following information:

[0048] Task start time sequence: indicates the starting position of each task on the time slot axis;

[0049] GPU type assignment sequence: indicates the GPU type index assigned to each task, which corresponds to the predefined fitness matrix column index;

[0050] GPU quantity allocation sequence: indicates the number of GPU cards of the same type occupied by each task;

[0051] When randomly generating the initial population, each gene individual is initialized according to the following rules:

[0052] The task start time is randomly distributed within the estimated time slot range. The estimated maximum time slot is 2-3 times the total computing power required by all tasks divided by the total computing power of all cards.

[0053] The GPU type assignment method is to randomly select cards with a fitness greater than zero based on the fitness matrix;

[0054] The number of GPUs must not exceed the total physical number of GPUs of the same type and must not be less than 1.

[0055] Furthermore, the GPU type repair principle is: if a task in a gene is assigned to a GPU type with a fitness of 0, it is randomly replaced with any GPU type among the feasible types;

[0056] The principle of GPU quantity repair is as follows: if the number of allocated GPUs exceeds the total physical number of GPUs of the same type, it will be corrected to the maximum available number; if the number of allocated GPUs is 0, it will be forcibly repaired to 1;

[0057] The time slot compression optimization method is to eliminate the idle gaps between tasks through time slot scanning, move the subsequent tasks forward to eliminate the idle gaps, and reduce the total completion time.

[0058] Furthermore, the calculation formula of the fitness value of the gene individual is:

[0059] Fitness=-MaxEndTime–Loss*Weight

[0060] Among them, MaxEndTime represents the latest completion time of all tasks in the corresponding allocation plan of the gene individual; Loss represents the penalty item caused by violating the constraint. If the total amount of each type of card used in a certain time slot exceeds the number of cards in the card pool, the excess amount will be penalized; the final Loss is the sum of the penalties of all time slots; Weight represents the preset penalty coefficient.

[0061] On the other hand, the present invention also provides a heterogeneous general-purpose GPU task optimization scheduling system based on the fitness matrix, and the heterogeneous general-purpose GPU task optimization scheduling system based on the fitness matrix includes a processor and a memory; at least one instruction is stored in the memory, and the instruction is loaded and executed by the processor to implement the above-mentioned heterogeneous general-purpose GPU task optimization scheduling method based on the fitness matrix.

[0062] The technical solution provided by the present invention improves the resource utilization efficiency and task execution performance of heterogeneous GPU clusters by introducing a genetic optimization scheduling method driven by a fitness matrix. The beneficial effects brought by the present invention include at least:

[0063] 1. Optimization of computing power utilization: This invention uses the adaptability matrix to quantify the matching degree between the task and the GPU, rationally allocates adaptability resources, and reduces the waste of computing power caused by insufficient utilization of hardware characteristics.

[0064] 2. Improved global scheduling efficiency: This invention utilizes the crossover and mutation mechanism of the genetic algorithm in combination with fitness scoring to achieve optimized scheduling effects.

[0065] 3. The improved blank time slot optimization strategy of the present invention can improve the convergence speed of the genetic algorithm.

[0066] 4. Enhanced flexibility and scalability: The fitness matrix in this invention is independent of the scheduling algorithm core and can be dynamically expanded to new task types and GPU architectures without modifying the genetic operators, reducing adaptation costs by 80%.

[0067] In summary, the present invention achieves global optimization scheduling driven by task-GPU adaptability while ensuring compliance with resource constraints, providing an efficient and scalable solution for heterogeneous computing scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0069] Figure 11 is a schematic diagram of the execution flow of a heterogeneous general-purpose GPU task optimization scheduling method based on a fitness matrix provided by an embodiment of the present invention;

[0070] Figure 2 Schematic diagram of the time slot compression optimization principle provided by an embodiment of the present invention;

[0071] Figure 3 This is a schematic diagram of a single-point crossover provided by an embodiment of the present invention;

[0072] Figure 4 This is a graph provided by an embodiment of the present invention showing the decrease in optimal fitness over multiple runs with the number of iterations when the population size is 50, two elite individuals are retained in each generation, 500 generations are run, the crossover probability is 0.3, the mutation probability is 0.05, and the penalty coefficient is 100;

[0073] Figure 5 This is a graph showing the decrease in optimal fitness as the number of iterations increases when performing multiple runs of the gene optimization strategy without considering blank time slot optimization, as provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0074] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0075] First, it should be noted that in the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplarily" is intended to present concepts in a concrete manner. In addition, in the embodiments of the present invention, the meaning of "and / or" can be both or either of the two.

[0076] First embodiment

[0077] This embodiment provides a heterogeneous general GPU task optimization scheduling method based on fitness matrix for multi-type tasks and multi-architecture GPU clusters, which is suitable for scenarios such as deep learning training and scientific simulation that require efficient use of heterogeneous GPU resources. Figure 1 As shown in the figure, this method designs a targeted scheduling scheme based on the fitness matrix between computing tasks and GPU hardware, including gene expression, crossover, mutation, and repair mechanisms, and achieves optimized resource scheduling through genetic algorithms. Specifically, the execution process of this method includes the following steps:

[0078] S1, based on the task characteristics and GPU hardware capabilities in multi-type tasks and multi-architecture GPU clusters, builds a compatibility matrix between different task types and card types to quantify the matching degree between tasks and GPUs;

[0079] Specifically, this embodiment obtains the adaptation relationship matrix between different task types and card types based on theoretical analysis or actual testing to complete the construction of the adaptation matrix. The process is as follows:

[0080] Consider a number of computing tasks {T1, T2, ..., T n}, they need a computing platform based on general-purpose GPUs to complete the calculation. For each task T i , associated with its memory requirement M(T i ), computing power requirements P(T i ) and task type K(T i ). The task type is expressed as a number from 1 to K. T A positive integer.

[0081] Consider several computing power pools {C1, C2, ..., C m} composed of heterogeneous GPU computing platforms, for each card pool C i , associated with the number of cards N(C i )、Single card memory M(C i ), the computing power of each time slot of a single card P(C i ) and card type K(C i ). The card type is expressed as a number from 1 to K C A positive integer.

[0082] For a certain computing card, it can only be used as a whole, that is, it can only be delivered to a certain task at a time.

[0083] For a single task, it is assumed that only a few cards from a certain computing power pool can be allocated, that is, the same task does not use multiple types of cards.

[0084] Considering the compatibility between task type and card type, it can be expressed as a matrix:

[0085]

[0086] Among them, h i,j is the fitness of the i-th task on the j-th card, i = 1, 2, 3, ..., K C , K C is the maximum positive integer value corresponding to the task type; j=1,2,3,…,K T , K T is the maximum positive integer value corresponding to the card type; h i,j The value of h is a real number between 0 and 1. i,j =1 indicates that there is no performance loss in adaptation, h i,j =0 indicates no fit at all.

[0087] S2, designing a scheduling optimization problem based on the fitness matrix;

[0088] If each task T i , assuming that the jth i Card pools, allocated N i Block card, starting time is S i , end time is E i , where the time is taken as the time unit of periodic scheduling, that is, time slot. Then:

[0089] The actual computing power is: Among them, P(C j ) represents the computing power of each time slot of a single card in the jth card pool; Represents task T i The fitness level in the jth card pool;

[0090] Actual video memory: N i ·M(C j ); where M(C j ) represents the memory of a single card in the j-th card pool;

[0091] End time: (Note: round up to the nearest integer based on the time slot);

[0092] Once a task is deployed, its running environment will not be modified until it is completed. Assume that the earliest running task starts at time slot 0, and the adjustable parameter set X = {A i ,N i ,S i} i＝1..n ; Where n represents the total number of computing tasks;

[0093] The optimal scheduling solution is:

[0094]

[0095] Among them, Z(t,j) is the set of tasks running on the jth card in time slot t; Indicates A i Single card memory of a card pool; N(C j ) represents the number of cards in the jth card pool; m represents the total number of computing power card pools.

[0096] S3, solving the scheduling optimization problem by using a genetic algorithm to achieve optimal resource scheduling;

[0097] It should be noted that the scheduling problem is essentially a complex combinatorial optimization problem, and it is difficult to obtain an optimal solution. Therefore, this embodiment adopts a genetic algorithm to obtain a near-optimal solution. The process is as follows:

[0098] Step 1: Gene encoding and population initialization.

[0099] Each scheduling scheme is encoded as a gene individual, which contains the following three sets of information:

[0100] Task start time sequence: indicates the starting position of each task on the time slot axis;

[0101] GPU type assignment sequence: indicates the GPU type index assigned to each task, which corresponds to the predefined fitness matrix column index;

[0102] GPU quantity allocation sequence: indicates the number of GPU cards of the same type occupied by each task.

[0103] The initial population is randomly generated, and each gene individual is initialized according to the following rules:

[0104] The task start time is randomly distributed within the estimated time slot range. The estimated maximum time slot is: the total computing power required by all tasks divided by the total computing power of all cards (equivalent to the optimal end time without considering adaptation and scheduling), multiplied by 2-3 times.

[0105] The GPU type assignment method is to randomly select cards with a fitness greater than zero based on the fitness matrix;

[0106] The number of GPUs must not exceed the total physical number of GPUs of the same type and must not be less than 1.

[0107] Step 2: Gene repair and optimization.

[0108] GPU type repair: If a task in the gene is assigned to a GPU type with a fitness of 0, it will be randomly replaced with any GPU type among the feasible types;

[0109] GPU quantity repair: If the number of allocated GPUs exceeds the total physical number of GPUs of the same type, it will be corrected to the maximum available number; if the number of allocated GPUs is 0, it will be forcibly repaired to 1;

[0110] Time slot compression optimization: Eliminate idle gaps between tasks through time slot scanning, move subsequent tasks forward to eliminate idle gaps, and reduce the total completion time. This is a direct optimization of the current gene.

[0111] like Figure 2 As shown, time slot compression can be performed for all card pools or for each card pool individually. In the former, since no tasks are scheduled between the end of T2 and the beginning of T3, T3 and all subsequent tasks can be shifted forward by d1 time slots. In the latter, since no tasks are scheduled for pool C1 between the end of T1 and the beginning of T3, T3 can be shifted forward by d2 time slots. The latter optimization is more refined.

[0112] Step 3: Fitness evaluation.

[0113] Calculate the fitness value of the gene individual, the calculation formula is:

[0114] Fitness=-MaxEndTime–Loss*Weight

[0115] Among them, MaxEndTime represents the latest completion time of all tasks in the allocation plan corresponding to the gene individual; Loss represents the penalty item caused by violating the constraint. If the total amount of each type of card used in a certain time slot exceeds the number of cards in the card pool, the excess amount will be penalized; the final Loss is the sum of the penalties of all time slots; Weight represents the preset penalty coefficient, which is usually around 10.

[0116] Therefore, the gene with the latest completion time and the less penalty has the greater fitness.

[0117] Step 4: Generation of offspring.

[0118] Roulette wheel selection: The population is weightedly selected according to the fitness value, and genes with higher fitness have a greater probability of entering the next generation.

[0119] Elite retention: retain several individuals with the highest fitness values ​​in each generation to avoid losing high-quality solutions.

[0120] Step 5: Genetic operations (crossover and mutation).

[0121] Single-point crossover: Randomly select the crossover point, cut the two parent genes into two segments, and then splice them together; Figure 3 shown.

[0122] Mutation: Randomly select the mutation task number, randomly select a feasible GPU type based on the fitness matrix, and randomly select the number of cards of the corresponding type (at least 1 and no more than the total number of cards of that type);

[0123] It should be noted that crossover and mutation are performed according to probability, and gene failure may occur after crossover and mutation.

[0124] Step 6: Determine the termination condition.

[0125] The judgment is to reach the maximum number of iterations (or other termination conditions). If the termination conditions are reached, the gene individual with the highest fitness is output as the optimal scheduling solution.

[0126] If the termination condition is not met, the current generation is replaced by the child generation, and the next round of iteration is entered, and the process goes to step 2.

[0127] In summary, this embodiment introduces a technology stack adaptation matrix to provide a heterogeneous general-purpose GPU task optimization scheduling method based on the fitness matrix. While ensuring resource constraint compliance, it implements task-GPU adaptability-driven global optimization scheduling, providing an efficient and scalable solution for heterogeneous computing scenarios.

[0128] Next, we will verify the effectiveness of this method through specific examples.

[0129] (1) Scenario description

[0130] Assume that there are 10 tasks, as shown in Table 1:

[0131] Table 1 Task list

[0132]

[0133]

[0134] Assume that there are 5 GPU card pools, as shown in Table 2:

[0135] Table 2 GPU card pool list

[0136] Serial number Number of cards Single card computing power Single card video memory Card Type 1 4 12 16 1 2 8 16 32 1 3 8 20 16 2 4 10 28 32 2 5 16 10 8 3

[0137] The adaptation matrix list is shown in Table 3:

[0138] Table 3 Adaptation matrix list

[0139]

[0140] According to the method of the present invention, when the population size is 50, 2 elite individuals are retained in each generation, 500 generations are run, the crossover probability is 0.3, the mutation probability is 0.05, and the penalty coefficient is 100, the curve of the optimal fitness obtained by multiple runs decreasing with the number of iterations is as follows: Figure 4 shown.

[0141] It can be seen that as the number of iterations increases, the fitness increases rapidly, the penalty term basically disappears after 200 generations, and gradually converges to the optimal solution or suboptimal solution.

[0142] Correspondingly, if the gene optimization strategy of blank time slot optimization is not considered, the curves of multiple runs are as follows: Figure 5 As shown in Figure 2, it can be seen that genetic optimization helps the solution converge quickly.

[0143] The solution finally obtained by the present invention is completed in 28 time slots, and the allocation of each task is shown in Table 4:

[0144] Table 4 Task allocation results

[0145]

[0146]

[0147] Among them, we can see that card pool 1 is not used in this allocation due to insufficient video memory and a small number of cards. The last card pool is used very intensively, with each time slot being used continuously.

[0148] Second embodiment

[0149] This embodiment provides a heterogeneous general-purpose GPU task optimization scheduling system based on a fitness matrix. The heterogeneous general-purpose GPU task optimization scheduling system based on a fitness matrix includes a processor and a memory; the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the above-mentioned heterogeneous general-purpose GPU task optimization scheduling method based on the fitness matrix.

[0150] Among them, it should be noted that the heterogeneous general-purpose GPU task optimization scheduling system based on the fitness matrix of this embodiment corresponds to the heterogeneous general-purpose GPU task optimization scheduling method based on the fitness matrix of the above-mentioned first embodiment; among them, the functions implemented by each functional module in the heterogeneous general-purpose GPU task optimization scheduling system based on the fitness matrix of this embodiment correspond one-to-one to each process step in the heterogeneous general-purpose GPU task optimization scheduling method based on the fitness matrix of the above-mentioned first embodiment; therefore, they will not be repeated here.

[0151] It should also be noted that, in this document, relational terms such as first and second are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any actual relationship or order between these entities or operations. The terms "include," "comprises," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. Without further limitation, an element defined by the phrase "comprising a..." does not preclude the presence of other identical elements in the process, method, article, or terminal device comprising the element. In addition, the term "and / or" is merely a description of an associative relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: the presence of A alone, the presence of A and B simultaneously, or the presence of B alone, where A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding. "At least one" means one or more, and "more" means two or more. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0152] In addition, it can be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0153] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0154] In the several embodiments provided herein, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of functional modules / units is merely a logical functional division. In actual implementation, other division methods may be used, such as multiple units or components being combined or integrated into another device, or some features being ignored or not implemented. Furthermore, the coupling or direct coupling or communication connection shown or discussed between each other may be through some interface, indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms. Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of the present embodiment according to actual needs. In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit.

[0155] If the method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0156] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention. It should be noted that, although preferred embodiments of the present invention have been described, those skilled in the art, once understanding the basic inventive concepts of the present invention, may make various improvements and modifications without departing from the principles of the present invention. Such improvements and modifications should also be considered within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as covering the preferred embodiments and all variations and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A heterogeneous general-purpose GPU task optimization scheduling method based on fitness matrix, characterized in that: The heterogeneous general-purpose GPU task optimization scheduling method based on the fitness matrix includes: Based on the task characteristics and GPU hardware capabilities of multi-type tasks and multi-architecture GPU clusters, a compatibility matrix between different task types and card types is constructed to quantify the matching degree between tasks and GPUs. Based on the fitness matrix, design a scheduling optimization problem; The scheduling optimization problem is solved by genetic algorithm to achieve optimal resource scheduling.

2. The heterogeneous general-purpose GPU task optimization scheduling method based on the fitness matrix according to claim 1 is characterized in that: Based on the task characteristics and GPU hardware capabilities of multi-type tasks and multi-architecture GPU clusters, a compatibility matrix between different task types and card types is constructed to quantify the degree of matching between tasks and GPUs, including: Assume that there are multiple computing tasks in a multi-type task and multi-architecture GPU cluster, which require a computing power platform based on general-purpose GPUs to complete the calculations; each task is associated with its memory requirement, computing power requirement, and task type; the task type is expressed as a positive integer; the heterogeneous GPU computing platform is composed of multiple computing power card pools; each card pool is associated with the number of cards, the memory of a single card, the computing power of a single card per time slot, and the card type; the card type is expressed as a positive integer; For a certain computing card, it can only be used as a whole, that is, it can only be delivered to a certain task at a time; For a single task, it is assumed that only a certain number of cards from a certain hashrate pool can be allocated, i.e. the same task does not use multiple types of cards; Considering the adaptability between task types and card types, a adaptability matrix between different task types and card types is constructed to quantify the matching degree between tasks and GPUs. The adaptability matrix is ​​expressed as: Among them, h i,j is the fitness of the i-th task on the j-th card, i = 1, 2, 3, ..., K C , K C is the maximum positive integer value corresponding to the task type; j=1,2,3,…,K T , K T is the maximum positive integer value corresponding to the card type; h i,j The value of h is a real number between 0 and 1. i,j =1 indicates that there is no performance loss in adaptation, h i,j =0 indicates no fit at all.

3. The heterogeneous general-purpose GPU task optimization scheduling method based on the fitness matrix according to claim 1 is characterized in that: Based on the fitness matrix, a scheduling optimization problem is designed, including: For the i-th task T i , assuming that the jth i Card pools, allocated N i Block card, starting time is S i , end time is E i , where the time is taken as the time unit of periodic scheduling, that is, time slot; then: The actual computing power is: Among them, P(C j ) represents the computing power of each time slot of a single card in the jth card pool; Represents task T i The fitness level in the jth card pool; Actual video memory: N i ·M(C j ); where M(C j ) represents the memory of a single card in the j-th card pool; End time: Among them, P(T i ) represents the computing power requirement of the i-th task; Once a task is deployed, its running environment will not be modified until it is completed. Assume that the earliest running task starts at time slot 0, and the adjustable parameter set X = {A i ,N i ,S i } i＝1..n ; Where n represents the total number of computing tasks; The scheduling optimization problem is then expressed as: s.t.S i ≥0 Among them, Z(t,j) is the set of tasks running on the jth card in time slot t; M(T i ) represents the memory requirement of the i-th task; Indicates A i Single card memory of a card pool; N(C j ) represents the number of cards in the jth card pool; m represents the total number of computing power card pools.

4. The heterogeneous general-purpose GPU task optimization scheduling method based on the fitness matrix according to claim 1 is characterized in that: Solving the scheduling optimization problem through genetic algorithms to achieve optimal resource scheduling includes: Encode each scheduling scheme as a gene individual and randomly generate the initial population; Completed the GPU type and quantity fixes and performed time slot compression optimization; Calculate the fitness value of individual genes; The population is weightedly selected according to the fitness value, and several individuals with the highest fitness value are retained in each generation; Randomly select the crossover point, split the two parent genes into two segments, and then splice them together; randomly select the mutation task number, randomly select the feasible GPU type according to the fitness matrix, and randomly select the number of cards of the corresponding type; Determine whether the preset termination condition is met. If the termination condition is met, the gene individual with the highest fitness value is output as the optimal scheduling plan; if the termination condition is not met, the current generation is replaced by the offspring and the next round of iteration is entered.

5. The heterogeneous general-purpose GPU task optimization scheduling method based on the fitness matrix according to claim 4 is characterized in that: When encoding each scheduling scheme into a gene individual, each gene individual contains the following information: Task start time sequence: indicates the starting position of each task on the time slot axis; GPU type assignment sequence: indicates the GPU type index assigned to each task, which corresponds to the predefined fitness matrix column index; GPU quantity allocation sequence: indicates the number of GPU cards of the same type occupied by each task; When randomly generating the initial population, each gene individual is initialized according to the following rules: The task start time is randomly distributed within the estimated time slot range. The estimated maximum time slot is 2-3 times the total computing power required by all tasks divided by the total computing power of all cards. The GPU type assignment method is to randomly select cards with a fitness greater than zero based on the fitness matrix; The number of GPUs must not exceed the total physical number of GPUs of the same type and must not be less than 1.

6. The heterogeneous general-purpose GPU task optimization scheduling method based on the fitness matrix according to claim 5 is characterized in that: The GPU type repair principle is: if a task in a gene is assigned to a GPU type with a fitness of 0, it is randomly replaced with any GPU type among the feasible types; The principle of GPU quantity repair is as follows: if the number of allocated GPUs exceeds the total physical number of GPUs of the same type, it will be corrected to the maximum available number; if the number of allocated GPUs is 0, it will be forcibly repaired to 1; The time slot compression optimization method is to eliminate the idle gaps between tasks through time slot scanning, move the subsequent tasks forward to eliminate the idle gaps, and reduce the total completion time.

7. The heterogeneous general-purpose GPU task optimization scheduling method based on the fitness matrix according to claim 4 is characterized in that: The calculation formula of the fitness value of a gene individual is: Fitness=-MaxEndTime–Loss*Weight Among them, MaxEndTime represents the latest completion time of all tasks in the corresponding allocation plan of the gene individual; Loss represents the penalty item caused by violating the constraint. If the total amount of each type of card used in a certain time slot exceeds the number of cards in the card pool, the excess amount will be penalized; the final Loss is the sum of the penalties of all time slots; Weight represents the preset penalty coefficient.

8. A heterogeneous general-purpose GPU task optimization scheduling system based on a fitness matrix, the heterogeneous general-purpose GPU task optimization scheduling system based on a fitness matrix comprising a processor and a memory; characterized in that: At least one instruction is stored in the memory, and the instruction is loaded and executed by the processor to implement the heterogeneous general-purpose GPU task optimization scheduling method based on the fitness matrix according to any one of claims 1 to 7.

Images