A container scheduling method of GPU resources

Abstract

The application discloses a container scheduling method of GPU resources, and the method comprises the following steps: receiving GPU container tasks of each platform, generating at least one task group by using a preset dynamic task dependency graph clustering mechanism, and extracting a group-level demand feature vector of the task group; generating a value vector of each GPU node by using a preset node multi-dimensional value evaluation algorithm based on the GPU node, obtaining a plurality of partitions and a partition value vector of the partitions; determining a target partition of each task group by using a group-partition matching model based on the group-level demand feature vector of the task group and the partition value vector; and generating a GPU allocation scheme of the task group by using a task group resource allocation decision mechanism based on the target partition of the task group. Thus, efficient utilization and allocation of GPU resources are realized, and container tasks are reasonably matched.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of graphic processor resource, and in particular to a GPU resource container scheduling method. BACKGROUND

[0002] With the rapid development of artificial intelligence, deep learning, high-performance computing and other fields, GPU (Graphics Processing Unit) has become the core hardware resource for processing large-scale data and complex computing tasks due to its powerful parallel computing capability. In cloud computing and data center environments, multiple users and applications need to use GPU resources simultaneously to accelerate the execution of their computing tasks. However, the scarcity and high cost of GPU resources make it a key problem to efficiently manage and schedule these resources to improve the overall performance and resource utilization of data centers. Containerization technology is increasingly popular, and Kubernetes and other container orchestration platforms have become the mainstream tools for managing distributed applications. Container technology encapsulates applications and their dependent environments, enabling rapid deployment and migration of applications, improving resource utilization and system flexibility. However, in the field of GPU resource scheduling, how to combine container technology with GPU resource scheduling to achieve more efficient and flexible resource management remains a pressing problem.

[0003] Traditional GPU resource scheduling methods usually adopt a static allocation strategy, that is, GPU resources are allocated to each task according to the initial request of the task. However, this method has many limitations. First, static allocation cannot adapt to dynamic changes during task execution, such as communication patterns between tasks, data dependency relationships, etc., which may lead to unreasonable resource allocation and affect task execution efficiency and overall system performance. Second, traditional scheduling methods often ignore data dependency relationships and communication overhead between tasks, and independently schedule multiple related tasks (such as multiple stages in an AI inference pipeline), resulting in increased cross-task communication overhead and reduced task execution efficiency. SUMMARY

[0004] The application provides a GPU resource container scheduling method, which realizes efficient utilization of GPU resources and precise allocation of container tasks.

[0005] The application provides a GPU resource container scheduling method, which includes:

[0006] S101, receiving GPU container tasks of each platform, generating at least one task group using a preset dynamic task dependency graph clustering mechanism, and extracting a group-level demand feature vector of the task group;

[0007] S102, based on the GPU node, using a preset node multi-dimensional value evaluation algorithm, generating a value vector of each GPU node, obtaining a plurality of partitions and their partition value vectors;

[0008] S103, based on the group-level demand feature vector of the task group and the partition value vector, determining the target partition of each task group through a group-partition matching model;

[0009] S104, based on the target partition determined by the task group, using a task group resource allocation decision mechanism to generate a GPU allocation scheme for the task group.

[0010] Preferably, the preset dynamic task dependency graph clustering mechanism specifically comprises:

[0011] S201, obtaining historical communication data between different container tasks in a preset time period, including source container ID, target container ID, communication timestamp, data transmission volume, communication frequency, aggregating data by container pair, calculating the historical total flow and average interaction frequency of the container pair;

[0012] S202, based on the received GPU container tasks of each platform, generating all container pairs, combining historical communication data, and calculating the dependency factor of the container pair;

[0013] S203, according to the task dependency factor of all container pairs, taking the container as a graph node and the task dependency factor as an edge weight, constructing a task dependency graph, retaining edges whose task dependency factors meet a preset dependency threshold, identifying connected components, each connected component forming a task group, and generating a task group set , is the kth task group, and K is the total number of task groups.

[0014] Preferably, the dependency factor is calculated according to the following formula:

[0015]

[0016] wherein, is the task dependency factor of container pair i and j, is the historical total flow from container i to container j, is the average interaction frequency of the container pair, is the total outflow of container i, is the global average interaction frequency of container i, is a preset weight factor for reflecting the importance of data size dependency to the task dependency factor.

[0017] Preferably, each graph node in the task dependency graph is bound with a demand feature vector of the container task, represented as , GPU card requirement of the i-th container task, computing time consumption of the i-th container task, communication sensitivity of the i-th container task.

[0018] Preferably, the group-level requirement feature vector of the extraction task group specifically comprises:

[0019] Based on the requirement feature vector of each container task in the task group, a group-level requirement feature vector corresponding to the task group is constructed , total GPU card requirement of all container tasks in the k-th task group, minimum communication sensitivity in the k-th task group, average task dependency factor of all dependency edges in the k-th task group, that is, the average value of the dependency factors corresponding to all dependency edges in the task group.

[0020] Preferably, the preset node multi-dimensional value evaluation algorithm specifically comprises:

[0021] S301, based on each GPU node, the value vector of the GPU node is obtained according to the topology value and the resource value of the GPU node; the value vector is obtained by: the fusion value is obtained by weighted summation of the topology value and the resource value, and the value vector is composed of the fusion value and the GPU generation standard deviation , fusion value of node m, GPU generation standard deviation of node m;

[0022] S302, according to the value vector of all GPU nodes, the GPU nodes are divided to obtain a plurality of partitions and partition value vectors , available GPU card quantity in the z-th partition, average fusion value of the z-th partition, maximum value of the GPU generation standard deviation in the z-th partition.

[0023] Preferably, the target partition of each task group is determined through the group-partition matching model, specifically comprising:

[0024] D1, a partition adaptation scoring algorithm S(k, z) is constructed to quantify the matching degree of the task group k and the partition z, and the scoring algorithm is composed of three sub-indexes of GPU resource matching degree, communication sensitivity adaptation degree and task dependency strength adaptation degree;

[0025] D2, the matching degree between the task group and each partition is obtained through the partition adaptation scoring algorithm, and the partition with the highest matching degree is selected as the target partition of the task group;

[0026] D3, locking GPU resources in the target partition for the task group.

[0027] Preferably, the task group resource allocation decision mechanism specifically comprises:

[0028] S401, sorting all task groups in ascending order of total GPU card demand in the group to form a scheduling queue;

[0029] S402, based on the container tasks in each task group, traversing the resource queue in the corresponding target partition, locking the resource pool, and in the pool, generating at least one resource allocation scheme, for each resource allocation scheme, using the pre-constructed dynamic loss function to obtain the loss value, selecting the resource allocation scheme with the smallest loss value as the target allocation scheme, and based on the target allocation scheme, scheduling resources for the container tasks in the task group.

[0030] Preferably, in D1, the adaptive scoring algorithm is set as:

[0031]

[0032] wherein, is the matching degree of the task group k and the partition z, 、 、 is a preset weight value, which is dynamically adjusted according to the task type and expert experience, and satisfies , is the GPU resource matching degree, is the communication sensitivity adaptive degree, is the task dependency strength adaptive degree.

[0033] Preferably, the demand feature vector of the container task is obtained in the following manner:

[0034] A1, extracting the attribute features of the container task;

[0035] A2, inputting the attribute features of the container task into a pre-trained task communication sensitivity prediction model to output the time-consuming information of the container task, including the calculation time-consuming and the communication sensitivity;

[0036] A3, the GPU card demand, the calculation time-consuming and the communication sensitivity of the container task are combined to form a demand feature vector.

[0037] One or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0038] Highly related tasks are aggregated into task groups by dynamic task dependency graph clustering to reduce cross-task communication overhead; node value is comprehensively evaluated in combination with topology position and hardware resources and hardware compatibility, the node is divided into multiple partitions to realize resource hierarchical management and ensure that high-demand tasks are allocated to high-quality resource areas. Through a multi-dimensional scoring algorithm (resource matching degree, communication sensitivity adaptation degree, and dependency strength adaptation degree), the task group is allocated to the most suitable partition to avoid resource mismatch. The dynamic loss function considers both computational intensity and communication overhead when allocating container tasks to achieve a balance between resource distribution balance and communication overhead minimization. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 A flowchart of a container scheduling method for GPU resources of an embodiment of the application. DETAILED DESCRIPTION

[0040] In order to facilitate the understanding of the present application, the present application will be described in more detail below with reference to the relevant drawings; the preferred embodiments of the present application are shown in the drawings, but the present application can be implemented in many different forms and is not limited to the embodiments described herein; on the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0041] It should be noted that the terms "vertical", "horizontal", "up", "down", "left", "right" and similar expressions used herein are only for illustrative purposes and do not represent the only implementation.

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs; the terms used herein in the specification of the present application are only for the purpose of describing the specific embodiments and are not intended to limit the present application; the term "and / or" used herein includes any and all combinations of one or more related listed items.

[0043] The conventional scheduling method often cannot accurately quantify the data dependency relationship between tasks and the communication overhead, resulting in unreasonable task group division and low resource allocation efficiency. In related technologies, the value evaluation of GPU nodes often only considers the hardware resource status (such as the number of GPU cards, memory utilization, etc.), and ignores the influence of network topology position on task execution efficiency, resulting in tasks being deployed on nodes with large communication delay, which reduces the task execution performance; each task is often independently scheduled, and the data dependency relationship and communication overhead within the task group are ignored, resulting in increased communication overhead within the task group, which affects the task execution efficiency and the overall performance of the system.

[0044] Embodiment one: Figure 1It is a flowchart of a container scheduling method of a GPU resource of an embodiment of the application.

[0045] As shown in the figure, a container scheduling method of a GPU resource includes the following steps: Figure 1

[0046] S101, receiving GPU container tasks of each platform, using a preset dynamic task dependency graph clustering mechanism to generate at least one task group, and extracting a group-level demand feature vector of the task group.

[0047] Specifically, the GPU tasks of each platform are encapsulated into containers (including the platform type to which the task belongs, the container image type, the number of GPU cards required, the task type label, the input data size, etc.), to obtain GPU container tasks, to ensure environmental compatibility and enable tasks to be easily moved and deployed between different clusters. Each task group includes at least one container task. Through container task group dependency modeling and clustering analysis, the data dependency relationship between container tasks is quantified, a task dependency graph is constructed, and highly correlated task groups are identified, providing a basis for subsequent task scheduling, resource allocation, or parallel optimization.

[0048] It should be noted that a container is defined as a task instantiation unit running in a Kubernetes cluster, and each container encapsulates an independent computing task, which exists before resource scheduling and is submitted by the application platform. The actual interaction behavior (communication event) between containers occurs through an Overlay network. Exemplarily, the communication event includes a connection event: TCP / UDP session establishment, data transmission: RPC call, HTTP request, custom protocol interaction, and meta-information exchange: task state synchronization message. Therefore, due to the communication event between containers, there is a dynamic indicator of the cooperation strength between container tasks, which can be understood as a dependency factor between tasks.

[0049] In some embodiments, the preset dynamic task dependency graph clustering mechanism specifically includes:

[0050] S201, obtaining historical communication data between different container tasks in a preset time period, including source container ID, target container ID, communication timestamp, data transmission volume (byte number), and communication frequency (number of interactions per unit time), aggregating data by container pair, and calculating the historical total flow and average interaction frequency of the container pair.

[0051] Specifically, the historical communication data is extracted from a cluster monitoring system or a network log, and the preset time period is set to 30 days to ensure data timeliness.

[0052] S202, based on the received GPU container tasks of each platform, generating all container pairs, combining the historical communication data, and calculating the dependency factor of the container pair, specifically calculated according to the following formula: ​

[0053]

[0054] wherein, is the task dependency factor of container pair i and j, is the historical total flow from container i to container j, i.e., the cumulative data transmission amount, is the average interaction frequency of container pair, which can be understood as the ratio of the total interaction times of the container pair to the window length of the preset time period, is the total outflow of container i, i.e., the total output flow of container i to all other containers, is the global average interaction frequency of container i, i.e., the average interaction frequency of container i with all other containers, which can be understood as the ratio of the total interaction times of container i with all other containers to the window length of the preset time period. is a preset weight factor, used to reflect the importance of data size dependence on the task dependency factor, if is larger, the data size dependence is emphasized, which is suitable for data-intensive tasks (such as video stream processing), otherwise, the real-time interaction dependence is emphasized, which is suitable for control-intensive tasks (such as microservice invocation), the value of is dynamically adjusted according to the actual situation (task type) and expert experience.

[0055] Therefore, the task dependency factor algorithm can represent the relative importance of container j in the output flow or interaction of container i, which helps to quantify the dependence strength of container i on container j and avoid the deviation of absolute value.

[0056] It should be noted that if there is no historical record of a container pair, the similar historical task is matched according to its image label or task type, and the dependence factor is inherited, if the similar historical task or image label matching fails, is set to 0; if the average interaction frequency of the container pair is lower than the preset threshold (determined according to the actual situation and expert experience, used to measure the contingency or direct irrelevance of the interaction between the container pair), is set to 0 (considered as no dependence).

[0057] S203, according to the task dependency factor of all container pairs, taking the container as the graph node and the task dependency factor as the edge weight, a task dependency graph is constructed, the edges whose task dependency factors meet the preset dependence threshold (for example, the dependence threshold is set to 0.6) are retained, a preset graph traversal algorithm (such as BFS or union-find set) is used to identify the connected components, each connected component forms a task group, and a task group set , is the kth task group, and K is the total number of task groups.

[0058] It should be noted that in the task dependency graph, the containers that do not exist in the connected component are independent task groups, and finally all the task groups formed by the received platform GPU container tasks are counted to form a task group set, and each task group includes at least one GPU container task.

[0059] In some embodiments, each graph node in the task dependency graph is bound with a demand feature vector of the container task, denoted as , including GPU card demand, calculation time consumption, communication sensitivity, is the GPU card demand of the i-th container task, is the calculation time consumption of the i-th container task, is the communication sensitivity of the i-th container task.

[0060] Specifically, the demand feature vector of each container task is obtained by analyzing the container, specifically:

[0061] A1, analyze each container task and extract the attribute features of the container task, including but not limited to time attribute, GPU card demand, GPU type (such as V100, A100, etc., which can be encoded as category features), container image type (such as TensorFlow, PyTorch, etc., which can be encoded as category features), task type label (such as "image training", "inference", "data processing", etc., which can be encoded as category features), data size, algorithm feature.

[0062] Among them, the time attribute represents the attribute of the current container task receiving time, for example, workday / weekend-time interval, which can be defined according to actual conditions and can reflect the attribute of the task execution time. It should be noted that the method for obtaining the attribute features of the container task can refer to the description of the related prior art, and the present application will not be described in detail. For example, the GPU card demand is obtained from the Kubernetes resource request, the data size is obtained from the image source data label, and the algorithm feature is obtained from the image metadata.

[0063] A2, input the attribute features of the container task into the pre-trained task communication sensitive prediction model, and output the time consumption information of the container task, including calculation time consumption and communication sensitivity.

[0064] Among them, the pre-trained task communication sensitive prediction model is obtained in the following way:

[0065] a1, collect a large number of historical container task data sets to obtain the attribute features of each historical container task and the actual calculation time (the effective calculation time of the GPU is collected from the NVIDIADCGM) and the actual communication time (the cross-container data transmission time is captured through eBPF).

[0066] a2, based on the attribute features of each historical container task, the communication sensitivity is calculated based on the actual calculation time and the actual communication time, the actual calculation time and the communication sensitivity are combined to form the time-consuming information for labeling the attribute features, and the training data set is obtained.

[0067] The communication sensitivity is set as the calculation-communication time ratio, and is calculated according to the following formula:

[0068]

[0069] is the actual calculation time of the historical container task (which can be understood as the effective calculation time of GPU), is the actual communication time of the historical container task (which can be understood as the time length of node data transmission and synchronization waiting).

[0070] a3, the preselected neural network structure is trained by using the training data set, the model parameters are continuously optimized, and the task communication sensitive prediction model is obtained.

[0071] A3, the GPU card demand, the calculation time consumption and the communication sensitivity of the container task are combined to form the demand feature vector.

[0072] Therefore, the communication sensitivity quantifies the weight proportion of the communication overhead in the total task time consumption, tends to 1, indicating that the calculation is dominant, and the communication influence can be ignored (low sensitivity, and the cross-node communication can be tolerated during scheduling), tends to 0, indicating that the communication is dominant, and the calculation efficiency is restricted by the network (high sensitivity, and the same node / same cabinet deployment is preferred during scheduling, indicating that the task has low tolerance to communication delay).

[0073] In some embodiments, the group-level demand feature vector of the task group is extracted, specifically including:

[0074] Based on the demand feature vector of each container task in the task group, the group-level demand feature vector corresponding to the task group is constructed , is the total GPU card demand of all container tasks in the kth task group, is the minimum communication sensitivity in the kth task group, is the average task dependency factor of all dependency edges in the kth task group, that is, the average value of the dependency factors corresponding to all dependency edges in the task group.

[0075] S102, based on the GPU node, a preset node multi-dimensional value evaluation algorithm is used to generate a value vector of each GPU node, and a plurality of partitions and their partition value vectors are obtained.

[0076] In some embodiments, the preset node multi-dimensional value evaluation algorithm specifically comprises:

[0077] S301, based on each GPU node, according to the topology value and the resource value of the GPU node, obtaining the value vector of the GPU node.

[0078] Specifically, step S301 comprises:

[0079] B1, obtaining the whole network topology data through the cluster management node, including the physical connection relationship between nodes (for example, the switch level: core layer / aggregation layer / access layer), the position label between nodes, and the communication weight between nodes defined in advance according to the position relationship (determined in advance according to expert experience and actual situation), and calculating the GPU node topology value according to the following formula:

[0080]

[0081] Wherein, is the topology value of node m, is the number of physical connections of node m (such as the number of direct switch ports), reflecting the hub position of node m in the network, is the set of physical adjacent nodes of node m, is the communication weight between node m and node n, quantifying the influence of physical distance on delay.

[0082] It should be noted that the communication weight between nodes is defined according to the position relationship, and is specifically set based on expert experience and actual situation. As an example, in the present embodiment, three position relationships are defined: same cabinet, same aggregation layer, and cross-machine room, which are typical data center network abstraction models. For example, the communication weight of nodes in the same cabinet is set to 1, the communication weight of nodes in the same aggregation layer is set to 2, and the communication weight of nodes across the machine room is set to 5.

[0083] In the data center network, the physical position relationship between nodes can be more refined, such as same server (communication weight 0.5), same cabinet (communication weight 1), same aggregation switch (communication weight 2), same core switch (communication weight 3), and cross-machine room (communication weight 5) nodes, etc. However, in the present embodiment, the typical data center architecture is abstracted and simplified, because the three relationships can cover the main network delay difference. The communication weights of 1, 2 and 5 are linear mappings based on the measured delay ratio (according to the typical ratio of network delay and bandwidth, such as 0.1ms, 1ms and 5ms). This integer setting is for easy calculation, although it is not normalized, but through the degree centrality weighting in the formula, the relative standardization of node value has been realized, which is equivalent to averaging the weight sum with the connection number, avoiding the high connection degree node from being overestimated.

[0084] Thus, the topology value reflects the position of the node in the network, and the communication weight between nodes is set according to the physical distance between nodes, such as 1 = same cabinet, 2 = same convergence layer, and 5 = cross machine room. Thus, the topology value falls within [1, 5], and in subsequent fusion value calculation, it can be normalized.

[0085] B2, real-time collection of node hardware indicators, including the number of available GPU cards and memory utilization, and the resource value is calculated according to the following formula:

[0086]

[0087] wherein, is the resource value of node m, is the number of available GPU cards of node m, is the memory utilization of node m.

[0088] Thus, the resource value reflects the hardware resource status of the node, avoiding zero-value interference, amplifying resource differences, and the higher the memory idle rate, the greater the value. In subsequent fusion resource calculation, it can be normalized.

[0089] B3, the topology value and the resource value are weighted and summed to obtain the fusion value, and the GPU generation standard deviation forms a value vector , is the fusion value of node m, is the GPU generation standard deviation of node m, N represents the number of GPU cards in node m, represents the rth GPU generation number, i.e., the hardware generation code, represents the average generation of GPU in node m, i.e., the average value of the generation of GPU in node m. For example, node m includes 2 A100 (gen = 10) and 2 V100 (gen = 9), = (10 + 10 + 9 + 9) / 4 = 9.5, = 0.5. It can be understood that there is a difference in the interconnection bandwidth of different generation GPUs. If = 0, it means that the GPU is fully homogeneous, and the performance is optimal. If is greater than 1, it means that mixed cross-generation hardware is used, and the NVLink bandwidth decreases.

[0090] wherein, the weight values of the topology value and the resource value are set according to the actual situation and expert experience, for balancing the topology advantage and the hardware performance. For example, the weight value of the topology value is set to 0.6, and the weight value of the resource value is set to 0.4. The GPU generation standard deviation represents the hardware generation difference in the node, avoiding the performance degradation of cross-generation GPU cooperation (such as mixing A100 and V100).

[0091] S302, dividing the GPU nodes according to the value vectors of all GPU nodes to obtain a plurality of partitions and partition value vectors , is the number of available GPU cards in the zth partition, is the average fusion value of the zth partition, is the maximum value of the standard deviation of the GPU generation in the zth partition.

[0092] Exemplarily, dividing the GPU nodes according to the value vectors of all GPU nodes specifically comprises:

[0093] C1, sorting all nodes in descending order of fusion value, and eliminating nodes with a standard deviation of GPU generation greater than a generation threshold (for example, set to 1) to generate a GPU node sequence;

[0094] C2, dividing the upper quartile nodes of the GPU node sequence to the core area, dividing the lower quartile nodes to the edge area, and dividing the remaining nodes to the intermediate area.

[0095] In other examples, based on the value vector, the fusion value and the standard deviation of the GPU generation, the standard deviation of the GPU generation can be used as an adjustment factor of the fusion value to determine a comprehensive score. The higher the fusion value, the higher the comprehensive score, and the node is more likely to be divided into the core area. The higher the standard deviation, the more likely the node is to be downgraded (for example, from the core area to the intermediate area or the edge area), because a high standard deviation means poor hardware compatibility. Exemplarily, the comprehensive score is set as: , is the fusion value, is the standard deviation of the GPU generation, is a preset compatibility tolerance threshold, set to a fixed value of 1 (an experienced safe value), and further, the nodes are sorted in descending order based on the comprehensive score, the upper quartile is taken as the core area, the lower quartile is taken as the edge area, and the rest is taken as the intermediate area.

[0096] In other examples, the value vectors of all nodes can be subjected to cluster analysis (K-means algorithm) to generate a plurality of clusters, each cluster including at least one node, and each cluster corresponding to a partition.

[0097] Thus, based on the value vectors of the GPU nodes, the network topology position and the hardware resource status are comprehensively considered to select the most suitable deployment area for different task groups.

[0098] S103, determining the target partition of each task group through a group-partition matching model based on the group-level demand feature vector of the task group and the partition value vector. ​

[0099] In some embodiments, the target partition of each task group is determined by a group-partition matching model, specifically including:

[0100] D1, a partition adaptation scoring algorithm S(k, z) is constructed to quantify the matching degree of task group k and partition z, and the scoring function is composed of three sub-indexes of GPU resource matching degree, communication sensitivity adaptation degree, and task dependency strength adaptation degree, which are weighted:

[0101]

[0102] wherein, is the matching degree of task group k and partition z, , , is a preset weight value, which is dynamically adjusted according to the task type (computation-intensive / communication-intensive) and expert experience, and satisfies The weight is dynamically adjusted according to the task type (for example, the computation-intensive task increases w1, and the communication-intensive task increases w2).

[0103] Specifically:

[0104] is the GPU resource matching degree, if the partition resource is insufficient, i.e. =0, directly exclude this partition;

[0105] is the communication sensitivity adaptation degree, , is a preset maximum value of the global GPU generation standard deviation, used for normalizing , if the task group communication sensitivity is low ( ), it means that the task is not sensitive to communication delay, at this time, the partition with low hardware heterogeneity ( ) is preferred to avoid the performance degradation of cross-generation GPU; if the communication sensitivity is high ( ), it means that the task is sensitive to communication delay, at this time, the partition with high topology value ( ) and low hardware heterogeneity is preferred to reduce network delay;

[0106] is the task dependency strength adaptation degree, , is a threshold value (for example, set to 0.6) preset for the task dependency factor, used to distinguish strong dependency task groups, if the dependency strength in the task group is high ( ), it means that the data interaction between tasks is frequent, at this time, the partition with high fusion value ( partitioning to support parallel optimization (e.g., data locality optimization), and if the dependency strength is low, only the partitioning base value is considered.

[0107] D2, obtain the matching degree between the task group and each partition by partition adaptation scoring algorithm, and select the partition with the highest matching degree as the target partition of the task group.

[0108] It should be noted that if multiple task groups compete for a unified partition, they are allocated in descending order of matching degree. Unallocated task groups enter the waiting queue, or enter the next round of matching, or are allocated to the partition corresponding to the next matching degree.

[0109] D3, lock GPU resources for the task group in the target partition, and the number of locks is the total number of GPU cards required by the tasks in the group multiplied by a dynamic coefficient (1.2-0.2 , that is, the group with high tightness reserves more resources to avoid fragmentation.

[0110] Thus, the task group is allocated to the most suitable partition, and resources are reserved to ensure the integrity of high-tightness task groups. By quantifying the matching degree of task group demand and partition value, efficient use of GPU resources and optimization of task performance are achieved. At the same time, considering resources, communication and dependency relationship, avoiding the limitation of single index, avoiding the performance degradation of mixed hardware through GPU generation standard deviation.

[0111] S104, based on the target partition determined by the task group, a resource allocation scheme for the container tasks in the task group is generated using a task group resource allocation decision mechanism.

[0112] In some embodiments, the task group resource allocation decision mechanism specifically includes:

[0113] S401, sort all task groups in ascending order of total GPU card demand in the group to form a scheduling queue.

[0114] S402, based on the container tasks in each task group (in the order of the scheduling queue), traverse the resource queue in the corresponding target partition, lock the resource pool (the locked node pool), and in the pool, generate at least one resource allocation scheme. For each resource allocation scheme, use the pre-constructed dynamic loss function to obtain the loss value, and select the resource allocation scheme with the smallest loss value as the target allocation scheme.

[0115] Wherein, the locked resource pool is: traversing the resource queue of the target partition, finding a set of nodes that can meet the total GPU card demand of the task group, locking the resources of these nodes to prevent other task groups from occupying. In the locked node pool, different resource allocation schemes for container tasks are tried. The resource allocation scheme needs to meet: the GPU card demand of each container task can be met on a certain node, and the total GPU card resources of the node are not over-allocated.

[0116] Specifically, the pre-constructed dynamic loss algorithm is set as:

[0117]

[0118] wherein, is the loss value of the resource allocation scheme, is the communication sensitivity of the container task, is the GPU card demand of the i-th container task, and S is the number of nodes allocated to the container task (i.e., the number of nodes involved in the resource allocation scheme of the container task), is the actual number of GPU cards allocated to the task container on node s, represents the number of GPU cards that need to perform cross-node data transmission among the GPU resources allocated to the container task on node s.

[0119] represents the total number of GPU pairs, which refers to the number of combinations of GPU cards that need to communicate with each other within the container task. For example, for a container task that requires N GPU cards, the total number of GPU pairs is ; represents the number of cross-node GPU pairs, which refers to the number of GPU pairs that are forced to be distributed across multiple nodes (these GPU pairs originally need to communicate, but due to allocation to different nodes, communication overhead is generated). For example, if 4 GPU cards are allocated to 2 nodes (2 cards per node), the number of cross-node pairs is: node A has GPU1-2, node B has GPU3-4, the cross-node pairs include (1,3)(1,4)(2,3)(2,4), so there are 4 pairs.

[0120] Therefore, when the number of cross-node pairs / total pairs tends to 1, it means that almost all GPU pairs are scattered, and the larger the ratio, the larger the communication overhead. Therefore, as a penalty term, the In combination, the communication overhead ratio is more accurately quantified, which can accurately distinguish the difference between "partial dispersion" and "complete dispersion". High-interaction GPU pairs are scattered with high penalties, and low-interaction GPU pairs are scattered with low penalties, which is particularly effective for medium-sized tasks (such as 8-16 GPU cards).

[0121] As a weight value, the weight is dynamically adjusted according to the task characteristics. For high tasks, avoid idle computing resources due to excessive optimization of communication, and for low tasks, prefer to minimize communication overhead. Exemplarily, when > 0.7, it indicates that the task type is biased towards computation-intensive, so the communication constraint needs to be relaxed to Dominant, allow GPU scattered deployment; When <0.3, it indicates that the task type is communication-intensive, so strict restrictions are needed for cross-node allocation to Dominant, strengthen cross-node punishment, and ensure that high-interaction GPUs are deployed on the same node.

[0122] It can be understood that when selecting a node combination for each container task within the target partition allocated by the task group, a dynamic loss function is used to determine the optimal combination, i.e., the target allocation scheme. When scheduling tasks within the group, high-dependence tasks are preferentially allocated to nodes with low communication overhead.

[0123] S403, according to the task dependency DAG (Directed Acyclic Graph) of the task group, determine the execution order of the tasks within the group, and schedule resources for the container tasks based on the target allocation scheme.

[0124] Specifically, topological sorting is performed on the DAG to determine the execution order of the container tasks, and each container task is started only after the completion of its preceding tasks. The container tasks are submitted to the allocated resource nodes for execution in topological order.

[0125] Thus, a group of nodes is allocated for the task group, and a node allocation scheme for each container task within the group is determined, reducing communication overhead within the task group, avoiding resource competition between groups, and improving overall resource utilization.

[0126] As an example, the implementation process of step S104 is described in detail:

[0127] E1, the task group set G={ }, each task group includes a container task set and the number of GPU cards required; the target partition resource queue R of the task group={ }, each node has available GPU card quantity .

[0128] E2, calculate the total GPU card requirement total_gpu( ) of each task group , and sort G in ascending order.

[0129] E3, for each task group in the sorted G:

[0130] find the smallest node set S R that satisfies total_gpu( )≤ ;

[0131] lock the resources in S;

[0132] generate all possible allocation schemes (assigning container tasks in to the nodes of S, satisfying the GPU card requirements of individual container tasks);

[0133] calculate the loss value for each allocation scheme, and select the allocation scheme with the smallest loss value ;

[0134] According to the allocation scheme allocate resources;

[0135] build DAG of, and perform topological sorting to obtain the execution order ;

[0136] According to schedule container task execution.

[0137] E4, output resource allocation scheme (container task to node mapping) and container task execution order.

[0138] Thus, the task group sorting prioritizes small demand groups, dynamically locks the resource pool and selects the smallest loss allocation scheme, relies on DAG scheduling to ensure task execution order, and can efficiently allocate resources to task groups in the target partition and minimize loss value.

[0139] In summary, in traditional scheduling, multiple related tasks (such as multiple stages in an AI inference pipeline) are independently scheduled, resulting in large cross-task communication overhead; the demand of the task group (such as communication-intensive) does not match the characteristics of the node (such as network location, hardware resources), causing performance bottlenecks; static task group division and resource allocation cannot adapt to dynamically changing communication patterns.

[0140] Therefore, in the embodiments of the present application, each platform GPU task is encapsulated into a container task containing various information, ensuring environmental compatibility, using a dynamic task dependency graph clustering mechanism to identify task groups, analyzing container task attribute characteristics, inputting task communication sensitive prediction model to obtain time-consuming information, forming a demand feature vector of the container task, and then extracting a group-level demand feature vector of the task group. The dynamic task dependency graph clustering mechanism quantifies the data dependency relationship between container tasks, constructs a task dependency graph and identifies highly correlated task groups, and the demand feature vector and group-level demand feature vector accurately reflect the resource demand of the task and task group, providing a basis for reasonable scheduling.

[0141] Based on the node multi-dimensional value evaluation algorithm, the GPU node topology value is calculated by acquiring the whole network topology data, the resource value is calculated by real-time collection of node hardware indicators, the fusion value is obtained by weighted summation, and the value vector is composed of the GPU generation standard deviation. According to the value vector of all GPU nodes, the GPU nodes are divided by sorting and eliminating, proportionally dividing or clustering analysis, and a plurality of partitions and partition value vectors are obtained. The value vector comprehensively considers the network topology position and hardware resource status, accurately evaluates the node value, and the partition and partition value vector select the most suitable deployment area for different task groups, thereby improving the resource utilization efficiency.

[0142] Through the group-partition matching model, a partition adaptation score algorithm is constructed to quantify the matching degree of the task group and the partition, select the partition with the highest matching degree as the target partition, and lock the GPU resources for the task group in the target partition. The task group is allocated to the most suitable partition, and resources are reserved to ensure the resource integrity of the high-tightness task group, realize the efficient utilization of GPU resources and the optimization of task performance, and at the same time avoid the limitation of a single index and the performance degradation of mixed hardware.

[0143] Using the task group resource allocation decision mechanism, the task groups are sorted in ascending order of total GPU card demand in the group to form a scheduling queue, the resource queue in the target partition is traversed to lock the resource pool, the resource allocation scheme is generated in the pool, the dynamic loss function (combined with communication sensitivity and cross-node GPU logarithm) is used to select the scheme with the minimum loss value as the target allocation scheme, and the container tasks are scheduled based on the target allocation scheme. A group of nodes is allocated to the task group, and the node allocation scheme of each container task in the group is determined, the communication overhead in the task group is reduced, the inter-group resource competition is avoided, and the overall resource utilization is improved.

[0144] Therefore, by dynamic task dependency graph clustering, high-correlation task groups are allocated to the same partition, the cross-node communication overhead is reduced, and the parallel optimization capability is improved; the node partition division comprehensively considers the topology position and hardware resources, avoids the performance degradation of mixed GPU cooperation, and ensures the accurate matching of resource allocation and task demand. When performing group-partition matching, the communication sensitivity adaptation index guides the deployment of high-sensitivity tasks to partitions with high topology value, reducing the influence of network delay; the task dependency strength adaptation prioritizes the allocation of partitions with high fusion value to strongly dependent task groups, supporting data locality optimization. The dynamic task dependency factor algorithm and the loss function design adapt to different task types (computation-intensive / communication-intensive) and real-time resource state changes. The resource locking and waiting queue mechanism solves the resource conflict problem when multiple task groups compete for the same partition.

[0145] The technical solutions in the above embodiments of the application have at least the following technical effects or advantages:

[0146] By dynamic task dependency graph clustering, high correlation tasks are aggregated into task groups to reduce cross-task communication overhead; combined with topology location and hardware resources and hardware compatibility, the node value is comprehensively evaluated. The node is divided into core area, edge area, etc., to realize resource hierarchical management and ensure that high demand tasks are allocated to high quality resource area. Through multi-dimensional scoring algorithm (resource matching degree, communication sensitivity adaptation degree, dependence strength adaptation degree), the task group is allocated to the most suitable partition to avoid resource mismatch. The dynamic loss function considers the computational intensity and communication overhead when allocating container tasks, achieving a balance between resource distribution balance and communication overhead minimization.

[0147] Through dynamic task dependency graph clustering, node multi-dimensional value evaluation, group-partition matching model and dynamic resource allocation decision, efficient utilization of GPU resources and task performance optimization are realized, solving the problems of insufficient task dependency quantization, node heterogeneity influence and single index limitation of resource allocation in traditional scheduling methods, and suitable for containerized task scheduling scenarios in large-scale, heterogeneous GPU cluster environment.

[0148] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A container scheduling method of GPU resources, characterized in that, The method comprises the following steps: S101, receive GPU container tasks of each platform, generate at least one task group by using a preset dynamic task dependency graph clustering mechanism, and extract a group-level demand feature vector of the task group; the preset dynamic task dependency graph clustering mechanism specifically comprises: S201, obtaining historical communication data between different container tasks in a preset time period, including source container ID, target container ID, communication timestamp, data transmission volume, and communication frequency, aggregating the data according to container pairs, and calculating the historical total flow and average interaction frequency of the container pairs; S202, based on the received GPU container tasks of each platform, generating all container pairs, combining the historical communication data, and calculating the dependency factor of the container pairs; S203, according to the task dependency factor of all container pairs, taking the container as a graph node, and taking the task dependency factor as an edge weight, constructing a task dependency graph, retaining edges whose task dependency factors satisfy a preset dependency threshold, identifying connected components, forming a task group for each connected component, and generating a task group set , is the kth task group, and K is the total number of task groups; In S102, based on the GPU nodes, a preset node multi-dimensional value evaluation algorithm is used to generate a value vector of each GPU node, to obtain a plurality of partitions and partition value vectors; the preset node multi-dimensional value evaluation algorithm specifically comprises: in S301, based on each GPU node, a value vector of the GPU node is obtained according to a topology value and a resource value of the GPU node; the value vector is obtained in the following manner: the topology value and the resource value are weighted and summed to obtain a fusion value, and the fusion value and a GPU generation standard deviation form the value vector , is the fusion value of the node m, is the GPU generation standard deviation of the node m; in S302, the GPU nodes are divided according to the value vectors of all the GPU nodes, to obtain a plurality of partitions and partition value vectors , is the number of available GPU cards in the zth partition, is the average fusion value of the zth partition, is the maximum value of the GPU generation standard deviation in the zth partition; S103, determining the target partition of each task group based on the group-level demand feature vector and the partition value vector of the task group through a group-partition matching model; S104, generating a GPU allocation scheme for the task group by using a task group resource allocation decision mechanism based on the target partition determined by the task group.

2. The method of claim 1, wherein, The dependency factor is calculated according to the following formula: wherein, is a task dependency factor for container pair i and j, is the historical total flow from container i to container j, is the average interaction frequency for container pair, is the total outflow of container i, is the global average interaction frequency of container i, is a preset weight factor for reflecting the importance degree of data size dependency on the task dependency factor. 3.The method of claim 1, wherein, Each graph node in the task dependency graph is bound with a demand feature vector of the container task, denoted as , is the GPU card demand of the i-th container task, is the computation time consumption of the i-th container task, is the communication sensitivity of the i-th container task.

4. The method of claim 3, wherein, The group-level demand feature vector of the extraction task group specifically comprises: Based on the demand feature vector of each container task in the task group, a group-level demand feature vector corresponding to the task group is constructed , is the total GPU card demand of all container tasks in the kth task group, is the minimum communication sensitivity in the kth task group, is the average task dependency factor of all dependency edges in the kth task group, that is, the average value of the dependency factors corresponding to all dependency edges in the task group.

5. The method of claim 4, wherein, The target partition of each task group is determined through a group-partition matching model, specifically comprising: D1, constructing a partition adaptation scoring algorithm S(k, z) to quantify the matching degree of the task group k and the partition z, and the scoring algorithm is composed of three sub-indexes of GPU resource matching degree, communication sensitivity adaptation degree and task dependency strength adaptation degree; D2, obtaining the matching degree between the task group and each partition through the partition adaptation scoring algorithm, and selecting the partition with the highest matching degree as the target partition of the task group; D3, locking the GPU resources in the target partition for the task group.

6. The method of claim 5, wherein, The task group resource allocation decision mechanism specifically comprises: S401, sorting all task groups in ascending order of total GPU card demand in the group to form a scheduling queue; S402, based on the container tasks in each task group, traversing the resource queue in the corresponding target partition, locking the resource pool, and in the pool, generating at least one resource allocation scheme, for each resource allocation scheme, using the pre-constructed dynamic loss function to obtain the loss value, selecting the resource allocation scheme with the smallest loss value as the target allocation scheme, and based on the target allocation scheme, scheduling the container tasks in the task group.

7. The method of claim 6, wherein, In D1, the adaptation scoring algorithm is set as: wherein, is the matching degree of task group k and partition z, 、 、 is a preset weight value, dynamically adjusted according to task type and expert experience, to meet , is the GPU resource matching degree, is the communication sensitivity adaptation degree, is the task dependency strength adaptation degree.

8. The method of claim 3, wherein, The demand feature vector of the container task is obtained in the following way: A1, extracting the attribute features of the container task; A2, inputting the attribute features of the container task into a pre-trained task communication sensitivity prediction model to output the time-consuming information of the container task, and the time-consuming information includes calculation time-consuming and communication sensitivity; A3, the GPU card demand, calculation time-consuming and communication sensitivity of the container task are combined to form a demand feature vector.

Images