An AI training scene-oriented computing power resource dynamic scheduling and optimization method

Abstract

The application discloses an AI training scene-oriented computing power resource dynamic scheduling and optimization method, and relates to the technical field of computing power resource scheduling. The method comprises the following steps: performing dynamic and static separation resource pool division on a cluster hardware node to obtain a static computing power pool and a dynamic computing power pool. The task graph and hardware core utilization rate of each training task are collected and input into a first dynamic scheduling model, and the correlation degree of software and hardware features is extracted by using an attention mechanism. Based on the correlation degree, a Nash equilibrium solver is used to calculate the resource allocation proportion of each task in the dynamic computing power pool. According to the allocation proportion, the hardware core is allocated to each training task for execution from the dynamic computing power pool. The application combines dynamic and static separation resource pool architecture and game theory optimization, realizes fine allocation and dynamic adaptation of heterogeneous computing power, effectively improves the cluster training efficiency and resource utilization rate, and guarantees the calculation fairness and system stability in the multi-task concurrent scene.

Description

Technical Field

[0001] This invention relates to the field of computing resource scheduling technology, specifically a method for dynamic scheduling and optimization of computing resources for AI training scenarios. Background Technology

[0002] In computing power scheduling systems for AI training scenarios, when multiple training tasks compete for limited hardware core resources, the system struggles to maintain a balanced resource allocation and maximize training efficiency under high load. Existing technologies typically employ static partitioning or dynamic allocation based on simple thresholds. Static partitioning pre-allocates hardware resources to fixed tasks, failing to adapt to real-time fluctuations in task computing power demands during training. This results in some hardware cores remaining idle while tasks wait for data, while other tasks experience congestion due to insufficient resources. Dynamic allocation based on simple thresholds triggers migration operations by monitoring hardware core occupancy, but these methods focus only on a single-dimensional load metric, ignoring the complex matching relationship between graph structure features such as operator types, tensor dimensions, and communication bandwidth within training tasks and hardware core cache hit rates and memory bandwidth utilization. This can easily lead to frequent task rescheduling, causing additional time overhead and cache invalidation.

[0003] Existing technical solutions suffer from the following drawbacks: Firstly, they lack methods for modeling the deep-level correlation between the computational graph features of training tasks and the microarchitecture state of hardware cores. This prevents the capture of the true affinity differences between tasks and different hardware cores at the operator level, leading to allocation decisions relying on coarse scalar load values ​​rather than multi-dimensional feature correlations, making it difficult for scheduling results to approach the global optimum. Secondly, when facing multiple tasks competing for the same batch of dynamic resources, existing solutions generally adopt a greedy first-come, first-served or proportional share allocation, failing to formalize the resource allocation process as a game problem where the gains of each task are mutually constrained. They lack consideration for the differentiated priorities and diminishing marginal returns of tasks, causing the resource ratios obtained by each task to deviate from the equilibrium point, with some tasks occupying redundant resources while the training throughput of other tasks is severely limited.

[0004] This invention aims to solve the problem of extracting the correlation between software and hardware features of task graphs and hardware core utilization, as well as the problem of solving the problem of balanced allocation ratio in dynamic resource allocation of multiple tasks. Summary of the Invention

[0005] The purpose of this invention is to provide a dynamic scheduling and optimization method for computing resources in AI training scenarios. By dividing the resource pool into static and dynamic pools to clearly define the schedulable boundaries, the multi-dimensional task graph and hardware core utilization are jointly modeled to extract the correlation between software and hardware features. Using this as input, a Nash equilibrium solution process for inter-task game is constructed to obtain the multi-task computing resource allocation ratio in the dynamic computing pool, thereby improving the adaptability of resource allocation and training throughput when multiple training tasks are concurrent in heterogeneous clusters.

[0006] To achieve the above objectives, the present invention provides the following technical solution: The present invention provides a method for dynamic scheduling and optimization of computing resources for AI training scenarios, comprising: dividing hardware nodes in a heterogeneous computing power cluster into static and dynamic resource pools to obtain a static computing power pool and a dynamic computing power pool; collecting the task graph of each training task and the hardware core utilization rate of the hardware nodes; inputting the task graph and the hardware core utilization rate into a first dynamic scheduling model, and extracting the software and hardware feature correlation degree through an attention mechanism; based on the software and hardware feature correlation degree, calculating the computing power resource allocation ratio of each training task in the dynamic computing power pool using a Nash equalization solver; and scheduling hardware cores from the dynamic computing power pool to allocate to each training task for execution according to the computing power resource allocation ratio.

[0007] This method separates static and dynamic nodes, incorporating high-utilization and stable nodes into a static computing pool to ensure deterministic resource supply for core training. Simultaneously, it clusters the remaining nodes into a dynamic computing pool based on memory frequency and network bandwidth, enabling elastic aggregation of heterogeneous resources and improving overall cluster resource flow efficiency. As a preferred implementation, the static-dynamic separation includes: marking nodes with utilization rates exceeding a historical average utilization threshold as static nodes, collecting topology information and maximum resource capacity of these static nodes to construct the static computing pool; and clustering the remaining nodes based on memory frequency and network bandwidth to construct the dynamic computing pool, ensuring that nodes within the dynamic pool exhibit consistent memory access and communication characteristics, reducing performance fluctuations during cross-node scheduling.

[0008] When collecting task graphs and hardware core utilization, the execution graphs of each training task are analyzed, and the operator type, data tensor dimension, and communication edge bandwidth of each computing node are extracted to construct a task graph in the form of a directed acyclic graph. The operator type is mapped to a one-hot encoded vector, the data tensor dimension is mapped to a multi-dimensional size feature, and the communication edge bandwidth is mapped to a scalar weight, thereby finely characterizing the computation and communication patterns of the task. The instantaneous occupancy rate, cache hit rate, and memory bandwidth utilization of each hardware core are polled from the cluster management interface to construct the hardware core utilization, which reflects the load of the hardware core and the microarchitecture competition status in real time, providing an accurate data foundation for subsequent feature association.

[0009] When inputting the task graph and hardware core utilization into the first dynamic scheduling model to extract the software-hardware feature correlation, preferably, the node feature matrix and adjacency matrix of the task graph are concatenated with the hardware core utilization to form a joint feature tensor. In the self-attention layer of the model, the similarity score between each query vector and all key vectors in the joint feature tensor is calculated to generate an attention weight matrix. This matrix is ​​then normalized to output a software-hardware feature correlation that reflects the matching strength between task computation requirements and hardware core load status. More preferably, the self-attention layer adopts a multi-head attention mechanism, dividing the joint feature tensor into multiple subspaces, calculating the attention weight matrix of each subspace, concatenating them, and performing a linear transformation to obtain a multi-angle fused software-hardware feature correlation. This effectively captures the deep dependencies between heterogeneous tasks and heterogeneous cores, improving the targeting of allocation decisions.

[0010] When calculating the computing resource allocation ratio using the Nash equilibrium solver based on the aforementioned hardware-software feature correlation, each training task is treated as a game participant, and the number of hardware cores in the dynamic computing pool is used as a resource constraint. A random resource allocation ratio matrix is ​​initialized. A payoff function for each task is constructed based on the hardware-software feature correlation, reflecting the partial derivative of the task's training throughput with respect to resource allocation. The resource allocation ratio matrix is ​​iteratively updated until the difference between the partial derivatives of the payoff functions of all tasks with respect to resource allocation is less than a preset convergence threshold. The computing resource allocation ratio of each training task in the dynamic computing pool is then output. The payoff function further incorporates a task priority factor, which is dynamically adjusted based on the historical convergence speed of the training tasks and serves as a weighting coefficient for the payoff function. This ensures that the scheduling process balances fairness and priority, prioritizing the resource needs of convergence bottleneck tasks and accelerating the convergence of global training.

[0011] When allocating hardware cores to each training task according to the computing power resource allocation ratio, the ratio is converted into an integer allocation scheme for the number of hardware cores. The allocated hardware cores are then bound to the corresponding training task processes through kernel binding system calls, eliminating cache invalidation and context switching overhead caused by task migration between different cores. At the same time, the input and output requests of the bound hardware cores are processed by priority-based sorting queue caching to reduce multi-task concurrent I / O contention, reduce data waiting latency, and improve the stability of training throughput.

[0012] As an extension, when new hardware nodes are dynamically added to the heterogeneous computing power cluster, the capacity configurations of the static and dynamic computing power pools are updated based on the information of the new nodes, and the resource constraint dimensions of the first dynamic scheduling model are adjusted accordingly. This allows the scheduling strategy to adapt to changes in cluster size and maintain the real-time optimality of scheduling decisions. Through the above steps, this method achieves multi-task dynamic game scheduling with deep integration of software and hardware features. While ensuring training performance and fair resource allocation, it significantly improves the overall utilization and concurrent execution efficiency of the heterogeneous computing power cluster.

[0013] The technical effects and advantages provided by the present invention in the above technical solution are as follows: By concatenating the node feature matrix and adjacency matrix of the task graph with hardware core utilization to form a joint feature tensor, and calculating the similarity score between each query vector and all key vectors in the self-attention layer of the first dynamic scheduling model to generate an attention weight matrix, the software and hardware feature correlation is output. This allows the capture of nonlinear cross-modal interaction features from operator type, data tensor dimension, communication edge bandwidth, hardware core instantaneous occupancy, cache hit rate, and memory bandwidth utilization. Compared with scheduling methods that rely solely on a single utilization threshold, this feature correlation accurately characterizes the response differences of different hardware cores to different computation graph regions. This enables subsequent allocation decisions to match the most suitable hardware core combination for the graph structure attributes of each task, avoiding the scheduling of computationally intensive operators to cores with high cache hit rates but weak floating-point computing power, reducing irregular fluctuations in operator execution time, and improving the stability of overall training throughput.

[0014] Based on the correlation between hardware and software features, each training task is treated as a game participant. The number of hardware cores in the dynamic computing power pool is used as a resource constraint, and a random resource allocation ratio matrix is ​​initialized. The payoff function, constructed using the partial derivative of the task training throughput with respect to resource allocation, is iteratively updated until the difference in the partial derivatives of the payoff functions of all tasks is less than a preset convergence threshold, thus obtaining the computing power resource allocation ratio. Under the condition of multi-task competition for limited dynamic resources, an equilibrium allocation is achieved where the marginal payoffs of all parties tend to be consistent. Unlike allocation methods based on proportional share or first-come, first-served, the Nash equilibrium solution, under the constraint of the correlation between hardware and software features, ensures that the amount of resources acquired by each task matches its actual affinity for hardware cores. The priority factor dynamically adjusts the weighting coefficient of the payoff function based on the historical convergence speed, preventing tasks that are close to convergence from occupying a large amount of redundant resources, and fully providing the remaining hardware cores to tasks with slow convergence, thus narrowing the gap in training completion time between different tasks and improving the overall training efficiency of the task group. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0016] Figure 1 It is a flowchart of a method for dynamic scheduling and optimization of computing resources for AI training scenarios; Figure 2 This is a flowchart of the task graph construction and hardware core utilization data acquisition process; Figure 3 This is a schematic diagram of the software and hardware feature correlation calculation process; Figure 4 This is a flowchart of the dynamic and static separation resource pool division and dynamic update process; Figure 5 It is a curve showing the distribution of tensor dimensions and communication edge bandwidth of the training task's computation nodes; Figure 6 This is a probability density distribution diagram of attention scores in the subspace of the multi-head self-attention mechanism; Figure 7 It is a curve showing the change in the proportion of computing resources allocated to the training task as a function of the number of iterations; Figure 8 It is a cumulative distribution curve of input / output request latency based on task priority factors. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] See Figure 1 This invention provides a method for dynamic scheduling and optimization of computing resources for AI training scenarios, comprising: dividing hardware nodes in a heterogeneous computing cluster into static and dynamic resource pools to obtain a static computing pool and a dynamic computing pool; collecting the task graph of each training task and the hardware core utilization of the hardware nodes; inputting the task graph and hardware core utilization into a first dynamic scheduling model, and extracting the correlation between software and hardware features through an attention mechanism; calculating the allocation ratio of computing resources for each training task in the dynamic computing pool based on the correlation between software and hardware features using a Nash equilibrium solver; and scheduling hardware cores from the dynamic computing pool to allocate to each training task for execution according to the allocation ratio of computing resources.

[0019] In the static / dynamic resource pool partitioning process, nodes are classified into static nodes and dynamic candidate nodes based on their historical average utilization rate. Static nodes form a static computing pool, while dynamic candidate nodes are clustered to form a dynamic computing pool. The task graph acquired during the acquisition phase stores the operator type, tensor dimension, and communication edge bandwidth of each training task in the form of a directed acyclic graph. Hardware core utilization is obtained through the cluster interface, including the instantaneous occupancy rate, cache hit rate, and memory bandwidth utilization of each hardware core. The first dynamic scheduling model concatenates the node feature matrix and adjacency matrix of the task graph with the hardware core utilization rate to form a joint feature tensor, and uses a self-attention mechanism to calculate the correlation between software and hardware features. The Nash equilibrium solver treats each training task as a game participant, uses the number of hardware cores in the dynamic computing pool as a resource constraint, constructs a payoff function based on the correlation between software and hardware features, and iteratively solves for the computing resource allocation ratio. During scheduling execution, the allocation ratio is converted into an integer core allocation scheme, and the hardware cores are bound to the training task processes through kernel binding to complete task scheduling.

[0020] Example 1

[0021] In specific implementation, please refer to Figure 2 The implementation method for collecting the task graph of each training task and the hardware core utilization of hardware nodes is as follows.

[0022] The training task is parsed to obtain the execution graph generated during compilation. The execution graph contains multiple computation nodes and multiple communication edges. Each computation node represents an operator operation, and each communication edge represents data transfer between two computation nodes. During execution graph parsing, each computation node is traversed to extract its operator type, input data tensor dimensions, and output data tensor dimensions. Each communication edge is traversed to extract the amount of data transmitted and its transmission delay metric. The communication edge bandwidth is then calculated based on these metrics. The communication edge bandwidth can be obtained by dividing the data volume by the transmission delay metric, representing the amount of data that the communication edge can transmit per unit time.

[0023] Based on the extracted information, a task graph is constructed, represented using a directed acyclic graph (DAG). During DAG construction, each computation node is mapped to a node in the DAG, and each communication edge is mapped to a directed edge in the DAG. For each node, the corresponding operator type is mapped to a one-hot encoded vector. Specifically, a set of operator types containing multiple operator types is pre-constructed, with each operator type corresponding to one dimension of the one-hot encoded vector. For a given computation node, the position of the node's operator type within the operator type set is determined, and the dimension of the one-hot encoded vector corresponding to that position is set to 1, while the remaining dimensions of the one-hot encoded vector are set to 0, forming the operator type one-hot encoded vector. Simultaneously, the input and output data tensor dimensions of the computation node are organized into multi-dimensional size features. The input data tensor dimension represents the size of each dimension of the input tensor, and the output data tensor dimension represents the size of each dimension of the output tensor. The multi-dimensional size feature is a numerical sequence formed by concatenating the input and output data tensor dimensions in a preset order. For each directed edge, the communication edge bandwidth is mapped to a scalar weight, which is the numerical value of the communication edge bandwidth. This scalar weight is directly assigned to the directed edge as a weight attribute.

[0024] In some embodiments, the node feature matrix of a directed acyclic graph is composed of the one-hot encoded vectors of all node operator types and multidimensional size features concatenated together, while the adjacency matrix records the connection relationships between nodes and the scalar weights of each directed edge.

[0025] On the other hand, the utilization rate of hardware cores of hardware nodes is collected. The instantaneous occupancy rate, cache hit rate, and memory bandwidth utilization rate of each hardware core are read from the cluster management interface of the heterogeneous computing power cluster using a polling method. The cluster management interface exposes the status register data of the hardware cores through the system performance monitoring interface or the hardware driver layer interface. The instantaneous occupancy rate is the percentage of clock cycles in which the hardware core is active within a sampling period; the cache hit rate is the ratio of the number of last-level cache access hits to the total number of cache accesses; and the memory bandwidth utilization rate is the proportion of currently used memory bandwidth to the theoretical peak memory bandwidth of the hardware core. A fixed polling interval can be set, and at each polling time, a status read request is sent to the cluster management interface to obtain the real-time values ​​of the instantaneous occupancy rate, cache hit rate, and memory bandwidth utilization rate of each hardware core.

[0026] The instantaneous utilization rate, cache hit rate, and memory bandwidth utilization rate read each time are combined to form the hardware core utilization information for the corresponding hardware core. Optionally, the various indicators in the hardware core utilization information are normalized. The instantaneous utilization rate is normalized by dividing the original instantaneous utilization rate by 100%. The cache hit rate is already between 0 and 1 and does not require additional scaling. The memory bandwidth utilization rate is normalized by dividing the original memory bandwidth utilization rate by 100%, thereby obtaining a hardware core utilization feature vector with values ​​in the range of 0 to 1 for each dimension. The hardware core utilization feature vectors of all hardware cores are summarized to obtain the hardware core utilization data.

[0027] In this way, the task graph is constructed as a directed acyclic graph structure with node one-hot encoding features, multi-dimensional size features, and edge scalar weights, and the hardware core utilization is constructed as a set of values ​​containing normalized hardware core indicators, thereby completing the acquisition of the task graph and hardware core utilization.

[0028] See Figure 5 In the figure, the vertical axis represents the cumulative probability, ranging from 0 to 1, and the horizontal axis represents the logarithmic scale of the values, indicating the numerical distribution of the input data tensor dimensions, output data tensor dimensions, and communication bandwidth. In the legend, the solid curve represents the cumulative distribution function of the input data tensor dimensions, the dashed curve represents the cumulative distribution function of the output data tensor dimensions, and the dotted dashed curve represents the cumulative distribution function of the communication bandwidth.

[0029] As can be seen from the figure, the numerical values ​​of the input data tensor dimensions are mainly distributed in approximately to Within the range of 0.1 to 0.9, the cumulative probability shows a gradual increase, indicating that the input tensor size has a wide numerical range and a median value of approximately 0.1. Order of magnitude. The numerical range of the output data tensor dimension is similar to that of the input data tensor dimension, but its cumulative probability curve is slightly shifted to the left, indicating that the output tensor size is generally smaller than the input tensor size, and its distribution range is relatively concentrated. The communication side bandwidth is significantly lower than the input and output data tensor dimensions, with most communication side bandwidth values ​​concentrated around [number missing]. to Within the range, the cumulative probability curve rises sharply, indicating that the communication side bandwidth values ​​are relatively concentrated and the range is narrow.

[0030] Example 2

[0031] In specific implementation, please refer to Figure 3 The task graph and hardware core utilization are input into the first dynamic scheduling model, and the implementation method of extracting the correlation between software and hardware features through the attention mechanism is as follows.

[0032] The first dynamic scheduling model receives two types of input data: graph structure data of the task graph and numerical data of hardware core utilization. Before being input into the first dynamic scheduling model, the task graph is matrix-processed, extracting node feature matrices and adjacency matrices from the directed acyclic graph structure of the task graph. Each row of the node feature matrix corresponds to a node in the task graph, and each row is composed of the one-hot encoded vector of the corresponding node's operator type and multi-dimensional size features. The dimension of the one-hot encoded vector of the operator type is denoted as... The dimension of a multidimensional size feature is denoted as . Then the dimension of each row of the node feature matrix is The adjacency matrix is ​​a square matrix, and its order is equal to the total number of nodes in the task graph. The nodes located at the th position in the adjacency matrix are... Line number The elements of the column represent the number of elements from the first column. The node points to the first The scalar weight of the directed edge of the i-th node, if the i-th node... The node and the first If there are no directed edges between nodes, the corresponding element is set to 0.

[0033] Hardware core utilization data is organized in vector form, and the dimension of the hardware core utilization vector is denoted as . In the process of concatenating the joint feature tensor, the node feature matrix is ​​first flattened into a one-dimensional vector along the row direction, and the adjacency matrix is ​​flattened into a one-dimensional vector in the same way, while the hardware core utilization vector is kept in its original dimension. The flattened one-dimensional vectors of the node feature matrix, the flattened one-dimensional vectors of the adjacency matrix, and the hardware core utilization vector are then concatenated along the feature dimensions to generate a one-dimensional vector that integrates task graph attributes and hardware core states. This one-dimensional vector is then reshaped to form the joint feature tensor. The shape of the joint feature tensor can be set to a two-dimensional matrix according to the requirements of subsequent layers of the first dynamic scheduling model. The rows of the matrix represent the sequence length dimension of the joint feature tensor, and the columns of the matrix represent the feature dimensions of the joint feature tensor. The feature dimensions of the joint feature tensor are denoted as . .

[0034] The core structure of the first dynamic scheduling model is a self-attention layer, which contains a set of learnable linear projection matrices, namely the query projection matrix. Key projection matrix Sum projection matrix Query the projection matrix Key projection matrix Sum projection matrix The number of rows is The number of columns is , The dimension of the representation within the self-attention layer. Joint feature tensor. After inputting into the self-attention layer, the query vector matrix is ​​calculated using matrix multiplication. Key vector matrix Sum value vector matrix The calculation method is as follows , , Query vector matrix It contains multiple query vectors, with one query vector per row; key vector matrix It contains multiple key vectors, with one key vector per row; value vector matrix It contains multiple value vectors, with one value vector per row.

[0035] In the self-attention layer, the query vector matrix Each query vector and key vector matrix Similarity is calculated for all key vectors in the query vector matrix. The similarity score is obtained by scaling the dot product operation. With key vector matrix Multiply by the transpose, then divide by the scaling factor. This process generates an attention score matrix. Each row in the attention score matrix corresponds to the similarity score between a query vector and all key vectors. A softmax normalization operation is then performed on each row of the attention score matrix so that the sum of the scores in each row equals 1. The normalized matrix is ​​the attention weight matrix.

[0036] The self-attention layer employs a multi-head attention mechanism to implement the above process. The joint feature tensor... The feature dimension is divided into multiple subspaces, and the number of subspaces is denoted as . , The value of is an integer greater than or equal to 2. Specifically, the joint feature tensor Uniformly divide the dimensional features into There are subspaces, and the feature dimension of each subspace is . The number of combinations is also... The query projection matrix, key projection matrix, and value projection matrix of the group, the first The group query projection matrix is ​​denoted as The key projection matrix is ​​denoted as The projection matrix is ​​denoted as ,in From 1 to Integer identifier. The joint feature tensor part within each subspace and the first Perform a linear transformation on the group of projection matrices to obtain the first... The query vector matrix, key vector matrix, and value vector matrix of each subspace, and in the ... The attention weight matrix is ​​calculated independently within each subspace, following the same calculation steps as for a single attention head. After independent calculation in each subspace, the attention weight matrix in each subspace is multiplied by the corresponding value vector matrix to obtain... The output matrix of each subspace. The output matrices are concatenated along the feature dimension to form a matrix of dimension . The concatenated matrix is ​​then subjected to a linear transformation, the weight matrix of which is denoted as . Weight matrix The number of rows and columns are both The linear transformation operation involves combining the concatenated matrix with the weight matrix. Multiplying them together, the output is the correlation degree between software and hardware features.

[0037] The software and hardware feature correlation degree is the final output of the first dynamic scheduling model. The software and hardware feature correlation degree contains multiple correlation degree values, and each correlation degree value represents the correlation strength between a certain task computing node and a certain hardware core.

[0038] The symbols used in the above process will be explained. This represents the dimension of the one-hot encoded vector of the operator type. The dimension representing a multidimensional size feature. The dimension representing the hardware core utilization vector. The feature dimension of the joint feature tensor is represented. Indicates querying the projection matrix. Represents the key projection matrix. Represents the value projection matrix, The dimension representing the internal representation of the self-attention layer. This represents the number of subspaces in a multi-head attention mechanism. Indicates the first The query projection matrix of each subspace. Indicates the first The key projection matrix of each subspace Indicates the first The value projection matrix of each subspace, This represents the weight matrix after the concatenation is linearly transformed.

[0039] See Figure 6 In the figure, the horizontal axis represents the attention score, ranging from -2 to 2, and the vertical axis represents the probability density. The legend shows the distribution of attention scores in the four subspaces of the first dynamic scheduling model, with solid lines, dashed lines, dotted lines, and dotted lines representing the attention scores of subspaces 1 to 4, respectively.

[0040] Looking at the trends in the curves, the probability density curve of the attention score in the first subspace has the highest peak value, concentrated in the range close to 0, indicating a relatively concentrated distribution and a strong tendency for concentration and a small fluctuation range in the attention score of this subspace. The probability density curve of the attention score in the third subspace has the second highest peak value, and its distribution is also relatively concentrated, but slightly more dispersed than that of the first subspace. The probability density peak value of the fourth subspace is slightly lower than that of the third subspace, and its distribution is wider, indicating a larger range of variation in its attention score. The probability density curve of the attention score in the second subspace has the lowest peak value and the widest distribution, especially with a significant extension in the positive range (approximately 0 to 1), reflecting that the attention score in this subspace exhibits greater fluctuations and a more dispersed distribution characteristic.

[0041] Overall, the attention scores of the four subspaces are all centered around 0 and exhibit a nearly normal distribution with left-right symmetry, which aligns with the extraction method of software and hardware feature correlation by the multi-head self-attention mechanism in the first dynamic scheduling model. The differences in the distribution of attention scores among the different subspaces reflect the different focuses of each subspace on the joint features of task graph and hardware core utilization, further supporting the design goal of the multi-head attention mechanism to capture multi-dimensional software and hardware correlation information in parallel across multiple subspaces.

[0042] Example 3

[0043] In practice, the following method is used to calculate the allocation ratio of computing resources for each training task in the dynamic computing pool based on the correlation between software and hardware features and the Nash equilibrium solver.

[0044] All training tasks currently running in the heterogeneous computing cluster are defined as game participants, and the total number of game participants is denoted as . The total number of available hardware cores within the dynamic computing power pool is taken as the total resource constraint, denoted as... Initialize a resource allocation ratio matrix with dimension 1. The resource allocation ratio matrix consists of rows, each corresponding to a training task. Each element in each row represents the proportion of resources allocated to the corresponding training task, and the sum of all elements in the matrix is ​​1. The initial values ​​of the resource allocation ratio matrix are generated randomly. A non-negative value, then... Normalize the sum of the non-negative values ​​so that the normalized values ​​are... The sum of the values ​​is 1, and the normalized values ​​are used to fill the resource allocation ratio matrix.

[0045] The reward function for each training task is constructed based on the hardware-software feature correlation. The hardware-software feature correlation comprises multiple correlation values, each representing the strength of the correlation between a computation node and a hardware core for a given training task. For a single training task, all correlation values ​​belonging to that training task are aggregated by averaging them to obtain the task-level hardware-software correlation value. This task-level correlation value reflects the overall potential of the training task to achieve accelerated response on the dynamic computing pool hardware cores.

[0046] The partial derivative of the training task's throughput with respect to resource allocation is fitted using the task-level hardware / software correlation value, and used as the reward function. Specifically, the training task is defined. The profit function is the training task. Training throughput for training tasks The partial derivative of the proportion of resources allocated, where From 1 to Integer identifier. The training task... The task-level hardware / software correlation value is denoted as Training tasks The resource allocation ratio in the current iteration step is denoted as . Then the training task The profit function Calculated using the following formula:

[0047] in, Indicates training task The value of the payoff function, For training mission Task-level hardware / software correlation value The value range is determined by the aggregation result of the correlation degree between software and hardware features, and is a real number between 0 and 1. For training mission The resource allocation ratio in the current iteration step. The value of is a real number between 0 and 1. It is a natural constant. The return function value... Follow As resources increase, the marginal benefit per unit of resources decreases, indicating that the training task's marginal benefit decreases as more resources are acquired.

[0048] The reward function further incorporates a task priority factor. This factor is dynamically adjusted based on the historical convergence rate of the training task. For each training task, the most recent convergence rate of the training task is recorded. The change in validation loss within each sampling window The value is 5, and the sampling window duration is a fixed value. The training task... exist The absolute values ​​of the changes in validation loss within each sampling window are averaged to obtain the training task. The average loss rate of all training tasks is ranked. The training tasks ranked higher converge faster, and their corresponding task priority factors are higher. The task priority factor is denoted as... , The value range is from 0.5 to 1.5. The task priority factor is calculated by performing min-max normalization on the average loss descent rate to the interval [0,1], and then linearly mapping the normalization result to the interval [0.5,1.5] to obtain... After introducing a task priority factor, the reward function is adjusted to... In the Nash equilibrium solution, the task priority factor is used as a weighting coefficient of the payoff function.

[0049] In the iterative process of solving the Nash equilibrium, the resource allocation ratio matrix is ​​repeatedly updated until the difference between the partial derivatives of the reward functions of all training tasks with respect to resource allocation is less than a preset convergence threshold. In each iteration, the marginal reward of each training task is calculated; the marginal reward is the adjusted reward function value. Compare the marginal returns of all training tasks and identify the training task with the highest and lowest marginal returns. If the absolute value of the difference between the marginal returns of the training task with the highest and lowest marginal returns is less than a preset convergence threshold (0.001), the iteration terminates. If the difference is not less than the preset convergence threshold, an adjustment step size is removed from the resource allocation ratio of the training task with the lowest marginal return and added to the resource allocation ratio of the training task with the highest marginal return (set to 0.01). The smaller value in the range. After the update is complete, proceed to the next iteration.

[0050] When the iteration terminates, the resource allocation ratio of each training task in the resource allocation ratio matrix is ​​the computing power resource allocation ratio output by the Nash equilibrium solver.

[0051] See Figure 7 The figure shows the curves illustrating the change in the computing resource allocation ratio as a function of the number of iteration steps during the iterative process of the Nash equilibrium solver described in Example 3. The horizontal axis represents the number of iteration steps of the Nash equilibrium solver, ranging from 0 to 200, and the vertical axis represents the computing resource allocation ratio obtained by each training task, ranging from 0 to 0.7. The figure contains five curves, corresponding to the changing trends of the computing resource allocation ratio for training tasks 1 to 5, respectively.

[0052] In the diagram, training task 3 initially had the highest computing resource allocation ratio, approximately 0.44. Subsequently, as the number of iterations increased, its allocation ratio showed a significant downward trend, dropping to approximately 0.22 around step 50, before stabilizing and slowly decreasing to approximately 0.15 by step 200. The initial allocation ratios for training tasks 2 and 4 were both between 0.15 and 0.25, showing an initial increase followed by stabilization during iterations. The allocation ratio for training task 2 continuously increased from approximately 0.17 to approximately 0.24, while the allocation ratio for training task 4 fluctuated slightly, eventually stabilizing at approximately 0.20. The computing resource allocation ratio for training task 1 remained consistently low, initially around 0.08, steadily increasing to approximately 0.10 during iterations, with relatively small overall changes. The allocation ratio for training task 5 rapidly increased from approximately 0.04 initially, reaching approximately 0.29 by step 100, before stabilizing and becoming the training task with the highest allocation ratio at the end of the iterations.

[0053] Example 4

[0054] In practice, hardware cores are scheduled from the dynamic computing power pool and allocated to each training task according to the computing power resource allocation ratio. The process includes converting the computing power resource allocation ratio into an integer allocation scheme for the number of hardware cores, implementing affinity binding through kernel binding system calls, and performing priority-based sorting queue caching on the input and output requests of the bound hardware cores.

[0055] When converting the computing resource allocation ratio into an integer allocation scheme for the number of hardware cores, the total number of currently schedulable hardware cores in the dynamic computing pool is obtained, and the total number of hardware cores is recorded as follows: Obtain the computing power resource allocation ratio output by the Nash equilibrium solver, for the number... The training task, whose computing resource allocation ratio is denoted as... ,in From 1 to Integer identifier, This represents the total number of training tasks. For each training task, calculate the initial allocated number of cores. The initial allocated number of cores is the ratio of the computing power resource allocation to the total number of hardware cores for that training task. The product is rounded down, i.e., the training task. The initial allocation of cores is Sum the initial allocation of cores for all training tasks, using the total number of hardware cores. Subtracting the sum of the initially allocated cores yields the number of unallocated remaining hardware cores. These remaining hardware cores are then allocated one by one to training tasks, following this rule: for each remaining hardware core, calculate the marginal revenue increment for all training tasks based on the current allocated core count. The training task with the highest marginal revenue increment is selected as the allocation target for this hardware core, and its allocated core count is incremented by 1. This process is repeated until all remaining hardware cores are allocated. The marginal revenue increment is calculated using the revenue function of the training task in Example 3. The difference between the revenue function value of the training task with the current allocated core count plus 1 and the revenue function value with the current allocated core count is used as the marginal revenue increment. After the remaining hardware cores are allocated, each training task receives an integer allocated core count, and the integer allocated core counts of all training tasks together constitute the integer allocation scheme.

[0056] When using the kernel binding system call to perform affinity binding between integer-allocated hardware cores and the corresponding training task processes, the system selects an equal number of idle hardware cores from the dynamic computing power pool's hardware core list. The process identifier corresponding to the training task is obtained; this identifier is the process number of the training task in the operating system. The kernel binding system call provided by the operating system is then invoked, taking the process identifier and the hardware core mask as input parameters. The hardware core mask is a bitmap data structure, where each bit corresponds to a hardware core in the dynamic computing power pool. The bits corresponding to the hardware cores allocated to the training task are set to 1, and the bits corresponding to the remaining hardware cores are set to 0. After receiving the kernel binding system call, the operating system restricts the process thread scheduling domain corresponding to the process identifier to the set of hardware cores specified by the hardware core mask, ensuring that the training task process only performs operations on the allocated hardware cores, thus completing the affinity binding.

[0057] When performing priority-based sorting queue caching on input / output requests for bound hardware cores, an input / output request sorting queue is maintained within the hardware core group occupied by each training task process. Input / output requests include data read requests and data write requests generated by training tasks. The task priority factor for each training task calculated in Example 3 is obtained. The task priority factor is denoted as Each input / output request carries the task priority factor value of its respective training task. In the input / output request sorting queue, requests are ordered from highest to lowest priority factor value. Input / output requests from training tasks with higher priority factor values ​​are placed at the head of the queue for priority processing, while those from training tasks with lower priority factor values ​​are placed at the tail for delayed processing. When multiple input / output requests have the same priority factor value, they are arranged according to the order in which they arrived in the queue. Data read and write requests cached in the input / output request sorting queue are sent to the storage system sequentially according to the sorting result, ensuring that data access operations for high-priority training tasks receive priority responses, while data access operations for low-priority training tasks are cached and wait in the queue.

[0058] in, This indicates the total number of hardware cores that are currently available for scheduling in the dynamic computing pool. Integer identifier for the training task. The total number of training tasks. Indicates training task The proportion of computing power resources allocated, Indicates training task The initial allocation of cores, Indicates training task Task priority factor.

[0059] See Figure 8 The figure shows the cumulative probability distribution curves of input / output request latency generated by training tasks with different task priority factor values ​​according to the method described in Example 4. The horizontal axis represents the input / output request latency in milliseconds, ranging from 0 to approximately 320 milliseconds; the vertical axis represents the cumulative probability of latency, ranging from 0 to 1. In the figure, the solid curves represent the input / output requests generated by training tasks with high task priority factor values, and the dashed curves represent the input / output requests generated by training tasks with low task priority factor values.

[0060] As shown in the figure, the cumulative probability of input / output request latency for training tasks with high task priority factors increases rapidly, reaching over 99% at approximately 80 milliseconds, with a maximum latency approaching 100 milliseconds. This demonstrates that input / output requests for high-priority tasks can be responded to and processed quickly. In contrast, the cumulative probability of input / output request latency for training tasks with low task priority factors increases more slowly, reaching a cumulative probability of 90% at approximately 110 milliseconds. Furthermore, the latency distribution extends to approximately 320 milliseconds before approaching 1, indicating that input / output requests for low-priority tasks wait in the queue for a longer period, resulting in greater response latency.

[0061] Example 5

[0062] In specific implementation, please refer to Figure 4 The implementation method for dividing the hardware nodes in the heterogeneous computing power cluster into static and dynamic computing power pools is as follows.

[0063] For each hardware node in the heterogeneous computing cluster, resource utilization data is collected over a historical operating period of 7 days. The data collection granularity is to record the average utilization value of each hardware node every minute. The arithmetic mean of all recorded values ​​for each hardware node over the historical operating period is calculated as the historical average utilization rate of the hardware node. The historical average utilization rate of the hardware node is compared with a preset historical average utilization rate threshold, which is set at 70%. Hardware nodes with a historical average utilization rate higher than the 70% threshold are marked as static nodes, and hardware nodes with a historical average utilization rate lower than the 70% threshold are marked as dynamic candidate nodes.

[0064] Collect the topology information and maximum resource capacity of all static nodes to construct a static computing pool. Topology information includes the physical location of each static node within the heterogeneous computing cluster, the communication link bandwidth between the static node and other hardware nodes, and the communication domain number to which the static node belongs. Maximum resource capacity includes the total number of hardware cores, total memory capacity, and total network interface bandwidth of each static node. Organize the topology information and maximum resource capacity of each static node into a static node configuration record. Use this set of static node configuration records as the configuration data for the static computing pool, thus completing the construction of the static computing pool.

[0065] All dynamic candidate nodes are clustered based on memory frequency and network bandwidth to construct a dynamic computing power pool. During clustering, the memory frequency and network bandwidth values ​​of each dynamic candidate node are extracted. The memory frequency is the highest operating frequency of the memory modules installed in the dynamic candidate node, and the network bandwidth is the nominal throughput bandwidth of the network interface of the dynamic candidate node. A two-dimensional feature vector is formed from the memory frequency and network bandwidth values ​​of each dynamic candidate node. A clustering algorithm is used to cluster all two-dimensional feature vectors, dividing the dynamic candidate nodes into multiple clusters. Within the same cluster, the memory frequency and network bandwidth values ​​of dynamic candidate nodes are similar, while there are differences in memory frequency or network bandwidth values ​​between different clusters. After clustering, dynamic candidate nodes belonging to the same cluster are grouped into the same dynamic computing power pool partition. All dynamic computing power pool partitions corresponding to clusters together constitute the dynamic computing power pool. Hardware nodes within each partition of the dynamic computing power pool have similar memory frequency and network bandwidth characteristics.

[0066] During the operation of the heterogeneous computing power cluster, when a dynamically added hardware node joins the cluster, the capacity configuration of the static and dynamic computing power pools is updated. For dynamically added hardware nodes, the average utilization rate of the newly added hardware node is collected during the initial stable period after joining, with the initial stable period duration set to 10 minutes. If the average utilization rate of the dynamically added hardware node during the initial stable period is consistently higher than the historical average utilization rate threshold, the dynamically added hardware node is marked as a static node. The topology information and maximum resource capacity of the dynamically added hardware node are extracted and added to the configuration data of the static computing power pool, thus updating the capacity configuration of the static computing power pool. If the average utilization rate of dynamically added hardware nodes during the initial stable period is not higher than the historical average utilization rate threshold, then the dynamically added hardware nodes are marked as dynamic candidate nodes. The memory frequency and network bandwidth values ​​of the dynamically added hardware nodes are extracted, the two-dimensional feature vector of the dynamically added hardware nodes is added to the feature set of existing dynamic candidate nodes, the clustering operation is re-executed for all dynamic candidate nodes, the partitioning of the dynamic computing power pool is re-determined, and the capacity configuration of the dynamic computing power pool is updated.

[0067] The resource constraint dimension of the first dynamic scheduling model is adjusted based on the updated capacity configuration. After the capacity configuration is updated, the new total number of schedulable hardware cores in the dynamic computing pool is obtained, and this new total number of hardware cores in the dynamic computing pool is used as the updated total resource constraint value. The updated total resource constraint value is written into the resource constraint parameters of the Nash equilibrium solver in the first dynamic scheduling model. At the same time, the hardware core utilization data of the newly added hardware cores is expanded into the hardware core utilization vector input to the first dynamic scheduling model, and the dimension of the hardware core utilization vector is increased accordingly. The dimension of the joint feature tensor received by the self-attention layer in the first dynamic scheduling model is adjusted according to the change in the dimension of the hardware core utilization vector, so that the resource constraints used by the first dynamic scheduling model are consistent with the actual capacity of the dynamic computing pool when performing software and hardware feature correlation extraction and computing resource allocation ratio calculation.

[0068] Optionally, after receiving the updated resource constraint parameters, the first dynamic scheduling model performs dimensional adaptation on the resource allocation ratio matrix inside the Nash equilibrium solver, keeping the number of rows in the resource allocation ratio matrix unchanged to the number of training tasks, and maintaining the relative proportion of the resource allocation ratio value in each row of the resource allocation ratio matrix under the updated total resource constraint, so that the Nash equilibrium solver can continue to iterate and solve under the updated resource constraint dimension.

[0069] In some embodiments, the clustering operation of dynamic candidate nodes can also introduce the total number of hardware cores of the dynamic candidate nodes as an auxiliary clustering feature, and expand the two-dimensional feature vector into a three-dimensional feature vector containing memory frequency values, network bandwidth values ​​and the total number of hardware cores, and then perform clustering processing to adapt to the partitioning requirements of dynamic computing power pools of different sizes.

[0070] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for dynamic scheduling and optimization of computing resources for AI training scenarios, characterized in that, include: The hardware nodes in the heterogeneous computing power cluster are divided into static and dynamic computing power pools to obtain static computing power pools and dynamic computing power pools. Collect the task graphs for each training task and the hardware core utilization of the hardware nodes; The task graph and the hardware core utilization are input into the first dynamic scheduling model, and the correlation between software and hardware features is extracted through an attention mechanism. Based on the correlation between the hardware and software features, the Nash equilibrium solver is used to calculate the computing resource allocation ratio of each training task in the dynamic computing pool. According to the computing power resource allocation ratio, hardware cores are scheduled from the dynamic computing power pool and allocated to each training task for execution.

2. The method for dynamic scheduling and optimization of computing resources for AI training scenarios according to claim 1, characterized in that, The process of collecting the task graphs for each training task and the hardware core utilization of the hardware nodes includes: The execution graphs of each training task are analyzed, and the operator type, data tensor dimension, and communication edge bandwidth of each computing node are extracted to construct the task graph. The instantaneous utilization rate, cache hit rate, and memory bandwidth utilization rate of each hardware core are read from the cluster management interface in a polling manner to construct the hardware core utilization rate.

3. The method for dynamic scheduling and optimization of computing resources for AI training scenarios according to claim 2, characterized in that, The task graph is represented by a directed acyclic graph, in which the operator type of each computing node is mapped to a one-hot encoded vector, the data tensor dimension is mapped to a multidimensional size feature, and the communication edge bandwidth is mapped to a scalar weight.

4. The method for dynamic scheduling and optimization of computing resources for AI training scenarios according to claim 1, characterized in that, The step of inputting the task graph and the hardware core utilization into the first dynamic scheduling model, and extracting the correlation between software and hardware features through an attention mechanism, includes: The node feature matrix and adjacency matrix of the task graph are concatenated with the hardware core utilization to form a joint feature tensor; In the self-attention layer of the first dynamic scheduling model, the similarity score between each query vector and all key vectors in the joint feature tensor is calculated to generate an attention weight matrix. The attention weight matrix is ​​normalized to output the correlation degree of the software and hardware features.

5. The method for dynamic scheduling and optimization of computing resources for AI training scenarios according to claim 4, characterized in that, The self-attention layer employs a multi-head attention mechanism, which divides the joint feature tensor into multiple subspaces, calculates the attention weight matrix in each subspace, concatenates and linearly transforms them to obtain the correlation degree between software and hardware features.

6. The method for dynamic scheduling and optimization of computing resources for AI training scenarios according to claim 1, characterized in that, The step of calculating the allocation ratio of computing resources for each training task in the dynamic computing pool based on the correlation between the hardware and software features using a Nash equalization solver includes: Each training task is treated as a game participant, and the number of hardware cores in the dynamic computing power pool is used as a resource constraint to initialize a random resource allocation ratio matrix. The revenue function for each task is constructed based on the correlation between the hardware and software features, wherein the revenue function is the partial derivative of the task training throughput with respect to resource allocation. The resource allocation ratio matrix is ​​iteratively updated until the difference between the partial derivatives of the revenue functions of all tasks with respect to resource allocation is less than a preset convergence threshold, and then the computing power resource allocation ratio is output.

7. The method for dynamic scheduling and optimization of computing resources for AI training scenarios according to claim 6, characterized in that, The reward function further incorporates a task priority factor, which is dynamically adjusted based on the historical convergence speed of the training task and serves as a weighting coefficient for the reward function in the Nash equilibrium solution.

8. The method for dynamic scheduling and optimization of computing resources for AI training scenarios according to claim 1, characterized in that, The step of scheduling hardware cores from the dynamic computing power pool and allocating them to each training task for execution according to the computing power resource allocation ratio includes: Convert the computing resource allocation ratio into an integer allocation scheme for the number of hardware cores; The kernel binds system calls to establish affinity between the integer-allocated hardware core and the corresponding training task process. Input / output requests for the bound hardware core are processed using a priority-based sorted queue caching process.

9. The method for dynamic scheduling and optimization of computing resources for AI training scenarios according to claim 1, characterized in that, The method further includes: Update the capacity configuration of the static computing pool and the dynamic computing pool based on the dynamically added hardware nodes in the heterogeneous computing power cluster. The resource constraint dimension of the first dynamic scheduling model is adjusted according to the capacity configuration.

10. A method for dynamic scheduling and optimization of computing resources for AI training scenarios according to claim 1, characterized in that, The process of dividing hardware nodes in a heterogeneous computing power cluster into static and dynamic resource pools to obtain a static computing power pool and a dynamic computing power pool includes: Based on the historical average utilization threshold of each hardware node, nodes with utilization rates higher than the threshold are marked as static nodes; Collect the topology information and maximum resource capacity of the static nodes to construct the static computing pool; The remaining nodes are clustered according to memory frequency and network bandwidth to construct the dynamic computing power pool.

Images