AI computing power efficiency improvement method and system based on heterogeneous resource pooling and dynamic scheduling

Abstract

The application is specifically an AI computing power efficiency increasing method and system based on heterogeneous resource pooling and dynamic scheduling, relates to the technical field of distributed AI computing power scheduling, and comprises the following steps: collecting bandwidth, time delay and packet loss rate data of a wide area network link between each regional node in real time, calculating a transmission cost value between any two nodes, and generating a cost matrix.In the application, a standard virtual computing power unit index is introduced, floating point operation capability, video memory bandwidth and operator compatibility coefficient are uniformly included in calculation, different architecture heterogeneous chips obtain unified dimension logical resource representation, thereby providing a quantitative basis for cross-brand task allocation, and the problem that the existing scheme cannot perform equivalent comparison on heterogeneous chips is solved.

Description

Technical Field

[0001] This invention relates to the field of distributed AI computing power scheduling technology, and in particular to a method and system for improving AI computing power efficiency based on heterogeneous resource pooling and dynamic scheduling. Background Technology

[0002] As the scale of AI training tasks continues to expand, cross-regional distributed computing environments are being widely used. In this environment, multiple data centers deploy acceleration chips of different brands and architectures, and these data centers are interconnected via wide area network links. Existing scheduling schemes are typically designed for homogeneous chip environments, relying on the chip's original hardware parameters for resource statistics, and cannot establish a unified metric standard across chips with different architectures.

[0003] In existing scheduling schemes, task allocation decisions are mainly based on local computing power reserves, and wide area network (WAN) transmission latency is not included in the scheduling model. When the training dataset is large and the data and computing power are located on different geographically distributed nodes, the latency caused by data transfer across the WAN significantly affects end-to-end task turnaround time, and existing schemes lack effective means to address this issue.

[0004] Furthermore, existing solutions generally employ static, exclusive resource allocation. During off-peak periods of task execution, the allocated computing resources remain idle and cannot be reused by other tasks, resulting in low overall computing power utilization. These issues collectively constrain the execution efficiency and resource utilization of AI tasks in geographically heterogeneous computing environments. Summary of the Invention

[0005] The purpose of this invention is to propose an AI computing power efficiency enhancement method and system based on heterogeneous resource pooling and dynamic scheduling in order to solve the above problems.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: AI computing power efficiency enhancement methods based on heterogeneous resource pooling and dynamic scheduling include: Obtain the physical parameters of heterogeneous chips in each regional node, calculate the standard virtual computing unit index corresponding to each chip based on the standardized physical parameters and operator compatibility coefficient, and form a resource view of the logical computing pool. Real-time data collection of bandwidth, latency, and packet loss rate of WAN links between nodes in different regions; calculation of transmission cost between any two nodes; generation of cost matrix. Receive AI task requests, obtain the source node location and dataset size of the training dataset, combine the cost matrix with the total idle standard virtual computing power unit index of each node, calculate the comprehensive execution cost when allocating AI tasks to each candidate node, and select the candidate node with the minimum comprehensive execution cost as the target execution node. During the execution of AI tasks, identify AI processes that are experiencing low computing power, reclaim a portion of their standard virtual computing unit index quota, and reallocate it to the queue of real-time inference tasks waiting to be executed within the same node.

[0007] Preferably, the calculation of the standard virtual computing unit index corresponding to each chip based on the standardized physical parameters and operator compatibility coefficient includes: The floating-point computing power and memory bandwidth of the chip are respectively processed by Z-score normalization; the normalized floating-point computing power and normalized memory bandwidth are multiplied by the corresponding preset weight coefficients and summed, and then multiplied by the operator compatibility coefficient to obtain the standard virtual computing unit index; wherein, the sum of the two weight coefficients is 1 and both are greater than 0.

[0008] Preferably, the operator compatibility coefficient is the performance equivalence value of a specific operator relative to a benchmark architecture when executed on the target chip architecture; the operator compatibility coefficient is obtained by performing standard operator benchmark tests on each architecture chip; when calculating the standard virtual computing unit index, the corresponding value is obtained by querying the pre-stored compatibility coefficient table according to the main operator types contained in the AI ​​task to be executed.

[0009] Preferably, calculating the transmission cost between any two nodes includes: The basic cost is obtained by dividing the link delay between the source node and the target node by the available bandwidth of the corresponding link; it is then determined whether the measured packet loss rate of the corresponding link exceeds a preset packet loss rate threshold. If it does, a preset penalty coefficient is applied to the basic cost; wherein the link delay and available bandwidth are respectively normalized by mean based on the range before being substituted into the calculation.

[0010] Preferably, the calculation of the overall execution cost when allocating AI tasks to each candidate node includes: The estimated transmission latency for transmitting training data from the source node to the candidate node is calculated, whereby the estimated transmission latency is obtained by dividing the dataset size by the available bandwidth of the link between the source node and the candidate node; the estimated computation latency for the candidate node to complete the AI ​​task is calculated, whereby the estimated computation latency is obtained by dividing the total computation of the AI ​​task by the total number of currently idle standard virtual computing power units indexed by the candidate node; the estimated transmission latency and the estimated computation latency are multiplied by their respective weight coefficients and then summed to obtain the comprehensive execution cost; wherein the sum of the two weight coefficients is 1 and both are greater than 0.

[0011] Preferably, when the estimated transmission delay exceeds a preset multiple threshold of the estimated computation delay, a computation-to-data migration strategy is triggered; the computation-to-data migration strategy migrates the computation logic of the AI ​​task to the data source node or the node with idle computing power closest to the data source node for execution.

[0012] Preferably, the period of low computing power includes a model parameter saving phase and a data loading waiting phase; the recovery of the occupied portion of the standard virtual computing unit index quota is performed in a non-intrusive manner, including: Before reclaiming quotas, check whether the target AI process is currently in a critical operation of gradient calculation or parameter update; quota reclamation is only performed when the target AI process is in a non-critical operation phase; during the reclamation process, the video memory page table of the target AI process is marked, and when the AI ​​process resumes calculation, its video memory access path is restored through page table remapping.

[0013] Preferably, the method further includes: when the original AI process exits the low computing power period, reverting the corresponding standard virtual computing power unit index quota from the real-time inference task queue and returning it to the AI ​​process.

[0014] Preferably, the method further includes: maintaining historical load time-series data for each AI process; predicting the time window for each AI process to enter the next computing power trough based on the historical load time-series data; and scheduling the real-time inference tasks to be assigned to the corresponding node's task waiting queue in advance before the predicted computing power trough arrives.

[0015] An AI computing power efficiency enhancement system based on heterogeneous resource pooling and dynamic scheduling includes: The computing power module is used to obtain the physical parameters of heterogeneous chips in nodes in various regions, calculate the standard virtual computing power unit index corresponding to each chip based on the standardized physical parameters and operator compatibility coefficient, and form a resource view of the logical computing power pool. The link monitoring module is used to collect bandwidth, latency and packet loss rate data of WAN links between nodes in different regions in real time, calculate the transmission cost between any two nodes, and generate a cost matrix. The task scheduling module is used to receive AI task requests, obtain the location of the source node where the training dataset is located and the size of the dataset, calculate the comprehensive execution cost by combining the cost matrix with the total idle standard virtual computing power unit index of each node, and select the candidate node with the minimum comprehensive execution cost as the target execution node. The dynamic resource management module is used to identify AI processes that are experiencing low computing power during AI task execution, reclaim a portion of their standard virtual computing unit index quota, and reallocate it to the real-time inference task queue waiting to be executed within the same node.

[0016] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. This invention introduces a standard virtual computing unit index to uniformly incorporate floating-point computing power, video memory bandwidth, and operator compatibility coefficients into the calculation, enabling heterogeneous chips with different architectures to obtain a unified dimension of logical resource representation, thereby providing a quantitative basis for cross-brand task allocation and solving the problem that existing solutions cannot perform equivalent comparisons of heterogeneous chips.

[0017] 2. This invention constructs a cost matrix and incorporates transmission cost into the comprehensive execution cost calculation. The scheduling decision can simultaneously consider computing power reserves and WAN transmission latency, making the task allocation scheme more closely reflect actual network conditions. In scenarios with extremely large datasets, the computation-to-data migration strategy moves computational logic closer to the data source for execution, avoiding the transmission bottleneck caused by large-scale data movement across WANs and helping to reduce end-to-end task turnaround time. Attached Figure Description

[0018] Further details, features, and advantages of this application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which: Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0019] Several embodiments of this application will now be described in more detail with reference to the accompanying drawings to enable those skilled in the art to implement this application. This application may be embodied in many different forms and for various purposes and should not be limited to the embodiments set forth herein. These embodiments are provided to make this application thorough and complete, and to fully convey the scope of this application to those skilled in the art. The embodiments described do not limit this application.

[0020] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It will be further understood that terms such as those defined in commonly used dictionaries shall be interpreted as having a meaning consistent with their meaning in the relevant field and / or the context of this specification, and shall not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0021] Example 1 Its specific implementation method is combined with the appendix Figure 1 Please provide a detailed explanation.

[0022] In this embodiment, it includes: Step 1: Obtain the physical parameters of each heterogeneous chip and calculate the standard virtual computing unit index. Obtain the floating-point computing power, memory bandwidth, and bus speed of heterogeneous chips deployed in each regional node. Before calculating the standard virtual computing unit index, Z-score normalization is applied to the floating-point computing power and memory bandwidth to eliminate the impact of their dimensional differences on the weighted calculation. Based on the normalized physical parameters, the standard virtual computing unit index corresponding to each chip is calculated according to the following formula: ; in, For standard virtual computing unit index, The standardized floating-point computing power of the chip. This refers to the standardized memory bandwidth of the chip. and The preset weighting coefficients, and , For operator compatibility coefficients, The main operator types included in the AI ​​task to be performed.

[0023] After the above calculations, each chip obtains a uniform dimension. The value is used to form the resource view of the logical computing power pool.

[0024] It should be noted that the above operator compatibility coefficients This refers to the performance equivalence ratio of a specific operator on a target chip architecture relative to a baseline architecture. Operator compatibility coefficient. The operator compatibility coefficient is obtained by performing standard operator benchmark tests on chips of various architectures in advance. This coefficient is calculated when the target chip has native acceleration support for a certain type of operator. Greater than 1; when the target chip needs to execute the operator through software simulation, the operator compatibility coefficient is greater than 1. Less than 1. The system maintains a compatibility coefficient table between operators and chip architectures, which is used to calculate the standard virtual computing unit index. At that time, the corresponding operator compatibility coefficient is obtained by looking up a table based on the main operator types contained in the AI ​​task to be executed. value.

[0025] Step 2: Monitor cross-regional link status and generate cost matrix Real-time data collection of bandwidth, latency, and packet loss rate of WAN links between nodes in different regions. Based on the collected link status data, the data transmission cost between any two nodes is calculated. The transmission costs between all node pairs are organized into a cost matrix, with rows and columns corresponding to nodes in different regions, and elements of the cost matrix representing the transmission cost between the corresponding node pairs.

[0026] It should be noted that the transmission cost mentioned above comprehensively reflects the link's latency and available bandwidth. Specifically, the transmission cost is positively correlated with link latency and negatively correlated with available link bandwidth. For nodes... With nodes The transmission cost of the link between them Calculate using the following formula: ; in, For nodes With nodes The transmission value between them Number the source node. Number the target node. For nodes With nodes Link latency between them, in milliseconds; The available bandwidth for the corresponding link, in megabits per second; The preset penalty coefficient, ; This represents the measured packet loss rate for the corresponding link. This is the preset packet loss rate threshold; This is an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise. and Before substituting into the formula, mean normalization based on the range is performed to eliminate the impact of inconsistent dimensions of latency and bandwidth in comparison calculations. When the link packet loss rate... Exceeding the threshold At that time, a penalty coefficient is applied to the transmission cost. This is to reduce the scheduling priority of the link.

[0027] Step 3: Calculate the overall execution cost based on the data gravity model and generate a task allocation scheme. Receive AI task requests to be executed, and obtain the location of the source node containing the training dataset corresponding to the AI ​​task, as well as the dataset size. Combine the cost matrix with the idle time of each node. Total, calculated as the overall execution cost when allocating AI tasks to each candidate node: ; in, To account for overall execution costs, The estimated transmission latency for transferring training data from the source node to the candidate node, in seconds; Candidate nodes are selected based on their idle time. The estimated computational latency for completing the total number of AI tasks, in seconds; and The preset weighting coefficients, and . and The units of all values ​​are time, so they can be directly weighted and summed.

[0028] It is obtained by dividing the dataset size by the available bandwidth of the link between the source node and the candidate node. The available bandwidth of the link is extracted from the corresponding element of the cost matrix. Divide the total computation of the AI ​​task by the current idle time of the candidate nodes. The total amount was obtained.

[0029] Selecting the overall execution cost The smallest candidate node is selected as the target execution node to generate a task allocation scheme.

[0030] In this embodiment of the application, to avoid bandwidth bottlenecks caused by transporting large-scale datasets across wide area networks, when the estimated transmission delay... Exceeding the estimated delay When the preset multiple threshold is reached, a computation-to-data migration strategy is triggered. This strategy migrates the computational logic of the AI ​​task to the data source node or the nearest node with available computing power, rather than moving data to a remote computing cluster. This strategy enables scheduling decisions to prioritize the use of computing resources near the data source in scenarios with extremely large datasets, thus avoiding wide area network transmission bottlenecks.

[0031] Step 4: Monitor the runtime memory utilization and perform dynamic resource shrinking and expansion. During AI task execution, the system collects real-time data on the memory usage and computational load of each AI process on the target execution node. It identifies AI processes currently experiencing low computational power, including the model parameter saving phase and the data loading waiting phase.

[0032] For AI processes experiencing low computing power, reclaim a portion of their occupied standard virtual computing unit indexes. Quota. Index of the standard virtual computing unit to be recycled. The quota is reallocated to the queue of real-time inference tasks waiting to be executed within the same node. When the original AI process exits the period of low computing power, the corresponding standard virtual computing unit index is rolled back from the real-time inference task queue. The quota is then returned to the AI ​​process.

[0033] It should be noted that the above recycling operation is performed in a non-intrusive manner. Specifically, the system recycles standard virtual computing unit indexes. Before allocating quotas, the system checks whether the target AI process is currently in a critical operation involving gradient calculation or parameter update. Quota reclamation is only performed if the target AI process is in a non-critical operation phase. During reclamation, the system marks the target AI process's video memory page table instead of immediately releasing the physical video memory. When the AI ​​process resumes computation, its video memory access path is restored through page table remapping, thereby avoiding process interruption or data loss.

[0034] In this embodiment, to achieve adaptive resource allocation during off-peak periods, the system maintains historical load time-series data for each AI process. Based on this historical load time-series data, the system predicts the time window for each AI process to enter the next computing power trough. Before the predicted computing power trough arrives, the real-time inference tasks to be allocated are pre-scheduled to the task waiting queue of the corresponding node, shortening the queuing time for real-time inference tasks.

[0035] Example 2 During its operation in 20XX, an AI service platform distributed its computing infrastructure across three geographically located nodes (Node A, Node B, and Node C). Node A deployed chips A and B specifically for accelerating matrix operations; Node B deployed a general-purpose graphics computing chip C; and Node C deployed a chip D optimized for sparse computing. The nodes were interconnected via a wide area network, resulting in significant differences in link quality between nodes. Currently, the platform needs to schedule a batch of large language model fine-tuning tasks (primarily using Transformer attention operators). The training dataset, measuring 480GB, is stored on Node A.

[0036] Step 1 implementation: Obtain physical parameters and calculate the standard virtual computing unit index. The local monitoring agent collects the raw values ​​of the floating-point arithmetic capability and memory bandwidth of each chip on nodes A, B, and C. Z-score normalization is applied to the floating-point arithmetic capability and memory bandwidth of the four chips to eliminate the difference in dimensions between the two metrics. For the Transformer attention operator, a pre-established operator compatibility table is consulted: Chips A and B have native acceleration support for this operator, with a compatibility coefficient greater than 1; Chip C has a general architecture, with a compatibility coefficient equal to 1; Chip D is optimized for sparse operators, but requires partial software simulation for dense attention operators, resulting in a compatibility coefficient less than 1. , Substitute into the formula Calculate the standard virtual computing unit index for each chip, where This is the standardized floating-point computing power value. This is the standardized memory bandwidth value. This is the operator compatibility coefficient.

[0037] Table 1 Results of physical parameter acquisition and standard virtual computing unit index calculation for each chip. ; The total number of idle VCU indices for node A is 3.08 (both chip A and chip B are idle). Node B has insufficient actual available computing power because the index of chip C is negative. The total number of idle VCU indices for node C is 0.77. This forms a resource view of the logic computing pool, where each heterogeneous chip is mapped to a logic resource of a uniform dimension.

[0038] Implementation of step 2: Monitor link status and generate cost matrix The local monitoring agent collects real-time WAN link status data between nodes A, B, and C, including round-trip latency, available bandwidth, and packet loss rate. A preset packet loss rate threshold is set. Penalty coefficient After performing mean normalization based on the range for the latency and bandwidth of each link, the results are substituted into the formula. Calculate the transmission cost, where This is the normalized link latency. The normalized available bandwidth, For link packet loss rate, This is an indicator function.

[0039] Table 2 Results of Cross-Node Link Status Collection and Cost Calculation ; After normalization, if the link from node A to node C experiences a packet loss rate exceeding a threshold, the transmission cost is multiplied by a coefficient. The cost is significantly higher than other links. The final cost matrix is ​​as follows: Table 3. Inter-node transmission cost matrix ; In the cost matrix, the path A→C has the highest cost due to packet loss penalty, which will directly affect subsequent task allocation decisions.

[0040] Step 3 implementation: Calculate the overall execution cost and generate a task allocation scheme. The training dataset is 480GB in size and stored on node A. The total computational cost of the task is estimated to be... This is a floating-point operation. The weighting coefficient is then determined. , Substitute into the formula Calculate the overall execution cost of each candidate node, where For data transmission latency, To calculate latency, nodes A, B, and C were evaluated as candidate execution nodes.

[0041] Since the dataset itself is located on node A, when assigning tasks to node A, (No cross-node transmission required). When allocated to node B, Seconds; when allocated to node C, The time difference is significant, and the penalty for packet loss is further amplified. The result is obtained by dividing the total computational load by the equivalent computing power of the total number of idle VCU indexes on each node. Node A has the strongest computing power and the shortest computational latency.

[0042] Table 4 Comparison of Overall Execution Costs of Each Candidate Node ; Node B has a negative VCU index, indicating insufficient equivalent computational power for the Transformer attention operator, and is therefore excluded from the candidate list. Node A has the lowest overall execution cost, so a task allocation scheme is generated: the large language model fine-tuning task is assigned to node A for execution. Furthermore, the system simultaneously verifies the triggering conditions for the computation-to-data migration strategy: node A itself is a data source node. The triggering condition does not apply; if in the future a scenario arises where data is located at node C but computing power is concentrated at node A, and Exceed If the preset multiple threshold is reached, the calculation logic will be moved to the vicinity of node C for execution.

[0043] Step 4 implementation: Monitor runtime memory utilization and perform dynamic shrinking and expansion. After the task is started on chips A and B on node A, the local monitoring agent continuously collects the memory usage and computational load of each AI process. During a certain time window of the training process, process P-A enters the model parameter saving phase (a period of low computational power), and its computational load drops sharply from its peak. The system checks that process P-A is not currently in a critical operation of gradient calculation or parameter update, confirming that quota reclamation can be safely performed. The system marks the memory page table of process P-A, reclaims 0.72 VCU index quotas, and temporarily allocates this quota to the real-time inference task R-01 queued in the task waiting queue on node A. Simultaneously, based on the historical load time-series data of process P-A, the system predicts that it will exit the parameter saving phase and resume gradient calculation in approximately 35 seconds, and pre-schedules the next batch of real-time inference tasks R-02 to the waiting queue on node A to shorten the queuing time.

[0044] Table 5. Records of Dynamic Resource Contraction and Allocation During Low Computing Power Periods ; Process P-A resumes computation within the prediction time window. The system restores its video memory access path through page table remapping, rolls back 0.72 VCU index quotas from real-time inference task R-01 and returns them to process P-A. No process interruption or data loss is triggered throughout the process.

[0045] Throughout the implementation process, data is collected from chip parameters at the physical layer and standardized using Z-scores to form a VCU index resource view with unified dimensions. Link status data is normalized and processed using cost formulas to form a cost matrix. Both types of data are incorporated into the comprehensive execution cost calculation, outputting a task allocation scheme. After task execution, runtime monitoring data drives dynamic shrinkage and expansion, and the reclaimed quota is reused by real-time inference tasks during off-peak periods. The output data of each step serves as the input for the next step, and the data flow maintains a logically complete closed loop.

[0046] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

[0047] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0048] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An AI computing power efficiency enhancement method based on heterogeneous resource pooling and dynamic scheduling, characterized in that, include: Obtain the physical parameters of heterogeneous chips in each regional node, calculate the standard virtual computing unit index corresponding to each chip based on the standardized physical parameters and operator compatibility coefficient, and form a resource view of the logical computing pool. Real-time data collection of bandwidth, latency, and packet loss rate of WAN links between nodes in different regions; calculation of transmission cost between any two nodes; generation of cost matrix. Receive AI task requests, obtain the source node location and dataset size of the training dataset, combine the cost matrix with the total idle standard virtual computing power unit index of each node, calculate the comprehensive execution cost when allocating AI tasks to each candidate node, and select the candidate node with the minimum comprehensive execution cost as the target execution node. During the execution of AI tasks, identify AI processes that are experiencing low computing power, reclaim a portion of their standard virtual computing unit index quota, and reallocate it to the queue of real-time inference tasks waiting to be executed within the same node.

2. The AI ​​computing power efficiency enhancement method based on heterogeneous resource pooling and dynamic scheduling according to claim 1, characterized in that, The standard virtual computing unit index for each chip is calculated based on the standardized physical parameters and operator compatibility coefficients, including: The floating-point computing power and memory bandwidth of the chip are respectively processed by Z-score normalization; the normalized floating-point computing power and normalized memory bandwidth are multiplied by the corresponding preset weight coefficients and summed, and then multiplied by the operator compatibility coefficient to obtain the standard virtual computing unit index; wherein, the sum of the two weight coefficients is 1 and both are greater than 0.

3. The AI ​​computing power efficiency enhancement method based on heterogeneous resource pooling and dynamic scheduling according to claim 2, characterized in that, The operator compatibility coefficient is the performance equivalence value of a specific operator when executed on a target chip architecture relative to a benchmark architecture. The operator compatibility coefficient is obtained by performing standard operator benchmark tests on chips of various architectures. When calculating the standard virtual computing unit index, the corresponding value is obtained by querying a pre-stored compatibility coefficient table according to the main operator types contained in the AI ​​task to be executed.

4. The AI ​​computing power efficiency enhancement method based on heterogeneous resource pooling and dynamic scheduling according to claim 1, characterized in that, Calculate the transmission cost between any two nodes, including: The basic cost is obtained by dividing the link delay between the source node and the target node by the available bandwidth of the corresponding link; it is then determined whether the measured packet loss rate of the corresponding link exceeds a preset packet loss rate threshold. If it does, a preset penalty coefficient is applied to the basic cost; wherein the link delay and available bandwidth are respectively normalized by mean based on the range before being substituted into the calculation.

5. The AI ​​computing power efficiency enhancement method based on heterogeneous resource pooling and dynamic scheduling according to claim 1, characterized in that, Calculate the overall execution cost of allocating AI tasks to candidate nodes, including: The estimated transmission latency for transmitting training data from the source node to the candidate node is calculated, whereby the estimated transmission latency is obtained by dividing the dataset size by the available bandwidth of the link between the source node and the candidate node; the estimated computation latency for the candidate node to complete the AI ​​task is calculated, whereby the estimated computation latency is obtained by dividing the total computation of the AI ​​task by the total number of currently idle standard virtual computing power units indexed by the candidate node; the estimated transmission latency and the estimated computation latency are multiplied by their respective weight coefficients and then summed to obtain the comprehensive execution cost; wherein the sum of the two weight coefficients is 1 and both are greater than 0.

6. The AI ​​computing power efficiency enhancement method based on heterogeneous resource pooling and dynamic scheduling according to claim 5, characterized in that, When the estimated transmission delay exceeds a preset multiple threshold of the estimated calculation delay, a calculation to data migration strategy is triggered; The computation-to-data migration strategy migrates the computational logic of AI tasks to the data source node or the node with idle computing power closest to the data source node for execution.

7. The AI ​​computing power efficiency enhancement method based on heterogeneous resource pooling and dynamic scheduling according to claim 1, characterized in that, The period of low computing power includes a model parameter saving phase and a data loading waiting phase; the recovery of the occupied standard virtual computing unit index quota is performed in a non-intrusive manner, including: Before reclaiming quotas, check whether the target AI process is currently in a critical operation of gradient calculation or parameter update; quota reclamation is only performed when the target AI process is in a non-critical operation phase; during the reclamation process, the video memory page table of the target AI process is marked, and when the AI ​​process resumes calculation, its video memory access path is restored through page table remapping.

8. The AI ​​computing power efficiency enhancement method based on heterogeneous resource pooling and dynamic scheduling according to claim 1, characterized in that, Also includes: When the original AI process exits the period of low computing power, the corresponding standard virtual computing unit index quota is rolled back from the real-time inference task queue and returned to the AI ​​process.

9. The AI ​​computing power efficiency enhancement method based on heterogeneous resource pooling and dynamic scheduling according to claim 1, characterized in that, Also includes: Maintain historical load time-series data for each AI process; Based on the historical load time series data, the time window for each AI process to enter the next computing power trough is predicted; before the predicted computing power trough arrives, the real-time inference tasks to be assigned are scheduled to the task waiting queue of the corresponding node in advance.

10. An AI computing power efficiency enhancement system based on heterogeneous resource pooling and dynamic scheduling, wherein the AI ​​computing power efficiency enhancement method based on heterogeneous resource pooling and dynamic scheduling according to any one of claims 1-9 is characterized in that, include: The computing power module is used to obtain the physical parameters of heterogeneous chips in nodes in various regions, calculate the standard virtual computing power unit index corresponding to each chip based on the standardized physical parameters and operator compatibility coefficient, and form a resource view of the logical computing power pool. The link monitoring module is used to collect bandwidth, latency and packet loss rate data of WAN links between nodes in different regions in real time, calculate the transmission cost between any two nodes, and generate a cost matrix. The task scheduling module is used to receive AI task requests, obtain the location of the source node where the training dataset is located and the size of the dataset, calculate the comprehensive execution cost by combining the cost matrix with the total idle standard virtual computing power unit index of each node, and select the candidate node with the minimum comprehensive execution cost as the target execution node. The dynamic resource management module is used to identify AI processes that are experiencing low computing power during AI task execution, reclaim a portion of their standard virtual computing unit index quota, and reallocate it to the real-time inference task queue waiting to be executed within the same node.

Images