A GPU computing power scheduling method based on a cloud multi-core heterogeneous computing power platform

Abstract

The application discloses a GPU computing power scheduling method based on a cloud multi-core heterogeneous computing power platform, relates to the technical field of heterogeneous computing power resource allocation, and comprises the following steps: carrying out bearing granularity arrangement and execution mode division according to a GPU task fingerprint adaptation matrix, generating a to-be-scheduled GPU task bearing mode, and performing sorting and screening according to a bearing adaptation degree to generate a GPU candidate bearing set; simulating and evaluating a GPU residual resource form occupied by the to-be-scheduled GPU task bearing mode in the GPU candidate bearing set to generate GPU fragment counterfactual evaluation information; selecting a target bearing mode according to the GPU fragment counterfactual evaluation information, binding GPU resources of the to-be-scheduled GPU task according to the target bearing mode, collecting running feedback of the to-be-scheduled GPU task, and generating GPU scheduling closed-loop update data. The application improves the continuity, stability and resource retention capability of GPU computing power scheduling.

Description

Technical Field

[0001] This invention relates to the field of heterogeneous computing power resource allocation technology, and in particular to a GPU computing power scheduling method based on a cloud-multi-core heterogeneous computing power platform. Background Technology

[0002] With the development of cloud computing and artificial intelligence training and inference services, GPU computing resources are gradually shifting from single-server deployment to centralized cloud management, with multiple models and architectures of GPU chips forming a heterogeneous computing resource pool. Related technologies typically use container orchestration, task queues, resource monitoring, and GPU instantiation allocation to uniformly schedule training tasks, inference tasks, and parallel computing tasks, thereby improving the centralized utilization of GPU resources.

[0003] In a multi-chip heterogeneous computing environment, different GPU chips differ in operator execution efficiency, memory specifications, supported precision, and interconnection methods. The operator structure and operational constraints of the GPU tasks to be scheduled also vary significantly. If scheduling is based solely on GPU load, memory usage, or static hardware specifications, it can easily lead to inaccurate matching between the GPU chip and the task operator requirements. This can result in fragmented GPU resources after the task is completed, affecting the continuous processing capacity of subsequent GPU tasks. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a GPU computing power scheduling method based on a cloud-multi-core heterogeneous computing power platform to solve the problem of difficulty in accurately matching task operator requirements with GPU chip carrying capacity in cloud-multi-core heterogeneous GPU resources.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: This invention provides a GPU computing power scheduling method based on a multi-chip heterogeneous computing power platform, comprising: collecting GPU operation data; filtering GPU chips that meet testing conditions based on idle stability and test security; issuing benchmark operator test tasks in the current idle window and recording benchmark operator test execution data; generating a GPU cross-chip operator fingerprint set; obtaining GPU task requests to be scheduled; parsing the task computation graph, operation framework, accuracy requirements, parallelism mode, and latency constraints; generating GPU task operator requirements to be scheduled; and performing item-by-item matching and matrix organization of the GPU task operator requirements with the operator performance characteristics in the GPU cross-chip operator fingerprint set. A GPU task fingerprint adaptation matrix is ​​generated; based on the GPU task fingerprint adaptation matrix, the granularity of the load and the execution mode are sorted and divided to generate the load modes of the GPU tasks to be scheduled, and sorted and filtered according to the degree of load adaptation to generate a set of GPU candidate load modes; the remaining GPU resource status after the load modes of the GPU tasks to be scheduled in the set of GPU candidate load modes are simulated and evaluated to generate GPU fragmentation counterfactual evaluation information; based on the GPU fragmentation counterfactual evaluation information, a target load mode is selected, the GPU tasks to be scheduled are bound to GPU resources according to the target load mode, and the running feedback of the GPU tasks to be scheduled is collected to generate GPU scheduling closed-loop update data.

[0007] As a preferred embodiment of the GPU computing power scheduling method based on a multi-core heterogeneous computing power platform in this invention, the GPU operation data collection specifically includes collecting GPU load status, GPU memory usage status, GPU temperature status, GPU power consumption status, GPU task usage status, GPU testability status, GPU memory specifications, GPU supported precision, GPU interconnection method, continuous range of available GPU memory, GPU slice support status, GPU slice specifications, GPU slice usage status, GPU whole card usage status, and GPU connection path usage status according to GPU chip identifier and GPU instance identifier, and organizing them to generate GPU operation data.

[0008] As a preferred embodiment of the GPU computing power scheduling method based on a cloud-multi-core heterogeneous computing power platform described in this invention, the GPU chip that meets the test conditions is a GPU chip selected based on GPU running data that is in a low-load state, has stable memory usage, normal temperature, normal power consumption, and is not exclusively occupied by a task. The benchmark operator test task is a lightweight operator test task based on the chip type, supported precision, memory specifications, interconnection method, and current idle window configuration of the GPU chip that meets the test conditions.

[0009] As a preferred embodiment of the GPU computing power scheduling method based on a cloud-multi-core heterogeneous computing power platform described in this invention, the generation of GPU cross-core operator fingerprint set specifically includes extracting load fluctuation status, memory usage stability status, temperature status, power consumption status and task usage status from GPU running data, and screening each GPU chip according to idle stability and test security to generate GPU chip test condition screening data. Extract GPU chips that meet the test conditions from the GPU chip test condition screening data, and configure benchmark operator test tasks based on the chip type, supported precision, memory specifications, interconnection method and current idle window of the GPU chips that meet the test conditions, and generate benchmark operator test task distribution information. Based on the information issued by the benchmark operator test task, the GPU chip that meets the test conditions is controlled to execute the benchmark operator test task, and the operator execution time, changes in video memory usage and communication waiting status are recorded to generate benchmark operator test execution data; Based on the benchmark operator test execution data, the differences in operator performance between different GPU chips are organized according to the GPU chip identifier and benchmark operator type. The differences in operator performance are organized into operator performance features, and the operator performance features are written into the fingerprint record of the corresponding GPU chip to generate a cross-chip GPU operator fingerprint set.

[0010] As a preferred embodiment of the GPU computing power scheduling method based on a multi-core heterogeneous computing power platform of the present invention, the GPU task request to be scheduled is obtained by receiving task submission information sent by the GPU task submission terminal and parsing the task fields of the task submission information.

[0011] As a preferred embodiment of the GPU computing power scheduling method based on a multi-core heterogeneous computing power platform of the present invention, the step of generating the GPU task fingerprint adaptation matrix specifically includes: performing field verification on the GPU task request to be scheduled, and extracting the task computation graph, task running framework, task accuracy requirements, task parallel mode and task latency constraints from the GPU task request to be scheduled, and integrating them to generate basic parsing data of the GPU task to be scheduled. Based on the basic parsing data of the GPU task to be scheduled, the operator type of the computing nodes in the task computation graph is identified, and the nodes are organized according to the operator naming rules and execution order of the task running framework to generate the operator structure data of the GPU task to be scheduled. Based on the operator structure data of the GPU task to be scheduled, the number of calls, call positions, dependencies, precision requirements and parallel execution relationships of different operator types are statistically analyzed to generate the operator requirements of the GPU task to be scheduled. Based on the operator requirements of the GPU tasks to be scheduled, the corresponding operator performance features are extracted from the GPU cross-chip operator fingerprint set. The operator requirements of the GPU tasks to be scheduled are matched and matrixed item by item with the operator performance features to generate a GPU task fingerprint adaptation matrix.

[0012] As a preferred embodiment of the GPU computing power scheduling method based on a multi-core heterogeneous computing power platform of the present invention, the method of generating the GPU task to be scheduled specifically includes reading the expected execution time, expected memory usage, expected communication wait and precision adaptation status from the GPU task fingerprint adaptation matrix, and merging them according to the continuity of memory usage and the continuity of computing power usage to generate GPU load granularity data. Based on the GPU load granularity data, the GPU load is divided into whole card load, slice load and time-sharing load. The execution path is divided by calling the expected communication wait and precision adaptation status from the GPU task fingerprint adaptation matrix, and the GPU execution mode data is generated. Based on the GPU execution mode data, the full-card load, slice load, time-sharing load and execution path are combined accordingly to generate the GPU task load mode to be scheduled.

[0013] As a preferred embodiment of the GPU computing power scheduling method based on a cloud-multi-core heterogeneous computing power platform described in this invention, the step of generating a GPU candidate bearer set specifically includes calling the expected execution time, expected memory usage, expected communication wait and precision adaptation status in the GPU task fingerprint adaptation matrix according to the GPU task bearing mode to be scheduled, and organizing the corresponding execution usage to generate GPU bearing adaptation data. Based on the GPU load adaptation data, the operator execution time, expected memory usage, expected communication wait and precision adaptation status corresponding to the GPU task load mode to be scheduled are compared and evaluated to generate GPU load evaluation data. Based on the GPU load evaluation data, the load methods of the GPU tasks to be scheduled are sorted, and the load methods of the GPU tasks to be scheduled that are mismatched in terms of memory, communication wait, and precision are eliminated, thereby generating a set of GPU candidate load methods.

[0014] As a preferred embodiment of the GPU computing power scheduling method based on a multi-core heterogeneous computing power platform in this invention, the generation of GPU fragment counterfactual evaluation information specifically includes: establishing a counterfactual baseline state based on the current continuous memory interval, slice specifications, GPU instance capacity, and interconnect path bandwidth in the GPU running data; taking each type of GPU task to be scheduled in the GPU candidate load set as a hypothetical load action; and deducting the corresponding resource occupancy in the counterfactual baseline state to generate a counterfactual occupancy state. In the counterfactual occupancy state, the remaining continuous interval of video memory, the remaining slice size, the remaining capacity of the GPU instance and the remaining bandwidth of the interconnect path are reconstructed respectively, and the remaining resource form corresponding to each counterfactual occupancy state is matched with the resource requirements of subsequent tasks in the scheduling queue to generate a counterfactual acceptable result. The difference between the acceptable counterfactual result and the acceptable result under the counterfactual baseline is compared to obtain the memory continuity loss, slice size loss, GPU instance capacity loss and interconnect path bandwidth loss. The fragmentation counterfactual cost is calculated by combining the current task carrying adaptation degree in the GPU task fingerprint adaptation matrix. The carrying methods of each GPU task to be scheduled are sorted according to the fragmentation counterfactual cost to generate GPU fragmentation counterfactual evaluation information.

[0015] As a preferred embodiment of the GPU computing power scheduling method based on a multi-core heterogeneous computing power platform in this invention, the generation of GPU scheduling closed-loop update data specifically includes: filtering the GPU task carrying mode to be scheduled based on GPU fragment counterfactual assessment information, and taking the GPU task carrying mode to be scheduled that meets the preset fragment control conditions as the target carrying mode. Based on the target bearing method, GPU resources are bound to the GPU task to be scheduled, and GPU resource binding execution data is generated. The GPU task to be scheduled is started based on the GPU resource binding execution data, and the actual execution time, actual memory usage, actual communication wait and task completion status are collected during the execution of the GPU task to be scheduled, and the execution feedback of the GPU task to be scheduled is generated. Based on the feedback from the GPU tasks to be scheduled, the performance characteristics of operators in the cross-core operator fingerprint set and the impact of GPU fragmentation in the GPU fragmentation counterfactual assessment information are updated to generate closed-loop update data for GPU scheduling.

[0016] The beneficial effects of this invention are as follows: By simulating the resource occupancy of each scheduled GPU task in the GPU candidate load set before execution, and generating simulation evaluation information on the impact of resource fragmentation, it is possible to pre-determine the impact of different load modes on the remaining continuous interval of video memory, the remaining slice combination relationship, the number of unoccupied whole cards, and the unoccupied multi-GPU connection paths before GPU resource binding. Based on this, the target load mode is selected, so that the GPU scheduling result not only meets the computing power, video memory, accuracy, and communication requirements of the current GPU task, but also retains as much of the resource form that subsequent GPU tasks can continue to undertake, thereby improving the matching accuracy, continuous undertaking capability, and resource utilization stability of GPU resource scheduling in a cloud multi-core heterogeneous computing power platform. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a GPU computing power scheduling method based on a multi-core heterogeneous computing power platform.

[0019] Figure 2 A flowchart for generating cross-core operator fingerprint sets for GPUs.

[0020] Figure 3 A flowchart for generating the GPU candidate bearer set.

[0021] Figure 4 A flowchart for generating closed-loop update data for GPU scheduling. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Reference Figures 1-4 This is one embodiment of the present invention, which provides a GPU computing power scheduling method based on a cloud-multi-core heterogeneous computing power platform, including the following steps: S1. Collect GPU running data, filter GPU chips that meet the test conditions according to idle stability and test security, issue benchmark operator test tasks in the current idle window and record benchmark operator test execution data, and generate GPU cross-chip operator fingerprint set.

[0026] S1.1 Collect GPU load status, GPU memory usage status, GPU temperature status, GPU power consumption status, GPU task usage status, GPU testable status, GPU memory specifications, GPU supported precision, GPU interconnect method, GPU available memory continuous range, GPU slice support status, GPU slice specifications, GPU slice usage status, GPU whole card usage status, and GPU connection path usage status according to GPU chip identifier and GPU instance identifier, and organize them to generate GPU running data.

[0027] Specifically, based on the GPU chip identifier and GPU instance identifier, the system reads the GPU load status, GPU memory usage status, GPU temperature status, GPU power consumption status, and continuous range of available GPU memory from the GPU driver monitoring interface; it reads the GPU task usage status, GPU slice usage status, GPU overall card usage status, and GPU connection path usage status from the GPU resource binding record; and it reads the GPU memory specifications, GPU supported precision, GPU interconnect method, GPU slice support status, and GPU slice specifications from the GPU device attribute list. The system then determines whether the GPU load status is below a preset load threshold, whether the GPU memory usage status is within a preset memory fluctuation range, and whether the GPU temperature status is within a preset range. The GPU's testable status is determined by checking whether the GPU power consumption is within the preset temperature range, whether the GPU task occupancy status indicates that it is not exclusively occupied by a task, and so on. The GPU load status, GPU memory occupancy status, GPU temperature status, GPU power consumption status, GPU task occupancy status, GPU testable status, GPU memory specifications, GPU supported precision, GPU interconnect method, continuous range of available GPU memory, GPU slice support status, GPU slice specifications, GPU slice occupancy status, GPU whole card occupancy status, and GPU connection path occupancy status under the same GPU chip identifier and GPU instance identifier are timestamped and fields are merged to generate GPU running data.

[0028] It should also be noted that the load threshold is set based on the GPU chip manufacturer's manual, historical stable operation records, and benchmark operator test error requirements, for example, the value range is 10% to 30% of GPU utilization; the memory fluctuation range is set based on the historical fluctuation records of GPU memory usage and the requirements of benchmark operator tests for memory stability, for example, the value range is no more than 3% to 8% change in memory usage within 30 consecutive seconds; the temperature range is set based on the GPU chip manufacturer's recommended operating temperature and historical stable operating temperature records, for example, the value range is 30℃ to 75℃; the power consumption range is set based on the GPU chip's rated power consumption, heat dissipation capacity, and benchmark operator test stability requirements, for example, the value range is 20% to 70% of the rated power consumption.

[0029] S1.2 The GPU chips that meet the test conditions are those selected based on GPU operating data that are in a low-load state, have stable memory usage, normal temperature, normal power consumption, and are not exclusively occupied by any task.

[0030] Specifically, based on GPU operating data, the GPU load status, GPU memory usage status, GPU temperature status, GPU power consumption status, and GPU task usage status are read item by item. GPU chips with GPU load status below a preset load threshold are marked as being in a low load state; GPU chips with GPU memory usage status within a preset memory fluctuation range are marked as being in a stable memory usage state; GPU chips with GPU temperature status within a preset temperature range are marked as being in a normal temperature state; GPU chips with GPU power consumption status within a preset power consumption range are marked as being in a normal power consumption state; and GPU chips with task usage status not indicating task exclusivity are marked as not being task exclusivity. GPU chips that simultaneously meet the requirements of low load state, stable memory usage state, normal temperature state, normal power consumption state, and not being task exclusivity are selected to obtain GPU chips that meet the test conditions.

[0031] S1.3 The benchmark operator test task is a lightweight operator test task based on the chip type, supported precision, memory specifications, interconnection method and current idle window configuration of the GPU chip that meets the test conditions.

[0032] Specifically, test operators for matrix computation, convolution computation, normalization computation, attention computation, and communication synchronization computation are selected according to the chip type of the GPU chip that meets the test conditions. The computational precision of the test operators is configured according to the supported precision of the GPU chip that meets the test conditions. The data scale of the test operators is configured according to the memory specifications of the GPU chip that meets the test conditions. The communication path for communication synchronization computation is configured according to the interconnection method of the GPU chip that meets the test conditions. The execution time and number of loops of the test operators are limited according to the current idle window to form a benchmark operator test task.

[0033] It should also be noted that the current idle window is determined by the GPU load status, GPU memory usage status, and GPU task usage status in the GPU running data, according to the timestamps, for a period of time when the GPU load status, GPU memory usage status, and GPU task usage status continuously meet the low load status, stable memory usage status, and are not exclusively occupied by a task.

[0034] S1.4 Extract the load fluctuation status, memory usage stability status, temperature status, power consumption status, and task usage status from the GPU operating data, and filter each GPU chip according to idle stability and test security to generate GPU chip test condition screening data.

[0035] Specifically, GPU load status, GPU memory usage status, GPU temperature status, GPU power consumption status, and GPU task usage status are extracted from GPU operating data. A sliding time window statistical method is used to calculate the mean and maximum difference of GPU load samples at consecutive time stamps to form a load fluctuation status. The rate of change and maximum fluctuation of GPU memory usage samples at consecutive time stamps are calculated to form a memory usage stability status. Idle stability is evaluated based on load fluctuation status and memory usage stability status. Test safety is evaluated based on temperature status, power consumption status, and task usage status. GPU chips that meet the requirements of low load status and stable memory usage status in idle state, and the requirements of normal temperature status and normal power consumption status in test safety status, and are not exclusively occupied by tasks, are written into the GPU chip test condition screening data.

[0036] S1.5 Extract GPU chips that meet the test conditions from the GPU chip test condition screening data, and configure the benchmark operator test task according to the chip type, supported precision, memory specifications, interconnection method and current idle window of the GPU chip that meets the test conditions, and generate benchmark operator test task distribution information.

[0037] Specifically, GPU chips that meet the test conditions are extracted from the GPU chip test condition screening data according to their GPU chip identifiers. The corresponding chip type, supported precision, memory specifications, available continuous memory range, interconnect method, and current free window are read. Based on the chip type and operator support relationship recorded in the benchmark operator configuration table, at least one of the following operators is selected: matrix multiplication operator, convolution operator, normalization operator, attention operator, and communication synchronization operator, to determine the benchmark operator type. The supported precision is compared with the precision field in the test precision configuration table, and the commonly supported computational precision is selected to determine the benchmark operator. The calculation precision is as follows: Calculate the available video memory capacity for testing according to the video memory specifications and the continuous range of available video memory in the GPU, and configure the input tensor size of the benchmark operator according to the test usage ratio to determine the data size of the benchmark operator; Determine the single-card execution path, PCIe interconnect path, NVLink interconnect path, or NVSwitch interconnect path corresponding to the GPU chip according to the interconnection method, and determine the matching execution path as the communication path of the benchmark operator; Determine the number of benchmark operator loops and the benchmark operator execution duration based on the duration of the current idle window and the upper limit of the test duration, and generate benchmark operator test task distribution information.

[0038] Among them, the benchmark operator configuration table, test accuracy configuration table, test occupancy ratio and test duration limit are jointly determined by the GPU chip manufacturer's operation manual, the platform's historical stable test records and the current GPU resource pool management strategy.

[0039] S1.6. Based on the information issued by the benchmark operator test task, control the GPU chip that meets the test conditions to execute the benchmark operator test task, and record the operator execution time, changes in video memory usage and communication waiting status, and generate benchmark operator test execution data.

[0040] Specifically, based on the benchmark operator test task information, the benchmark operator test task is sent to the GPU chip that meets the test conditions, and the benchmark operator test task is executed according to the benchmark operator type, benchmark operator calculation precision, benchmark operator data size, benchmark operator communication path, and benchmark operator execution duration in the benchmark operator test task information. During the execution of the benchmark operator test task, the operator start time and operator end time are recorded to obtain the operator execution duration, the difference in video memory usage before and after execution is recorded to obtain the video memory usage change, and the communication synchronization wait time is recorded to obtain the communication wait status. The operator execution duration, video memory usage change, and communication wait status are written into the same test record according to the GPU chip identifier to generate benchmark operator test execution data.

[0041] It should also be noted that the benchmark operator test task is executed within an isolated micro-window. The isolated micro-window is the time interval during which the GPU load status, GPU memory usage status, GPU temperature status, GPU power consumption status, and GPU task usage status all meet the test conditions within a continuous sampling period. During the execution of the benchmark operator test task, the load disturbance amplitude, memory disturbance amplitude, and communication waiting disturbance amplitude are collected simultaneously, and the test fluctuation coefficient is calculated based on the load disturbance amplitude, memory disturbance amplitude, and communication waiting disturbance amplitude. When the test fluctuation coefficient exceeds the test fluctuation limit, the corresponding benchmark operator test execution data is marked as invalid test data, and writing to the GPU cross-chip operator fingerprint set is prohibited.

[0042] S1.7. Based on the benchmark operator test execution data, organize the operator performance differences between different GPU chips according to the GPU chip identifier and benchmark operator type, organize the operator performance differences into operator performance features, and write the operator performance features into the fingerprint record of the corresponding GPU chip to generate a GPU cross-chip operator fingerprint set.

[0043] Specifically, based on the benchmark operator test execution data, the operator execution time, memory usage changes, and communication wait times for the same GPU chip are collected according to the GPU chip identifier and benchmark operator type. The normalization comparison uses the preset standard execution time, preset standard memory usage changes, and preset standard communication wait times corresponding to the same benchmark operator type as the normalization benchmark. The operator execution time, memory usage changes, and communication wait times for each GPU chip are compared with the normalization benchmark to obtain the operator performance differences between different GPU chips. The differences in execution time, memory usage, and communication wait times under the same benchmark operator type are written into the same operator performance record to form operator performance characteristics. These operator performance characteristics are then written into the fingerprint record of the corresponding GPU chip to generate a cross-chip GPU operator fingerprint set.

[0044] It should also be noted that the standard execution time, standard memory change amount, and standard communication wait time can be jointly set by historical stable test records, manufacturer-published performance parameters, and benchmark test records of similar GPU chips.

[0045] S2. Obtain the GPU task request to be scheduled, parse the task computation graph, running framework, accuracy requirements, parallel mode and latency constraints, generate the GPU task operator requirements to be scheduled, and match and matrix-organize the GPU task operator requirements to be scheduled with the operator performance features in the GPU cross-chip operator fingerprint set to generate the GPU task fingerprint adaptation matrix.

[0046] S2.1 The GPU task request to be scheduled is obtained by receiving the task submission information sent by the GPU task submission end and parsing the task fields of the task submission information.

[0047] Specifically, the system receives task submission information from the GPU task submission client, performs integrity verification and field parsing on the task submission information, reads the task identifier, task computation graph, task execution framework, task precision requirements, task parallelism mode, and task latency constraints, establishes a task record according to the task identifier, writes the task computation graph into the task structure field, writes the task execution framework into the runtime environment field, writes the task precision requirements into the precision constraint field, writes the task parallelism mode into the execution method field, writes the task latency constraints into the time constraint field, and binds the task structure field, runtime environment field, precision constraint field, execution method field, and time constraint field to the same task identifier to obtain the GPU task request to be scheduled.

[0048] S2.2 Perform field validation on the GPU task request to be scheduled, and extract the task computation graph, task execution framework, task precision requirements, task parallelism mode and task latency constraints from the GPU task request to be scheduled, and integrate them to generate basic parsing data of the GPU task to be scheduled.

[0049] Specifically, when performing field validation on the GPU task request to be scheduled, the number of computing nodes and the number of node connections in the task computation graph are compared with the preset minimum number. The task computation graph is considered complete when both the number of computing nodes and the number of node connections are not zero. The framework name and framework version in the task execution framework are matched with the preset supported framework list. The task execution framework is considered complete when the match is successful. The target computation precision in the task precision requirement is validated against the preset precision field format. The task precision requirement is considered complete when the validation passes. The matching relationship between the target computation precision and the GPU supported precision is written into the subsequent precision adaptation state. The single GPU execution flag or multi GPU execution flag in the task parallel mode is checked for consistency with the execution dependency relationship in the task computation graph. If they match, the task parallel mode is considered complete. At least one of the maximum waiting time and target completion time in the task latency constraint is validated against the preset time format. The task latency constraint is considered complete when the validation passes. After the task computation graph, task execution framework, task precision requirement, task parallel mode, and task latency constraint are all complete, they are written into the same task record according to the task identifier to generate the basic parsing data of the GPU task to be scheduled.

[0050] It should also be noted that the minimum number is set according to the requirement that the task computation graph can form an effective computation link, such as the number of computation nodes is not less than 1 and the number of node connection relationships is not less than 1; the list of supported frameworks is set according to the framework versions installed in the GPU runtime environment; the time format is set according to the unified parsing requirements of the task scheduling time field, such as second-level integer timestamps or ISO 8601 time format.

[0051] A successful match means that the framework name in the task execution framework can be found in the list of supported frameworks, and the framework version falls within the range of versions allowed to run the corresponding framework name in the list of supported frameworks. For example, the task execution framework is PyTorch and the version number is within the range of PyTorch versions that are installed and compatible with the current GPU runtime environment. A successful verification means that the format, value, and interface recognizability of the field to be verified all meet the scheduling parsing requirements. For example, the target calculation precision in the task precision requirement belongs to one of FP32, FP16, BF16, and INT8, and can be recognized by the GPU scheduling interface. The maximum waiting time in the task latency constraint is a resolvable numerical time, or the target completion time is a resolvable timestamp.

[0052] S2.3. Based on the basic parsing data of the GPU task to be scheduled, identify the operator type of the computing nodes in the task computation graph, and organize them according to the operator naming rules and execution order of the computing nodes corresponding to the task running framework to generate the operator structure data of the GPU task to be scheduled.

[0053] Specifically, based on the basic parsing data of the GPU task to be scheduled, the task graph structure and task execution framework are read, and the node name, operator identifier, and node connection relationship of each node in the task graph structure are extracted. According to the operator naming rules corresponding to the task execution framework, each node is classified into matrix operators, convolution operators, normalization operators, attention operators, or communication and synchronization operators. According to the node connection relationship and node execution order, the call position, before and after connection relationship, and parallel connection relationship of each operator type are sorted out to generate the operator structure data of the GPU task to be scheduled.

[0054] S2.4. Based on the operator structure data of the GPU task to be scheduled, count the number of calls, call positions, dependencies, precision requirements and parallel execution relationships of different operator types to generate the operator requirements for the GPU task to be scheduled.

[0055] Specifically, based on the operator structure data of the GPU task to be scheduled, the number of calls in the task computation graph is counted according to the operator type, and the call position of each operator type in the execution order of the computation nodes is recorded; the input source node and output destination node corresponding to each operator type are extracted according to the node connection relationship, and the succession order between adjacent operator types is determined according to the node execution order to form the sequential dependency relationship between each operator type; the task precision requirements and task parallel mode are read from the basic parsing data of the GPU task to be scheduled, and the precision requirements and parallel execution relationship are written into the corresponding operator type to generate the operator requirements of the GPU task to be scheduled.

[0056] S2.5. Based on the operator requirements of the GPU task to be scheduled, extract the corresponding operator performance features from the GPU cross-chip operator fingerprint set, and match and matrix-organize the operator requirements of the GPU task to be scheduled with the operator performance features item by item to generate a GPU task fingerprint adaptation matrix.

[0057] Specifically, based on the operator type, number of calls, call location, dependencies, precision requirements, and parallel execution relationships in the operator requirements of the GPU tasks to be scheduled, the performance characteristics of operators of the same type are extracted from the GPU cross-chip operator fingerprint set. The operator requirements and performance characteristics of the GPU tasks to be scheduled are matched item by item according to the GPU chip identifier. The expected execution time, expected memory usage, expected communication wait, and precision adaptation status of each GPU chip are organized and arranged in a matrix according to the GPU task identifier and the GPU chip identifier to generate a GPU task fingerprint adaptation matrix.

[0058] It should also be noted that, based on the number of calls, call locations, and dependencies of the same operator type in the requirements of the GPU task to be scheduled, the corresponding operator performance characteristics in the GPU cross-chip operator fingerprint set are called to accumulate the expected execution time; based on the changes in video memory usage and call locations in the corresponding operator performance characteristics, the expected video memory usage is obtained; and based on the communication waiting situation and parallel execution relationship corresponding to the communication synchronization calculation, the expected communication waiting time is obtained.

[0059] S3. Based on the GPU task fingerprint adaptation matrix, organize the load granularity and divide the execution mode to generate the load mode of the GPU task to be scheduled, and sort and filter according to the load adaptation degree to generate the GPU candidate load set.

[0060] S3.1 Read the estimated execution time, estimated memory usage, estimated communication wait time and accuracy adaptation status from the GPU task fingerprint adaptation matrix, and merge them according to the continuity of memory usage and the continuity of computing power usage to generate GPU load granular data.

[0061] Specifically, the estimated execution time, estimated memory usage, estimated communication wait time, and precision adaptation status are read from the GPU task fingerprint adaptation matrix according to the GPU task identifier and GPU chip identifier. The estimated memory usage is calculated as a ratio to the continuous interval of available GPU memory to obtain the continuous memory usage ratio. The memory usage requirement of the scheduled GPU task is marked according to the hierarchical interval corresponding to the continuous memory usage ratio to form memory usage continuity. The estimated execution time and estimated communication wait time are summed to obtain the estimated continuous usage time. The ratio of the estimated continuous usage time to the task latency constraint is calculated to obtain the computing power continuous usage ratio. The computing power continuous usage degree of the scheduled GPU task is marked according to the hierarchical interval corresponding to the computing power continuous usage ratio to form computing power usage continuity. The memory usage continuity, computing power usage continuity, and precision adaptation status are merged according to the GPU chip identifier to generate GPU load granularity data.

[0062] S3.2. Based on the GPU load granularity data, divide the load into whole card load, slice load, and time-sharing load, and call the expected communication wait and precision adaptation status from the GPU task fingerprint adaptation matrix to divide the execution path and generate GPU execution mode data.

[0063] Specifically, based on the continuity of memory usage, the continuity of computing power usage, and the precision adaptation status in the GPU load granularity data, the scheduled GPU tasks with high continuity of memory usage and high continuity of computing power usage are classified as whole-card loads. When the precision adaptation status characterization is adapted and the GPU slice support status characterization supports slice execution, the corresponding scheduled GPU tasks are classified as slice loads. The scheduled GPU tasks with low continuity of memory usage and low continuity of computing power usage are classified as time-sharing loads. The expected communication wait and precision adaptation status are retrieved from the GPU task fingerprint adaptation matrix. The scheduled GPU tasks whose expected communication wait meets the task latency constraints are classified as single-GPU execution paths or multi-GPU parallel execution paths. The scheduled GPU tasks whose precision adaptation status does not meet the GPU chip identifier in the GPU task fingerprint adaptation matrix are classified as cross-chip alternative execution paths, generating GPU execution mode data.

[0064] In one embodiment, a continuous range of available GPU memory is read, and the ratio of the expected memory usage to the continuous range of available GPU memory is calculated to obtain the continuous memory usage ratio. When the continuous memory usage ratio is greater than 70%, the corresponding GPU task to be scheduled is marked as having high memory usage continuity. When the continuous memory usage ratio is greater than 30% but not greater than 70%, the corresponding GPU task to be scheduled is marked as having medium memory usage continuity. When the continuous memory usage ratio is not greater than 30%, the corresponding GPU task to be scheduled is marked as having low memory usage continuity. The high, medium, and low computing power usage continuity is divided according to the ratio range of the expected execution time and the expected communication waiting time to the task latency constraint.

[0065] S3.3. Based on the GPU execution mode data, combine the full-card load, slice load, time-sharing load and execution path accordingly to generate the GPU task load mode to be scheduled.

[0066] Specifically, based on the GPU execution mode data, the whole card load, slice load, time-sharing load and execution path are extracted according to the GPU task identifier to be scheduled. The whole card load is combined with the corresponding single GPU execution path or multi-GPU parallel execution path, the slice load is combined with the corresponding single GPU execution path, the time-sharing load is combined with the corresponding single GPU execution path, and the cross-chip replacement execution path is combined with the whole card load, slice load or time-sharing load corresponding to the replaceable GPU chip to generate the GPU task load mode to be scheduled.

[0067] It should also be noted that, based on the GPU task fingerprint adaptation matrix, a GPU chip that has been adapted and whose expected execution time, expected memory usage, and expected communication wait meet the running constraints of the GPU task to be scheduled is generated, and an alternative GPU chip is generated.

[0068] S3.4. Based on the GPU task bearing mode to be scheduled, call the estimated execution time, estimated memory usage, estimated communication wait and precision adaptation status in the GPU task fingerprint adaptation matrix, organize the corresponding execution usage, and generate GPU bearing adaptation data.

[0069] Specifically, based on the GPU task carrying method to be scheduled, the estimated execution time, estimated memory usage, estimated communication wait time, and precision adaptation status in the GPU task fingerprint adaptation matrix are retrieved according to the GPU task identifier and GPU chip identifier. The estimated execution time, estimated memory usage, estimated communication wait time, and precision adaptation status of the GPU task to be scheduled are then organized according to the whole card carrying, slice carrying, time-sharing carrying, and execution path to generate GPU carrying adaptation data.

[0070] S3.5. Based on the GPU load adaptation data, compare and evaluate the operator execution time, expected memory usage, expected communication wait time, and accuracy adaptation status corresponding to the GPU task load mode to be scheduled, and generate GPU load evaluation data.

[0071] Specifically, based on GPU load adaptation data, the estimated execution time, estimated memory usage, estimated communication wait time, and precision adaptation status are read according to the load mode of the GPU task to be scheduled. Based on the task latency constraints of the GPU task to be scheduled, the estimated execution time is added to the estimated communication wait time to obtain the estimated total time. If the estimated total time is not greater than the task latency constraints, it is marked as latency feasible. Based on the GPU memory specifications, GPU memory usage status, and the continuous range of available GPU memory, the remaining memory capacity and the maximum continuous available memory capacity are calculated. If the estimated memory usage is not greater than both the remaining memory capacity and the maximum continuous available memory capacity, it is marked as memory feasible. Based on the task precision requirements, the target computation precision is compared with the GPU's supported precision. If the target computation precision is included in the GPU's supported precision, it is marked as precision feasible. The latency feasible results, memory feasible results, and precision feasible results are merged according to the load mode of the GPU task to be scheduled to generate GPU load evaluation data.

[0072] S3.6. Based on the GPU load evaluation data, sort the GPU task load methods to be scheduled and remove those with mismatched memory, communication wait, or precision, and generate a GPU candidate load set.

[0073] Specifically, based on the GPU load evaluation data, the GPU task load methods to be scheduled are arranged in order of load priority. The GPU task load methods to be scheduled that fail the memory feasibility judgment are eliminated, the GPU task load methods to be scheduled that fail the latency feasibility judgment in the expected communication wait do not meet the task latency constraints are eliminated, and the GPU task load methods to be scheduled that fail the accuracy feasibility judgment are eliminated. The remaining GPU task load methods to be scheduled are then written into the GPU candidate load set in order of load priority.

[0074] S4. Simulate and evaluate the remaining GPU resource status after the scheduling of GPU tasks in the GPU candidate bearer set is occupied, and generate GPU fragmentation counterfactual evaluation information.

[0075] S4.1. Based on the current continuous memory range, slice size, GPU instance capacity and interconnect path bandwidth in the GPU running data, establish a counterfactual baseline state. Take each scheduled GPU task in the GPU candidate load set as a hypothetical load action and deduct the corresponding resource usage in the counterfactual baseline state to generate a counterfactual occupancy state.

[0076] Specifically, based on GPU runtime data, the current contiguous memory range, slice size, GPU instance capacity, and interconnect path bandwidth are read and organized into pre-load resource records according to GPU chip identifier, GPU instance identifier, and interconnect path identifier, establishing a counterfactual baseline state. Each scheduled GPU task load mode is read item by item from the GPU candidate load set, and each scheduled GPU task load mode is set as a hypothetical load action. According to the memory contiguous range occupancy, slice size occupancy, GPU instance capacity occupancy, and interconnect path bandwidth occupancy corresponding to the hypothetical load action, the corresponding resource occupancy is deducted item by item in the counterfactual baseline state, and the remaining memory range, remaining slice size, remaining GPU instance capacity, and remaining interconnect path bandwidth after deduction are recorded to generate a counterfactual occupancy state.

[0077] S4.2. Under the counterfactual occupancy state, reconstruct the remaining continuous interval of video memory, the remaining slice size, the remaining capacity of the GPU instance, and the remaining bandwidth of the interconnect path, and match the remaining resource form corresponding to each counterfactual occupancy state with the resource requirements of subsequent tasks in the scheduling queue to generate a counterfactual acceptable result.

[0078] Specifically, under the counterfactual occupancy state, based on the remaining video memory range, remaining slice specifications, remaining GPU instance capacity, and remaining interconnect path bandwidth after merging and deducting GPU chip identifiers, GPU instance identifiers, and interconnect path identifiers, the remaining continuous video memory range, remaining slice specifications, remaining GPU instance capacity, and remaining interconnect path bandwidth are reconstructed to form the remaining resource form. The video memory requirements, instance requirements, and parallel communication requirements of subsequent tasks are read from the resource requirements of subsequent tasks in the scheduling queue. The video memory requirements of subsequent tasks are matched with the remaining continuous video memory range, the instance requirements of subsequent tasks are matched with the remaining slice specifications and remaining GPU instance capacity, and the parallel communication requirements of subsequent tasks are matched with the remaining interconnect path bandwidth. The number of acceptable tasks, the reasons for inadmissibility, and the location of resource gaps are recorded to generate the counterfactual acceptable result.

[0079] S4.3. Compare the difference between the acceptable counterfactual result and the acceptable result under the counterfactual baseline state to obtain the memory continuity loss, slice size loss, GPU instance capacity loss and interconnect path bandwidth loss. Combine the current task carrying adaptation degree in the GPU task fingerprint adaptation matrix to calculate the fragmentation counterfactual cost. Sort the carrying methods of each GPU task to be scheduled according to the fragmentation counterfactual cost to generate GPU fragmentation counterfactual evaluation information.

[0080] Specifically, for each GPU task loadout method to be scheduled, the number of acceptable tasks, reasons for inacceptability, and resource gap locations in the counterfactual acceptable results are compared with the acceptable results in the counterfactual baseline state, item by item, to obtain memory continuity loss, slice specification loss, GPU instance capacity loss, and interconnect path bandwidth loss. The current task loadout adaptation degree is read from the GPU task fingerprint adaptation matrix, and the memory continuity loss, slice specification loss, GPU instance capacity loss, interconnect path bandwidth loss, and current task loadout adaptation degree are normalized using a normalized weighted difference method. The memory continuity loss, slice specification loss, GPU instance capacity loss, and interconnect path bandwidth loss are used as cost terms, and the current task loadout adaptation degree is used as a deduction term to calculate the fragmentation counterfactual cost for each GPU task loadout method to be scheduled. The GPU task loadout methods to be scheduled are arranged from low to high according to the fragmentation counterfactual cost, and the corresponding loss source, resource gap location, and current task loadout adaptation degree are written in to generate GPU fragmentation counterfactual evaluation information.

[0081] S5. Select the target bearing method based on the GPU fragment counterfactual assessment information, bind the GPU tasks to be scheduled to GPU resources according to the target bearing method, collect the running feedback of the GPU tasks to be scheduled, and generate GPU scheduling closed-loop update data.

[0082] S5.1. Based on the GPU fragment counterfactual assessment information, the scheduling mode of the GPU task to be scheduled is screened, and the scheduling mode of the GPU task to be scheduled that meets the preset fragment control conditions is selected as the target mode.

[0083] Specifically, based on the GPU fragmentation counterfactual assessment information, the scheduling methods of each GPU task are checked in order of GPU fragmentation impact. The scheduling methods of GPU tasks that meet the preset fragmentation control conditions in terms of memory fragmentation impact, slice fragmentation impact, whole card retention impact, and multi-GPU connection impact are retained as bindable methods. The comprehensive load evaluation value is calculated based on the expected total time, memory usage ratio, communication waiting ratio, and precision adaptation status of the bindable methods. The bindable methods are arranged in ascending order of comprehensive load evaluation value, and the scheduling method of the GPU task with the smallest comprehensive load evaluation value is selected as the target method.

[0084] It should also be noted that the fragmentation control conditions are screening conditions set according to GPU resource retention requirements, subsequent GPU task acceptance requirements and GPU resource pool management strategies. These include the following conditions: the remaining continuous interval of video memory is not less than the preset video memory retention threshold; the remaining slice combination relationship meets the subsequent slice reorganization requirements; the number of unoccupied whole cards is not less than the preset whole card retention number; and the unoccupied multi-GPU connection path meets the subsequent multi-GPU parallel carrying requirements. The number of full cards to be reserved is set based on the proportion of high-priority tasks in the GPU resource pool, the historical arrival frequency of full card tasks, the average waiting time of full card tasks, and the total number of full cards in the GPU resource pool. For example, the value range is usually 5% to 20% of the total number of full cards in the GPU resource pool, and no less than 1 full GPU card. The memory retention threshold is set based on the GPU memory specifications, the frequency of historical large memory tasks, and the demand for subsequent GPU tasks. For example, the value ranges from 10% to 30% of the GPU memory specifications, and is not lower than the median value of memory demand for historical large memory tasks. The subsequent slice reassembly requirements and subsequent multi-GPU parallel processing requirements are determined based on the waiting queue of GPU task requests to be scheduled, historical GPU task arrival records, and GPU resource pool management strategies.

[0085] S5.2. Based on the target carrying method, bind GPU resources to the GPU task to be scheduled and generate GPU resource binding execution data.

[0086] Specifically, based on the target carrying method, the corresponding GPU chip identifier, GPU instance identifier, whole card carrying, slice carrying, time-sharing carrying, and execution path are read. The GPU task to be scheduled is bound to one of the GPU chip identifier, GPU instance identifier, whole card carrying, slice carrying, time-sharing carrying, and execution path, and the binding time, binding range, and binding status are written synchronously to generate GPU resource binding execution data.

[0087] S5.3. Start the GPU task to be scheduled based on the GPU resource binding execution data, and collect the actual execution time, actual video memory usage, actual communication wait and task completion status during the execution of the GPU task to be scheduled, and generate the execution feedback of the GPU task to be scheduled.

[0088] Specifically, the GPU task to be scheduled is started based on the GPU chip identifier, GPU instance identifier, bearer type, execution path, binding time and binding range in the GPU resource binding execution data. During the execution of the GPU task to be scheduled, the task start time, task end time, peak memory usage, communication synchronization wait time and task completion mark are recorded. The actual execution time is generated based on the task start time and task end time, the actual memory usage is generated based on the peak memory usage, and the actual communication wait time is generated based on the communication synchronization wait time. The actual execution time, actual memory usage, actual communication wait and task completion status are then organized according to the GPU task identifier to be scheduled to generate the GPU task execution feedback.

[0089] S5.4. Based on the running feedback of the GPU tasks to be scheduled, update the operator performance characteristics in the GPU cross-chip operator fingerprint set and the GPU fragmentation counterfactual assessment information to generate GPU scheduling closed-loop update data.

[0090] Specifically, based on the feedback from the GPU tasks to be scheduled, the actual execution time, actual memory usage, actual communication wait time, and task completion status are extracted according to the GPU task identifier and GPU chip identifier. Based on the operator type, number of calls, and dependencies in the GPU task operator requirements, the actual execution time, actual memory usage, and actual communication wait time are allocated to the corresponding operator types. The allocated actual execution time, actual memory usage, and actual communication wait time are then written back to the corresponding operator performance characteristics in the GPU cross-chip operator fingerprint set. Based on the task completion status, actual memory usage, actual communication wait time, and GPU resource release status after task completion, the GPU fragmentation impact in the GPU fragmentation counterfactual assessment information is corrected. The updated operator performance characteristics and updated GPU fragmentation impact are then organized according to the GPU task identifier to generate closed-loop update data for GPU scheduling.

[0091] In summary, this invention simulates the resource occupancy of each scheduled GPU task in the candidate GPU loader set before execution, generates simulation evaluation information on the impact of resource fragmentation, and can pre-determine the impact of different loaders on the remaining continuous memory intervals, the remaining slice combination relationships, the number of unoccupied whole cards, and the unoccupied multi-GPU connection paths before GPU resource binding. Based on this, the target loader is selected, so that the GPU scheduling result not only meets the computing power, memory, accuracy, and communication requirements of the current GPU task, but also preserves as much of the resource form as possible for subsequent GPU tasks to continue to be undertaken. This improves the matching accuracy, continuous undertaking capability, and resource utilization stability of GPU resource scheduling in a cloud multi-core heterogeneous computing platform.

[0092] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A GPU computing power scheduling method based on a multi-core heterogeneous computing power platform, characterized in that: include, Collect GPU running data, filter GPU chips that meet the test conditions according to idle stability and test security, issue benchmark operator test tasks in the current idle window and record benchmark operator test execution data, and generate GPU cross-chip operator fingerprint set; The system obtains GPU task requests to be scheduled, parses the task computation graph, running framework, accuracy requirements, parallel mode and latency constraints, generates the GPU task operator requirements to be scheduled, and performs item-by-item matching and matrix organization of the GPU task operator requirements with the operator performance features in the GPU cross-chip operator fingerprint set to generate a GPU task fingerprint adaptation matrix. Based on the GPU task fingerprint adaptation matrix, the granularity of task handling and execution methods are organized, and the task handling methods for GPU tasks to be scheduled are generated, specifically including... Read the estimated execution time, estimated memory usage, estimated communication wait and accuracy adaptation status from the GPU task fingerprint adaptation matrix, and merge them according to the continuity of memory usage and the continuity of computing power usage to generate GPU-load granular data. Based on the GPU load granularity data, the GPU load is divided into whole card load, slice load and time-sharing load. The execution path is divided by calling the expected communication wait and precision adaptation status from the GPU task fingerprint adaptation matrix, and the GPU execution mode data is generated. Based on the GPU execution method data, the full card load, slice load, time-sharing load and execution path are combined accordingly to generate the GPU task load method to be scheduled, and sorted and filtered according to the load adaptability to generate a GPU candidate load set; The remaining GPU resources after the scheduling methods of the GPU tasks in the candidate GPU loadout set are occupied are simulated and evaluated to generate GPU fragmentation counterfactual assessment information, specifically including: Based on the current continuous memory interval, slice size, GPU instance capacity and interconnect path bandwidth in the GPU running data, a counterfactual baseline state is established. Each GPU task to be scheduled in the GPU candidate load set is taken as a hypothetical load action, and the corresponding resource occupancy is deducted in the counterfactual baseline state to generate a counterfactual occupancy state. In the counterfactual occupancy state, the remaining continuous interval of video memory, the remaining slice size, the remaining capacity of the GPU instance, and the remaining bandwidth of the interconnect path are reconstructed respectively, and the remaining resource form corresponding to each counterfactual occupancy state is matched with the resource requirements of subsequent tasks in the scheduling queue to generate a counterfactual acceptable result. The difference between the acceptable counterfactual result and the acceptable result under the counterfactual baseline is compared to obtain the memory continuity loss, slice size loss, GPU instance capacity loss and interconnect path bandwidth loss. The fragmentation counterfactual cost is calculated by combining the current task carrying adaptation degree in the GPU task fingerprint adaptation matrix. The carrying mode of each GPU task to be scheduled is sorted according to the fragmentation counterfactual cost to generate GPU fragmentation counterfactual evaluation information. The target bearing method is selected based on the GPU fragment counterfactual assessment information. The GPU tasks to be scheduled are bound to GPU resources according to the target bearing method, and the running feedback of the GPU tasks to be scheduled is collected to generate GPU scheduling closed-loop update data.

2. The GPU computing power scheduling method based on a multi-core heterogeneous computing power platform in one cloud as described in claim 1, characterized in that: The collection of GPU runtime data specifically includes, According to the GPU chip identifier and GPU instance identifier, collect GPU load status, GPU memory usage status, GPU temperature status, GPU power consumption status, GPU task usage status, GPU testability status, GPU memory specifications, GPU supported precision, GPU interconnect method, GPU available memory continuous range, GPU slice support status, GPU slice specifications, GPU slice usage status, GPU whole card usage status, and GPU connection path usage status, and organize them to generate GPU running data.

3. The GPU computing power scheduling method based on a multi-core heterogeneous computing power platform in one cloud as described in claim 1, characterized in that: The GPU chips that meet the test conditions are those selected based on GPU operating data that are in a low-load state, have stable memory usage, normal temperature, normal power consumption, and are not exclusively occupied by any task. The benchmark operator test task is a lightweight operator test task based on the chip type, supported precision, memory specifications, interconnection method, and current idle window configuration of the GPU chip that meets the test conditions.

4. The GPU computing power scheduling method based on a multi-core heterogeneous computing power platform in one cloud as described in claim 1, characterized in that: The generation of the GPU cross-core operator fingerprint set specifically includes, The GPU operation data includes load fluctuation status, memory usage stability status, temperature status, power consumption status, and task usage status. The GPU chips are then filtered according to idle stability and test safety to generate GPU chip test condition filtering data. Extract GPU chips that meet the test conditions from the GPU chip test condition screening data, and configure benchmark operator test tasks based on the chip type, supported precision, memory specifications, interconnection method and current idle window of the GPU chips that meet the test conditions, and generate benchmark operator test task distribution information. Based on the information issued by the benchmark operator test task, the GPU chip that meets the test conditions is controlled to execute the benchmark operator test task, and the operator execution time, changes in video memory usage and communication waiting status are recorded to generate benchmark operator test execution data; Based on the benchmark operator test execution data, the differences in operator performance between different GPU chips are organized according to the GPU chip identifier and benchmark operator type. The differences in operator performance are organized into operator performance features, and the operator performance features are written into the fingerprint record of the corresponding GPU chip to generate a cross-chip GPU operator fingerprint set.

5. The GPU computing power scheduling method based on a multi-core heterogeneous computing power platform in one cloud as described in claim 1, characterized in that: The GPU task request to be scheduled is obtained by receiving task submission information sent by the GPU task submission client and parsing the task fields of the task submission information.

6. The GPU computing power scheduling method based on a multi-core heterogeneous computing power platform in one cloud as described in claim 5, characterized in that: The generation of the GPU task fingerprint adaptation matrix specifically includes, The fields of the GPU task request to be scheduled are validated, and the task computation graph, task execution framework, task precision requirements, task parallelism mode and task latency constraints are extracted from the GPU task request to be scheduled. The data are then integrated to generate the basic parsing data of the GPU task to be scheduled. Based on the basic parsing data of the GPU task to be scheduled, the operator type of the computing nodes in the task computation graph is identified, and the nodes are organized according to the operator naming rules and execution order of the task running framework to generate the operator structure data of the GPU task to be scheduled. Based on the operator structure data of the GPU task to be scheduled, the number of calls, call positions, dependencies, precision requirements and parallel execution relationships of different operator types are statistically analyzed to generate the operator requirements of the GPU task to be scheduled. Based on the operator requirements of the GPU tasks to be scheduled, the corresponding operator performance features are extracted from the GPU cross-chip operator fingerprint set. The operator requirements of the GPU tasks to be scheduled are matched and matrixed item by item with the operator performance features to generate a GPU task fingerprint adaptation matrix.

7. The GPU computing power scheduling method based on a multi-core heterogeneous computing power platform in one cloud as described in claim 1, characterized in that: The generation of the GPU candidate bearer set specifically includes, Based on the GPU task bearing method to be scheduled, the estimated execution time, estimated memory usage, estimated communication wait and precision adaptation status in the GPU task fingerprint adaptation matrix are called, and the corresponding execution usage is organized to generate GPU bearing adaptation data. Based on the GPU load adaptation data, the operator execution time, expected memory usage, expected communication wait and precision adaptation status corresponding to the GPU task load mode to be scheduled are compared and evaluated to generate GPU load evaluation data. Based on the GPU load evaluation data, the load methods of the GPU tasks to be scheduled are sorted, and the load methods of the GPU tasks to be scheduled that are mismatched in terms of memory, communication wait, and precision are eliminated, thereby generating a set of GPU candidate load methods.

8. The GPU computing power scheduling method based on a multi-core heterogeneous computing power platform in one cloud as described in claim 1, characterized in that: The generation of GPU scheduling closed-loop update data specifically includes, Based on the counterfactual assessment information of GPU fragmentation, the scheduling mode of GPU task is screened, and the scheduling mode of GPU task that meets the preset fragmentation control conditions is selected as the target mode. Based on the target bearing method, GPU resources are bound to the GPU task to be scheduled, and GPU resource binding execution data is generated. The GPU task to be scheduled is started based on the GPU resource binding execution data, and the actual execution time, actual memory usage, actual communication wait and task completion status are collected during the execution of the GPU task to be scheduled, and the execution feedback of the GPU task to be scheduled is generated. Based on the feedback from the GPU tasks to be scheduled, the performance characteristics of operators in the cross-core operator fingerprint set and the impact of GPU fragmentation in the GPU fragmentation counterfactual assessment information are updated to generate closed-loop update data for GPU scheduling.

Images