A heterogeneous graphics processor resource scheduling method and system

Abstract

The present application relates to the technical field of computer, disclose a kind of heterogeneous graphics processor resource scheduling method and system, including reading hardware parameter, virtualization capability, interconnection capability and operating state from heterogeneous graphics processor interface, uniform conversion is resource information library record;Task type, computing power accuracy, video memory, software stack, communication demand and priority are extracted from task description and code environment, form task demand record;According to hardware advantage, residual computing power, residual video memory and load, filtering, scoring and sorting are carried out, and candidate resource is obtained;Single graphics processor allocation, virtual computing power slice allocation or multi-graphics processor aggregation allocation are generated;Abnormality is checked in running and checkpoint, resource screening, container reconstruction and breakpoint recovery are executed.The present application improves heterogeneous graphics processor resource utilization and task stability by uniform resource modeling and task matching.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and specifically to a method and system for scheduling heterogeneous graphics processor resources. Background Technology

[0002] Heterogeneous graphics processor clusters typically contain graphics processor resources with different architectures, driver interfaces, runtime interfaces, and interconnect capabilities. Existing scheduling methods often use the number of individual graphics processors, node idle status, or single vendor resource interfaces as scheduling criteria, making it difficult to simultaneously represent the static hardware parameters, virtualization capabilities, interconnect capabilities, and runtime status of graphics processors in a single resource view.

[0003] In actual scheduling, different tasks have significantly different requirements for computing power precision, video memory capacity, software stack version, inter-GPU communication, and task priority. If the scheduling system cannot extract these requirements from the task submission description and task code environment, problems such as mismatch between task and GPU hardware advantages, software stack incompatibility, and imbalance between video memory or computing power supply and demand can easily occur.

[0004] Furthermore, heterogeneous graphics processors (GPUs) employ different virtualization and interconnection methods. Some GPUs are suitable for hardware-level virtual instances, while others are better suited for software quota sharding. Some multi-GPU tasks are best aggregated within the same high-speed interconnect domain, while others require collaboration across different architectures via a common network layer path. Existing scheduling methods, if unable to differentiate between single GPU allocation, virtual computing power shard allocation, and multi-GPU aggregation allocation, can easily lead to resource fragmentation, unreasonable communication paths, and untimely task migration processing.

[0005] Therefore, this application provides a heterogeneous graphics processor resource scheduling method and system to solve the above-mentioned technical problems. Summary of the Invention

[0006] The purpose of this invention is to provide a heterogeneous graphics processor resource scheduling method and system to solve the technical problems in the prior art, such as inconsistent resource views, insufficient matching of tasks and graphics processor advantages, imprecise allocation of virtualization resources, and untimely adjustment of operational anomalies.

[0007] To address the aforementioned technical problems, this invention provides a heterogeneous graphics processor resource scheduling method, comprising: Static hardware parameters, virtualization capabilities, interconnect capabilities, and runtime status are read from the driver interface, runtime interface, and monitoring interface of the heterogeneous graphics processor, respectively, and converted into resource information database records according to a unified field dictionary and a unified unit of measurement. Extract task type, computing power and precision requirements, video memory requirements, software stack dependencies, inter-GPU communication requirements, and task priority from the task submission description and task code environment to form a task requirement record; Based on the task type, computing power precision requirements, software stack dependencies, and hardware advantage type, remaining computing power, remaining video memory, and load status recorded in the task requirement record, filter, score, and sort to obtain candidate graphics processors or candidate virtual computing power slices. Based on task priority, the sorting results of candidate graphics processors or candidate virtual computing power slices, and the communication requirements between graphics processors, generate single graphics processor allocation, virtual computing power slice allocation, or multi-graphics processor aggregate allocation results. During task execution, the runtime state is read and preset exception conditions are verified. For running tasks that meet the preset exception conditions, checkpoint saving, target resource filtering, container reconstruction and breakpoint recovery are performed, and the updated resource information database record and task scheduling results are output.

[0008] In some specific embodiments, the conversion of records into resource information databases according to a unified field dictionary and a unified unit of measurement further includes: Write the unique identifier of the graphics processor, the graphics processor type, and the node address into the resource index field; Write the total video memory, double-precision peak computing power, virtualization support type, and interconnect type into the static constraint field; Write the load rate, used video memory, temperature, power consumption, and health status into the running constraint field; Write the hardware advantage type and software ecosystem compatibility score into the matching score field; The resource index field, static constraint field, runtime constraint field, and matching score field are encapsulated into a resource information database record.

[0009] In some specific embodiments, forming a task requirement record further includes: Read the explicit declaration fields of task type, computing power precision, video memory capacity, software stack version, inter-GPU communication requirements, and task priority from the task submission description; When explicitly declared fields are missing or inconsistent with the task code environment, scan the data types in the task code, vendor runtime calls, and software stack paths in the container image documentation to infer computing power accuracy requirements and software stack dependencies. After performing conflict checks on the explicitly declared fields and the inferred results, a task requirement record is generated and pushed into the priority queue.

[0010] In some specific embodiments, performing filtering, scoring, and sorting further includes: Based on the software stack dependencies, video memory requirements, and computing power precision requirements recorded in the task requirements, graphics processors or virtual computing power slices that do not meet the hard operating conditions are eliminated. The hardware adaptation score is calculated based on the preset mapping relationship between hardware advantage type and task type; the computing power adaptation score is calculated based on the supply-demand ratio between remaining computing power and computing power demand; the video memory adaptation score is calculated based on the supply-demand ratio between remaining video memory and video memory demand; and the load distribution score is calculated based on the load status. The hardware adaptation score, computing power adaptation score, video memory adaptation score, and load distribution score are summed according to preset weights to obtain the ranking results of candidate graphics processors or candidate virtual computing power slices.

[0011] In some specific embodiments, generating a single graphics processor allocation, a virtual computing power slice allocation, or a multi-graphics processor aggregate allocation result further includes: When the computing power and video memory requirements in the task requirement record are lower than the preset slicing conditions, the hardware-level virtual instance corresponding to the graphics processor that supports hardware-level virtual instance will be converted into a fixed-granularity hard slice object, and the software quota slice corresponding to the graphics processor that supports software quota slicing will be converted into an adjustable-granularity soft slice object. The hardware preset granularity is verified for hard slice objects and the allocation request is rolled back when the granularity does not match. Runtime configuration instructions are generated for soft slice objects according to the computing power quota and the video memory limit. Write the slice type, memory quota, computing power quota, and isolation level into the virtual computing power slice allocation result.

[0012] In some specific embodiments, generating the multi-graphics processor aggregation allocation result further includes: Transform high-speed interconnect links, high-speed bus links, and remote direct memory access links into a weighted topology map that includes link bandwidth, link latency, and topological location; Based on the communication requirements between graphics processors, candidate graphics processor groups within the same high-speed interconnect domain are selected in the weighted topology graph, and a common network layer path is selected for candidate graphics processor groups with different architectures. The aggregated communication operations are mapped to aggregated communication primitives that can be executed by the vendor's communication library, and ring, tree, or hierarchical communication algorithms are selected according to the data volume and topology location to generate aggregated allocation results for multiple graphics processors.

[0013] In some specific embodiments, verifying preset abnormal conditions further includes: Read the temperature threshold, health status threshold, video memory usage threshold, computing load threshold, communication latency threshold, and priority preemption conditions according to the graphics processor type; Threshold comparisons are performed on the continuously sampled temperature values, device heartbeat, video memory page swapping status, computing unit utilization, and graphics processor link status, respectively. An anomaly signal is generated when any comparison result meets a preset anomaly condition; When the comparison result does not meet the preset abnormal conditions, the sampling result is written back to the resource information database and the current task allocation result is continued.

[0014] In some specific embodiments, reading the runtime state during task execution further includes: Perform fragmentation and reassembly processing on the virtual computing power slice: When a new task requests insufficient resources and multiple allocated slices on the same physical graphics processor are under low load, a reorganization scheme is calculated based on the unused video memory quota, unused computing power quota, and isolation level of each slice. The computing power quota and video memory limit can be adjusted for soft slice objects through the runtime quota interface; For hard slice objects, the slice composition is changed only when the slice is released or a new task is assigned; Write the adjusted slice occupancy relationship into the resource information database record.

[0015] In some specific embodiments, the preset weights are formed through policy parameter update processing, including: Extract task requirement records, graphics processor allocation results, actual runtime, peak video memory usage, energy consumption, migration records, and task completion status from historical scheduling logs to form a sliding window sample; The objective function is composed of cluster utilization, task waiting time, energy consumption and matching prediction bias, and candidate weight parameters are generated by Bayesian optimization. The candidate weight parameters are replayed and verified in shadow mode. Once the verification is successful, they are hot-loaded into the filtering, scoring, and sorting processes.

[0016] Based on the same concept, the present invention also provides a heterogeneous graphics processor resource scheduling system, comprising: The resource record generation module is configured to read static hardware parameters, virtualization capabilities, interconnection capabilities, and runtime status from the driver interface, runtime interface, and monitoring interface of the heterogeneous graphics processor, and convert them into resource information database records according to a unified field dictionary and a unified unit of measurement. The task parsing module is configured to extract task type, computing power and precision requirements, video memory requirements, software stack dependencies, inter-GPU communication requirements, and task priority from the task submission description and task code environment, forming a task requirement record. The candidate sorting module is configured to perform filtering, scoring and sorting according to the task type, computing power precision requirements, software stack dependencies and hardware advantage type, remaining computing power, remaining video memory and load status recorded in the task requirement record to obtain candidate graphics processors or candidate virtual computing power slices. The resource allocation module is configured to generate single graphics processor allocation, virtual computing power slice allocation, or multi-graphics processor aggregate allocation results based on task priority, the sorting results of candidate graphics processors or candidate virtual computing power slices, and the communication requirements between graphics processors. The operation adjustment module is configured to read the runtime status and verify preset abnormal conditions during task execution. For running tasks that meet the preset abnormal conditions, checkpoint saving, target resource filtering, container reconstruction and breakpoint recovery are performed, and the updated resource information database record and task scheduling results are output.

[0017] Compared with existing technologies, its advantages are as follows: This invention discloses a heterogeneous graphics processor resource scheduling method and system, which unifies the outputs of the driver interface, runtime interface and monitoring interface of different graphics processors into resource information database records. The scheduling core can read static hardware parameters, virtualization capabilities, interconnection capabilities and runtime status in the same resource view, reducing resource identification errors caused by inconsistent interface formats and field units.

[0018] The scheduling process extracts task type, computing power and precision requirements, video memory requirements, software stack dependencies, inter-GPU communication requirements, and task priority from the task submission description and task code environment. It can combine explicit declarations and task environment inference results to reduce the risk of scheduling failure caused by missing task requirements or software stack mismatch.

[0019] First, the system filters based on hard operating conditions, and then scores and sorts tasks according to hardware advantages, remaining computing power, remaining video memory, and load status. This allows tasks to be matched with candidate graphics processors or candidate virtual computing slices that have corresponding hardware advantages and whose current resource status meets the requirements, thereby reducing resource waste caused by resource mismatch and single-dimensional scheduling.

[0020] The scheduling system converts the hardware-level virtual instances corresponding to graphics processors that support hardware-level virtual instances into fixed-granularity hard slice objects, and converts the software quota slices corresponding to graphics processors that support software quota slicing into adjustable-granularity soft slice objects. The scheduling system can express different underlying virtualization mechanisms in a unified way and generate individual graphics processor allocation or virtual computing power slice allocation results according to task requirements.

[0021] Multiple interconnect links are converted into a weighted topology graph that includes link bandwidth, link delay and topological location. Candidate graphics processor groups and aggregated communication algorithms are selected according to the communication requirements between graphics processors. Multi-graphics processor tasks can combine topological location and communication requirements to form aggregated allocation results, reducing the coordination overhead caused by low-speed links or mismatched communication paths.

[0022] During task execution, the system continuously reads the runtime status and verifies temperature, health status, memory usage, computing load, communication latency, and priority preemption conditions. When abnormal conditions are met, the scheduling system can trigger checkpoint saving, target resource filtering, container reconstruction, and breakpoint recovery processing to reduce the impact of runtime anomalies on task continuity.

[0023] The virtual computing power slices are processed by fragmentation and reorganization. Based on the unused video memory quota, unused computing power quota and isolation level, the soft slice quota or subsequent hard slice combination is adjusted. The scheduling system can update the resource occupancy relationship when small tasks and fragmented resources coexist, thereby improving the consistency between the resource information database records and the actual allocable resources.

[0024] Historical scheduling logs are used to form a sliding window sample, and the scoring weights are updated using cluster utilization, task waiting time, energy consumption and matching prediction deviation as objective functions. The filtering, scoring and sorting processes can adjust parameters according to actual operation feedback, so that the task matching and resource allocation strategies can adapt to the changes in the operating status of heterogeneous graphics processor clusters. Attached Figure Description

[0025] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating some specific embodiments of the heterogeneous graphics processor resource scheduling method of the present invention; Figure 2 This is a schematic diagram of the structure of a heterogeneous graphics processor resource scheduling system according to some specific embodiments of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms "a," "the," and "the" as used in the embodiments of this application are also intended to include the plural forms, unless the context clearly indicates otherwise, and "multiple" generally includes at least two.

[0028] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0029] It should be understood that although the terms first, second, third, etc., may be used in the embodiments of this application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, first may also be referred to as second without departing from the scope of the embodiments of this application, and similarly, second may also be referred to as first.

[0030] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0031] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0032] It should be noted that any symbols and / or numbers present in the specification that are not marked in the accompanying drawings are not reference numerals.

[0033] Reference Figure 1 A method for scheduling resources of heterogeneous graphics processors, comprising: S101 reads static hardware parameters, virtualization capabilities, interconnect capabilities, and runtime status from the driver interface, runtime interface, and monitoring interface of the heterogeneous graphics processor, respectively, and converts them into resource information database records according to a unified field dictionary and a unified unit of measurement. S102, extract the task type, computing power and precision requirements, video memory requirements, software stack dependencies, inter-graphics processor communication requirements and task priority from the task submission description and task code environment to form a task requirement record; S103, according to the task type, computing power precision requirements, software stack dependencies and hardware advantage type, remaining computing power, remaining video memory and load status recorded in the task requirement record, perform filtering, scoring and sorting to obtain candidate graphics processors or candidate virtual computing power slices. S104, based on task priority, the sorting results of candidate graphics processors or candidate virtual computing power slices, and the communication requirements between graphics processors, generate single graphics processor allocation, virtual computing power slice allocation, or multi-graphics processor aggregate allocation results. S105 reads the runtime status and verifies the preset exception conditions during task execution. For running tasks that meet the preset exception conditions, it performs checkpoint saving, target resource filtering, container reconstruction and breakpoint recovery processing, and outputs the updated resource information database record and task scheduling results.

[0034] To provide a clearer explanation, the steps in the embodiments of the present invention are described in detail below: S101 reads static hardware parameters, virtualization capabilities, interconnect capabilities, and runtime status from the driver interface, runtime interface, and monitoring interface of the heterogeneous graphics processor, respectively, and converts them into resource information database records according to a unified field dictionary and a unified unit of measurement. Furthermore, the unique identifier of the graphics processor, the graphics processor type, and the node address are written into the resource index field; Write the total video memory, double-precision peak computing power, virtualization support type, and interconnect type into the static constraint field; Write the load rate, used video memory, temperature, power consumption, and health status into the running constraint field; Write the hardware advantage type and software ecosystem compatibility score into the matching score field, and encapsulate the resource index field, static constraint field, runtime constraint field and matching score field into a resource information database record.

[0035] In this embodiment, resource acquisition is performed during system initialization, graphics processor (GPU) access, GPU hot-plugging, and the arrival of runtime monitoring cycles. In this embodiment, the first type of GPU is a GPU with hardware-level virtual instance capabilities, the second type is a GPU with software quota sharding capabilities, and the third type is a GPU with high-bandwidth video memory and software sharding capabilities. For the first type of GPU, its unique identifier, driver version, double-precision peak computing power, total video memory, hardware-level virtualization support status, compute unit status, load rate, video memory usage, temperature, power consumption, and health status are read through its management commands, driver library functions, or runtime interface. For the second type of GPU, its video memory capacity, computing power precision support, software ecosystem compatibility parameters, computing power sharding capabilities, operating load, video memory usage, temperature, and health status are read through its runtime interface, management daemon, or deep learning adaptation interface. For the third type of graphics processor, the high-bandwidth memory capacity, memory bandwidth, interconnect bandwidth, multi-precision computing capability, software fragmentation capability, operating load, memory usage, temperature, and health status are read through the third type of graphics processor software stack monitoring interface.

[0036] The resource index field is used to uniquely locate the graphics processor resource in subsequent scheduling processing. The resource index field may include a globally unique identifier, graphics processor type, node address, and driver version. The static constraint field describes the hardware boundaries of the resource, which are determined before task allocation. The static constraint field may include total video memory, peak double-precision computing power, supported computing precision sets, virtualization support type, interconnect type, and interconnect domain identifier. The runtime constraint field describes the currently allocable capacity. The runtime constraint field may include load rate, used video memory, idle video memory, temperature, power consumption, health status, link status, and number of active tasks. The matching score field is used for subsequent filtering, scoring, and sorting steps. The matching score field may include hardware advantage type, software ecosystem compatibility score, interconnect affinity identifier, and isolation level. These fields are encapsulated into resource information base records and written to an in-memory database, a time-series database, or a shared cache. The key of the resource information base record can be formed by combining the graphics processor unique identifier and the node address, and the value of the resource information base record can be stored using a hierarchical key-value structure or a relational table structure.

[0037] In this embodiment, a unified field dictionary is used to mask the differences in data names between interfaces from different vendors. The computational utilization field in the first type of interface, the graphics processor utilization field in the third type of interface, and the device load field in the second type of interface are all mapped to load rate; the video memory usage field in the first type of interface, the high-bandwidth video memory usage field in the third type of interface, and the video memory occupancy field in the second type of interface are all mapped to used video memory; the temperature field output by different interfaces is converted to degrees Celsius; the video memory capacity output by different interfaces is converted to a unified capacity unit; and the computing power values ​​output by different interfaces are converted to a unified computing power unit. When parsing the raw output, the resource acquisition adapter extracts the raw fields according to the field mapping table and then converts them to a unified data format according to unit conversion rules. If an interface outputs a text table, the resource acquisition adapter extracts the raw fields according to preset field positions or regular expressions; if an interface outputs structured data, the resource acquisition adapter reads the raw fields according to the key-name mapping relationship; if an interface outputs log-style data, the resource acquisition adapter parses the valid fields according to the timestamp and field template.

[0038] In this embodiment, the software ecosystem compatibility score is used to describe the degree of adaptation between the task code and the graphics processor software stack. Taking compatibility with the general-purpose parallel computing ecosystem as an example, a compatibility score can be formed by weighting interface coverage, kernel migration success rate, and performance penalty coefficient: ; in, This indicates the software ecosystem compatibility score. Indicates interface coverage. Indicates kernel run success rate. This indicates the performance ratio.

[0039] Interface coverage represents the proportion of target ecosystem interfaces supported by the graphics processor driver to the total number of target ecosystem standard interfaces; kernel run pass rate represents the proportion of standard kernel samples that successfully compile and run on the graphics processor; performance ratio represents the performance ratio of the same task in compatibility mode and native mode. After the compatibility score is written to the resource information database, it serves as input for software stack dependency filtering and soft scoring in the execution filtering, scoring, and sorting steps. For native tasks that do not require a compatibility layer, the software ecosystem compatibility score can be directly formed by the native software stack adaptation status, driver version compatibility, and runtime interface availability.

[0040] In this embodiment, when a graphics processor (GPU) is first connected, static hardware parameters are read and an initial resource information database record is generated. When the runtime monitoring cycle arrives, only the runtime constraint fields and necessary matching score fields are updated. Static hardware parameters can be collected during system startup or hardware hot-plugging. The runtime status can be collected at a cycle of 5 seconds, which can be adjusted to 1 second per cycle under high load scenarios. If a vendor's interface fails to return data within a certain collection cycle, the resource acquisition adapter writes the health status in the resource information database record to an alert status and retains the static constraint fields from the previous cycle. If the runtime status of the same GPU cannot be read in multiple consecutive collection cycles, the GPU is marked as unallocable, and this unallocable mark is provided to the steps of filtering, scoring, and sorting. Through the above processing, a unified resource view of three types of GPUs can be read in the same resource information database.

[0041] In this embodiment, the resource information database record can be further divided into a basic identifier layer, a static specification layer, a dynamic status layer, and an advantage type layer. The basic identifier layer includes the unique identifier of the graphics processor, the graphics processor type, the node address, and the driver version; the static specification layer includes the total video memory, the set of supported computing precision, the double-precision peak computing power, the virtualization support type, and the interconnect type; the dynamic status layer includes the load rate, used video memory, temperature, power consumption, and health status; the advantage type layer includes advantages in double-precision scientific computing, high video memory bandwidth, integrated training and push adaptation, software ecosystem compatibility score, and interconnect domain identifier. After the task requirement record is generated, the above-mentioned hierarchical fields are read from the resource information database record, avoiding re-accessing the underlying driver interface each time a task is submitted.

[0042] S102, extract the task type, computing power and precision requirements, video memory requirements, software stack dependencies, inter-graphics processor communication requirements and task priority from the task submission description and task code environment to form a task requirement record; Furthermore, read the explicit declaration fields in the task submission description, such as task type, computing power precision, video memory capacity, software stack version, inter-GPU communication requirements, and task priority. When explicitly declared fields are missing or inconsistent with the task code environment, scan the data types in the task code, vendor runtime calls, and software stack paths in the container image documentation to infer computing power accuracy requirements and software stack dependencies. After performing conflict checks on the explicitly declared fields and the inferred results, a task requirement record is generated and pushed into the priority queue.

[0043] In this embodiment, the task submission description can originate from a structured interface call or a job script. The structured interface call can use structured data including task type, computing power precision, GPU memory capacity, software stack version, inter-GPU communication requirements, task priority, estimated runtime, and image identifier. The job script can use a script header declaration carrying computing power precision, number of GPUs, GPU memory requirements, software stack version, and queue priority. Explicitly declared fields are read first and written into the direct constraint section of the task requirement record. Task types can include scientific computing tasks, AI training tasks, AI inference tasks, graphics rendering tasks, and hybrid training inference tasks. Computing power precision requirements can include double precision, single precision, half precision, low-bit floating-point, or integer inference precision. GPU memory requirements can be explicitly filled in by the user or estimated from model file size, intermediate tensor estimation, and batch processing parameters. Software stack dependencies can include driver version, runtime interface, communication library, and container image environment. Inter-GPU communication requirements can include whether multi-GPU collaboration is needed, the required set of communication types, and acceptable cross-node communication boundaries. Task priorities can be divided into different queues according to production-critical tasks, offline batch processing tasks, and test and debugging tasks.

[0044] When the explicitly declared fields in the task submission description are complete and consistent with the task code environment, a task requirement record is formed. When an explicitly declared field is missing, the task code environment is read to complete the task requirement record. For computational precision requirements, the data type keywords, deep learning framework tensor types, compilation options, and numerical library calls in the code are scanned. When a double-precision data type or double-precision matrix operation call is detected, the computational precision requirement is written as a double-precision requirement; when a half-precision tensor, mixed-precision training configuration, or low-bit inference configuration is detected, the computational precision requirement is written as the corresponding low-precision requirement. For software stack dependencies, the container image documentation, environment variables, dynamic library paths, and vendor runtime calls are scanned. When a third-type graphics processor software stack path is detected, the software stack dependency is written as the third-type graphics processor native environment; when a first-type graphics processor toolchain path is detected, the software stack dependency is written as the first-type graphics processor adapted environment; when a second-type graphics processor runtime interface is detected, the software stack dependency is written as the second-type graphics processor adapted environment. For inter-graphics processor communication requirements, the multi-process parameters, communication library configuration, and distributed training parameters in the task startup command are scanned to form an inter-graphics processor communication requirement field.

[0045] In this embodiment, conflict checking is used to avoid non-executable combinations between explicitly declared fields and the task code environment. If the task submission description declares the use of a third-type graphics processor software stack while the container image specification file only contains a first-type graphics processor toolchain, the task requirement record is marked as a software stack conflict, and the software stack conflict mark is provided to the filtering, scoring, and sorting steps. If the task submission description declares a single graphics processor running while the task code environment contains multi-process set communication startup parameters, the graphics processor communication requirement is rewritten to a pending verification state, and the filtering, scoring, and sorting steps are required to generate candidate graphics processor groups only when resources that meet the video memory and interconnection conditions exist. If the video memory requirement declared by the user is lower than the video memory usage estimated based on the model file and batch processing parameters, the higher video memory requirement is used as the hard condition for video memory, and the estimation source is written into the task requirement record. After the task requirement record is formed, it is pushed into a priority queue, which can be maintained as a high-priority queue, a medium-priority queue, and a low-priority queue. Tasks in the high-priority queue are given priority to enter the filtering, scoring, and sorting steps; when high-priority tasks lack resources, medium-priority tasks and low-priority tasks can wait or participate in backfilling.

[0046] In this embodiment, an interruptibility field can also be added to the task requirement record during task parsing. The interruptibility field can be formed by the checkpoint support declaration in the task submission description, the training framework configuration, and the shared storage mount status. If the task code environment includes periodic checkpoint saving logic and the container is mounted with shared storage, the interruptibility field is set to support migration; if the task lacks checkpoint saving logic, the interruptibility field is set to not support migration.

[0047] S103, according to the task type, computing power precision requirements, software stack dependencies and hardware advantage type, remaining computing power, remaining video memory and load status recorded in the task requirement record, perform filtering, scoring and sorting to obtain candidate graphics processors or candidate virtual computing power slices. Furthermore, based on the software stack dependencies, video memory requirements, and computing power precision requirements recorded in the task requirements, graphics processors or virtual computing power slices that do not meet the hard operating conditions are eliminated.

[0048] When the score difference between the first-ranked candidate graphics processor and the second-ranked candidate graphics processor is lower than a preset hysteresis threshold, the load fluctuation amplitude and the predicted amount of queued tasks occupied by the two sets of candidate resources within a preset time window are read. For the candidate resources with a score difference lower than the preset hysteresis threshold, the waiting time weighting and fragmentation occupation deduction are re-executed to generate a corrected ranking result.

[0049] The hardware adaptation score is calculated based on the preset mapping relationship between hardware advantage type and task type; the computing power adaptation score is calculated based on the supply-demand ratio between remaining computing power and computing power demand; the video memory adaptation score is calculated based on the supply-demand ratio between remaining video memory and video memory demand; and the load distribution score is calculated based on the load status.

[0050] The hardware adaptation score, computing power adaptation score, video memory adaptation score, and load distribution score are summed according to preset weights to obtain the ranking results of candidate graphics processors or candidate virtual computing power slices.

[0051] Extract task requirement records, graphics processor allocation results, actual runtime, peak video memory usage, energy consumption, migration records, and task completion status from historical scheduling logs to form a sliding window sample.

[0052] The objective function is composed of cluster utilization, task waiting time, energy consumption and matching prediction deviation. Bayesian optimization is used to generate candidate weight parameters. The candidate weight parameters are replayed and verified in shadow mode. After the verification is passed, they are hot-loaded into the filtering, scoring and sorting processes.

[0053] In this embodiment, during hard filtering, the software stack dependencies, video memory requirements, and computing power precision requirements are read from the task requirement record. The driver version, software ecosystem compatibility score, remaining video memory, supported computing power precision set, and virtualization support type are read from the resource information database record. If the task explicitly requires a native runtime environment for a third-type graphics processor, graphics processors or virtual computing power slices whose software stack dependencies satisfy the native runtime environment of a third-type graphics processor are retained. If the task explicitly requires a first-type graphics processor toolchain, graphics processors or virtual computing power slices that meet the runtime conditions of the first-type graphics processor toolchain are retained. If the task is a general compatibility task, the software ecosystem compatibility score is used to determine whether the candidate resources meet a preset compatibility threshold. During video memory requirement filtering, the task's video memory requirements and the remaining video memory of the candidate resources are compared, and candidate resources with remaining video memory lower than the task's video memory requirements are eliminated. During computing power precision filtering, it is checked whether the computing power precision set supported by the candidate resources includes the task's computing power precision requirements. When the task's computing power precision requirements can be executed by the compatibility layer, the compatibility score and performance degradation coefficient are further read to determine whether the candidate resources are retained.

[0054] In this embodiment, the following approach is used to handle the ranking jitter and resource fragmentation issues when candidate resource scores are close: A preset hysteresis threshold can be set to 5 points. A preset time window can be set to 10 minutes. The load fluctuation amplitude can be calculated based on the difference between the highest and lowest load rates within the time window. The predicted amount of queued tasks can be calculated from the estimated memory requirements, estimated computing power requirements, and estimated start times of the queued tasks on the candidate resources. When the score difference between the first-ranked candidate resource and the second-ranked candidate resource is lower than the preset hysteresis threshold, a correction score is calculated for each group of candidate resources: ; in, Indicates a corrected fraction. This indicates the original comprehensive adaptation score. Indicates the weight of waiting time. This represents the waiting time coefficient. Indicates the weight of fragment usage. Indicates the fragmentation occupancy coefficient. Indicates the volatility penalty weight. This indicates the magnitude of load fluctuation.

[0055] The waiting time coefficient is obtained by normalizing the waiting time after a task enters the priority queue, and the fragmentation occupancy coefficient is obtained by the proportion of small fragments formed by the remaining video memory and computing power after the candidate resources are allocated. If the allocation of a candidate resource leaves video memory fragments and computing power fragments that are insufficient to satisfy any scheduled task, the fragmentation occupancy is reduced. The corrected sorting result is sent back to the step of generating the candidate graphics processor or candidate virtual computing power slice sorting result as the final sorting basis.

[0056] In this embodiment, hardware adaptation score, computing power adaptation score, memory adaptation score, and load distribution score are calculated. The hardware adaptation score is obtained by mapping the task type to the hardware advantage type. For example, when a double-precision scientific computing task is matched with a graphics processor with double-precision scientific computing advantages, the hardware adaptation score is 100; when the same task is assigned to a graphics processor that supports multi-precision capabilities, a lower score can be taken based on its multi-precision capabilities; when the same task is assigned to a graphics processor with software ecosystem compatibility, an intermediate score can be taken based on software ecosystem compatibility. When a training-inference integrated task is matched with a graphics processor with training-inference integrated adaptation advantages, a higher hardware adaptation score is taken. When a large model task with high memory bandwidth is matched with a graphics processor with high memory bandwidth advantages, a higher hardware adaptation score is taken. For lightweight inference tasks, load status and virtualization slicing capabilities can be prioritized.

[0057] The computing power matching score is determined based on the supply-demand ratio between remaining computing power and the task's computing power requirement. When the remaining computing power is less than the task's computing power requirement, the computing power matching score is 0; when the remaining computing power is greater than or equal to the task's computing power requirement, the computing power matching score is 100 multiplied by the ratio of the task's computing power requirement to the remaining computing power. This calculation method ensures that candidate resources that just meet the requirement receive a higher computing power matching score, while candidate resources that retain a large amount of unused computing power after allocation receive a lower computing power matching score.

[0058] The memory matching score is determined based on the supply-demand ratio between the remaining memory and the memory requirement. When the remaining memory is less than the task's memory requirement, the memory matching score is 0; when the remaining memory is greater than or equal to the task's memory requirement, the memory matching score is 100 multiplied by the ratio of the task's memory requirement to the remaining memory.

[0059] The load distribution score is the difference between 100 and the current load rate, so that candidate resources with lower load rates get higher load distribution scores.

[0060] The hardware adaptation score, computing power adaptation score, memory adaptation score, and load distribution score are multiplied by their respective hardware weights, computing power weights, memory weights, and load weights, and then summed to obtain the comprehensive adaptation score. For example, the hardware weight can be 0.4, the computing power weight can be 0.2, the memory weight can be 0.2, and the load weight can be 0.2. If the task is a communication-sensitive multi-GPU training task, interconnect affinity can also be included as part of the hardware adaptation score, or topology constraint verification can be performed after candidate resource sorting. After the comprehensive adaptation score is formed, candidate resources are sorted according to task priority queues. For high-priority tasks, candidate resources that meet the hard operating conditions and have the highest comprehensive adaptation score are selected first; for medium-priority and low-priority tasks, backfill scheduling can be performed by combining waiting time coefficients and fragmentation deduction results.

[0061] In this embodiment, the step for updating the preset weights is as follows: each scheduling decision is written to the historical scheduling log, which includes task requirement records, candidate resource score details, graphics processor allocation results, actual runtime, peak memory usage, energy consumption, migration records, communication latency, task completion status, and queuing time. A sliding window sample with a length of 7 days or 5000 tasks is maintained. The objective function is formed by a weighted combination of cluster utilization, average waiting time, unit task energy consumption, and matching prediction bias, where the cluster utilization term has a positive weight, and the average waiting time, unit task energy consumption, and matching prediction bias terms have negative weights. This objective function value is used as the optimization objective for Bayesian optimization.

[0062] Starting with the previous round of weight parameters, a proxy model is constructed and candidate weight parameters are sampled. The candidate weight parameters are first replayed in shadow mode using historical tasks from the past hour, during which the simulated target value for using the candidate weight parameters is calculated. If the simulated target value increases by more than 5% relative to the target value corresponding to the current weight parameter, the candidate weight parameter is hot-loaded into the scheduling core; otherwise, the current weight parameter continues to be used. Through the above processing, the candidate ranking results formed by the filtering, scoring, and sorting steps can be updated with parameters according to changes in task load, hardware status, and resource fragmentation.

[0063] In this embodiment, a sudden increase in task waiting time and a decrease in cluster utilization can also be used as triggering conditions for parameter updates. For example, when the average task waiting time increases by 50% relative to the previous statistical window, or when cluster utilization is below 40% for two consecutive hours, the most recent window sample is extracted from the historical scheduling log, and an incremental Bayesian optimization is performed. The candidate weight parameters obtained from the incremental Bayesian optimization are still published after being verified by shadow mode replay. If the number of historical scheduling log samples is insufficient, the default weights are retained, and filtering, scoring, and sorting are performed only based on the current resource information database records.

[0064] S104, based on task priority, the sorting results of candidate graphics processors or candidate virtual computing power slices, and the communication requirements between graphics processors, generate single graphics processor allocation, virtual computing power slice allocation, or multi-graphics processor aggregate allocation results. Furthermore, when the computing power and video memory requirements in the task requirement record are lower than the preset slicing conditions, the hardware-level virtual instance corresponding to the graphics processor that supports hardware-level virtual instances will be converted into a fixed-granularity hard slice object, and the software quota slice corresponding to the graphics processor that supports software quota slicing will be converted into an adjustable-granularity soft slice object.

[0065] The hardware preset granularity is verified for hard slice objects and the allocation request is rolled back when the granularity does not match. Runtime configuration instructions are generated for soft slice objects according to the computing power quota and the video memory limit. The slice type, video memory quota, computing power quota and isolation level are written into the virtual computing power slice allocation result.

[0066] Transform high-speed interconnect links, high-speed bus links, and remote direct memory access links into a weighted topology graph that includes link bandwidth, link latency, and topological location.

[0067] Based on the communication requirements between graphics processors, candidate graphics processor groups within the same high-speed interconnect domain are selected in the weighted topology graph, and a common network layer path is selected for candidate graphics processor groups with different architectures.

[0068] The aggregated communication operations are mapped to aggregated communication primitives that can be executed by the vendor's communication library, and ring, tree, or hierarchical communication algorithms are selected according to the data volume and topology location to generate aggregated allocation results for multiple graphics processors.

[0069] In this embodiment, scheduling allocation results are generated based on the ranking results and task priorities formed by the filtering, scoring, and sorting steps. For tasks with high computing power and memory requirements that require full device capabilities, a single graphics processor allocation result is generated. For example, when a double-precision scientific computing task requires 150 TFLOPS of computing power and 32 GB of memory, it can be allocated to a single graphics processor with double-precision computing advantages for execution, according to the ranking results. For small tasks with low computing power and memory requirements, virtual computing power slice allocation results are generated. Preset slice conditions may include a task's memory requirement being less than 50% of the total memory of a single graphics processor, a task's computing power requirement being less than 50% of the available computing power of a single graphics processor, and the task having no strict requirements for strong isolation or allowing the task to share resources. For training or inference tasks that require multiple graphics processors to complete together, a multi-graphics processor aggregate allocation result is generated.

[0070] In the specific implementation of virtual computing power slicing allocation, graphics processors supporting hardware-level virtual instances are mapped to fixed-granularity hard slice objects. For example, a single graphics processor supporting hardware-level virtual instances can be mapped to four 1 / 4-granularity hard slice objects, each with a fixed memory quota, a fixed computing power quota, and a strong isolation level. When a graphics processor supporting software quota slicing uses software quota slicing, the heterogeneous virtualization adaptation layer treats a single physical graphics processor as an elastic resource pool and generates soft slice objects according to the computing power and memory requirements recorded in the task requirement record. Soft slice objects can have a memory quota granularity of 1GB and a computing power quota granularity of 5% to 10%. A unified slice creation instruction is sent to the heterogeneous virtualization adaptation layer, which includes the graphics processor's unique identifier, memory quota, computing power quota, and isolation level requirements. The heterogeneous virtualization adaptation layer performs hardware-preset granularity verification on hard slice objects. When the requested granularity is 25% and the hardware supports 1 / 4 slices, a hard slice configuration command is generated. When the requested granularity is 30% and the hardware only supports 1 / 4 or 1 / 2 slices, the allocation request is rolled back and a granularity mismatch result is returned. The heterogeneous virtualization adaptation layer generates runtime configuration instructions for soft slice objects. These runtime configuration instructions can include container visible device configuration, memory limit configuration, compute time slice quota configuration, and process resource limit configuration. After configuration, the slice type, memory quota, compute power quota, and isolation level are written to the virtual compute power slice allocation results, and the resource information database is updated synchronously.

[0071] In this embodiment, isolation levels can be set for hard slice objects and soft slice objects respectively. Hard slice objects are marked with a strong isolation level, suitable for tasks that are latency-sensitive, have high security isolation requirements, or have high task interruption costs. Soft slice objects are marked with a soft isolation level, suitable for lightweight inference, testing and debugging, and small tasks that can tolerate resource sharing. If multiple soft slice objects run simultaneously on the same physical graphics processor, the load status and memory bandwidth contention status are read, and slice reassembly can be triggered during the run adjustment step. If the task requirement record explicitly requires strong isolation, the resource allocation step only selects the allocation result of hard slice objects or a single graphics processor.

[0072] In the specific implementation of multi-GPU aggregation allocation, the topology information of high-speed interconnect links, high-speed bus links, and remote direct memory access links is read from the resource information database. GPUs are treated as graph nodes, and interconnections between GPUs or between nodes are treated as graph edges. Link bandwidth, link latency, and topological location are written into the graph edge weights. The weighted topology graph can distinguish links within the same high-speed interconnect domain, the same high-speed bus domain, the same server node, the same rack, and across racks into different topology levels. For tasks with high inter-GPU communication requirements, candidate GPU groups are prioritized within the same high-speed interconnect domain. When resources within the same high-speed interconnect domain are insufficient, candidate GPU groups within the same node or rack are selected. When collaboration between different architectures is unavoidable, a common network layer path, such as a remote direct memory access link or a high-bandwidth Ethernet link, is chosen, and communication groups of different architectures are written into the multi-GPU aggregation allocation results.

[0073] In this embodiment, aggregated communication operations can include aggregated contracts, aggregated collection, many-to-many exchange, broadcasting, and point-to-point transmission. A unified aggregated communication interface is exposed to the upper-layer training or inference framework, and vendor communication library adaptation information is written into the allocation results. For the first type of graphics processors that support software quota sharding, the unified aggregated communication interface can call its communication library to execute aggregated communication primitives; for the second type of graphics processors that support software quota sharding, the unified aggregated communication interface can call its communication library or deep learning adapted communication library to execute aggregated communication primitives; for graphics processors that support hardware-level virtual instances, the unified aggregated communication interface can call its distributed communication library to execute aggregated communication primitives. For different architecture candidate graphics processor groups, the unified aggregated communication interface maps the aggregated communication operations of different vendors to a unified semantics and completes the synchronization between graphics processors or between nodes through a common network layer path. The communication algorithm is selected based on the data volume and topology location. When the data volume is large and the topology is circularly connected, a ring algorithm can be selected; when the node hierarchy is obvious, a hierarchical algorithm can be selected; when a small number of graphics processors are aggregated, a tree algorithm can be selected. The communication algorithm selection results, communication group members, link paths, and vendor communication library configurations are combined to form the multi-graphics processor aggregation allocation result.

[0074] In this embodiment, a parallel strategy selection can also be incorporated into the multi-GPU aggregation allocation. For large model training tasks, tensor parallelism, pipelined parallelism, or expert parallelism is selected based on the model size, memory requirements, and inter-GPU communication requirements recorded in the task requirements log. For expert parallel tasks, high-speed interconnect domains that support many-to-many exchanges are prioritized, and corresponding set communication primitives are configured in the unified set communication interface. For inference tasks, the same session or the same batch of requests can be placed in the same high-speed interconnect domain as much as possible based on request routing information and cache hit status to reduce data exchange between GPUs. The results of the above parallel strategy selection are written into the communication configuration section of the multi-GPU aggregation allocation result.

[0075] S105 reads the runtime status and verifies the preset exception conditions during task execution. For running tasks that meet the preset exception conditions, it performs checkpoint saving, target resource filtering, container reconstruction and breakpoint recovery processing, and outputs the updated resource information database record and task scheduling results.

[0076] Furthermore, based on the graphics processor type, the temperature threshold, health status threshold, video memory usage threshold, computing load threshold, communication latency threshold, and priority preemption conditions are read.

[0077] When the target resource's graphics processor type is different from the type of the graphics processor resource where the abnormal running task is located, the target architecture container image is filtered according to the software stack dependencies in the task requirement record and the runtime version information in the checkpoint file.

[0078] When the target architecture container image and checkpoint file cannot meet the preset compatibility conditions, the target resources are removed and resources of the same type or the same software stack are re-selected to obtain the migration execution resources.

[0079] Threshold comparisons are performed on the continuously sampled temperature values, device heartbeat, video memory page swapping status, computing unit utilization, and graphics processor link status. When any comparison result meets the preset abnormal conditions, an abnormal signal is generated. When the comparison result is within the normal range, the sampling result is written back to the resource information database and the current task allocation result is continued.

[0080] When a new task requests insufficient resources and multiple allocated slices on the same physical graphics processor are under low load, a reorganization scheme is calculated based on the unused video memory quota, unused computing power quota, and isolation level of each slice; the computing power quota and video memory limit are adjusted for soft slice objects through the runtime quota interface; the slice combination is changed only when the slice is released or a new task is assigned for hard slice objects; and the adjusted slice occupancy relationship is written to the resource information database record.

[0081] In this embodiment, the resource information database records and runtime status are continuously read during task execution. The runtime status can be periodically written by the resource acquisition adapter in the steps described above for reading static hardware parameters, virtualization capabilities, interconnect capabilities, and runtime status, or it can be directly sent to the scheduling core by the unified monitoring module. Temperature thresholds can be configured differently according to the graphics processor type. For example, the warning threshold for the first type of graphics processor can be set to 85℃, and the abnormal threshold can be set to 90℃; the abnormal threshold for the second type of graphics processor can be set to 85℃; and the abnormal threshold for the third type of graphics processor can be set to 88℃. Health status thresholds can include device offline, graphics processor disconnection, uncorrectable error count growth, and driver heartbeat loss. The video memory occupancy threshold can be set to 95% and combined with the video memory page swapping frequency to determine the risk of video memory overflow. Computational load thresholds can include a sudden drop in load rate to 0% and a sustained period exceeding a preset time, or a load rate reaching 100% and causing a scheduling heartbeat timeout. Communication latency thresholds can include inter-graphics processor link latency exceeding 5ms or a packet loss rate exceeding 1%. Priority preemption conditions can include a high-priority task entering the queue and a low-priority task currently occupying the target resource.

[0082] During anomaly detection, the unified monitoring module continuously samples temperature, device heartbeat, memory page swapping status, compute unit utilization, and inter-GPU link status. Temperature anomalies can be detected by checking if three consecutive samples exceed the corresponding GPU anomaly threshold; if the sampling period is 3 seconds, then three consecutive samples correspond to a 9-second detection window. Device heartbeat anomalies can be caused by driver interface access failure, inaccessible device files, or lost heartbeat packets. Memory overflow risk can be caused by a persistent memory occupancy rate exceeding 95% and a memory page swapping frequency exceeding a preset threshold. Computational load anomalies can be caused by a sudden drop in compute unit utilization, process freeze, or scheduling heartbeat timeout. Communication anomalies can be caused by a combination of inter-GPU link latency and packet loss rate. When any comparison result meets the preset anomaly conditions, an anomaly signal is generated and written to the scheduling status of the running task. If all comparison results are within the normal range, the sampling results are written back to the resource information database, and the current task allocation results continue to be executed.

[0083] Upon the generation of an anomaly signal, checkpoint saving, target resource filtering, container reconstruction, and breakpoint recovery are executed. For training tasks that support checkpoint saving, a preset control signal is sent to the running task container or the application management interface is called. The application saves model parameters, optimizer status, iteration steps, and necessary runtime context to shared storage. For inference tasks, request queue position, cache identifier, and processed batch identifier can be saved. After the checkpoint file is saved, target resources are filtered from the resource information database records. Target resources are preferentially selected from graphics processors or virtual computing power slices of the same type, model, and software stack that meet idle conditions. Idle conditions may include computing unit utilization below 5% for 10 consecutive seconds, memory usage below 10%, no active computing processes, and no resources reserved by high-priority tasks. After target resource filtering is completed, the source container is destroyed or paused, the source resources are released, and the resource information database records are updated. A new container instance is created on the node where the target resource is located, shared storage is mounted, the checkpoint file is loaded, the target graphics processor driver interface is called to initialize the memory context, and the training or inference process is resumed from the checkpoint position.

[0084] The architecture compatibility handling during cross-GPU type migration is as follows: When the GPU type of the target resource differs from the GPU type of the GPU resource where the abnormal running task resides, the software stack dependencies, container image identifier, runtime version requirements, and framework version, parameter precision format, and communication library version in the task requirement record are read. If the source resource is a Type I GPU and the target resource is a Type III GPU, a target architecture container image with a Type III GPU software stack is selected; if the source resource is a Type II GPU and the target resource is a Type I GPU, a target architecture container image with a Type I GPU toolchain is selected. Preset compatibility conditions may include that the model parameter format of the checkpoint file can be read by the target framework, the task code environment supports the target runtime interface, the target container image contains the task dependency library, and the target communication library version meets the inter-GPU communication requirements. If the target architecture container image and the checkpoint file cannot meet the preset compatibility conditions, the target resource is removed, and resources of the same type, the same software stack, or resources with compatibility layer support are re-selected. If all candidate resources cannot meet the preset compatibility conditions, the task status is set to "awaiting migration resource," and the checkpoint file and source task release information are retained, waiting for subsequent resources to become available.

[0085] The fragment reorganization process is as follows: When a new task requests insufficient resources and multiple allocated fragments on the same physical graphics processor are under low load, the unused memory quota, unused computing power quota, isolation level, task priority, and adjustable status of each fragment are read. For soft fragment objects, the computing power quota and memory limit can be adjusted online through the runtime quota interface. For example, when there are 3 soft fragment objects on a graphics processor that supports software quota fragmentation, and the load rate of 2 of the soft fragment objects is consistently below 5%, while a new task requires an additional 20% computing power quota and 8GB of video memory, the unused quota of the low-load soft fragment objects is calculated and reclaimed, and the computing power quota and memory limit of each soft fragment object are reset through the runtime interface. For hard fragment objects, if a graphics processor that supports hardware-level virtual instances has already created fixed-granularity hard fragment objects, the hard fragment topology is not changed during the running task; when a hard fragment object is released or a new task is assigned, the subsequent fragment combination is changed. After fragment reorganization is completed, the adjusted fragment occupancy relationship, resource availability status, and task binding relationship are written to the resource information database record.

[0086] In this embodiment, runtime logs can also be output at key nodes in the above steps. The runtime logs may include task identifier, unique GPU identifier, resource database record version, task requirement record summary, candidate resource score, allocation result, slice quota, communication group members, anomaly signals, checkpoint file path, migration target resource, and recovery status. The runtime logs can be written to a time-series database or object storage and a query interface can be provided. The recording trigger conditions for the runtime logs may include task submission, candidate sorting completion, allocation result generation, anomaly signal generation, checkpoint saving completion, container reconstruction completion, and task termination. By reading the runtime logs, it is possible to verify whether the scheduling results are formed by the task requirement record and resource database record, and also to verify whether task migration and slice reassembly are performed according to the above steps.

[0087] The following description, using specific examples, elaborates on this embodiment.

[0088] In an exemplary implementation scenario, the heterogeneous graphics processor cluster includes a first type of graphics processor, a second type of graphics processor, and a third type of graphics processor. The resource acquisition adapter reads the static hardware parameters and runtime status of the three types of graphics processors in the steps described above, which involve reading static hardware parameters, virtualization capabilities, interconnect capabilities, and runtime status. In the exemplary resource information database record, the first type of graphics processor is a first-type graphics processor with a peak double-precision computing power of 180 TFLOPS, a total video memory of 32GB, supports 4 hard slice objects, has a current load rate of 20%, 4GB of used video memory, and a temperature of 68℃; the second type of graphics processor has a total video memory of 64GB, a software ecosystem compatibility score of 96, a current load rate of 35%, and 20GB of used video memory; the third type of graphics processor has a total video memory of 80GB, a video memory bandwidth of 2.4TB / s, supports high-speed interconnect, has a current load rate of 30%, and 10GB of used video memory. The resource acquisition adapter converts the fields output from different interfaces into resource index fields, static constraint fields, runtime constraint fields, and matching score fields, and writes them into the resource information database records.

[0089] In the step of creating task requirement records, three tasks are received. Task A is a high-priority scientific computing task, explicitly declaring a double-precision computing requirement of 150 TFLOPS, a GPU memory requirement of 32 GB, and a software stack dependency of a Type I graphics processor toolchain. Task B is a medium-priority training and inference integrated task, requiring 60 GB of GPU memory, supporting multi-GPU collaboration, and its code environment includes a general parallel computing interface and distributed training startup parameters. Task C is a low-priority lightweight inference task, requiring 5 TFLOPS of computing power, 8 GB of GPU memory, and without specifying a fixed software stack. Tasks A, B, and C are each created as task requirement records and pushed into their respective priority queues.

[0090] In the steps of filtering, scoring, and ranking, task A is filtered, scored, and ranked. Task A requires double-precision computing power and a Type I graphics processor toolchain. The Type I graphics processor meets the software stack dependency and computing power precision requirements. Type II and Type III graphics processors receive lower hardware compatibility scores due to differences in hardware advantage types and software stack adaptability. For example, Task A's hardware compatibility score for the Type I graphics processor is 100, the computing power compatibility score is calculated by multiplying 100 by the ratio of the task's computing power requirement to the remaining computing power, approximately 83.3, and the memory compatibility score is 0 due to insufficient remaining memory. If another Type I graphics processor has 32GB of remaining memory and 180 TFLOPS of remaining computing power, then its memory compatibility score is 100, and its load distribution score is 80. Summing these scores with weights of 0.4, 0.2, 0.2, and 0.2, this other Type I graphics processor receives a higher overall compatibility score and becomes the first candidate resource for ranking. For Task B, the second and third types of graphics processors are grouped into candidate graphics processor groups, and a comprehensive adaptation score is calculated based on software ecosystem compatibility, high memory bandwidth, and interconnect domain status. For Task C, hard slice objects of the first type of graphics processor and soft slice objects of the third type of graphics processor are read, and the candidate virtual computing power slice ranking results are calculated.

[0091] In the step of generating the scheduling and allocation results, Task A is allocated to a Type 1 graphics processor system that meets the requirements for remaining video memory and double-precision computing power. Task B is allocated to a multi-graphics processor aggregate resource composed of Type 2 and Type 3 graphics processors. The same high-speed interconnect domain and common network layer path are selected in the weighted topology graph, and the aggregated communication operation is mapped to the vendor's communication library. Task C, meeting the preset slicing conditions, sends a slice creation command to the heterogeneous virtualization adaptation layer. For Type 1 graphics processor hard slice objects, a hard slice object with a 25% computing power quota and an 8GB video memory quota is requested. After granularity verification passes, a virtual computing power slice allocation result is generated. If Task C selects a Type 3 graphics processor soft slice object, a runtime configuration command with a 20% computing power quota and an 8GB video memory limit is generated, and the slice type, video memory quota, computing power quota, and isolation level are written to the resource information database.

[0092] During the adjustment process, the runtime status is continuously read. If the temperature of the first type of graphics processor exceeds 90°C for three consecutive samples during the execution of Task A, an abnormal signal is generated. When Task A supports checkpoint saving, a control signal is sent to the Task A container, and Task A saves model parameters or intermediate calculation states, iteration steps, and runtime context to shared storage. Another first type of graphics processor that meets the idle condition is selected, the source container is destroyed, the overheated graphics processor resources are released, a new container is created on the target node, shared storage is mounted, the checkpoint file is loaded, and Task A is restored. If the target resource needs to be migrated across graphics processor types, the runtime version information in the checkpoint file and the software stack dependencies in the task requirement record are read, and the target architecture container image is selected; if the image and checkpoint file cannot meet the compatibility conditions, the target resource is removed and resources of the same type are re-selected. After the task migration is completed, the updated resource information database record and task scheduling results are output.

[0093] For the purpose of simplicity, the method steps disclosed in the above embodiments are described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0094] like Figure 2 As shown, the present invention also provides a heterogeneous graphics processor resource scheduling system, comprising: The resource record generation module 201 is configured to read static hardware parameters, virtualization capabilities, interconnection capabilities, and runtime status from the driver interface, runtime interface, and monitoring interface of the heterogeneous graphics processor, respectively, and convert them into resource information database records according to a unified field dictionary and a unified unit of measurement. The task parsing module 202 is configured to extract task type, computing power and precision requirements, video memory requirements, software stack dependencies, inter-graphics processor communication requirements and task priority from the task submission description and task code environment, and form a task requirement record. The candidate sorting module 203 is configured to perform filtering, scoring and sorting according to the task type, computing power precision requirements, software stack dependencies and hardware advantage type, remaining computing power, remaining video memory and load status recorded in the task requirement record to obtain candidate graphics processors or candidate virtual computing power slices. The resource allocation module 204 is configured to generate single graphics processor allocation, virtual computing power slice allocation, or multi-graphics processor aggregate allocation results according to task priority, the sorting results of candidate graphics processors or candidate virtual computing power slices, and the communication requirements between graphics processors. The operation adjustment module 205 is configured to read the runtime status and verify preset abnormal conditions during task execution. For running tasks that meet the preset abnormal conditions, it performs checkpoint saving, target resource filtering, container reconstruction and breakpoint recovery processing, and outputs updated resource information database records and task scheduling results.

[0095] It is worth noting that although only some basic functional modules are disclosed in the embodiments of this invention, it does not mean that the composition of this system is limited to the above-mentioned basic functional modules. On the contrary, based on the above-mentioned basic functional modules, those skilled in the art can arbitrarily add one or more functional modules in combination with existing technology to form an infinite number of embodiments or technical solutions. That is to say, this system is open rather than closed. The fact that this embodiment only discloses a few basic functional modules should not be considered as the scope of protection of this invention being limited to the disclosed basic functional modules. At the same time, for the convenience of description, the above devices are described separately according to their functions as various units and modules. Of course, in implementing this invention, the functions of each unit and module can be implemented in one or more software and / or hardware.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for scheduling resources of heterogeneous graphics processors, characterized in that, include: Static hardware parameters, virtualization capabilities, interconnect capabilities, and runtime status are read from the driver interface, runtime interface, and monitoring interface of the heterogeneous graphics processor, respectively, and converted into resource information database records according to a unified field dictionary and a unified unit of measurement. Extract task type, computing power and precision requirements, video memory requirements, software stack dependencies, inter-GPU communication requirements, and task priority from the task submission description and task code environment to form a task requirement record; Based on the task type, computing power precision requirements, software stack dependencies, and hardware advantage type, remaining computing power, remaining video memory, and load status recorded in the task requirement record, filter, score, and sort to obtain candidate graphics processors or candidate virtual computing power slices. Based on task priority, the sorting results of candidate graphics processors or candidate virtual computing power slices, and the communication requirements between graphics processors, generate single graphics processor allocation, virtual computing power slice allocation, or multi-graphics processor aggregate allocation results. During task execution, the runtime state is read and preset exception conditions are verified. For running tasks that meet the preset exception conditions, checkpoint saving, target resource filtering, container reconstruction and breakpoint recovery are performed, and the updated resource information database record and task scheduling results are output.

2. The heterogeneous graphics processor resource scheduling method according to claim 1, characterized in that, Converted into resource information database records according to a unified field dictionary and unified unit of measurement, further including: Write the unique identifier of the graphics processor, the graphics processor type, and the node address into the resource index field; Write the total video memory, double-precision peak computing power, virtualization support type, and interconnect type into the static constraint field; Write the load rate, used video memory, temperature, power consumption, and health status into the running constraint field; Write the hardware advantage type and software ecosystem compatibility score into the matching score field; The resource index field, static constraint field, runtime constraint field, and matching score field are encapsulated into a resource information database record.

3. The heterogeneous graphics processor resource scheduling method according to claim 1, characterized in that, Create a task requirement record, which further includes: Read the explicit declaration fields of task type, computing power precision, video memory capacity, software stack version, inter-GPU communication requirements, and task priority from the task submission description; When explicitly declared fields are missing or inconsistent with the task code environment, scan the data types in the task code, vendor runtime calls, and software stack paths in the container image documentation to infer computing power accuracy requirements and software stack dependencies. After performing conflict checks on the explicitly declared fields and the inferred results, a task requirement record is generated and pushed into the priority queue.

4. The heterogeneous graphics processor resource scheduling method according to claim 1, characterized in that, Perform filtering, scoring, and sorting, further including: Based on the software stack dependencies, video memory requirements, and computing power precision requirements recorded in the task requirements, graphics processors or virtual computing power slices that do not meet the hard operating conditions are eliminated. The hardware adaptation score is calculated based on the preset mapping relationship between hardware advantage type and task type; the computing power adaptation score is calculated based on the supply-demand ratio between remaining computing power and computing power demand; the video memory adaptation score is calculated based on the supply-demand ratio between remaining video memory and video memory demand; and the load distribution score is calculated based on the load status. The hardware adaptation score, computing power adaptation score, video memory adaptation score, and load distribution score are summed according to preset weights to obtain the ranking results of candidate graphics processors or candidate virtual computing power slices.

5. The heterogeneous graphics processor resource scheduling method according to claim 1, characterized in that, The generation of single graphics processor allocation, virtual compute slice allocation, or multi-graphics processor aggregate allocation results further includes: When the computing power and video memory requirements in the task requirement record are lower than the preset slicing conditions, the hardware-level virtual instance corresponding to the graphics processor that supports hardware-level virtual instance will be converted into a fixed-granularity hard slice object, and the software quota slice corresponding to the graphics processor that supports software quota slicing will be converted into an adjustable-granularity soft slice object. The hardware preset granularity is verified for hard slice objects and the allocation request is rolled back when the granularity does not match. Runtime configuration instructions are generated for soft slice objects according to the computing power quota and the video memory limit. Write the slice type, memory quota, computing power quota, and isolation level into the virtual computing power slice allocation result.

6. The heterogeneous graphics processor resource scheduling method according to claim 1, characterized in that, The generated multi-graphics processor aggregation allocation result further includes: Transform high-speed interconnect links, high-speed bus links, and remote direct memory access links into a weighted topology map that includes link bandwidth, link latency, and topological location; Based on the communication requirements between graphics processors, candidate graphics processor groups within the same high-speed interconnect domain are selected in the weighted topology graph, and a common network layer path is selected for candidate graphics processor groups with different architectures. The aggregated communication operations are mapped to aggregated communication primitives that can be executed by the vendor's communication library, and ring, tree, or hierarchical communication algorithms are selected according to the data volume and topology location to generate aggregated allocation results for multiple graphics processors.

7. The heterogeneous graphics processor resource scheduling method according to claim 1, characterized in that, Validating preset exception conditions further includes: Read the temperature threshold, health status threshold, video memory usage threshold, computing load threshold, communication latency threshold, and priority preemption conditions according to the graphics processor type; Threshold comparisons are performed on the continuously sampled temperature values, device heartbeat, video memory page swapping status, computing unit utilization, and graphics processor link status, respectively. An anomaly signal is generated when any comparison result meets a preset anomaly condition; When the comparison result does not meet the preset abnormal conditions, the sampling result is written back to the resource information database and the current task allocation result is continued.

8. A heterogeneous graphics processor resource scheduling method according to claim 5, characterized in that, During task execution, the runtime state is read, further including: Perform fragmentation and reassembly processing on the virtual computing power slice: When a new task requests insufficient resources and multiple allocated slices on the same physical graphics processor are under low load, a reorganization scheme is calculated based on the unused video memory quota, unused computing power quota, and isolation level of each slice. The computing power quota and video memory limit can be adjusted for soft slice objects through the runtime quota interface; For hard slice objects, the slice composition is changed only when the slice is released or a new task is assigned; Write the adjusted slice occupancy relationship into the resource information database record.

9. A heterogeneous graphics processor resource scheduling method according to claim 4, characterized in that, The preset weights are formed through policy parameter updates, including: Extract task requirement records, graphics processor allocation results, actual runtime, peak video memory usage, energy consumption, migration records, and task completion status from historical scheduling logs to form a sliding window sample; The objective function is composed of cluster utilization, task waiting time, energy consumption and matching prediction bias, and candidate weight parameters are generated by Bayesian optimization. The candidate weight parameters are replayed and verified in shadow mode. Once the verification is successful, they are hot-loaded into the filtering, scoring, and sorting processes.

10. A heterogeneous graphics processor resource scheduling system, characterized in that, include: The resource record generation module is configured to read static hardware parameters, virtualization capabilities, interconnection capabilities, and runtime status from the driver interface, runtime interface, and monitoring interface of the heterogeneous graphics processor, and convert them into resource information database records according to a unified field dictionary and a unified unit of measurement. The task parsing module is configured to extract task type, computing power and precision requirements, video memory requirements, software stack dependencies, inter-GPU communication requirements, and task priority from the task submission description and task code environment, forming a task requirement record. The candidate sorting module is configured to perform filtering, scoring and sorting according to the task type, computing power precision requirements, software stack dependencies and hardware advantage type, remaining computing power, remaining video memory and load status recorded in the task requirement record to obtain candidate graphics processors or candidate virtual computing power slices. The resource allocation module is configured to generate single graphics processor allocation, virtual computing power slice allocation, or multi-graphics processor aggregate allocation results based on task priority, the sorting results of candidate graphics processors or candidate virtual computing power slices, and the communication requirements between graphics processors. The operation adjustment module is configured to read the runtime status and verify preset abnormal conditions during task execution. For running tasks that meet the preset abnormal conditions, checkpoint saving, target resource filtering, container reconstruction and breakpoint recovery are performed, and the updated resource information database record and task scheduling results are output.

Images