Heterogeneous computing scheduling method and device and heterogeneous computing slice resource scheduling system

Abstract

The application provides a heterogeneous computing scheduling method and device and a heterogeneous computing slice resource scheduling system. The method comprises the following steps: when the state of a node accessing a container cloud platform or equipment changes, the basic capability and running environment information of a heterogeneous device are acquired; the heterogeneous device is divided into a plurality of slice resource objects; candidate positions meeting the requirements of device type, architecture compatibility, isolation and tenant policy constraints are screened; the continuous available slice proportion of the candidate positions is calculated, and the candidate positions are determined to have carrying capacity in the case that the continuous available slice proportion of the candidate positions is greater than the task demand proportion; the candidate positions with carrying capacity that do not meet the minimum resource vector, operator compatibility requirement, tenant quota, priority protection and interference limitation are eliminated, and the preferred candidate positions are obtained; the slice resource objects are allocated in the preferred candidate positions, and the to-be-processed tasks are started, thereby solving the problem that the service quality fluctuation is caused by simply pursuing resource utilization.

Description

Technical Field

[0001] This application relates to the field of heterogeneous computing power scheduling technology, and more specifically, to a heterogeneous computing power scheduling method, apparatus, computer program product, and heterogeneous computing power slice resource scheduling system. Background Technology

[0002] In recent years, with the rapid development of large model training, inference services, retrieval enhancement, agent workflows, and data-intensive computing, the demand for heterogeneous acceleration resources such as GPUs and DCUs in container cloud platforms has continued to grow. Unlike traditional general-purpose CPU resources, heterogeneous acceleration resources are characterized by expensive equipment, significant differences, strong runtime dependencies, complex resource sharing, and high isolation requirements. Their scheduling and management capabilities have become a key foundational capability for cloud platforms to support AI services.

[0003] In traditional container orchestration systems, accelerators such as GPUs and DCUs are typically allocated as entire cards, entire devices, or coarse-grained quotas. While this approach is simple to implement, in real-world business scenarios, many tasks do not require exclusive access to the entire device, but only need a portion of the video memory, some computing power, or specific operator processing capabilities. If the entire card allocation method is still used, it can easily lead to idle resources and wasted costs. To address this, the industry has gradually introduced technologies such as GPU virtualization, time-slice reuse, multi-instance partitioning, video memory splitting, and device sharing to support multiple tasks using a single physical device in parallel.

[0004] At the same time, as AI workloads become increasingly diverse, heterogeneous computing power management on container cloud platforms is no longer simply a matter of "splitting and allocating," but requires consideration of the following issues simultaneously:

[0005] First, the long-term co-location of multiple small tasks will cause the accumulation of resource fragments, resulting in subsequent larger tasks being unable to be scheduled even when there is "sufficient global free space".

[0006] Second, different tenants and tasks with different priorities have different requirements for resource fairness, isolation and latency protection. Simply pursuing resource utilization will cause fluctuations in service quality.

[0007] Third, there are differences between heterogeneous devices such as GPUs and DCUs in terms of drivers, frameworks, operators, and runtime. Unified pooling management cannot stop at the resource registration level, but needs to explicitly consider compatibility constraints during scheduling.

[0008] Fourth, online services, inference workloads, training tasks, and batch jobs have different tolerance levels for reordering and migration. If there is a lack of online reordering paths that match the workload type, resource reordering will be difficult to implement in a production environment.

[0009] Therefore, the key to heterogeneous computing power scheduling at this stage has evolved from "whether it can be partitioned and shared" to "whether it can make unified decisions based on the slice status that the scheduler can perceive, and complete online optimization in an implementable and rollback-able manner when fragments accumulate." Summary of the Invention

[0010] The main objective of this application is to provide a method, apparatus, computer program product, and heterogeneous computing power scheduling system for scheduling heterogeneous computing power slices, so as to at least solve the problem that simply pursuing resource utilization in the prior art can cause fluctuations in service quality.

[0011] To achieve the above objectives, according to one aspect of this application, a method for scheduling heterogeneous computing power is provided, comprising: obtaining basic capabilities and operating environment information of heterogeneous devices when a node accesses a container cloud platform or the device status changes; dividing the heterogeneous devices into multiple slice resource objects; filtering candidate locations that meet device type, architecture compatibility requirements, isolation requirements, and tenant policy constraints, wherein the candidate locations are a set of slice resource objects; calculating the proportion of continuously available slices at the candidate locations, and determining that the candidate locations have carrying capacity if the proportion of continuously available slices at the candidate locations is greater than the task demand proportion, wherein the task demand proportion is the ratio of the number of slices required for the task to be processed to the total number of slices on the device; eliminating the candidate locations with carrying capacity that do not meet the minimum resource vector, operator compatibility requirements, tenant quota, priority protection, and interference restrictions, thereby obtaining preferred candidate locations; allocating the slice resource objects at the preferred candidate locations and starting the task to be processed.

[0012] Optionally, allocating the slice resource object at the preferred candidate location and starting the pending task includes: scoring each of the preferred candidate locations based on fragmentation waste, node diffusion, energy consumption, and tenant penalty calculations, and determining the preferred candidate location with the highest score as the optimal candidate location; allocating the slice resource object at the optimal candidate location and starting the pending task.

[0013] Optionally, the method further includes: if the preferred candidate position does not exist, determining whether to enter the online reordering process based on the proportion of continuously available slices, waiting latency, high-priority task status, and estimated benefits; if the online reordering process is entered, performing slice reordering on the heterogeneous device.

[0014] Optionally, determining whether to enter the online reordering process based on the proportion of continuously available slices, waiting latency, high-priority task status, and estimated benefits includes: determining the task status as a high-priority task if there are pending tasks with a priority higher than a predetermined priority; initiating the online evaluation process if the task status is high-priority; initiating the online evaluation process if the proportion of continuously available slices is less than a corresponding threshold and the waiting latency is greater than a corresponding threshold; performing a benefit analysis on the candidate reordering scheme to obtain the benefit of the candidate reordering scheme, the benefit including the freed-up continuously available slice space, the improvement in waiting latency for tasks with a priority higher than the predetermined priority, and the degree of optimization of the score; and entering the online reordering process if the benefit is greater than a preset benefit threshold.

[0015] Optionally, when entering the online reordering process, the heterogeneous devices are sliced ​​and reordered, including: when the task to be processed is an online service task, a model inference task, or a streaming task, the slice reordering is completed using a dual-instance parallel and grayscale switching method; when the task to be processed is a training task, an offline inference task, or a batch processing task, the slice reordering is completed using a checkpoint or savepoint recovery method.

[0016] Optionally, when entering the online reordering process, after reordering the slices of the heterogeneous devices, the method further includes: if the task to be processed is an online service task, a model inference task, or a streaming task, verifying whether the latency, error rate, throughput, and latency tail ripple of the task to be processed meet the corresponding requirements, and obtaining a first verification result; if the task to be processed is a training task, an offline inference task, or a batch processing task, verifying whether the recovery success rate, task progress retention, and execution integrity of the task to be processed meet the corresponding requirements, and obtaining a second verification result; if the first verification result or the second verification result is a successful verification, confirming the reordering is successful and updating the slice resource status; if the first verification result or the second verification result is a failed verification, executing rollback logic to restore the slice resource status to the state before reordering.

[0017] Optionally, after successful rearrangement, the method further includes: re-filtering the preferred candidate positions, allocating the slice resource object at the preferred candidate position and starting the pending task; and releasing the slice resource object corresponding to the preferred candidate position when the pending task ends or stops.

[0018] According to another aspect of this application, a heterogeneous computing power scheduling device is provided, comprising: an acquisition unit, configured to acquire basic capabilities and operating environment information of heterogeneous devices when a node accesses a container cloud platform or the device status changes; a partitioning unit, configured to partition the heterogeneous devices into multiple slice resource objects; a filtering unit, configured to filter candidate locations that meet device type, architecture compatibility requirements, isolation requirements, and tenant policy constraints, wherein the candidate locations are a set of slice resource objects; a calculation unit, configured to calculate the proportion of continuously available slices at the candidate locations, and determine that the candidate locations have carrying capacity if the proportion of continuously available slices at the candidate locations is greater than the task demand proportion, wherein the task demand proportion is the ratio of the number of slices required for the task to be processed to the total number of slices on the device; a rejection unit, configured to reject the candidate locations with carrying capacity that do not meet the minimum resource vector, operator compatibility requirements, tenant quota, priority protection, and interference restrictions, thereby obtaining preferred candidate locations; and a scheduling unit, configured to allocate the slice resource objects at the preferred candidate locations and start the task to be processed.

[0019] According to another aspect of this application, a computer program product is provided, comprising a computer program that, when executed by a processor, implements any of the methods described.

[0020] According to another aspect of this application, a heterogeneous computing power slicing resource scheduling system is provided for performing any of the methods described.

[0021] By applying the technical solution of this application, the above-mentioned heterogeneous computing power scheduling method divides heterogeneous computing power into slice resource objects visible to the scheduler. In the candidate selection and continuous available slice determination, compatibility, isolation, priority and service quality constraints are introduced to ensure that the binning behavior always occurs within a controllable boundary. This achieves constrained and efficient reuse rather than unlimited resource exploitation. Therefore, it is more conducive to balancing resource efficiency and business stability in the production environment and solves the problem that simply pursuing resource utilization will cause service quality fluctuations. Attached Figure Description

[0022] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0023] Figure 1 A flowchart illustrating a heterogeneous computing power scheduling method according to an embodiment of this application is shown.

[0024] Figure 2 An online defrag state machine diagram according to an embodiment of this application is shown;

[0025] Figure 3 The diagram illustrates a timing diagram of a dual-instance grayscale switching process for an online service / inference task according to an embodiment of this application.

[0026] Figure 4 A timing diagram illustrating checkpoint recovery for a training / batch processing task according to an embodiment of this application is shown.

[0027] Figure 5 A flowchart illustrating another method for scheduling heterogeneous computing power according to an embodiment of this application is shown.

[0028] Figure 6 A structural block diagram of a heterogeneous computing power scheduling device according to an embodiment of this application is shown;

[0029] Figure 7 A general technical architecture diagram of a heterogeneous computing power slicing resource scheduling system according to an embodiment of this application is shown;

[0030] Figure 8 A diagram showing the relationship between a slice resource object and a data model according to an embodiment of this application is illustrated. Detailed Implementation

[0031] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0032] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] For ease of description, the following explains some of the nouns or terms used in the embodiments of this application:

[0035] A slice resource object is the smallest unit of scheduling resources that can be independently allocated, isolated, released, and rearranged on a single physical heterogeneous accelerator device. A slice resource object includes at least compute and storage quotas, and may further include attributes such as bandwidth level, isolation method, and compatibility constraints.

[0036] Co-location deployment refers to a placement method that, under the premise of meeting compatibility constraints, isolation requirements, QoS requirements, and multi-tenant policy constraints, deploys multiple tasks on different slice resource objects of the same physical device to reduce the number of new devices that need to be started and improve resource utilization.

[0037] Continuous available slices: This is not limited to contiguousness at the physical address level, but refers to a set of slice resource objects that meet composable bearer conditions on the same physical device. The composable bearer conditions preferably include at least: being in the same isolation domain, meeting the minimum resource vector requirement of the task, meeting compatibility matrix constraints, and meeting the policy and QoS restrictions of the task.

[0038] Continuous availability slice ratio: For a candidate device d, let MaxContig(d) be the maximum size of the continuous availability slice set that can meet the target task requirements, and Total(d) be the total number of slices for the device. Then, the continuous availability slice ratio of this device can preferably be defined as: R_contiguous(d) = MaxContig(d) / Total(d). At the cluster level, the overall continuous availability slice status can be characterized by the minimum value, average value, weighted average value, or representative value of key candidate devices, depending on the scheduling strategy.

[0039] As described in the background section, simply pursuing resource utilization in the prior art can lead to fluctuations in service quality. To address this issue, embodiments of this application provide a method, apparatus, computer program product, and heterogeneous computing power slice resource scheduling system for scheduling heterogeneous computing power.

[0040] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0041] This embodiment provides a method for scheduling heterogeneous computing power. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0042] Figure 1This is a flowchart of a heterogeneous computing power scheduling method according to an embodiment of this application. For example... Figure 1 As shown, the method includes the following steps:

[0043] Step S101: When a node connects to the container cloud platform or the device status changes, obtain the basic capabilities and operating environment information of the heterogeneous device.

[0044] Step S102: Divide the above-mentioned heterogeneous devices into multiple slice resource objects;

[0045] Step S103: Filter candidate locations that meet the requirements of device type, architecture compatibility, isolation requirements and tenant policy constraints. The candidate locations are the set of the slice resource objects.

[0046] Step S104: Calculate the proportion of consecutive available slices at the above candidate locations, and if the proportion of consecutive available slices at the above candidate locations is greater than the task requirement proportion, determine that the above candidate locations have carrying capacity. The task requirement proportion is the ratio of the number of slices required for the task to be processed to the total number of slices in the device.

[0047] Step S105: Eliminate the above-mentioned candidate locations with carrying capacity that do not meet the minimum resource vector, operator compatibility requirements, tenant quota, priority protection and interference restrictions, and obtain the preferred candidate locations;

[0048] Step S106: Allocate the slice resource object at the preferred candidate position and start the pending task.

[0049] In this embodiment, the above-mentioned heterogeneous computing power scheduling method divides heterogeneous computing power into slice resource objects visible to the scheduler. It introduces compatibility, isolation, priority, and service quality constraints in candidate screening and continuous available slice determination, so that the binning behavior always occurs within a controllable boundary. It achieves constrained and efficient reuse rather than unlimited resource exploitation. Therefore, it is more conducive to balancing resource efficiency and business stability in a production environment, and solves the problem that simply pursuing resource utilization will cause service quality fluctuations.

[0050] To improve the overall scheduling quality and the schedulability of subsequent tasks, in one optional implementation, step S106 includes:

[0051] Step S1061: Calculate and score each of the above preferred candidate locations based on fragmentation waste, node diffusion, energy consumption, and tenant penalty, and determine the preferred candidate location with the highest score as the optimal candidate location.

[0052] Step S1062: Allocate the slice resource object at the optimal candidate position and start the pending task.

[0053] In the above implementation, the scoring formula is: Score = α × FragmentWaste + β × NodeSpread + γ × EnergyCost + δ × TenantPenalty, where FragmentWaste represents the fragmentation waste item, used to measure the degree of fragmentation added to the target device after task placement; NodeSpread represents the node diffusion item, used to measure the degree to which the same task or job is deployed across multiple nodes; EnergyCost represents the energy consumption item, used to reflect the energy consumption pressure of the device or the estimated energy consumption per unit job corresponding to the placement scheme; TenantPenalty represents the tenant penalty item, used to reflect the placement cost under conditions of tenant over-provisioning, low priority, or policy restrictions; α, β, γ, and δ are scoring weight coefficients, whose values ​​can be configured according to the scenario, and preferably their sum is 1. The scheduling control module first normalizes each scoring item, and then performs a weighted summation to prevent any one item from dominating the overall result due to excessively large dimensions. For candidate positions with similar or identical scores, a secondary sorting can be performed in order of less fragmentation waste, lower energy consumption, or less impact on high-priority tenants. By using the multi-objective scoring method described above, the placement decision is no longer solely based on "whether it can be placed," but also considers the impact of placement behavior on subsequent fragment states, node diffusion, energy consumption levels, and tenant fairness, thereby improving the overall scheduling quality and the schedulability of subsequent tasks.

[0054] To determine whether to initiate the online defragmentation process, in one optional implementation, the method further includes:

[0055] Step S201: If there are no preferred candidate positions as mentioned above, determine whether to enter the online reordering process based on the proportion of continuously available slices, waiting latency, high-priority task status, and estimated benefits.

[0056] Step S202: In the case of entering the above-mentioned online rearrangement process, the heterogeneous devices are sliced ​​and rearranged.

[0057] In the above implementation, if no candidate location that meets the direct placement conditions is found after candidate screening and multi-target scoring of the task to be scheduled, the task is not suspended indefinitely. Instead, it is further determined whether to enter the online defragmentation process. Online defragmentation is preferably triggered when any one or a combination of the following conditions is met: 1. The proportion of continuously available slices of the candidate device is lower than a preset threshold; 2. The waiting latency of the task to be scheduled exceeds a preset latency limit; 3. The currently arriving task is a high-priority task, and the main reason for its inability to be scheduled is slice fragmentation rather than global resource exhaustion; 4. According to the evaluation of monitoring data, it is expected that slice rearrangement can significantly reduce the comprehensive score result or restore the schedulability of the target task; 5. The system is in a typical fragmented state of "sufficient total quantity but difficult to combine".

[0058] To avoid system disturbances caused by excessively frequent online defragmentation, in one optional implementation, step S201 includes:

[0059] Step S2011: If there are pending tasks with a priority higher than the predetermined priority, determine that the task is in the high-priority task state.

[0060] Step S2012: In the case of the above-mentioned high-priority task status, initiate the online evaluation process;

[0061] Step S2013: If the proportion of continuously available slices is less than the corresponding threshold and the waiting delay is greater than the corresponding threshold, the online evaluation process is initiated.

[0062] Step S2014: Perform a benefit analysis on the candidate rearrangement scheme to obtain the benefits of the candidate rearrangement scheme. The benefits include the freed-up continuous available slice space, the improvement of the waiting latency for tasks with a priority higher than the predetermined priority, and the degree of optimization of the score.

[0063] Step S2015: If the above-mentioned revenue is greater than the preset revenue threshold, proceed with the above-mentioned online rearrangement process.

[0064] In the above implementation, the triggering logic can adopt a combined judgment method of "continuously available slice ratio + waiting latency + high-priority task status". That is, when R_contiguous is lower than the threshold and the maximum waiting time exceeds the threshold, or when there are high-priority pending tasks, online defragmentation evaluation is initiated. To avoid system disturbances caused by excessively frequent online defragmentation, the following protection conditions can also be configured: limit the maximum number of reorderings for the same task; set a reordering cooling window; allow entry into the execution phase only when the estimated benefit is greater than the preset benefit threshold; restrict reordering operations during critical business windows or periods of high system pressure.

[0065] In order to perform slice-level rearrangement for different tasks, in one optional implementation, step S202 above includes:

[0066] Step S2021: When the above-mentioned task to be processed is an online service task, a model inference task, or a streaming task, the slice rearrangement is completed by using dual-instance parallelism and grayscale streaming.

[0067] Step S2022: If the task to be processed is a training task, an offline inference task, or a batch processing task, the slice rearrangement is completed using a checkpoint or savepoint recovery method.

[0068] In the above embodiments, such as Figure 2As shown, online defragmentation is preferably performed using a state machine approach of "trigger—evaluation—execution—verification—confirmation or rollback". In the trigger phase, when direct placement fails and the online defragmentation trigger conditions are met, the scheduling control module enters the Defrag trigger phase and selects several potential reordering objects as candidates from the currently running tasks. In the evaluation phase, the scheduling control module combines monitoring feedback, task attributes, and reordering strategy objects to perform cost and benefit analysis on multiple candidate reordering schemes. The preferred costs include switching costs, disturbance costs to existing tasks, service quality fluctuation risks, and reordering time costs; the preferred benefits include the freed-up continuous available slice space, improvement in waiting latency for high-priority tasks, and optimization of the overall scoring results. The execution phase only begins when the benefits meet a preset threshold. In the execution phase, for tasks with high latency and continuity requirements, such as online services, model inference, and streaming processing, a dual-instance parallel and gray-scale switching approach is preferred for slice reordering. Figure 3 As shown, the preferred process is as follows: pre-allocate the required slice resource objects on the target node or target device; start the new replica and complete initialization or warm-up; gradually redirect traffic from the old replica to the new replica according to preset weights; monitor metrics such as latency, error rate, and throughput within the observation window; if the metrics continuously meet the preset SLO, release the slice resource objects occupied by the old replica; if the metrics do not meet the threshold, perform a rollback operation and restore the original state. This method can complete slice-level reordering with minimal business interruption, making it suitable for online services and inference workloads. For training tasks, offline inference tasks, or batch processing tasks, checkpoint or savepoint recovery is preferred for slice reordering. Figure 4 As shown, the preferred process is as follows: generate a consistency checkpoint at the source location; pre-allocate the required slice resource objects at the target location; resume task execution based on the checkpoint; if the recovery is successful, release the slice at the original location; if the recovery fails, roll back to the original location to continue execution or restore to the waiting state. This approach does not rely on vendor-independent general process hot migration capabilities and is more suitable for engineering implementation in heterogeneous computing environments.

[0069] To ensure the rearrangement effect, in an optional implementation, after performing slice rearrangement on the heterogeneous devices in the above-mentioned online rearrangement process, the method further includes:

[0070] Step S301: If the task to be processed is an online service task, a model inference task, or a streaming task, verify whether the latency, error rate, throughput, and latency tail fluctuation of the task to be processed meet the corresponding requirements, and obtain the first verification result.

[0071] Step S302: If the task to be processed is a training task, an offline inference task, or a batch processing task, verify whether the recovery success rate, task progress retention, and execution integrity of the task to be processed meet the corresponding requirements, and obtain the second verification result.

[0072] Step S303: If the first verification result or the second verification result is verified as passed, confirm that the rearrangement is successful and update the slice resource status.

[0073] Step S304: If the first verification result or the second verification result is that the verification fails, execute the rollback logic to restore the slice resource state to the state before the rearrangement.

[0074] In the above embodiments, such as Figure 2 As shown, during the verification and confirmation phase, after the reordering is completed, the system verifies the corresponding metrics within a preset observation window. For online service tasks, latency, error rate, throughput, and latency tail fluctuations are preferred for verification; for training or batch processing tasks, recovery success rate, task progress retention, and execution integrity are preferred for verification. If the verification passes, the reordering is confirmed as successful and the slice resource status is updated; if the verification fails, the rollback logic is executed to restore the state before the reordering.

[0075] To achieve complete closed-loop scheduling, after successful rearrangement, the above method also includes:

[0076] Step S401: Re-filter the above preferred candidate positions, allocate the above slice resource objects to the above preferred candidate positions, and start the above pending task;

[0077] Step S402: When the above-mentioned pending task ends or stops, release the slice resource object corresponding to the preferred candidate position.

[0078] In the above implementation, the resource usage status is continuously reported during task execution; when the task ends or stops, the corresponding slice resource object is released and the global resource status of the platform is updated, thus achieving complete closed-loop scheduling.

[0079] To enable those skilled in the art to better understand the technical solution of this application, the implementation process of the heterogeneous computing power scheduling method of this application will be described in detail below with reference to specific embodiments.

[0080] This embodiment relates to a specific method for scheduling heterogeneous computing power, such as... Figure 5 As shown, it includes the following steps:

[0081] Step S1: Discover and register heterogeneous accelerator resources.

[0082] When a node accesses a platform or device status changes, it detects heterogeneous devices such as GPUs and DCUs and collects their basic capabilities and operating environment information.

[0083] Step S2: Generate slice resource objects.

[0084] The physical device is divided into multiple slice resource objects according to the preset slicing strategy, and a global slice resource view that can be directly perceived by the scheduler is established.

[0085] Step S3: Receive the task to be scheduled and parse the task scheduling specification.

[0086] Obtain information such as task resource requirements, priority, tenant attributes, compatibility requirements, workload type, and QoS requirements.

[0087] Step S4: Perform candidate screening.

[0088] Filter candidate locations from the global slice resource view that meet the constraints of device type, compatibility matrix, isolation requirements, and tenant policy.

[0089] Step S5: Calculate the continuous available slice capacity.

[0090] For each candidate location, calculate the set of consecutive available slices and the proportion of consecutive available slices to assess its actual carrying capacity for the target mission.

[0091] Step S6: Perform hard constraint verification.

[0092] Candidate locations that do not meet the minimum resource vector, compatibility, tenant quota, priority protection, and interference restrictions are eliminated.

[0093] Step S7: Perform multi-objective scoring.

[0094] Candidate locations that pass the hard constraint verification are scored based on factors such as fragmentation waste, node diffusion, energy consumption, and tenant penalties, and the location with the best comprehensive score is selected.

[0095] Step S8: Determine if there is a solution that can be placed directly.

[0096] If a candidate location that meets the placement criteria exists, a slice resource object is allocated at the target location and the task is started; otherwise, proceed to step S9.

[0097] Step S9: Determine if online defragmentation has been triggered.

[0098] The decision to enter the online defragment process is based on a combination of factors including the proportion of continuously available slices, waiting latency, the status of high-priority tasks, and estimated revenue.

[0099] Step S10: Perform online defragmentation.

[0100] If an online Defrag is triggered, select the appropriate reordering object and, depending on the task type, use either dual-instance parallelism and grayscale switching, or checkpoint recovery to complete the slice reordering.

[0101] Step S11: Verify and update the slice resource status.

[0102] Verify the results of online Defrag using an observation window; if verification passes, update the global slice resource view; if it fails, roll back and restore the original state.

[0103] Step S12: Retry candidate scoring and task placement.

[0104] After a successful online defragment, candidate filtering, scoring, and placement are re-executed based on the updated slice resource status to complete the final scheduling of the original task.

[0105] Step S13: Runtime monitoring and resource release.

[0106] The resource usage status is continuously reported during task execution; when the task ends or stops, the corresponding slice resource object is released and the global resource status of the platform is updated.

[0107] This application also provides a heterogeneous computing power scheduling device. It should be noted that the heterogeneous computing power scheduling device of this application can be used to execute the heterogeneous computing power scheduling method provided in this application. This device is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0108] The following describes the heterogeneous computing power scheduling device provided in the embodiments of this application.

[0109] Figure 6 This is a schematic diagram of a heterogeneous computing power scheduling device according to an embodiment of this application. Figure 6 As shown, the device includes:

[0110] The acquisition unit 10 is used to acquire the basic capabilities and operating environment information of heterogeneous devices when a node connects to the container cloud platform or when the device status changes.

[0111] The partitioning unit 20 is used to divide the aforementioned heterogeneous devices into multiple slice resource objects;

[0112] The filtering unit 30 is used to filter candidate locations that meet the requirements of device type, architecture compatibility, isolation requirements and tenant policy constraints. The candidate locations are a set of the slice resource objects.

[0113] The calculation unit 40 is used to calculate the proportion of consecutive available slices at the candidate location, and if the proportion of consecutive available slices at the candidate location is greater than the task requirement ratio, it determines that the candidate location has the carrying capacity. The task requirement ratio is the ratio of the number of slices required for the task to be processed to the total number of slices in the device.

[0114] Elimination unit 50 is used to eliminate the above-mentioned candidate positions with carrying capacity that do not meet the minimum resource vector, operator compatibility requirements, tenant quota, priority protection and interference restrictions, so as to obtain the preferred candidate positions;

[0115] The scheduling unit 60 is used to allocate the slice resource object at the preferred candidate position and start the pending task.

[0116] In this embodiment, the heterogeneous computing power scheduling device divides heterogeneous computing power into slice resource objects visible to the scheduler. It introduces compatibility, isolation, priority, and service quality constraints in candidate screening and continuous available slice determination. This ensures that the binning behavior always occurs within controllable boundaries, achieving constrained and efficient reuse rather than unlimited resource exploitation. Therefore, it is more conducive to balancing resource efficiency and business stability in a production environment, solving the problem that simply pursuing resource utilization will cause service quality fluctuations.

[0117] To improve the overall scheduling quality and the schedulability of subsequent tasks, in one optional implementation, the scheduling unit includes:

[0118] The calculation module is used to score each of the above preferred candidate locations based on fragmentation waste, node diffusion, energy consumption and tenant penalty, and to determine the preferred candidate location with the highest score as the optimal candidate location.

[0119] The allocation module is used to allocate the slice resource objects at the optimal candidate positions and start the pending tasks.

[0120] In the above implementation, the scoring formula is: Score = α × FragmentWaste + β × NodeSpread + γ × EnergyCost + δ × TenantPenalty, where FragmentWaste represents the fragmentation waste item, used to measure the degree of fragmentation added to the target device after task placement; NodeSpread represents the node diffusion item, used to measure the degree to which the same task or job is deployed across multiple nodes; EnergyCost represents the energy consumption item, used to reflect the energy consumption pressure of the device or the estimated energy consumption per unit job corresponding to the placement scheme; TenantPenalty represents the tenant penalty item, used to reflect the placement cost under conditions of tenant over-provisioning, low priority, or policy restrictions; α, β, γ, and δ are scoring weight coefficients, whose values ​​can be configured according to the scenario, and preferably their sum is 1. The scheduling control module first normalizes each scoring item, and then performs a weighted summation to prevent any one item from dominating the overall result due to excessively large dimensions. For candidate positions with similar or identical scores, a secondary sorting can be performed in order of less fragmentation waste, lower energy consumption, or less impact on high-priority tenants. By using the multi-objective scoring method described above, the placement decision is no longer solely based on "whether it can be placed," but also considers the impact of placement behavior on subsequent fragment states, node diffusion, energy consumption levels, and tenant fairness, thereby improving the overall scheduling quality and the schedulability of subsequent tasks.

[0121] To determine whether to initiate the online defragmentation process, in one optional implementation, the above-mentioned device further includes:

[0122] The judgment unit is used to determine whether to enter the online reordering process when there are no preferred candidate positions as described above, based on the proportion of continuously available slices, waiting latency, high-priority task status and estimated benefits.

[0123] The rearrangement unit is used to slice and rearrange the heterogeneous devices when the above-mentioned online rearrangement process is entered.

[0124] In the above implementation, if no candidate location that meets the direct placement conditions is found after candidate screening and multi-target scoring of the task to be scheduled, the task is not suspended indefinitely. Instead, it is further determined whether to enter the online defragmentation process. Online defragmentation is preferably triggered when any one or a combination of the following conditions is met: 1. The proportion of continuously available slices of the candidate device is lower than a preset threshold; 2. The waiting latency of the task to be scheduled exceeds a preset latency limit; 3. The currently arriving task is a high-priority task, and the main reason for its inability to be scheduled is slice fragmentation rather than global resource exhaustion; 4. According to the evaluation of monitoring data, it is expected that slice rearrangement can significantly reduce the comprehensive score result or restore the schedulability of the target task; 5. The system is in a typical fragmented state of "sufficient total quantity but difficult to combine".

[0125] To avoid system disturbances caused by excessively frequent online defragmentation, in one optional implementation, the aforementioned judgment unit includes:

[0126] The determination module is used to determine the state of the high-priority task when there are pending tasks with a priority higher than the predetermined priority.

[0127] The first judgment module is used to initiate the online evaluation process when the task is in the aforementioned high-priority state.

[0128] The second judgment module is used to initiate the online evaluation process when the proportion of continuously available slices is less than the corresponding threshold and the waiting delay is greater than the corresponding threshold.

[0129] The analysis module is used to perform a benefit analysis on the candidate rearrangement schemes to obtain the benefits of the candidate rearrangement schemes. The benefits include the freed-up contiguous available slice space, the improvement in waiting latency for tasks with higher priority than the predetermined priority, and the degree of optimization of the above score.

[0130] The third judgment module is used to enter the online rearrangement process when the above-mentioned revenue is greater than the preset revenue threshold.

[0131] In the above implementation, the triggering logic can adopt a combined judgment method of "continuously available slice ratio + waiting latency + high-priority task status". That is, when R_contiguous is lower than the threshold and the maximum waiting time exceeds the threshold, or when there are high-priority pending tasks, online defragmentation evaluation is initiated. To avoid system disturbances caused by excessively frequent online defragmentation, the following protection conditions can also be configured: limit the maximum number of reorderings for the same task; set a reordering cooling window; allow entry into the execution phase only when the estimated benefit is greater than the preset benefit threshold; restrict reordering operations during critical business windows or periods of high system pressure.

[0132] To perform slice-level rearrangement for different tasks, in one optional implementation, the rearrangement unit includes:

[0133] The first reordering module is used to complete the slice reordering in the case that the above-mentioned tasks to be processed are online service tasks, model inference tasks or streaming tasks, using dual-instance parallelism and gray-scale streaming.

[0134] The second rearrangement module is used to complete the slice rearrangement by using checkpoint or savepoint recovery methods when the above-mentioned task to be processed is a training task, an offline inference task, or a batch processing task.

[0135] In the above embodiments, such as Figure 2As shown, online defragmentation is preferably performed using a state machine approach of "trigger—evaluation—execution—verification—confirmation or rollback". In the trigger phase, when direct placement fails and the online defragmentation trigger conditions are met, the scheduling control module enters the Defrag trigger phase and selects several potential reordering objects as candidates from the currently running tasks. In the evaluation phase, the scheduling control module combines monitoring feedback, task attributes, and reordering strategy objects to perform cost and benefit analysis on multiple candidate reordering schemes. The preferred costs include switching costs, disturbance costs to existing tasks, service quality fluctuation risks, and reordering time costs; the preferred benefits include the freed-up continuous available slice space, improvement in waiting latency for high-priority tasks, and optimization of the overall scoring results. The execution phase only begins when the benefits meet a preset threshold. In the execution phase, for tasks with high latency and continuity requirements, such as online services, model inference, and streaming processing, a dual-instance parallel and gray-scale switching approach is preferred for slice reordering. Figure 3 As shown, the preferred process is as follows: pre-allocate the required slice resource objects on the target node or target device; start the new replica and complete initialization or warm-up; gradually redirect traffic from the old replica to the new replica according to preset weights; monitor metrics such as latency, error rate, and throughput within the observation window; if the metrics continuously meet the preset SLO, release the slice resource objects occupied by the old replica; if the metrics do not meet the threshold, perform a rollback operation and restore the original state. This method can complete slice-level reordering with minimal business interruption, making it suitable for online services and inference workloads. For training tasks, offline inference tasks, or batch processing tasks, checkpoint or savepoint recovery is preferred for slice reordering. Figure 4 As shown, the preferred process is as follows: generate a consistency checkpoint at the source location; pre-allocate the required slice resource objects at the target location; resume task execution based on the checkpoint; if the recovery is successful, release the slice at the original location; if the recovery fails, roll back to the original location to continue execution or restore to the waiting state. This approach does not rely on vendor-independent general process hot migration capabilities and is more suitable for engineering implementation in heterogeneous computing environments.

[0136] To ensure the rearrangement effect, in one optional embodiment, the above-mentioned device further includes:

[0137] The first verification unit is used to verify, when entering the above-mentioned online reordering process, whether the latency, error rate, throughput and latency tail fluctuation of the above-mentioned task to be processed meet the corresponding requirements after the above-mentioned heterogeneous device is sliced ​​and reordered, and when the above-mentioned task to be processed is an online service task, a model inference task or a streaming task, and obtain the first verification result.

[0138] The second verification unit is used to verify whether the recovery success rate, task progress retention and execution integrity of the task to be processed meet the corresponding requirements when the task to be processed is a training task, an offline inference task or a batch processing task, and to obtain the second verification result.

[0139] The update unit is used to confirm the successful rearrangement and update the slice resource status if the first verification result or the second verification result is verified as passed.

[0140] The rollback unit is used to execute rollback logic if the first verification result or the second verification result is that the verification fails, so that the slice resource state is restored to the state before the rearrangement.

[0141] In the above embodiments, such as Figure 2 As shown, during the verification and confirmation phase, after the reordering is completed, the system verifies the corresponding metrics within a preset observation window. For online service tasks, latency, error rate, throughput, and latency tail fluctuations are preferred for verification; for training or batch processing tasks, recovery success rate, task progress retention, and execution integrity are preferred for verification. If the verification passes, the reordering is confirmed as successful and the slice resource status is updated; if the verification fails, the rollback logic is executed to restore the state before the reordering.

[0142] To achieve complete closed-loop scheduling, after successful rearrangement, the above-mentioned device also includes:

[0143] The repeating unit is used to re-filter the above preferred candidate positions, allocate the above slice resource objects at the above preferred candidate positions, and start the above pending tasks.

[0144] The release unit is used to release the slice resource object corresponding to the preferred candidate position when the above-mentioned pending task ends or stops.

[0145] In the above implementation, the resource usage status is continuously reported during task execution; when the task ends or stops, the corresponding slice resource object is released and the global resource status of the platform is updated, thus achieving complete closed-loop scheduling.

[0146] This invention provides a heterogeneous computing power slicing resource scheduling system, such as... Figure 7 As shown, it includes a heterogeneous resource discovery and slice management module, a scheduling control module, a multi-tenant and quality of service strategy module, an execution and reordering module, and a monitoring and optimization feedback module.

[0147] Heterogeneous resource discovery and slice management module:

[0148] This module is responsible for discovering, registering, acquiring capabilities, and managing slices of heterogeneous accelerator devices such as GPUs and DCUs in the cluster nodes. When a node connects to the container cloud platform or the device status changes, this module obtains basic information such as the type, model, device identifier, computing power, video memory or on-chip storage capacity, bandwidth capacity, driver environment, and runtime environment of the corresponding device, and divides the physical device into slices according to a preset slicing strategy or a dynamic slicing strategy.

[0149] The results of slice partitioning are not only used by the underlying driver or runtime, but are further abstracted into slice resource objects that the scheduler can directly perceive. These slice resource objects include at least the following information: node identifier, device type, device number, compute quota, storage quota, isolation policy, bandwidth level, compatibility constraints, and occupancy status. The heterogeneous resource discovery and slice management module is also responsible for maintaining the lifecycle status of slice resource objects, including slice creation, occupancy, release, reclamation, rebinding after rearrangement, and status updates.

[0150] Scheduling and control module:

[0151] The scheduling control module is the core decision-making module of this invention. It is used to receive tasks to be scheduled, parse the task's resource requirements, priority, tenant attributes, workload type, compatibility requirements and service quality requirements, and perform candidate screening, hard constraint verification, multi-objective scoring and placement decision in combination with the real-time status of slice resource objects.

[0152] Within a scheduling cycle, the scheduling control module first filters candidate locations from the global slice resource view that meet the basic resource vector and compatibility constraints of the task. Then, it determines whether the candidate locations have actual carrying capacity based on the status of continuously available slices. For candidate locations that pass the screening, a comprehensive evaluation is performed based on a multi-objective scoring function to select the optimal placement scheme. When no directly placeable scheme exists, the scheduling control module does not terminate immediately but further determines whether the online defragmentation trigger condition is met. If it is met, the execution and reordering module is invoked to complete online defragmentation, and the candidate screening and scoring process is re-executed after defragmentation.

[0153] Multi-tenancy and Quality of Service Strategy Module:

[0154] This module is used to maintain tenant quotas, priorities, isolation requirements, and QoS constraints, and incorporates them as hard constraints or penalties in the scheduling process. Specifically, this module can configure maximum slice quotas, allowed device types, preemption permissions, preemption ranges, priority ranges, and mixed deployment limits for different tenants, and provide policy constraints when the scheduling control module performs candidate screening and scoring.

[0155] Furthermore, this module can restrict slice combinations, device combinations, or tenant combinations that may pose significant interference risks based on historical task behavior and current load status, thereby avoiding performance fluctuations caused by excessive consolidation. This module can improve resource utilization while controlling fairness risks and service quality degradation risks in a shared environment.

[0156] Execution and Reordering Module:

[0157] The execution and reordering module is responsible for performing task instance startup, switching, resumption, stopping, and reordering operations on the node side according to the placement or reordering instructions from the scheduling control module. For normal placement scenarios, this module binds the corresponding slice resource object to the target node and starts the task instance according to the scheduling result; for online defragmentation scenarios, this module performs differentiated slice reordering based on the selected reordering object.

[0158] The execution and reordering module preferably selects different reordering paths based on the task workload type: for online services, model inference, or streaming tasks, a dual-instance parallel approach and canary-scale streaming are preferred for slice reordering; for training, offline inference, or batch processing tasks, a checkpoint or savepoint recovery approach is preferred for slice reordering. This differentiated path design avoids imposing a uniform migration method on all task types, thereby improving the engineering feasibility and business acceptability of online reordering.

[0159] Monitoring and optimization feedback module:

[0160] This module is responsible for continuously collecting resource usage, slice occupancy status, latency, service quality metrics, and energy consumption information from both the node and task sides, and feeding this data back to the scheduling control module and the multi-tenant and service quality strategy module. On one hand, this feedback data can be used to calculate fragmentation waste, energy consumption, and tenant penalties in the candidate scoring; on the other hand, it can also be used to determine whether the triggering conditions for online defragmentation are met and to verify the effect after reordering.

[0161] Through this monitoring and optimization feedback module, the scheduling method of the present invention can continuously adjust the scoring, triggering and reordering strategies according to the operating status, thereby forming a closed-loop scheduling system with continuous optimization capabilities.

[0162] To achieve the above technical solution, the following resource objects and data models are preferably introduced at the implementation level: Figure 8As shown, a slice resource object represents the smallest unit of scheduling resources that can be independently allocated, isolated, reclaimed, and rearranged on a single physical heterogeneous accelerator device. This object preferably includes the following attributes: node identifier, device type, device model, device instance number, compute quota, video memory or on-chip storage quota, bandwidth level, isolation policy, isolation domain, current status, occupant identifier, and compatibility constraint information.

[0163] Slice resource objects can be generated by native hardware partitioning capabilities, or by virtualization technology, time-slice multiplexing, logical quotas, or a combination of device plugins and runtime quotas. Regardless of the underlying slice implementation method, as long as a slice resource object that can be directly perceived and manipulated by the scheduler is ultimately formed, it falls within the protection scope of this invention.

[0164] DeviceSliceClass:

[0165] Device slice categories are used to describe the capability boundaries and constraint characteristics of different types of heterogeneous device slices. They preferably include attributes such as: device type, manufacturer identifier, model identifier, computing resource category, storage resource scale, bandwidth category, energy consumption characteristics, isolation method, and compatibility matrix.

[0166] The compatibility matrix is ​​preferably used to characterize driver version, runtime environment, framework type, operator set, or other execution dependencies to determine whether a task can run safely on the corresponding device slice. This invention does not require that any GPU and any NPU be freely interchangeable; rather, it uses the compatibility matrix to filter out candidate heterogeneous device types that meet the task requirements.

[0167] TaskSchedulingSpec:

[0168] Task scheduling specifications describe the resource and policy requirements of tasks to be scheduled during the scheduling phase. Preferred attributes include: the number of slices or minimum resource vector required for the task, requirements or preferences for device type, task priority, tenant identifier, workload type, QoS or SLO requirements, whether reordering is allowed, whether preemption is allowed, and task isolation requirements.

[0169] Through task scheduling specifications, the scheduling control module can map and match task requirements with slice resource objects and the capability boundaries of DeviceSliceClass, thereby providing a basis for candidate selection and scoring decisions.

[0170] RepackingPolicy object:

[0171] The reordering strategy object is used to describe the execution boundaries and control conditions during online defragmentation. Its preferred features include: maximum number of reorderings, minimum benefit threshold, task reordering cooldown time, whether cross-node reordering is allowed, whether cross-device type candidate substitution is allowed, gray-scale switching parameters for online services, checkpoint recovery parameters for training tasks, and rollback conditions for reordering failures.

[0172] By introducing a rearrangement strategy object, this invention can standardize the description of the online defragmentation process, avoiding writing the rearrangement logic as a purely functional statement, thereby enhancing the clarity and feasibility of the technical solution.

[0173] The aforementioned heterogeneous computing power scheduling device includes a processor and a memory. The acquisition unit, partitioning unit, filtering unit, calculation unit, elimination unit, and scheduling unit are all stored as program units in the memory. The processor executes these program units stored in the memory to achieve the corresponding functions. All of the above modules are located in the same processor; alternatively, the modules may be located in different processors in any combination.

[0174] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can address the issue of service quality fluctuations caused by simply pursuing higher resource utilization.

[0175] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0176] This invention provides a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform the heterogeneous computing power scheduling method.

[0177] Specifically, the scheduling methods for heterogeneous computing power include:

[0178] Step S101: When a node connects to the container cloud platform or the device status changes, obtain the basic capabilities and operating environment information of the heterogeneous device.

[0179] Step S102: Divide the above-mentioned heterogeneous devices into multiple slice resource objects;

[0180] Step S103: Filter candidate locations that meet the requirements of device type, architecture compatibility, isolation requirements and tenant policy constraints. The candidate locations are the set of the slice resource objects.

[0181] Step S104: Calculate the proportion of consecutive available slices at the above candidate locations, and if the proportion of consecutive available slices at the above candidate locations is greater than the task requirement proportion, determine that the above candidate locations have carrying capacity. The task requirement proportion is the ratio of the number of slices required for the task to be processed to the total number of slices in the device.

[0182] Step S105: Eliminate the above-mentioned candidate locations with carrying capacity that do not meet the minimum resource vector, operator compatibility requirements, tenant quota, priority protection and interference restrictions, and obtain the preferred candidate locations;

[0183] Step S106: Allocate the slice resource object at the preferred candidate position and start the pending task.

[0184] This invention provides a processor for running a program, wherein the program executes the heterogeneous computing power scheduling method during runtime.

[0185] Specifically, the scheduling methods for heterogeneous computing power include:

[0186] Step S101: When a node connects to the container cloud platform or the device status changes, obtain the basic capabilities and operating environment information of the heterogeneous device.

[0187] Step S102: Divide the above-mentioned heterogeneous devices into multiple slice resource objects;

[0188] Step S103: Filter candidate locations that meet the requirements of device type, architecture compatibility, isolation requirements and tenant policy constraints. The candidate locations are the set of the slice resource objects.

[0189] Step S104: Calculate the proportion of consecutive available slices at the above candidate locations, and if the proportion of consecutive available slices at the above candidate locations is greater than the task requirement proportion, determine that the above candidate locations have carrying capacity. The task requirement proportion is the ratio of the number of slices required for the task to be processed to the total number of slices in the device.

[0190] Step S105: Eliminate the above-mentioned candidate locations with carrying capacity that do not meet the minimum resource vector, operator compatibility requirements, tenant quota, priority protection and interference restrictions, and obtain the preferred candidate locations;

[0191] Step S106: Allocate the slice resource object at the preferred candidate position and start the pending task.

[0192] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having at least the following method steps:

[0193] Step S101: When a node connects to the container cloud platform or the device status changes, obtain the basic capabilities and operating environment information of the heterogeneous device.

[0194] Step S102: Divide the above-mentioned heterogeneous devices into multiple slice resource objects;

[0195] Step S103: Filter candidate locations that meet the requirements of device type, architecture compatibility, isolation requirements and tenant policy constraints. The candidate locations are the set of the slice resource objects.

[0196] Step S104: Calculate the proportion of consecutive available slices at the above candidate locations, and if the proportion of consecutive available slices at the above candidate locations is greater than the task requirement proportion, determine that the above candidate locations have carrying capacity. The task requirement proportion is the ratio of the number of slices required for the task to be processed to the total number of slices in the device.

[0197] Step S105: Eliminate the above-mentioned candidate locations with carrying capacity that do not meet the minimum resource vector, operator compatibility requirements, tenant quota, priority protection and interference restrictions, and obtain the preferred candidate locations;

[0198] Step S106: Allocate the slice resource object at the preferred candidate position and start the pending task.

[0199] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0200] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0201] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0202] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0203] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0204] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0205] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0206] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0207] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0208] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0209] As can be seen from the above description, the embodiments of this application achieve the following technical effects:

[0210] 1) This application divides heterogeneous computing power into slice resource objects visible to the scheduler, and determines the actual carrying capacity of candidate devices based on continuously available slices. This allows the scheduler to move beyond simply judging whether the total amount of resources is sufficient, and further identify whether the resources can be combined according to constraints to carry tasks. Combined with a binning strategy, it can prioritize the use of remaining slices of devices that are already partially occupied, reducing the generation of new fragments. This not only improves the surface resource utilization rate, but also the actual availability and schedulable utilization rate of heterogeneous computing power resources.

[0211] 2) When direct placement fails, this application does not simply suspend the task, but triggers an online defragmentation process based on conditions such as the proportion of continuously available slices, waiting latency, and the status of high-priority tasks. By restoring the continuous carrying space required for subsequent tasks through slice-level rearrangement, it can significantly improve the schedulability of high-priority tasks and large-scale tasks and reduce long-term pending situations caused by resource fragmentation.

[0212] 3) This application does not simply pursue "as many mixed deployments as possible", but introduces compatibility, isolation, priority and service quality constraints in the candidate screening, continuous available slice determination and scoring process, so that the consolidation behavior always occurs within a controllable boundary, achieving constrained and efficient reuse rather than unlimited resource squeezing. Therefore, it is more conducive to balancing resource efficiency and business stability in the production environment.

[0213] 4) This application adopts differentiated rearrangement methods for different task types: For online service or inference tasks, dual-instance parallelism and gray-scale switching are used to reduce the risk of business interruption; for training or batch processing tasks, checkpoint or savepoint recovery is used to avoid dependence on general hot migration capabilities. The proposed online defragmentation closed loop is not just a concept, but has strong engineering feasibility and rollback controllability, making it more suitable for real loads in production environments.

[0214] 5) When a task cannot be placed directly, this application restores the carrying space through online defragmentation and re-executes candidate screening and placement after the rearrangement is completed, thus forming a closed loop of "placement failure - online defragmentation - status update - retry placement". It has dynamic adaptive optimization capabilities and can continuously improve the cluster resource layout as the running status changes, rather than remaining at the static scheduling level.

[0215] 6) This application enables different types of heterogeneous devices to have a unified description method and a unified candidate decision basis on the scheduler side through the slice resource object and continuous available slice determination mechanism; combined with the scoring mechanism and online sorting closed loop, it realizes unified processing for scheduling optimization. It does not realize a unified resource pool in form, but an integrated heterogeneous computing power decision capability for scheduling optimization, which is more conducive to unified resource utilization in complex heterogeneous environments.

[0216] 7) This application enables the reorganization of fragmented resources that were originally difficult to recombine and reuse through the visibility of sliced ​​resource objects and online fragmentation mechanism. This allows more tasks to be carried with the same hardware investment, reducing the expansion pressure and resource waste caused by fragmentation, and helping to reduce the overall operating cost of the platform.

[0217] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for scheduling heterogeneous computing power, characterized in that, include: When a node connects to the container cloud platform or the device status changes, it obtains the basic capabilities and operating environment information of the heterogeneous device. The heterogeneous device is divided into multiple slice resource objects; Candidate locations that meet the requirements of device type, architecture compatibility, isolation requirements, and tenant policy constraints are selected, and the candidate locations are a set of slice resource objects; Calculate the proportion of consecutive available slices at the candidate location, and determine that the candidate location has carrying capacity if the proportion of consecutive available slices at the candidate location is greater than the task demand proportion. The task demand proportion is the ratio of the number of slices required for the task to be processed to the total number of slices in the device. Candidate locations with carrying capacity that do not meet the minimum resource vector, operator compatibility requirements, tenant quota, priority protection, and interference restrictions are eliminated to obtain preferred candidate locations; The slice resource object is allocated at the preferred candidate location and the pending task is initiated.

2. The method according to claim 1, characterized in that, Allocating the slice resource object at the preferred candidate location and initiating the task to be processed includes: Each of the preferred candidate locations is scored based on factors such as fragmentation waste, node diffusion, energy consumption, and tenant penalties, and the preferred candidate location with the highest score is determined as the optimal candidate location. Allocate the slice resource object at the optimal candidate location and start the task to be processed.

3. The method according to claim 2, characterized in that, The method further includes: If no preferred candidate position is available, the decision to enter the online reordering process is made based on the proportion of continuously available slices, waiting latency, high-priority task status, and estimated benefits. When entering the online rearrangement process, the heterogeneous devices are sliced ​​and rearranged.

4. The method according to claim 3, characterized in that, The decision to enter the online reordering process is based on the proportion of continuously available slices, waiting latency, the status of high-priority tasks, and estimated revenue, including: If there are pending tasks with a priority higher than the predetermined priority, it is determined that the task is in the high-priority task state; If the task is in a high-priority state, initiate the online evaluation process; The online evaluation process is initiated when the proportion of continuously available slices is less than the corresponding threshold and the waiting delay is greater than the corresponding threshold. A benefit analysis is performed on the candidate rearrangement schemes to obtain the benefits of the candidate rearrangement schemes. The benefits include the freed-up contiguous available slice space, the improvement in waiting latency for tasks with a priority higher than the predetermined priority, and the degree of optimization of the score. If the revenue exceeds a preset revenue threshold, the online rearrangement process begins.

5. The method according to claim 3, characterized in that, When entering the online reordering process, the heterogeneous devices are sliced ​​and reordered, including: When the task to be processed is an online service task, a model inference task, or a streaming task, the slice rearrangement is completed using dual-instance parallelism and grayscale streaming. When the task to be processed is a training task, an offline inference task, or a batch processing task, the slice rearrangement is completed using a checkpoint or savepoint recovery method.

6. The method according to claim 3, characterized in that, In the case of entering the online rearrangement process, after slicing and rearranging the heterogeneous devices, the method further includes: When the task to be processed is an online service task, a model inference task, or a streaming task, verify whether the latency, error rate, throughput, and latency tail ripple of the task to be processed meet the corresponding requirements to obtain the first verification result. When the task to be processed is a training task, an offline inference task, or a batch processing task, verify whether the recovery success rate, task progress retention, and execution integrity of the task to be processed meet the corresponding requirements, and obtain a second verification result. If either the first verification result or the second verification result is successful, the rearrangement is confirmed to be successful and the slice resource status is updated. If either the first verification result or the second verification result is a verification failure, a rollback logic is executed to restore the slice resource state to its state before rearrangement.

7. The method according to claim 6, characterized in that, After successful rearrangement, the method further includes: The preferred candidate positions are re-selected, and the slice resource object is allocated at the preferred candidate positions and the pending task is started; When the pending task ends or stops, the slice resource object corresponding to the preferred candidate position is released.

8. A heterogeneous computing power scheduling device, characterized in that, include: The acquisition unit is used to acquire basic capabilities and operating environment information of heterogeneous devices when a node connects to the container cloud platform or when the device status changes. A partitioning unit is used to partition the heterogeneous device into multiple slice resource objects; The filtering unit is used to filter candidate locations that meet the requirements of device type, architecture compatibility, isolation requirements and tenant policy constraints, wherein the candidate locations are a set of slice resource objects; The calculation unit is used to calculate the proportion of consecutive available slices at the candidate location, and determine that the candidate location has carrying capacity if the proportion of consecutive available slices at the candidate location is greater than the task demand ratio. The task demand ratio is the ratio of the number of slices required for the task to be processed to the total number of slices in the device. The elimination unit is used to eliminate the candidate positions with carrying capacity that do not meet the minimum resource vector, operator compatibility requirements, tenant quota, priority protection and interference restrictions, so as to obtain the preferred candidate positions; The scheduling unit is used to allocate the slice resource object at the preferred candidate location and start the task to be processed.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 7.

10. A heterogeneous computing power slicing resource scheduling system, characterized in that, Used to perform the method according to any one of claims 1 to 7.

Images