Resource scheduling method and electronic device

Abstract

The application provides a resource scheduling method and an electronic device, which can be applied to the technical field of computers. The resource scheduling method comprises the following steps: in response to a resource request for a target task, obtaining resource utilization scores of nodes and hardware accelerators in a cluster according to resource usage amounts of the hardware accelerators, resource allocation amounts of the nodes, and a resource demand amount carried by the resource request; determining a target node in the nodes and a target hardware accelerator of the target node according to the resource utilization scores and a scheduling strategy carried by the resource request, the scheduling strategy indicating a scheduling rule matched with resource integration requirements and / or load balancing requirements of the target task; virtually dividing the target hardware accelerator according to the resource demand amount to obtain a virtual hardware accelerator; and binding the virtual hardware accelerator to a container of the target node to execute the target task by using the container.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically to a resource scheduling method and electronic device. Background Technology

[0002] With the rapid development of fields such as scientific computing, heterogeneous computing has become a core means of improving computing power. Clusters support various heterogeneous physical devices through a device plug-in mechanism, allowing physical devices on nodes to be reported as schedulable resources and allocated by the cluster scheduler. However, tasks with low computational resource requirements do not need to occupy the entire computing cores and memory of a physical device, leading to resource fragmentation and waste, and reducing the overall resource utilization of the cluster. Summary of the Invention

[0003] In view of the above problems, this application provides a resource scheduling method and an electronic device.

[0004] According to a first aspect of this application, a resource scheduling method is provided, comprising: responding to a resource request for a target task, obtaining resource utilization scores for each node and the hardware accelerator based on resource usage, resource allocation, and resource demand carried in the resource request of hardware accelerators of multiple nodes in a cluster; determining a target node and a target hardware accelerator of the target node based on the resource utilization scores and a scheduling policy carried in the resource request, wherein the scheduling policy indicates scheduling rules that match the resource integration and / or load balancing requirements of the target task; virtually partitioning the target hardware accelerator according to the resource demand to obtain a virtual hardware accelerator; and binding the virtual hardware accelerator to a container of the target node to execute the target task using the container.

[0005] According to embodiments of this application, a quantitative scoring model based on integrated hardware capabilities and real-time load correlates the resource requirements of the target task, the real-time resource usage of hardware devices, and the resource allocation, accurately characterizing the potential load of the pre-allocated cluster. A flexibly configurable strategy engine is embedded in the cluster scheduler, supporting two strategies that match the resource integration or load balancing requirements of the target task. This allows users to adaptively switch scheduling strategies based on the computing power requirements of the target task. The cluster scheduler then makes decisions based on the scheduling strategy and potential load, selecting the target hardware accelerator under the target node. This enables the scheduling process to perceive the real-time resource utilization of the underlying hardware and achieve global collaborative optimization, improving the adaptability of the scheduling system. Furthermore, based on the resource requirements of the target task, the target hardware accelerator is virtually partitioned at a fine granular level, achieving dynamic allocation and efficient utilization of hardware accelerator computing resources. This meets the needs of parallel execution of multiple target tasks and fine-grained resource scheduling, improving the overall resource utilization and task stability of the cluster.

[0006] According to embodiments of this application, resource utilization scores for nodes and hardware accelerators are obtained based on the resource usage, resource allocation, and resource requirements carried in resource requests of the hardware accelerators in the cluster. This includes: determining the total resource usage and total resource allocation of a node based on the resource usage and resource allocation of each of the multiple hardware accelerators in the cluster; obtaining a resource utilization score for a node based on its total resource usage, total resource allocation, and a first resource requirement, wherein the first resource requirement is determined based on the number of virtual hardware accelerators and the resource requirements carried in resource requests; and obtaining a resource utilization score for a hardware accelerator based on its resource usage, resource allocation, and a second resource requirement, wherein the second resource requirement is determined based on the resource requirements carried in resource requests for virtual hardware accelerators.

[0007] According to embodiments of this application, resource usage and allocation data from multiple hardware accelerators are aggregated at the node level to form a total index. A first resource requirement, quantified by the number of virtual hardware accelerators and their resource volume, is used to calculate a node-level resource utilization score, ensuring load balancing and fragmentation integration capabilities of the macro-level resource pool. Simultaneously, a second resource requirement based on the resource needs of a single virtual hardware accelerator is introduced at the hardware accelerator level for micro-level scoring, accurately matching the computing power of the target task with that of the physical device. Finally, through collaborative decision-making at both the node and hardware accelerator levels, cross-node communication overhead and performance degradation caused by hardware capability mismatch are reduced, significantly improving the scheduling efficiency and task execution stability of large-scale target tasks in heterogeneous clusters.

[0008] According to embodiments of this application, resource usage includes at least one of memory usage and computing unit usage, resource allocation includes at least one of memory allocation and computing unit allocation, and second resource demand includes at least one of memory demand and computing unit demand. The resource utilization score of the hardware accelerator is obtained based on its resource usage, resource allocation, and second resource demand, including: obtaining a memory utilization score based on the ratio of the sum of memory usage and memory demand to memory allocation; obtaining a computing unit utilization score based on the ratio of the sum of computing unit usage and computing unit demand to computing unit allocation; and weighting the memory utilization score and the computing unit utilization score to obtain a resource utilization score.

[0009] According to the embodiments of this application, the quantitative scoring model integrates hardware capabilities and real-time load, and calculates the resource requirements of the target task, the real-time resource usage of the hardware device, and the resource allocation, which accurately characterizes the potential load after pre-allocation. The cluster scheduler then makes decisions based on the scheduling strategy, upgrading the scheduling process from "if it is available" to "selecting the best".

[0010] According to an embodiment of this application, the scheduling strategy includes a node scheduling strategy and a hardware accelerator scheduling strategy; wherein, determining the target node and the target hardware accelerator of the target node in the cluster based on the resource utilization score and the scheduling strategy carried in the resource request includes: determining the target node from multiple nodes in the cluster based on the node scheduling strategy and the node's resource utilization score; and determining the target hardware accelerator of the target node from multiple hardware accelerators based on the hardware accelerator scheduling strategy and the respective resource utilization scores of the multiple hardware accelerators of the target node.

[0011] According to embodiments of this application, the scheduling strategy is deepened from the node level to the hardware accelerator level. The multi-strategy scheduling engine can simultaneously operate at two levels: node selection and selection of hardware accelerators within a node. The system can not only select the most suitable target node, but also select the most suitable target hardware accelerator on the target node, achieving global collaborative optimization from macro to micro levels and maximizing resource utilization efficiency.

[0012] According to embodiments of this application, determining a target node from multiple nodes based on a node scheduling strategy and the node's resource utilization score includes: when the node scheduling strategy is a scheduling rule that matches resource integration requirements, determining the node corresponding to the highest resource utilization score among multiple resource utilization scores as the target node; when the node scheduling strategy is a scheduling rule that matches load balancing requirements, determining the node corresponding to the lowest resource utilization score among multiple resource utilization scores as the target node.

[0013] According to embodiments of this application, node scheduling strategies can be flexibly configured and selected based on the computing power requirements of the target task. Simultaneously, a flexibly configurable strategy engine is embedded within the cluster scheduler, supporting two strategies that match either resource integration or load balancing requirements. This engine allows users to switch scheduling orientations with a single click based on the computing power requirements of the target task, achieving a leap from "fixed logic" to "strategy-driven" scheduling, significantly improving the adaptability and intelligence of the scheduling system.

[0014] According to embodiments of this application, determining the target hardware accelerator for a target node from multiple hardware accelerators based on a hardware accelerator scheduling policy and the respective resource utilization scores of multiple hardware accelerators of the target node includes: when the hardware accelerator scheduling policy is a scheduling rule that matches resource consolidation requirements, determining the hardware accelerator corresponding to the highest resource utilization score among multiple resource utilization scores as the target hardware accelerator; when the hardware accelerator scheduling policy is a scheduling rule that matches load balancing requirements, determining the hardware accelerator corresponding to the lowest resource utilization score among multiple resource utilization scores as the target hardware accelerator.

[0015] According to the embodiments of this application, a two-level collaborative scheduling mechanism is set up for hardware accelerators between nodes and within nodes, and a scheduling policy engine is configured to realize flexible switching between the two-level scheduling policies and global optimal scheduling from macro to micro.

[0016] According to an embodiment of this application, the resource scheduling method further includes: determining the target device attribute carried in the resource request, wherein the target device attribute is determined from the respective device attributes of multiple hardware accelerators based on the computing power requirement, and the device attribute includes at least one of computing power attribute and device model; determining multiple initial hardware accelerators corresponding to the target device attribute from the multiple hardware accelerators, so as to determine a resource utilization score based on the initial hardware accelerators.

[0017] According to embodiments of this application, the configurable hardware discovery engine of the device plugin can automatically identify device attributes such as the model and specific computing power attributes of hardware accelerators on the node based on preset configuration file driving rules. This concretizes physical hardware from the abstract "resource" into entities with different performance labels, laying a data foundation for subsequent differentiated scheduling based on hardware capabilities. This enables the cluster scheduler to distinguish hardware accelerators with different performance levels, allowing users to make initial selections based on the computing power requirements of the target task. It ensures that high-performance target tasks are prioritized for scheduling on high-performance hardware, improving task execution efficiency and cluster computing power output. This solves the technical problem of managing the heterogeneity of hardware accelerators in the cluster and realizes hardware-aware intelligent scheduling.

[0018] According to an embodiment of this application, the resource scheduling method further includes: in response to the completion of the target task, updating the resource usage of the target hardware accelerator and the total resource usage of the target node based on the resource demand.

[0019] According to the embodiments of this application, the device plugin of each node continuously monitors the allocation and usage of computing units and memory of each hardware accelerator, and synchronizes this dynamic information global status to the cluster scheduler by updating node annotations in real time. This mechanism breaks the limitation of traditional device plugins only reporting the static resource quantity, and provides the cluster scheduler with a global, real-time and accurate resource level view required for decision-making, thus avoiding resource allocation conflicts.

[0020] According to an embodiment of this application, a virtual hardware accelerator is obtained by virtually partitioning a target hardware accelerator based on resource requirements, including: generating a virtual partitioning strategy for the target hardware accelerator based on a second resource requirement; dividing the resources of the target hardware accelerator according to the virtual partitioning strategy to obtain resource partitions; and instantiating the resource partitions into virtual hardware accelerators with independent logical interfaces.

[0021] According to embodiments of this application, each virtual hardware accelerator can independently access the driver, allocate tasks, and ensure performance isolation. It can continue to be virtually split based on other target tasks with continuous response, thereby realizing the secure sharing of a single hardware accelerator among multiple target tasks, improving resource utilization and avoiding resource fragmentation.

[0022] A second aspect of this application provides a resource scheduling apparatus, comprising: a response module, configured to respond to a resource request for a target task and obtain resource utilization scores for each node and hardware accelerator based on the resource usage, resource allocation, and resource demand carried in the resource request of multiple nodes in the cluster; a determination module, configured to determine a target node and a target hardware accelerator among the multiple nodes based on the resource utilization scores and the scheduling policy carried in the resource request, wherein the scheduling policy indicates scheduling rules that match the resource integration and / or load balancing requirements of the target task; a splitting module, configured to virtually split the target hardware accelerator based on the resource demand to obtain virtual hardware accelerators; and a binding module, configured to bind the virtual hardware accelerators to a container of the target node to utilize the container to execute the target task.

[0023] A third aspect of this application provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above. Attached Figure Description

[0024] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments of this application with reference to the accompanying drawings.

[0025] Figure 1 The diagram illustrates an application scenario of a resource scheduling method and an electronic device according to embodiments of this application.

[0026] Figure 2 A flowchart of a resource scheduling method according to an embodiment of this application is shown.

[0027] Figure 3 A schematic diagram illustrating the device attributes for identifying a hardware accelerator according to an embodiment of this application is shown.

[0028] Figure 4 A schematic diagram illustrating the processing of resource requests according to an embodiment of this application is shown.

[0029] Figure 5 A schematic diagram of resource scheduling according to an embodiment of this application is shown.

[0030] Figure 6 A structural block diagram of a resource scheduling apparatus according to an embodiment of this application is shown.

[0031] Figure 7 A block diagram of an electronic device suitable for implementing a resource scheduling method according to an embodiment of this application is shown. Detailed Implementation

[0032] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0033] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0034] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0035] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0036] A cluster is a unified computing resource pool formed by multiple nodes connected through a network, providing the ability to deploy, run, and manage containerized applications.

[0037] A Pod is the smallest application instance to be scheduled in a cluster. It is the carrier of the target task and contains one or more tightly coupled containers that share network, storage, and computing resources.

[0038] A node is a physical machine or virtual machine in the cluster that runs a Pod. It is equipped with computing resources such as a central processing unit (CPU), memory, storage and hardware accelerators, and is the actual running platform of the Pod.

[0039] Hardware accelerators are dedicated computing devices deployed on nodes. For example, a hardware accelerator can be an accelerator device such as a graphics processing unit (GPU) or a deep computing unit (DCU).

[0040] The cluster scheduler is the core control component responsible for listening to unscheduled Pods and selecting the optimal target node for a Pod based on the cluster's global resource status and scheduling policy.

[0041] Virtualization partitioning technology aims to divide a single hardware accelerator into multiple virtual hardware accelerators for multiple Pods to share. It focuses on the partitioning capabilities of the device itself, but its scheduling decisions at the cluster level are still relatively crude. The cluster scheduler only knows the number of "virtual devices" on the node, but lacks awareness of the differences in the underlying hardware models and the real-time resource usage, resulting in blindness in the scheduling process.

[0042] In relevant scheduling methods, scheduling accelerator hardware resources mainly relies on device plugins and the scheduler's default logic. However, when facing hardware accelerator resources (such as DCUs) requiring dynamic allocation and fine-grained resource management, the following problems mainly exist:

[0043] First, the cluster scheduler lacks fine-grained hardware awareness. The cluster scheduler and most custom device plugins can only manage virtual hardware accelerators as discrete units. The scheduler cannot perceive the specific model and computing power of the hardware accelerator behind each virtual hardware accelerator, nor can it perceive the real-time, fine-grained resource availability (such as remaining computing cores and memory) on each hardware accelerator. This prevents the scheduler from making optimal node selections and from prioritizing computationally intensive tasks to high-performance hardware accelerators, potentially leading to high-performance computing tasks being scheduled to low-end devices.

[0044] Second, resource scheduling strategies are simplistic and inflexible. Related resource allocation strategies are typically implicit or fixed, unable to be flexibly adjusted according to the intentions of users or cluster administrators. For example, when maximizing single-machine resource utilization to save power or costs, there is a lack of effective "boxing" strategies to compactly distribute tasks across a small number of nodes and DCUs; when avoiding single-point overheating and achieving load balancing to improve system fault tolerance, there is a lack of effective "distribution" strategies to evenly distribute tasks across the entire cluster. This lack of flexibility in scheduling strategies makes cluster resource management unable to adapt to diverse business scenarios and operational needs.

[0045] Third, the static resource view can easily lead to allocation conflicts and decreased utilization. Since device plugins typically report static resource quantities, the scheduler makes decisions based on this static view, failing to perceive dynamic changes in resources at the second or millisecond level. This can result in multiple Pods being simultaneously scheduled onto the same nearly saturated physical DCU, leading to resource allocation conflicts at the device level. Furthermore, it prevents optimal scheduling based on real-time load, impacting the overall resource utilization and task stability of the cluster.

[0046] In summary, in the cluster ecosystem, how to achieve hardware accelerator resource scheduling that integrates hardware awareness, real-time monitoring, dynamic partitioning, and multi-strategy intelligent scheduling has become an urgent technical problem to be solved.

[0047] To address at least one of the aforementioned problems, embodiments of this application provide a resource scheduling method. In response to a resource request for a target task, a resource utilization score is obtained for each node and hardware accelerator based on the resource usage, resource allocation, and resource demand carried in the resource request of multiple nodes in the cluster. Based on the resource utilization score and the scheduling policy carried in the resource request, a target node and its target hardware accelerator are determined from among the multiple nodes. The scheduling policy indicates scheduling rules that match the resource integration and / or load balancing requirements of the target task. The target hardware accelerator is virtually partitioned based on the resource demand to obtain virtual hardware accelerators. The virtual hardware accelerators are then bound to containers on the target node to utilize the containers to execute the target task.

[0048] In scenarios involving automated decision-making using personal information, the methods, devices, and systems provided in this application all offer users corresponding entry points for choosing to agree to or reject the automated decision-making results. If the user chooses to reject, the process proceeds to the expert decision-making stage. Here, "automated decision-making" refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests, or economic, health, and credit status through computer programs, and then making a decision. Here, "expert decision-making" refers to the activity of making decisions by personnel who specialize in a particular field, possess specialized experience, knowledge, and skills, and have reached a certain level of professional expertise.

[0049] Figure 1 The diagram illustrates an application scenario of a resource scheduling method and an electronic device according to embodiments of this application.

[0050] like Figure 1 As shown, the application scenario according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0051] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication terminal applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103.

[0052] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays that support browsing or uploading, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0053] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0054] It should be noted that the resource scheduling method provided in this application embodiment can generally be executed by server 105. Correspondingly, the resource scheduling device provided in this application embodiment can generally be located in server 105. The resource scheduling method provided in this application embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the resource scheduling device provided in this application embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0055] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0056] The following will be based on Figure 1 The described scene, through Figures 2-6 The resource scheduling method according to the embodiments of this application will be described in detail.

[0057] Figure 2 A flowchart of a resource scheduling method according to an embodiment of this application is shown.

[0058] like Figure 2 As shown, the resource scheduling method in this embodiment includes operations S210 to S240.

[0059] In operation S210, in response to resource requests for the target task, resource utilization scores are obtained for each node and hardware accelerator based on the resource usage, resource allocation, and resource requirements carried in the resource requests of the hardware accelerators of multiple nodes in the cluster.

[0060] In operation S220, based on resource utilization scores and scheduling policies carried by resource requests, the target node and the target hardware accelerator of the target node are determined among multiple nodes.

[0061] In operation S230, the target hardware accelerator is virtually partitioned according to the resource requirements to obtain a virtual hardware accelerator.

[0062] In operation S240, a virtual hardware accelerator is bound to a container on the target node to perform the target task using the container.

[0063] The target task is a computational task that needs to be scheduled onto nodes in the cluster. For example, training or inference tasks for deep learning models.

[0064] Resource usage represents the actual resources consumed by the hardware accelerator.

[0065] Resource allocation represents the total amount of resources allocated by the system to the hardware accelerator.

[0066] The resource request carries the minimum resource requirement for the target task.

[0067] Resources include the number of computing units or the amount of memory.

[0068] The cluster scheduler responds to resource requests for a target task by obtaining the resource requirements carried in the resource request, as well as the real-time resource usage and allocation of each hardware accelerator.

[0069] Multiple hardware accelerators are deployed on each node. The total resource usage and total resource allocation of the node are obtained by summing the resource usage and resource allocation of each hardware accelerator on the node.

[0070] A quantitative scoring model is used to process the total resource usage, total resource allocation, and resource demand of multiple nodes to calculate the resource utilization score of each node.

[0071] A quantitative scoring model is used to calculate the resource utilization score of each hardware accelerator by processing the resource usage, resource allocation, and resource demand of multiple hardware accelerators under a node.

[0072] Resource utilization scores are used to predict the load level of nodes or hardware accelerators after resources are allocated based on resource demand.

[0073] In resource requests, predefined scheduling policies can be set for nodes and hardware accelerators separately. The scheduling policy indicates the scheduling rules that match the resource integration and / or load balancing requirements of the target task.

[0074] The resource integration requirement is to aggregate the scattered target tasks to the same node and the same hardware accelerator on the same node as much as possible, so as to reduce cross-node communication; the load balancing requirement is to distribute the target tasks as evenly as possible to multiple nodes or multiple hardware accelerators to avoid single point overload.

[0075] When the scheduling policy indicates scheduling rules that match the resource integration requirements, the node or hardware accelerator with the highest resource utilization score can be selected; when the scheduling policy indicates scheduling rules that match the load balancing requirements, the node or hardware accelerator with the lowest resource utilization score can be selected.

[0076] First, a target node is selected from multiple nodes according to the node scheduling rules. Then, a target hardware accelerator is selected from multiple hardware accelerators on the target node according to the hardware accelerator scheduling rules.

[0077] The cluster scheduler invokes the device plugins on the target node to perform logical virtual partitioning on the target hardware accelerator based on resource requirements. This partitioning results in fine-grained and isolated virtual hardware accelerators, completing resource mounting. For example, an 80GB target hardware accelerator A100 can be partitioned into two 40GB or eight 10GB virtual hardware accelerators, allowing multiple target tasks to securely share the same target hardware accelerator. Simultaneously, hardware-level isolation ensures strict resource boundary isolation between each virtual hardware accelerator.

[0078] For example, if the target task requires 50 computing units and the target hardware accelerator has 100 computing units, a virtual hardware accelerator with 50 computing units can be partitioned from the target hardware accelerator.

[0079] Bind the virtual hardware accelerator to the container on the target node, and start the container to run the target task.

[0080] According to embodiments of this application, a quantitative scoring model based on integrated hardware capabilities and real-time load correlates the resource requirements of the target task, the real-time resource usage of hardware devices, and the resource allocation, accurately characterizing the potential load of the pre-allocated cluster. A flexibly configurable strategy engine is embedded in the cluster scheduler, supporting two strategies that match the resource integration or load balancing requirements of the target task. This allows users to adaptively switch scheduling strategies based on the computing power requirements of the target task. The cluster scheduler then makes decisions based on the scheduling strategy and potential load, selecting the target hardware accelerator under the target node. This enables the scheduling process to perceive the real-time resource utilization of the underlying hardware and achieve global collaborative optimization, improving the adaptability of the scheduling system. Furthermore, based on the resource requirements of the target task, the target hardware accelerator is virtually partitioned at a fine granular level, achieving dynamic allocation and efficient utilization of hardware accelerator computing resources. This meets the needs of parallel execution of multiple target tasks and fine-grained resource scheduling, improving the overall resource utilization and task stability of the cluster.

[0081] According to an embodiment of this application, the resource scheduling method further includes: determining the target device attribute carried in the resource request, wherein the target device attribute is determined from the respective device attributes of multiple hardware accelerators based on the computing power requirement, and the device attribute includes at least one of computing power attribute and device model; determining multiple initial hardware accelerators corresponding to the target device attribute from the multiple hardware accelerators, so as to determine a resource utilization score based on the initial hardware accelerators.

[0082] The device attributes of a hardware accelerator include computing power, manufacturer, and device model.

[0083] When the device plugin on each node starts up, it automatically identifies the device attributes of the hardware accelerators on each node in the cluster through the configuration file driven rules in the configurable hardware discovery engine, monitors the resource status changes of the hardware accelerators on each node, and stores the device attributes and resource status changes in the form of node annotations to obtain storage files.

[0084] Based on the computing power requirements of the target task, matching device attributes can be adaptively selected from the storage file, and the device attributes can be specified in the resource request for the target task.

[0085] For example, the target device attributes are determined based on the computing power requirements of the target task. The target device attributes include the manufacturer NV and the device model H10. Multiple initial hardware accelerators that meet the requirements of manufacturer NV and device model H10 are first selected from the cluster.

[0086] The resource utilization score of the initial hardware accelerator is calculated based on multiple initial hardware accelerators, and the resource utilization score of the node is calculated based on the node associated with the initial hardware accelerator.

[0087] Figure 3 A schematic diagram illustrating the device attributes for identifying a hardware accelerator according to an embodiment of this application is shown.

[0088] like Figure 3 As shown, a device plugin, vdcu-device-plugin, is deployed on each of the multiple nodes (node1, node2 to nodeN) in the cluster. The device plugin automatically detects the device attributes of various hardware accelerators deployed on the nodes through the detect mechanism. For example, it identifies hardware accelerators with device models A10 and H10 deployed on node1, hardware accelerators with device models K100_AI and BW100 deployed on node2, and hardware accelerators with device model MI250 deployed on nodeN. It collects the device attributes and resource status information of the hardware accelerators in real time and adds this information to the annotations of the corresponding nodes. This enables the cluster control plane to perceive the global heterogeneous computing resource topology of the cluster, providing a data foundation for the cluster scheduler to make intelligent scheduling decisions across vendors and models.

[0089] According to embodiments of this application, the configurable hardware discovery engine of the device plugin can automatically identify device attributes such as the model and specific computing power attributes of hardware accelerators on the node based on preset configuration file driving rules. This concretizes physical hardware from the abstract "resource" into entities with different performance labels, laying a data foundation for subsequent differentiated scheduling based on hardware capabilities. This enables the cluster scheduler to distinguish hardware accelerators with different performance levels, allowing users to make initial selections based on the computing power requirements of the target task. It ensures that high-performance target tasks are prioritized for scheduling on high-performance hardware, improving task execution efficiency and cluster computing power output. This solves the technical problem of managing the heterogeneity of hardware accelerators in the cluster and realizes hardware-aware intelligent scheduling.

[0090] According to embodiments of this application, resource utilization scores for nodes and hardware accelerators are obtained based on the resource usage, resource allocation, and resource requirements carried in resource requests of the hardware accelerators in the cluster. This includes: determining the total resource usage and total resource allocation of a node based on the resource usage and resource allocation of each of the multiple hardware accelerators in the cluster; obtaining a resource utilization score for a node based on its total resource usage, total resource allocation, and a first resource requirement, wherein the first resource requirement is determined based on the number of virtual hardware accelerators and the resource requirements carried in resource requests; and obtaining a resource utilization score for a hardware accelerator based on its resource usage, resource allocation, and a second resource requirement, wherein the second resource requirement is determined based on the resource requirements carried in resource requests for virtual hardware accelerators.

[0091] The resource request carries the required number of virtual hardware accelerators and the resource requirement for a single virtual hardware accelerator. The first resource requirement is obtained by multiplying the required number of virtual hardware accelerators and the resource requirement.

[0092] The total resource usage of a node is obtained by summing the resource usage of each hardware accelerator in the cluster; the total resource allocation of a node is obtained by summing the resource allocation of each hardware accelerator in the cluster.

[0093] A node's resource utilization score is obtained by calculating the ratio of its total resource usage and primary resource demand to its total resource allocation. This score is used to predict the load distribution among nodes after resources are allocated based on their primary resource demand.

[0094] The resource requirement for the virtual hardware accelerator carried in the resource request is determined as the second resource requirement.

[0095] The resource utilization score of a hardware accelerator is obtained by calculating the ratio of its resource usage and second resource demand to the total resource allocation. This score is used to predict the load among hardware accelerators after resource allocation based on the second resource demand.

[0096] According to embodiments of this application, resource usage and allocation data from multiple hardware accelerators are aggregated at the node level to form a total index. A first resource requirement, quantified by the number of virtual hardware accelerators and their resource volume, is used to calculate a node-level resource utilization score, ensuring load balancing and fragmentation integration capabilities of the macro-level resource pool. Simultaneously, a second resource requirement based on the resource needs of a single virtual hardware accelerator is introduced at the hardware accelerator level for micro-level scoring, accurately matching the computing power of the target task with that of the physical device. Finally, through collaborative decision-making at both the node and hardware accelerator levels, cross-node communication overhead and performance degradation caused by hardware capability mismatch are reduced, significantly improving the scheduling efficiency and task execution stability of large-scale target tasks in heterogeneous clusters.

[0097] According to embodiments of this application, resource usage includes at least one of memory usage and computing unit usage, resource allocation includes at least one of memory allocation and computing unit allocation, and second resource demand includes at least one of memory demand and computing unit demand. The resource utilization score of the hardware accelerator is obtained based on its resource usage, resource allocation, and second resource demand, including: obtaining a memory utilization score based on the ratio of the sum of memory usage and memory demand to memory allocation; obtaining a computing unit utilization score based on the ratio of the sum of computing unit usage and computing unit demand to computing unit allocation; and weighting the memory utilization score and the computing unit utilization score to obtain a resource utilization score.

[0098] Memory usage refers to the actual amount of video memory used by the hardware accelerator; computing unit usage refers to the actual number of computing cores used by the hardware accelerator.

[0099] The memory allocation is the amount of video memory allocated by the cluster scheduler to the hardware accelerator; the computing unit allocation is the number of computing cores allocated by the cluster scheduler to the hardware accelerator.

[0100] The memory requirement is the amount of video memory required for the target task; the computing unit requirement is the number of computing cores required for the target task.

[0101] Memory utilization scores are used to predict the load level among hardware accelerators after memory resources are allocated based on memory demand.

[0102] The computing unit uses a score to predict the load level among hardware accelerators after allocating computing unit resources based on computing unit demand.

[0103] In one embodiment, the resource utilization score of the hardware accelerator As shown in formula (1):

[0104] (1).

[0105] in, Characterizes the demand for computing units. Characterizes the amount of computing units used. Characterizes the amount of computational units allocated. Indicates memory requirements. Indicates memory usage. It represents the amount of memory allocated.

[0106] A quantitative scoring model is used to process the resource usage, resource allocation, and secondary resource requirements of the hardware accelerator to obtain its resource utilization score. The resource utilization score of a node can be calculated using the same quantitative scoring model, which will not be elaborated upon here.

[0107] According to the embodiments of this application, the quantitative scoring model integrates hardware capabilities and real-time load, and calculates the resource requirements of the target task, the real-time resource usage of the device, and the resource allocation, accurately depicting the potential load after allocation. The cluster scheduler then makes a decision based on the scheduling strategy, upgrading the scheduling process from "if it is available" to "selecting the best".

[0108] According to an embodiment of this application, the scheduling strategy includes a node scheduling strategy and a hardware accelerator scheduling strategy; wherein, determining the target node and the target hardware accelerator of the target node in the cluster based on the resource utilization score and the scheduling strategy carried in the resource request includes: determining the target node from multiple nodes in the cluster based on the node scheduling strategy and the node's resource utilization score; and determining the target hardware accelerator of the target node from multiple hardware accelerators based on the hardware accelerator scheduling strategy and the respective resource utilization scores of the multiple hardware accelerators of the target node.

[0109] Based on the resource utilization scores of multiple nodes in the cluster, the nodes are sorted in descending order to obtain the node sequence.

[0110] The target node is determined from the node sequence based on the node scheduling strategy.

[0111] Based on the resource utilization scores of multiple hardware accelerators on the target node, the multiple hardware accelerators are sorted in descending order to obtain a hardware accelerator sequence.

[0112] The target hardware accelerator is determined from the hardware accelerator sequence based on the hardware accelerator scheduling policy.

[0113] According to embodiments of this application, the scheduling strategy is deepened from the node level to the hardware accelerator level. The multi-strategy scheduling engine can simultaneously operate at two levels: node selection and selection of hardware accelerators within a node. The system can not only select the most suitable target node, but also select the most suitable target hardware accelerator on the target node, achieving global collaborative optimization from macro to micro levels and maximizing resource utilization efficiency.

[0114] According to embodiments of this application, determining a target node from multiple nodes based on a node scheduling strategy and the node's resource utilization score includes: when the node scheduling strategy is a scheduling rule that matches resource integration requirements, determining the node corresponding to the highest resource utilization score among multiple resource utilization scores as the target node; when the node scheduling strategy is a scheduling rule that matches load balancing requirements, determining the node corresponding to the lowest resource utilization score among multiple resource utilization scores as the target node.

[0115] The node scheduling strategy is a scheduling rule that matches the resource integration requirements. It prioritizes the allocation of target tasks to the nodes with the highest resource utilization, fills the existing resource containers as much as possible, reduces the number of nodes used in the cluster, and is conducive to resource integration and energy saving.

[0116] When the node scheduling strategy is a scheduling rule that matches the resource integration requirements, the node corresponding to the highest resource utilization score among multiple resource utilization scores is determined as the target node, and computing resources are scheduled for the target task based on the target node.

[0117] The node corresponding to the maximum resource utilization score represents a large ratio of resource usage to resource allocation, indicating a high resource utilization rate.

[0118] The node scheduling strategy is a scheduling rule that matches the load balancing requirements. It prioritizes the allocation of target tasks to the nodes with the lowest resource utilization, aiming to distribute the workload evenly throughout the cluster, avoid single-point overload, and improve the overall fault tolerance and stability of the system.

[0119] When the node scheduling strategy is a scheduling rule that matches the load balancing requirements, the node corresponding to the minimum resource utilization score among multiple resource utilization scores is determined as the target node, and computing resources are scheduled for the target task based on the target node.

[0120] The node corresponding to the minimum resource utilization score represents a small ratio of resource demand to resource allocation, indicating low resource utilization.

[0121] According to embodiments of this application, node scheduling strategies can be flexibly configured and selected based on the computing power requirements of the target task. Simultaneously, a flexibly configurable strategy engine is embedded within the cluster scheduler, supporting two strategies that match either resource integration or load balancing requirements. This engine allows users to switch scheduling orientations with a single click based on the computing power requirements of the target task, achieving a leap from "fixed logic" to "strategy-driven" scheduling, significantly improving the adaptability and intelligence of the scheduling system.

[0122] According to embodiments of this application, determining the target hardware accelerator for a target node from multiple hardware accelerators based on a hardware accelerator scheduling policy and the respective resource utilization scores of multiple hardware accelerators of the target node includes: when the hardware accelerator scheduling policy is a scheduling rule that matches resource consolidation requirements, determining the hardware accelerator corresponding to the highest resource utilization score among multiple resource utilization scores as the target hardware accelerator; when the hardware accelerator scheduling policy is a scheduling rule that matches load balancing requirements, determining the hardware accelerator corresponding to the lowest resource utilization score among multiple resource utilization scores as the target hardware accelerator.

[0123] The hardware accelerator scheduling strategy is a scheduling rule that matches the resource integration requirements. It prioritizes the allocation of target tasks to the hardware accelerators with the highest resource utilization, while filling the existing resource containers as much as possible, which is conducive to resource integration and energy saving.

[0124] When the hardware accelerator scheduling strategy is a scheduling rule that matches the resource integration requirements, the hardware accelerator corresponding to the highest resource utilization score among multiple resource utilization scores is determined as the target hardware accelerator, and computing resources are provided for the target task based on the target hardware accelerator.

[0125] The hardware accelerator scheduling strategy is a scheduling rule that matches the load balancing requirements. It prioritizes the allocation of target tasks to the hardware accelerators with the lowest resource utilization, aiming to distribute the workload evenly across the entire cluster and avoid single-point overload.

[0126] When the hardware accelerator scheduling strategy is a scheduling rule that matches the load balancing requirements, the hardware accelerator corresponding to the lowest resource utilization score among multiple resource utilization scores is determined as the target hardware accelerator, and computing resources are provided for the target task based on the target hardware accelerator.

[0127] According to the embodiments of this application, a two-level collaborative scheduling mechanism is set up for hardware accelerators between nodes and within nodes, and a scheduling policy engine is configured to realize flexible switching between the two-level scheduling policies and global optimal scheduling from macro to micro.

[0128] Figure 4 A schematic diagram illustrating the processing of resource requests according to an embodiment of this application is shown.

[0129] like Figure 4 As shown, the user submits a resource request for the target task, specifying the following in the request: nodePolicy (a scheduling rule matching resource consolidation requirements), gpuPolicy (a scheduling rule matching load balancing requirements), target device attributes including device model K100_AI, and resource requirements (quantity: 1; memory requirement: 10GB; compute unit requirement: 50). After receiving the resource request, the cluster scheduler first calculates the node score using the node policy, selecting node2 as the target node. Then, using the target hardware accelerator scheduling policy, it selects the second available card from the four K100_AI hardware accelerators on node2 as the target hardware accelerator. The current status of the target hardware accelerator is: Virtual Hardware Accelerator Status number: 0 / 4 (Used). The system displays 0 sheets / 4 sheets that can be split into, memory status: memory: 0 / 64G (0 memory usage / 64G memory allocation), and compute unit status: cores: 0 / 100 (0 compute unit usage / 64G compute unit allocation). The cluster scheduler then sends the resource request to the device plugin on node2. This plugin performs virtualization splitting on the target hardware accelerator, creating virtual hardware accelerators that meet the resource requirements and binding them to the user container. Finally, the device plugin updates the resource usage status of the target hardware accelerator on node2 (e.g., virtual hardware accelerator status becomes 1 / 4, memory status becomes 10 / 64G, compute unit status becomes 50 / 100), and synchronizes the latest resource information back to the node annotation for subsequent scheduling decisions, achieving closed-loop management of the entire process from resource application, intelligent scheduling, virtual splitting to status feedback.

[0130] According to an embodiment of this application, a virtual hardware accelerator is obtained by virtually partitioning a target hardware accelerator based on resource requirements, including: generating a virtual partitioning strategy for the target hardware accelerator based on a second resource requirement; dividing the resources of the target hardware accelerator according to the virtual partitioning strategy to obtain resource partitions; and instantiating the resource partitions into virtual hardware accelerators with independent logical interfaces.

[0131] For example, if the second resource requirement is 10GB of memory and 50 computing units, a virtual partitioning strategy is formulated to partition the target hardware accelerator to require 10GB of memory and 50 computing units.

[0132] Resource partitioning includes memory partitioning and compute unit partitioning. After receiving the binding information, the device plugin on the target node calls the underlying interface to logically divide the physical resources of the target hardware accelerator according to the virtual partitioning strategy, forming a mutually isolated 10GB memory partition and a compute unit partition containing 50 compute units.

[0133] The memory partition and compute unit partition are instantiated as virtual hardware accelerators with independent logical interfaces, and the virtual hardware accelerators are object instances.

[0134] According to embodiments of this application, each virtual hardware accelerator can independently access the driver, allocate tasks, and ensure performance isolation. It can continue to be virtually split based on other target tasks with continuous response, thereby realizing the secure sharing of a single hardware accelerator among multiple target tasks, improving resource utilization and avoiding resource fragmentation.

[0135] According to an embodiment of this application, the resource scheduling method further includes: in response to the completion of the target task, updating the resource usage of the target hardware accelerator and the total resource usage of the target node based on the resource demand.

[0136] In response to the completion of the target task, the cluster scheduler calls the device plugin of the target node to update the resource usage of the target hardware accelerator and the total resource usage of the target node according to the resource requirements of the target task.

[0137] For example, before the target task is executed, the total resource usage of the target node is 128GB, the resource usage of the target hardware accelerator is 0, and the resource requirement is 10GB. After the target task is completed, the resource usage of the target hardware accelerator is updated to 10GB and the total resource usage of the target node is 138GB.

[0138] The device plugin uploads the update results to the cluster scheduler.

[0139] According to the embodiments of this application, the device plugin of each node continuously monitors the allocation and usage of computing units and memory of each hardware accelerator, and synchronizes this dynamic information global status to the cluster scheduler by updating node annotations in real time. This mechanism breaks the limitation of traditional device plugins only reporting the static resource quantity, and provides the cluster scheduler with a global, real-time and accurate resource level view required for decision-making, thus avoiding resource allocation conflicts.

[0140] Figure 5 A schematic diagram of resource scheduling according to an embodiment of this application is shown.

[0141] like Figure 5As shown, after a user submits a resource request through the Application Programming Interface (API), the API server, a service component of the cluster control plane, receives the container creation request and analyzes the resource request content according to the resource manager resource.limit. It then sends the request information to the cluster scheduler. The cluster scheduler filters nodes and hardware accelerators based on the resource requirements, node scheduling policies, hardware accelerator scheduling policies, and the real-time reported device attributes, resource usage, and resource allocation of hardware accelerators from each node. Simultaneously, it persists the API request information through a distributed key-value store. After selecting a target node, the cluster scheduler sends a request carrying the target node information to the target node's device plugin vdcu-device-plugin. The device plugin dynamically allocates the target hardware accelerator based on the resource requirements, ultimately generating complete container creation information and completing resource binding and container startup, achieving fully automated scheduling from resource request to virtualization allocation.

[0142] Each node's device plugin continuously monitors the resource allocation, usage, and remaining resources of each hardware accelerator, including computing units and memory. The real-time monitoring data is not only used for local resource allocation decisions, but also reported to the API Server in real time as node annotations after each target task is executed, providing the cluster scheduler with a global and dynamic resource view.

[0143] Based on the above resource scheduling method, this application also provides a resource scheduling apparatus. The following will be combined with... Figure 6 The device is described in detail.

[0144] Figure 6 A structural block diagram of a resource scheduling apparatus according to an embodiment of this application is shown.

[0145] like Figure 6 As shown, the resource scheduling device 600 in this embodiment includes a response module 610, a determination module 620, a splitting module 630, and a binding module 640.

[0146] The response module 610 is used to respond to resource requests for the target task and obtain resource utilization scores for each node and hardware accelerator based on the resource usage, resource allocation, and resource demand carried in the resource request of the hardware accelerators of multiple nodes in the cluster.

[0147] The determination module 620 is used to determine the target node and the target hardware accelerator of the target node among multiple nodes based on the resource utilization score and the scheduling policy carried by the resource request. The scheduling policy indicates the scheduling rules that match the resource integration requirements and / or load balancing requirements of the target task.

[0148] The segmentation module 630 is used to virtually segment the target hardware accelerator according to the resource requirements to obtain a virtual hardware accelerator.

[0149] Binding module 640 is used to bind a virtual hardware accelerator to a container on a target node in order to utilize the container to execute the target task.

[0150] According to embodiments of this application, any plurality of modules among the response module 610, determination module 620, segmentation module 630, and binding module 640 may be combined into one module, or any one of these modules may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this application, at least one of the response module 610, determination module 620, segmentation module 630, and binding module 640 may be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the response module 610, the determination module 620, the segmentation module 630, and the binding module 640 may be implemented at least partially as a computer program module that can perform corresponding functions when the computer program module is run.

[0151] Figure 7 A block diagram of an electronic device suitable for implementing a resource scheduling method according to an embodiment of this application is shown.

[0152] like Figure 7As shown, an electronic device 700 according to an embodiment of this application includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage portion 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0153] RAM 703 stores various programs and data required for the operation of electronic device 700. Processor 701, ROM 702, and RAM 703 are interconnected via bus 704. Processor 701 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 702 and / or RAM 703. It should be noted that programs may also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in one or more memories.

[0154] According to embodiments of this application, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to a bus 704. The electronic device 700 may also include one or more of the following components connected to the input / output (I / O) interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0155] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the resource scheduling method according to the embodiments of this application.

[0156] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include ROM 702 and / or RAM 703 and / or one or more memories other than ROM 702 and RAM 703 described above.

[0157] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to enable the computer system to implement the resource scheduling method provided in the embodiments of this application.

[0158] When the computer program is executed by the processor 701, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0159] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 709, and / or installed from a removable medium 711. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0160] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, it performs the functions defined in the system of this application embodiment. According to the embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0161] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0162] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0163] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A resource scheduling method, characterized in that, Applied to cluster schedulers, including: In response to a resource request for a target task, a resource utilization score is obtained for each node and the hardware accelerator based on the resource usage, resource allocation, and resource demand carried by the resource request of the hardware accelerators of multiple nodes in the cluster. Based on the resource utilization score and the scheduling policy carried by the resource request, a target node and the target hardware accelerator of the target node are determined among multiple nodes. The scheduling policy indicates scheduling rules that match the resource integration requirements and / or load balancing requirements of the target task. The target hardware accelerator is virtually partitioned according to the resource requirements to obtain a virtual hardware accelerator. The virtual hardware accelerator is bound to a container on the target node to execute the target task using the container.

2. The method according to claim 1, characterized in that, The process of obtaining resource utilization scores for each node and hardware accelerator based on the resource usage and allocation of the hardware accelerators in the cluster, as well as the resource requirements carried in the resource requests, includes: The total resource usage and total resource allocation of the node are determined based on the resource usage and resource allocation of each of the multiple hardware accelerators in the cluster. The resource utilization score of the node is obtained based on the total resource usage, total resource allocation, and first resource requirement of the node. The first resource requirement is determined based on the number of virtual hardware accelerators and the resource requirement carried in the resource request. Based on the resource usage, resource allocation, and second resource requirement of the hardware accelerator, a resource utilization score for the hardware accelerator is obtained. The second resource requirement is determined based on the resource requirement for the virtual hardware accelerator carried in the resource request.

3. The method according to claim 2, characterized in that, The resource usage includes at least one of memory usage and computing unit usage, the resource allocation includes at least one of memory allocation and computing unit allocation, and the second resource requirement includes at least one of memory requirement and computing unit requirement; The step of obtaining the resource utilization score of the hardware accelerator based on its resource usage, resource allocation, and second resource demand includes: A memory utilization score is obtained based on the ratio of the sum of the memory usage and the memory demand to the memory allocation. A computing unit utilization score is obtained based on the ratio of the sum of the computing unit usage and the computing unit demand to the computing unit allocation. The resource utilization score is obtained by weighting the memory utilization score and the computing unit utilization score.

4. The method according to claim 2, characterized in that, The scheduling strategy includes a node scheduling strategy and a hardware accelerator scheduling strategy. The step of determining the target node in the cluster and the target hardware accelerator of the target node based on the resource utilization score and the scheduling policy carried in the resource request includes: Based on the node scheduling strategy and the node's resource utilization score, a target node is determined from multiple nodes in the cluster; Based on the hardware accelerator scheduling policy and the resource utilization scores of the multiple hardware accelerators of the target node, the target hardware accelerator of the target node is determined from the multiple hardware accelerators.

5. The method according to claim 4, characterized in that, The step of determining the target node from among the multiple nodes based on the node scheduling strategy and the node's resource utilization score includes: When the node scheduling strategy is a scheduling rule that matches the resource integration requirements, the node corresponding to the highest resource utilization score among the multiple resource utilization scores is determined as the target node; When the node scheduling strategy is a scheduling rule that matches the load balancing requirements, the node corresponding to the minimum resource utilization score among the multiple resource utilization scores is determined as the target node.

6. The method according to claim 4, characterized in that, The step of determining the target hardware accelerator for the target node from among the multiple hardware accelerators based on the hardware accelerator scheduling policy and the resource utilization scores of each hardware accelerator for the target node includes: When the hardware accelerator scheduling strategy is a scheduling rule that matches the resource integration requirements, the hardware accelerator corresponding to the highest resource utilization score among the multiple resource utilization scores is determined as the target hardware accelerator. When the hardware accelerator scheduling strategy is a scheduling rule that matches the load balancing requirements, the hardware accelerator corresponding to the minimum resource utilization score among the multiple resource utilization scores is determined as the target hardware accelerator.

7. The method according to claim 1, characterized in that, The method further includes: The target device attribute carried in the resource request is determined. The target device attribute is determined from the device attributes of the multiple hardware accelerators based on the computing power requirement. The device attribute includes at least one of computing power attribute and device model. Multiple initial hardware accelerators corresponding to the target device attributes are determined from the multiple hardware accelerators, and a resource utilization score is determined based on the initial hardware accelerators.

8. The method according to claim 2, characterized in that, The method further includes: In response to the completion of the target task, the resource usage of the target hardware accelerator and the total resource usage of the target node are updated according to the resource requirements.

9. The method according to claim 2, characterized in that, The step of virtually partitioning the target hardware accelerator according to the resource requirement to obtain a virtual hardware accelerator includes: Based on the second resource requirement, a virtual partitioning strategy is generated for the target hardware accelerator. The resources of the target hardware accelerator are divided according to the virtual partitioning strategy to obtain resource partitions; The resource partition is instantiated as a virtual hardware accelerator with an independent logical interface.

10. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 9.

Images