System and method for sharing hybrid centralized distributed scheduling on physical hosts

Abstract

A system and method for hybrid centralized distributed scheduling and conflict resolution of multiple scheduler instances sharing physical resources in a cloud computing system. The cloud computing system includes multiple scheduler instances, a global resource manager (GRM) for advanced resource management and conflict resolution of the scheduler instances, and multiple physical hosts. Each physical host has a respective local resource manager (LRM). The scheduler instances are responsible for initial processing of resource requests and resource allocation, and suggest candidate physical hosts (and respective resource allocations) for the resource requests to the GRM. The GRM is responsible for resolving conflicts through its generic conflict resolver that filters, ranks, and counts. The GRM decides which of the candidate physical hosts will run runtime instances of the resource requests after resolving conflicts among the scheduler instances.

Description

Technical Field

[0001] The example embodiments relate to cloud computing, and more particularly to sharing physical resources in a cloud computing environment. Background Technology

[0002] Cloud computing is a network-based form of computing that allows access to a shared pool of configurable computing resources and higher-level services that can be rapidly provisioned with minimal management effort and are typically available to clients via the internet. Cloud computing involves client-server based computing implemented as a service. Cloud service providers typically offer three main types of services (hereinafter referred to as cloud computing services) by creating virtual machines and containers on demand for client use: Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). IaaS provides computing infrastructure that clients can rent and use. This computing infrastructure includes physical computing resources (such as processors, memory, storage devices, servers, networking components, etc.) that are virtualized and shared among clients. PaaS provides a platform that allows clients to develop, run, and manage software applications without building and maintaining the computing infrastructure and middleware. SaaS is a subscription-based service that delivers software applications running on computing infrastructure on demand via the internet.

[0003] Resource requests are typically submitted to a central scheduler instance by clients, applications, or other cloud services. Resource allocation is then performed by the central scheduler instance to shared physical computing resources, such as physical hosts. Conflicts may occur when different scheduler instances allocate the same physical resource to the same physical host multiple times, and the physical host's capacity is insufficient. Inefficiency may occur when physical host resources are insufficient or when some allocations need to be rejected due to scheduling conflicts; subsequently, some scheduler instances may wait and retry or allocate to different physical hosts.

[0004] In some existing cloud systems, available resources are provided to a single central scheduler instance at a time. This can lead to unfairness among scheduler instances, resulting in slow scheduling performance when idle resources are available. In other existing cloud systems, scheduler instances simply use and follow predefined resource-sharing policies without coordination, potentially leading to underutilization of available resources. Still other existing cloud systems rely on request and rejection processes, where resource requests are rescheduled one at a time on a single host. This is inefficient for different types of runtime instances and fails to resolve filtering or ordering conflicts.

[0005] We hope to use a master resource manager on other devices, responsible for coordinating high-level and distributed scheduling processes, to provide hybrid centralized-distributed scheduling.

[0006] We hope to provide additional scheduling processing performed by the scheduler instance and the local resource manager on each physical host. Summary of the Invention

[0007] Example embodiments relate to a method and system for hybrid centralized-distributed scheduling of concurrent heterogeneous scheduler instances sharing physical resources in a cloud computing system.

[0008] The example embodiment is a cloud computing system that includes multiple scheduler instances, a global resource manager (GRM) for high-level resource management of the scheduler instances for allocating resources to resource requests or workloads, and multiple physical hosts as runtime instances to provide resources to execute the resource requests. In the example embodiment, the GRM is responsible for high-level scheduling and conflict resolution.

[0009] It should be understood that because the central scheduler instance and GRM have determined the high-level resource allocation among physical hosts, the occurrence of scheduling conflicts at the local physical host level is reduced. The central scheduler instance does not need to rely on a request and rejection iteration model when interacting with the physical hosts.

[0010] In the example, the scheduler instance is responsible for initial processing of resource requests and resource allocation, and suggests candidate physical hosts (and corresponding resource allocations) for the resource requests to the GRM. The GRM resolves conflicts between this scheduler instance and other scheduler instances, determines which physical hosts among the candidate physical hosts will run the runtime instance of the resource request, and returns the decision to the scheduler instance. The GRM is responsible for selecting an appropriate candidate physical host to execute the runtime instance of the resource request after resolving scheduling conflicts among the candidate physical hosts proposed by the scheduler instances of these resource requests.

[0011] In the examples, the GRM can be used to identify candidate (qualified) physical hosts and filter out ineligible ones. In some examples, the filtering performed by the GRM can only be performed by the GRM itself. In other examples, the filtering performed by the GRM can supplement the initially suggested candidate physical hosts with the help of a scheduler instance.

[0012] In the example, each physical host has a corresponding local resource manager (LRM) that manages the details of local resource allocation arrangements for executing runtime instances on that physical host. The LRM on each physical host is responsible for the local scheduling and execution of runtime instances, as instructed by the central scheduler instance. Each scheduler instance can send resource allocations and runtime details to the physical host's LRM. The LRM receives resource allocations from various scheduler instances and runs the applicable runtime instances specified in the allocations and their runtime details. The LRM can also continuously report the capacity (total resource capacity), availability (available resources), and usage of the corresponding physical host to the GRM.

[0013] It should be understood that each host's LRM is responsible for managing the details of the local resource allocation arrangement for the corresponding physical host, so the GRM does not need to be responsible for all aspects of runtime scheduling.

[0014] In the examples, the scheduler instance determines a score or ranking for each candidate physical host and sends this score or ranking information to the GRM. In some examples, the score information may include weights or other applicable criteria. The GRM can use the received score or ranking information to determine the most suitable physical host for each scheduler instance's resource request (while still resolving scheduling conflicts). In some examples, the GRM is used to optimize or improve resource allocation or recovery costs when selecting physical hosts for resource requests scheduled by the scheduler instance.

[0015] In the example, there are different types of scheduler instances (called heterogeneous), which allocate resources for resource requests from different types of runtime instances executed by different types of runtime agents, and the GRM receives and processes resource allocations from these different types of scheduler instances. The GRM can make high-level resource management decisions between different types of scheduler instances, such as resolving conflicts between candidate physical hosts suggested by different types of scheduler instances for their scheduled resource requests.

[0016] It should be understood that GRM can resolve conflicts between heterogeneous scheduler instances and their heterogeneous runtime agents, and is not limited to a single type of scheduler instance or runtime agent.

[0017] It should be understood that the capacity and resource information (e.g., availability and usage information) of the relevant physical hosts can be continuously communicated to the GRM and LRM, which together facilitates the coordination of runtime instances and scheduler instances between the GRM and LRM.

[0018] An example embodiment is a method for scheduling and conflict resolution, executed by a global resource manager, comprising: receiving resource information of a physical host from a local resource manager on each of a plurality of physical hosts; receiving a corresponding candidate physical host from each of a plurality of scheduler instances, each having a corresponding resource request, the corresponding candidate physical host being adaptable to the resource allocation of the scheduler instance independently of other scheduler instances; selecting, based on the scheduler instance and the resource information, a corresponding physical host from the corresponding candidate physical hosts for at least one of the scheduler instances to resolve scheduling conflicts of the corresponding resource requests of the plurality of scheduler instances; and sending to the at least one of the scheduler instances an identifier of the selected corresponding physical host for running at least one runtime instance of the corresponding resource request scheduled by the scheduler instance.

[0019] In an example embodiment of any of the foregoing, the first at least one of the scheduler instances is for a runtime instance of a first type, and the second at least one of the scheduler instances is for a runtime instance of a second type that is of the same type as the runtime instance of the first type.

[0020] In an example embodiment of any of the foregoing, the first at least one of the scheduler instances is for a first type of runtime instance, and the second at least one of the scheduler instances is for a second type of runtime instance that is a different type from the first type of runtime instance.

[0021] In an example embodiment of any of the foregoing, the selection further includes selecting the same physical host for the first at least one and the second at least one of the scheduler instances.

[0022] In an example embodiment of any of the foregoing, the selection further includes selecting at least one of the physical hosts for the first at least one of the scheduler instances, and selecting at least one of the physical hosts for the second at least one of the scheduler instances, wherein the first at least one of the physical hosts is different from the second at least one of the physical hosts.

[0023] In an example embodiment of any of the foregoing, the method further includes reserving resources on the first at least one of the physical hosts in the first resource pool for resource requests of the first type of runtime instance, and reserving resources on the second at least one of the physical hosts in the second resource pool for resource requests of the second type of runtime instance.

[0024] In an example embodiment of any of the foregoing, the method further includes releasing the reserved resources on the first at least one or the second at least one of the physical hosts in the corresponding resource pool when the reserved resources on the corresponding physical host are no longer needed.

[0025] In an example embodiment of any of the foregoing, the method further includes migrating workloads from the first at least one of the physical hosts to another at least one of the physical hosts, such that the first at least one of the physical hosts has more resource free space, so that the resources of the respective physical hosts can be retained in the respective resource pool.

[0026] In any of the example embodiments above, the first type of runtime instance is one of a virtual machine, a container, a Pod, or a node; and the second type of runtime instance is a different type of the virtual machine, the container, the Pod, or the node.

[0027] In an example embodiment of any of the above, each scheduler instance is used to cut off unqualified physical hosts in order to identify qualified physical hosts as the corresponding candidate physical hosts.

[0028] In any of the example embodiments above, the ineligible physical host is a type of physical host that is more expensive than another type of physical host that is determined to be the ineligible physical host.

[0029] In an example embodiment of any of the foregoing, the selection of the respective physical host is performed for at least two of the scheduler instances, and the sending of the identifier of the selected respective physical host is performed for at least two of the scheduler instances.

[0030] In any of the example embodiments above, the method further includes sending the resource information to the scheduler instance after receiving the resource information.

[0031] In an example embodiment of any of the foregoing, the method further includes repeating the receipt of the resource information to monitor the at least one runtime instance.

[0032] In an example embodiment of any of the foregoing, the resource allocation includes reclaiming resources from one of the physical hosts.

[0033] In any of the above example embodiments, the resource information includes real-time actual resource usage information or scheduled resource allocation information of the corresponding physical host.

[0034] In an example embodiment of any of the foregoing, sending to at least one of the scheduler instances includes sending the number of scheduled resource allocations for the selected corresponding physical host, without sending details of the local resource allocation arrangements for the at least one runtime instance on the selected corresponding physical host.

[0035] In an example embodiment of any of the foregoing, each corresponding local resource manager receives information from the global resource manager and runtime agent on the physical host and is used to perform a local resource allocation arrangement on the physical host without receiving details of the local resource allocation arrangement.

[0036] In any of the example embodiments above, the details of the local resource allocation arrangement include local scheduling, local conflict resolution, and local optimization.

[0037] In any of the example embodiments above, the details of the local resource allocation arrangement include detailed placement of local resources, local resource sharing, isolation, binding, placement, replacement, or migration.

[0038] In an example embodiment of any of the foregoing, at least one physical host is configured with a plurality of non-uniform memory access nodes, wherein the details of the local resource allocation arrangement of the at least one physical host include migrating local resource allocation from a first non-uniform memory access node to a second non-uniform memory access node.

[0039] In an example embodiment of any of the foregoing, the method further includes receiving from each of the plurality of scheduler instances a rating or ranking information for the respective candidate physical host for the scheduler instance, wherein the selection of the respective physical host for the scheduler instance is also based on the received rating or ranking information.

[0040] In any of the example embodiments above, the scoring or ranking information is generated by the corresponding scheduler instance based on the following: expansion, stacking, application proximity, fragmentation reduction, interference reduction, or expansion resilience, or a combination or sub-combination of these.

[0041] In an example embodiment of any of the foregoing, the scoring information includes weights for ranking the respective candidate physical hosts based on one or more of the scaling, stacking, application proximity, fragmentation reduction, interference reduction, or scaling resilience.

[0042] In an example embodiment of any of the foregoing, the method further includes, before selecting the corresponding physical host from the corresponding candidate physical hosts, the global resource manager resolving rating or sorting conflicts based on the received rating or sorting information among the candidate physical hosts.

[0043] In any of the example embodiments above, the resolution of scoring or ranking conflicts is based on the cost of each of the plurality of physical hosts.

[0044] In an example embodiment of any of the foregoing, the method further includes, before selecting the corresponding physical host from the corresponding candidate physical hosts, the global resource manager filtering out at least one of the received corresponding candidate physical hosts as an unqualified physical host.

[0045] In an example embodiment of any of the foregoing, the selection of the appropriate physical host from the appropriate candidate physical hosts for the scheduler instance is also based on optimizing the resource allocation or reclamation costs of all the scheduler instances.

[0046] In any of the example embodiments above, the optimization includes optimizing the performance, isolation, and resource usage of the physical host.

[0047] In example embodiments of any of the foregoing, the scheduling conflict is resolved by scaling, stacking, application proximity, fragmentation reduction, interference reduction, or scaling resilience, or a combination or sub-combination of these.

[0048] In an example embodiment of any of the foregoing, the scheduling conflict is resolved by weighting one or more of the expansion, the stacking, the application proximity, the fragmentation reduction, the interference reduction, or the expansion resilience.

[0049] In any of the example embodiments above, the resource information includes allocation information, usage information, or both the allocation information and the usage information.

[0050] In an example embodiment of any of the foregoing, the method further includes receiving a conflict resolution request from at least one of the scheduler instances, wherein the selection is performed in small batches of the conflict resolution requests received from at least one of the scheduler instances of the respective candidate physical hosts within a specified time interval.

[0051] In any of the above example embodiments, selecting the corresponding physical host from the corresponding candidate physical hosts means selecting all physical hosts among fewer than the plurality of physical hosts.

[0052] In an example embodiment of any of the foregoing, the method is executed by at least one processor outside the physical host.

[0053] In an example embodiment of any of the foregoing, the method further includes sending to the local resource manager of the selected corresponding physical host at least one runtime instance of the selected corresponding physical host that will run the corresponding resource request.

[0054] Another example embodiment is a method for facilitating scheduling and conflict resolution, the method being executed by a scheduler instance, comprising: receiving a resource request for resource allocation; receiving resource information of a plurality of physical hosts; based on the resource information, selecting an identifier of a corresponding candidate physical host for the resource allocation of the resource request that can be adapted to resource requests of other scheduler instances independently; sending a conflict resolution request and the identifier of the corresponding candidate physical host to a global resource manager; receiving from the global resource manager an identifier of a physical host for resolving a scheduling conflict from the corresponding candidate physical host; and sending the resource allocation for the identified physical host to a runtime agent of the identified physical host to run at least one runtime instance of the resource allocation.

[0055] In an example embodiment of any of the foregoing, the selection includes filtering out ineligible physical hosts in order to identify ineligible physical hosts as the corresponding candidate physical hosts.

[0056] In any of the example embodiments above, the ineligible physical host is a type of physical host that is more expensive than another type of physical host that is determined to be the ineligible physical host.

[0057] In an example embodiment of any of the above, the method further includes scoring or ranking the corresponding candidate physical hosts and generating scoring or ranking information, and sending the scoring or ranking information to the global resource manager.

[0058] In any of the example embodiments above, the generation of the score or ranking information is based on: expansion, stacking, application proximity, fragmentation reduction, interference reduction, or expansion resilience, or a combination or sub-combination of these.

[0059] In an example embodiment of any of the foregoing, the scoring information includes weights generated based on one or more of the expansion, the stacking, the application proximity, the fragmentation reduction, the interference reduction, or the expansion resilience.

[0060] Another example embodiment is a method for facilitating scheduling and conflict resolution, performed by a local resource manager of a physical host, comprising: sending resource information of the physical host to a global resource manager; receiving a first resource allocation of a scheduled quantity from a first runtime agent or from the global resource manager, without receiving details of the local resource allocation arrangement, for the physical host to run at least one first runtime instance of the first runtime agent; receiving a second resource allocation of a scheduled quantity from a second runtime agent of a different type than the first runtime agent or from the global resource manager, without receiving details of the local resource allocation arrangement, for the physical host to run at least one second runtime instance of the second runtime agent; and performing local scheduling, local conflict resolution, and local optimization within the physical host to concurrently run the at least one first runtime instance and the at least one second runtime instance, resolving scheduling conflicts between the at least one first runtime instance and the at least one second runtime instance.

[0061] Another example embodiment is a computing system comprising: a plurality of scheduler instances, each scheduler instance being configured to receive resource requests for resource allocation; a plurality of physical hosts, each physical host having a local resource manager for sending resource information of the physical host; a global resource manager; and at least one processor, the at least one processor being configured to perform any of the methods described above.

[0062] Another example embodiment is a non-transitory computer-readable medium that stores instructions that, when executed by one or more processors, cause the one or more processors to perform any of the methods described above. Attached Figure Description

[0063] To gain a more complete understanding of the exemplary embodiments and their advantages, reference is now made to the following description taken in conjunction with the accompanying drawings, wherein:

[0064] Figure 1 This is a block diagram illustrating an example cloud computing system according to some example embodiments;

[0065] Figure 2 This is a block diagram of a cloud computing system that implements centralized distributed scheduling on a shared physical host, according to an example embodiment.

[0066] Figure 3 This illustrates the example embodiment by Figure 2 A flowchart of an example method executed by a cloud computing system;

[0067] Figure 4 This is a flowchart illustrating an example method executed by the Global Resource Manager (GRM) of a cloud computing system according to an example embodiment;

[0068] Figure 5 This is a flowchart illustrating an example method executed by a scheduler instance of a cloud computing system according to an example embodiment; and

[0069] Figure 6 This is a flowchart illustrating an example method performed by a Local Resource Manager (LRM) of one of the physical hosts according to an example embodiment.

[0070] Similar reference numerals can be used to denote similar components in different accompanying drawings. Detailed Implementation

[0071] In cloud data centers, clients (e.g., customers) need to run many different applications, which are developed, packaged, and deployed in different forms using new and old technologies, ecosystems, and runtime instances, such as virtual machines (VMs), Docker™ containers, Kubernetes™ (K8s) Pods, Java™ virtual machines (JVMs), etc. Typically, in cloud usage scenarios, the cluster has sufficient physical resources to meet all on-demand requests and run runtime instances as quickly as possible. When a certain percentage of resources have been sold to current on-demand users, more physical resources may be added to the cluster to prevent a "sold-out" situation.

[0072] A physical host is a physical hardware resource, typically a physical machine or device used to provide data or services. For example, a server can be a physical host. A virtual machine (VM) is an emulation of a computer system. A physical host can be used to run a VM. In cloud data centers, physical hosts can be used to process runtime instances based on various runtime requests.

[0073] To run runtime instances on shared physical hosts in a cluster to achieve optimal resource utilization, lowest cost, shortest request latency, and highest scheduling performance, rather than through resource silos, resource sharing is required. This allows different runtime instances to share not only the physical hosts in the cluster but also the resources within those physical hosts, such as graphics processing units (GPUs), non-uniform memory access (NUMA) / storage nodes, a central processing unit (PU), or a set of CPUs (which may include NUMA nodes or CPUs).

[0074] A scheduler instance is a module implemented by the processor that receives resource requests (e.g., from clients or applications) and retrieves those requests from the physical host. It is difficult or impractical to schedule all these heterogeneous runtime instances using a monolithic master scheduler, as they are best suited for native scheduling using different objects and settings from different ecosystems. For example, Kubernetes node / Pod affinity scheduling requests cannot be scheduled by scheduler instances of different types. For scalability and performance reasons, multiple concurrent scheduler instances are typically used for requests of the same or different types. Scheduler instances of different types can be referred to as heterogeneous.

[0075] When multiple concurrent scheduler instances exist, the cloud needs to resolve scheduling conflicts between requests competing for the same physical resources. If these instances come from heterogeneous scheduler instances, conflict resolution becomes even more difficult.

[0076] If the scheduler instance determines too many details of resource allocation on the physical host, such as CPU set, GPU, NUMA / storage node, and which CPU to bind to, then it becomes difficult to resolve conflicts.

[0077] Mesos™ Resource Manager is an existing application that supports multiple scheduler instances by providing global resources to one scheduler instance after another. There is no conflict resolution between scheduler instances. Example problems with Mesos™ Resource Manager include unfairness between scheduler instances and slow scheduling performance when idle resources exist.

[0078] Other existing applications include Hadoop™ Resource Manager and IBM™ Platform EGO, which support multiple application scheduler instances through predefined resource sharing policies among application scheduler instances. Example issues include not all global resources being available to every scheduler, and slow scheduling performance when idle resources exist.

[0079] Other existing applications, including Google™ Borg and Omega™, support multiple scheduler instances by providing global resources to multiple scheduler instances simultaneously and resolving conflicts through optimistic concurrency, attempting and retrying to allocate a host for a request one at a time. The problem is that conflict resolution is inefficient for multiple heterogeneous scheduler instances and may fail to resolve filtering and ordering conflicts.

[0080] The existing solutions described above lack local resource management capabilities that enable local scheduling, conflict resolution, and optimization within the physical host. Optimization refers to performing operations that improve outcomes, rather than being limited to optimal results. Local resource management can offload, coordinate, and balance centralized scheduling, and significantly reduce conflicts in centralized scheduling to improve the scheduling performance of large clusters.

[0081] The example embodiments relate to systems and methods for effectively and efficiently reducing and resolving filtering, ordering, and countable resource conflicts between scheduler instances at the cluster level using a processor-implemented global resource manager (GRM). The global resource manager (GRM) is used for high-level resource management by the scheduler instance. The GRM has specific processing details for scheduling distributed to other devices, and is therefore referred to herein as hybrid centralized-distributed scheduling. The GRM is responsible for high-level scheduling and conflict resolution, while other scheduling functions are performed by the scheduler instance and a local resource manager (LRM) implemented on each physical host.

[0082] In the example implementation, the scheduler instance sends a candidate list of top-level candidate physical hosts for each resource request provided by the scheduler instance to the GRM. The GRM may also receive sorting information, such as host sorting scores, and common sorting criteria from the scheduler instance.

[0083] In some examples, LRM is set up on each physical host to reduce cluster-level conflicts. LRM offloads detailed scheduling and conflict resolution from the scheduler instance and GRM to the local host level, and LRM can be used within the physical host of the LRM for local resource sharing, isolation, NUMA, CPU, GPU, and peripheral component interconnect express (PCIe) binding, placement, replacement, migration, and optimization.

[0084] In some examples, GRM can resolve scheduling conflicts of scheduler instances in mini-batches, each mini-batch containing one or more conflict resolution requests.

[0085] Some example implementations of cloud computing can be used for resource reclamation requests, which are the reverse of resource requests, to reclaim resources that have been allocated to applications in a cluster or physical host. For reclamation, an example sorting score can be the cost of reclamation, and conflicts can occur when multiple requests reclaim the same reclaimable resource on a physical host.

[0086] Figure 1This is a schematic logical block diagram illustrating the architecture of a cloud computing system (also known as cloud 100) that can provide cloud computing services. The cloud 100 shown typically includes an infrastructure platform 102 (e.g., an IaaS layer), an application platform 104 (e.g., a PaaS layer), and applications 106 (e.g., a SaaS layer). The infrastructure platform 102 includes physical hardware resources 108 and a virtualization layer 110 that presents an abstraction of the physical hardware resources 108 to the application platform 104. The abstraction presented by the virtualization layer 110 depends on the requirements of the applications 112 hosted on the application platform 104. Physical hardware resources 108 include physical machines 114 and physical memory 116. The physical machines include processing resources (e.g., central processing unit (CPU), graphics processing unit (GPU), accelerator, tensor processing unit (TPU)). The physical memory includes storage resources such as memory (e.g., static random access memory (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), read-only memory (ROM), permanent storage devices (e.g., hard disk drive, optical drive), or combinations thereof), as well as connectivity resources (not shown) typically residing within the data center. An example of connectivity resources is network resources, and any example embodiment including connectivity resources can similarly include network resources. As will be understood in the art, a data center comprises a collection of physical hardware resources 108 (typically in the form of servers) that can be used as a collection of computing resources including processing, storage, and connectivity resources. Within a data center, multiple servers can be connected together to provide a pool of computing resources on which virtualized entities can be instantiated. Data centers can interconnect to form pools of computing resources that are linked together by connectivity resources. These connectivity resources can take the form of physical connections such as Ethernet or optical communication links.

[0087] A physical host refers to a specific subset of physical hardware resources 108, typically one physical host per server. Each individual physical host comprises a subset of physical machines 114 and physical storage 116. In other examples, a server may have more than one physical host (each with its own physical hardware resources 108). A group of physical hosts may be called a cluster.

[0088] Virtualization layer 110 supports a flexible and efficient multi-tenant runtime and hosting environment for application 112 by providing Infrastructure as a Service (IaaS) facilities. Virtualization layer 110 includes a virtualization manager or hypervisor (not shown) that can provide a security and resource "sandbox" for each application 112 hosted by application platform 104. Each "sandbox" can be implemented as a virtual machine (VM) 118 in a physical host, which may include an appropriate operating system and controlled access to virtualized storage resources 120. An example implementation of the VM is a Java™ Virtual Machine (JVM).

[0089] The virtualization of physical hardware resources 108 by the virtualization layer 110 is considered a fundamental technology of the cloud 100. Virtualization is a technology that allows the creation of virtual computing resource pools connected to each computing resource (e.g., processing, storage, and connectivity resources) by connectivity resources. Virtualization can take the form of instantiating VMs 118, which are indistinguishable from physical computing devices to other entities on the network and to the software running on the VMs 118. The VMs 118 have their own set of computing resources (e.g., processing, storage, and connectivity resources) on which the operating system can run. The VMs 118 may have virtual network interfaces that can be assigned network addresses. Between the underlying resources and the VMs 118, there is typically a hypervisor (not shown) that manages resource isolation and network interaction. One purpose of the VMs 118 is to provide isolation from other processes running on the cloud 100. When initially developed, the VMs 118 were a mechanism that allowed different processes to run without concern that a single faulty process could cause the entire system to crash. Instead, the faulty process would be contained within its own VM 118. This isolation allows each VM to have its own set of network interfaces. Typically, a single underlying compute resource can support multiple virtualization entities.

[0090] Those skilled in the art will understand that containers can be used in place of VMs 118. As mentioned above, each VM 118 typically includes its own operating system, which often adds redundant computing, storage, and connectivity resources. Containers allow a single operating system (OS) kernel to support many isolated applications. Instead of a hypervisor that allows each VM 118 to run its own operating system, a single OS hosts containers that are responsible for implementing the resource isolation that would otherwise be provided by the VM 118. An example application for implementing containers is Kubernetes™ (K8s).

[0091] Application platform 104 provides the capability to host application 112 and includes application platform service 122. Application platform service 122 provides a set of middleware application services and infrastructure services to application 112 hosted on application platform 104. Application 112 hosted on application platform 104 can run on VM 118 or physical machine 114. Figure 1 In the illustrated example embodiment, application platform service 122 includes a cloud caching service system 124 for data storage in memory, a database service 126 for applications, a messaging service 128 for publishing messages to subscriber clients, and an application program interface (API) gateway service that enables clients to create, publish, and maintain APIs to access other cloud services. Those skilled in the art will understand that application 112 can provide clients with other middleware application services, such as notification services, runtime services, etc. Application 112 from the client can be deployed and executed within the corresponding VM 118 or physical machine 114.

[0092] The example resources for Cloud 100 include processing resources, storage resources, and connectivity resources. For example, any or all of these resources can be allocated for instantiating VM 118 or for implementing container applications. In some examples, resources can be reclaimed, which can be considered as reversing or subtracting from a specific resource allocation of one of these resources.

[0093] Cloud100 workloads refer to discrete capabilities or workloads that are intended to run or are running as runtime instances on Cloud100. For example, workloads can be used for web servers, containers, or Hadoop™ nodes.

[0094] Figure 2 This is a block diagram of Cloud 100, which implements hybrid centralized-distributed heterogeneous scheduling of multiple scheduler instances on a shared physical host, according to an example embodiment. (See diagram for example.) Figure 2 As shown, Cloud 100 includes GRM 200, multiple scheduler instances 202, and multiple physical hosts 204. In some examples, physical hosts 204 operate in parallel. "Hybrid centralized-distributed" refers to the GRM 200 performing globally centralized processing of certain aspects of scheduling, resource management, and conflict resolution, while other local aspects of scheduling and resource management are performed by other devices (such as scheduler instances 202 and physical hosts 204) directly affected by these local aspects. In some examples, GRM 200 includes a generic filter conflict resolver, a generic sorting program conflict resolver, and a generic counter conflict resolver.

[0095] In an example embodiment, each physical host 204 includes an LRM 206. A GRM 200 works with each LRM 206 and acts as a centralized scheduler and resource manager to coordinate required resource allocations and resolve conflicts between scheduler instances 202. The LRM 206 of each physical host 204 is used to perform detailed runtime instance scheduling of the applicable allocated resources within the corresponding physical host 204. The LRM 206 of each physical host 204 can perform local scheduling, conflict resolution, and optimization within the corresponding physical host 204. Therefore, in some example embodiments, neither the GRM 200 nor the scheduler instance 202 is responsible for the detailed scheduling of local resources of the physical host 204 for runtime instances.

[0096] Figure 2 Details of a physical host 204(1) are shown, including an LRM 206 and one or more runtime agents, such as runtime agent 1 (208(1)) and runtime agent 2 (208(2)). Runtime agents 208(1) and 208(2) can each execute one or more corresponding runtime instances using the physical resources of physical host 204. In this example, runtime agent 1 (208(1)) executes multiple corresponding runtime instances (runtime 1 (210(1))) against physical host 204(1), and runtime agent 2 (208(2)) executes multiple corresponding runtime instances (runtime 2 (210(2))) against the same physical host 204(1). The resources of runtime 1 and runtime 2 do not overlap on physical host 204(1), and therefore can run concurrently without scheduling conflicts. Other physical hosts 204 can be configured in a similar manner. GRM 200 coordinates the high-level selection of which physical host 204 will execute which runtime instance, and the amount of resource allocation to be performed on said physical host 204, to run the appropriate runtime instance. Runtime agents 208(1) and 208(2) each negotiate with LRM 206 the detailed implementation of their runtime instances. In some examples, runtime 1 (210(1)) and runtime 2 (210(2)) are used to execute different types of runtime instances, which may be referred to as heterogeneous runtime instances.

[0097] like Figure 2As shown, scheduler instance 202 includes a plurality of first scheduler instances of a first type, one of which is shown as scheduler 1 (202(1)). Scheduler instance 202 includes a plurality of second scheduler instances of a second type, one of which is shown as scheduler 2 (202(2)). In the example embodiment, the first type of scheduler instance and the second type of scheduler instance are scheduler instances of different types. For example, the first type of scheduler instance may be an OpenStack™ virtual machine (VM); and the second type of scheduler instance may be a Kubernetes Pod or container. The different types of scheduler instances may be referred to as heterogeneous scheduler instances. In the example embodiment, scheduler instance 202 may include even further different types of scheduler instances (not shown here).

[0098] In the example embodiment, each scheduler instance 202 communicates with a corresponding runtime agent 208(1), 208(2), which is used to specifically handle the particular type of scheduler instance. Furthermore, each scheduler instance 202 can receive usage, availability, and capacity data related to all physical hosts 204 from the GRM 200 to evaluate candidate physical hosts and perform scheduling for the required resource allocation. Figure 2 In the example, scheduler 1 (202(1), which may be called S1) is used to communicate with runtime agent 1 (208(1)) and scheduler 2 (202(2), which may be called S2) is used to communicate with runtime agent 2 (208(3)).

[0099] Different types of scheduler instances 202 receive the current global resource availability and usage status of all physical hosts in the cluster from GRM 200. In cloud computing systems, resource scheduler instances typically perform three steps to schedule resource requests: a) filtering to obtain a list of qualified physical hosts that can meet all constraints and have sufficient remaining resources; b) sorting the (qualified) physical hosts; and c) selecting a physical host from the sorted list.

[0100] Scheduler instance 202 can be used to receive resource request 212 (the resource request may be referred to as "rereq"). Figure 2 In the example, scheduler 1 (202(1)) and other scheduler instances of the first type can receive resource request 1 (212(1)) of the first type to start a runtime instance of the first type. Similarly, scheduler 2 (202(2)) and other scheduler instances of the second type can receive resource request 2 (212(2)) of the second type to start a runtime instance of the second type.

[0101] In some examples, each scheduler instance 202 is also used to receive capacity and resource information (e.g., availability and usage information) related to all physical hosts 204 from GRM 200. Scheduler instance 202 receives the global resource usage status of all physical hosts 204 in a given cluster from GRM 200 and schedules resource requests for different types of runtime instances. In example embodiments, scheduler instance 202 may allocate or reclaim resources. In some examples, scheduler instance 202 may have two or more types.

[0102] Then, scheduler instance 202 can satisfy the received resource request 212 by first suggesting to GRM 200 which physical hosts 204 can run the resource request 212; these physical hosts are called candidate physical hosts. The suggestion from scheduler instance 202 to GRM 200 also includes the amount of resource allocation and the type of runtime instance. In some examples, each scheduler instance 202 is used to determine which of the physical hosts 204 is a candidate physical host that can accommodate the required resource allocation of the scheduler instance 202, based on the type, capacity, and resource information of the physical host. At this stage, scheduler instances 202 do not coordinate with each other, and the possible candidate physical hosts are determined by each scheduler instance 202 independently of the other scheduler instances 202.

[0103] In some examples, each scheduler instance 202 first filters out ineligible physical hosts 204, such as those that cannot handle a certain type of request or do not have sufficient capacity to handle a certain type of request. In some examples, scheduler instance 202 is used to determine and provide GRM 200 with additional scoring or ranking information about eligible candidate physical hosts. For example, scheduler instance 202 may score or rank physical hosts 204 (or alternatively, only eligible candidate physical hosts 204) based on resource request 121, the real-time or scheduled capacity of each physical host 204, availability, or usage data, and send the scoring or ranking information of candidate physical hosts 204 to GRM 200 for further consideration so that GRM 200 can make a final high-level scheduling decision.

[0104] In some examples, scheduler instance 202 may also filter physical hosts 204 as candidate physical hosts to present to GRM 200 based on, for example, a specific cut-off threshold or a specified number of candidate physical hosts.

[0105] GRM 200 is responsible for resolving scheduling and resource allocation conflicts between scheduler instances. In some examples, GRM200 can be used to optimize resource allocation or recover costs.

[0106] GRM 200 receives from each scheduler instance 202 a plurality of corresponding candidate hosts for scheduling each resource request for scheduler instance 202. GRM 200 resolves filtering, sorting, and counting conflicts between scheduler instances for their respective resource requests and determines which physical host 204 will run the runtime instance for each resource request. GRM 200 then sends the conflict resolution result back to scheduler instance 202, the conflict resolution result including an identifier of which physical host 204 will run the runtime instance for the resource request. In some embodiments, such as Figure 2 As shown, scheduler 1 202(1) can be the default scheduler. In some example embodiments, GRM 200 and default scheduler instance 202(1) can be implemented in a single component, allowing them to share the same filtering, sorting, and counting framework.

[0107] In the example embodiment, scheduler instance 202 communicates with runtime agents 208(1), 208(2) of its corresponding type to launch runtime instances 210(1), 210(2) of different types using resources and resource allocation quantities scheduled by scheduler instance 202 and determined by conflict resolution of GRM 200.

[0108] LRM 206 is responsible for the details of resource allocation and local scheduling of runtime instances on each physical host 204. LRM 206 on each physical host 204 ensures that the resources allocated to runtime instances 210(1), 210(2) on physical host 204 do not exceed the host's resource capacity. LRM 206 also coordinates between different runtime agents 208(1), 208(2) and makes local scheduling decisions to place or replace runtime instances on NUMA, CPU, GPU, memory, and PCIe devices in the local physical host 204 to optimize the performance, isolation, and resource usage of requested workloads on the local physical host 204. Each LRM 206 communicates with GRM 200 to report its "heartbeat," including usage or resource information (actual or scheduled), and synchronizes the resource allocation determined by GRM 200 with the actual usage on the local physical host 204.

[0109] The runtime agents 208(1), 208(2) of different types on each physical host 204 communicate with the LRM 206 and its corresponding scheduler instances 202(1), 202(2). The runtime agents 208(1), 208(2) work with the operating system (OS) and virtualization and container subsystems on the local physical host 204, and locally start and monitor the runtime instances of their corresponding types for the scheduler instances 202(1), 202(2).

[0110] LRM 206 collects information about the physical resources and topology of its respective host and reports the information to GRM 200 for global resource scheduling and conflict resolution. In example embodiments, LRM 206 may directly invoke operating system commands to collect resources, or LRM 206 may interact with each runtime agent 208(1), 208(2) to collect resource information. In some example embodiments, LRM 206 and runtime agents 208(1), 208(2) may be implemented in a single component.

[0111] Scheduler instances 202(1) and 202(2) receive the current global resource availability and usage status of all physical hosts 204 in the cluster from GRM 200. In the example embodiment, scheduler instances 202(1) and 202(2) may still have their own ecosystem-specific resource filters to filter out non-candidate physical hosts and determine candidate physical hosts. Each scheduler instance 202(1) and 202(2) has a corresponding sorting procedure to score candidate physical hosts and then sort the candidate physical hosts based on their scores to satisfy resource allocation or reclamation requests.

[0112] In the example embodiment, each scheduler instance 202(1), 202(2) can cut off the candidate list of the requested top-level candidate host and send the candidate list to GRM 200 for conflict resolution. For resource allocation, GRM 200 ensures that the countable resources (e.g., CPU cores, memory, GPU, IO bandwidth, storage space) on physical host 204 do not become negative after deducting the amount of resource allocation required from all resource requests from all scheduler instances 202.

[0113] In the example embodiment, GRM 200 may have eligibility rules to resolve filtering conflicts, filtering out or disqualifying one or more received candidate physical hosts from the candidate lists received from each scheduler instance 202(1), 202(2). For example, if some resources, such as GPUs, are more expensive than others, such as CPUs, then there might be a general rule in GRM 200 that "any physical host with a large number of unused GPUs is disqualified from allocating resource requests that do not require GPUs." Generally, workloads require at least some CPU and memory to run. Only some specific workloads require GPUs in addition to CPU and memory. Such a general rule can prevent a particular GPU physical host from exhausting its CPU or memory, while some of its expensive GPUs are left unused and wasted.

[0114] In the example embodiment, scheduler instances 202(1) and 202(2) can determine the corresponding final total score "SchedulerSortScore" for each candidate physical host 204 in the candidate list and send the corresponding final total score to GRM 200 to show how much better the physical host 204 is than other hosts in terms of resource allocation or reclamation. For reclamation, a preferred physical host 204 means a lower reclamation cost, and each physical host 204 can include a list of existing candidate allocation requests to be preempted. In this way, GRM 200 can select hosts from the list from top to bottom and randomly select hosts from some physical hosts 204 in the candidate list with the same score.

[0115] In the example embodiment, scheduler instances 202(1), 202(2) may further include a set of common ranking criterion scores for each physical host 204 from the candidate lists of scheduler instances 202(1), 202(2), such that GRM 200 does not need to score the physical hosts 204 using any GRM ranking procedure. For example, example criteria for scoring include spreading, stacking, application proximity, fragmentation reduction, interference reduction, or spreading resilience (referred to as the following variables or policies: Spreading, Stacking, AppProximity, ReduceFragment, ReduceInterference, SpreadResilience, respectively). In the example embodiment, all scheduler instances 202(1), 202(2), and GRM 200 are aware of the scoring. GRM 200 may optionally have its own GRM ranking procedure to score physical hosts based on these example criteria and take into account its own weights to combine scores according to multiple criteria into a single final ranking score. Thus, GRM 200 resolves ranking policy conflicts between scheduler instances and GRM 200. For example, if some scheduler instances prefer Spreading, but the current GRM 200 policy gives more weight to Stacking, then GRM 200 is used to rank physical hosts in the candidate list for resource requests based on the GRM 200 calculation for each host, where the current weight of each host is based on... <SchedulerSortScore,Spreading,Stacking,AppProximity,ReduceFragment,ReduceInterference,SpreadResilience> The weights on the stacking are greater than those on the score vector. Each scheduling instance 202(1), 202(2) may have a sorting procedure to generate its own sorting score "SchedulerSortScore" based on similar vector weights or other criteria. In some examples, GRM 200 treats the sorting score ("SchedulerSortScore") of a single scheduler instance 202(1), 202(2) more carefully than other items, and may also add more weights to the SchedulerSortScore of one or more other scheduler instances to guide the preferred direction of GRM200.

[0116] In the example embodiment, scheduler instances 202(1) and 202(2) may send small batches of such scheduling requests to GRM 200.

[0117] When receiving a candidate list of candidate hosts for scheduling requests from scheduler instances 202(1) and 202(2), GRM 200 can resolve scheduling conflicts between scheduler instances 202(1) and 202(2) for different types of concurrent resources. In some examples, GRM 200 can resolve one request at a time using optimistic concurrency. In examples, GRM 200 can also resolve scheduling conflicts in mini-batch manner. For example, after receiving a set of candidate lists from scheduler instances 202(1) and 202(2) at specified resolution intervals or configurable numbers of candidate lists, GRM 200 can optimize conflict resolution and physical host selection to collect requests received from any or all scheduler instances 202, based on the first-come, first-served principle. In some examples, requests are independent of each other. If the set contains mixed allocation and reclamation requests, GRM 200 can consider allocation and reclamation together to maximize the combined score (for reclamation, optimization means to minimize or reduce reclamation costs), including allowing allocation requests to use resources reclaimed from reclamation requests.

[0118] After resolving the scheduling conflict, GRM 200 sends the identifier of the corresponding physical host 204 and any results of the scheduling conflict resolution back to scheduler instances 202(1) and 202(2).

[0119] Each scheduler instance 202(1), 202(2) communicates with the corresponding type of runtime agent 208(1), 208(2) to launch different types of runtime instances 210(1), 210(2) using resources scheduled by the corresponding scheduler instance 202(1), 202(2) and determined by GRM 200.

[0120] To achieve better performance, reduce conflicts, and avoid complex and slow centralized scheduling within the scheduler instance and GRM 200, the sorting algorithm in GRM 200 is kept as simple and fast as possible, with some detailed low-level resource scheduling handled by LRM 206 in physical host 204. The combination and balance of centralized and local scheduling can be termed hybrid distributed scheduling. In example embodiments, the boundary between centralized and local scheduling can be adjusted based on applicability, production system settings, and requirements. In some example embodiments, GRM 200 can automatically (or configurably) adjust the responsibility boundary between LRM 206 and GRM 200 to further optimize performance.

[0121] Central scheduling in scheduler instances 202 and GRM 200 can be maintained at a high level to limit and quantify host selection (e.g., the amount of CPU sets, CPU cores, GPUs, memory, NUMA nodes, and PCIe devices) without requiring micro-scheduling to optimize the detailed placement of local resources (e.g., NUMA nodes, memory, CPU sets, CPU cores, GPUs, PCIe devices and their topology, affinity, and proximity) for each request within physical host 204. This simplicity can significantly reduce the chance of conflicts in central scheduling at the global level of the cluster.

[0122] LRM 206's local scheduling on each physical host 204 can perform low-level scheduling (micro-scheduling) to optimize the detailed placement of local resources (such as NUMA nodes, memory, CPU sets, CPU cores, GPUs, PCIe devices and their topology, affinity, and proximity) for each request within the physical host 204. LRM 206 also works with the runtime agent, OS, and virtualization / container subsystems within the host to coordinate and manage detailed local resource sharing, isolation, binding, placement, replacement, and migration for requests within the host. Because the scale of local scheduling optimization and conflict resolution is reduced from the cluster to the physical host 204, Cloud 100 with LRM 206 performs more efficiently compared to without it.

[0123] Each LRM 206 on physical host 204 ensures that the resources allocated to runtime instances on physical host 204 do not exceed the host's resource capacity. It coordinates between different runtime agents and makes local scheduling decisions to place / replace / migrate runtime instances on local resources within the local physical host 204 (e.g., NUMA nodes, memory, CPU sets, CPU cores, GPUs, PCIe devices and their topology, affinity, and proximity) to optimize performance, isolation, and resource utilization. Each LRM 206 communicates with GRM 200 to report the "heartbeat" of the local host 204 (resource information such as availability and usage information allocated by the scheduler and actually used by the runtime instances) and synchronizes the resource allocation scheduled by the scheduler instance and determined by GRM 200 with the actual usage on the local physical host 204.

[0124] To optimize workload performance through local scheduling, in an example embodiment, scheduler instance 202 may send workload-NUMA affinity information to LRM 206 in a request, so that LRM 206 may attempt to satisfy workload-NUMA requirements by bringing relevant workloads close to each other for local execution in physical host 204.

[0125] Different types of runtime agents 208(1), 208(2) on each physical host 204 are used to communicate with LRM 206 and its corresponding scheduler instances 202(1), 202(2), work with the OS and virtualization and container subsystems on the local physical host 204, and start and monitor different types of runtime instances 210(1), 210(2) such as virtual machines and containers locally for their corresponding scheduler instances 202(1), 202(2).

[0126] In the example, GRM 200 can have a leader instance and multiple parallel worker instances. Multiple parallel worker instances can resolve performance and scalability conflicts in parallel, such as one instance per partition of physical host 204. The leader instance can be used to coordinate the final resolution determined by the multiple parallel worker instances.

[0127] In the example, different types of scheduler instances 202(1) and 202(2) can be configured with advanced sharing strategies for resource sharing and conflict resolution, such as based on elastic pools, schedulers, workloads, and priorities.

[0128] Figure 3 This is a flowchart illustrating an example method 300 according to an example embodiment, the example method including messages executed by cloud 100 for centralized distributed scheduling of multiple scheduler instances 202 and GRM 200. While some steps and messages are shown sequentially and discretely, in some example embodiments, some steps and messages are executed concurrently. In some example embodiments, some steps and messages are executed continuously, intermittently, or repeatedly. In example embodiments, a particular step does not necessarily need to wait for the next step to complete before repeating or iterating that step; for example, due to an ongoing resource request or information update, the step may be repeatedly received and processed before or simultaneously with the next step.

[0129] Cloud 100 internally transmits capacity and resource information (e.g., availability and usage information) related to the physical resources of each physical host 204. Availability information may include the available resources of a specific physical host 204; simply put, the available resources can generally be considered as the total resource capacity of that type of resource on physical host 204 minus the usage of that type of resource executed on physical host 204. GRM 200 is continuously informed of usage information, availability information, or both for each physical host 204. Usage information may be the actual real-time usage information of each physical host 204, the scheduled usage information of physical host 204, or both. Figure 3The example details the steps of a physical host 204 (1), and those skilled in the art will understand that cloud 100 can perform similar processes with other physical hosts 204 and can run scheduler instance 202 and run runtime instances across different physical hosts 204 in the cluster.

[0130] exist Figure 3 In the diagram, large white arrows indicate the transmission of resource information (e.g., availability and / or usage information) to illustrate that transmission can be performed continuously or intermittently throughout method 300. In step 302, runtime agent 2 (208(2)) sends its resource information to LRM 206. In step 304, runtime agent 1 (208(1)) sends its resource information to LRM 206. One or both of runtime agents 1 (208(1)) and runtime agent 2 (208(2)) may also send the total capacity of physical host 204(1) to LRM 206 during the corresponding steps 302 and 304. In other examples, the total capacity of physical host 204(1) is sent at an initial time, or periodically, or LRM 206 already knows this total capacity. In step 306, LRM 206 sends the resource information to GRM 200. The resource information in step 306 may be resource information received from runtime agents 208(1) and 208(2), or it may be resource information determined based on other processes or runtime instances running on physical host 204(1). The resource information in step 306 may be actual or scheduled resource information. GRM 200 receives resource information from all LRMs on all physical hosts 204.

[0131] In step 308, GRM 200 sends resource information to scheduler 2 (202(2)). In step 310, GRM 200 sends resource information to scheduler 1 (202(1)). Availability or usage information is related to all physical hosts 204. In step 312, scheduler 1 (202(1)) receives resource request 1 (212(1)), see [link to relevant documentation]. Figure 2 Using the received resource information or the required resource type compared to the type of each physical host 204, scheduler 1 determines which physical hosts 204 are suitable for the resource allocation of the resource request; these physical hosts are called candidate (qualified) physical hosts. At this point, the selection of those candidate physical hosts is independent of the resource requests of other scheduler instances 202. Other criteria can be used to add candidate physical hosts to the candidate list. Additional rating or ranking information for each candidate physical host can also be determined from scheduler 1 (202(1)) at this time; for example, ranking information is determined from the ranking of candidate physical hosts based on their total score or other rating information.

[0132] In step 314, scheduler 1 (202(1)) sends the identifiers of the candidate physical hosts to GRM 200. In some examples, scheduler 1 (202(1)) also sends rating or ranking information for each candidate physical host to GRM 200. For example, the rating information could be a single weight or total score for each candidate physical host. For example, the ranking information could be a ranked list of candidate physical hosts. Step 316 can be the same as step 312 performed by scheduler 2 (202(2)), which generates corresponding candidate physical hosts for scheduler 2 (202(2)). Step 318 can be the same as step 314 performed by scheduler 2 (202(2)).

[0133] In step 320, GRM 200 has now received the corresponding candidate physical host from each of scheduler instances 202 (e.g., scheduler 1 (202(1)) and scheduler 2 (202(2))), and received the rating or ranking information of the candidate physical hosts from each of scheduler instances 202. In step 320, GRM 200 selects the corresponding physical host from the corresponding candidate physical hosts for each of the scheduler instances 202. The specific decision for selecting the corresponding physical host may be based on all receive requests from scheduler instances 202 and capacity and resource information for resolving scheduling conflicts in scheduler instances 202. The specific decision for selecting the corresponding physical host may also be based on the actual or scheduled usage information of each physical host 204.

[0134] In some example embodiments, step 320 may be performed in a batch manner, for example, after a threshold of candidate physical hosts has been collected from scheduling instance 202 or after a specified resolution time interval.

[0135] In some example embodiments, GRM 200 performs filtering and determines which physical hosts will be candidate or qualified physical hosts. In some examples, GRM 200 performing filtering may be an alternative to steps 312 and 316, where scheduler instance 202 does not perform any filtering. In most examples, GRM 200 performing filtering may be the final decision in step 320 or further filtering, which takes into account the initial filtering proposed by each of scheduler instance 202 in steps 312 and 316.

[0136] exist Figure 3 In the example shown, scheduler 1 (202(1)) and scheduler 2 (202(2)) both select the same physical host 204(1). Physical host 204(1) can execute different types of runtime instances for different types of scheduler instances. In other example embodiments not shown here, a different physical host 204 is selected for each of scheduler 1 (202(1)) and scheduler 2 (202(2)).

[0137] In step 322, GRM 200 sends the identifier of the selected physical host 204(1) to scheduler 1 (202(1)). In step 324, GRM 200 sends the identifier of the selected physical host 204(1) to scheduler 2 (202(2)). In some examples (not shown here), GRM 200 sends the identifier of the selected physical host 204(1) to LRM 206 of the selected physical host 204(1).

[0138] In step 326, scheduler 1 sends a resource request (e.g., from resource request 1 (212 (1))) for a specific resource request runtime to runtime agent 1 (208 (1)) of the identified physical host 204 (1). In step 328, scheduler 2 sends a resource request (e.g., from resource request 2 (212 (2))) for a runtime instance of a specific resource request to runtime agent 2 (208 (2)) of the identified physical host 204 (1).

[0139] Steps 330 and 332 can be performed simultaneously in the example embodiment. In step 330, runtime agent 1 (208(1)) runs the runtime instance using allocated resources. Figure 3 As shown, step 330 may include runtime agent 1 (208(1)) working with LRM 206. For example, LRM 206 is used to locally schedule the scheduling and runtime instance execution of resource requests received from scheduler 1 (202(1)). LRM 206 implements local resource allocation arrangement details, including local scheduling, local conflict resolution, and local optimization. In some examples, local resource allocation arrangement details may include detailed placement of local resources, local resource sharing, isolation, binding, placement, replacement, or migration. LRM 206 can schedule heterogeneous runtime instances. LRM 206 performs local resource allocation and may inform runtime agent 1 (208(1)) which resources to use and when to use them. Similarly, step 332 may be performed by runtime agent 2 (208(2)) in collaboration with LRM 206. For runtime instances of a different type from runtime agent 1 (208(1)), step 332 may be performed concurrently with step 330.

[0140] In step 334, LRM 206 sends resource information to GRM 200 related to actual usage information from runtime instances implementing runtime 1 and runtime 2. It is understood that step 334 can be an example instance of step 306.

[0141] The following are example implementations of specific implementations of Cloud 100 described in more detail. The first example implementation illustrates how scheduler instance 202 and GRM 200 work together. The second example implementation illustrates how GRM 200 resolves filtering and sorting conflicts. According to the embodiment, sorting conflicts may also refer to scoring conflicts. The third example implementation illustrates how LRM 206 performs resource gatekeeping, local resource scheduling, and best-effort optimization on its local physical host 204. The fourth example implementation uses an advanced sharing policy that utilizes elastic pool-based sharing configured across different types of scheduler instances 202, which can help resolve scheduling conflicts. The fifth example implementation uses scheduler, workload, and priority-based sharing configured across different types of scheduler instances 202, which can help resolve scheduling conflicts.

[0142] The first example implementation will now be described, illustrating how scheduler instance 202 and GRM 200 work together.

[0143] The OpenStack scheduler instance S1 obtains the current global resource usage status of physical hosts 204 in the cluster from GRM 200, and retrieves the resource request ResReq1 (2 cores, 4GB) in terms of CPU cores and GB of memory. S1 uses its filters to obtain a list of qualified physical hosts that meet all constraints and have sufficient remaining resources. S1 uses its sorting algorithm to sort qualified physical hosts with sufficient remaining CPU cores and memory, such as Host2 (4 cores, 8GB), Host1 (8 cores, 16GB), and Host3 (16 cores, 32GB), where the sorting strategy is to pack physical hosts as compactly as possible (minimum remaining resources). S1 cuts the top two candidates, Host2 (4 cores, 8GB) and Host1 (8 cores, 16GB), back to GRM200 to resolve conflicts.

[0144] The Kubernetes scheduler instance S2 receives the current global resource usage status of physical hosts in the cluster from GRM 200, receiving resource requests ResReq2 (4 cores, 8GB) in terms of CPU cores and GB of memory. S2 uses its filters to obtain a list of qualified physical hosts that meet all constraints and have sufficient remaining resources. S2 uses its sorting algorithm to sort qualified physical hosts with sufficient remaining CPU cores and memory, such as Host2 (4 cores, 8GB), Host3 (16 cores, 32GB), Host4 (16 cores, 32GB), and Host5 (32 cores, 64GB), where the sorting strategy is to pack physical hosts as compactly as possible (minimum remaining resources). S2 cuts the top 3 candidates, namely Host2 (4 cores, 8GB), Host3 (16 cores, 32GB), and Host4 (16 cores, 32GB), back to GRM 200 to resolve conflicts.

[0145] GRM 200 receives the candidate list of ResReq1 from S1. GRM 200 finds that the current remaining resources of the relevant physical hosts in a global resource usage state are Host2 (4 cores, 8GB) and Host1 (6 cores, 12GB). GRM 200 decides to allocate the resources of Host2 to ResReq1 (2 cores, 4GB) and deduct them from Host2, so that the remaining resources of Host2 become Host2 (2 cores, 4GB) in a global resource usage state.

[0146] The GRM 200 receives the candidate list for ResReq2 from S2. The GRM 200 finds that the currently remaining resources of the relevant physical hosts in a global resource usage state are Host2 (2 cores, 4GB) and Host3 (16 cores, 32GB). Since Host2 cannot currently satisfy ResReq2 (4 cores, 8GB), the GRM 200 decides to allocate the resources of Host3 to ResReq2 and deduct them from Host3, making the remaining resources in a global resource usage state Host2 (2 cores, 4GB), Host1 (6 cores, 12GB), Host3 (12 cores, 24GB), and Host4 (16 cores, 32GB).

[0147] The GRM 200's selection of Host2 and Host3 is made when the GRM 200 sequentially parses the candidate lists of S1 and S2, or when they are missed within the same mini-batch time window. In another example embodiment, the GRM 200 can use mini-batch processing for resource requests. If resource requests from S1 and S2 are received within the same mini-batch time window, the GRM 200 can perform some optimizations, such as using a minimum host maximum request priority packing strategy based on the current state of Host2 (4 cores, 8GB), Host1 (6 cores, 12GB), Host3 (16 cores, 32GB), and Host4 (16 cores, 32GB). This involves prioritizing small hosts in the mini-batch and then prioritizing large requests, deciding to allocate resources from Host2 to ResReq2 (4 cores, 8GB) and resources from Host1 to ResReq1 (2 cores, 4GB). Note that Host1 does not meet the conditions of ResReq2, and neither do Host3 and Host4. After deduction, the remaining resources of Host2 (0 cores, 0GB), Host1 (4 cores, 8GB), Host3 (16 cores, 32GB), and Host4 (16 cores, 32GB) are in a state of global resource usage. Compared to the result of parsing the candidate lists of S1 and S2 one by one, there are fewer remaining memory fragments.

[0148] The first exemplary implementation allows heterogeneous scheduler instances 202(1), 202(2) and their runtime instances to share physical host 204 by combining central parallel scheduling and local micro-scheduling, which can greatly improve resource utilization and scheduling performance, and reduce resource costs and request waiting time.

[0149] Existing heterogeneous scheduler instances and their runtime instances in different ecosystems can be easily adapted to GRM 200 and scheduler instance 202 as microservices without having to be reimplemented. Integrating GRM 200 and scheduler instance 202 saves design and engineering effort and allows them to continue to evolve within their ecosystems.

[0150] The second example implementation will now be described, illustrating how GRM 200 resolves filtering and sorting conflicts.

[0151] For the resource request ResReq (2 cores, 4GB), scheduler instance 202(1) uses a diffusion sorting strategy to cut off the top 3 in the candidate list, namely Host3 (16 cores, 32GB), Host4 (8 cores, 16GB), and Host2 (4 cores, 8GB), to GRM 200. Scheduler instance 202(1) also has a sorting procedure that generates a sorting process for each candidate physical host 204 in the candidate list.<SchedulerSortScore,Spreading,Stacking> The score vector. For example, the scheduler instance 202(1) sends ScoreVector(Host3) = <9,7,3>, ScoreVector(Host4) = <8,5,4>, and ScoreVector(Host2) <7,3,6> to GRM 200.

[0152] In one example, the GRM 200 has a filtering conflict resolution strategy for eligibility rules to resolve filtering conflicts and disqualify candidate physical hosts. For example, this rule could be, "Any physical host with many unused GPUs will be disqualified from allocating resource requests that do not require GPUs." Such a rule prevents expensive unused GPUs on a physical host from being wasted because other, less important resources, such as CPUs, are being exhausted on that physical host. By filtering candidate physical hosts, the GRM 200 finds that Host4 has unused GPUs, therefore Host4 is disqualified. The GRM 200 also has a sorting conflict resolution strategy that prioritizes Stacking for sorting and does not run its own sorting process.<SchedulerSortScore,Spreading,Stacking> Compared to other items, the GRM 200 sorting strategy places more weight on stacking. For example, WeightOnScoreVector<10,2,20> treats SchedulerSortScore cautiously but favors stacking. The final scores for each candidate physical host are then: FinalScore(Host3) = 10*9 + 2*7 + 20*3 = 164; FinalScore(Host2) = 10*7 + 2*3 + 20*6 = 196. Based on the final score ranking, ResReq (2 cores, 4GB) is likely assigned to Host2, which has the highest score of 196.

[0153] In the second example implementation, GRM 200 resolves filtering and sorting conflicts and has a consistent scheduling filtering and sorting strategy, while still carefully considering the sorting score of each scheduler instance and other common sorting criteria.

[0154] In the example, GRM 200 resolves scoring or ranking conflicts based on the received scoring or ranking information among the corresponding candidate physical hosts before selecting the final physical host from the corresponding candidate physical hosts. In the example, GRM 200 resolves scoring or ranking conflicts based on the cost of each of the multiple physical hosts. In the example, GRM 200 filters out at least one of the received corresponding candidate physical hosts as an unqualified physical host before selecting the final physical host from the corresponding candidate physical hosts.

[0155] The third example implementation will now be described, illustrating how LRM 206 performs resource gatekeeping, local resource scheduling, and best-effort optimizations on the corresponding local physical host 204 to achieve workload-NUMA affinity on physical host 204. The example implementation assumes that physical host 204 has sufficient memory for the VM, so only CPU cores need to be considered.

[0156] Physical host 204(1) has two NUMA nodes, namely NUMA1 and NUMA2, each with eight cores. NUMA1 has two cores in use and six cores available for VM1. NUMA2 has four cores in use and four cores available for VM1.

[0157] A new VM with 4 cores is preferentially allocated to this physical host 204(1) on a NUMA node. LRM 206 places this VM on NUMA2. Now, NUMA1 has 6 cores available. NUMA2 has 0 cores available.

[0158] A new VM with 10 cores will be allocated to this host. LRM 206 sees that the 10 cores required for this VM on this host are greater than the 6 cores currently available, and refuses to start this VM on this physical host 204(1), which may have been incorrectly scheduled to this physical host 204(1) due to some temporary data synchronization error.

[0159] The two-core VM previously running on NUMA2 is now complete. NUMA1 now has six cores available. NUMA2 now has two cores available.

[0160] A new VM2 with 8 cores is preferentially assigned to this physical host 204(1) on one or two NUMA nodes. LRM 206 checks whether VM1 on NUMA1 has a NUMA preference and uses a large amount of memory. If it does, then LRM 206 places VM2 on both NUMA1 and NUMA2. If it does not, then LRM 206 migrates VM1 from NUMA1 to NUMA2 and places VM2 on NUMA1.

[0161] The third example balances centralized scheduling performed by GRM 200 with local scheduling performed by LRM 206 to achieve local optimization of centralized scheduling and conflict resolution performance, as well as workload performance and resource utilization.

[0162] A fourth example implementation using an advanced sharing strategy will now be described, which utilizes elastic pool-based sharing configured between different types of scheduler instances 202, which can help resolve scheduling conflicts.

[0163] In a cluster consisting of many physical hosts 204, scheduler instance 202(1) of scheduler type1 maintains a host pool POOL1 that initially contains 0 hosts. Scheduler instance 202(2) of scheduler type2 maintains a host pool POOL2 that initially contains 0 hosts. There can be more than two such scheduler instances 202 in the cluster.

[0164] Each physical host 204 in the cluster is initially idle (i.e., no scheduler has allocated resources for any workload) and does not belong to any host pool. GRM 200 can be used to define the sharing policy between scheduler types of scheduler instance 202. For example, a scheduler type can use a certain percentage or a limited number of physical hosts 204 in the cluster. If a scheduler instance of this scheduler type queries GRM 200 for more than the percentage or limit of physical hosts in the cluster, or if there are no idle physical hosts in the cluster, GRM 200 will return an error to reject it.

[0165] Type 1 scheduler instance 202(1) receives resource request 212(1). Scheduler instance 202(1) first attempts to schedule the resource request to any physical host in POOL1. If scheduler instance 202(1) finds that no physical host in POOL1 can satisfy the request, then scheduler instance 202(1) requests an idle physical host in the cluster from GRM 200 that can satisfy the request. If scheduler instance 202(1) obtains an idle host, then scheduler instance 202(1) marks the idle host belonging to POOL1 and then schedules the resource request to the idle host. Now an idle host is allocated and reserved for any resource request in POOL1 and Type 1. Otherwise, scheduler instance 202(1) returns an error to the relevant resource request 212(1), indicating that no resource can be allocated to the request.

[0166] Similarly, the Type2 scheduler instance 202(2) receives and schedules resource request 212(2) with POOL2, and GRM200 performs operations in a manner similar to that described with respect to Type1.

[0167] When GRM 200 resolves conflicts using pooled resources in the example embodiment, GRM 200 only needs to resolve conflicts between scheduler instances 202 of the same scheduler type, because different scheduler types use different host pools that do not overlap. Optionally, in the example embodiment, each scheduler type may have its own conflict resolver among its instances, without requiring conflict resolution by GRM 200.

[0168] The Type 1 scheduler instance 202(1) or the Type 2 scheduler instance 202(2) will periodically check whether there are any idle hosts in their respective host pools. If so, the scheduler instances 202(1) and 202(2) will unmark the idle hosts and release them from their host pools back to GRM 200 in the cluster.

[0169] The Type 1 scheduler instance 202(1) or the Type 2 scheduler instance 202(2) will periodically check whether the scheduler instances 202(1) and 202(2) can make a host idle in its host pool by migrating the workload running on the currently running physical host to another physical host in the host pool. If it is possible and after the migration, the scheduler instances 202(1) and 202(2) can unmark the current physical host and release the current physical host from its host pool back to GRM200 in the cluster.

[0170] A fifth example implementation will now be described, which uses schedulers, workloads, and priority-based sharing configured between different types of scheduler instances 202 to help resolve scheduling conflicts.

[0171] GRM 200 can define scheduler and workload types, such as VM scheduler, container scheduler, x86 workloads, reduced instruction set computer (RISC) machine (ARM) workloads, latency-sensitive workloads, and batch workloads. GRM 200 can then set conflict probabilities between the two types of scheduler instances and workloads, which can be defined through configuration or estimated through machine learning from historical data. For example, the conflict probability between x86 workloads and ARM workloads is 0.

[0172] In the example, GRM 200 can also optionally determine scores and ranking information between different types of scheduler instances and workloads. Ranking information can be calculated based on scheduler type or the type of workload currently scheduled. For example, on-demand VM requests have a higher ranking value than Spot VM requests. Latency-sensitive workload requests have a higher ranking value than batch workload requests. Ranking can also be selectively adjusted based on some dynamic data. An example of such dynamic data is the sum of weighted resource amounts used by scheduler types or workload types; if a scheduler type or workload type uses fewer resources than others, the ranking value will be increased.

[0173] In the example, the GRM 200 receives multiple requests for resource conflict resolution in small batches. The GRM 200 can resolve scheduling conflicts within groups (i.e., small groups for small batches). For resource requests with a high probability of conflict, the GRM 200 can resolve conflicts in the group using more pessimistic concurrency control (such as transactions or locking). For resource requests with a low probability of conflict, the GRM 200 can resolve conflicts in the group using more optimistic concurrency control (such as lock-free and retries). For resource requests with a zero probability of conflict, such as the conflict probability between x86 and ARM workloads, the GRM 200 can resolve conflicts in parallel in different groups.

[0174] If a set of conflicting requests contains requests with high ordering values, such as On-Demand VMs, and requests with low ordering values, such as Spot VMs, then GRM 200 sorts the requests based on the received ordering information. GRM 200 first resolves conflicts for requests with high ordering values. Then, GRM 200 resolves conflicts for requests with low ordering values.

[0175] In the fifth example, advanced sharing policies are used to resolve conflicts within groups, whether in host groups such as elastic pools, or in scheduler, workload, and priority-based groups. Advanced sharing policies can help narrow down the scope of conflict resolution and can improve performance. Using sorting values ​​allows for better scheduling control and balancing across different priority-based requests.

[0176] Figure 4 This is a flowchart illustrating an example method 400 performed by a GRM 200 according to an example embodiment. The GRM 200 communicates with a physical host 204 and a scheduler instance 202, where each scheduler instance 202 has a corresponding resource request that requires resource allocation.

[0177] In step 402, GRM 200 receives resource information (e.g., allocation or usage information) from the local resource manager 206 on physical host 204. In step 404, GRM 200 sends the received resource information to scheduling instance 202. In step 406, GRM 200 receives corresponding candidate physical hosts from each scheduler instance, these candidate physical hosts can be adapted to the resource allocation of the scheduler instance 202 independently of other scheduler instances 202. In step 408, based on the scheduler instance 202 and the resource information, GRM 200 selects a corresponding physical host from the corresponding candidate physical hosts for each scheduler instance 202 to resolve scheduling conflicts of corresponding resource requests from multiple scheduler instances. In step 410, GRM 200 sends to scheduler instance 202 the identifier of the selected corresponding physical host 204 for at least one runtime instance to run the corresponding resource request scheduled by scheduler instance 202. In the example (not shown here), GRM 200 also sends to the local resource manager 206 of the selected physical host 204 at least one runtime instance that the selected physical host 204 will run for the corresponding resource request.

[0178] In the example, method 400 also includes GRM 200 reserving resources on at least one first physical host 204 in the first resource pool for resource requests of the first type of runtime instance, and reserving resources on at least one second physical host 204 in the second resource pool for resource requests of the second type of runtime instance.

[0179] In the example, method 400 also includes releasing the reserved resources on the first at least one or the second at least one of the physical hosts 204 in the corresponding resource pool when the reserved resources on the corresponding physical host 204 are no longer needed.

[0180] In the example, method 400 also includes migrating workloads from one of the first at least one physical hosts 204 to another of the first at least one physical hosts 204 to make one of the first at least one physical hosts 204 idle so that physical host 204 can remain in the resource pool.

[0181] Figure 5 This is a flowchart illustrating an example method 500 for facilitating centralized distributed scheduling of a type of runtime instance on a physical host 204, executed by a scheduler instance (e.g., scheduler 1 (202 (1))) according to an example embodiment.

[0182] In step 502, scheduler 1 (202(1)) receives a resource request for resource allocation. In step 504, scheduler 1 (202(1)) receives resource information from multiple physical hosts 204. In step 506, scheduler 1 (202(1)) selects the identifiers of the corresponding candidate physical hosts based on the resource information, which can adapt the resource allocation of the resource request to the resource request independently of the resource requests of other scheduler instances.

[0183] In some example embodiments, in step 508, scheduler 1 (202(1)) further filters out unqualified physical hosts in order to identify qualified physical hosts as corresponding candidate physical hosts.

[0184] In some example embodiments, in step 509, scheduler 1 202(1)) further scores or ranks the corresponding candidate physical hosts and generates score or ranking information. In this case, scheduler 1 (202(1)) also sends the score or ranking information to GRM 200.

[0185] In step 510, scheduler 1 (202(1)) sends a resource request and the identifier of the corresponding candidate physical host to GRM 200. In step 512, scheduler 1 (202(1)) receives from the Global Resource Manager the identifier of the physical host 204 that has resolved the scheduling conflict from the corresponding candidate physical host. In step 514, scheduler 1 (202(1)) sends a resource allocation for the identified physical host 204 to the runtime agent of the identified physical host 204 to run at least one runtime instance of the resource allocation.

[0186] Figure 6 This is a flowchart illustrating an example method 600 for facilitating centralized distributed scheduling of multiple scheduler instances 202, performed by an LRM 206 of one of the physical hosts 204(1) according to an example embodiment.

[0187] In step 602, LRM 206 sends resource information (e.g., availability or usage information) of the corresponding physical host 204 to GRM 200. In step 604, LRM 206 receives a first resource allocation of a scheduled quantity from runtime agent 1 (208(1)) without receiving local resource allocation arrangement details, for physical host 204 to run at least one first runtime instance of runtime agent 1 (208(1)). In some examples (not shown here), LRM 206 receives the first resource allocation from GRM 200 instead of runtime agent 1 (208(1)). In step 606, LRM 206 receives a second resource allocation of a scheduled quantity from runtime agent 2 (208(2)) without receiving local resource allocation arrangement details, for physical host 204 to run at least one second runtime instance of runtime agent 2 (208(2)). In some examples (not shown here), LRM 206 receives a second resource allocation from GRM 200 instead of runtime agent 2 (208(2)).

[0188] In step 608, LRM 206 performs local scheduling, local conflict resolution, and local optimization within physical host 204 to run at least one first runtime instance and at least one second runtime instance concurrently, resolving scheduling conflicts between at least one first runtime instance and at least one second runtime instance.

[0189] The above-described example embodiments can be implemented using only hardware or by using software and necessary general-purpose hardware platforms. Based on this understanding, some of the technical solutions of the example embodiments can be embodied in the form of software products. Software products can be stored in non-volatile or non-transitory storage media, such as compact disk read-only memory (CD-ROM), universal serial bus (USB) flash drives, or portable hard drives. Software products include multiple instructions that enable a computer device (personal computer, server, or network device) to perform the methods provided in the example embodiments. According to example embodiments, software products may additionally include multiple instructions that enable a computer device to perform operations for configuring or programming digital logic devices.

[0190] According to exemplary embodiments, the exemplary apparatus and methods described herein can be implemented by one or more controllers. A controller may include hardware, software, or a combination of hardware and software, depending on the specific application, component, or function. In some exemplary embodiments, one or more controllers may include analog or digital components and may include one or more processors, one or more non-transitory storage media, such as a memory storing instructions executed by one or more processors, one or more transceivers (or separate transmitters and receivers), one or more signal processors (analog or digital), and one or more analog circuit components.

[0191] In the described methods or block diagrams, boxes may represent events, steps, functions, processes, modules, messages, and / or state-based operations, etc. Although some of the examples above have been described as occurring in a specific order, those skilled in the art will understand that some steps or processes may be performed in a different order, provided that changing the order of any given step does not prevent or impair the occurrence of subsequent steps. Furthermore, in other embodiments, some of the messages or steps described above may be deleted or combined, and in other embodiments, some of the messages or steps described above may be separated into multiple sub-messages or sub-steps. Even more so, some or all of the steps may be repeated as needed. Elements described as methods or steps similarly apply to systems or subcomponents, and vice versa. References to terms such as "send" or "receive" may be interchangeable depending on the perspective of the particular device.

[0192] The embodiments described above are considered illustrative, not limiting. The example embodiments described as methods are similarly applicable to systems, and vice versa.

[0193] Variations can be made to some of the example embodiments, which may include combinations and sub-combinations of any of the above. The example embodiments presented above are merely examples and are not intended to limit the scope of this disclosure. The innovative variations described herein will be apparent to those skilled in the art, and these variations are within the scope of this disclosure. In particular, features from one or more of the above embodiments can be selected to create alternative embodiments consisting of sub-combinations of features that may not be explicitly described above. Furthermore, features from one or more of the above embodiments can be selected and combined to create alternative embodiments consisting of combinations of features that may not be explicitly described above. Features applicable to such combinations and sub-combinations will be apparent to those skilled in the art upon a thorough review of this disclosure. The subject matter described herein is intended to include all suitable variations in the art.

Claims

1. A method for scheduling and collision resolution, characterized by, The method is performed by a global resource manager, comprising: receiving resource information of a physical host from a local resource manager on each of a plurality of physical hosts; receiving a respective candidate physical host from each of a plurality of scheduler instances each having a respective resource request, the respective candidate physical host being each filtered out by the respective scheduler instance and being adaptable to resource allocation of the scheduler instance independently of other scheduler instances; filtering out at least one ineligible physical host from the respective candidate physical host for at least one of the scheduler instances based on the scheduler instances and the resource information to select a respective physical host resolving scheduling conflicts of the respective resource requests of the plurality of scheduler instances, the resource information including real-time actual resource usage information or scheduling resource allocation information of the respective physical host; and sending an identification of the selected respective physical host to the at least one of the scheduler instances to run at least one runtime instance of the respective resource request scheduled by the scheduler instance.

2. The method of claim 1, wherein, A first at least one of the scheduler instances is for a first type of runtime instance, and a second at least one of the scheduler instances is for a second type of runtime instance that is of a same type as compared to the first type of runtime instance.

3. The method of claim 1, wherein, A first at least one of the scheduler instances is for a first type of runtime instance, and a second at least one of the scheduler instances is for a second type of runtime instance that is of a different type as compared to the first type of runtime instance.

4. The method of claim 1, wherein, The selection further comprises selecting a same physical host for the first at least one of the scheduler instances and the second at least one of the scheduler instances.

5. The method of claim 3, wherein, The selection further comprises selecting a first at least one of the physical hosts for the first at least one of the scheduler instances, and selecting a second at least one of the physical hosts for the second at least one of the scheduler instances, wherein the first at least one of the physical hosts is different from the second at least one of the physical hosts.

6. The method of claim 3, wherein, Further comprising reserving resources on the first at least one of the physical hosts in a first resource pool for resource requests of the first type of runtime instance, and reserving resources on the second at least one of the physical hosts in a second resource pool for resource requests of the second type of runtime instance.

7. The method of claim 6, wherein, Further comprising releasing the reserved resources on the first at least one of the physical hosts or the second at least one of the physical hosts in a respective resource pool when the reserved resources on the respective physical host are no longer needed.

8. The method of claim 6, wherein, Further comprising migrating workloads from the first at least one of the physical hosts to another at least one of the physical hosts so that the first at least one of the physical hosts has more resources free for the resources of the respective physical host to be reserved in a respective resource pool.

9. The method according to any one of claims 3, and 5 to 8, characterized in that, The first type of runtime instance is one of a virtual machine, a container, a Pod, or a node; and the second type of runtime instance is a different one of the virtual machine, the container, the Pod, or the node.

10. The method according to any one of claims 1 to 8, characterized in that, Each scheduler instance is configured to select a respective physical host from a plurality of candidate physical hosts for a respective runtime instance.

11. The method of claim 10, wherein, The respective physical host is selected from a plurality of candidate physical hosts.

12. The method according to any one of claims 1 to 8, characterized in that, The respective physical host is selected from a plurality of candidate physical hosts.

13. The method according to any one of claims 1 to 8, characterized in that, The selection of the respective physical host is made for at least two of the scheduler instances, and wherein the sending of the identification of the selected respective physical host is made for at least two of the scheduler instances.

14. The method according to any one of claims 1 to 8, characterized in that, Further comprising sending the resource information to the scheduler instances after the receiving of the resource information.

15. The method according to any one of claims 1 to 8, characterized in that, Further comprising repeating the receiving of the resource information to monitor the at least one runtime instance.

16. The method according to any one of claims 1 to 8, characterized in that, The resource allocation includes reclaiming resources from one of the physical hosts.

17. The method of any one of claims 1 to 8, wherein, The sending to at least one of the scheduler instances includes sending a scheduling quantity of the resource allocation for the selected respective physical host, and without sending local resource allocation arrangement details of the at least one runtime instance on the selected respective physical host.

18. The method of claim 17, wherein, Each respective local resource manager receives information from the global resource manager and runtime agent on the physical host, and is configured to perform local resource allocation arrangement of the physical host without receiving local resource allocation arrangement details.

19. The method of claim 17, wherein, The local resource allocation arrangement details include local scheduling, local conflict resolution, and local optimization.

20. The method of claim 17, wherein, The local resource allocation arrangement details include detailed placement on local resources, local resource sharing, isolation, binding, placement, replacement, or migration.

21. The method of any one of claims 1 to 8, wherein, At least one physical host is configured with a plurality of non-uniform memory access nodes, wherein the local resource allocation arrangement details of the at least one physical host include migrating local resource allocation from a first non-uniform memory access node to a second non-uniform memory access node.

22. The method of claim 21, wherein, Further comprising receiving scoring or ranking information for the respective candidate physical host of the scheduler instance from each of the plurality of scheduler instances, wherein the selecting of the respective physical host for the scheduler instance is further based on the received scoring or ranking information.

23. The method of claim 22, wherein, The scoring or ranking information is generated by a respective scheduler instance based on one or more of: scaling, stacking, application proximity, fragmentation reduction, interference reduction, or scaling elasticity, or combinations or sub-combinations thereof.

24. The method of claim 21, wherein, The scoring information includes weights for ranking the respective candidate physical host according to one or more of the scaling, the stacking, the application proximity, the fragmentation reduction, the interference reduction, or the scaling elasticity.

25. The method of claim 24, wherein, Further comprising resolving scoring or ranking conflicts between the candidate physical hosts according to received scoring or ranking information prior to selecting the respective physical host from the respective candidate physical hosts. The resolving of the scoring or ranking conflicts is based on a cost of each of the plurality of physical hosts.

26. The method of any one of claims 1 to 8, wherein, The selecting the respective physical host from the respective candidate physical hosts for the scheduler instance is further based on optimizing the resource allocation or reclamation cost for all the scheduler instances.

27. The method of claim 26, wherein, The optimizing includes optimizing performance, isolation, and resource usage of the physical hosts.

28. The method of any one of claims 1 to 8, wherein, The scheduling conflict is resolved by spreading, stacking, application proximity, fragmentation reduction, interference reduction, or expansion elasticity, or a combination or sub-combination of these.

29. The method of claim 28, wherein, The scheduling conflict is resolved by weighting one or more of the spreading, the stacking, the application proximity, the fragmentation reduction, the interference reduction, or the expansion elasticity.

30. The method of any one of claims 1 to 8, wherein, The resource information includes allocation information, usage information, or both the allocation information and the usage information.

31. The method of any one of claims 1 to 8, wherein, Further comprising receiving a conflict resolution request from at least one of the scheduler instances, wherein the selecting is performed in small batches for the conflict resolution requests received from the at least one of the scheduler instances from the respective candidate physical hosts within a specified time interval.

32. The method of any one of claims 1 to 8, wherein, The selecting the respective physical host from the respective candidate physical hosts is selecting less than all of the plurality of physical hosts.

33. The method of any one of claims 1 to 8, wherein, The method is performed by at least one processor external to the physical hosts.

34. The method of any one of claims 1 to 8, wherein, Further comprising sending to the local resource manager of the selected respective physical host that the selected respective physical host is to run at least one runtime instance of the respective resource request.

35. A method of facilitating scheduling and conflict resolution, the method comprising: The method is performed by a scheduler instance, comprising: receiving a resource request for resource allocation; receiving resource information of a plurality of physical hosts, the resource information including real-time actual resource usage information or scheduled resource allocation information of the physical hosts; based on the resource information, selecting an identification of respective candidate physical hosts that are capable of accommodating the resource allocation of the resource request independently of resource requests of other scheduler instances; sending a conflict resolution request and the identification of the respective candidate physical hosts to a global resource manager; receiving from the global resource manager an identification of physical hosts that resolve scheduling conflicts of resource requests of multiple scheduler instances selected by the global resource manager by filtering out at least one unqualified physical host of the respective candidate physical hosts; and sending to a runtime agent of the identified physical hosts the resource allocation for the identified physical hosts to run at least one runtime instance of the resource allocation.

36. The method of claim 35, wherein, The selecting includes filtering out unqualified physical hosts so that qualified physical hosts are determined as the respective candidate physical hosts.

37. The method of claim 36, wherein, The unqualified physical hosts are a type of physical host that is more expensive than another type of physical host that is determined as the qualified physical hosts.

38. The method of any one of claims 35-37, wherein, Further comprising scoring or ranking the respective candidate physical hosts and generating scoring or ranking information, and sending the scoring or ranking information to the global resource manager.

39. The method of claim 38, wherein, The generating the scoring or ranking information is based on: spreading, stacking, application proximity, fragmentation reduction, interference reduction, or expansion elasticity, or a combination or sub-combination of these.

40. The method of claim 39, wherein, The score information includes a weight generated according to one or more of the extension, the stacking, the application proximity, the fragmentation reduction, the interference reduction, or the extension elasticity.

41. A method of facilitating scheduling and conflict resolution, the method comprising: The method is performed by a local resource manager of a physical host, comprising: sending resource information of the physical host to a global resource manager, the resource information including real-time actual resource usage information or scheduled resource allocation information of the physical host; receiving, from a first runtime agent or from the global resource manager, a scheduled number of first resource allocations without receiving local resource allocation arrangement details for the physical host to run at least one first runtime instance in the first runtime agent; receiving, from a second runtime agent of a different type compared to the first runtime agent or from the global resource manager, a scheduled number of second resource allocations without receiving the local resource allocation arrangement details for the physical host to run at least one second runtime instance in the second runtime agent; and performing local scheduling, local conflict resolution, and local optimization within the physical host to concurrently run the at least one first runtime instance and the at least one second runtime instance, resolving scheduling conflicts of the at least one first runtime instance and the at least one second runtime instance.

42. A computing system, comprising: comprising: a plurality of scheduler instances, each scheduler instance to receive a respective resource request for resource allocation; a plurality of physical hosts, each physical host having a local resource manager to send resource information of the physical host; a global resource manager, the global resource manager comprising at least one processor to: receive resource information of each of the plurality of physical hosts from the local resource manager on each of the plurality of physical hosts; receive, from each of the plurality of scheduler instances each having the respective resource request, a respective candidate physical host, the respective candidate physical host being each screened out by the respective scheduler instance and being adaptable to resource allocation of the scheduler instance independently of other scheduler instances; based on the scheduler instances and the resource information, filter out, for at least one of the scheduler instances, at least one disqualification physical host from the respective candidate physical host to select a respective physical host resolving scheduling conflicts of the respective resource requests of the plurality of scheduler instances, the resource information including real-time actual resource usage information or scheduled resource allocation information of the respective physical host; send, to the at least one of the scheduler instances, an identification of the selected respective physical host to run at least one runtime instance of the respective resource request scheduled by the scheduler instance.

43. The computing system of claim 42, wherein, the at least one processor to send, to the local resource manager of the selected respective physical host, that the selected respective physical host is to run at least one runtime instance of the respective resource request.

44. A non-transitory computer readable medium storing instructions, wherein, the instructions, when executed by one or more processors, cause the one or more processors to perform scheduling and conflict resolution, the instructions comprising: instructions for receiving resource information of the physical host from a local resource manager on each of a plurality of physical hosts; instructions for receiving, from each of a plurality of scheduler instances each having a respective resource request, a respective candidate physical host that is each filtered out by the respective scheduler instance and is capable of accommodating resource allocation of the scheduler instance independently of other scheduler instances; instructions for filtering out, based on the scheduler instances and the resource information, at least one ineligible physical host from the respective candidate physical hosts for at least one of the scheduler instances to select a respective physical host that resolves scheduling conflicts of the respective resource requests of the plurality of scheduler instances, the resource information comprising real-time actual resource usage information or scheduling resource allocation information of the respective physical host; and instructions for sending, to the at least one of the scheduler instances, an identification of a respective physical host that is to run a selection of at least one runtime instance of the respective resource request scheduled by the scheduler instance.

45. A computing system, comprising: comprising: a plurality of scheduler instances each for receiving a resource request for resource allocation; a plurality of physical hosts each having a local resource manager for sending resource information of the physical host; a global resource manager; and at least one processor for performing the method according to any one of claims 1 to 41. The instructions, when executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 1 to 41. 46.A non-transitory computer-readable medium storing instructions, wherein the instructions, when executed by a processor, cause the processor to perform operations comprising: comprising:

47. A computing system, comprising: a plurality of scheduler instances each for performing the method according to any one of claims 35 to 40; a plurality of physical hosts each having a local resource manager for performing the method according to claim 41; a global resource manager for performing the method according to any one of claims 1 to 34. ​

Images