Container scheduling method in virtualization environment
By scheduling containers based on the resource utilization rate of physical machines in a virtualized environment, the problem that container schedulers cannot optimize their resources in a virtualized environment is solved, and more efficient resource management and utilization is achieved.
Patent Information
- Application Number
- CN202411810311.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-10
- Publication Date
- 2025-05-30
AI Technical Summary
In a virtualized environment, the scheduler that deploys and manages containers fails to optimize the use of resources, resulting in poor resource management.
By scheduling containers between virtual machines executed on multiple physical machines, the container scheduler determines the optimal virtual machine to deploy containers based on the resource utilization of the physical machine, and realizes efficient utilization of resources.
Improved system resource management solutions to ensure that containers are deployed on virtual machines with the best resource utilization rate, thereby improving resource utilization efficiency.
Smart Images
Figure CN120066671A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of computer system software, and particularly relates to a container scheduling method in a virtualized environment. Background Art
[0002] Virtualization enables workloads to execute in an environment isolated from their underlying hardware through an intermediate software layer. Two typical representatives of current virtualization technologies include virtual machines and containers. The above virtualization technologies can run on physical machines. For example, a physical machine cluster can run a first workload as a container and a second workload as a virtual machine. Containers can be deployed either directly on a physical machine or within a virtual machine running on a physical machine. Due to this deployment complexity, the scheduler for deploying and managing containers cannot optimally utilize resources. The present invention is specifically proposed to solve the above problems. Summary of the Invention
[0003] In view of this, the present invention provides a container scheduling method in a virtualized environment, which schedules containers based on the resource utilization rate of physical machines and can improve the system resource management scheme.
[0004] The technical solution of the present invention is as follows:
[0005] A container scheduling method in a virtualized environment, comprising:
[0006] Sending first information of multiple virtual machines executed on multiple physical machines from a virtual machine scheduler to a container scheduler, wherein the container scheduler has access to the resource utilization rate of the virtual machines executed on the multiple physical machines;
[0007] The virtual machine scheduler receives second information from the container scheduler, and the virtual machine scheduler uses the multiple virtual machines as candidate virtual machines for deploying containers; each of the virtual machines is executed on the multiple physical machines;
[0008] The virtual machine is used to deploy a container, wherein the virtual machine scheduler has access to the resource utilization rate of each physical machine;
[0009] The container scheduler receives a request for deploying a container, and sends container information to the virtual machine scheduler. The virtual machine scheduler uses the container information and the resource utilization rate of the virtual machines to determine the optimal virtual machine for deploying the container; the virtual machine scheduler returns the optimal virtual machine ID to the container scheduler, and the container scheduler deploys the container on the optimal virtual machine based on the optimal virtual machine ID.
[0010] Further, the container scheduler is configured to:
[0011] Identify an initial candidate list of virtual machines for deploying containers according to user-defined constraints and policies;
[0012] The container information can be sent to the virtual machine scheduler;
[0013] The virtual machines can be scheduled according to the resource usage of the cluster composed of physical machines;
[0014] The container information includes: the resource requirements of the containers to be scheduled and the policies configured for the containers to be scheduled.
[0015] Furthermore, the virtual machine scheduler is based on the information provided by the container scheduler, the resource utilization rate of the physical machines, and the resource utilization rate of all virtual machines in the cluster to identify candidate virtual machines for deploying containers.
[0016] Furthermore, the number of candidate virtual machines is not less than one, and the virtual machine scheduler selects the optimal candidate virtual machine from each of the candidate virtual machines for deploying containers.
[0017] Furthermore, the first information includes the resource requirements of the containers to be deployed.
[0018] Furthermore, the first information includes the policies associated with the container scheduler, and the policies are used to identify the virtual machines of the containers to be deployed.
[0019] Furthermore, the first information includes the rankings of the multiple first virtual machines;
[0020] The rankings of the first virtual machines in the first information are determined by the container scheduler.
[0021] Furthermore, the second information includes the rankings of the virtual machines;
[0022] The information of the virtual machines is identified by the container scheduler.
[0023] Furthermore, the virtual machine scheduler executes on a separate virtual machine isolated from the cluster.
[0024] Furthermore, each of the virtual machines is configured to be capable of executing containers.
[0025] Advantages of the present invention:
[0026] The present invention schedules containers based on the resource utilization rate of physical machines, so that the container scheduler deploys containers on the optimal virtual machines, improving the management scheme of system resources. Description of the Drawings
[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required to be used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.
[0028] Figure 1 is a distributed virtualization system.
[0029] Figure 2 is a flowchart for deploying containers.
[0030] Figure 3 is an example diagram for efficient resource management.
[0031] Figure 4 is an example diagram for efficient resource management.
[0032] Figure 5 is an example diagram for ensuring high availability.
[0033] Figure 6 is an example diagram for ensuring high availability.
[0034] Figure 7 is the process of deploying containers. Detailed Implementation Manner
[0035] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.
[0036] The following uses specific specific examples to illustrate the implementation manners of the present disclosure. Those skilled in the art can easily understand other advantages and effects of the present disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. The present disclosure can also be implemented or applied through other different specific implementation manners. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present disclosure without creative efforts belong to the scope of protection of the present disclosure.
[0037] It should be noted that the following describes various aspects of the embodiments within the scope of the appended claims. It should be obvious that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, those skilled in the art should understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement a device and / or practice a method. Additionally, this device and / or this method can be implemented using other structures and / or functions in addition to one or more of the aspects described herein.
[0038] It should also be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present disclosure schematically. The diagrams only show the components related to the present disclosure, rather than being drawn according to the number, shape, and size of the components in actual implementation. The types, quantities, and proportions of the components in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.
[0039] In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.
[0040] An embodiment of the present invention provides a container scheduling method in a virtualized environment. This method is based on the resource utilization rate of physical machines and can improve the resource management method.
[0041] The container scheduler receives a request to deploy a container. In response to this request, the container scheduler is responsible for: (1) using user-defined constraints and policies to identify a list of candidate virtual machines for deploying the container; (2) sending the container information to the virtual machine scheduler.
[0042] The virtual machine scheduler uses the received information and the resource utilization rate of the physical machines to identify candidate virtual machines for deploying the container.
[0043] Figure 1 A distributed virtualization system is depicted, which includes multiple physical machines (e.g., Physical Machine 1 and Physical Machine 2).
[0044] A physical machine can execute one or more virtual machines (e.g., Virtual Machine 1, Virtual Machine 2, and Virtual Machine 3), where a virtual machine is simulation software of a physical machine.
[0045] For a virtual machine, the virtual machine manager acts as a mediator between the virtual machine environment and the underlying hardware, providing an abstraction layer. The virtual machine can be executed using the virtual machine manager, which can run directly on bare hardware (Type 1 virtual machine manager) or in combination with the host operating system (Type 2 virtual machine manager).
[0046] As Figure 1 shown, Physical Machine 1 executes Virtual Machine 1, and Virtual Machine 1 in turn executes Container 1 and Container 2. Physical Machine 1 also executes Virtual Machine 2, and Virtual Machine 2 does not execute any containers.
[0047] Physical Machine 2 executes Virtual Machine 3, and Virtual Machine 3 in turn executes Container 3. Physical Machine 2 also executes Virtual Machine 4, and Virtual Machine 4 in turn executes Container 4.
[0048] The distributed virtualization system also includes a virtual machine scheduler for managing virtual machines in the distributed virtualization system. Managing virtual machines includes migrating, creating, and deleting virtual machines within the distributed virtualization system. The virtual machine scheduler can manage the virtual machines of the distributed virtualization system such that the resource consumption across physical machines in the distributed virtualization system is optimized (e.g., load balancing).
[0049] The virtual machine scheduler can execute at the container, virtual machine, or host level. Additionally, the virtual machine scheduler can execute on the physical machines (e.g., physical machine 1, physical machine 2) of the distributed virtualization system.
[0050] The distributed virtualization system includes a container scheduler for managing containers. Managing containers includes creating and deleting containers. The container scheduler determines the resource utilization of virtual machines in the distributed virtualization system and optimizes (e.g., load balances) the resource utilization of virtual machines to optimize the execution of containers within the distributed virtualization system.
[0051] The container scheduler only determines the resource utilization of the virtual machines that execute containers. Therefore, the container scheduler does not consider the resource utilization of virtual machines that do not execute containers (e.g., virtual machine 2).
[0052] Figure 2 The operations depicted are implemented using a two-stage scheduling process, enabling efficient resource management of containers and virtual machines.
[0053] The container scheduler identifies the virtual machines on which to deploy containers based on container information. Container information includes the resource requirements of the container, the virtual machines that execute the container, and the virtual machines assigned to the container scheduler.
[0054] The container information can be combined with the policies associated with the container scheduler to identify virtual machines. The policies of the container scheduler include a container load balancing policy between virtual machines, a policy for consolidating containers between virtual machines (bin packing problem: deploying containers into virtual machines to ensure the fewest virtual machines are used), grouping containers by type (deploying containers of the same type to the same virtual machine), and allocating containers among a predetermined number of containers (to improve the availability of processes). The virtual machines can be ranked in order from most desirable to least desirable. The container scheduler provides the virtual machine scheduler with a list of virtual machines and the resource requirements of the containers.
[0055] The virtual machine scheduler determines the resource utilization by responding to receiving the list of virtual machines from the container scheduler.
[0056] Resource utilization represents the consumption of different types of resources (e.g., processor, memory, external storage, network bandwidth resources). Resource utilization can also represent which containers are deployed on which virtual machines or physical machines respectively.
[0057] The virtual machine scheduler identifies candidate virtual machines for deploying containers based on resource utilization. The physical machine with the lowest resource utilization can be determined as a candidate virtual machine; the physical machines below the resource usage threshold can be determined as candidate virtual machines. The candidate virtual machines can be sorted based on their resource utilization.
[0058] The virtual machine scheduler can send a list of the group of candidate virtual machines to the container scheduler.
[0059] The container scheduler can identify virtual machines from the list of candidate virtual machines.
[0060] Once the container scheduler has identified the virtual machines, the container scheduler can deploy containers on the identified virtual machines.
[0061] Figure 3 Depicts a scheduling decision using only a single scheduling process (e.g., the container scheduler). Figure 4 Depicts a scheduling decision using multiple scheduling processes (e.g., the container scheduler and the virtual machine scheduler).
[0062] In Figure 3 are depicted multiple physical machines (e.g., physical machine 1 and physical machine 2). Physical machine 1 is executing virtual machines 3A-1, 3A-2, and 3A-4. Virtual machines 3A-1 and 3A-2 execute containers while virtual machine 3A-4 does not execute containers. Physical machine 2 is executing virtual machine 3A-3. Virtual machine 3A-3 executes containers.
[0063] Each virtual machine has its own load and its own amount of unused resources. The load represents the percentage of resources dedicated to the virtual machine that the virtual machine is currently using, and the unused resources represent the percentage of the resources of the virtual machine that are not being used by the virtual machine.
[0064] When determining the virtual machine for deploying a container, the scheduling process can determine the resource utilization of each of the three virtual machines 3A-1, 3A-2, and 3A-3, but cannot determine the resource utilization of virtual machine 3A-4 (for virtual machines that do not execute containers, the container scheduler cannot determine the resource utilization). Resource utilization represents: (1) the amount of resources consumed by each virtual machine; (2) whether the virtual machine includes a copy of the container to be deployed.
[0065] Based on the resource utilization, the scheduling process can determine to deploy a new container on virtual machine 3A-2 because virtual machine 3A-2 uses less of its allocated resources than the other virtual machines that the scheduling process can view.
[0066] As Figure 4 shown, multiple scheduling processes are employed to deploy containers.
[0067] After identifying the virtual machines, the container scheduler sends a list identifying the virtual machines to the virtual machine scheduler. The virtual machine scheduler identifies the physical machines associated with the container scheduler that sent the list.
[0068] Thereafter, the virtual machine scheduler determines the resource utilization of the identified physical machines. Based on the resource utilization, the virtual machine scheduler identifies a physical machine. For example, the virtual machine scheduler may determine that physical machine 2 has a higher percentage of unused resources compared to physical machine 1 and is thus more suitable for the deployment of the requested container. After identifying physical machine 2, the virtual machine scheduler may: (1) identify a subset of the virtual machines executing on physical machine 2; (2) identify each virtual machine running on physical machine 2. The identified virtual machines are referred to as candidate virtual machines.
[0069] The virtual machine scheduler sends a list of the candidate virtual machines to the container scheduler. Based on the list, the container scheduler identifies the virtual machine on which to deploy the new container.
[0070] As Figure 5 shown, physical machine 1 executes virtual machines 4A-1, 4A-2, and physical machine 2 executes virtual machine 4A-3. Each of the virtual machines 4A-1, 4A-2, and 4A-3 executes containers. Virtual machine 4A-1 executes the same container as the requested container. The scheduling process manages the containers of the system such that the same virtual machine does not execute replica containers. This provides high availability for any process in the container. High availability allows the requested container to be scheduled onto a virtual machine on which its replica is not yet running, such that if the virtual machine fails, at least one instance of the requested container remains running. Thus, in operation, based on the resource utilization, the scheduling process deploys the requested container on virtual machine 4A-2 or virtual machine 4A-3 because virtual machine 4A-1 includes a replica of the container.
[0071] In Figure 6 it, multiple scheduling processes are implemented. The container scheduler may identify: (1) virtual machines 4B-2 and 4B-3 (virtual machines that do not include replica containers); (2) virtual machine 4B-2 (the virtual machine selected by the container scheduler). The container scheduler may also identify virtual machine 4B-1, which is running a replica of the container to be deployed. When multiple virtual machines are identified, the container scheduler may also rank the multiple virtual machines based on a policy associated with the container to be scheduled.
[0072] Thereafter, the container scheduler sends a list of the identified virtual machines to the virtual machine scheduler. The virtual machine scheduler identifies the physical machines associated with the container scheduler.
[0073] After identifying a physical machine, the virtual machine scheduler can determine the resource utilization in the physical machine. The resource utilization indicates which physical machines include virtual machines, and the virtual machines include replicas of containers. The resource utilization is related to the virtual machines associated with the container scheduler, the virtual machines not associated with the container scheduler, the running virtual machines, and the containers.
[0074] Based on the resource utilization, the virtual machine scheduler identifies physical machines. For example, the virtual machine scheduler determines that physical machine 2 does not include a virtual machine containing a container replica. After identifying physical machine 2, the virtual machine scheduler can: (1) identify a group of virtual machines running on physical machine 2; (2) identify each virtual machine running on physical machine 2. The identified virtual machines can be used as candidate virtual machines.
[0075] The virtual machine scheduler sends a list of candidate virtual machines to the container scheduler. Based on this list, the container scheduler identifies the virtual machines on which to deploy new containers. By selecting physical machines that do not execute replica containers, the scheduling process ensures that the requested containers are deployed on physical machines different from the replica containers. If any physical machine / its virtual machines / containers fail, at least one instance of the container will continue to execute.
[0076] Figure 7 The process of deploying containers is depicted.
[0077] In addition to identifying virtual machines, the first information also includes the resource requirements of a container (e.g., container 1 as shown Figure 1 ), the policy to be used by a second container, the identity of the containers running on the virtual machine, the identity of whether the virtual machine contains a replica of the container to be deployed, or any combination thereof. The resource requirements can include computing power (e.g., in cores), external storage (e.g., in bytes), memory (e.g., in bytes), remaining running time (e.g., in time units), or any combination. The policy to be used by the virtual machine scheduler identifies virtual machines based on the physical machines or the information determined by the virtual machine scheduler (e.g., container 1 as shown Figure 1 ).
[0078] After receiving the first information, the virtual machine scheduler identifies the physical machines associated with the virtual machines included in the first information. For example, the virtual machine scheduler identifies the first physical machine that executes the first virtual machine. The container scheduler identifies the second physical machine that executes the second virtual machine.
[0079] The resource utilization of each physical machine is determined by the identified virtual machine scheduler. The resource utilization can be based on one or more different types of resources, such as processors, external storage, memory, network bandwidth resources. The resource utilization represents the percentage of each type of resource used (or not used) by each physical machine.
[0080] When determining resource utilization, the virtual machine scheduler identifies the resources of the physical machine that a virtual machine not executing a container is using (e.g., virtual machine 3B-4).
[0081] Based on the resource utilization and the first information, the virtual machine scheduler can identify candidate virtual machines.
[0082] Based on the resource utilization, candidate virtual machines are identified by the physical machine. For example, a physical machine can be identified when it satisfies: (1) the least amount of resources are used; (2) the amount of resources within the resource consumption threshold is being consumed; (3) no replicas of the container to be deployed are included; (4) any combination thereof.
[0083] Thereafter, the virtual machine scheduler provides the second information to the container scheduler. Based on the second information, the container scheduler deploys the container on the virtual machine based on the information associated with the physical machine.
[0084] The second information can identify a single candidate virtual machine such that the container is deployed on the candidate virtual machine. When a list of candidate virtual machines is received, the container scheduler uses the associated policy to determine the virtual machine from the list to deploy the container. The second information includes the ranking of the virtual machine determined by the virtual machine scheduler. The container scheduler identifies the virtual machine based on the ranking determined by the container scheduler and the ranking determined by the virtual machine scheduler.
[0085] As described above, it is only the specific implementation manner of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present disclosure should be covered by the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.
Claims
1. A container scheduling method in a virtualized environment, characterized in that ,include: First information of the plurality of virtual machines executed on the plurality of physical machines is sent from the virtual machine scheduler to the container scheduler, wherein the container scheduler has access to resource utilization of the virtual machines executed on the plurality of physical machines; The virtual machine scheduler receives the second information from the container scheduler, and the virtual machine scheduler uses the multiple virtual machines as candidate virtual machines for deploying the container; each of the virtual machines is executed on the multiple physical machines; The virtual machine is used to deploy a container, wherein the virtual machine scheduler has access to resource utilization of each physical machine; The container scheduler receives a request to deploy a container and sends the container information to the virtual machine scheduler. The virtual machine scheduler uses the container information and the resource utilization of the virtual machine to determine the optimal virtual machine for deploying the container. The virtual machine scheduler returns the optimal virtual machine ID to the container scheduler, and the container scheduler deploys the container on the optimal virtual machine based on the optimal virtual machine ID.
2. The container scheduling method in a virtualized environment according to claim 1, characterized in that ,The container scheduler is configured as: An initial candidate list of virtual machines for deploying containers may be identified based on user-defined constraints and policies; Container information can be sent to the virtual machine scheduler; Virtual machines can be scheduled based on the resource usage of the cluster composed of each physical machine; The container information includes: resource requirements of the container to be scheduled and the configuration strategy of the container to be scheduled.
3. The container scheduling method in a virtualized environment according to claim 2, characterized in that ,The virtual machine scheduler identifies candidate virtual machines for deploying containers based on the ,information provided by the container scheduler, the resource utilization of the ,physical machines, and the resource utilization of all virtual machines in the ,cluster.
4. The container scheduling method in a virtualized environment according to claim 3, characterized in that ,The number of candidate virtual machines is not less than one, and the virtual machine scheduler selects the best candidate virtual machine from each of the candidate virtual machines for deploying the container.
5. The container scheduling method in a virtualized environment according to claim 4, characterized in that ,The first information includes the resource requirements of the container to be deployed.
6. The container scheduling method in a virtualized environment according to claim 5, characterized in that ,The first information includes a policy associated with the container scheduler, and the policy is used to identify the virtual machine to which the container is to be deployed.
7. The container scheduling method in a virtualized environment according to claim 6, characterized in that ,The first information includes rankings of the plurality of first virtual machines; The ranking of the first virtual machine in the first information is determined by the container scheduler.
8. The container scheduling method in a virtualized environment according to claim 7, characterized in that ,The second information includes the ranking of the virtual machine; The information of the virtual machine is identified by the container scheduler.
9. The container scheduling method in a virtualized environment according to claim 8, characterized in that ,The virtual machine scheduler executes on a separate virtual machine isolated from the cluster.
10. The container scheduling method in a virtualized environment according to claim 9, characterized in that ,Each of the virtual machines is configured as an executable container.