Resource hybrid scheduling system and method
By using a resource hybrid scheduling system and method, the problem that virtual machines and containers can only be deployed separately is solved, enabling the hybrid deployment of virtual machines and containers on the same physical machine, reducing deployment costs and improving scheduling efficiency.
Patent Information
- Application Number
- CN202411054636.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2026-02-03
AI Technical Summary
The existing virtual machine and container scheduling architecture manages physical machines independently, which means that virtual machines and containers can only be deployed on different physical machines, increasing the deployment cost of physical machines and reducing scheduling efficiency.
This paper provides a resource hybrid scheduling system and method that enables the matching and scheduling of different types of resources by customizing resource forms. It supports the hybrid deployment of virtual machines and containers on the same physical machine and utilizes components such as API servers, resource schedulers and agents to schedule resources based on resource requirements and resource pool usage.
It enables the hybrid deployment of virtual machines and containers on the same physical machine, reducing the deployment cost of physical machines and improving the efficiency and utilization of resource scheduling.
Smart Images
Figure CN121455618A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of cloud computing technology, and in particular to a resource hybrid scheduling system and method. Background Technology
[0002] With the development and promotion of container technology, most stateless services are being deployed in containers, and the application of containers and virtual machines in different business scenarios is becoming increasingly widespread. Currently, distributed systems require the simultaneous use of containers and virtual machines to run application instances. Virtual machine-specific scheduling systems are used to create and schedule virtual machine instances, while container-specific scheduling systems are used to create and schedule container instances.
[0003] Since virtual machine scheduling architecture and container scheduling architecture are two independent architectures that manage different physical machines, virtual machine scheduling architecture can perform virtual machine scheduling on the physical machines it manages, while container scheduling architecture can perform container scheduling on the physical machines it manages.
[0004] However, based on the existing virtual machine scheduling architecture and container scheduling architecture, virtual machines and containers can only be deployed on different physical machines, which undoubtedly increases the deployment cost of physical machines. Summary of the Invention
[0005] This application provides a resource hybrid scheduling system and method to solve the technical problem in the prior art that it is not possible to support the hybrid deployment of virtual machines and containers on the same physical machine.
[0006] In a first aspect, this application provides a resource hybrid scheduling system, including an API server, resource schedulers corresponding to at least two types of resources, and a resource pool. The resource pool includes multiple physical machines, each physical machine is configured with a physical machine agent, and the physical machines support the deployment of at least two types of resources, wherein;
[0007] The API server is used to create resource instances based on user resource creation requests, whereby the resource creation requests indicate the resource type and resource requirements.
[0008] The resource scheduler is used to discover newly created resource instances of the corresponding type, and determine the target physical machine from the resource pool according to the resource pool resource usage list and the resource requirements of the newly created resource instance, and schedule the newly created resource instance to the target physical machine;
[0009] The agent of the target physical machine is used to discover newly created resource instances scheduled to this physical machine, and to create corresponding resources according to the resource requirements of the newly created resource instances.
[0010] Optionally, in the system described above, the API server is also used for:
[0011] Receive a custom resource type registration request sent by a user. The custom resource type registration request contains a custom resource definition, which is used to characterize the custom resource and the resource type of the custom resource.
[0012] After successful verification, the custom resource definition is stored in a distributed key-value storage system.
[0013] Optionally, in the system described above, the API server is specifically used for:
[0014] After creating a resource instance, the resource instance is stored in the distributed key-value storage system;
[0015] The resource scheduler is specifically used for:
[0016] By monitoring the API server, newly created resource instances of the corresponding type can be detected.
[0017] Optionally, in the system described above, the resource scheduler is further configured to:
[0018] After discovering a newly created resource instance of the corresponding type, obtain the node name of the newly created resource instance;
[0019] If the node name of the newly created resource instance is empty, then the newly created resource instance will be scheduled.
[0020] The resource scheduler is also used for:
[0021] After scheduling the newly created resource instance to the target physical machine, the scheduling information is synchronized with the API server.
[0022] The API server is also used for:
[0023] Receive scheduling information sent by the resource scheduler and update the node name of the newly created resource instance to the node name of the target physical machine.
[0024] Optionally, in the system described above, the system further includes a coordinator, which is specifically used for:
[0025] Receive resource usage information reported by the agent of each physical machine in the resource pool;
[0026] Based on the resource usage of each physical machine, a resource pool resource usage list is generated and stored in the database;
[0027] The resource scheduler is specifically used for:
[0028] Send a resource pool resource usage list retrieval request to the coordinator;
[0029] The coordinator is specifically used for:
[0030] The system receives a request from the resource scheduler to retrieve the resource pool resource usage list, retrieves the resource pool resource usage list from the database, and sends the resource pool resource usage list to the resource scheduler.
[0031] Optionally, in the system described above, the resource scheduler is further configured to:
[0032] Based on the resource requirements and scheduling rules of the newly created resource instance, select multiple candidate physical machines from the resource pool that meet the resource requirements and scheduling rules;
[0033] Based on resource utilization, topology, and load balancing, each candidate physical machine is scored, and the target physical machine is selected from the candidate physical machines whose scores are greater than or equal to the preset scores.
[0034] Optionally, in the system described above, the resource scheduler is specifically used for:
[0035] After scheduling the newly created resource instance to the target physical machine, the coordinator sends a resource reservation request to the database so that the database updates the resource pool resource usage list. The resource reservation request is used to request the database to reserve resources for the newly created resource instance on the target physical machine.
[0036] Optionally, in the system described above, the agent of the target physical machine is also used for:
[0037] After creating the corresponding resources based on the resource requirements of the newly created resource instance, the creation success information is synchronized to the API server.
[0038] The API server is also used for:
[0039] Receive the creation success message sent by the agent of the target physical machine, and update the status of the newly created resource instance to creation success.
[0040] Optionally, in the system described above, the agent of the target physical machine is specifically used for:
[0041] If the corresponding resources cannot be created according to the resource requirements of the newly created resource instance, the coordinator requests the database to update the resource pool resource usage list in order to return the resources reserved on the target physical machine for the newly created resource instance.
[0042] Synchronize creation failure information with the API server;
[0043] The API server is specifically used for:
[0044] Receive the creation failure information sent by the agent of the target physical machine, and update the status of the newly created resource instance to creation failure.
[0045] Secondly, this application provides a resource hybrid scheduling method. The resource hybrid scheduling system includes an API server, resource schedulers corresponding to at least two types of resources, and a resource pool. The resource pool includes multiple physical machines, each physical machine is configured with a physical machine agent, and each physical machine supports the deployment of at least two types of resources. The method includes:
[0046] The API server creates resource instances based on user resource creation requests, whereby the resource creation requests indicate the resource type and resource requirements.
[0047] The resource scheduler discovers a newly created resource instance of the corresponding type, and determines the target physical machine from the resource pool based on the resource pool resource usage list and the resource requirements of the newly created resource instance, and schedules the newly created resource instance to the target physical machine;
[0048] The agent of the target physical machine discovers and schedules newly created resource instances on the physical machine, and creates corresponding resources according to the resource requirements of the newly created resource instances.
[0049] The resource hybrid scheduling system and method provided in this application use a technical means of describing the type attributes of resources in the form of custom resources to achieve matching of corresponding resource schedulers for different types of resources, supporting the hybrid deployment of multiple types of resources on the same physical machine and sharing the same resource pool. Attached Figure Description
[0050] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0051] Figure 1 A signaling interaction diagram for a resource hybrid scheduling system embodiment 1 provided in this application;
[0052] Figure 2 for Figure 1 A flowchart;
[0053] Figure 3 A signaling interaction diagram for a second embodiment of a resource hybrid scheduling system provided in this application;
[0054] Figure 4 for Figure 3A flowchart.
[0055] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0056] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0057] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0058] The terms “preset,” “first,” “second,” “third,” etc. (if applicable) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein.
[0059] In this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.
[0060] First, let me explain the terms used in this application:
[0061] Kubernetes is a brand-new distributed architecture solution based on container technology. It is an open-source container cluster management system from Google, often abbreviated as K8S. In Kubernetes, multiple containers can be created, each running an application instance. Then, through built-in load balancing strategies, the management, discovery, and access of this group of application instances can be achieved, without requiring complex manual configuration and handling by operations personnel.
[0062] kubectl: k8s is a command-line interface used to send client commands.
[0063] API Server: The API Server is the unified entry point for the cluster and the coordinator of all components. It provides interface services via HTTP REST. All CRUD operations on object resources are handled by the API Server before being submitted to Etcd storage. It provides the sole entry point for resource object operations; all other components must operate on resource data through its API. Only the API Server communicates with the storage; other modules access the cluster state through the API Server. First, this ensures the security of cluster state access. Second, it isolates the cluster state access method from the backend storage implementation: the API Server provides state access and its method remains unchanged regardless of changes in the backend storage technology, etcd. As the entry point for the Kubernetes system, it encapsulates the CRUD operations of core objects, providing them to external clients and internal components via RESTful interfaces. It performs "full queries" and "change monitoring" of relevant resource data, completing related business functions in real time.
[0064] The scheduler, responsible for deciding which node to run a Pod on, takes into full account the cluster topology, the current load of each node, and considerations for high availability, performance, data affinity, and other requirements. 1. The scheduler collects and analyzes the resource (memory, CPU, etc.) load of all nodes in the current Kubernetes cluster and then distributes newly created Pods to available nodes accordingly. 2. It monitors all running Pods in the Kubernetes cluster, both undistributed and distributed, in real time. 3. The scheduler also monitors node information; due to frequent node lookups, it caches the latest information locally. 4. Finally, after distributing a Pod to the designated node, the scheduler writes the Pod-related binding information back to the API Server.
[0065] Etcd is the default storage system provided by Kubernetes. It is a distributed, consistent key-value store, mainly used for sharing configuration and service discovery, and storing cluster state data, such as information about Pods, Services, and other objects.
[0066] Node: In Kubernetes, a node is a worker machine in a cluster. Each node runs containers and is managed by Kubernetes. In Kubernetes, a Node refers to a worker node within the cluster. Worker nodes are computing resources in the cluster, responsible for running containerized applications and managing related resources. Each node is an independent physical machine or virtual machine, and together they form the infrastructure of a Kubernetes cluster.
[0067] The node agent is a daemon process running on each node. Its main function is to report the usage of resources such as CPU, memory, and disk on the node, and to create various types of resources, such as Pod resources and VM resources. Once the resource scheduler determines to run a Pod on a certain node, it sends the Pod's specific configuration information to the agent on that node. The agent then creates and runs the container based on this information and reports the running status to the API Server.
[0068] Pod: The smallest unit in a Kubernetes cluster. A Pod can run one or more containers. In most cases, a Pod contains only one container.
[0069] In Kubernetes, Custom Resources (CR) is a way to extend the API server, allowing users to define new resource types according to their own needs; Custom Resources Definitions (CRD) is an extension mechanism that allows users to define and use non-native resource types. CRD is actually part of the Kubernetes API, which allows users to create and manipulate custom resources.
[0070] Custom resource definitions consist of two main components:
[0071] Custom resource definition (API definition): describes the structure, attributes, and behavior of the custom resource;
[0072] Operator: A controller is a custom controller that allows you to manage the lifecycle and behavior of custom resources in a Kubernetes cluster.
[0073] To create a Custom Resource Description (CRD), you can use the APIs and tools provided by Kubernetes. By defining a CRD object, you can tell Kubernetes how to handle custom resources. When creating a CRD, you need to use the specification definition language provided by the Kubernetes API to describe the structure and attributes of the custom resource, including its name, fields, and tags. The CRD controller is a key component for handling the lifecycle and behavior of custom resources. By writing a custom controller, you can define the behavior for creating, updating, and deleting custom resources. Once you have completed writing the CRD and controller, you can deploy them to your Kubernetes cluster.
[0074] Currently, virtual machine scheduling architecture and container scheduling architecture are two independent architectures that manage different physical machines. The virtual machine scheduling architecture can perform virtual machine scheduling on the physical machines it manages, while the container scheduling architecture can perform container scheduling on the physical machines it manages.
[0075] Based on the existing virtual machine scheduling architecture and container scheduling architecture, virtual machines and containers can only be deployed on different physical machines, which undoubtedly increases the deployment cost of physical machines. The independent scheduling methods of virtual machine scheduling architecture and container scheduling architecture are inefficient.
[0076] To address the aforementioned issues, this application provides a method for mixed scheduling of different types of resources, such as containers and virtual machines. This application first creates a CRD for each type of custom resource, such as a virtual machine resource. This allows for the expansion of different types of resources within the Kubernetes cluster. After pre-deploying non-native resource types, users can request the creation of a target type of resource from the API server. The target type can be a native pod resource in the Kubernetes cluster or a non-native user-defined resource. The API server then creates resource instances based on the user's resource creation request. All resource schedulers, by listening to the API server, can discover newly created resource instances of the corresponding type. Based on the resource requirements of the discovered newly created resource instances and the resource pool's resource usage list, they determine the target physical machine from the resource pool and schedule the newly created resource instance to the target physical machine. When the agent of the target physical machine discovers a newly created resource instance scheduled to its own physical machine, it creates the corresponding resource on the physical machine based on the resource requirements of the newly created resource instance.
[0077] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.
[0078] For example, Figure 1 This is a signaling interaction diagram for a resource hybrid scheduling system embodiment one provided in this application. (See diagram below.) Figure 1 As shown, the resource hybrid scheduling system in this embodiment includes an API server, resource schedulers corresponding to at least two types of resources, and a resource pool. The resource pool includes multiple physical machines, each physical machine is configured with a physical machine agent, and the physical machines support the deployment of at least two types of resources.
[0079] The API server is used to create resource instances based on user resource creation requests. Resource creation requests indicate the resource type and resource requirements.
[0080] The resource scheduler is used to discover newly created resource instances of the corresponding type, and based on the resource pool resource usage list and the resource requirements of the newly created resource instance, it determines the target physical machine from the resource pool and schedules the newly created resource instance to the target physical machine.
[0081] The agent of the target physical machine is used to discover newly created resource instances scheduled to this physical machine, and to create corresponding resources based on the resource requirements of the newly created resource instances.
[0082] Optionally, the API server is also used for:
[0083] Receive a custom resource type registration request sent by the user. The custom resource type registration request contains a custom resource definition, which is used to represent the custom resource and the resource type of the custom resource.
[0084] After successful verification, the custom resource definition will be stored in the distributed key-value storage system.
[0085] Users can pre-create CRDs for each type of custom resource through the above operations, enabling the deployment of non-native resource types and facilitating the subsequent creation of corresponding resources on physical machines.
[0086] To better illustrate the working principle of the resource schedulers corresponding to at least two types of resources, this embodiment provides a brief explanation using resource schedulers 1, 2, and 3 corresponding to three types of resources, respectively. Figure 2As shown, users can submit resource creation requests to the API server. These requests specify the resource type and requirements. For example, users can use the kubectl command to request the API server to create a resource. Based on the user's resource creation request, the API server will create a resource instance.
[0087] Assume that the first type of resource 1 corresponds to resource scheduler 1, such as pod resources corresponding to resource scheduler 1; the second type of resource 2 corresponds to resource scheduler 2, such as virtual machine resources (vm resources) corresponding to resource scheduler 2; and the third type of resource 3 corresponds to resource scheduler 3.
[0088] If a user requests the API server to create a resource of type 1, then resource scheduler 1 will find the newly created resource 1 instance by the API server, while resource schedulers 2 and 3 will not find the newly created resource instance of the corresponding type by the API server.
[0089] If a user requests the API server to create a resource of type 2, then resource scheduler 2 will find the newly created resource 2 instance by the API server, while resource scheduler 1 and resource scheduler 3 will not find the newly created resource instance of the corresponding type by the API server.
[0090] If a user requests the API server to create a resource of type 3, then resource scheduler 3 will find the newly created resource 3 instance by the API server, while resource scheduler 1 and resource scheduler 2 will not find the newly created resource instance of the corresponding type by the API server.
[0091] Subsequently, the resource scheduler will determine the target physical machine from the resource pool based on the resource pool resource usage list and the resource requirements of the newly created resource instance of the corresponding type, and schedule the newly created resource instance to the target physical machine. In this way, resource scheduling can be performed through the dedicated resource scheduler corresponding to different types of resources.
[0092] Finally, once the agent of the target physical machine discovers a newly created resource instance scheduled to this physical machine, it will create the corresponding resources according to the resource requirements of the newly created resource instance. Resource scheduler 1, resource scheduler 2, and resource scheduler 3 may all schedule newly created resource instances of the corresponding type to the same physical machine, thereby achieving mixed deployment of different types of resources on the same physical machine.
[0093] For example, a physical machine's agent can listen to the physical machine, obtain all resource instances whose node names are the same as the physical machine's node names, and discover newly created resource instances scheduled to the physical machine.
[0094] This embodiment exemplifies the process of a physical machine's agent creating pod resources and virtual machine resources:
[0095] I. Pod resource creation process:
[0096] 1. Initialize the Informer factory
[0097] 2. Obtain a podinformer instance
[0098] 3. Add callback functions to podInformer: add event to create pod resources; update event to execute the process of updating pod resources; delete event to execute the process of deleting pod resources.
[0099] The creation, updating, and deletion of pod resources are the native Kubernetes kubelet pod creation processes, which will not be elaborated upon in this embodiment.
[0100] II. VM resource creation process:
[0101] 1. Initialize the Informer factory;
[0102] 2. Obtain a vminformer instance;
[0103] 3. Add callback functions to vminformer: add event to create VM resources; update event to execute the VM resource update process; delete event to execute the VM resource deletion process.
[0104] in,
[0105] ① Creating VM resources:
[0106] 1. Obtain the resource requirements parameters such as CPU, memory, and disk for the VM to be created based on the VM instance's spec field;
[0107] 2. Call the virt-install command line to create the VM based on the resource requirement parameters and rendering parameters mentioned above.
[0108] virt-install is a command-line tool used to create and install virtual machines. The virt-install command has many options, which can be broadly categorized as follows:
[0109] General options: Specify the virtual machine's name, memory size, number of VCPUs, and characteristics, etc.;
[0110] Installation method: Specify the installation method, GuestOS type, etc.;
[0111] Storage configuration: Specifies the storage type, location, and attributes;
[0112] Network configuration: Specifies the network type and interface attributes of the network interface, such as MAC address, driver mode, etc.
[0113] Graphics configuration: Defines configurations related to virtual machine display functions, such as VNC-related configurations;
[0114] Device options: Specify file console, sound device, serial interface, parallel interface, display interface, etc.;
[0115] Virtualization platform: virtualization model (HVM or Paravirt), type of emulated CPU platform, type of emulated host, type of hypervisor (such as KVM, Xen, or QEMU), and the UUID of the current virtual machine, etc.
[0116] ② Deleting VM resources:
[0117] 1. Use `virsh destroy` to force shutdown the VM;
[0118] 2. Use the `virsh undefin` command to delete the VM;
[0119] 3. Delete the VM's disk files, network configurations, etc.
[0120] ③ Update VM resources process:
[0121] 1. Invoke the process to delete VM resources and delete the VM;
[0122] 2. Invoke the VM resource creation process to create a VM and achieve the purpose of updating the VM.
[0123] For example, Figure 3 This is a signaling interaction diagram for a second embodiment of a resource hybrid scheduling system provided in this application. This embodiment builds upon the previous embodiment, as follows... Figure 3 As shown, the resource hybrid scheduling system also includes a coordinator, a distributed key-value storage system, and a database.
[0124] Optionally, the API server is specifically used for:
[0125] After creating a resource instance, the resource instance is stored in a distributed key-value storage system;
[0126] Optionally, the resource scheduler is specifically used for:
[0127] By monitoring the API server, newly created resource instances of the corresponding type can be discovered, and the node name of the newly created resource instance can be obtained;
[0128] If the node name of the newly created resource instance is empty, then the newly created resource instance will be scheduled.
[0129] For example, when the resource scheduler starts up, it can obtain the data required by the resource scheduler from the API server through Kubernetes' inspector mechanism and the list + watch mechanism, such as the Pods, VMs and Nodes to be scheduled.
[0130] The process by which the resource scheduler selects Pods to be scheduled for a Pod resource:
[0131] 1. Obtain the kubeconfig file;
[0132] 2. Initialize the clientset;
[0133] 3. Create an Informer factory;
[0134] 4. Obtain a pod informer instance;
[0135] 5. Provide a custom callback function for the pod informer. When the add or update event occurs, check if the spec.nodeName of the object is null. If nodeName == nil, add the object to the scheduling queue.
[0136] 6. Start Informer.
[0137] The process by which the resource scheduler selects VMs to be scheduled for virtual machine resources:
[0138] 1. Obtain the kubeconfig file;
[0139] 2. Initialize the dynamicClient client;
[0140] 3. Define groupVersionResource;
[0141] 4. Create an Informer and add a callback function. When the add or update event occurs, check if the spec.nodeName of the object is null. If nodeName == nil, add the object to the scheduling queue.
[0142] 5. Start Informer.
[0143] Optionally, the resource scheduler is specifically used for:
[0144] Send a request to the coordinator to retrieve the resource pool usage list.
[0145] Optionally, the coordinator is specifically used for:
[0146] Receive the resource pool resource usage list retrieval request sent by the resource scheduler, retrieve the resource pool resource usage list from the database, and send the resource pool resource usage list to the resource scheduler.
[0147] Optionally, the resource scheduler is also used for:
[0148] Based on the resource requirements and scheduling rules of the newly created resource instance, select multiple candidate physical machines from the resource pool that meet the resource requirements and scheduling rules;
[0149] Based on resource utilization, topology, and load balancing, each candidate physical machine is scored, and the target physical machine is selected from the candidate physical machines whose scores are greater than or equal to the preset scores.
[0150] Optionally, the resource scheduler is also used for:
[0151] After scheduling the newly created resource instance to the target physical machine, synchronize the scheduling information with the API server.
[0152] Optionally, the API server is also used for:
[0153] Receive scheduling information sent by the resource scheduler and update the node name of the newly created resource instance to the node name of the target physical machine.
[0154] After the resource scheduler completes its scheduling task, it needs to request the coordinator to pre-reserve the CPU, memory, and disk resources used by newly created resource instances of the corresponding type to prevent them from being occupied. Optionally, the resource scheduler is specifically used for:
[0155] After scheduling the newly created resource instance to the target physical machine, the coordinator sends a resource reservation request to the database so that the database can update the resource pool resource usage list. The resource reservation request is used to request the database to reserve resources for the newly created resource instance on the target physical machine.
[0156] Of course, since there are multiple resource schedulers corresponding to different types of resources in the resource hybrid scheduling system, multiple resource schedulers may simultaneously request the database through the coordinator to reserve resources for newly created resource instances of the corresponding type on the same physical machine. In this case, an optimistic locking mechanism can be used. Only the resource scheduler that acquires the lock first can reserve resources on that physical machine and update the resource pool resource usage list. Optimistic locking is a prior art known to those skilled in the art, and will not be elaborated upon in this embodiment.
[0157] Optionally, the agent of the target physical machine is also used for:
[0158] After creating the corresponding resources based on the resource requirements of the newly created resource instance, the creation success information is synchronized to the API server.
[0159] Optionally, the API server is also used for:
[0160] Receive the creation success message sent by the agent of the target physical machine, and update the status of the newly created resource instance to creation success.
[0161] Optionally, the proxy for the target physical machine is specifically used for:
[0162] If the corresponding resources cannot be created according to the resource requirements of the newly created resource instance, the coordinator requests the database to update the resource pool resource usage list in order to return the resources reserved on the target physical machine for the newly created resource instance.
[0163] Synchronize creation failure information with the API server;
[0164] Optionally, the API server is specifically used for:
[0165] Receive the creation failure information sent by the agent of the target physical machine, and update the status of the newly created resource instance to creation failure.
[0166] In this embodiment, the resource pool resource usage list is obtained by the coordinator based on the CPU, memory, and disk resource usage information reported by the agents of the physical machines. For example, such as... Figure 4 As shown, the coordinator is specifically used for:
[0167] Receive resource usage information reported by the agent of each physical machine in the resource pool;
[0168] Based on the resource usage of each physical machine, a resource pool resource usage list is generated and stored in the database.
[0169] For example, a physical machine's agent can report the usage of resources such as CPU, memory, and disk on its local node in the following ways:
[0170] Use the / proc / stat file to get CPU usage, the free command to get memory usage, and the df command to get disk usage;
[0171] After obtaining the metrics, request the coordinator's metrics interface to periodically push the data to the coordinator so that the coordinator can record it in the database.
[0172] The resource pool resource usage list contains the resource usage information of all physical machines in the resource pool. By viewing the resource pool resource usage list, the resource scheduler can know the resource usage information of all nodes globally. Combined with the resource requirements of newly created resource instances of the corresponding type, the scheduler can determine the target physical machine that meets the conditions from the resource pool, and then bind the newly created resource instance to the target physical machine to complete the scheduling.
[0173] In this embodiment, the agents of multiple physical machines in the resource pool can periodically report the resource usage of their physical machines to the coordinator, or they can report the resource usage of their physical machines to the coordinator only when they receive an instruction from the coordinator. This embodiment does not impose any restrictions on this.
[0174] This application also provides a resource hybrid scheduling method. The resource hybrid scheduling system includes an API server, resource schedulers corresponding to at least two types of resources, and a resource pool. The resource pool includes multiple physical machines, each physical machine is configured with a physical machine agent, and each physical machine supports the deployment of at least two types of resources. The method includes:
[0175] The API server creates resource instances based on users' resource creation requests. The resource creation request indicates the resource type and resource requirements.
[0176] The resource scheduler discovers newly created resource instances of the corresponding type, and determines the target physical machine from the resource pool based on the resource pool resource usage list and the resource requirements of the newly created resource instance, and schedules the newly created resource instance to the target physical machine;
[0177] The agent of the target physical machine discovers and schedules newly created resource instances on the local physical machine, and creates corresponding resources based on the resource requirements of the newly created resource instances.
[0178] In summary, the resource hybrid scheduling system and method provided in this application pre-create a CRD for each type of custom resource, such as virtual machine resources, and deploy non-native resource types in the system to extend different types of resources. After deployment, users can request the API server to create resources of the target type. The target type can be a native pod resource in the Kubernetes cluster or a non-native user-defined resource in the Kubernetes cluster. At this time, the API server will create resource instances according to the user's resource creation request. All resource schedulers can discover newly created resource instances of the corresponding type by listening to the API server, and determine the target physical machine from the resource pool based on the resource requirements of the newly created resource instances of the corresponding type and the resource usage list of the resource pool, and then schedule the newly created resource instances to the target physical machine. When the agent of the target physical machine discovers that a newly created resource instance has been scheduled to the physical machine, it will create the corresponding resources on the physical machine according to the resource requirements of the newly created resource instance.
[0179] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.
[0180] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.
[0181] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as these combinations of technical features do not contradict each other, they should be considered within the scope of this specification.
[0182] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.
[0183] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
Claims
1. A resource hybrid scheduling system, characterized in that, It includes an API server, a resource scheduler corresponding to at least two types of resources, and a resource pool. The resource pool includes multiple physical machines, each physical machine is configured with a physical machine agent, and each physical machine supports the deployment of at least two types of resources. The API server is used to create resource instances based on user resource creation requests, whereby the resource creation requests indicate the resource type and resource requirements. The resource scheduler is used to discover newly created resource instances of the corresponding type, and determine the target physical machine from the resource pool according to the resource pool resource usage list and the resource requirements of the newly created resource instance, and schedule the newly created resource instance to the target physical machine; The agent of the target physical machine is used to discover newly created resource instances scheduled to the physical machine, and to create corresponding resources according to the resource requirements of the newly created resource instances.
2. The system according to claim 1, characterized in that, The API server is also used for: Receive a custom resource type registration request sent by a user. The custom resource type registration request contains a custom resource definition, which is used to characterize the custom resource and the resource type of the custom resource. After successful verification, the custom resource definition is stored in a distributed key-value storage system.
3. The system according to claim 2, characterized in that, The API server is specifically used for: After creating a resource instance, the resource instance is stored in the distributed key-value storage system; The resource scheduler is specifically used for: By monitoring the API server, newly created resource instances of the corresponding type can be detected.
4. The system according to claim 3, characterized in that, The resource scheduler is also used for: After discovering a newly created resource instance of the corresponding type, obtain the node name of the newly created resource instance; If the node name of the newly created resource instance is empty, then the newly created resource instance will be scheduled. The resource scheduler is also used for: After scheduling the newly created resource instance to the target physical machine, the scheduling information is synchronized with the API server. The API server is also used for: Receive scheduling information sent by the resource scheduler and update the node name of the newly created resource instance to the node name of the target physical machine.
5. The system according to claim 4, characterized in that, The system also includes a coordinator, which is specifically used for: Receive resource usage information reported by the agent of each physical machine in the resource pool; Based on the resource usage of each physical machine, a resource pool resource usage list is generated and stored in the database; The resource scheduler is specifically used for: Send a resource pool resource usage list retrieval request to the coordinator; The coordinator is specifically used for: The system receives a request from the resource scheduler to retrieve the resource pool resource usage list, retrieves the resource pool resource usage list from the database, and sends the resource pool resource usage list to the resource scheduler.
6. The system according to claim 5, characterized in that, The resource scheduler is also used for: Based on the resource requirements and scheduling rules of the newly created resource instance, select multiple candidate physical machines from the resource pool that meet the resource requirements and scheduling rules; Based on resource utilization, topology, and load balancing, each candidate physical machine is scored, and the target physical machine is selected from the candidate physical machines whose scores are greater than or equal to the preset scores.
7. The system according to claim 6, characterized in that, The resource scheduler is specifically used for: After scheduling the newly created resource instance to the target physical machine, the coordinator sends a resource reservation request to the database so that the database updates the resource pool resource usage list. The resource reservation request is used to request the database to reserve resources for the newly created resource instance on the target physical machine.
8. The system according to claim 7, characterized in that, The agent of the target physical machine is also used for: After creating the corresponding resources based on the resource requirements of the newly created resource instance, the creation success information is synchronized to the API server. The API server is also used for: Receive the creation success message sent by the agent of the target physical machine, and update the status of the newly created resource instance to creation success.
9. The system according to claim 8, characterized in that, The proxy of the target physical machine is specifically used for: If the corresponding resources cannot be created according to the resource requirements of the newly created resource instance, the coordinator requests the database to update the resource pool resource usage list in order to return the resources reserved on the target physical machine for the newly created resource instance. Synchronize creation failure information with the API server; The API server is specifically used for: Receive the creation failure information sent by the agent of the target physical machine, and update the status of the newly created resource instance to creation failure.
10. A resource hybrid scheduling method, characterized in that, The resource hybrid scheduling system includes an API server, resource schedulers corresponding to at least two types of resources, and a resource pool. The resource pool includes multiple physical machines, each with a physical machine agent. Each physical machine supports the deployment of at least two types of resources. The method includes: The API server creates resource instances based on user resource creation requests, whereby the resource creation requests indicate the resource type and resource requirements. The resource scheduler discovers a newly created resource instance of the corresponding type, and determines the target physical machine from the resource pool based on the resource pool resource usage list and the resource requirements of the newly created resource instance, and schedules the newly created resource instance to the target physical machine; The agent of the target physical machine discovers and schedules newly created resource instances on the physical machine, and creates corresponding resources according to the resource requirements of the newly created resource instances.