Multi-tenant virtual cluster creation method and device, equipment, medium and program product

Abstract

The invention discloses a multi-tenant virtual cluster creating method and device, equipment, a medium and a program product, and relates to the technical field of cloud computing, and the multi-tenant virtual cluster creating method comprises the steps of obtaining a parameter description file, and submitting the parameter description file to a preset physical cluster, the parameter description file is used for describing parameters of the multi-tenant virtual cluster; according to the parameter description file, a multi-tenant virtual cluster is created, and the multi-tenant virtual cluster comprises namespaces corresponding to a plurality of tenants respectively; creating a tenant cluster namespace in the physical cluster according to the parameter description file; and creating a control plane component for the multi-tenant virtual cluster based on the tenant cluster namespace to obtain a control plane, the control plane being used for controlling the multi-tenant virtual cluster according to the tenant demand. According to the method and the device, the isolation among the API resources of the tenants in the cluster is improved, and the flexibility of cluster configuration adjustment is improved.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a method, device, electronic device, computer-readable storage medium, and computer program product for creating a multi-tenant virtual cluster. Background Art

[0002] In the current field of cloud computing, big data, and edge computing, Kubernetes (a container orchestration engine) can be used to use Namespace as an abstract isolation mechanism to divide the cluster into multiple virtual logical parts, thereby creating multiple isolated work environments in a Kubernetes cluster. However, when faced with the need for multi-tenant work management, the namespace of the above method lacks cluster-scoped resource management capabilities. Moreover, since each namespace shares the same control plane, it is difficult to achieve access isolation and flow control between the namespaces corresponding to the control plane. In addition, tenants cannot choose different k8s versions and adjust API (Application Programming Interface) feature switches according to their personalized needs. There are technical problems such as poor isolation between API resources of each tenant in the cluster and poor flexibility in cluster configuration adjustment. Summary of the Invention

[0003] The main purpose of this application is to provide a multi-tenant virtual cluster creation method, device, electronic device, computer-readable storage medium and computer program product, aiming to improve the isolation between API resources of each tenant in the cluster and the flexibility of cluster configuration adjustment.

[0004] To achieve the above objectives, the present application provides a multi-tenant virtual cluster creation method, which is applied to a K8s cluster system. The multi-tenant virtual cluster creation method includes:

[0005] Obtain a parameter description file, and submit the parameter description file to a preset physical cluster, wherein the parameter description file is used to describe the parameters of the multi-tenant virtual cluster;

[0006] Creating a multi-tenant virtual cluster according to the parameter description file, wherein the multi-tenant virtual cluster includes namespaces corresponding to multiple tenants respectively;

[0007] Creating a tenant cluster namespace in the physical cluster according to the parameter description file;

[0008] A control plane component is created for the multi-tenant virtual cluster based on the tenant cluster namespace to obtain a control plane, wherein the control plane is used to control the multi-tenant virtual cluster according to tenant requirements.

[0009] Optionally, after the step of creating a control plane component for the multi-tenant virtual cluster based on the tenant cluster namespace, the method further includes:

[0010] Create a corresponding Pod manager for the Etcd of the multi-tenant virtual cluster, and create a Coredns Pod based on the Pod manager;

[0011] Creating an API server service for the controller plane of the multi-tenant virtual cluster in the physical cluster for tenant access;

[0012] Create a Kube-dns service corresponding to the physical cluster for the multi-tenant virtual cluster to perform domain name resolution.

[0013] Optionally, after the step of creating a Kube-dns service corresponding to the physical cluster, the method further includes:

[0014] An adapter mapper for creating the multi-tenant virtual cluster through the tenant cluster namespace;

[0015] Creating a corresponding service account module, a permission rule module, and a permission rule binding module in the physical cluster based on the adapter mapper, so that the adapter mapper can access the control plane of the physical cluster;

[0016] Based on the number and specifications of the virtual working nodes in the multi-tenant virtual cluster, Pods corresponding to the virtual working nodes are created in the physical cluster, wherein the Pods are mutually exclusively scheduled on different physical nodes.

[0017] Optionally, after the step of creating an adapter mapper corresponding to the multi-tenant virtual cluster through the tenant cluster namespace, the method further includes:

[0018] Monitor the preset core API resources in the multi-tenant virtual cluster and the physical cluster respectively through the adapter mapper;

[0019] When a first event of the multi-tenant virtual cluster is monitored, namespace conversion is performed on the API resource corresponding to the first event;

[0020] Submit the converted API resources to the API server of the physical cluster;

[0021] When a second event of the physical cluster is monitored, namespace conversion is performed on the API resource corresponding to the second event;

[0022] The converted API resource is submitted to the API service of the multi-tenant virtual cluster to execute the second event through the controller of the multi-tenant virtual cluster.

[0023] Optionally, the multi-tenant virtual cluster creation method further includes:

[0024] When a workload created by a tenant is received, the workload Pod corresponding to the workload is scheduled and bound to the virtual node through the multi-tenant virtual cluster, and the workload Pod is associated with the node name of the virtual node;

[0025] Removing the node name associated with the workload Pod through the adapter mapper of the multi-tenant virtual cluster, and adding the affinity attribute of the corresponding virtual node Pod to the workload Pod;

[0026] Submit the workload Pod to the API server of the physical cluster;

[0027] The workload Pod and the virtual node Pod are affinity-scheduled to corresponding physical nodes through the scheduler of the physical cluster.

[0028] Optionally, workloads of multiple tenants in the multi-tenant virtual cluster are mapped to the same tenant cluster namespace through an adapter mapper; network traffic between different namespaces in the multi-tenant virtual cluster is controlled and isolated by the network policy of the CNI network plug-in.

[0029] The present application also provides a multi-tenant virtual cluster creation device, the multi-tenant virtual cluster creation device comprising:

[0030] A parameter acquisition module is used to acquire a parameter description file and submit the parameter description file to a preset physical cluster, wherein the parameter description file is used to describe the parameters of the multi-tenant virtual cluster;

[0031] A cluster creation module, configured to create a multi-tenant virtual cluster according to the parameter description file, wherein the multi-tenant virtual cluster includes namespaces corresponding to a plurality of tenants respectively;

[0032] A space creation module, configured to create a tenant cluster namespace within the physical cluster according to the parameter description file;

[0033] A control plane creation module is configured to create a control plane component for the multi-tenant virtual cluster based on the tenant cluster namespace to obtain a control plane, wherein the control plane is used to control the multi-tenant virtual cluster according to tenant requirements.

[0034] The present application also provides an electronic device, which is a physical device, and includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the multi-tenant virtual cluster creation method as described above.

[0035] The present application also provides a readable storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores a program for implementing the multi-tenant virtual cluster creation method. The program for implementing the multi-tenant virtual cluster creation method is executed by a processor to implement the steps of the multi-tenant virtual cluster creation method as described above.

[0036] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned multi-tenant virtual cluster creation method when executed by a processor.

[0037] The present application provides a method for creating a multi-tenant virtual cluster, which is applied to a Kubernetes cluster system. The multi-tenant virtual cluster creation method first obtains a parameter description file and submits the parameter description file to a preset physical cluster, wherein the parameter description file is used to describe the parameters of the multi-tenant virtual cluster, so that users can create a multi-tenant virtual cluster that meets their own personalized needs by adjusting the parameter description file. Then, the multi-tenant virtual cluster is created according to the parameter description file, wherein the multi-tenant virtual cluster includes namespaces corresponding to multiple tenants, so that resource isolation between tenants is achieved by each tenant using a separate namespace. Then, a tenant cluster namespace is created within the physical cluster according to the parameter description file, wherein the multiple namespaces in the virtual cluster share the underlying physical cluster, thereby improving resource utilization. Then, a control plane component is created for the multi-tenant virtual cluster based on the tenant cluster namespace to obtain a control plane, wherein the control plane is used to control the multi-tenant virtual cluster according to tenant requirements. Each multi-tenant virtual cluster corresponds to a control plane, so that users can customize the version of the virtual cluster created, and tenants can separately configure corresponding namespaces for different services within the multi-tenant virtual cluster, thereby achieving more refined business resource management and control, and further improving the flexibility of Kubernetes cluster configuration adjustment. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0040] Figure 1 This is a flowchart of the method for creating a multi-tenant virtual cluster in Example 1 of the present application;

[0041] Figure 2 This is a schematic diagram of the multi-tenant resource architecture in the K8s cluster system in an embodiment of the present application;

[0042] Figure 3 Schematic diagram of the working principle of the adapter mapper in the embodiment of the present application;

[0043] Figure 4 A schematic diagram of the working nodes and scheduling principles of a multi-tenant virtual cluster in an embodiment of the present application;

[0044] Figure 5 This is a schematic diagram of a multi-tenant network architecture in an embodiment of the present application;

[0045] Figure 6 A schematic diagram of a multi-tenant storage architecture in an embodiment of the present application;

[0046] Figure 7 This is a schematic diagram of the structure of a multi-tenant virtual cluster creation device in an embodiment of the present application;

[0047] Figure 8 This is a schematic diagram of the device structure of the hardware operating environment involved in the multi-tenant virtual cluster creation method in the embodiment of the present application.

[0048] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0049] To make the above-mentioned purposes, features, and advantages of the present application more clearly understood, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making any creative work are within the scope of protection of this application.

[0050] Example 1

[0051] To meet the needs of multi-tenant workload management, there are currently two common Kubernetes cluster management approaches. The first is based on Kubernetes namespaces, which divide workloads into multiple virtualized logical parts. The key idea is to create multiple isolated workload environments within a single Kubernetes cluster. This approach achieves resource isolation between tenants by creating multiple namespaces for a multi-tenant Kubernetes cluster, with each tenant using a separate namespace. However, namespaces are a soft isolation mechanism and have the following drawbacks: 1. They lack cluster-scoped resource control capabilities; 2. They share a control plane, making it impossible to isolate and limit access to the control plane. Tenants cannot choose different Kubernetes versions based on their needs, nor can they adjust API feature toggles; 3. Complex business resource management is difficult within a single namespace. Another approach is to adopt a Kubernetes multi-cluster strategy, an architecture and strategy that connects and manages multiple Kubernetes clusters together. A multi-cluster architecture allows multiple Kubernetes clusters to be treated as a single, large cluster, providing higher availability, flexibility, and scalability. However, this approach has the following drawbacks: high management and resource costs, low physical cluster resource utilization, difficulty in sharing between clusters, and a relatively complex scaling process, requiring advance capacity planning.

[0052] In order to overcome the technical defects of the above-mentioned prior art, the embodiment of the present application provides a multi-tenant virtual cluster creation method applied to the K8s cluster system, please refer to Figure 1 The steps of the multi-tenant virtual cluster creation method include:

[0053] Step S10: obtaining a parameter description file and submitting the parameter description file to a preset physical cluster, wherein the parameter description file is used to describe the parameters of the multi-tenant virtual cluster;

[0054] The parameter description file is written by the cluster administrator according to the quota and k8s version requirements to describe the tenant cluster API YAML (used to express the data serialization format). Exemplarily, the parameter description file may include:

[0055]

[0056] After writing the API YAML, you can submit the YAML to the API server of the physical cluster.

[0057] Step S20: creating a multi-tenant virtual cluster according to the parameter description file, wherein the multi-tenant virtual cluster includes namespaces corresponding to a plurality of tenants respectively;

[0058] After the apiserver of the physical cluster receives the api yaml, it can create a multi-tenant virtual cluster through the built-in controller. In the multi-tenant virtual cluster, it can include namespaces corresponding to multiple tenants. Each tenant's namespace is used to bear the tenant's own workload. The namespaces are independent of each other, and different tenant clusters can create namespaces and API resources with the same name without causing conflicts. The control plane components of the tenant cluster are all deployed in container mode. Each component runs in the form of a Pod (the smallest basic unit of deployment and management in a kubernetes cluster) and is co-located with the workload of the application namespace in the tenant namespace of the k8s physical cluster. It should also be noted that multiple different multi-tenant virtual clusters can be created based on the same physical cluster, so multiple tenant virtual clusters share the underlying k8s physical cluster, which has high resource utilization.

[0059] It's also important to note that all workloads and API resources in a multi-tenant virtual cluster are mapped to a namespace in the Kubernetes physical cluster, managed by the mapping adapter controller within the multi-tenant virtual cluster. Furthermore, tenants can directly interact with the virtual cluster's API server and utilize all standard Kubernetes cluster features. These multi-tenant virtual clusters are applied for and used on demand, eliminating the need for physical cluster resource management and partitioning. This reduces the cost of creating and maintaining these clusters, making expansion and contraction easy. Capacity allocation for each namespace is simple, and tenant namespaces are highly isolated from one another.

[0060] Step S30: creating a tenant cluster namespace in the physical cluster according to the parameter description file;

[0061] It should be noted that the controller in the physical cluster creates a tenant namespace named TenantCluster (tenant cluster)-demo (example) in the physical cluster based on the API specifications in the obtained parameter description file. This TenantCluster-demo namespace is deployed in the actual physical cluster and corresponds one-to-one with the multi-tenant virtual cluster. It can be considered a mapping of the multi-tenant virtual cluster in the physical cluster and is used to implement control, component creation, resource management, etc. of the multi-tenant cluster through this TenantCluster-demo namespace. Only administrators have management rights to the K8s physical cluster, while tenants only have management rights to the multi-tenant virtual cluster. This allows for hierarchical management of virtual clusters and actual physical clusters in the K8s cluster, improving the standardization of the K8s cluster.

[0062] Step S40: creating a control plane component for the multi-tenant virtual cluster based on the tenant cluster namespace to obtain a control plane, wherein the control plane is used to control the multi-tenant virtual cluster according to tenant requirements.

[0063] Specifically, a deployment is used to create the tenant cluster's Kubernetes control plane components, including the apiserver, controller-manager, and scheduler, in the physical cluster TenantCluster-demo namespace, with a replica count of 3. It should be noted that the control plane components belong to the kube-system namespace within the tenant cluster. The kube-system namespace is a namespace used to deploy the control plane and adapter mapper in a multi-tenant virtual cluster.

[0064] In addition, the technical solution of the embodiment of the present application implements the lifecycle management of the virtual cluster with a declarative API and controller mode. The administrator can submit API resource objects of corresponding specifications according to the needs of the tenants to conveniently create and manage the cluster. In the technical solution of the embodiment of the present application, each multi-tenant virtual cluster has a complete control plane, including highly available apiserver, etcd, controller-manager, scheduler and other control components; moreover, the multi-tenant virtual cluster can customize the K8s version and feature switch of the cluster; in addition, multiple namespaces can be created in a multi-tenant virtual cluster to perform logical division of application services of different tenants; the multi-tenant virtual cluster can extend the CRD (custom resource) API and operator controller, and can complete cluster-level CRD API integration. The technical solution of the embodiment of the present application can not only provide complete tenant cluster functions and flexibility, but also quickly create, expand, and delete tenant virtual clusters, and automatically manage the cluster lifecycle.

[0065] It should also be noted that compared with the conventional multi-K8s cluster model, this technical solution does not use a separate physical node pool and does not require the planning and deployment of K8s physical clusters. Instead, it adopts a declarative API and controller model. The cluster administrator submits the corresponding cluster description API according to the needs of the tenant, and the operator controller tunes and completes the creation and operation and maintenance of the virtual cluster.

[0066] Furthermore, in a feasible embodiment, after the step of creating a control plane component for the multi-tenant virtual cluster based on the tenant cluster namespace, the method may further include:

[0067] Step S50: Create a corresponding Pod manager for the Etcd of the multi-tenant virtual cluster, and create a Coredns Pod based on the Pod manager;

[0068] First, a statefulset (which can be understood as a Pod manager) can be created for the etcd of the tenant cluster in the TenantCluster-demo namespace in the physical cluster. In the statefulset, each Pod has a stable network identifier that can be accessed through DNS (Domain Name System) or other service discovery mechanisms. The identifier is automatically assigned by the statefulset controller when the Pod is created and remains unchanged when the Pod is restarted, which enables stateful applications to be stably accessed on the network. The embodiment of the present application also includes creating a Pod of Coredns (a DNS server software) for domain name resolution.

[0069] Step S60: creating an API server service on the controller plane of the multi-tenant virtual cluster in the physical cluster for tenant access;

[0070] Step S70: Create a Kube-dns service corresponding to the physical cluster for the multi-tenant virtual cluster to perform domain name resolution.

[0071] In this embodiment of the present application, an API server (application programming interface server) service is created in the physical cluster as the control plane of the multi-tenant virtual cluster for tenant access. A kube-dns service (a core component for service discovery and DNS resolution in the Kubernetes cluster) is also created for domain name resolution within the tenant cluster.

[0072] Furthermore, after the step of creating the Kube-dns service corresponding to the physical cluster, the method may further include:

[0073] Step S71, creating an adapter mapper of the multi-tenant virtual cluster through the tenant cluster namespace;

[0074] In the process of creating the adapter mapper, the adapter mapper of the multi-tenant virtual cluster is mainly created in the TenantCluster-demo namespace of the physical cluster. It can be understood that the TenantCluster-demo namespace corresponds to the multi-tenant virtual cluster, and the management, creation, and resource division of the multi-tenant virtual cluster all need to be implemented through the namespace in the real physical cluster. Therefore, after the adapter mapper of the multi-tenant virtual cluster is created through the tenant cluster namespace, the adapter mapper exists in the kube-system namespace of the multi-tenant virtual cluster, and the physical cluster resources corresponding to the adapter mapper are available in the TenantCluster-demo namespace, thereby completing the mapping between virtual cluster resources and physical cluster resources.

[0075] Step S72: creating a corresponding service account module, permission rule module, and permission rule binding module in the physical cluster based on the adapter mapper, so that the adapter mapper can access the control plane of the physical cluster;

[0076] In the adapter mapper, it is also necessary to create corresponding ServiceAccount (service account module), ClusterRole (authority rule module), and ClusterRoleBinding (authority rule binding module) for the adapter mapper to access the physical cluster control plane.

[0077] Step S73: Based on the number and specifications of the virtual working nodes in the multi-tenant virtual cluster, create Pods corresponding to the virtual working nodes in the physical cluster, wherein the Pods are mutually exclusively scheduled on different physical nodes.

[0078] Finally, based on the number and specifications of the virtual worker nodes, virtual node pods are created in the physical cluster. It should be noted that virtual node pods are mutually exclusive scheduled to different physical nodes to avoid node conflicts.

[0079] For example, referring to Figure 2In the K8s cluster system, there is a physical cluster, which includes tenant cluster namespace 1 (corresponding to the TenantCluster-demo namespace) and tenant cluster namespace 2. Each tenant cluster namespace corresponds to a tenant virtual cluster (i.e., a multi-tenant virtual cluster). The tenant cluster namespace also includes the tenant virtual cluster control plane for controlling the tenant virtual cluster, including modules such as apiserver, controller-manager, and scheduler. In addition, the tenant cluster namespace also includes configmap (configuration center) and secret (used to store sensitive information). The tenant cluster namespace corresponding to the adapter mapper of the tenant virtual cluster also includes deployment (used to create Pod). In the tenant virtual cluster, at least tenant namespaces (such as A and B) are included. The tenant namespaces also have configmaps, secrets, and deployments corresponding to the configmaps and secrets in the physical cluster respectively. In addition, the tenant virtual cluster also includes the kube-system namespace, which includes a control plane composed of modules such as apiserver, controller-manager, and scheduler, for tenants to personalize and adjust the multi-tenant virtual cluster. In addition, each tenant virtual cluster has an adapter mapper deployed in the kube-system namespace to complete the mapping between virtual cluster resources and physical cluster resources. The adapter mapper has the same life cycle as the tenant virtual cluster and is created at the same time when the administrator creates the virtual cluster. All workloads and API resources of the tenant cluster are mapped by the adapter mapper to the namespace of the tenant corresponding to the k8s physical cluster, completing the physical layer isolation.

[0080] Therefore, the technical solution of the embodiment of the present application has the following advantages: different multi-tenant virtual clusters can create namespaces and API resources with the same name without causing conflicts; the control plane components of the multi-tenant virtual cluster are all deployed in container mode, and each component runs in the form of Pod, and is co-located with the workload of the application namespace in the tenant namespace of the k8s physical cluster; the workload of each tenant is mapped and created in the namespace of the k8s physical cluster through the adapter mapper of each multi-tenant virtual cluster, thereby enabling multiple virtual clusters to share the same physical cluster and improving resource utilization.

[0081] Furthermore, in a feasible embodiment, after the step of creating the adapter mapper corresponding to the multi-tenant virtual cluster through the tenant cluster namespace, the method may further include:

[0082] Step A10: monitoring the preset core API resources in the multi-tenant virtual cluster and the physical cluster respectively through the adapter mapper;

[0083] It should be noted that, referring to Figure 3 The adapter mapper is an asynchronous operator controller that can list and watch the tenant's virtual cluster API server and physical cluster API server simultaneously. Specifically, the adapter mapper connects to the kube-apiserver in the tenant's virtual cluster and the kube-apiserve in the physical cluster, respectively, and monitors the following preset k8s core API resources of the virtual and physical clusters: Services, Endpoints (endpoints that implement services), Configmaps, Secrets, Events, Pods, PVCs (Persistent Volumes, storage resources configured in the cluster), and virtual node pods.

[0084] Step A20: When a first event of the multi-tenant virtual cluster is monitored, namespace conversion is performed on the API resource corresponding to the first event;

[0085] Step A30: submitting the converted API resources to the API server of the physical cluster;

[0086] When the adapter mapper listens to the first event of the core API resource above on the virtual cluster control plane (for example, an add, delete, or modify event), it performs namespace conversion on the corresponding resource (for example, converting pod1 in the default namespace of virtual cluster A to pod1-ns-default in the namespace of the physical cluster named TenantCluster-A), and then submits the converted API resource to the apiserver of the physical cluster. The technical solution of the embodiment of the present application achieves event resource synchronization between the virtual cluster and the physical cluster by converting and sending events of the multi-tenant virtual cluster.

[0087] Step A40: When a second event of the physical cluster is monitored, namespace conversion is performed on the API resource corresponding to the second event;

[0088] Step A50: Submit the converted API resource to the API service of the multi-tenant virtual cluster to execute the second event through the controller of the multi-tenant virtual cluster.

[0089] The adapter mapper monitors all core resources of the tenant cluster namespace of the physical cluster, converts and adapts the second event it monitors (for example, converts pod1:pod1-ns-default in the TenantCluster-A namespace of the physical cluster into pod1 under the default namespace of virtual cluster A), and then submits the converted API resources to the apiserver of the virtual cluster. Finally, the controller of the virtual cluster completes the tuning logic and lifecycle management (corresponding to the first event and the second event). Compared with the traditional multi-cluster isolation solution, the tenant cluster controller automatically tunes the status of the virtual cluster according to the API resource object, and the management cost is lower. It should be noted that the adapter mapper plays the role of a resource conversion bridge between the virtual cluster and the physical cluster in the technical solution of the embodiment of the present application, thereby realizing the synchronization of events and resources between the virtual cluster and the physical cluster.

[0090] Furthermore, in another feasible embodiment, the multi-tenant virtual cluster creation method may further include:

[0091] Step B10: When a workload created by a tenant is received, the workload Pod corresponding to the workload is scheduled and bound to a virtual node through the multi-tenant virtual cluster, and the workload Pod is associated with the node name of the virtual node;

[0092] Reference Figure 4 In the technical solution of the embodiment of the present application, a two-level scheduling method is adopted when scheduling workloads: first, the scheduler of the tenant virtual cluster control plane schedules the workload to the virtual node, and then the physical cluster scheduler schedules the Pod corresponding to the virtual node to the physical node, and co-locates the tenant workload and the virtual node Pod.

[0093] It should be noted that when the administrator creates a tenant virtual cluster, he or she can configure the corresponding virtual node pool and node instance according to the tenant quota and virtual cluster specifications. Among them, the virtual node corresponds to a Pod with kubelet-like functions (agent component on the node) and is scheduled to the physical node pool. It is responsible for the status maintenance and heartbeat reporting of the virtual node (node) API, and realizes the lifecycle management of Pod network, volume (storage volume) and other resources through interaction with the physical node kubelet. The virtual node pod is mapped and adapted to the physical cluster by the adapter mapper, and has a mandatory anti-affinity feature between each other to ensure that each virtual node pod is scheduled to a different host node.

[0094] Specifically, after a tenant (Actor) creates a workload (e.g., kubectl, the command-line tool for the Kubernetes cluster), the virtual cluster's K8s scheduler first schedules and binds the Pod to the virtual node (including complete filter and score execution strategies), and associates the nodename of the virtual node.

[0095] Step B20: removing the node name associated with the workload Pod through the adapter mapper of the multi-tenant virtual cluster, and adding the affinity attribute of the corresponding virtual node Pod to the workload Pod;

[0096] Step B30: submitting the workload Pod to the API server of the physical cluster;

[0097] In an embodiment of the present application, after the workload Pod is associated with the node name, it is further necessary to remove the association between the workload Pod and the node name through the adapter mapper of the virtual cluster. This is because in the workload scheduling process, the nodename of the virtual node only plays an associative role, which is used to correspond the workload Pod with the virtual node Pod to facilitate increasing its affinity attributes. After the workload Pod is attached with the affinity attribute of the virtual node, it is submitted to the apiserver of the physical cluster for further scheduling processing by the physical cluster.

[0098] Step B40: Affinity-schedule the workload Pod and the virtual node Pod to the corresponding physical node through the scheduler of the physical cluster.

[0099] After the apiserver of the physical cluster receives the workload Pod with affinity attributes to the virtual node, the scheduler of the physical cluster can schedule the workload Pod and the virtual node Pod to the corresponding physical node based on affinity, thereby realizing the mapping and scheduling between the virtual cluster and the real physical cluster.

[0100] This technical solution adopts a two-level scheduling approach, which can not only meet the tenants' needs for workload failure domain control, node grouping and resource optimization configuration, but also meet the efficient utilization of physical resources.

[0101] Furthermore, in a feasible embodiment, the workloads of multiple tenants in the multi-tenant virtual cluster are mapped to the same tenant cluster namespace through an adapter mapper; the network traffic between different namespaces in the multi-tenant virtual cluster is controlled and isolated by the network policy of the CNI network plug-in.

[0102] Figure 5The tenant virtual cluster 1 shown in the figure can include tenant namespace 1 (and may also include tenant namespace 2, etc.). The workload of tenant namespace 1 is mapped to the tenant cluster namespace corresponding to the tenant VPC of the physical cluster through the adapter mapper in the kube-system namespace. The workload is then mapped to the physical node Pod by resolving DNS requests and service requests. It should also be noted that the SVC (Switching Virtual Circuit) is the switching virtual circuit between Pod-1 and Pod-2.

[0103] In the technical solution of the embodiment of the present application, the Pod network of the tenant cluster namespace is protected by the SDN (Software Defined Network) network VPC (Virtual Private Cloud) of the underlying infrastructure. Different VPCs are isolated by network protocols and cannot communicate with each other by default. Network hard isolation is achieved with the help of SDN overlay vxlan (Virtual Extensible Local Area Network). The network traffic between different namespaces in a multi-tenant virtual cluster is controlled and isolated by the network policy of the CNI network plug-in.

[0104] On the other hand, internal network traffic within a tenant cluster namespace is considered intra-VPC traffic at the physical cluster level. When tenant network connectivity is required, Pods communicate with each other using regular internal IP traffic, without NAT (Network Address Translation). Network traffic between Pods and services, if accessed using DNS names, must be resolved to the corresponding service through the coreDNS service within the tenant namespace. If Cluster-IP (intranet IP) is used, load balancing is achieved using the CNI network plugin or kube-proxy.

[0105] In a feasible embodiment, in a multi-tenant virtual cluster, container storage is implemented by an adapter mapper. Figure 6When an administrator creates a multi-tenant virtual cluster, they open the physical cluster's storage class to the tenant virtual clusters as needed. When tenants create PVCs 1-3 (Persistent Volume Claims), the virtual cluster's adapter mapper modifies the PVC's name and attributes and submits it to the physical cluster's tenant cluster namespace. The physical cluster's CSI driver then performs lifecycle management, including PVC creation and mounting.

[0106] Example 2

[0107] The present invention provides a multi-tenant virtual cluster creation device. Figure 7 , the multi-tenant virtual cluster creation device includes:

[0108] A parameter acquisition module 10 is configured to acquire a parameter description file and submit the parameter description file to a preset physical cluster, wherein the parameter description file is used to describe the parameters of the multi-tenant virtual cluster;

[0109] A cluster creation module 20 is configured to create a multi-tenant virtual cluster according to the parameter description file, wherein the multi-tenant virtual cluster includes namespaces corresponding to a plurality of tenants respectively;

[0110] A space creation module 30 is configured to create a tenant cluster namespace in the physical cluster according to the parameter description file;

[0111] The control plane creation module 40 is configured to create a control plane component for the multi-tenant virtual cluster based on the tenant cluster namespace to obtain a control plane, wherein the control plane is used to control the multi-tenant virtual cluster according to tenant requirements.

[0112] Optionally, the multi-tenant virtual cluster creation device is further configured to:

[0113] Create a corresponding Pod manager for the Etcd of the multi-tenant virtual cluster, and create a Coredns Pod based on the Pod manager;

[0114] Creating an API server service for the controller plane of the multi-tenant virtual cluster in the physical cluster for tenant access;

[0115] Create a Kube-dns service corresponding to the physical cluster for the multi-tenant virtual cluster to perform domain name resolution.

[0116] Optionally, the multi-tenant virtual cluster creation device is further configured to:

[0117] An adapter mapper for creating the multi-tenant virtual cluster through the tenant cluster namespace;

[0118] Creating a corresponding service account module, a permission rule module, and a permission rule binding module in the physical cluster based on the adapter mapper, so that the adapter mapper can access the control plane of the physical cluster;

[0119] Based on the number and specifications of the virtual working nodes in the multi-tenant virtual cluster, Pods corresponding to the virtual working nodes are created in the physical cluster, wherein the Pods are mutually exclusively scheduled on different physical nodes.

[0120] Optionally, the multi-tenant virtual cluster creation device further includes a monitoring module, and the monitoring module is configured to:

[0121] Monitor the preset core API resources in the multi-tenant virtual cluster and the physical cluster respectively through the adapter mapper;

[0122] When a first event of the multi-tenant virtual cluster is monitored, namespace conversion is performed on the API resource corresponding to the first event;

[0123] Submit the converted API resources to the API server of the physical cluster;

[0124] When a second event of the physical cluster is monitored, namespace conversion is performed on the API resource corresponding to the second event;

[0125] The converted API resource is submitted to the API service of the multi-tenant virtual cluster to execute the second event through the controller of the multi-tenant virtual cluster.

[0126] Optionally, the multi-tenant virtual cluster creation device is used to:

[0127] When a workload created by a tenant is received, the workload Pod corresponding to the workload is scheduled and bound to the virtual node through the multi-tenant virtual cluster, and the workload Pod is associated with the node name of the virtual node;

[0128] Removing the node name associated with the workload Pod through the adapter mapper of the multi-tenant virtual cluster, and adding the affinity attribute of the corresponding virtual node Pod to the workload Pod;

[0129] Submit the workload Pod to the API server of the physical cluster;

[0130] The workload Pod and the virtual node Pod are affinity-scheduled to corresponding physical nodes through the scheduler of the physical cluster.

[0131] Optionally, workloads of multiple tenants in the multi-tenant virtual cluster are mapped to the same tenant cluster namespace through an adapter mapper; network traffic between different namespaces in the multi-tenant virtual cluster is controlled and isolated by the network policy of the CNI network plug-in.

[0132] The multi-tenant virtual cluster creation device provided in the embodiment of the present application adopts the multi-tenant virtual cluster creation method in the above embodiment, which can solve the technical problems of poor isolation between API resources of each tenant in the cluster and poor flexibility of cluster configuration adjustment. Compared with the existing technology, the beneficial effects of the multi-tenant virtual cluster creation device provided in the embodiment of the present application are the same as the beneficial effects of the multi-tenant virtual cluster creation method provided in the above embodiment, and the other technical features of the multi-tenant virtual cluster creation device are the same as the features disclosed in the above embodiment method, which are not repeated here.

[0133] Example 3

[0134] An embodiment of the present application provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the multi-tenant virtual cluster creation method of the above-mentioned embodiment 1.

[0135] Reference below Figure 8 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions: tablet computers), PMPs (Portable Media Players: portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 8 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0136] like Figure 8As shown, the electronic device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. Various programs and data required for the operation of the electronic device are also stored in the RAM 1004. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (ICD), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have instead.

[0137] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0138] The electronic device provided in this application, using the multi-tenant virtual cluster creation method in the above-mentioned embodiment, can solve the technical problems of poor isolation between API resources of each tenant in the cluster and poor flexibility in cluster configuration adjustment. Compared with the prior art, the beneficial effects of the electronic device provided in the embodiment of this application are the same as the beneficial effects of the multi-tenant virtual cluster creation method provided in the above-mentioned embodiment, and the other technical features of the electronic device are the same as the features disclosed in the method of the previous embodiment, which are not repeated here.

[0139] It should be understood that various parts of the present disclosure can be implemented with hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in an appropriate manner.

[0140] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0141] Example 4

[0142] An embodiment of the present application provides a computer-readable storage medium having computer-readable program instructions stored thereon, and the computer-readable program instructions are used to execute the multi-tenant virtual cluster creation method in the above-mentioned embodiment 1.

[0143] The computer-readable storage medium provided in the embodiments of the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0144] The computer-readable storage medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0145] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by an electronic device, the electronic device: obtains a parameter description file and submits the parameter description file to a preset physical cluster, wherein the parameter description file is used to describe the parameters of the multi-tenant virtual cluster; creates a multi-tenant virtual cluster according to the parameter description file, wherein the multi-tenant virtual cluster includes namespaces corresponding to multiple tenants respectively; creates a tenant cluster namespace within the physical cluster according to the parameter description file; creates a control plane component for the multi-tenant virtual cluster based on the tenant cluster namespace to obtain a control plane, wherein the control plane is used to control the multi-tenant virtual cluster according to tenant requirements.

[0146] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smaillk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0147] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0148] The modules involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0149] The computer-readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions for executing the aforementioned multi-tenant virtual cluster creation method. This computer-readable storage medium can address the technical issues of poor isolation between API resources of tenants in a cluster and poor flexibility in cluster configuration adjustment. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this embodiment of the application are the same as those of the multi-tenant virtual cluster creation method provided in Example 1 above, and are not further elaborated here.

[0150] Example 5

[0151] An embodiment of the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned multi-tenant virtual cluster creation method.

[0152] The computer program product provided in this application can address the technical issues of poor isolation between API resources of tenants in a cluster and poor flexibility in cluster configuration adjustment. Compared to the prior art, the beneficial effects of the computer program product provided in this embodiment of the application are the same as those of the multi-tenant virtual cluster creation method provided in Example 1 above, and are not further elaborated here.

[0153] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. A method for creating a multi-tenant virtual cluster, characterized in that: Applied to the K8s cluster system, the multi-tenant virtual cluster creation method includes: Obtain a parameter description file, and submit the parameter description file to a preset physical cluster, wherein the parameter description file is used to describe the parameters of the multi-tenant virtual cluster; Creating a multi-tenant virtual cluster according to the parameter description file, wherein the multi-tenant virtual cluster includes namespaces corresponding to multiple tenants respectively; Creating a tenant cluster namespace in the physical cluster according to the parameter description file; A control plane component is created for the multi-tenant virtual cluster based on the tenant cluster namespace to obtain a control plane, wherein the control plane is used to control the multi-tenant virtual cluster according to tenant requirements.

2. The multi-tenant virtual cluster creation method according to claim 1, wherein: After the step of creating a control plane component for the multi-tenant virtual cluster based on the tenant cluster namespace, the method further includes: Create a corresponding Pod manager for the Etcd of the multi-tenant virtual cluster, and create a Coredns Pod based on the Pod manager; Creating an API server service for the controller plane of the multi-tenant virtual cluster in the physical cluster for tenant access; Create a Kube-dns service corresponding to the physical cluster for the multi-tenant virtual cluster to perform domain name resolution.

3. The multi-tenant virtual cluster creation method according to claim 2, wherein: After the step of creating the Kube-dns service corresponding to the physical cluster, the method further includes: An adapter mapper for creating the multi-tenant virtual cluster through the tenant cluster namespace; Creating a corresponding service account module, a permission rule module, and a permission rule binding module in the physical cluster based on the adapter mapper, so that the adapter mapper can access the control plane of the physical cluster; Based on the number and specifications of the virtual working nodes in the multi-tenant virtual cluster, Pods corresponding to the virtual working nodes are created in the physical cluster, wherein the Pods are mutually exclusively scheduled on different physical nodes.

4. The method for creating a multi-tenant virtual cluster according to claim 3, wherein: After the step of creating an adapter mapper corresponding to the multi-tenant virtual cluster through the tenant cluster namespace, the method further includes: Monitor the preset core API resources in the multi-tenant virtual cluster and the physical cluster respectively through the adapter mapper; When a first event of the multi-tenant virtual cluster is monitored, namespace conversion is performed on the API resource corresponding to the first event; Submit the converted API resources to the API server of the physical cluster; When a second event of the physical cluster is monitored, namespace conversion is performed on the API resource corresponding to the second event; The converted API resource is submitted to the API service of the multi-tenant virtual cluster to execute the second event through the controller of the multi-tenant virtual cluster.

5. The method for creating a multi-tenant virtual cluster according to claim 1, wherein: The multi-tenant virtual cluster creation method further includes: When a workload created by a tenant is received, the workload Pod corresponding to the workload is scheduled and bound to the virtual node through the multi-tenant virtual cluster, and the workload Pod is associated with the node name of the virtual node; Removing the node name associated with the workload Pod through the adapter mapper of the multi-tenant virtual cluster, and adding the affinity attribute of the corresponding virtual node Pod to the workload Pod; Submit the workload Pod to the API server of the physical cluster; The workload Pod and the virtual node Pod are affinity-scheduled to corresponding physical nodes through the scheduler of the physical cluster.

6. The method for creating a multi-tenant virtual cluster according to any one of claims 1 to 5, wherein: The workloads of multiple tenants in the multi-tenant virtual cluster are all mapped to the same tenant cluster namespace through the adapter mapper; the network traffic between different namespaces in the multi-tenant virtual cluster is controlled and isolated by the network policy of the CNI network plug-in.

7. A multi-tenant virtual cluster creation device, characterized in that: The multi-tenant virtual cluster creation device includes: A parameter acquisition module is used to acquire a parameter description file and submit the parameter description file to a preset physical cluster, wherein the parameter description file is used to describe the parameters of the multi-tenant virtual cluster; A cluster creation module, configured to create a multi-tenant virtual cluster according to the parameter description file, wherein the multi-tenant virtual cluster includes namespaces corresponding to a plurality of tenants respectively; A space creation module, configured to create a tenant cluster namespace within the physical cluster according to the parameter description file; A control plane creation module is configured to create a control plane component for the multi-tenant virtual cluster based on the tenant cluster namespace to obtain a control plane, wherein the control plane is used to control the multi-tenant virtual cluster according to tenant requirements.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the multi-tenant virtual cluster creation method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program for implementing a multi-tenant virtual cluster creation method, and the program for implementing a multi-tenant virtual cluster creation method is executed by a processor to implement the steps of the multi-tenant virtual cluster creation method according to any one of claims 1 to 6.

10. A computer program product, characterized in that When the computer program is executed by a processor, the steps of the multi-tenant virtual cluster creation method according to any one of claims 1 to 6 are implemented.

Images