Multi-cluster access method and system

Abstract

The present disclosure relates to the technical field of computers. A multi-cluster access method and system are provided. The method includes steps of synchronizing resources between a primary cluster and a sub-cluster, sending related information of the synchronized resources to a data plane, receiving feedback of a plurality of virtual IP addresses from the data plane, and configuring each resource using a virtual IP address corresponding to a cluster access request.

Description

Technical Field

[0001] Cross - reference to Related Applications This disclosure is based on and claims priority to CN Patent Application No. 202210847314.X, filed on July 19, 2022, the disclosure of which is incorporated herein by reference in its entirety.

[0002] This disclosure relates to the field of computer technology, particularly to multi - cluster access methods and systems.

Background Art

[0003] Ingress provides a set of routing rules for requests entering the cluster. As a cluster gateway resolution for k8s (Kubernetes), Ingress enables calls between services within the cluster but cannot provide multi - cluster access.

Summary of the Invention

Means for Solving the Problems

[0004] According to one aspect of the present disclosure, there is provided a multi - cluster access method including steps of synchronizing resources between a primary cluster and sub - clusters, sending related information of the synchronized resources to a data plane, receiving feedback of a plurality of virtual IP addresses from the data plane, and configuring each resource using a virtual IP address corresponding to a cluster access request.

[0005] In some embodiments, resource synchronization includes steps of a cluster manager in the primary cluster interacting with cluster managers in each sub - cluster, obtaining resources of each pod in each sub - cluster, and synchronizing resources of each pod in the primary cluster to each sub - cluster.

[0006] In some embodiments, resource synchronization further includes storing the resources of each pod in a database and storing snapshot information corresponding to the resources of each pod in a Custom Resource Definition (CRD) file.

[0007] In some embodiments, the step of configuring a virtual IP address includes binding snapshot information corresponding to the resources of each pod to the virtual IP address.

[0008] In some embodiments, one or more of query, creation, update, and deletion operations are performed on the CRD file based on a user instruction.

[0009] In some embodiments, the method further includes binding the virtual IP address of each pod to a corresponding domain name.

[0010] In some embodiments, the method includes sending a load balancing rule to a data plane, where the virtual IP address is generated based on the load balancing rule and the associated information of the resources.

[0011] In some embodiments, the step of obtaining the resources of each pod within each sub-cluster includes obtaining the kubeconfig file of each sub-cluster in a ConfigMap manner, and the clusterinfo crd in the kubeconfig file is configured to store the configuration information of the corresponding cluster.

[0012] According to one aspect of the present disclosure, there is provided a multi-cluster access system comprising a cluster manager configured to synchronize resources between a primary cluster and a sub-cluster, transmit related information of the synchronized resources to a data plane, receive feedback of a plurality of virtual IP addresses from the data plane, and configure each resource using a virtual IP address corresponding to a cluster access request, and a load balancer controller.

[0013] In some embodiments, the cluster manager is configured to interact with cluster managers within each sub-cluster, obtain resources of each pod within each sub-cluster, and synchronize the resources of each pod within the primary cluster to each sub-cluster.

[0014] In some embodiments, the multi-cluster access system further comprises a custom resource definition (CRD) file configured to store snapshot information corresponding to the resources of each pod.

[0015] In some embodiments, the load balancer controller is configured to bind snapshot information of the resources of each pod to a virtual IP address.

[0016] In some embodiments, based on a user instruction, one or more of query, creation, update, and deletion operations are performed on the CRD file.

[0017] In some embodiments, the multi-cluster access system comprises a global domain name system configured to bind the virtual IP address of each pod to a corresponding domain name.

[0018] In some embodiments, the load balancer controller is further configured to send the load balancing rules to the data plane, and the virtual IP address is generated based on the load balancing rules and the associated information of the resources.

[0019] In some embodiments, the cluster manager is configured to obtain the kubeconfig file of each sub-cluster in the form of a ConfigMap, and the clusterinfo crd in the kubeconfig file is configured to store the configuration information of the corresponding cluster.

[0020] According to one aspect of the present disclosure, there is provided a multi-cluster access system including a memory and a processor coupled to the memory, the processor being configured to execute the multi-cluster access method described above based on instructions stored in the memory.

[0021] According to another aspect of the present disclosure, there is also provided a non-transitory computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the multi-cluster access method described above.

[0022] According to still another aspect of the present disclosure, there is further provided a computer program including instructions that, when executed by a processor, cause the processor to perform the multi-cluster access method described above.

[0023] Other features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings.

[0024] The accompanying drawings, which are incorporated herein and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0025] The present disclosure will be more clearly understood by reading the following detailed description with reference to the accompanying drawings.

Brief Description of the Drawings

[0026]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Modes for Carrying Out the Invention

[0027] Next, various embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the relative arrangements, numerical expressions, and values of the components and steps described in these examples do not limit the scope of the present invention.

[0028] At the same time, for the sake of ease of explanation, it should be understood that the dimensions of the various components shown in the drawings are not drawn to actual scale.

[0029] The following description of at least one exemplary embodiment is, in fact, merely exemplary and is not intended to limit the present invention, its application, or its use.

[0030] Techniques, methods, and devices known to those skilled in the relevant art may not be described in detail, but where appropriate, these techniques, methods, and devices should be considered as part of this specification.

[0031] In all of the examples shown and described in this specification, any particular value should be construed as illustrative only and not as a limitation. Thus, other examples of exemplary embodiments may have different values.

[0032] Note that like reference numerals and letters in the accompanying drawings indicate like symbols, and thus, once an item is defined in the drawings, no further explanation in the accompanying drawings is required.

[0033] To clearly understand the objectives, technical solutions, and advantages of the present disclosure, the present disclosure will be further described in detail below in conjunction with the accompanying drawings and embodiments.

[0034] Note that the collection, use, storage, sharing, and transfer of users' personal information related to the technical solutions of the present disclosure need to comply with relevant laws and regulations and require notification to users and obtaining users' consent or permission. When applicable, users' personal information is subject to de-identification, anonymization, and / or encryption processes.

[0035] FIG. 1 is a flowchart of a multi-cluster access method according to some embodiments of the present disclosure and is executed by a control plane.

[0036] In step 110, the primary cluster synchronizes resources with the sub-clusters.

[0037] In some embodiments, in order to achieve resource synchronization between the primary cluster and the sub-clusters, a cluster manager within the primary cluster can be used, and the primary cluster can obtain the resource information of each pod within each sub-cluster.

[0038] In some embodiments, the resources of each pod are stored in a database, and the snapshot information corresponding to the resources of each pod is stored in a Custom Resource Definition (CRD) file.

[0039] In step 120, the related information of the synchronized resources is sent to the data plane.

[0040] In some embodiments, the load balancer controller monitors the snapshot information of each pod in the CRD file and synchronizes the snapshot information of the pod to the data plane.

[0041] In some embodiments, the load balancer controller synchronizes the load balancing rule information stored in the CRD file to the data plane.

[0042] In step 130, feedback of a plurality of virtual IP addresses is received from the data plane.

[0043] In some embodiments, the data plane generates a plurality of virtual IP addresses based on the load balancing rules and the snapshot information of each pod, and sends the virtual IP addresses to the load balancer controller.

[0044] In step 140, each resource is configured using the virtual IP address corresponding to the cluster access request.

[0045] In some embodiments, after the user sends an access request to the data plane, the data plane can send the access request to the pod in the corresponding cluster based on the virtual IP address.

[0046] In the above embodiment, the control plane synchronizes the resources of the primary cluster and the sub-cluster, and sends the related information of the synchronized resources to the data plane. The data plane generates a plurality of virtual IP addresses. After configuring each resource using the corresponding virtual IP address, the data plane can transfer the access request to the corresponding cluster, thereby solving the problem of multi-cluster access.

[0047] Figure 2 is a flowchart of a multi-cluster access method according to another embodiment of the present disclosure.

[0048] In step 210, the cluster manager synchronizes resources between the primary cluster and the sub-cluster.

[0049] In some embodiments, in order to achieve global load balancing, the capabilities of the pods in multiple clusters are collected. As a core component, the cluster manager can solve the problem of pod collection across multiple clusters. For example, the cluster manager in the primary cluster interacts with the cluster managers in each sub-cluster, obtains the resources of each pod in each sub-cluster, and synchronizes the resources of each pod in the primary cluster to each sub-cluster. In this way, the sub-cluster can also obtain the pod resources of other clusters.

[0050] In some embodiments, the cluster manager obtains the kubeconfig file from each sub-cluster in the ConfigMap manner, and the clusterinfo crd in the kubeconfig file is configured to store the configuration information of the corresponding cluster.

[0051] In some embodiments, the CRD controller located within the primary cluster is configured to generate a cluster manager pod synchronized based on cluster information and synchronize resources with the sub-cluster. The cluster manager acquires kubeconfig configuration information in a mounting ConfigMap manner and implements a master-slave backup mode for multiple pods via a k8s lock.

[0052] In some embodiments, the ConfigMap can realize the configuration management of the applications within the container, the kubeconfig file is an authentication file of the k8s API server (application programming interface server) that includes Cluster, User, namespace, and Authentication Mechanism information, and spec.config stores the configuration files of the primary cluster and the sub-cluster.

[0053] In step 220, the cluster manager stores the resources of each pod in a database and stores the snapshot information corresponding to the resources of each pod in a CRD file.

[0054] When a single pod exchanges data with multiple clusters, in the case of a large amount of data, for example, when there is a large amount of cluster data, the performance of etcd (distributed key-value storage system) that supports big data storage also deteriorates. In this embodiment, the resources of the pod are stored in a database, and the snapshot information corresponding to the resources of each pod is stored in a CRD file. By querying the CRD, information about each pod can be found, and the storage pressure on the etcd of the primary cluster can be reduced.

[0055] In some embodiments, the CRD file also stores load balancing rule information.

[0056] In some embodiments, the four-layer load balancing CRD file in the primary cluster, i.e., the L4CRD file, and the seven-layer load balancing CRD file, i.e., the L7CRD file, are used to record the relationship between the resources and load balancing within the cluster. The federalEndpoint CRD file in the primary cluster is used to record the pods within the sub-cluster.

[0057] In some embodiments, when processing the L4CRD file and the L7CRD file, the cluster manager creates a service within the sub-cluster and binds the Endpoint information to the federalEnjoin.

[0058] In some embodiments, the CRD file can be queried, created, updated, or deleted based on user instructions, thereby realizing the query, creation, update, and deletion of multi-cluster resources.

[0059] In step 230, the load balancer controller sends the load balancing rule information stored in the CRD file, as well as the snapshot information corresponding to the resources of each pod, to the data plane.

[0060] In step 240, the data plane generates a plurality of virtual IP addresses and feeds them back to the load balancer controller.

[0061] In some embodiments, the data plane generates a plurality of virtual IP addresses based on the load balancing rule information and the snapshot information corresponding to each pod, and feeds the virtual IP addresses back to the load balancer controller.

[0062] In step 250, the load balancer controller binds the snapshot information corresponding to the resources of each pod to the virtual IP address.

[0063] In step 260, after receiving an access request, the data plane transfers the access request to the pod in the corresponding cluster based on the IP address.

[0064] In some embodiments, the load balancer controller obtains the backend IP address through the federalEndpoint of the primary cluster, obtains the load balancing rules from the L4CRD and L7CRD, and binds the virtual IP address obtained from the data plane to the L4CRD and L7CRD, thereby enabling the distinction between public IP and private IP.

[0065] In some embodiments, Kubernetes provides users with an excellent cloud service experience as a Docker (container) scheduling solution, and the load balancer controller, as an important component of Kubernetes, provides users with a proxy means for accessing pods. However, changing the IP address of Docker may cause inconvenience in business use. In this embodiment, since the data plane feeds back the virtual IP address and the load balancer controller stores the correspondence between the pod and the virtual IP address in the CRD file, the access to the service can be simplified.

[0066] In some embodiments, GDNS (Global Domain Name System) is used to bind the virtual IP address of each pod to the corresponding domain name.

[0067] For example, GDNS searches for the virtual IP address of each pod from the CRD file, binds it to the corresponding domain name, and enables the data plane to transfer the access request to the pod in the corresponding cluster based on the domain name.

[0068] In the related art, with external load balancing, the Ingress of multiple clusters can be mounted, but two load balancings are required before reaching the service container, and traffic cannot be evenly distributed. In this embodiment, an access request can directly reach a pod within the corresponding cluster.

[0069] In the above embodiment, the primary cluster acquires information about each pod within the sub-cluster and sends the information about each pod to the data plane through the load balancer controller. The data plane generates a virtual IP address, and the load balancer controller configures the virtual IP address for each pod, thereby enabling access to the pods within multiple clusters. This process does not violate the sub-cluster and provides a consistent access method to the management side by hiding the underlying differences. Regardless of which management side the access request comes from, the present disclosure provides a smooth interface with the sub-cluster to achieve load balancing.

[0070] FIG. 3 is a schematic diagram of the structure of a multi-cluster access system according to some embodiments of the present disclosure. This system includes a cluster manager 310 and a load balancer controller 320 arranged on the primary cluster side.

[0071] The cluster manager 310 is configured to synchronize resources between the primary cluster and the sub-clusters.

[0072] In some embodiments, the cluster manager 310 located within the primary cluster is configured to interact with the cluster managers within each sub-cluster, acquire the resources of each container instance pod within each sub-cluster, and synchronize the resources of each pod within the primary cluster to each sub-cluster.

[0073] In some embodiments, the cluster manager 310 is configured to obtain the kubeconfig file of each sub-cluster in the form of a ConfigMap, and the clusterinfo crd in the kubeconfig file is configured to store the configuration information of the corresponding cluster.

[0074] In some embodiments, as shown in FIG. 4, the system further includes a CRD file 410 configured to store snapshot information corresponding to the resources of each pod. For example, the cluster manager 310 stores the resources of each pod in a database and stores the snapshot information corresponding to the resources of each pod in the CRD file 410.

[0075] In some embodiments, the CRD file 410 also stores load balancing rule information.

[0076] In some embodiments, based on a user instruction, query, creation, update, and deletion operations are performed on the CRD file 410. The CRD file 410 is an L4CRD file or an L7CRD file.

[0077] The load balancer controller 320 is configured to send the related information of the synchronized resources to the data plane, receive a plurality of virtual IP addresses as feedback from the data plane, and configure each resource using the virtual IP address corresponding to the cluster access request.

[0078] In some embodiments, the load balancer controller 320 sends the load balancing rule information stored in the CRD file and the snapshot information corresponding to the resources of each pod to the data plane. The data plane generates a plurality of virtual IP addresses based on the load balancing rule information and the resource-related information, and feeds back the virtual IP addresses to the load balancer controller 320.

[0079] In some embodiments, the load balancer controller 320 is configured to bind the snapshot information of the resources of each pod to a virtual IP address. After receiving an access request, the data plane transfers the access request to the pod within the corresponding cluster based on the IP address.

[0080] In other embodiments of the present disclosure, the system further includes a global domain name system 420 configured to bind the virtual IP address of each pod to a corresponding domain name. The data plane can transfer the access request to the pod within the corresponding cluster based on the domain name.

[0081] In the above embodiments, the control plane realizes resource synchronization between the primary cluster and the sub-cluster and sends the related information of the synchronized resources to the data plane. The data plane generates a plurality of virtual IP addresses. After configuring each resource using the corresponding virtual IP address, the data plane can transfer the access request to the corresponding cluster, thereby realizing multi-cluster access. Further, the present disclosure provides a unified access method to the management side by hiding the underlying differences, enabling a smooth interface with the sub-cluster to achieve load balancing regardless of which management side the access request comes from.

[0082] In some specific embodiments, as shown in FIG. 5, the CRD controller generates a cluster manager pod synchronized based on cluster information. The administrator can create, update, and delete the cluster manager. When processing the L4CRD file or the L7CRD file, the cluster manager creates services within the sub-cluster and binds the Endpoint information to federalEnjoin. As the core of the cluster, the API server is responsible for communication between various functional modules within the cluster. Each functional module within the cluster stores information in etcd through the API server. CCM (Cloud Provider Manager) is a single cluster load balancer that implements the load balancing function.

[0083] In the Kubernetes promotion process, multiple deployment methods for disaster recovery between data centers / regions, such as gray release, canary release, A / B testing, etc., are supported. To address the differences in the multi-cluster load balancing solution in the business scenario and the differences in the domain name server, the present disclosure provides a unified access method for the management side by hiding the underlying differences and can interface smoothly with the sub-cluster for load balancing.

[0084] FIG. 6 is a structural diagram of a multi-cluster access system according to a further embodiment of the present disclosure. The system 600 includes a memory 610 and a processor 620. Here, the memory 610 may be a magnetic disk, a flash memory, or any other non-volatile storage medium. The memory 610 is configured to store the instructions of the corresponding embodiments described above. The processor 620 is coupled to the memory 610 and may be implemented as one or more integrated circuits such as a microprocessor or a microcontroller. The processor 620 is configured to execute the instructions stored in the memory.

[0085] In some embodiments, processor 620 is coupled to memory 610 via bus 630. System 600 may be further connected to external storage device 650 through storage interface 640 to access external data, and may be further connected to a network or another computer system (not shown) through network interface 660, which will not be described in detail herein.

[0086] In the above embodiment, multi-cluster access can be realized by storing data instructions in the memory and processing the above instructions using a processor.

[0087] In other embodiments, a computer-readable storage medium storing computer program instructions for implementing the steps of the above embodiments when executed by a processor is provided. Those skilled in the art should understand that the embodiments of the present disclosure can be provided as a method, an apparatus, or a computer program product. Therefore, the embodiments of the present disclosure can take the form of completely hardware embodiments, completely software embodiments, or embodiments including both hardware elements and software elements. Furthermore, the present disclosure can take the form of a computer program product incorporated on one or more non-transitory storage media usable by a computer (including, but not limited to, disk storage, CD-ROM, optical storage devices, etc.) incorporating program code usable by a computer.

[0088] This disclosure will be described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowcharts and / or block diagrams, and combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. The computer program instructions can be provided to a processor of a computer or other programmable data processing apparatus, such that the instructions executed by the processor of the computer or other programmable data processing apparatus generate means for implementing the functions specified in one or more flows of the flowchart and / or one or more blocks of the block diagram, to generate a machine, which can be a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus.

[0089] The computer program instructions can also be stored in a computer-readable storage device that can direct a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable storage device produce a manufacture including instruction means for implementing the functions specified in one or more flows of the flowchart and / or one or more blocks of the block diagram.

[0090] These computer program instructions can also be loaded onto a computer or other programmable device to cause a series of operational steps to be performed on the computer or other programmable device to generate a computer-implemented process, such that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more flows of the flowchart and / or one or more blocks of the block diagram.

[0091] According to some embodiments of the present disclosure, when executed by a processor, there is further provided a computer program comprising instructions for causing the processor to execute the multi-cluster access method described above.

[0092] So far, the present disclosure has been described in detail. To avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can understand how to implement the technical solutions disclosed in this specification.

[0093] Although some specific embodiments of the present disclosure have been described in detail by way of example, those skilled in the art should understand that the above examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. It should be understood that by those skilled in the art, the above embodiments can be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the following claims.

Description of Reference Numerals

[0094] 310 Cluster Manager 320 Load Balancer Controller 410 CRD File 420 Global Domain Name System 600 System 610 Memory 620 Processor 630 Bus 640 Storage Interface 650 External Storage Device 660 Network Interface

Claims

1. synchronizing resources between a primary cluster and a sub-cluster; sending related information of the synchronized resources to a data plane; receiving feedback of a plurality of virtual IP addresses from the data plane; configuring each resource using a virtual IP address corresponding to a cluster access request A multi-cluster access method comprising the steps of:

2. The step of synchronizing resources is interacting, by a cluster manager in the primary cluster, with cluster managers in each sub-cluster; obtaining the resources of each pod in each sub-cluster; and synchronizing the resources of each pod in the primary cluster to each sub-cluster. The multi-cluster access method according to claim 1.

3. The step of synchronizing resources is further comprising storing the resources of each pod in a database and storing snapshot information corresponding to the resources of each pod in a custom resource definition (CRD) file. The multi-cluster access method according to claim 2.

4. The step of configuring the virtual IP address is comprising binding the snapshot information corresponding to the resources of each pod to the virtual IP address. The multi-cluster access method according to claim 3.

5. Based on a user instruction, one or more of querying, creating, updating, and deleting operations are performed on the CRD file. The multi-cluster access method according to claim 3 or 4.

6. The multi-cluster access method according to any one of claims 2 to 4, further comprising binding the virtual IP address of each pod to a corresponding domain name.

7. The step of sending a load balancing rule to the data plane, wherein the virtual IP address is generated based on the load balancing rule and related information of resources. The multi-cluster access method according to any one of claims 1 to 4.

8. The step of obtaining the resources of each pod in each sub-cluster is The method for multi-cluster access according to any one of claims 2 to 4, comprising the step of obtaining the kubeconfig file of each sub-cluster in the form of a ConfigMap, wherein the clusterinfo crd in the kubeconfig file is configured to store the configuration information of the corresponding cluster.

9. A cluster manager configured to synchronize resources between a primary cluster and sub-clusters, a load balancer controller configured to send relevant information of the synchronized resources to a data plane, receive feedback of a plurality of virtual IP addresses from the data plane, and configure each resource using the virtual IP address corresponding to a cluster access request. A multi-cluster access system comprising the above.

10. The multi-cluster access system according to claim 9, wherein the cluster manager is configured to interact with the cluster managers in each sub-cluster, obtain the resources of each pod in each sub-cluster, and synchronize the resources of each pod in the primary cluster to each sub-cluster.

11. The multi-cluster access system according to claim 10, further comprising a custom resource definition (CRD) file configured to store snapshot information corresponding to the resources of each pod.

12. The multi-cluster access system according to claim 11, wherein the load balancer controller is configured to bind the snapshot information of the resources of each pod to a virtual IP address.

13. The multi-cluster access system according to claim 11 or 12, wherein one or more of querying, creating, updating, and deleting operations are performed on the CRD file based on a user instruction.

14. The multi-cluster access system according to any one of claims 10 to 12, further comprising a global domain name system configured to bind the virtual IP address of each pod to a corresponding domain name.

15. The load balancer controller is further configured to send a load balancing rule to the data plane, and the virtual IP address is generated based on the load balancing rule and the resource association information. The multi-cluster access system according to any one of claims 9 to 12.

16. The cluster manager is configured to obtain the kubeconfig file of each sub-cluster in the form of a ConfigMap, and the clusterinfo crd in the kubeconfig file is configured to store the configuration information of the corresponding cluster. The multi-cluster access system according to any one of claims 10 to 12.

17. A memory, A processor coupled to the memory Comprising, the processor is configured to execute the multi-cluster access method according to any one of claims 1 to 8 based on instructions stored in the memory. A multi-cluster access system.

18. A non-transitory computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the multi-cluster access method according to any one of claims 1 to 8.

19. A computer program comprising instructions that, when executed by a processor, cause the processor to perform the multi-cluster access method according to any one of claims 1 to 8.

Images