Cross-network multi-cluster system, its access method, and cloud computing device

Abstract

The present application provides a cross-network multi-cluster system, an access method thereof, and a cloud computing device. The method for accessing the cross-network multi-cluster system is used for a multi-cluster system including a central cluster and a working cluster. The central cluster includes a proxy server, and the working cluster includes proxy clients. The method includes: the proxy client sends a control plane connectivity tunnel request to the central cluster, so as to establish a long connection control plane tunnel between the proxy client and the proxy server; the proxy client establishes a long connection reverse proxy tunnel with the proxy server through the long connection control plane tunnel; the proxy client receives and processes an API request that traverses the long connection reverse proxy tunnel and is forwarded by the proxy server. According to the technical solution of the embodiment, transparent and secure cross-network API calls can be realized.

Description

Technical Field

[0001] This application relates to the field of cloud computing technology. Specifically, it relates to a cross-network multi-cluster system, an access method thereof, a cloud computing device, and a computer-readable medium. Background Art

[0002] Traditional application deployment methods cannot define resource boundaries for applications in physical servers, often resulting in resource allocation problems. Even when running each application on different physical servers, it is difficult to scale due to insufficient resource utilization, and the cost of maintaining many physical servers is high.

[0003] Virtualization technology allows multiple virtual machines to run on the CPU of a single physical server. Virtualization allows applications to be isolated between virtual machines and provides a certain degree of security. Virtualization technology can better utilize the resources on physical servers, and can achieve better scalability and reduce hardware costs because applications can be easily added or updated.

[0004] With the rapid development of cloud computing, containerized deployment of applications has received more and more attention and development. Containers are similar to virtual machines, but they have relaxed isolation attributes and can share resources such as the operating system between applications, so they are considered lightweight. Containers are similar to virtual machines and can have their own file systems, CPUs, memories, process spaces, etc. Since they are separated from the infrastructure, they can be ported across clouds and operating systems.

[0005] In addition, with the expansion of cloud computing scale and the improvement of security requirements, cross-region multi-cloud clusters or hybrid cloud clusters have become the infrastructure of data centers. In such a multi-cluster scenario, it is often necessary to call the APIs of different clusters. However, there is currently a lack of a solution for cross-network access to different clusters that does not depend on network infrastructure configuration. Summary of the Invention

[0006] This application aims to provide a cross-network multi-cluster system, an access method thereof, and a cloud computing device, which can achieve transparent and secure cross-network API calls through a two-layer nested long connection tunnel.

[0007] The user characteristics and advantages of this application will become obvious through the following detailed description, or will be learned partially through the practice of this application.

[0008] According to one aspect of the present application, a method for accessing a cross-network multi-cluster system for a K8S multi-cluster system is provided. The K8S multi-cluster system includes a central cluster and a working cluster. Cross-network access between the central cluster and the working cluster is achieved through a proxy mechanism. The central cluster includes a proxy server, and the working cluster includes a proxy client. The method includes: the proxy client initiating a control-plane connectivity tunnel request to the central cluster, thereby establishing a long-connection control-plane tunnel between the proxy client and the proxy server; the proxy client establishing a long-connection reverse proxy tunnel with the proxy server through the long-connection control-plane tunnel; the proxy client receiving and processing API requests forwarded by the proxy server through the long-connection reverse proxy tunnel.

[0009] According to one aspect of the present application, a cross-network multi-cluster system based on a K8S cluster is provided. The cross-network multi-cluster system includes a central cluster and a working cluster. Cross-network access between the central cluster and the working cluster is achieved through a proxy mechanism. The central cluster includes a proxy server, and the working cluster includes a proxy client. The proxy server and the proxy client are configured such that: the proxy client initiates a control-plane connectivity tunnel request to the central cluster, thereby establishing a long-connection control-plane tunnel between the proxy client and the proxy server; the proxy client establishes a long-connection reverse proxy tunnel with the proxy server through the long-connection control-plane tunnel; the proxy server forwards API requests to the proxy client through the long-connection reverse proxy tunnel; the proxy client receives and processes API requests passing through the long-connection reverse proxy tunnel.

[0010] According to another aspect of the present application, a cloud computing device is provided, including: a processor; a memory having a computer program stored thereon; when the processor executes the computer program, the foregoing method is implemented.

[0011] According to another aspect of the present application, a computer-readable medium is provided, having a computer program stored thereon, and when the program is executed by a processor, the foregoing method is implemented.

[0012] According to another aspect of the present application, a computer program product is provided, including a computer program or instruction, and when the computer program or instruction is executed by a processor, the foregoing method is implemented.

[0013] According to an exemplary embodiment, the long - connection reverse proxy tunnel operates on the long - connection control plane tunnel, and the reverse proxy tunnel includes two nested long - connections. The proxy server can forward API requests to the proxy client through the long - connection reverse proxy tunnel, and the proxy client can receive and process the API requests passing through the long - connection reverse proxy tunnel. Thus, users can securely access different working clusters through the network control plane of the central cluster without caring about the topology of the underlying network infrastructure, realizing transparent and secure invocation of the K8S API.

[0014] It should be understood that the above general description and the following detailed description are only exemplary and do not limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] By referring to the accompanying drawings and describing its exemplary embodiments in detail, the above and other objectives, features, and advantages of the present application will become more apparent.

[0016] Figure 1 A schematic diagram showing an application scenario of the technical solution of the present application.

[0017] Figure 2 A schematic architecture diagram showing a cross - network multi - cluster system according to an exemplary embodiment of the present application.

[0018] Figure 3 A flowchart showing a method for accessing a cross - network multi - cluster system according to an embodiment of the present application.

[0019] Figure 4 A schematic diagram showing the process of establishing a long - connection control plane tunnel between a proxy client and a proxy server.

[0020] Figure 5 A timing diagram showing an API call initiated for a cross - network multi - cluster system according to an exemplary embodiment.

[0021] Figure 6 A block diagram showing a cloud computing device according to an exemplary embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in various forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. Like reference numerals in the figures denote the same or similar parts, and thus their repeated description will be omitted.

[0023] In addition, the described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present application. However, those skilled in the art will realize that the technical solutions of the present application may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be employed. In other cases, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present application.

[0024] The flowcharts shown in the accompanying drawings are merely illustrative and not necessarily include all the content and operations / steps, nor are they necessarily executed in the described order. For example, some operations / steps may be decomposed, while some operations / steps may be combined or partially combined, so the actual execution order may change according to the actual situation.

[0025] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or devices.

[0026] Referring to "embodiments" herein means that the specific features, structures, or characteristics described in connection with the embodiments may be included in at least one embodiment of the present application. The phrase appears in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art can understand that the embodiments described herein may be combined with other embodiments.

[0027] Before describing the embodiments of the present application, some terms related to the embodiments of the present application are explained.

[0028] K8S: Kubernetes, a cloud-native resource orchestration and scheduling engine used to manage containerized workloads and application services, making it simple and efficient to deploy containerized applications. Each container is isolated from each other, each container has its own file system, and the processes between containers do not affect each other, and computing resources can be distinguished. Compared with virtual machines, containers can be deployed quickly. Since containers are decoupled from the underlying infrastructure and the machine file system, they can be migrated between different clouds and different versions of operating systems.

[0029] API Request: K8S adopts a hub-and-spoke API model. All API calls made from the cluster terminate at the kube-apiserver, and other control plane components are not designed to expose remote services. The kube-apiserver is configured to listen for remote connection requests on a secure HTTPS port (443). The kube-apiserver is the declarative API server of K8S and provides a bridge for other components to interact.

[0030] OCM: Open Cluster Management, a hybrid cloud multi-cluster management platform technology for K8S. Through the simple and open API of OCM, functions such as cluster registration, workload distribution, and dynamic resource configuration routing can be achieved, enabling users to complete development work in multi-cluster and hybrid environments as if they were on a single K8S cluster platform.

[0031] port-forward: A mechanism provided natively by K8S to access the internal network of the K8S cluster.

[0032] Reverse proxy tunnel: A technology that initiates a tunnel from the network being proxied to other networks, enabling proxy communication across network restrictions such as firewalls.

[0033] Konnectivity: The reverse tunnel protocol adopted in K8S.

[0034] Hub cluster: The K8S cluster that serves as the control center in a multi-cluster platform.

[0035] Worker cluster: The K8S cluster managed by the control center in a multi-cluster platform.

[0036] TLS: The Transport Layer Security protocol. The authentication process of TLS depends on the completion of the handshake.

[0037] Mutual TLS (mTLS): A security protection technology that requires mutual authentication between the server and the client in the TLS authentication framework. Two sets of certificates, namely the client certificate and the server certificate, are generated by the same root certification authority (root CA). When the client accesses the server using a secure protocol, the two parties will exchange certificates and conduct authentication, and communication can only be established after successful authentication.

[0038] Long connection: It means that multiple data packets can be continuously sent on a connection. During the connection maintenance period, if no data packets are sent, both parties need to send link detection packets. The operation steps of a long connection are: establish a connection - data transmission... (maintain the connection)... data transmission - close the connection.

[0039] Figure 1A schematic diagram showing an application scenario of the technical solution of the present application.

[0040] like Figure 1 As shown in the figure, with the rapid development of enterprise business, cross-regional multi-cloud clusters have become the basic architecture of data centers. In a cross-network multi-cloud cluster scenario, the entire system includes a central cluster and n working clusters. At least two working clusters can be deployed on different private clouds to ensure disaster recovery.

[0041] In some scenarios, Figure 1 The infrastructure shown can include dozens of K8S clusters, each with thousands of (virtual) servers. Applications and required components (middleware, database, security, load balancing, etc.) can be organized into logical units of virtual logical data centers at the architectural level and planned for deployment on physical infrastructure.

[0042] See also Figure 1 The central cluster can provide cluster registration, cluster lifecycle management, load scheduling, plug-in registration and management based on OCM or other management architectures. The working cluster can provide load scheduling, resource distribution, plug-in management, etc. based on OCM or other management architectures.

[0043] In many scenarios, it is necessary to call the APIs of other working clusters through the central cluster. In the process of API calling, the first thing to solve is the forward network connectivity problem. This is because the working cluster itself may be deployed in a different VPC (virtual private cloud) network, and there may be firewalls between the central cluster and the working cluster, denying direct access.

[0044] Traditional methods of network connectivity include manually opening / maintaining a security access whitelist (ACL: access control list) for the central cluster, or directly establishing routing rules at the infrastructure level to connect the network planes between clusters. However, these network connectivity solutions vary greatly for different infrastructure environments, and it is difficult to provide an easy-to-use automated connectivity solution without considering the basic network environments of different clusters.

[0045] Another solution for the API push link from the central cluster to the working clusters in a multi-cluster architecture is that the administrator of the working cluster installs an agent client in the working cluster. After that, the client will establish a reverse proxy tunnel of the SSH protocol (Secure Shell) to the central cluster. When the central cluster accesses each working cluster, it can pass through this SSH proxy tunnel to the network plane where the control plane of the working cluster is located, and then initiate an API call request. This solution requires opening the SSH service port, which poses a relatively large security risk. The SSH reverse proxy tunnel cannot restrict the target network that the central cluster can access. A client initiating a malicious network request from the network of the central cluster may use the tunnel to attack the network where the working cluster is located. In addition, there are also limitations in automated operation and maintenance deployment, and each SSH agent needs to be manually installed by the administrator of the working cluster.

[0046] Therefore, this application proposes a method for accessing a working cluster across networks to achieve API access from the central cluster to the working clusters, without the requirement that the central cluster network and the working cluster network be in the same network plane or other network infrastructure configurations.

[0047] It is easy to understand that Figure 1 the application scenarios and architectures shown are schematic to make it easier for readers to understand the technical solutions of this application. The technical solutions according to the embodiments of this application described below can be applied to other multi-cluster scenarios.

[0048] The technical solutions of this application and their advantages will be described in detail below with reference to the embodiments.

[0049] Figure 2 A schematic architecture diagram of a cross-network multi-cluster system according to an example embodiment of this application is shown.

[0050] See Figure 2 , the cross-network multi-cluster system according to the example embodiment is based on a K8S cluster. The cross-network multi-cluster system includes a central cluster 200 and working clusters 260. The working clusters 260 are managed clusters of the central cluster 200, and the number can be multiple. The nth (#n) working cluster is shown in the figure. According to the example embodiment, cross-network access between the central cluster 200 and the working clusters 260 can be achieved through a proxy mechanism. The central cluster 260 includes a proxy server 202, and the working cluster 260 includes a proxy client 262. As Figure 2 shown, the central cluster 260 may include multiple proxy servers 202, and each working cluster 260 may include multiple proxy clients 262.

[0051] According to an example embodiment, the proxy client 262 may initiate a control plane connectivity tunnel request to the central cluster 200, so that the central cluster 200 establishes a long connection tunnel between the proxy client 262 and the proxy server 202. For example, the long connection tunnel of the control plane may be established through the K8S Port-forward mechanism.

[0052] According to some embodiments, each proxy client 262 is configured with a deployed auxiliary service, which can pull the container instance name corresponding to the proxy server 202 in the central cluster 200 in real time. The proxy client 262 may initiate a control plane connectivity tunnel request including the container instance name to the central cluster 200, so as to initiate a long connection tunnel for container access. The long connection tunnel may be based on the Port-forward mechanism. Through this long connection tunnel, the four-layer network connectivity between the proxy client 262 of each worker cluster 260 and the instance of the proxy server 202 in the central cluster 200 can be established, thus preparing for the establishment of a long connection reverse proxy tunnel in the next step.

[0053] According to an example embodiment, after the control plane connectivity tunnel, the proxy client 262 establishes a long connection reverse proxy tunnel with the proxy server 202 through the control plane connectivity tunnel. Through this long connection reverse proxy tunnel, the internal network of each worker cluster 260 can be directly accessed from the control plane of the central cluster 200.

[0054] According to an example embodiment, the proxy client 262 may initiate a request to register an instance of the proxy client to the proxy server 202, and the request may include the name of the worker cluster 260 where the proxy client 262 is located. The proxy server 202 approves the instance registration request of the proxy client 262 and returns the metadata information of the proxy server 202 to the proxy client 262, such as the unique identifier of the proxy server 202 and configuration parameters, etc. Then, the proxy client 262 may establish a long connection reverse proxy tunnel to the proxy server 202. The long connection reverse proxy tunnel may be a native K8S Konnectivity tunnel.

[0055] According to an example embodiment, after the long connection reverse proxy tunnel is established, the proxy server 202 can forward API requests to the proxy client 262 through the long connection reverse proxy tunnel, and the proxy client 262 can receive and process the API requests passing through the long connection reverse proxy tunnel.

[0056] According to the technical solution of the exemplary embodiment, the long - connection reverse proxy tunnel operates on the long - connection control - plane tunnel, and this reverse proxy tunnel is essentially mainly composed of two nested long - connections. Users can securely access different working clusters through the network control - plane of the central cluster without caring about the topology of the underlying network infrastructure, realizing the transparent and secure invocation of the K8S API. The technical solution according to the embodiment of the present application does not require the central - cluster network and the working - cluster network to be in the same network plane or other network infrastructure configurations, enabling the multi - cluster control - plane to have the ability of basic network connectivity.

[0057] According to some embodiments, the long - connection reverse proxy tunnel is based on the K8S native Konnectivity tunnel, and can initiate client API calls through the client or native development toolkit provided by Konnectivity natively, further simplifying the problem of multi - cluster API calls. After the target domain - name address in the API call request is registered in the proxy server 202 by the proxy client 262, it will be dynamically routed to the corresponding working cluster 260, ensuring transparency and security. By using the default - registered logical name of the working cluster as the target domain name, a client call to the K8S API can be directly initiated to each working cluster 260.

[0058] According to the exemplary embodiment, there can be multiple instances of the proxy server 202 and the proxy client 262, each located in a different container, thus realizing disaster recovery. In this way, when a certain instance of the proxy client 262 or the proxy server 202 has a problem of crashing, it can be automatically recovered to achieve the purpose of "high availability".

[0059] See Figure 2 , the central cluster 200 may include a registration controller 204, and the working cluster 260 may include a registration proxy 264. Through the registration controller 204 and the registration proxy 264, the working cluster 260 is registered in the central cluster 200 in the OCM framework as a managed - cluster resource. The OCM platform technology provides native plug - in extensibility, which enables the systems deployed therein to be fully automated. During the registration process of the working cluster 260, the registration proxy 264 can establish communication with the https service access entry of the OCM.

[0060] According to the exemplary embodiment, the https service of the OCM is protected by mTLS technology, and there is no interface - security problem even if it is directly exposed to the public network by default. In addition, before the proxy clients in each working cluster 260 are started, the OCM will automatically issue mTLS client certificates for them to initiate long - connection tunnels. After the issuance, the OCM will also dynamically roll the certificates used by these proxy clients to prevent security problems such as certificate - copy leakage.

[0061] SeeFigure 2 Moreover, the central cluster 200 may further include a cluster proxy plugin manager 206. Based on the OCM and the native plugin framework, the cluster proxy plugin manager 206 is used to automatically install the proxy client 262 for the working cluster 260. In the OCM framework, through the automated plugin installation mechanism of the native plugin framework (Addon-Framework), when a new working cluster 260 is registered to the central cluster 200, the process of installing the proxy client 262 in the working cluster can be triggered dynamically. These proxy clients 262 will first establish a long connection control plane tunnel, and then establish a long connection reverse proxy tunnel thereon and complete the registration with the proxy server 202. When registering with the proxy server 202 of the central cluster 200, the proxy client 262 can provide the domain names it supports. In multi-cluster technology, these domain names may include the name of the working cluster during registration, such as the logical name of the working cluster.

[0062] In this way, according to the exemplary embodiment, the establishment process of the tunnel can also be fully automated by means of the cluster registration dynamic discovery ability of the OCM, so as to automatically complete the network connectivity, and the deployment of the reverse proxy tunnel can be completed without manual intervention, so that the client does not need to care about the topology of the underlying network infrastructure and can securely access different working clusters from the network plane of the central cluster.

[0063] Figure 3 The flowchart of the method for accessing a cross-network multi-cluster system according to an embodiment of the present application is shown.

[0064] Figure 3 The method shown can be used in a K8S multi-cluster system, which includes a central cluster and a working cluster. Cross-network access between the central cluster and the working cluster is realized through a proxy mechanism. The central cluster includes a proxy server, and the working cluster includes a proxy client.

[0065] See Figure 3 In S301, the proxy client sends a control plane connectivity tunnel request to the central cluster, thereby establishing a long connection control plane tunnel between the proxy client and the proxy server.

[0066] According to the exemplary embodiment, the control plane connectivity tunnel request includes the container instance name of the proxy server, and the long connection control plane tunnel is a container access long connection tunnel. For example, the container access long connection tunnel is a tunnel based on the K8S port-forward mechanism.

[0067] In S303, the proxy client establishes a long connection reverse proxy tunnel with the proxy server through the long connection control plane tunnel.

[0068] According to an exemplary embodiment, the proxy client may initiate a request to register a client instance with the proxy server, and the request may include the name of the working cluster where the proxy client is located. The proxy server approves the registration request of the proxy client instance and returns metadata information of the proxy server to the proxy client, including the unique identifier of the proxy server itself, configuration parameters, etc.

[0069] Based on the metadata information of the proxy server returned due to the proxy server's approval of the registration of the proxy client instance, the proxy client sends a request to the proxy server, and the request includes the identifier of the proxy client itself, etc., thereby establishing a long - connection reverse proxy tunnel. For example, the long - connection reverse proxy tunnel can be a native K8S Konnectivity tunnel. Thereafter, the proxy server can forward API requests to the proxy client through the long - connection reverse proxy tunnel.

[0070] In S305, the proxy client receives and processes the API request forwarded by the proxy server through the long - connection reverse proxy tunnel.

[0071] According to an exemplary embodiment, the proxy server receives an API request, which is based on the long - connection reverse proxy tunnel and includes the name of the working cluster. Then, the proxy server sends a channel request to the proxy client. The proxy client can receive the channel request sent by the proxy server and return the ID of the available channel.

[0072] The proxy server determines the ID of the available channel according to the response and sends the API request to the proxy client through the available channel.

[0073] The proxy client receives the API request sent by the proxy server through the available channel, thereby initiating an API request to the control plane of the working cluster and can return the request result according to the response.

[0074] Figure 4 Schematic diagram showing the process of establishing a long - connection control plane tunnel between the proxy client and the proxy server.

[0075] See Figure 4 , in S401, the proxy client initiates a request to Kube - Apiserver.

[0076] This request can be a request based on the SPDY (speedy: a fast content transfer protocol) protocol or an http / 2 request. The request may include the container instance name corresponding to the target proxy server. As discussed before, each proxy client is configured with an auxiliary service, and this auxiliary service can pull the container instance name corresponding to the proxy server in the central cluster in real - time.

[0077] In S403, Kube-Apiserver locates the container and further locates the node where the container is located.

[0078] In S405, Kube-Apiserver sends a request to the node where the container is located. Similarly, this request can be a request based on the SPDY protocol or an http / 2 request.

[0079] In S407, the node then sends a request to the container runtime where the node is located.

[0080] In S409, the container runtime where the node is located then sends a request to the corresponding container of the proxy server.

[0081] In S411, the proxy client can access the corresponding container of the proxy server at this time, so that a long connection control plane tunnel can be established between the proxy client and the proxy server.

[0082] Figure 5 A timing diagram showing an API call initiated for a cross-network multi-cluster system according to an exemplary embodiment.

[0083] See Figure 5 , at timing node 1, the proxy client can send a control plane connectivity tunnel request to the central cluster, so that the central cluster establishes a long connection tunnel between the proxy client and the proxy server. According to an exemplary embodiment, this request may include information such as the container instance name of the proxy server, and a long connection tunnel of the control plane can be established based on the K8S Port-forward mechanism. Each proxy client is configured with an auxiliary service, and this auxiliary service can pull the container instance name corresponding to the proxy server in the central cluster in real time.

[0084] At timing node 2.1, the proxy client sends a request to register a client instance to the proxy server, and this request includes the name of the working cluster where the proxy client is located, etc.

[0085] At timing node 2.2, the proxy server approves the registration of the proxy client instance and returns the metadata information of the proxy server, including the unique identifier of the proxy server itself and configuration parameters, etc.

[0086] At timing node 2.3, the proxy client establishes a long connection reverse proxy tunnel to the proxy server. During the tunnel establishment process, the proxy client can provide information such as its own identifier (ID) to the proxy server. The long connection reverse proxy tunnel can be a native K8S Konnectivity tunnel. Thereafter, the proxy server can forward API requests to the proxy client through the long connection reverse proxy tunnel.

[0087] At timing node 3.1, the user initiates an API request, which is assigned to a certain proxy server through the load balancer of the central cluster or other means. Information such as the name of the target working cluster for this request may be included in the API request. According to the example embodiment, the long connection reverse proxy tunnel is based on the K8S native Konnectivity tunnel, and the client can initiate a client API call through the client or native development kit provided by Konnectivity natively.

[0088] At timing node 3.2, the proxy server sends a channel request to the proxy client. The target domain name address in the API call request has been registered by the proxy client to the proxy server and is thus dynamically routed to the corresponding working cluster.

[0089] At timing node 3.3, the proxy client returns the ID of the available channel and prepares to process the API request data.

[0090] At timing node 3.4, the proxy server sends the API request data to the proxy client through the available channel. According to the example embodiment, the routing to the API server of the working cluster can automatically resolve the domain name through CNAME.

[0091] At timing node 3.5, the proxy client receives the API request sent by the proxy server through the available channel and initiates an API request to the control plane of the working cluster.

[0092] So far, the process of initiating the API call is completed, and the proxy client can return the request result according to the response of the control plane of the working cluster hereafter.

[0093] Figure 6 A block diagram of a cloud computing device according to an example embodiment of the present application is shown.

[0094] As Figure 6 shown, the cloud computing device 30 includes a processor 12 and a memory 14. The cloud computing device 30 may further include a bus 22, a network interface 16, and an I / O interface 18. The processor 12, the memory 14, the network interface 16, and the I / O interface 18 can communicate with each other through the bus 22.

[0095] The processor 12 may include one or more general-purpose CPUs (Central Processing Unit), microprocessors, or application-specific integrated circuits, etc., for executing relevant program instructions.

[0096] The memory 14 may include a machine system-readable medium in the form of volatile memory, such as random access memory (RAM), read-only memory (ROM), and / or cache memory. The memory 14 is used to store one or more programs containing instructions as well as data. The processor 12 can read the instructions stored in the memory 14 to execute the methods according to the embodiments of the present application described above. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile storage devices.

[0097] The cloud computing device 12 can also communicate with one or more networks through the network interface 16. The network interface 16 can be a wired network interface or a wireless network interface, or can also be a virtual network interface.

[0098] The cloud computing device 12 can also communicate with one or more external devices (such as audio input devices, audio output devices, cameras, keyboards, mice, displays, various sensors, etc.) through the input / output (I / O) interface 18.

[0099] The bus 22 may include an address bus, a data bus, a control bus, etc. The bus 22 provides a path for exchanging information between the components.

[0100] It should be noted that in the specific implementation process, the cloud computing device 30 may further include other components necessary for normal operation. In addition, those skilled in the art can understand that the above devices may also only include the components necessary to implement the solutions of the embodiments of this specification, and do not necessarily include all the components shown in the figure.

[0101] The present application also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the steps of the above method. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nano-systems (including molecular memory ICs), network storage devices, cloud storage devices, or any type of medium or device suitable for storing instructions and / or data.

[0102] The embodiments of the present application also provide a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps of any one of the methods described in the above method embodiments.

[0103] Those skilled in the art can clearly understand that the technical solution of this application can be implemented by means of software and / or hardware. The "units" and "modules" in this specification refer to software and / or hardware that can independently complete or cooperate with other components to complete specific functions, where the hardware can be, for example, a field programmable gate array, an integrated circuit, etc.

[0104] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0105] In the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0106] In the several embodiments provided by this application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the displayed or discussed coupling or direct coupling or communication connection between each other can be through some service interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical or other form.

[0107] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0108] In addition, the functional units in each embodiment of this application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0109] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application.

[0110] In the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0111] The embodiments of this application have been described and explained in detail above. It should be clearly understood that this application describes how to form and use specific examples, but this application is not limited to any details of these examples. On the contrary, based on the teachings of the content disclosed in this application, these principles can be applied to many other embodiments.

[0112] Through the description of the exemplary embodiments, it is easy for those skilled in the art to understand that the technical solution according to the embodiments of this application has at least one or more of the following advantages.

[0113] According to the exemplary embodiments, through two-layer nested long connection tunnels, transparent and secure cross-network API calls can be realized.

[0114] According to the exemplary embodiments, the long connection reverse proxy tunnel works on the long connection control plane tunnel. The proxy server can forward API requests to the proxy client through the long connection reverse proxy tunnel. The proxy client can receive and process the API requests passing through the long connection reverse proxy tunnel. Thus, users do not need to care about the topology of the underlying network infrastructure and can securely access different working clusters through the network control plane of the central cluster, realizing transparent and secure calls of the K8S API.

[0115] The technical solution according to the embodiments of this application does not require the central cluster network and the working cluster network to be in a network plane or other network infrastructure configurations, enabling the multi-cluster control plane to have the ability of basic network connectivity.

[0116] According to some embodiments, the long connection reverse proxy tunnel is based on the K8S native Konnectivity tunnel, and can initiate client API calls through the client or native development toolkit provided by Konnectivity natively, further simplifying the problem of multi-cluster API calls.

[0117] According to some embodiments, after the target domain name address in the API call request is registered in the proxy server by the proxy client, it will be dynamically routed to the corresponding working cluster to ensure transparency and security.

[0118] According to some embodiments, the tunnel establishment process can be fully automated with the help of the cluster registration dynamic discovery capability of OCM, thereby automatically completing the opening of network connectivity and completing the deployment of the reverse proxy tunnel without manual intervention.

[0119] The foregoing content can be better understood in accordance with the following terms:

[0120] Clause 1. A method for accessing a cross-network multi-cluster system, for a K8S multi-cluster system, wherein the K8S multi-cluster system includes a central cluster and a working cluster, wherein the central cluster and the working cluster achieve cross-network access through a proxy mechanism, wherein the central cluster includes a proxy server, and the working cluster includes a proxy client, and the method includes:

[0121] The proxy client initiates a control plane connectivity tunnel request to the central cluster, thereby establishing a long connection control plane tunnel between the proxy client and the proxy server;

[0122] The proxy client establishes a persistent reverse proxy tunnel with the proxy server through the persistent control plane tunnel;

[0123] The proxy client receives and processes the API request forwarded by the proxy server and traversing the long connection reverse proxy tunnel.

[0124] Clause 2. The method as described in Clause 1 is characterized in that the control plane connectivity tunnel request includes the container instance name of the proxy server, and the long connection control plane tunnel is a container access long connection tunnel.

[0125] Clause 3. The method as described in Clause 2 is characterized in that the container accesses the long connection tunnel based on the K8Sport-forward mechanism.

[0126] Clause 4. The method as described in Clause 1 is characterized in that the proxy client establishes a persistent reverse proxy tunnel with the proxy server through the persistent control plane tunnel, comprising:

[0127] The proxy client initiates a request to the proxy server to register a client instance, wherein the request includes the name of the working cluster where the proxy client is located;

[0128] Based on the metadata information of the proxy server returned by the proxy server in response to the proxy server approving the registration of the proxy client instance, the proxy client establishes a persistent reverse proxy tunnel to the proxy server.

[0129] Clause 5. The method as described in Clause 4 is characterized in that the long connection reverse proxy tunnel is a K8S native Konnectivity tunnel.

[0130] Clause 6. The method as described in Clause 4, characterized in that the proxy client receives and processes the API request forwarded by the proxy server and traverses the persistent reverse proxy tunnel, comprising:

[0131] The proxy client receives a channel request sent by the proxy server;

[0132] The proxy client returns the ID of the available channel;

[0133] The proxy client receives the API request sent by the proxy server through the available channel;

[0134] The proxy client initiates an API request to the control plane of the working cluster.

[0135] Clause 7. The method according to clause 6, characterized in that, for the central cluster, the method further comprises:

[0136] The proxy server approves the registration request of the proxy client instance and returns metadata information of the proxy server to the proxy client;

[0137] The proxy server forwards the API request to the proxy client through the long connection reverse proxy tunnel.

[0138] Clause 8. The method as described in Clause 7, characterized in that the proxy server forwards the API request to the proxy client through the persistent reverse proxy tunnel, comprising:

[0139] The proxy server receives an API request, where the API request is based on the persistent reverse proxy tunnel and includes the name of the working cluster;

[0140] The proxy server sends a channel request to the proxy client and determines the ID of an available channel according to the response;

[0141] The proxy server sends an API request to the proxy client through the available channel.

[0142] Clause 9. The method according to clause 1, further comprising:

[0143] The working cluster is registered in the central cluster in the OCM framework as a managed cluster resource.

[0144] Clause 10. The method according to clause 9, further comprising:

[0145] When the working cluster is registered with the central cluster, the central cluster dynamically triggers the installation of the proxy client in the working cluster through a native plug-in framework.

[0146] Clause 11. A cross-network multi-cluster system, the cross-network multi-cluster system is based on a K8S cluster, the cross-network multi-cluster system includes a central cluster and a working cluster, characterized in that cross-network access is achieved between the central cluster and the working cluster through a proxy mechanism, the central cluster includes a proxy server, the working cluster includes a proxy client, wherein the proxy server and the proxy client are configured as follows:

[0147] The proxy client initiates a control plane connectivity tunnel request to the central cluster, thereby establishing a long connection control plane tunnel between the proxy client and the proxy server;

[0148] The proxy client establishes a persistent reverse proxy tunnel with the proxy server through the persistent control plane tunnel;

[0149] The proxy server forwards the API request to the proxy client through the persistent reverse proxy tunnel;

[0150] The proxy client receives and processes API requests that traverse the persistent reverse proxy tunnel.

[0151] Clause 12. The cross-network multi-cluster system as described in Clause 11, characterized in that:

[0152] The long connection control plane tunnel is based on the K8S port-forward mechanism;

[0153] The long connection reverse proxy tunnel is the K8S native Konnectivity tunnel.

[0154] Clause 13. The cross-network multi-cluster system as described in Clause 11, characterized in that it also includes:

[0155] A cluster proxy plug-in manager, the cluster proxy plug-in manager is based on OCM and a native plug-in framework, and is used to automatically install the proxy client for a working cluster.

[0156] Clause 14. A cloud computing device, comprising:

[0157] processor;

[0158] A memory, on which a computer program is stored;

[0159] When the processor executes the computer program, the method described in any one of Clauses 1-10 is implemented.

[0160] The exemplary embodiments of the present application have been specifically shown and described above. It should be understood that the present application is not limited to the detailed structures, settings or implementation methods described herein; on the contrary, the present application is intended to cover various modifications and equivalent settings included within the spirit and scope of the appended clauses.

Claims

1. A method for accessing a cross-network multi-cluster system, for a K8S multi-cluster system, wherein the K8S multi-cluster system includes a central cluster and a working cluster. It is characterized in that The central cluster and the working cluster achieve cross-network access through a proxy mechanism, the central cluster includes a proxy server, the working cluster includes a proxy client, and the method includes: The proxy client initiates a control plane connectivity tunnel request to the central cluster, thereby establishing a long connection control plane tunnel between the proxy client and the proxy server; the control plane connectivity tunnel request includes the container instance name of the proxy server, the long connection control plane tunnel is a container access long connection tunnel, and the container access long connection tunnel is based on the K8S port-forward mechanism; The proxy client establishes a persistent reverse proxy tunnel with the proxy server through the persistent control plane tunnel; the persistent reverse proxy tunnel is a K8S native Konnectivity tunnel; The proxy client receives and processes the API request forwarded by the proxy server and traversing the long connection reverse proxy tunnel.

2. The method according to claim 1, It is characterized in that The proxy client establishes a persistent connection reverse proxy tunnel with the proxy server through the persistent connection control plane tunnel, including: The proxy client initiates a request to the proxy server to register a client instance, wherein the request includes the name of the working cluster where the proxy client is located; Based on the metadata information of the proxy server returned by the proxy server in response to the proxy server approving the registration of the proxy client instance, the proxy client establishes a persistent reverse proxy tunnel to the proxy server.

3. The method according to claim 2, It is characterized in that The proxy client receives and processes the API request forwarded by the proxy server and traversing the persistent reverse proxy tunnel, including: The proxy client receives a channel request sent by the proxy server; The proxy client returns the ID of the available channel; The proxy client receives the API request sent by the proxy server through the available channel; The proxy client initiates an API request to the control plane of the working cluster.

4. The method according to claim 3, It is characterized in that For the central cluster, the method further includes: The proxy server approves the registration request of the proxy client instance and returns metadata information of the proxy server to the proxy client; The proxy server forwards the API request to the proxy client through the long connection reverse proxy tunnel.

5. The method according to claim 4, It is characterized in that The proxy server forwards the API request to the proxy client through the persistent reverse proxy tunnel, including: The proxy server receives an API request, where the API request is based on the persistent reverse proxy tunnel and includes the name of the working cluster; The proxy server sends a channel request to the proxy client and determines the ID of an available channel according to the response; The proxy server sends an API request to the proxy client through the available channel.

6. The method according to claim 1, It is characterized in that The method further comprises: The working cluster is registered in the central cluster in the OCM framework as a managed cluster resource.

7. The method according to claim 6, It is characterized in that Also includes: When the working cluster is registered with the central cluster, the central cluster dynamically triggers the installation of the proxy client in the working cluster through a native plug-in framework.

8. A cross-network multi-cluster system, the cross-network multi-cluster system is based on a K8S cluster and includes a central cluster and a working cluster, It is characterized in that The central cluster and the working cluster achieve cross-network access through a proxy mechanism, the central cluster includes a proxy server, the working cluster includes a proxy client, wherein the proxy server and the proxy client are configured as follows: The proxy client initiates a control plane connectivity tunnel request to the central cluster, thereby establishing a long connection control plane tunnel between the proxy client and the proxy server; the control plane connectivity tunnel request includes the container instance name of the proxy server, the long connection control plane tunnel is a container access long connection tunnel, and the container access long connection tunnel is based on the K8S port-forward mechanism; The proxy client establishes a persistent reverse proxy tunnel with the proxy server through the persistent control plane tunnel; The long connection reverse proxy tunnel is the K8S native Konnectivity tunnel; The proxy server forwards the API request to the proxy client through the persistent reverse proxy tunnel; The proxy client receives and processes API requests that traverse the persistent reverse proxy tunnel.

9. The cross-network multi-cluster system according to claim 8, It is characterized in that Also includes: A cluster proxy plug-in manager, the cluster proxy plug-in manager is based on OCM and a native plug-in framework, and is used to automatically install the proxy client for a working cluster.

10. A cloud computing device, It is characterized in that include: processor; A memory having a computer program stored therein; When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images