Methods, devices, equipment and media for cross-network multi-cloud and multi-cluster application distribution

Abstract

This invention relates to the field of Kubernetes cloud platform technology, and discloses a method, apparatus, device, and medium for cross-network multi-cloud and multi-cluster application distribution. The method includes: creating a server application and k8s APIService resources in a central cluster; creating proxy applications in multiple access clusters and initiating remote procedure call requests to the API server through the proxy applications; when the API server receives the remote procedure call request from the proxy applications, creating a bidirectional long-connection tunnel between the server application and the proxy applications; creating an access cluster key in the central cluster for application distribution using continuous delivery tools, and distributing the application to multiple access clusters through the bidirectional long-connection tunnel using continuous delivery tools based on the access cluster key. This invention fully leverages the advanced features of ArgoCD multi-cluster distribution, reduces user costs, and lowers the risk of leakage of user access cluster configuration files.

Description

Technical Field

[0001] This invention relates to the field of Kubernetes cloud platform technology, specifically to a method, apparatus, device, and medium for distributing multi-cloud, multi-cluster applications across networks. Background Technology

[0002] With the widespread use of Kubernetes clusters (hereinafter referred to as k8s clusters), multi-cloud and multi-cluster has become a trend. Mainstream cloud vendors have launched distributed container cloud products, in which a cloud vendor provides a k8s cluster as a central cluster, which manages multiple k8s clusters in the cloud vendor's own cloud, other cloud vendors' cloud, and the user's local IDC (Internet Data Center) as access clusters. By deploying ArgoCD (Argo Continuous Delivery, CD) in the central cluster, applications are distributed and deployed to multiple access clusters and maintained uniformly.

[0003] ArgoCD is a continuous delivery tool based on GitOps principles. It uses GitOps to distribute applications across multiple Kubernetes clusters. Its architecture is as follows: Figure 1 As shown, the ArgoCD application itself is deployed in a Kubernetes cluster. It stores the kubeconfig (a configuration file used by the Kubernetes client to interact with the Kubernetes cluster) of the cluster to be deployed as a secret (key), using the secret name as the name of the cluster to be deployed. By creating a custom Application resource object, it configures information such as the Git / Helm (Helm plugin) repository address and the name of the cluster to be deployed in the resource object's spec (the spec attribute field in a Kubernetes resource object represents the expected value of the resource). ArgoCD automatically pulls the application's YAML (a highly readable format used to express data serialization) configuration file from the Git / Helm repository, connects to the rendered native Kubernetes YAML using the kubeconfig stored in the secret, and pushes it to the cluster to be deployed. Simultaneously, it monitors changes to the Git / Helm repository files and application-related resources in the cluster to be deployed, ensuring that the application resources in the cluster to be deployed are dynamically consistent with the expected values ​​in the repository.

[0004] It can be seen that ArgoCD's push mechanism heavily relies on network reachability from ArgoCD to the cluster to be deployed. In a distributed container cloud scenario, ArgoCD is deployed on the cloud provider's central cluster, while the clusters to be deployed (i.e., the access clusters) are scattered across the same cloud provider, other cloud providers, and the user's local data center. For security and cost considerations, the kube-apiserver service (a core component of the Kubernetes cluster, i.e., the API server, primarily responsible for providing the entry point for kube-apiserver access) typically does not provide a public IP address or domain name for external access. Therefore, ArgoCD on the cloud cannot directly access other VPCs of the same cloud provider, other cloud providers, or the user's local data center, limiting ArgoCD's ability to distribute multi-cloud, multi-cluster applications across network scenarios.

[0005] In summary, when using ArgoCD to distribute applications to multiple Kubernetes access clusters, the kubeconfig of the access clusters must be provided, and the network between ArgoCD and the kube-apiserver of the access clusters must be reachable. When the access clusters are located in other VPCs, other cloud vendors, or user-built data centers, network unavailability will prevent the use of ArgoCD for application distribution. Summary of the Invention

[0006] In view of this, the present invention provides a method, apparatus, device and medium for distributing multi-cloud and multi-cluster applications across networks, in order to solve the problem of limitations when using continuous delivery tools to distribute applications to multi-access clusters in cross-network scenarios.

[0007] In a first aspect, the present invention provides a method for distributing multi-cloud, multi-cluster applications across networks, the method comprising:

[0008] Create server applications and Kubernetes APIService resources in a pre-established central cluster, and listen for remote procedure call requests after the server applications start; the central cluster includes API servers;

[0009] In multiple pre-established access clusters, proxy applications are created, and remote procedure call requests are initiated to the API server through the proxy applications.

[0010] When the API server receives a remote procedure call request from the proxy application, it forwards it to the server application through the k8s APIService resource to create a bidirectional long-lived connection tunnel between the server application and the proxy application.

[0011] Deploy continuous delivery tools in the central cluster and create access cluster keys for application distribution using the continuous delivery tools. Based on the access cluster keys, the continuous delivery tools distribute the application to multiple access clusters through bidirectional long connection tunnels.

[0012] The cross-network, multi-cloud, multi-cluster application distribution method provided by this invention is a method for distributing applications to multiple Kubernetes clusters across networks based on Kubernetes APIService resources and reverse proxy tunnels, supporting continuous delivery tools (such as ArgoCD). This method involves deploying a server application and an agent application (hereinafter referred to as the agent application) in the central cluster and access clusters respectively. The agent application actively requests the APIService address exposed by the API server (hereinafter referred to as kube-apiserver) in the central cluster to access the server application behind the APIService. The server application and the agent application establish a bidirectional long-lived connection tunnel, which forwards the request to the kube-apiserver in the access cluster for application distribution and resource monitoring. This method achieves the goal of deploying only one continuous delivery tool in the central cluster, eliminating the need to deploy it in each access cluster, thus increasing the utilization rate of customer cluster resources. Simultaneously, the continuous delivery tool can distribute applications to all access clusters, fully leveraging its advanced features such as cluster-based tagging and flexible multi-cluster differentiated deployment, solving the limitation problem encountered when using continuous delivery tools to distribute applications to multiple access clusters in cross-network scenarios.

[0013] In one alternative implementation, the central cluster and multiple access clusters are established in the following manner:

[0014] Create multiple Kubernetes clusters, with one of them serving as the central cluster and the rest as multiple access clusters. The central cluster includes the API Service address exposed via public IP or domain name, CA certificate, client certificate, and client key. The API Service address exposed via public IP or domain name is integrated on the API server.

[0015] The cross-network multi-cloud multi-cluster application distribution method provided by this invention creates multiple Kubernetes clusters, with one of the Kubernetes clusters serving as the central cluster and the remaining Kubernetes clusters serving as multiple access clusters. The central cluster includes an APIService address exposed via public IP or domain name, a CA certificate, a client certificate, and a client key. The APIService address exposed via public IP or domain name is integrated on the API server, which is beneficial for proxy applications to access the APIService address on the API server.

[0016] In one alternative implementation, creating server applications and k8sAPIService resources in a pre-established central cluster includes:

[0017] Create a k8s APIService resource in a pre-established central cluster and point the k8s APIService resource to the server application so that the server application can be accessed by accessing the APIService address exposed by public IP or domain name integrated into the API server.

[0018] The cross-network multi-cloud multi-cluster application distribution method provided by this invention creates a k8s APIService resource in a pre-established central cluster and points the k8s APIService resource to the server application. The server application can be accessed by accessing the APIService address exposed by public IP or domain name integrated into the API server. Only the public IP or domain name needs to be added to the kube-apiserver in the central cluster. There is no need to configure the public IP or domain name in each access cluster, which reduces user costs, increases users' willingness to access the distributed container cloud, and provides conditions for establishing bidirectional long connection tunnels between server applications and proxy applications.

[0019] In one alternative implementation, proxy applications are created in multiple pre-established access clusters, and remote procedure call requests are initiated to the API server through the proxy applications, including:

[0020] In multiple pre-established access clusters, a proxy application containing startup parameters is created. The startup parameters include the APIService address of the central cluster exposed by public IP or domain name.

[0021] When the proxy application starts, it initiates a remote procedure call request to the API server through the APIService address exposed by the central cluster in the startup parameters, which is either a public IP address or a domain name.

[0022] The cross-network multi-cloud multi-cluster application distribution method provided by this invention creates a proxy application containing startup parameters in multiple pre-established access clusters. The startup parameters include the APIService address of the central cluster exposed by public IP or domain name. This enables the proxy application to initiate a remote procedure call request to the API server through the APIService address of the central cluster exposed by public IP or domain name in the startup parameters when it starts, thereby achieving the purpose of accessing the server application.

[0023] In one optional implementation, when the API server receives a remote procedure call request from the proxy application, it forwards it to the server application via the k8s APIService resource to create a bidirectional long-lived connection tunnel between the server application and the proxy application, including:

[0024] When the API server receives a remote procedure call request from the proxy application, it matches the k8sAPIService resource based on the URL path and forwards the remote procedure call request to the server application. When the server application receives the remote procedure call request, it communicates with the proxy application and creates a bidirectional long connection tunnel between the server application and the proxy application.

[0025] The cross-network multi-cloud multi-cluster application distribution method provided by this invention, when the API server receives a remote procedure call request from the proxy application, it matches the k8s APIService resource according to the URL path and forwards the remote procedure call request to the server application. When the server application receives the remote procedure call request, it realizes communication between the server application and the proxy application and establishes a bidirectional long connection tunnel between the server application and the proxy application, thus providing conditions for subsequent application distribution.

[0026] In one alternative implementation, deploying continuous delivery tools in a central cluster and creating access cluster keys for continuous delivery tool application distribution includes:

[0027] The APIService address exposed by the central cluster in the form of public IP or domain name is assembled into the API server address of the access cluster. The CA certificate of the central cluster is used as the CA certificate of the access cluster, the client certificate of the central cluster is used as the client certificate of the access cluster, and the client key of the central cluster is used as the client key of the access cluster to create an access cluster key for continuous delivery tool application distribution.

[0028] The cross-network multi-cloud multi-cluster application distribution method provided by this invention enables ArgoCD located in the central cluster to use the certificate private key of the central cluster to request the API server address of the central cluster. Through the tunnel forwarding created by the server application and the agent application, it accesses the API server (kube-apiserver) of the access cluster for application distribution and resource monitoring. This fully leverages the advanced features of ArgoCD multi-cluster distribution, reduces user costs, and reduces the risk of leakage of the user's access cluster kubeconfig configuration file.

[0029] In one alternative implementation, distributing applications to multiple access clusters via a bidirectional long-connection tunnel using a continuous delivery tool based on the access cluster key includes:

[0030] When a continuous delivery tool distributes an application to multiple access clusters, the name of the access cluster key is used as the name of the access cluster to which the application is distributed.

[0031] The cross-network multi-cloud multi-cluster application distribution method provided by this invention uses the name of the access cluster key as the name of the distributed access cluster when the continuous delivery tool distributes the application to multiple access clusters. This distinguishes the access clusters by name, which facilitates the management and resource monitoring of multiple access clusters.

[0032] Secondly, the present invention provides a cross-network multi-cloud multi-cluster application distribution device, the device comprising:

[0033] The central cluster configuration module is used to create server applications and k8sAPIService resources in a pre-established central cluster, and to listen for remote procedure call requests after the server application starts.

[0034] The access cluster configuration module is used to create proxy applications in multiple pre-established access clusters and initiate remote procedure call requests to the API server through the proxy applications.

[0035] The bidirectional long-connection tunnel creation module is used to create a bidirectional long-connection tunnel between the server application and the proxy application when the API server receives a remote procedure call request from the proxy application and forwards it to the server application through the k8s APIService resource.

[0036] The application distribution module is used to deploy continuous delivery tools in the central cluster and create access cluster keys for application distribution by the continuous delivery tools. Based on the access cluster keys, the continuous delivery tools distribute applications to multiple access clusters through bidirectional long connection tunnels.

[0037] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the cross-network multi-cloud multi-cluster application distribution method of the first aspect or any corresponding embodiment described above.

[0038] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the cross-network multi-cloud multi-cluster application distribution method of the first aspect or any corresponding embodiment thereof.

[0039] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the cross-network multi-cloud multi-cluster application distribution method of the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0040] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0041] Figure 1 This is the existing ArgoCD architecture diagram;

[0042] Figure 2 This is a flowchart illustrating a cross-network, multi-cloud, multi-cluster application distribution method according to an embodiment of the present invention.

[0043] Figure 3 This is a flowchart illustrating another cross-network multi-cloud multi-cluster application distribution method according to an embodiment of the present invention;

[0044] Figure 4 This is a flowchart illustrating another cross-network multi-cloud multi-cluster application distribution method according to an embodiment of the present invention;

[0045] Figure 5 This is a schematic diagram of the cross-network multi-cloud multi-cluster application distribution architecture according to an embodiment of the present invention;

[0046] Figure 6 This is a schematic diagram of another cross-network multi-cloud multi-cluster application distribution architecture according to an embodiment of the present invention;

[0047] Figure 7 This is a schematic diagram illustrating the relationship between kube-apiserver, k8s APIService resources, and server applications according to an embodiment of the present invention.

[0048] Figure 8 This is a structural block diagram of a cross-network multi-cloud multi-cluster application distribution device according to an embodiment of the present invention;

[0049] Figure 9 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] According to an embodiment of the present invention, a method for distributing multi-cloud and multi-cluster applications across networks is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0052] This embodiment provides a cross-network, multi-cloud, multi-cluster application distribution method, which can be used for the aforementioned computer terminals, such as central processing units and servers. Figure 2 This is a flowchart of a cross-network multi-cloud multi-cluster application distribution method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0053] Step S101: Create a server application and k8s APIService resources in the pre-established central cluster, and listen for remote procedure call requests after the server application starts; the central cluster includes an API server.

[0054] Specifically, k8s is an abbreviation for Kubernetes, an open-source container cluster management system used to automatically deploy, scale, and manage containerized applications.

[0055] Server applications, also known as server-side applications, are deployed as workloads in a central cluster. The API server is the Kubernetes API server, and the Kubernetes API Service is a resource object deployed alongside the server application in the central cluster. Kubernetes allows users to register custom API services in the cluster and access these services through the Kubernetes API server. When a user creates a Kubernetes API Service, Kubernetes creates a RESTful interface for that API service and generates a Uniform Resource Name (URN). Using the URN, other components and users can access and manipulate the API service through the Kubernetes API server. In the central cluster, the Kubernetes API Service resource acts as the front-end application, and the server application acts as the back-end application. The relationship between kube-apiserver, Kubernetes API Service, and the server application is as follows: Figure 7 As shown.

[0056] A remote procedure call request, or gRPC (gRemote Procedure Call) request for short, is a type of remote procedure call request that listens for gRPC requests to the cluster in real time after the server application starts.

[0057] Step S102: Create proxy applications in multiple pre-established access clusters and initiate remote procedure call requests to the API server through the proxy applications.

[0058] Specifically, such as Figure 6 As shown in step 1, the agent application, also known as the proxy application, initiates a remote procedure call request to the kube-apiserver using the APIService address, which carries the public IP address or domain name of the central cluster, in the startup parameters. The APIService address is short for the Kubernetes APIService address.

[0059] Step S103: When the API server receives the remote procedure call request from the proxy application, it forwards it to the server application through the k8sAPIService resource to create a bidirectional long connection tunnel between the server application and the proxy application.

[0060] Specifically, such as Figure 6Steps 2, 3, and 4 are shown below. After receiving a request, the kube-apiserver in the central cluster matches the k8s APIService resource based on the URL path. According to the backend service bound to the k8s APIServer, it forwards the request to the server application. The server application and the agent application establish a bidirectional long-lived connection tunnel, also known as a gRPC long-lived connection tunnel.

[0061] Step S104: Deploy the continuous delivery tool in the central cluster and create an access cluster key for application distribution using the continuous delivery tool. Based on the access cluster key, the continuous delivery tool distributes the application to multiple access clusters through a bidirectional long connection tunnel.

[0062] Specifically, the continuous delivery tool refers to ArgoCD. ArgoCD is a continuous delivery tool based on GitOps principles (GitOps is a model for implementing continuous delivery, the core idea of ​​which is to store the declarative infrastructure and applications of an application system in a Git version control repository). It uses a configuration repository (such as a Git / Helm repository) as the sole trusted source for application deployment and management. Users define the expected state of the application in the configuration repository. ArgoCD continuously listens for and pulls the configuration from the repository, delivers the application to the designated Kubernetes cluster according to the expected state, and continuously monitors changes to resources in the Kubernetes cluster to ensure that the actual resources in the Kubernetes cluster are consistent with the expected values ​​in the repository. The access cluster key is also called the access cluster secret.

[0063] For example, when distributing applications, ArgoCD uses the APIService address carrying the public IP address or domain name of the central cluster kube-apiserver, the CA certificate of the central cluster, the client certificate of the central cluster, and the client key of the central cluster to create an ArgoCD access cluster secret for use in ArgoCD application distribution and resource monitoring.

[0064] This embodiment provides a cross-network, multi-cloud, multi-cluster application distribution method. It utilizes a continuous delivery tool (such as ArgoCD) based on Kubernetes APIService resources and reverse proxy tunnels to distribute applications across multiple Kubernetes clusters. By deploying a server application and an agent application (hereinafter referred to as the agent application) in the central cluster and access clusters respectively, the agent application actively requests the APIService address exposed by the API server (hereinafter referred to as kube-apiserver) in the central cluster to access the server application behind the APIService. A bidirectional long-lived connection tunnel is established between the server application and the agent application. The request is forwarded through this tunnel to the kube-apiserver in the access cluster for application distribution and resource monitoring. This method eliminates the need to deploy a single continuous delivery tool in the central cluster, thus increasing the utilization of customer cluster resources. Furthermore, the continuous delivery tool can distribute applications to all access clusters, fully leveraging its advanced features such as cluster-based tagging and flexible multi-cluster deployment. This solves the limitations encountered when using continuous delivery tools to distribute applications to multiple access clusters in cross-network scenarios.

[0065] This embodiment provides a cross-network, multi-cloud, multi-cluster application distribution method, which can be used for the aforementioned computer terminals, such as central processing units and servers. Figure 3 This is a flowchart of a cross-network multi-cloud multi-cluster application distribution method according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps:

[0066] Step S201: Create a server application and k8s APIService resources in the pre-established central cluster, and listen for remote procedure call requests after the server application starts; the central cluster includes an API server.

[0067] Specifically, step S201 includes:

[0068] Step a. Create a k8s APIService resource in the pre-established central cluster and point the k8s APIService resource to the server application so that the server application can be accessed by accessing the APIService address exposed by public IP or domain name integrated into the API server.

[0069] Among them, such as Figure 5As shown, the central cluster and multiple access clusters are established in the following way: multiple Kubernetes clusters are created, one of which is used as the central cluster, and the remaining Kubernetes clusters are used as multiple access clusters. The central cluster includes the APIService address exposed by public IP or domain name, CA certificate, client certificate and client key. The APIService address exposed by public IP or domain name is integrated on the API server and is assembled by the public IP or domain name and the URN corresponding to the Kubernetes APIService.

[0070] like Figure 6 As shown, the central cluster creates a server application's deployment, configmap (an API object used to store configuration data), secret, service, and corresponding RBAC (Role-Based Access Control) resource requests.

[0071] Create a k8s APIService resource in the central cluster and point the k8s APIService resource to the backend server application. The server application can be accessed by accessing the APIService address, which is assembled from the public IP or domain name bound to the API server and the URN corresponding to the k8s APIService.

[0072] Step S202: Create proxy applications in multiple pre-established access clusters and initiate remote procedure call requests to the API server through the proxy applications.

[0073] Specifically, step S202 includes:

[0074] Step S2021: Create a proxy application containing startup parameters in each of the pre-established multiple access clusters. The startup parameters include the APIService address of the central cluster exposed by public IP or domain name.

[0075] Specifically, an agent application, or agent application for short, refers to a system with autonomy and intelligence, capable of sensing the external environment, making decisions, and executing corresponding actions to achieve predetermined goals. In this embodiment, the agent application is deployed in each access cluster and establishes a communication connection with the server application in the central cluster.

[0076] Step S2022: When the proxy application starts, it initiates a remote procedure call request to the API server through the APIService address exposed by the central cluster in the startup parameters using a public IP address or domain name.

[0077] Specifically, when the agent application starts, it sends a request to the kube-apiserver of the central cluster through the APIService address, which carries the public IP or domain name of the central cluster, in the startup parameters.

[0078] Step S203: After the API server receives the remote procedure call request from the proxy application, it forwards it to the server application via the k8sAPIService resource to create a bidirectional long-lived connection tunnel between the server application and the proxy application. For details, please refer to [link to relevant documentation]. Figure 2 Step S103 of the illustrated embodiment will not be described again here.

[0079] Step S204: Deploy the continuous delivery tool in the central cluster and create an access cluster key for application distribution using the continuous delivery tool. Based on the access cluster key, the continuous delivery tool distributes the application to multiple access clusters through a bidirectional long-connection tunnel. For details, please refer to [link to details]. Figure 2 Step S104 of the illustrated embodiment will not be described again here.

[0080] This embodiment provides a cross-network, multi-cloud, multi-cluster application distribution method. It creates a Kubernetes APIService resource in a pre-established central cluster and points this resource to a server application. This allows access to the server application via the APIService address exposed by a public IP address or domain name integrated into the API server. Only the public IP address or domain name needs to be added to the API server (kube-apiserver) in the central cluster; configuring a public IP address or domain name in each access cluster is unnecessary, reducing user costs and increasing user willingness to access the distributed container cloud. This also provides the conditions for establishing a bidirectional long-connection tunnel between the API server application and the proxy application. Proxy applications containing startup parameters are created in multiple pre-established access clusters. These startup parameters include the APIService address exposed by the central cluster using a public IP address or domain name. When the proxy application starts, it initiates a remote procedure call request to the API server via the APIService address exposed by the central cluster in the startup parameters, thereby achieving access to the server application.

[0081] This embodiment provides a cross-network, multi-cloud, multi-cluster application distribution method, which can be used for the aforementioned computer terminals, such as central processing units and servers. Figure 4 This is a flowchart of a cross-network multi-cloud multi-cluster application distribution method according to an embodiment of the present invention, such as... Figure 4 As shown, the process includes the following steps:

[0082] Step S301: Create a server application and k8s APIService resources in the pre-established central cluster, and listen for remote procedure call requests after the server application starts; the central cluster includes an API server. For details, please refer to [link to details]. Figure 3 Step S201 of the illustrated embodiment will not be described again here.

[0083] Step S302: Create proxy applications in each of the pre-established access clusters, and initiate remote procedure call requests to the API server through the proxy applications. See details below. Figure 3 Step S202 of the illustrated embodiment will not be described again here.

[0084] Step S303: When the API server receives the remote procedure call request from the proxy application, it forwards it to the server application through the k8sAPIService resource to create a bidirectional long connection tunnel between the server application and the proxy application.

[0085] Specifically, step S303 includes:

[0086] Step b. When the API server receives the remote procedure call request from the proxy application, it forwards the remote procedure call request to the server application based on the URL path matching k8s APIService resource. When the server application receives the remote procedure call request, it communicates with the proxy application and creates a bidirectional long connection tunnel between the server application and the proxy application.

[0087] Specifically, such as Figure 6 As shown, the URL path refers to the part of the network address used to specify the specific location of the resource. When the central cluster kube-apiserver receives a remote procedure call request, it matches the URN of the k8s APIService according to the URL path and forwards it to the server application. After communication, a bidirectional long connection tunnel, namely the gRPC long connection tunnel, is established between the server application and the agent application.

[0088] Step S304: Deploy the continuous delivery tool in the central cluster and create an access cluster key for application distribution using the continuous delivery tool. Based on the access cluster key, the continuous delivery tool distributes the application to multiple access clusters through a bidirectional long connection tunnel.

[0089] Specifically, step S304 includes:

[0090] Step S3041: Assemble the APIService address exposed by the central cluster in the form of public IP or domain name into the API server address of the access cluster, use the CA certificate of the central cluster as the CA certificate of the access cluster, use the client certificate of the central cluster as the client certificate of the access cluster, and use the client key of the central cluster as the client key of the access cluster to create an access cluster key for continuous delivery tool application distribution.

[0091] The access cluster key, or access cluster secret for short, is created by assembling the API Service address exposed by the central cluster kube-apiserver using a public IP address or domain name. This assembly consists of the API server address of the access cluster, the CA certificate of the central cluster, the client certificate of the central cluster, and the client key of the central cluster. The access cluster secret is used by the ArgoCD continuous delivery tool for application distribution and resource monitoring.

[0092] Step S3042: When the continuous delivery tool distributes the application to multiple access clusters, the name of the access cluster key is used as the name of the access cluster to which the application is distributed.

[0093] Specifically, when the continuous delivery tool ArgoCD distributes applications and monitoring resources to the access cluster, it uses the name of the access cluster secret created above as the name of the access cluster.

[0094] This embodiment provides a cross-network, multi-cloud, multi-cluster application distribution method. When the API server receives a remote procedure call request from a proxy application, it matches the k8s APIService resource based on the URL path and forwards the request to the server application. Upon receiving the request, the server application establishes communication with the proxy application and creates a bidirectional long-connection tunnel between them, facilitating subsequent application distribution. This allows ArgoCD, located in the central cluster, to use its central cluster's certificate and private key to request the k8s APIService address of the central cluster's API server. Through the tunnel created by the server and agent applications, the request is forwarded to access the k8s cluster resource object (kube-apiserver) of the access cluster's API server for application distribution and resource monitoring. This fully leverages ArgoCD's advanced multi-cluster distribution features, reduces user costs, and lowers the risk of kubeconfig file leakage from the user's access cluster. When the continuous delivery tool distributes applications to multiple access clusters, the name of the access cluster key is used as the name of the distributed access cluster, distinguishing access clusters by name and facilitating management and resource monitoring of multiple access clusters.

[0095] As one or more specific application embodiments of the present invention, combined with Figure 5 and Figure 6 The cross-network multi-cloud multi-cluster application distribution method provided by this invention will be further described in detail, such as... Figure 6 As shown, the specific process is as follows:

[0096] Step 1. As Figure 5 As shown, multiple Kubernetes clusters, i.e., k8s clusters, are created, one of which is planned as the central cluster, and the rest are planned as access clusters.

[0097] Step 2. Install and deploy the server application and k8s APIService resources (k8s cluster resource objects) in the central cluster, and mount them with public IP addresses or domain names.

[0098] Step 3. After the server application starts, it listens for gRPC requests.

[0099] Step 4. Deploy the agent application in the access cluster.

[0100] Step 5. After the agent application starts, it uses the APIService address, which carries the public IP address or domain name of the central cluster in the startup parameters, to send a request to the kube-apiserver of the central cluster.

[0101] Step 6. After receiving the request, the central cluster kube-apiserver forwards it to the backend server application for processing according to the k8s APIService resource configuration.

[0102] Step 7. After receiving the request from the agent application, the server application creates a bidirectional gRPC long connection tunnel.

[0103] Step 8. Create an access cluster secret in the central cluster for ArgoCD application distribution and resource monitoring. Configure the access cluster configuration file kubeconfig in the access cluster secret. Specifically, assemble the API server address of the access cluster using the API server address of the central cluster and the URN corresponding to the k8s APIService. Use the CA certificate of the central cluster as the CA certificate of the access cluster, the client certificate of the central cluster as the client certificate of the access cluster, and the client private key of the central cluster as the client private key of the access cluster. ArgoCD will then use the access cluster secret for application distribution and resource monitoring.

[0104] Step 9. When ArgoCD distributes applications and resource monitoring to the access cluster, it uses the name of the access cluster secret created above as the name of the access cluster.

[0105] This embodiment provides a cross-network, multi-cloud, multi-cluster application distribution method. By deploying server and agent applications in the central cluster and access clusters respectively, the agent application actively requests the APIService address exposed by the central cluster's kube-apiserver to access the server application behind the APIService. A bidirectional, long-lived connection tunnel is established between the two. The ArgoCD located in the central cluster uses the certificate and private key of this central cluster to request the k8sAPIService address of this central cluster. This request is forwarded through the tunnel established by the server and agent to access the kube-apiserver in the access cluster for application distribution and resource monitoring. In this embodiment, only one ArgoCD needs to be deployed in the central cluster, eliminating the need to deploy ArgoCD in each access cluster, thus increasing the utilization of customer cluster resources. Simultaneously, ArgoCD can distribute applications to all access clusters, fully leveraging its advanced features such as cluster-based tagging and flexible multi-cluster deployment. Only a public IP address or domain name needs to be added to the central cluster's kube-apiserver, eliminating the need to configure a public IP address or domain name in each access cluster, reducing user costs and increasing users' willingness to access distributed container clouds. There is no need to provide the kubeconfig for accessing the cluster for ArgoCD to create the cluster secret, reducing the risk of cluster kubeconfig leakage.

[0106] This embodiment also provides a cross-network multi-cloud, multi-cluster application distribution device, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0107] This embodiment provides a cross-network, multi-cloud, multi-cluster application distribution device, such as... Figure 8 As shown, it includes:

[0108] The central cluster configuration module 801 is used to create server applications and k8sAPIService resources in a pre-established central cluster, and to listen for remote procedure call requests after the server applications start; the central cluster includes an API server.

[0109] The access cluster configuration module 802 is used to create proxy applications in multiple pre-established access clusters and initiate remote procedure call requests to the API server through the proxy applications.

[0110] The bidirectional long-connection tunnel creation module 803 is used to create a bidirectional long-connection tunnel between the server application and the proxy application by forwarding the remote procedure call request from the proxy application to the server application through the k8s APIService resource after the API server receives the request.

[0111] Application distribution module 804 is used to deploy continuous delivery tools in the central cluster and create access cluster keys for application distribution by the continuous delivery tools. Based on the access cluster keys, the continuous delivery tools distribute applications to multiple access clusters through bidirectional long connection tunnels.

[0112] In some optional implementations, the central cluster configuration module 801 includes:

[0113] The central cluster configuration unit is used to create k8s APIService resources in a pre-established central cluster and point the k8s APIService resources to the server application, so that the server application can be accessed by accessing the APIService address exposed by public IP or domain name integrated into the API server.

[0114] In some optional implementations, the access cluster configuration module 802 includes:

[0115] The proxy application creation unit is used to create proxy applications containing startup parameters in multiple pre-established access clusters. The startup parameters include the APIService address of the central cluster exposed by public IP or domain name.

[0116] The request initiation unit is used to initiate a remote procedure call request to the API server when the proxy application starts, by using the APIService address exposed by the central cluster in the startup parameters in the form of a public IP address or domain name.

[0117] In some alternative implementations, the bidirectional long-connection tunnel creation module 803 includes:

[0118] The forwarding request unit is used to forward the remote procedure call request to the server application when the API server receives the remote procedure call request from the proxy application, based on the URL path matching the k8s APIService resource. When the server application receives the remote procedure call request, it communicates with the proxy application and creates a bidirectional long connection tunnel between the server application and the proxy application.

[0119] In some alternative implementations, the application distribution module 804 includes:

[0120] The access cluster key creation unit is used to assemble the APIService address exposed by the central cluster in the form of public IP or domain name into the API server address of the access cluster, use the CA certificate of the central cluster as the CA certificate of the access cluster, use the client certificate of the central cluster as the client certificate of the access cluster, and use the client key of the central cluster as the client key of the access cluster, so as to create an access cluster key for continuous delivery tool application distribution.

[0121] The cluster name determination unit is used to use the name of the access cluster key as the name of the access cluster being distributed when the continuous delivery tool distributes the application to multiple access clusters.

[0122] In some optional implementations, the cross-network multi-cloud, multi-cluster application distribution apparatus further includes:

[0123] The cluster creation module is used to create multiple Kubernetes clusters, with one of the Kubernetes clusters serving as the central cluster and the other Kubernetes clusters serving as multiple access clusters. The central cluster includes the APIService address exposed via public IP or domain name, CA certificate, client certificate, and client key. The APIService address exposed via public IP or domain name is integrated on the API server.

[0124] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0125] In this embodiment, the cross-network multi-cloud multi-cluster application distribution device is presented in the form of functional units. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0126] This invention also provides a computer device having the above-described features. Figure 8 The example shown is a cross-network, multi-cloud, multi-cluster application distribution device.

[0127] Please see Figure 9 , Figure 9 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 9As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 9 Take a processor 10 as an example.

[0128] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0129] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0130] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0131] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0132] The computer device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means. Figure 9 Taking the example of a connection between China and Israel via a bus.

[0133] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the computer device, such as a touchscreen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0134] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0135] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0136] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for distributing multi-cloud, multi-cluster applications across networks, characterized in that, The method includes: A server application and a k8s APIService resource are created in a pre-established central cluster, and remote procedure call requests are listened for after the server application starts; the central cluster includes an API server. In each of the pre-established access clusters, a proxy application is created, and a remote procedure call request is initiated to the API server through the proxy application. The central cluster and multiple access clusters are established in the following manner: Multiple Kubernetes clusters are created, with one of them serving as the central cluster and the remaining Kubernetes clusters serving as multiple access clusters. The central cluster includes an APIService address exposed via public IP or domain name, a CA certificate, client certificates, and client keys. The APIService address exposed via public IP or domain name is integrated on the API server. When the API server receives a remote procedure call request from the proxy application, it forwards it to the server application through the k8s APIService resource to create a bidirectional long connection tunnel between the server application and the proxy application. Deploy continuous delivery tools in the central cluster and create access cluster keys for application distribution using the continuous delivery tools. Based on the access cluster keys, the continuous delivery tools distribute the application to multiple access clusters through bidirectional long connection tunnels.

2. The method according to claim 1, characterized in that, The creation of server applications and Kubernetes APIService resources in the pre-established central cluster includes: Create a k8s APIService resource in a pre-established central cluster and point the k8s APIService resource to the server application so that the server application can be accessed by accessing the APIService address exposed by public IP or domain name integrated into the API server.

3. The method according to claim 2, characterized in that, The step of creating proxy applications in multiple pre-established access clusters and initiating remote procedure call requests to the API server through the proxy applications includes: In multiple pre-established access clusters, a proxy application containing startup parameters is created, including the APIService address of the central cluster exposed by public IP or domain name. When the proxy application starts, it initiates a remote procedure call request to the API server through the APIService address exposed by the central cluster in the startup parameters, which is either a public IP address or a domain name.

4. The method according to claim 1, characterized in that, The step of the API server forwarding a remote procedure call request from the proxy application to the server application via the k8s APIService resource to create a bidirectional long-lived connection tunnel between the server application and the proxy application includes: When the API server receives a remote procedure call request from the proxy application, it matches the k8sAPIService resource according to the URL path and forwards the remote procedure call request to the server application. When the server application receives the remote procedure call request, it communicates with the proxy application and creates a bidirectional long connection tunnel between the server application and the proxy application.

5. The method according to claim 1, characterized in that, Deploying continuous delivery tools in the central cluster and creating access cluster keys for application distribution of continuous delivery tools includes: The APIService address exposed by the central cluster in the form of public IP or domain name is assembled into the API server address of the access cluster. The CA certificate of the central cluster is used as the CA certificate of the access cluster, the client certificate of the central cluster is used as the client certificate of the access cluster, and the client key of the central cluster is used as the client key of the access cluster to create an access cluster key for continuous delivery tool application distribution.

6. The method according to claim 5, characterized in that, The method of distributing applications to multiple access clusters via a bidirectional long-connection tunnel using a continuous delivery tool based on the access cluster key includes: When the continuous delivery tool distributes the application to multiple access clusters, the name of the access cluster key is used as the name of the access cluster to which the application is distributed.

7. A cross-network multi-cloud, multi-cluster application distribution device, characterized in that, The device includes: The central cluster configuration module is used to create server applications and k8s APIService resources in a pre-established central cluster, and to listen for remote procedure call requests after the server applications start; the central cluster includes an API server; The access cluster configuration module is used to create proxy applications in multiple pre-established access clusters and initiate remote procedure call requests to the API server through the proxy applications. The central cluster and multiple access clusters are established in the following manner: Multiple Kubernetes clusters are created, with one of them serving as the central cluster and the remaining Kubernetes clusters serving as multiple access clusters. The central cluster includes an APIService address exposed via public IP or domain name, a CA certificate, client certificates, and client keys. The APIService address exposed via public IP or domain name is integrated on the API server. The bidirectional long-connection tunnel creation module is used to forward the remote procedure call request from the proxy application to the server application through the k8s APIService resource after the API server receives the request; in order to create a bidirectional long-connection tunnel between the server application and the proxy application. The application distribution module is used to deploy continuous delivery tools in the central cluster and create access cluster keys for application distribution by the continuous delivery tools. Based on the access cluster keys, the continuous delivery tools distribute the application to multiple access clusters through bidirectional long connection tunnels.

8. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the cross-network multi-cloud multi-cluster application distribution method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the cross-network multi-cloud multi-cluster application distribution method according to any one of claims 1 to 6.

Images