A service mesh disaster recovery method, device, medium

Abstract

This invention relates to a service mesh disaster recovery method, device, and medium, applied to a service mesh cluster. The method includes the following steps: setting up a global gateway within the service mesh cluster to resolve requests issued by the service mesh cluster to the global gateway; configuring IPTABLES to forward request information from the service mesh cluster to a target service; and deleting the policy in IPTABLES that redirects requests issued by the service mesh cluster to the SIDECAR when the SIDECAR fails, thereby achieving service mesh disaster recovery. Compared with existing technologies, this invention resolves requests issued by the cluster to the GLOBAL-SIDECAR IP through hardware DNS configuration rules, enabling disaster recovery switching in the event of a SIDECAR failure without restarting the POD, thus reducing business impact.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a service grid disaster recovery method, device, and medium. Background Technology

[0002] A service mesh is an infrastructure layer specifically designed to handle service communication; it represents the next generation of microservice architecture. Its responsibility is to ensure reliable request delivery within the complex topology of services composed of cloud-native applications. It is a set of lightweight network proxies deployed alongside application services and transparent to those services. The overall architecture of a service mesh consists of a data plane and a control plane.

[0003] The management component, known as the control plane, is responsible for communicating with agents in the data plane and distributing policies and configurations.

[0004] In the service mesh, the agent is called the sidecar or data plane. It directly handles inbound and outbound packets, including forwarding, routing, health checks, load balancing, authentication, authorization, and generating monitoring data.

[0005] ISITO is a service mesh technology. Existing service meshes are generally developed based on the open-source ISITO and then extended with additional functionality. In a Kubernetes cluster, Pods are the foundation for all business types. Currently, the service mesh platform, which is supposed to act as a transparent proxy for business services, sometimes causes business failures due to its own issues. The following problems exist when using a service mesh platform:

[0006] (1) When POD fails to start, the business container keeps restarting because of the health check configuration: When the business starts, it needs to call other services. If it fails, it will exit. There is no retry logic or disaster recovery logic, which makes the traffic sent by the business container after it starts unable to be processed.

[0007] (2) In some cases, DNS resolution may fail, making it impossible to process traffic sent by the business container: Because ISTIO's smart DNS is configured, the format of the response DNS data packet is different from that of ordinary DNS, which may lead to resolution failure when using other DNS clients. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the prior art by providing a service mesh disaster recovery method, device, and medium to solve the problem that existing methods lack a disaster recovery mechanism when the service mesh SIDECAR is abnormal.

[0009] The objective of this invention can be achieved through the following technical solutions:

[0010] In one aspect, the present invention provides a service mesh disaster recovery method applied to a service mesh cluster, the method comprising the following steps:

[0011] A global gateway is set up within the service mesh cluster to resolve requests issued by the service mesh cluster to the global gateway.

[0012] By configuring IPTABLES, request information from the service mesh cluster is forwarded to the target service;

[0013] When SIDECAR encounters an anomaly, the policy of redirecting requests from the service mesh cluster to SIDECAR in the IPTABLES is removed, and the global gateway is used for routing to achieve disaster recovery for the service mesh.

[0014] As a preferred technical solution, the SIDECAR includes a first unit for acquiring XDS rules and performing availability checks, and a second unit for forwarding request information from the service mesh cluster to the target service based on the XDS rules.

[0015] As a preferred technical solution, DNS is configured to resolve requests issued by the service mesh cluster to the IP address of the global gateway.

[0016] As a preferred technical solution, the service mesh cluster is configured by starting the POD and injecting the SIDECAR.

[0017] As a preferred technical solution, the criteria for determining whether SIDECAR is abnormal are as follows:

[0018] The unified registration center did not receive the heartbeat packet or the reception was abnormal.

[0019] As a preferred technical solution, the service mesh cluster is a K8S cluster.

[0020] As a preferred technical solution, the following steps are also included:

[0021] Obtain availability information for each container in the service mesh cluster and send heartbeat packets to the external unified registry center.

[0022] As a preferred technical solution, the service mesh cluster includes multiple containers.

[0023] In another aspect, an electronic device is provided, comprising: one or more processors and a memory, wherein the memory stores one or more programs, the one or more programs including instructions for performing the above-described service mesh disaster recovery method.

[0024] In another aspect, the present invention provides a computer-readable storage medium including one or more programs executable by one or more processors of an electronic device, said one or more programs including instructions for performing the above-described service mesh disaster recovery method.

[0025] Compared with existing technologies, this invention enables disaster recovery switching in the event of a SIDECAR failure without restarting the POD, thus reducing business impact. In this invention, requests issued by the service mesh cluster are resolved to the global gateway through hardware DNS configuration rules. During disaster recovery switching, the IPTABLES policy redirecting to SIDECAR is deleted, thereby solving or partially solving the problem of existing methods lacking a disaster recovery mechanism when the service mesh SIDECAR fails. Attached Figure Description

[0026] Figure 1 This is a flowchart of the service grid disaster recovery method in Example 1. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] Example 1

[0029] A service mesh is an infrastructure layer specifically designed to handle service communication and represents the next generation of microservice architecture. Its responsibility is to reliably deliver requests within the complex topology of services composed of cloud-native applications. It is a set of lightweight network proxies deployed alongside application services and transparent to those services. The overall architecture of a service mesh consists of a data plane and a control plane.

[0030] The management component, known as the control plane, is responsible for communicating with agents in the data plane and distributing policies and configurations.

[0031] In the service mesh, the agent is called the sidecar or data plane. It directly handles inbound and outbound packets, including forwarding, routing, health checks, load balancing, authentication, authorization, and generating monitoring data.

[0032] The traffic hijacking process of SIDECAR is as follows: IPTABLES is the management tool for the Linux kernel firewall software NETFILTER, located in user space and also part of NETFILTER. NETFILTER resides in kernel space and not only has network address translation capabilities but also firewall functions such as packet content modification and packet filtering. The inbound and outbound traffic of the service mesh is hijacked to SIDECAR through techniques such as IPTABLES.

[0033] The specific interception process is as follows:

[0034] Within the NETWORK NAMESPACE where the POD (container) resides, in addition to the traffic originating from ENVOY, the IPTABLES rules will intercept both incoming and outgoing traffic and redirect it to port 15001, which ENVOY is listening on, via NAT REDIRECT.

[0035] ENVOY will forward traffic according to the XDS rules obtained from PILOT.

[0036] ENVOY's LISTENER0.0.0.0:15001 receives all traffic entering and leaving the POD, and then forwards the requests to the corresponding VIRTUAL LISTENER.

[0037] For this POD service, there is an HTTP LISTENER PODIP + port to accept INBOUND traffic.

[0038] For each SERVICE+non-HTTP port, the listener-paired OUTBOUND non-HTTP traffic

[0039] Each SERVICE+HTTP port has an HTTP LISTENER: 0.0.0.0+ port to accept OUTBOUND traffic.

[0040] The entire interception and forwarding process is transparent to the business container. The business container still uses the SERVICE domain name and port to communicate, and the SERVICE domain name is still translated into the SERVICE IP. However, the SERVICE IP is directly translated into the POD IP in the SIDECAR. Traffic coming out of the container that has already used the POD IP will be directly forwarded to the corresponding POD. In contrast to the traditional KUBERNETES service mechanism, the SERVICE IP to POD IP conversion is performed on the NODE and implemented by IPTABLES maintained by KUBE-PROXY.

[0041] Service discovery in SIDECAR works as follows: DNS resolution is a crucial component of any application infrastructure. When application code attempts to access another service within a cluster, or even a service on the internet, it must first look up the IP address corresponding to the service hostname before initiating a connection. This name lookup process is commonly referred to as service discovery. In Kubernetes, the cluster DNS server, for services of type CLUSTERIP, resolves the service hostname to a unique, unroutable Virtual IP (VIP). The ISTIO SIDECAR proxy introduces a smart DNS proxy that controls the application's DNS resolution, returning a random IP. This request is then intercepted by IPTABLES in the POD and forwarded to the SIDECAR by the SIDECAR proxy.

[0042] Most existing service meshes are developed based on the open-source ISITO platform and have their functionality extended from there. Sidecar, which is supposed to act as a transparent proxy for business services, sometimes causes business failures due to its own inherent problems. The following issues are common when using a service mesh platform:

[0043] (1) When POD startup fails, the business container keeps restarting because of the health check configuration. When the business starts, it needs to call other services (such as pulling configuration from the configuration center). If it fails, it will exit without retry or disaster recovery logic. The reason for the failure is that the SIDECAR is not ready (the SIDECAR needs to pull configuration from the control plane, which takes time), which makes it impossible to process the traffic sent by the business container after startup.

[0044] (2) After enabling ISTIO's Smart DNS, DNS resolution fails in some cases, making it impossible to process traffic sent by the business container. The reason is that there is a problem with the implementation of Smart DNS. The format of the response DNS packet is different from that of ordinary DNS. It works fine when using the underlying library GLIBC, but it may fail when using other DNS clients.

[0045] like Figure 1 This embodiment provides a service mesh disaster recovery method. This embodiment uses the domain name suffix as part of the underlying service name of the service mesh, and disables the smart DNS service in ISTIO, using hardware DNS for resolution.

[0046] Deploy a GLOBAL-SIDECAR (global gateway, using the same technology as SIDECAR, but removing the service governance functions from the original SIDECAR and retaining only the east-west service discovery function) in each service mesh cluster. Configure the hardware DNS resolver server to resolve all requests from this KUBERNETES cluster to the IP address of the GLOBAL-SIDECAR.

[0047] Normally, when POD starts, it will automatically inject SIDECAR. At this time, SIDECAR will start an initialization container to initialize IPTABLES. Requests will be intercepted by IPTABLES and then SIDECAR will act as a proxy to forward the requests to the target service.

[0048] To ensure that business operations are not affected when the SIDECAR fails, this embodiment provides a SIDECAR disaster recovery switch. When the SIDECAR container fails, the IPTABLES policy that redirects requests to the SIDECAR is removed by controlling the IPTABLES maintained by the service mesh KUBE-PROXY. In this way, all traffic from the business service will be forwarded to GLOBAL-SIDECAR, which will then route and forward the traffic.

[0049] This method, during disaster recovery failover, controls the KUBE-PROXY to remove the IPTABLES policy redirecting to SIDECAR via the control plane, using the domain name suffix as the underlying service name of the service mesh. Requests from this KUBERNETES cluster are resolved to the IP address of GLOBAL-SIDECAR through hardware DNS configuration rules, enabling disaster recovery failover even in the event of a SIDECAR failure without restarting the POD, thus minimizing business impact.

[0050] Example 2

[0051] This embodiment provides an electronic device, including: one or more processors and a memory, wherein the memory stores one or more programs, the one or more programs including instructions for executing the service mesh disaster recovery method as described in Embodiment 1.

[0052] Example 3

[0053] This embodiment provides a computer-readable storage medium including one or more programs executable by one or more processors of an electronic device, the one or more programs including instructions for performing the service mesh disaster recovery method as described in Embodiment 1.

[0054] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A service grid disaster recovery method, characterized in that, Applied to service mesh clusters, the method includes the following steps: Set up a global gateway within the service mesh cluster to resolve requests from the service mesh cluster to the global gateway: Deploy a GLOBAL-SIDECAR in each service mesh cluster and configure the hardware DNS resolution server to resolve all requests from this KUBERNETES cluster to the IP address of the GLOBAL-SIDECAR. By configuring IPTABLES, request information from the service mesh cluster is forwarded to the target service: Under normal circumstances, POD is automatically injected into SIDECAR when it starts. SIDECAR starts and initializes the initialization container to initialize IPTABLES. Requests are intercepted by IPTABLES and SIDECAR, which then proxies and forwards the requests to the target service. When SIDECAR fails, the policy of redirecting requests from the service mesh cluster to SIDECAR in the IPTABLES is removed, and the global gateway is used for routing to achieve service mesh disaster recovery: the IPTABLES policy of redirecting requests to SIDECAR is removed by the service mesh control KUBE-PROXY-maintained IPTABLES, and all traffic from business services is forwarded to GLOBAL-SIDECAR, which then routes and forwards it.

2. The service mesh disaster recovery method according to claim 1, characterized in that, Requests from the service mesh cluster can be resolved to the IP address of the global gateway by configuring DNS.

3. The service mesh disaster recovery method according to claim 1, characterized in that, Configure the service mesh cluster by launching POD and injecting SIDECAR.

4. The service mesh disaster recovery method according to claim 1, characterized in that, The SIDECAR includes a first unit for obtaining XDS rules and performing availability checks, and a second unit for forwarding request information from the service mesh cluster to the target service based on the XDS rules.

5. A service mesh disaster recovery method according to claim 1, characterized in that, The service mesh cluster mentioned is a Kubernetes cluster.

6. The service mesh disaster recovery method according to claim 1, characterized in that, The service mesh cluster comprises multiple containers.

7. A service mesh disaster recovery method according to claim 1, characterized in that, It also includes the following steps: Obtain availability information for each container in the service mesh cluster and send heartbeat packets to the external unified registry center.

8. A service mesh disaster recovery method according to claim 7, characterized in that, The conditions for determining if a SIDECAR error has occurred are as follows: The unified registration center did not receive the heartbeat packet or the reception was abnormal.

9. An electronic device, characterized in that, include: One or more processors and a memory, wherein the memory stores one or more programs, the one or more programs including instructions for executing the service mesh disaster recovery method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, Includes one or more programs that are executed by one or more processors of an electronic device, said one or more programs including instructions for performing the service mesh disaster recovery method as described in any one of claims 1-8.

Images