Method, device and medium for eliminating LVS performance bottleneck when accessing K8S across clusters

Abstract

The present invention proposes a method, device and medium for eliminating the LVS performance bottleneck when K8S accesses across clusters, and relates to the fields of cloud computing and computer network communication technology. The method includes: constructing a service that can modify the IPVS rules under the Pod network namespace; generating a container image from the constructed service, deploying the container image in the K8S cluster using the daemon set mode, installing ipvsadm and related dependent package components in the container image, and granting the host network namespace data acquisition and IPVS rule modification permissions to the container process; using commands in the container image service to scan all network namespaces under the current machine, and traversing the scanned network namespaces; using commands to modify the IPVS rules under all scanned network namespaces, so that the traffic of the target cluster LVS virtual IP address is directly forwarded to the RS node of the target cluster. The present invention shortens the call path of cross-cluster call traffic and improves service call performance.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing and computer network communication technology, and in particular to a method, device and medium for eliminating LVS performance bottleneck during K8S cross-cluster access. Background Art

[0002] In the existing technology, the business system consists of hundreds of microservices, which are distributed in multiple K8S clusters and need to call each other frequently. In the current architecture, cross-cluster call traffic needs to be forwarded through the LVS DS (Director Server) node of the target cluster. As the traffic increases, the LVS DS node gradually becomes the performance bottleneck of the system, which is mainly manifested as follows:

[0003] 1. Single-point load problem: Since the LVS DS (Director Server) node is bound to the LVS virtual IP, it is impossible for multiple nodes to provide services at the same time, so it cannot be expanded horizontally. All traffic passing through is concentrated on one node. As the traffic increases, its processing capacity will reach its upper limit.

[0004] 2. Latency issue: Cross-cluster call traffic needs to be forwarded through the LVS DS (DirectorServer) and RS (RealServer) nodes of the target cluster in sequence, which increases network latency and affects the response speed of the request.

[0005] That is, in the existing architecture, the traffic path for calling services across clusters is: source cluster pod → target cluster LVSDS (Director Server) node → target cluster LVS RS (RealServer) node → target cluster pod. Since the target cluster DS (Director Server) node is bound to the target cluster LVS virtual IP, only a single node can provide services, and horizontal expansion is not supported. When large-scale traffic passes through this node, bottlenecks in some resources (such as network bandwidth, number of network connections, CPU or memory, etc.) may cause problems such as long call times or timeout failures, thereby affecting the normal operation of the business and system performance. Summary of the invention

[0006] The present invention proposes the following technical solutions to address one or more technical deficiencies in the above-mentioned prior art.

[0007] A method for eliminating LVS performance bottlenecks during K8S cross-cluster access, the method comprising:

[0008] Construction step: Construct a service that can modify the IPVS rules under the Pod network namespace. The service is used to maintain the IPVS rules of all network namespaces under a machine. The maintenance operations are as follows: Obtain all LVS rules under the network namespace. If there are no rules for the target LVS virtual IP and Real Server, create them; if they exist, update them based on the Real Server list.

[0009] Deployment step: Generate a container image of the constructed service, deploy the container image in the K8S cluster in the form of a DaemonSet, install ipvsadm and related dependency package components in the container image, and grant the container process the permission to obtain data from the host network namespace and modify IPVS rules.

[0010] Scanning step: Use the ip netns list command in the service of the container image to scan all network namespaces under the current machine and traverse the scanned network namespaces.

[0011] Modification step: Use the command ip netns exec <network namespace> <ipvsadm command> to modify the IPVS rules under all scanned network namespaces, so that the traffic of the target cluster LVS virtual IP address is directly forwarded to the RS (Real Server) nodes of the target cluster.

[0012] Furthermore, the specific operation in the scanning step is as follows: Start the service and obtain the virtual IP address of the target cluster LVS and the list of RS nodes of the target cluster LVS; Refresh and select the RS nodes of the target cluster LVS, and filter out the healthy nodes and the list of unhealthy nodes through port connectivity detection; Obtain the list of all network namespaces of the current machine and traverse them. If the service is in the cleaning mode or the number of healthy nodes is less than the first threshold, clear the IPVS rules of the namespace. Otherwise, add the IP addresses of the healthy nodes to the Real Server list of the LVS virtual IP address in the IPVS rules, and remove the IP addresses of the unhealthy nodes from the Real Server list of the LVS virtual IP in the IPVS rules. Obtain the list of all Real Servers under the current IPVS rules, remove the IP addresses not in the configuration list from the rules, and then start traversing the next network namespace.

[0013] Furthermore, mount the / lib / modules, / proc, and / var / run / netns directories of the host of the target cluster to the Pod.

[0014] Further, in the deployment step, a K8S configuration file for deploying the service is written in YAML format, and the service is deployed to the source K8S cluster in the form of a DaemonSet.

[0015] Further, enable hostNetwork, hostPID, and securityContext.privileged in the configuration file.

[0016] Further, the operation of filtering out the list of healthy nodes and unhealthy nodes through port connectivity detection is as follows: the service sends encrypted data packets to the Real Server through the port, and after receiving the encrypted data packets, the Real Server returns a confirmation message to the service, records the total duration of this process, repeats it three times, and obtains three durations T1, T2, and T3 respectively. Calculate the variance of T1, T2, and T3 as σ. If σ is less than the second threshold, add this node to the list of healthy nodes, otherwise add it to the list of unhealthy nodes.

[0017] The present invention also proposes a device for eliminating the LVS performance bottleneck during K8S cross-cluster access, and this device includes:

[0018] A construction unit that constructs a service capable of modifying IPVS rules under the Pod network namespace. This service is used to maintain the IPVS rules of all network namespaces under the machine, and the maintenance operation is as follows: obtain all LVS rules under the network namespace. If there are no rules for the target LVS virtual IP and Real Server, create them; if they exist, update them based on the Real Server list.

[0019] A deployment unit that generates a container image of the constructed service, deploys the container image in the K8S cluster in the mode of a DaemonSet, installs ipvsadm and related dependency package components in the container image, and grants the container process the permission to obtain data from the host network namespace and modify IPVS rules.

[0020] A scanning unit that uses the ip netns list command in the service of the container image to scan out all network namespaces under the current machine and traverses the scanned network namespaces.

[0021] A modification unit that uses ip netns exec <network namespace> <ipvsadm command> to modify the IPVS rules under all scanned network namespaces, so that the traffic of the target cluster LVS virtual IP address is directly forwarded to the RS (Real Server) nodes of the target cluster.

[0022] Furthermore, the specific operations of the scanning unit are: starting the service and obtaining the virtual IP address of the target cluster LVS and the RS node list of the target cluster LVS; refreshing and selecting the RS node of the target cluster LVS, and filtering out the healthy node and unhealthy node lists through port connectivity detection; obtaining the network namespace list of all the current machine and traversing it, if the service is in the cleaning mode or the number of healthy nodes is less than the first threshold, then clearing the IPVS rule of the namespace, otherwise adding the IP address of the healthy node to the Real Server list of the LVS virtual IP address in the IPVS rule, and removing the IP address of the unhealthy node from the Real Server list of the LVS virtual IP in the IPVS rule, obtaining all Real Server lists under the current IPVS rule, removing the IP addresses that are not in the configuration list from the rule, and then starting to traverse the next network namespace.

[0023] Furthermore, mount the / lib / modules, / proc, and / var / run / netns directories of the target cluster's host into the Pod.

[0024] Furthermore, in the deployment unit, a K8S configuration file for deploying the service is written in YAML format, and the service is deployed to the source K8S cluster in the form of a daemon process set.

[0025] Furthermore, hostNetwork, hostPID, and securityContext.privileged are enabled in the configuration file.

[0026] Furthermore, the operation of filtering out the list of healthy nodes and unhealthy nodes through port connectivity detection is: the service sends an encrypted data packet to the Real Server through the port, and the Real Server returns a confirmation message to the service after receiving the encrypted data packet, and records the total duration of the process, which is repeated three times to obtain three durations, namely T1, T2, and T3, and calculates the variance of T1, T2, and T3 as σ. If σ is less than the second threshold, the node is added to the healthy node list, otherwise it is added to the unhealthy node list.

[0027] The present invention further provides a computer-readable storage medium, wherein the storage medium stores computer program code, and when the computer program code is executed by a computer, any of the above methods is executed.

[0028] The technical effect of the present invention is as follows: A method, device and medium for eliminating the LVS performance bottleneck during cross-cluster access in K8S. The method includes: Construction step S101, constructing a service that can modify the IPVS rules under the Pod network namespace. The service is used to maintain the IPVS rules of all network namespaces under the machine. The maintenance operation is: obtaining all LVS rules under the network namespace, and if there are no rules for the target LVS virtual IP and Real Server, creating them; if they exist, updating them based on the Real Server list; Deployment step S102, generating a container image of the constructed service, deploying the container image in the K8S cluster in the mode of a DaemonSet, installing ipvsadm and related dependency package components in the container image, and granting the container process the permission to obtain data from the host network namespace and modify IPVS rules; Scanning step S103, using the ip netns list command in the service of the container image to scan all network namespaces under the current machine, and traversing the scanned network namespaces; Modification step S104, using ip netns exec <network namespace> <ipvsadm command> to modify the IPVS rules under all the scanned network namespaces, so that the traffic of the target cluster LVS virtual IP address is directly forwarded to the RS (Real Server) node of the target cluster.In the present invention, a service for modifying IPVS rules under a Pod network namespace is constructed. The service is used to maintain IPVS rules for all network namespaces under a machine. The maintenance operation is as follows: obtain all LVS rules under the network namespace. If there are no rules for the target LVS virtual IP and Real Server, create them; if they exist, update them based on the Real Server list. Generate a container image of the constructed service and deploy the container image in a K8S cluster. The service uses the ipnetns list command to scan all network namespaces under the current machine and traverses the scanned network namespaces. The ip netns exec <network namespace> <ipvsadm command> is used to modify the IPVS rules under all the scanned network namespaces, so that the traffic of the target cluster LVS virtual IP address is directly forwarded to the RS (Real Server) nodes of the target cluster, realizing that when making cross-cluster service calls, the calling traffic bypasses the target cluster DS (DirectorServer) node and directly reaches the RS (Real Server) node, solving the performance bottleneck problem of the target cluster LVS DS (DirectorServer) single point. And by directly routing the cross-cluster traffic to the target cluster RS (Real Server) node, the calling path of the cross-cluster call traffic is shortened, improving the service call performance. In the present invention, by traversing each obtained network namespace, the IP address of the unhealthy node is removed from the RealServer list of the LVS virtual IP in the IPVS rules, and all Real Server lists under the current IPVS rules are obtained, and the IP addresses not in the configured list are removed from the rules, thus avoiding directly sending data to an invalid node that cannot process data, resulting in data sending failure and reducing the data sending efficiency. In the present invention, in order to efficiently and accurately filter out healthy nodes and unhealthy nodes, so as to prevent directly sending data to a healthy node. The present invention improves the method of judging connectivity by the handshake protocol. The service sends encrypted data packets to the Real Server through a port. After receiving the encrypted data packets, the RealServer returns a confirmation message to the service, records the total duration of this process, repeats it three times, and obtains three durations T1, T2, and T3 respectively. Calculate the variance of T1, T2, and T3 as σ. If σ is less than the second threshold, add the node to the list of healthy nodes, otherwise add it to the list of unhealthy nodes. Since encrypted data packets are used, the security and reliability of the system can be ensured, and the method of calculating the variance of the duration is used to determine whether a node is healthy, and the judgment result is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Other features, objects and advantages of the present application will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings.

[0030] Figure 1 It is a flowchart of a method for eliminating the LVS performance bottleneck when K8S is accessed across clusters according to an embodiment of the present invention.

[0031] Figure 2 It is a structural diagram of a device for eliminating the LVS performance bottleneck when K8S is accessed across clusters according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0033] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0034] Figure 1 A method of eliminating the LVS performance bottleneck during K8S cross-cluster access of the present invention is shown, and the method includes:

[0035] Construct step S101, construct a service that can modify the IPVS rules under the Pod network namespace, the service is used to maintain the IPVS rules of all network namespaces under the machine, and the maintenance operation is: obtain all LVS rules under the network namespace, if the target LVS virtual IP and Real Server rules do not exist, create them; if they exist, update them based on the Real Server list;

[0036] Deployment step S102, generating a container image for the constructed service, deploying the container image in the K8S cluster using the daemon set (DeamoSet) mode, installing ipvsadm and related dependent package components in the container image, and granting the container process the host network namespace data acquisition and IPVS rule modification permissions;

[0037] Scanning step S103, using the ip netns list command in the service of the container image to scan all network namespaces under the current machine, and traverse the scanned network namespaces;

[0038] Modify step S104, and use the ip netns exec <network namespace> <ipvsadm command> to modify the IPVS rules under all scanned network namespaces, so that the traffic of the target cluster LVS virtual IP address is directly forwarded to the RS (Real Server) nodes of the target cluster.

[0039] In the present invention, a service for modifying IPVS rules under the Pod network namespace is constructed. The service is used to maintain the IPVS rules of all network namespaces under the machine. The maintenance operation is as follows: obtain all LVS rules under the network namespace. If there are no rules for the target LVS virtual IP and Real Server, create them; if they exist, update them based on the Real Server list. Generate a container image of the constructed service and deploy the container image in the K8S cluster. The service uses the ip netns list command to scan all network namespaces under the current machine and traverse the scanned network namespaces. Use the ip netns exec <network namespace> <ipvsadm command> to modify the IPVS rules under all scanned network namespaces, so that the traffic of the target cluster LVS virtual IP address is directly forwarded to the RS (Real Server) nodes of the target cluster, realizing that when making calls between cross-cluster services, the calling traffic bypasses the target cluster DS (Director Server) node and directly reaches the RS (RealServer) node, solving the performance bottleneck problem of the target cluster LVS DS (Director Server) single point, and by directly routing the cross-cluster traffic to the target cluster RS (RealServer) node, shortening the calling path of the cross-cluster call traffic and improving the service call performance, which is one of the important inventive concepts of the present invention.

[0040] In one embodiment, the specific operations in the scanning step S103 are: starting the service and obtaining the virtual IP address of the target cluster LVS and the RS node list of the target cluster LVS; refreshing and selecting the RS node of the target cluster LVS, and filtering out the healthy node and unhealthy node list through port connectivity detection; obtaining the network namespace list of all the current machine and traversing it, if the service is in the cleaning mode or the number of healthy nodes is less than the first threshold, such as the first threshold is 10, then clearing the IPVS rule of the namespace, otherwise adding the IP address of the healthy node to the Real Server list of the LVS virtual IP address in the IPVS rule, and removing the IP address of the unhealthy node from the Real Server list of the LVS virtual IP in the IPVS rule, obtaining all Real Server lists under the current IPVS rule, removing the IP address that is not in the configuration list from the rule, and then starting to traverse the next network namespace.

[0041] An important step in realizing the present invention is to scan and obtain the Real Server, so that the direct sending of data can be realized by modifying the IPVS rules. In the present invention, the constructed service is started and the virtual IP address of the target cluster LVS and the RS node list of the target cluster LVS are obtained; the healthy node and the unhealthy node list are filtered out through port connectivity detection; the network namespace list of all the current machines is obtained and traversed. If the service is in the cleaning mode or the number of healthy nodes is less than the first threshold, such as the first threshold is 10, the IPVS rule of the namespace is cleared, otherwise the IP address of the healthy node is added to the Real Server list of the LVS virtual IP address in the IPVS rule. In the present invention, by traversing each obtained network namespace, the IP address of the unhealthy node is removed from the Real Server list of the LVS virtual IP in the IPVS rule, and all Real Server lists under the current IPVS rule are obtained, and the IP address not in the configuration list is removed from the rule, so as to avoid the node directly sent being an invalid node, unable to process data, resulting in data sending failure, and reducing the efficiency of data sending. This is another important inventive concept of the present invention.

[0042] In another embodiment, in order to enable the service to scan all network namespaces on the host of the target cluster, the host's / lib / modules, / proc, and / var / run / netns directories need to be mounted into the Pod. The following is a relevant configuration code snippet for directory mounting.

[0043] YAML

[0044] apiVersion:apps / v1

[0045] kind:DaemonSet

[0046] metadata:

[0047] name:ipvs-rule-k8s0-to-k8s1

[0048] spec:

[0049] template:

[0050] spec:

[0051] containers:

[0052] - name:ipvsadm

[0053] volumeMounts:

[0054] - name:lib-modules

[0055] mountPath: / lib / modules

[0056] readOnly:true

[0057] - name:proc

[0058] mountPath: / proc

[0059] readOnly:true

[0060] - name:var-run-netns

[0061] mountPath: / var / run / netns

[0062] readOnly:true

[0063] volumes:

[0064] - name:lib-modules

[0065] hostPath:

[0066] path: / lib / modules

[0067] - name:proc

[0068] hostPath:

[0069] path: / proc

[0070] - name:var-run-netns

[0071] hostPath:

[0072] path: / var / run / netns

[0073] In another embodiment, in the deployment step S102, a K8S configuration file for deploying the service is written in YAML format, and the service is deployed to the source K8S cluster in the form of a daemon set.

[0074] In another embodiment, hostNetwork, hostPID and securityContext.privileged are enabled in the configuration file. That is, in order to allow the service to have the authority to normally use the system's ipvsadm command and query all process information of the machine, hostNetwork, hostPID and securityContext.privileged need to be enabled in the deployed YAML configuration file. The following is the relevant configuration snippet.

[0075] YAML

[0076] apiVersion:apps / v1

[0077] kind:DaemonSet

[0078] metadata:

[0079] name:ipvs-rule-k8s0-to-k8s1

[0080] spec:

[0081] template:

[0082] spec:

[0083] containers:

[0084] - name:ipvsadm

[0085] securityContext:

[0086] privileged:true

[0087] hostNetwork:true

[0088] hostPID:true

[0089] In another embodiment, the operation of filtering out healthy nodes and unhealthy node lists through port connectivity detection is: the service sends an encrypted data packet to the Real Server through the port, and the Real Server returns a confirmation message to the service after receiving the encrypted data packet, and records the total duration of the process, which is repeated three times to obtain three durations, namely T1, T2, and T3. The variance of T1, T2, and T3 is calculated as σ. If σ is less than the second threshold, the node is added to the healthy node list, otherwise it is added to the unhealthy node list.

[0090] In order to efficiently and accurately filter out healthy nodes and unhealthy nodes in the present invention, thereby preventing the directly sent nodes from being healthy nodes, the present invention improves the way of determining connectivity through the handshake protocol, and uses the service to send an encrypted data packet to the Real Server through a port. After receiving the encrypted data packet, the Real Server returns a confirmation message to the service, and records the total duration of the process, which is repeated three times to obtain three durations, namely T1, T2, and T3. The variance of T1, T2, and T3 is calculated as σ. If σ is less than a second threshold, the node is added to the healthy node list, otherwise it is added to the unhealthy node list. Since an encrypted data packet is used, the security and reliability of the system can be ensured, and the method of calculating the duration variance is used to determine whether the node is healthy, and the judgment result is more accurate. This is another inventive concept of the present invention.

[0091] Figure 2 A device for eliminating LVS performance bottleneck during K8S cross-cluster access of the present invention is shown, and the device includes:

[0092] The construction unit 201 constructs a service that can modify the IPVS rules under the Pod network namespace, and the service is used to maintain the IPVS rules of all network namespaces under the machine. The maintenance operation is: obtain all LVS rules under the network namespace, and if the target LVS virtual IP and Real Server rules do not exist, create them; if they exist, update them based on the Real Server list;

[0093] The deployment unit 202 generates a container image for the constructed service, deploys the container image in the K8S cluster using the daemon set (DeamoSet) mode, installs ipvsadm and related dependent package components in the container image, and grants the container process the host network namespace data acquisition and IPVS rule modification permissions;

[0094] The scanning unit 203 uses the ip netns list command to scan all network namespaces under the current machine in the service of the container image, and traverses the scanned network namespaces;

[0095] The modification unit 204 uses the ip netns exec <network namespace> <ipvsadm command> to modify the IPVS rules under all the scanned network namespaces, so that the traffic of the target cluster LVS virtual IP address is directly forwarded to the RS (Real Server) nodes of the target cluster.

[0096] In the present invention, a service for modifying the IPVS rules under the Pod network namespace is constructed. The service is used to maintain the IPVS rules of all network namespaces under the machine. The maintenance operation is as follows: obtain all LVS rules under the network namespace. If there are no rules for the target LVS virtual IP and Real Server, create them; if they exist, update them based on the Real Server list. Generate a container image of the constructed service and deploy the container image in the K8S cluster. The service uses the ip netns list command to scan all network namespaces under the current machine and traverses the scanned network namespaces. The ip netns exec <network namespace> <ipvsadm command> is used to modify the IPVS rules under all the scanned network namespaces, so that the traffic of the target cluster LVS virtual IP address is directly forwarded to the RS (Real Server) nodes of the target cluster. Thus, when making an inter-cluster service call, the calling traffic bypasses the target cluster DS (DirectorServer) node and directly reaches the RS (RealServer) node, solving the performance bottleneck problem of the target cluster LVS DS (DirectorServer) single point. Moreover, by directly routing the inter-cluster traffic to the target cluster RS (Real Server) node, the calling path of the inter-cluster call traffic is shortened, and the service call performance is improved. This is one of the important inventive concepts of the present invention.

[0097] In one embodiment, the specific operations of the scanning unit 203 are: starting the service and obtaining the virtual IP address of the target cluster LVS and the RS node list of the target cluster LVS; refreshing and selecting the RS node of the target cluster LVS, and filtering out the healthy node and unhealthy node list through port connectivity detection; obtaining the network namespace list of all the current machine and traversing it, if the service is in the cleaning mode or the number of healthy nodes is less than the first threshold, such as the first threshold is 10, then clearing the IPVS rule of the namespace, otherwise adding the IP address of the healthy node to the Real Server list of the LVS virtual IP address in the IPVS rule, and removing the IP address of the unhealthy node from the Real Server list of the LVS virtual IP in the IPVS rule, obtaining all Real Server lists under the current IPVS rule, removing the IP addresses that are not in the configuration list from the rule, and then starting to traverse the next network namespace.

[0098] An important step in realizing the present invention is to scan and obtain the Real Server, so that the direct sending of data can be realized by modifying the IPVS rules. In the present invention, the constructed service is started and the virtual IP address of the target cluster LVS and the RS node list of the target cluster LVS are obtained; the healthy node and the unhealthy node list are filtered out through port connectivity detection; the network namespace list of all the current machines is obtained and traversed. If the service is in the cleaning mode or the number of healthy nodes is less than the first threshold, such as the first threshold is 10, the IPVS rule of the namespace is cleared, otherwise the IP address of the healthy node is added to the Real Server list of the LVS virtual IP address in the IPVS rule. In the present invention, by traversing each obtained network namespace, the IP address of the unhealthy node is removed from the Real Server list of the LVS virtual IP in the IPVS rule, and all Real Server lists under the current IPVS rule are obtained, and the IP address not in the configuration list is removed from the rule, so as to avoid the node directly sent being an invalid node, unable to process data, resulting in data sending failure, and reducing the efficiency of data sending. This is another important inventive concept of the present invention.

[0099] In another embodiment, in order to enable the service to scan all network namespaces on the host of the target cluster, the host's / lib / modules, / proc, and / var / run / netns directories need to be mounted into the Pod. The following is a relevant configuration code snippet for directory mounting.

[0100] YAML

[0101] apiVersion:apps / v1

[0102] kind:DaemonSet

[0103] metadata:

[0104] name:ipvs-rule-k8s0-to-k8s1

[0105] spec:

[0106] template:

[0107] spec:

[0108] containers:

[0109] - name:ipvsadm

[0110] volumeMounts:

[0111] - name:lib-modules

[0112] mountPath: / lib / modules

[0113] readOnly:true

[0114] - name:proc

[0115] mountPath: / proc

[0116] readOnly:true

[0117] - name:var-run-netns

[0118] mountPath: / var / run / netns

[0119] readOnly:true

[0120] volumes:

[0121] - name:lib-modules

[0122] hostPath:

[0123] path: / lib / modules

[0124] - name:proc

[0125] hostPath:

[0126] path: / proc

[0127] - name:var-run-netns

[0128] hostPath:

[0129] path: / var / run / netns

[0130] In another embodiment, in the deployment unit 202, a K8S configuration file for deploying the service is written in YAML format, and the service is deployed to the source K8S cluster in the form of a daemon set.

[0131] In another embodiment, hostNetwork, hostPID and securityContext.privileged are enabled in the configuration file. That is, in order to allow the service to have the authority to normally use the system's ipvsadm command and query all process information of the machine, hostNetwork, hostPID and securityContext.privileged need to be enabled in the deployed YAML configuration file. The following is the relevant configuration snippet.

[0132] YAML

[0133] apiVersion:apps / v1

[0134] kind:DaemonSet

[0135] metadata:

[0136] name:ipvs-rule-k8s0-to-k8s1

[0137] spec:

[0138] template:

[0139] spec:

[0140] containers:

[0141] - name:ipvsadm

[0142] securityContext:

[0143] privileged:true

[0144] hostNetwork:true

[0145] hostPID:true

[0146] In another embodiment, the operation of filtering out healthy nodes and unhealthy node lists through port connectivity detection is: the service sends an encrypted data packet to the Real Server through the port, and the Real Server returns a confirmation message to the service after receiving the encrypted data packet, and records the total duration of the process, which is repeated three times to obtain three durations, namely T1, T2, and T3. The variance of T1, T2, and T3 is calculated as σ. If σ is less than the second threshold, the node is added to the healthy node list, otherwise it is added to the unhealthy node list.

[0147] In order to efficiently and accurately filter out healthy nodes and unhealthy nodes in the present invention, thereby preventing the directly sent nodes from being healthy nodes, the present invention improves the way of determining connectivity through the handshake protocol, and uses the service to send an encrypted data packet to the Real Server through a port. After receiving the encrypted data packet, the Real Server returns a confirmation message to the service, and records the total duration of the process, which is repeated three times to obtain three durations, namely T1, T2, and T3. The variance of T1, T2, and T3 is calculated as σ. If σ is less than a second threshold, the node is added to the healthy node list, otherwise it is added to the unhealthy node list. Since an encrypted data packet is used, the security and reliability of the system can be ensured, and the method of calculating the duration variance is used to determine whether the node is healthy, and the judgment result is more accurate. This is another inventive concept of the present invention.

[0148] In one embodiment of the present invention, a computer storage medium is provided, on which a computer program is stored. When the computer program on the computer storage medium is executed by a processor, the above method is implemented. The computer storage medium can be a hard disk, DVD, CD, flash memory or other memory.

[0149] For the convenience of description, the present invention describes the above device by dividing it into various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0150] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a storage medium, such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the devices described in each embodiment of the present application or some parts of the embodiments.

[0151] Finally, it should be noted that the above embodiments are only intended to illustrate rather than limit the technical solutions of the present invention. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the present invention can still be modified or replaced by equivalents. Any modification or partial replacement that does not depart from the spirit and scope of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A method for eliminating LVS performance bottlenecks during K8S cross-cluster access, characterized in that: The method includes: A construction step of constructing a service that can modify the IPVS rules under the Pod network namespace. The service is used to maintain the IPVS rules of all network namespaces under a machine. The maintenance operation is as follows: obtain all LVS rules under the network namespace, and if there are no rules for the target LVS virtual IP and Real Server, create them; if they exist, update them based on the Real Server list. A deployment step of generating a container image of the constructed service, deploying the container image in the K8S cluster in the mode of a DaemonSet, installing ipvsadm and related dependent package components in the container image, and granting the container process the permission to obtain host network namespace data and modify IPVS rules. A scanning step of using the ip netns list command in the service of the container image to scan out all network namespaces under the current machine and traverse the scanned network namespaces. A modification step of using ip netns exec <network namespace> <ipvsadm command> to modify the IPVS rules under all the scanned network namespaces, so that the traffic of the target cluster LVS virtual IP address is directly forwarded to the RS (Real Server) nodes of the target cluster.

2. The method according to claim 1, characterized in that The specific operation in the scanning step is as follows: start the service and obtain the virtual IP address of the target cluster LVS and the RS node list of the target cluster LVS; refresh and select the RS nodes of the target cluster LVS, and filter out the healthy nodes and the list of unhealthy nodes through port connectivity detection; obtain the list of all network namespaces of the current machine and traverse it. If the service is in the cleaning mode or the number of healthy nodes is less than the first threshold, clean up the IPVS rules of the namespace. Otherwise, add the IP addresses of the healthy nodes to the Real Server list of the LVS virtual IP address in the IPVS rules, and remove the IP addresses of the unhealthy nodes from the Real Server list of the LVS virtual IP in the IPVS rules. Obtain the list of all Real Servers under the current IPVS rules, remove the IP addresses not in the configuration list from the rules, and then start traversing the next network namespace.

3. The method according to claim 2, characterized in that Mount the / lib / modules, / proc, and / var / run / netns directories of the host of the target cluster into the Pod.

4. The method according to claim 3, characterized in that In the deployment step, write a K8S configuration file for deploying the service in YAML format, and deploy the service to the source K8S cluster in the way of a DaemonSet.

5. The method according to claim 4, characterized in that Enable hostNetwork, hostPID, and securityContext.privileged in the configuration file.

6. A device for eliminating LVS performance bottlenecks during K8S cross-cluster access, characterized in that: The device includes: Building unit, which builds a service that can modify IPVS rules under the Pod network namespace. The service is used to maintain IPVS rules for all network namespaces under a machine. The maintenance operations are as follows: obtain all LVS rules under the network namespace, if there are no rules for the target LVS virtual IP and Real Server, create them; if they exist, update them based on the Real Server list. Deployment unit, which generates a container image for the built service, deploys the container image in the K8S cluster in the form of a DaemonSet, installs ipvsadm and related dependent package components in the container image, and grants the container process the permission to obtain data from the host network namespace and modify IPVS rules. Scanning unit, which uses the ip netns list command in the service of the container image to scan all network namespaces under the current machine and traverses the scanned network namespaces. Modifying unit, which uses ip netns exec <network namespace> <ipvsadm command> to modify the IPVS rules under all the scanned network namespaces, so that the traffic of the target cluster LVS virtual IP address is directly forwarded to the RS (Real Server) nodes of the target cluster.

7. The device according to claim 6, characterized in that The specific operation of the scanning unit is as follows: start the service and obtain the virtual IP address of the target cluster LVS and the list of RS nodes of the target cluster LVS; refresh and select the RS nodes of the target cluster LVS, and filter out healthy nodes and unhealthy node lists through port connectivity detection; obtain and traverse the list of all network namespaces of the current machine. If the service is in the cleaning mode or the number of healthy nodes is less than the first threshold, clean up the IPVS rules of the namespace. Otherwise, add the IP addresses of healthy nodes to the Real Server list of the LVS virtual IP address in the IPVS rules, and remove the IP addresses of unhealthy nodes from the Real Server list of the LVS virtual IP in the IPVS rules. Obtain the list of all Real Server under the current IPVS rules, remove the IP addresses not in the configuration list from the rules, and then start traversing the next network namespace.

8. The device according to claim 7, characterized in that Mount the / lib / modules, / proc, and / var / run / netns directories of the host of the target cluster to the Pod.

9. The device according to claim 8, characterized in that In the deployment unit, use the YAML format to write the K8S configuration file for deploying the service, and deploy the service to the source K8S cluster in the form of a DaemonSet.

10. A computer storage medium, characterized in that: A computer program is stored on the computer storage medium. When the computer program on the computer storage medium is executed by a processor, the method described in any one of claims 1-5 is implemented.

Images