Pod migration method under Kubernetes cluster architecture system

Abstract

The invention discloses a Pod migration method under a Kubernetes cluster architecture system, which comprises the following steps of: 1, continuously monitoring the resource imbalance degree of a cluster, judging whether the resource imbalance degree exceeds a threshold value or not, and if so, executing a step 2; if not, continuing monitoring; 2, finding out nodes with serious resource imbalance, finding out Pod with the maximum influence on the resource imbalance in the nodes, and determining that the Pod is the Pod to be migrated; 3, state information of all nodes in the current cluster is collected, a Pod migration program is called for the to-be-migrated Pod, and finally a new target node is obtained; and 4, firstly creating the Pod on the new target node, and then deleting the original Pod until the Pod is successfully migrated to the new target node and runs. According to the invention, the limitation problem of Kubernetes static scheduling is solved, and the load balance of cluster resources and the reliability of services are ensured.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a Pod migration method under a Kubernetes cluster architecture system. Background Art

[0002] With the rapid development of cloud computing and containerization, enterprises are increasingly demanding the ability to build, deploy, and manage applications. Kubernetes, a highly scalable container orchestration platform, provides enterprises with powerful tools and features to simplify application deployment and management. It automates container deployment, scaling, load balancing, and fault recovery, significantly reducing the workload for developers and operations personnel.

[0003] However, the Kubernetes scheduler performs resource scheduling within the cluster on a one-time basis. This means it only considers the cluster's load balancing and the matching degree between the cluster's nodes and the scheduled pods when initially deploying a service. Once scheduling is complete, it does not adjust again. In today's era of big data and microservices, servers receive and process massive amounts of data daily. Over time, the resource load on each working node in the cluster fluctuates significantly, causing the load on the entire cluster to deteriorate, leading to decreased service quality or even interruption. In other words, the initial scheduling choices are no longer applicable, so this static scheduling approach cannot address the issue of severely unbalanced utilization of various resources in the cluster over time.

[0004] Therefore, to address the problem of unbalanced cluster load over time in Kubernetes clusters, a Pod migration method under the Kubernetes cluster architecture system is needed to maintain and improve the resource load balance of the cluster. Summary of the Invention

[0005] 1. Technical problem to be solved by the invention

[0006] To address the problem of resource utilization imbalance caused by changes in cluster resource load over time, the present invention proposes a Pod migration method under a Kubernetes cluster architecture system. The present invention monitors cluster resources and then migrates Pods that need to be migrated on certain nodes to the most suitable new target node at the moment according to the Pod migration method. This approach solves the limitations of Kubernetes static scheduling and ensures load balancing of cluster resources and service reliability.

[0007] 2. Technical solution

[0008] In order to achieve the above object, the technical solution provided by the present invention is:

[0009] The present invention provides a Pod migration method under a Kubernetes cluster architecture system, comprising the following steps:

[0010] Step 1: Continuously monitor the resource imbalance of the cluster to determine whether it exceeds the threshold. If so, proceed to step 2; if not, continue monitoring.

[0011] Step 2: Find the node with severe resource imbalance and the pod on this node that has the greatest impact on resource imbalance. Determine the pod to be migrated.

[0012] Step 3: Collect the status information of all nodes in the current cluster, call the Pod migration program for the Pod to be migrated, and finally obtain a new target node;

[0013] Step 4: Create a Pod on the new target node first, then delete the original Pod. The Pod is now successfully migrated to the new target node and is running. Return to step 1.

[0014] 3. Beneficial effects

[0015] Compared with the existing known technologies, the technical solution provided by the present invention has the following significant effects:

[0016] (1) The present invention provides a Pod migration method under a Kubernetes cluster architecture system, which continuously monitors the imbalance of cluster resources, finds Pods with heavy loads for certain nodes with resource imbalance, calls a Pod migration program, and adopts a mechanism of first adding and then deleting Pods after finding the new target node. While maintaining the balance of cluster resources, it also effectively prevents service interruptions.

[0017] (2) In the era of the Internet of Things, a large number of devices generate massive amounts of data of different specifications every day. The load balance of the cluster will gradually deteriorate over time. The present invention uses a similarity formula to calculate the matching degree of the Pod in the unbalanced node with a certain node, and then calculates the new target node with the node performance coefficient, which effectively improves the resource load of the cluster and enhances the stability of service operation.

[0018] (3) The Pod migration method proposed in the present invention completes the Pod migration by first creating a Pod on the new target node and then deleting the original old Pod. This avoids the situation where the new Pod is not created but the old Pod is deleted during the Pod migration process, and effectively prevents the possible Pod service interruption problem. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of a Pod migration method under a Kubernetes cluster architecture system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to further understand the content of the present invention, the present invention is described in detail with reference to the accompanying drawings and embodiments.

[0021] Example

[0022] like Figure 1 As shown, the Pod migration method under a Kubernetes cluster architecture system of this embodiment can improve the problem of cluster resource load imbalance caused by the passage of time in the Kubernetes cluster, thereby improving service efficiency and stability. The method includes the following steps:

[0023] Step 1: Continuously monitor the resource imbalance of the cluster to determine whether it exceeds the threshold. If so, proceed to step 2; if not, continue monitoring.

[0024] When a service is deployed as a pod in a Kubernetes cluster, the default Kubernetes scheduler assigns the pod to a suitable node. However, in the era of big data, various services need to receive and process massive amounts of data daily. As time passes, resource usage across cluster nodes can become increasingly uneven, leading to an imbalance in cluster resource load. At this point, the initial scheduling results are no longer applicable to the current cluster state, necessitating a monitoring program to monitor cluster resource imbalances in real time.

[0025] The monitoring program, running as a standalone service, continuously monitors resource usage across all nodes in the cluster using monitoring tools such as Prometheus and node-exporter. The monitoring program uses the prometheus-api-client library to call PromQL queries to obtain real-time resource usage for each node. The Kubernetes library provides status information for components such as Pods and Deployments in the Kubernetes cluster. Simple data preprocessing, such as rounding or unit conversion, is performed as necessary.

[0026] Use the cluster resource imbalance calculation formula to determine the cluster resource imbalance status. The formula is as follows:

[0027]

[0028] Where B represents the cluster resource imbalance, C represents the number of nodes in the cluster, and std(i) represents the standard deviation of the resource utilization of node i.

[0029]

[0030] Among them, U(i, r) represents the utilization rate of resource r in node i, U AVR (i) represents the comprehensive resource utilization rate of node i. m represents the number of resources involved in the calculation, and R includes but is not limited to CPU, memory, network I / O, and disk I / O.

[0031] Step 2: Identify the node with a severe resource imbalance and the pod on this node that has the greatest impact on the resource imbalance. These are the pods to be migrated. To identify the node with a severe resource imbalance, look for the node with the largest standard deviation in resource utilization. A larger standard deviation indicates that two resources on this node have significantly different utilization rates, indicating a severe resource imbalance on this node. Then, query the current resource usage of all pods on this node to identify the pod with the greatest impact on the resource imbalance.

[0032] Step 3: Collect the status information of all nodes in the current cluster, call the Pod migration program for the Pod to be migrated, and finally obtain a new target node.

[0033] Under the condition that the resource information of all nodes in the cluster and the resource information of the Pod to be migrated have been obtained, this embodiment matches it with the idleness of each resource of the nodes in the cluster according to the similarity formula. The calculation formula is as follows:

[0034]

[0035] Similar(A, B) indicates the similarity between vectors A and B. The range of similar(A, B) is [-1, 1]. The closer the value is to 1, the higher the similarity between the Pod and the node. m indicates that there are m types of resources involved in the calculation.

[0036] A represents the resource consumption of the Pod to be migrated over the past period of time, for example, A = [A1, A2...A m ], where A1 represents the CPU utilization of the Pod to be migrated over the past period of time, A2 represents the memory utilization of the Pod to be migrated over the past period of time, and so on;

[0037] B represents the idle resources of the node to be matched, for example, B = [B1, B2...B m ], where B1 represents the current CPU idleness of the node to be matched, B2 represents the current memory idleness of the node to be matched, and so on.

[0038] If the calculated similarity between A and B is close to 1, it means that this node is very suitable for providing the various dimensions of resources required for the Pod to be migrated to run. Otherwise, it means that this node is not suitable for running the Pod to be migrated.

[0039] Calculate the performance coefficient of each node. The reference indicators of the node's performance coefficient include the node's hardware performance and the node's current comprehensive resource idleness. When the comprehensive resource utilization of the nodes is the same, the node with higher hardware performance has a larger coefficient, indicating that the node currently has stronger performance; similarly, when the hardware performance of the nodes is the same, the node with higher comprehensive resource idleness has a larger coefficient, indicating that the node currently has better performance. The calculation formula is as follows:

[0040] Assume that the set of nodes in the cluster is S. First, calculate the performance coefficients of all nodes in S. This embodiment designs a calculation formula for the node performance coefficient as follows:

[0041] N(i)=H(i)*F AVR (i)

[0042] Among them, N(i) represents the performance coefficient of node i, H(i) represents the hardware performance reference score of node i, and F AVR (i) represents the comprehensive resource idleness of node i.

[0043]

[0044] Among them, node(i, p) represents the value of hardware resource p on node i, Indicates the maximum value of hardware resource p in S, where P includes but is not limited to the number of CPU cores, CPU frequency, memory capacity, disk capacity, and network bandwidth.

[0045]

[0046] Where F(i, r) represents the idleness of resource r in node i. m indicates that there are m types of resources involved in the calculation, and R includes but is not limited to CPU, memory, network I / O, and disk I / O.

[0047] The final score is obtained by calculating the similarity of each node and the node performance coefficient. The calculation formula is as follows:

[0048] New_Score(i)=N(i)*similar(A,B)

[0049] Here, New_Score(i) represents the score of the node to be matched, N(i) represents the node's performance coefficient, and similar(A,B), as above, represents the resource similarity between the pod and the node to be matched. For example, if node a has a performance coefficient of 4 and a resource similarity of 0.8, then node a's score is 3.2; if node b has a performance coefficient of 5 and a resource similarity of 0.6, then node b's score is 3, making node a more suitable than node b.

[0050] Step 4: Create a Pod on the new target node first, then delete the original Pod. The Pod is now successfully migrated to the new target node and is running. Return to step 1.

[0051] After scoring, the node with the highest score becomes the new target node. The migration program then scales the Pod replicas on the new target node by calling the kubectlscale command with the previously determined target node as a parameter. Once the Pod is created and running, a status code is returned. If everything is normal, the old Pod is deleted by calling the kubectl delete command. At this point, the Pod migration is complete, and the program returns to monitoring cluster resource imbalance.

[0052] This embodiment adopts the method of creating first and then deleting the migrated Pod, which can effectively prevent the problem of the new Pod not being successfully created while the old Pod has been deleted during the Pod migration process, thereby avoiding the risk of the corresponding request not being processed and the request not being responded to in time.

[0053] The above is a schematic description of the present invention and its embodiments, which is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. Therefore, if a person skilled in the art is inspired by this and, without departing from the purpose of the present invention, designs a structure and embodiment similar to this technical solution without inventiveness, they shall fall within the scope of protection of the present invention.

Claims

1. A Pod migration method under a Kubernetes cluster architecture system, characterized in that: The steps are: Step 1: Continuously monitor the resource imbalance of the cluster to determine whether it exceeds the threshold. If so, proceed to step 2; if not, continue monitoring. Step 2: Find the node with severe resource imbalance and the pod on this node that has the greatest impact on resource imbalance. Determine the pod to be migrated. Step 3: Collect the status information of all nodes in the current cluster, call the Pod migration program for the Pod to be migrated, and finally obtain a new target node; Step 4: Create a Pod on the new target node first, then delete the original Pod. The Pod is now successfully migrated to the new target node and is running. Return to step 1.

2. The Pod migration method under a Kubernetes cluster architecture system according to claim 1, characterized in that: Step 1: Set up an independent background monitoring program to monitor the resource usage of all nodes in the Kubernetes cluster and various status information of components in the Kubernetes cluster.

3. The Pod migration method under a Kubernetes cluster architecture system according to claim 1 or 2, characterized in that: Step 1: Use the cluster resource imbalance calculation formula to determine the cluster resource imbalance status. The formula is as follows: Where B represents the cluster resource imbalance, C represents the number of nodes in the cluster, and std(i) represents the standard deviation of the resource utilization of node i.

4. The Pod migration method under a Kubernetes cluster architecture system according to claim 3, characterized in that: The node with the severe resource imbalance in step 2 is the node with the largest standard deviation of resource utilization.

5. The Pod migration method under a Kubernetes cluster architecture system according to claim 4, characterized in that: Step 3: Use the similarity formula to match the Pod to be migrated with the idleness of each resource on the cluster node. The calculation formula is as follows: Here, similar(A, B) represents the similarity between vectors A and B. A represents the resource consumption of the Pod to be migrated over the past period of time, and B represents the resource availability of the node to be matched.

6. The Pod migration method under a Kubernetes cluster architecture system according to claim 5, characterized in that: Step 3 calculates the performance coefficient of each node. The reference indicators of the performance coefficient of the node include the hardware performance of the node and the current comprehensive resource idleness of the node.

7. The Pod migration method under a Kubernetes cluster architecture system according to claim 6, characterized in that: The performance coefficient of a node is calculated as follows: N(i)=H(i)*F AVR (i) Among them, N(i) represents the performance coefficient of node i, H(i) represents the hardware performance reference score of node i, and F AVR (i) represents the comprehensive resource idleness of node i.

8. The Pod migration method under a Kubernetes cluster architecture system according to claim 7, characterized in that: The calculation formula for the hardware performance reference score of node i is as follows: Among them, node(i, p) represents the value of hardware resource p on node i, Indicates the maximum value of hardware resource p in the node set S.

9. The Pod migration method under a Kubernetes cluster architecture system according to claim 8, characterized in that: The calculation formula for the comprehensive resource idleness of node i is as follows: Among them, F(i, r) represents the idleness of resource r in node i, and m represents the number of resources involved in the calculation.

10. The Pod migration method under a Kubernetes cluster architecture system according to claim 9, characterized in that: Step 3: Calculate the similarity of each node and the node performance coefficient to obtain the final score. The calculation formula is as follows: New_Score(i)=N(i)*similar(A,B) The node with the highest score becomes the new target node.

Images