Containerized RDMA application migration method based on load evaluation

Abstract

The invention discloses a containerized RDMA (Remote Direct Memory Access) application migration method based on load evaluation, which comprises the following steps: step 1, a load evaluator divides a node cluster of which the load is in a normal range by quantifying the resource load state of each node in a cluster and through a multi-threshold algorithm, and further screens migratable nodes in combination with the resource demand of container operation, so as to obtain a node cluster of which the load is in a normal range; 2, a mirror image pre-downloading device is responsible for pre-downloading a container mirror image on a target node so as to reduce delay in the migration process, and 3, a migration executor loads and executes real-time migration of the container. In the migration process, the recovery of the RDMA connection is quicker, the dependence on a special network card is avoided, the latest version mirror image can be predicted and deployed in the migrated target node in advance, the data transmission efficiency is improved, the migration delay is reduced, the application program is more flexible and extensible, the application program can be quickly deployed and adjusted according to actual requirements, and the application program migration efficiency is improved. Therefore, the method adapts to continuously changing service requirements and traffic loads.

Description

Technical Field

[0001] The present invention relates to the field of container migration of virtualization technology, and in particular to a containerized RDMA application migration method based on load evaluation. Background Art

[0002] With the development of cloud computing and big data technologies, large-scale computing service scenarios such as data centers have an increasing demand for high-performance communications and stable and scalable computing services. RDMA (Remote Direct Memory Access) technology is widely used in high-performance computing and large-scale data storage and processing scenarios due to its high data transmission speed and low latency. Containerization technology is widely used in application deployment, improving application portability, elasticity and security due to its lightweight virtualization characteristics.

[0003] At present, the mainstream solution to the migration problem of containerized RDMA applications basically relies on RDMA dedicated network cards. The process is to suspend the RDMA connection first, migrate the container, and then map the network card to the node by registering MR (Memory Region), so that the network card can access the data at the application layer through MR, and then re-establish the RDMA connection on the new computing node, and then resume from the initialization state. However, the opacity of this migration will lead to interruption of data transmission, thereby affecting the performance of the application, increasing the difficulty of application deployment and maintenance, and failing to meet user needs in a timely manner in some load imbalance scenarios, affecting the continuity, convenience, flexibility and efficiency of actual business. In addition, in actual applications, with the rapid development of DevOps technology, the speed and frequency of application version updates have increased rapidly. In some Internet companies, the frequency of application releases 10 times a day has become a basic standard configuration. Frequent image updates will cause each container migration to need to pull the new version of the image from the registry warehouse again, and then restore the original state through checkpoint i nt. This is limited by the influence of network bandwidth and image size, and will affect the container migration time when resources such as network bandwidth are tight.

[0004] The existing technologies for implementing containerized RDMA application migration have problems such as reliance on dedicated hardware, long migration delay, and lack of transparency. These problems limit the flexibility and scalability of containerized RDMA applications.

[0005] Aiming at the shortcomings of the existing virtualization technology in the container migration process, a solution is proposed. Summary of the invention

[0006] The purpose of the present invention is to design a faster container migration solution, which integrates RDMA hot migration technology and load evaluation, pre-downloads images on the node to be migrated, and reduces the time of containerized RDMA application migration.

[0007] The object of the present invention can be achieved by the following technical solutions: A containerized RDMA application migration method based on load evaluation, step 1: the load evaluator quantifies the resource load status of each node in the cluster, and divides the node cluster whose load is in a normal range through a multi-threshold algorithm, and further screens the nodes that can be migrated in combination with the resource requirements of the container operation;

[0008] Step 2: The image pre-downloader is responsible for pre-downloading the container image on the target node to reduce the delay during the migration process;

[0009] Step 3: The migration executor performs real-time migration of the container, including checkpoint creation, data transfer, and container recovery.

[0010] Preferably, the load evaluator comprises:

[0011] Load evaluation model: Each node to be migrated J has three indicators, CPU configuration z J1 , memory configuration J2 and network I / O configuration J3 , the weights corresponding to each indicator are k1, k2, k3, where k1+k2+k3=1. To evaluate and compare different heterogeneous nodes, we set a standard node N st , its CPU configuration is z st1 , memory utilization z st2 , network I / O utilization z st3 , the resource allocation ratio of any node to the standard node is the weight ratio of the corresponding indicator as follows:

[0012]

[0013] The load value of node J is:

[0014] L J =k1*n J1 +k2*n J2 +k3*n J3

[0015] Among them, n J1 represents the CPU utilization of the node J to be migrated, n J2 Represents the memory utilization of the node J to be migrated, n J3 Represents the network utilization rate of the node J to be migrated;

[0016] Multi-threshold algorithm: Set two thresholds for node load, namely the low threshold Th1 and the high threshold Th2. Set Th1 to 10% and Th2 to 90%. Determine the load value L of node J.J The size relationship between the two thresholds of the node load determines whether the node is suitable as a candidate for migration.

[0017] Preferably, the load value L of the judgment node J J The specific relationship between the two thresholds of node load is:

[0018] If L J If it is less than Th1, it means that the node J has a small load and is not suitable as a candidate for migration;

[0019] If L J If it is between Th1 and Th2, it means that the load of node J is normal and it can be used as a candidate node to be migrated;

[0020] If L J If it is greater than Th2, it means that node J is overloaded and is not suitable as a migrated node;

[0021] Select the nodes to be migrated with a load value between Th1 and Th2 as the nodes that can be migrated, which will be represented by j later;

[0022] Based on the resources occupied by the container and the resources available on the node, the similarity y between the target container i and the node j that can be migrated is calculated by combining the Pearson product-moment correlation coefficient:

[0023]

[0024] Among them, x i Represents the resource vector required by the container to be migrated, n j Represents the resource vectors that are being used by the nodes that can be migrated. Each vector contains CPU utilization, memory utilization, and network I / O utilization. i1 represents the CPU utilization required by container i, x i2 represents the memory utilization required by container i, x i3 represents the network utilization required by container i, Represents the CPU utilization x required by the container i1 、Memory utilization x i2 and network I / O utilization x i3 The mean value, n j1 represents the CPU utilization of the node j that can be migrated, n j2 represents the memory utilization of the node j that can be migrated, n j3 represents the network utilization rate of node j that can be migrated, represents the average of CPU utilization, memory utilization, and network I / O utilization in the node resources that can be migrated, y(x i ,n j) represents the jth node n that can be migrated j and the i-th container x to be migrated i degree of similarity.

[0025] Preferably, based on the resource situation that the node can provide to meet the resources occupied by the container operation, and the more remaining resources, the more suitable the node is as the first choice for container migration. This is defined as the satisfaction rate of the node suitable for container migration. The satisfaction rate should be:

[0026] satisfy(x i ,n j )=1-y(x i ,n j )

[0027] The containers are sorted according to their satisfaction rates, and the top one or several container nodes are used as the target nodes to be migrated. The load evaluator records the addresses of these target nodes.

[0028] Preferably, in step 2, the optimal target node is selected based on the quantitative result of the load evaluation, and the image corresponding to the container is pulled.

[0029] Preferably, the specific operation of step three is: using CRIU to create a checkpoint on the source node, and generating a container migration instruction, migrating the container-related data from the source node to the target node from which the image has been pulled, and using CRIU on the target node to restore the container state from the checkpoint.

[0030] The beneficial effects of the present invention are as follows:

[0031] 1. The present invention can restore RDMA connections more quickly during the migration process and get rid of the dependence on dedicated network cards. It can predict and deploy the latest version of the image in the target node to be migrated in advance, improve data transmission efficiency and reduce migration latency, make the application more flexible and scalable, and can be quickly deployed and adjusted according to actual needs, so as to adapt to changing business needs and traffic loads;

[0032] 2. The present invention pre-selects the most suitable node for migration in the cluster through a dynamic load evaluation algorithm, and pre-pulls the image version of the corresponding container, saving the inevitable image download part during container migration, thereby accelerating the container migration execution process.

[0033] 3. The present invention uses two parts, image pre-downloading and migration executor, to execute container migration. The container migration time can be stably maintained at the millimeter level, saving the image pulling part of image downloading that takes tens of seconds during the migration process. It is not affected by network fluctuations, etc., and increases the stability of container migration time. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The present invention will be further described below in conjunction with the accompanying drawings;

[0035] Figure 1 It is a schematic diagram of the system architecture of this method.

[0036] Figure 2 is the dynamic load evaluation result of this method.

[0037] Figure 3 This is the delay effect diagram of migrating three containers using this method. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] Example:

[0040] See also Figures 1 to 3 As shown, the present invention is a containerized RDMA application migration method based on load evaluation, step 1: the load evaluator quantifies the resource load status of each node in the cluster, and divides the node cluster whose load is in a normal range through a multi-threshold algorithm, and further screens the nodes that can be migrated in combination with the resource requirements of the container operation. The load evaluator monitors the CPU, memory, and network I / O utilization of each node in the cluster in real time, and collects basic resource usage data of the node system;

[0041] Step 2: The image pre-downloader is responsible for pre-downloading the container image on the target node to reduce the delay during the migration process. The optimal target node is selected based on the quantitative results of the load evaluation. The optimal node refers to satisfy(x i ,n j ) maximum value, if there are two identical satisfy(x i ,n j ), then both nodes download the image corresponding to the image pull container in advance;

[0042] Step 3: The migration executor load performs real-time migration of containers, including checkpoint creation, data transfer, and container recovery. It uses CRIU to create a checkpoint on the source node and generates container migration instructions to migrate container-related data from the source node to the target node where the image has been pulled, and uses CRIU on the target node to restore the container status from the checkpoint.

[0043] Preferably, the load evaluator comprises:

[0044] Load evaluation model: Each node to be migrated J has three indicators, CPU configuration z J1 , memory configuration J2 and network I / O configuration J3 , the weights corresponding to each indicator are k1, k2, k3, where k1+k2+k3=1. To evaluate and compare different heterogeneous nodes, we set a standard node N st , its CPU configuration is z st1 , memory utilization z st2 , network I / O utilization z st3 , the resource allocation ratio of any node to the standard node is the weight ratio of the corresponding indicator as follows:

[0045]

[0046] The load value of node J is:

[0047] L J =k1*n J1 +k2*n J2 +k3*n J3

[0048] Among them, n J1 represents the CPU utilization of the node J to be migrated, n J2 Represents the memory utilization of the node J to be migrated, n J3 represents the network utilization of the node J to be migrated. Multiple nodes J to be migrated are denoted as {J1,J2...J m};

[0049] The multi-threshold algorithm divides the load into the normal range as follows: set two thresholds for node load, namely the low threshold Th1 and the high threshold Th2, set Th1 to 10%, and set Th2 to 90%. More accurate Th1 and Th2 can be adjusted based on the node load data of the system after a period of time to determine the load value L of node J. J The size relationship between the two thresholds of the node load determines whether the node is suitable as a candidate for migration.

[0050] The load value L of the judgment node J J The specific relationship between the two thresholds of node load is:

[0051] If L J If it is less than Th1, it means that the node J has a light load and is not suitable as a candidate for migration. It is reserved for processing sudden or high-priority tasks.

[0052] If L JIf it is between Th1 and Th2, it means that the load of node J is normal and it can be used as a candidate node to be migrated;

[0053] If L J If it is greater than Th2, it means that node J is overloaded and is not suitable as a migrated node;

[0054] Select the nodes to be migrated whose load values ​​are between Th1 and Th2 as the nodes that can be migrated, and then use j to represent them. Obviously, {j1,j2...j m} belongs to {J1,J2...J m};

[0055] Based on the resources occupied by the container and the resources available on the node, the similarity y between the target container i and the node j that can be migrated is calculated by combining the Pearson product-moment correlation coefficient:

[0056]

[0057] Among them, x i Represents the resource vector required by the container to be migrated, n j Represents the resource vectors that are being used by the nodes that can be migrated. Each vector contains CPU utilization, memory utilization, and network I / O utilization. i1 represents the CPU utilization required by container i, x i2 represents the memory utilization required by container i, x i3 represents the network utilization required by container i, Represents the CPU utilization x required by the container i1 、Memory utilization x i2 and network I / O utilization x i3 The mean value, n j1 represents the CPU utilization of the node j that can be migrated, n j2 represents the memory utilization of the node j that can be migrated, n j3 represents the network utilization rate of node j that can be migrated, represents the average of CPU utilization, memory utilization, and network I / O utilization in the node resources that can be migrated, y(x i ,n j ) represents the jth node n that can be migrated j and the i-th container x to be migrated i The absolute value of the similarity must be positive.

[0058] Based on the resource situation that the node can provide to meet the resources occupied by the container operation, and the more remaining resources, the more suitable the node is as the first choice for container migration. This is defined as the satisfaction rate of the node suitable for container migration. The satisfaction rate should be:

[0059] satisfy(x i ,n j )=1-y(x i ,n j )

[0060] The containers are sorted according to their satisfaction rates, and the top one or several container nodes are used as the target nodes to be migrated. The load evaluator records the addresses of these target nodes.

[0061] This method creates 4 nodes based on vmvare, creates a cluster through docker swarm, and uses node 1 as the source node; establishes an rdma connection between nodes 2 and 3, and uses the perftest library to run 64 threads in parallel. The message size sent by each thread is 64KB, that is, keeping nodes 2 and 3 busy; node 4 does not perform any operation, keeping node 4 idle, and selects 3 containers busybox (30.2MB), migatest (214.1MB), and rdma_send (917.9M) as migration targets. Through the dynamic load evaluation algorithm, we get the satisfaction rate of each node as follows Figure 2 As shown, the image pre-download and migration executor parts are based on Figure 2 , execute container migration, and get the result as follows Figure 3 As shown in the figure, we can see that the container migration time can be stably maintained at the millimeter level, saving the image download process, which takes tens of seconds (affected by network bandwidth), and is not affected by network fluctuations, which increases the stability of container migration time.

[0062] During operation, the algorithm first quantifies the resource load status of each node in the cluster through the load evaluator, and selects nodes suitable for container migration. This process includes real-time monitoring of the CPU, memory, network I / O utilization and configuration of each node in the cluster, and collecting basic resource usage data of the node system; quantifying the node load status through the load evaluation model, combining the multi-threshold algorithm to divide the node cluster that can be migrated with a normal load range, and calculating the similarity between the target container i and the node j that can be migrated based on the Pearson product-moment correlation coefficient. According to the satisfaction rate (c i ,n j ) to sort and select the satisfaction rate satisfy(c i ,n j ) is used as the target node and the address of the target node is recorded. Then, the image pre-downloader pre-downloads the container image on the target node to reduce the delay during the migration process. Finally, the migration executor is responsible for performing the real-time migration of the container, including creating checkpoints, data transmission, and container recovery.

[0063] The above formulas are obtained by collecting a large amount of data for software simulation and selecting a formula that is close to the real value. The coefficients in the formula are set by technical personnel in this field according to actual conditions. The above are only preferred specific implementations of the present invention, but the protection scope of the present invention is not limited to this. Any technical personnel familiar with the technical field within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, make equivalent replacement or change, which should be covered within the protection scope of the present invention.

Claims

1. A containerized RDMA application migration method based on load assessment, characterized in that: include: Step 1: The load evaluator quantifies the resource load status of each node in the cluster, and uses a multi-threshold algorithm to divide the node clusters with normal loads. It further screens the nodes that can be migrated based on the resource requirements of the container operation. Step 2: The image pre-downloader is responsible for pre-downloading the container image on the target node to reduce the delay during the migration process; Step 3: The migration executor performs real-time migration of the container, including checkpoint creation, data transfer, and container recovery.

2. The method for migrating containerized RDMA applications based on load assessment according to claim 1, characterized in that: The load evaluator comprises: Load evaluation model: Each node to be migrated J has three indicators, CPU configuration z J1 , memory configuration J2 and network I / O configuration J3 , the weights corresponding to each indicator are k1, k2, k3, where k1+k2+k3=1. To evaluate and compare different heterogeneous nodes, we set a standard node N st , its CPU configuration is z st1 , memory utilization z st2 , network I / O utilization z st3 , the resource allocation ratio of any node to the standard node is the weight ratio of the corresponding indicator as follows: The load value of node J is: L J =k1*n J1 +k2*n J2 +k3*n J3 Among them, n J1 represents the CPU utilization of the node J to be migrated, n J2 Represents the memory utilization of the node J to be migrated, n J3 Represents the network utilization rate of the node J to be migrated; Multi-threshold algorithm: Set two thresholds for node load, namely the low threshold Th1 and the high threshold Th2. Set Th1 to 10% and Th2 to 90%. Determine the load value L of node J. J The size relationship between the two thresholds of the node load determines whether the node is suitable as a candidate for migration.

3. The method for migrating containerized RDMA applications based on load assessment according to claim 1, characterized in that: The load value L of the judgment node J J The specific relationship between the two thresholds of node load is: If L J If it is less than Th1, it means that the node J has a light load and is not suitable as a candidate for migration; If L J If it is between Th1 and Th2, it means that the load of node J is normal and it can be used as a candidate node to be migrated; If L J If it is greater than Th2, it means that node J is overloaded and is not suitable as a migrated node; Select the nodes to be migrated with load values ​​between Th1 and Th2 as the nodes that can be migrated, which are subsequently represented by j; Based on the resources occupied by the container and the resources available on the node, the similarity y between the target container i and the node j that can be migrated is calculated by combining the Pearson product-moment correlation coefficient: Among them, x i Represents the resource vector required by the container to be migrated, n j Represents the resource vectors that are being used by the nodes that can be migrated. Each vector contains CPU utilization, memory utilization, and network I / O utilization. i1 represents the CPU utilization required by container i, x i2 represents the memory utilization required by container i, x i3 represents the network utilization required by container i, Represents the CPU utilization x required by the container i1 、Memory utilization x i2 and network I / O utilization x i3 The mean value, n j1 represents the CPU utilization of the node j that can be migrated, n j2 represents the memory utilization of the node j that can be migrated, n j3 represents the network utilization rate of node j that can be migrated, represents the average of CPU utilization, memory utilization, and network I / O utilization in the node resources that can be migrated, y(x i ,n j ) represents the jth node n that can be migrated j and the i-th container x to be migrated i degree of similarity.

4. The method for containerized RDMA application migration based on load assessment according to claim 3 is characterized in that: Based on the resource situation that the node can provide to meet the resources occupied by the container operation, and the more remaining resources, the more suitable the node is as the first choice for container migration. This is defined as the satisfaction rate of the node suitable for container migration. The satisfaction rate should be: satisfy(x i ,n j )=1-y(x i ,n j ) The containers are sorted according to their satisfaction rates, and the top one or several container nodes are used as the target nodes to be migrated. The load evaluator records the addresses of these target nodes.

5. The method for migrating containerized RDMA applications based on load assessment according to claim 4, characterized in that: In step 2, the optimal target node is selected based on the quantitative results of the load assessment, and the image corresponding to the container is pulled.

6. The method for containerized RDMA application migration based on load assessment according to claim 1, characterized in that: The specific operations of step three are: use CRIU to create a checkpoint on the source node, generate container migration instructions, migrate container-related data from the source node to the target node where the image has been pulled, and use CRIU on the target node to restore the container status from the checkpoint.

Images