Container cloud scheduling system and method based on intermittent network environment

Abstract

The application provides a container cloud scheduling system based on an intermittent network environment. A node resource evaluation module is included, which determines a node resource remaining available score based on node characteristics and node load; wherein the node characteristics are determined by CPU frequency S(Ci), memory capacity S(Mi) and network bandwidth S(Ni) of the node; and the node load is determined by CPU utilization L(Ci), memory utilization L(Mi) and network bandwidth utilization L(Ni) of the node. A node balance factor module is included, which determines a comprehensive score based on the node resource remaining available score and a balance score; wherein the balance score is determined by CPU frequency balance degree, memory capacity balance degree and network bandwidth balance degree of the node; and the CPU frequency balance degree is determined by CPU frequency utilization average Lavg(C), the memory capacity balance degree is determined by memory capacity utilization average Lavg(M), and the network bandwidth balance degree is determined by network bandwidth utilization average Lavg(N). A scheduling decision module is included, which specifies a scheduling strategy based on the comprehensive score.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of resource scheduling, in particular to a container cloud scheduling system and method based on an intermittent network environment. BACKGROUND

[0002] With the continuous deepening of information revolution, information confrontation has gradually replaced the traditional confrontation model, and the form of confrontation has also transitioned from the past single combat form to integrated joint confrontation form. However, the past single confrontation model has been unable to adapt to the needs of current integrated joint confrontation. Not only must it collect confrontation information in multiple fields such as land, sea, space, aerospace, and network, but it also needs to effectively fuse these information. In the confrontation network, the distribution range of low-level confrontation nodes is generally within tens of square kilometers. Due to the large number of communication nodes, the electromagnetic environment and geographical environment where they are located are complex and changeable, and the interference is serious. Due to the communication ability, channel resources, and interference, the network exists in a situation of intermittent and good or bad.

[0003] “Tactical cloud” is a temporary network system that does not rely on permanent computer network facilities, but is temporarily constructed under the requirements of harsh confrontation environment conditions. It refers to a cloud computing environment that realizes computing technology and information service capability for a large number of terminal users in the harsh communication environment conditions of no network connection, intermittent, low bandwidth, and the highly dynamic, high mobility, strong competition, and limited network information technology resources of the military tactical front natural environment conditions between network nodes. It is mainly used to handle the situation problem that the cloud service can be quickly, smoothly, and effectively carried out under the constraints of the mobile environment.

[0004] Current container cloud scheduling has the characteristics of lightweight, millisecond-level startup, and anti-destroying. The container scheduler is not only an important driving force for the popularization of container technology, but also a key factor that determines the performance and resource utilization of the cluster. There are three major mainstream container orchestration engines including Docker Swarm, Apache Mesos, and Google Kubernetes. Among them, the most widely used is the lightweight open source and powerful Google Kubernetes. Several scheduling models and algorithms have their own advantages and disadvantages. Users can select the corresponding container scheduling framework to build their container cloud environment according to their own needs to achieve better resource allocation and scheduling.

[0005] The container cloud Google Kubernetes core scheduling algorithm, such as LeastRequestedPriority, BalancedResourceAllocation and the like, only considers the evaluation indexes of two dimensions of CPU and memory, and is easy to cause overload of other dimension resources, thereby causing low cluster resource utilization. The intermittent network instability factor is not effectively considered, and the scheduling system cannot be effectively applied to the harsh communication environment conditions and the highly dynamic, high mobility, strong competition and limited network information technology resources of the front natural environment conditions. SUMMARY

[0006] In view of the above problems, the purpose of the present application is to provide a container cloud scheduling system based on intermittent network environment, which is suitable for the scheduling of confrontation tasks in harsh network environment and has better load balancing performance.

[0007] In the first aspect, the present application provides a container cloud scheduling system based on intermittent network environment, which comprises a node resource evaluation module, a node resource balance factor module and a scheduling decision module. The node resource evaluation module determines the residual available score of node resources based on node characteristics and node load. The node characteristics are determined by the CPU frequency S(Ci), memory capacity S(Mi) and network bandwidth S(Ni) of the node. The node load is determined by the CPU utilization L(Ci), memory utilization L(Mi) and network bandwidth utilization L(Ni) of the node. The node resource balance factor module determines the comprehensive score based on the residual available score of node resources and the balance score. The balance score is determined by the CPU frequency balance degree, memory capacity balance degree and network bandwidth balance degree of the node. The CPU frequency balance degree is determined by the CPU frequency utilization average Lavg(C), the memory capacity balance degree is determined by the memory capacity utilization average Lavg(M), and the network bandwidth balance degree is determined by the network bandwidth utilization average Lavg(N). The scheduling decision module specifies the scheduling strategy based on the comprehensive score.

[0008] Compared with the prior art, the container cloud scheduling system based on the intermittent network environment provided by the application increases the network evaluation factor, comprehensively considers the cluster node CPU, memory and network bandwidth factors, better adapts to the application environment, and thus improves the load balancing efficiency of the cluster. In addition, the node balancing factor is used to reflect the balancing of each dimension of each node, so as to avoid the condition that the single-dimensional factor utilization rate is relatively high and the score is still relatively high. For the balance and stability of the network factor, the corresponding effect is played. Furthermore, in the conventional container cloud algorithm, the resource weight parameters of the node screening algorithm are usually set by human subjective experience. Due to the difference between people's general understanding and application scenarios, the inaccurate and unreasonable weight parameter setting will cause the imbalance of the cluster load and the low resource utilization rate. The application sets the weight of each node factor by using the fuzzy analytic hierarchy process combining subjective and objective factors.

[0009] In a second aspect, the application provides a container cloud scheduling method based on an intermittent network environment, comprising the following steps:

[0010] S10. Determine the node resource remaining available score based on the node characteristics and the node load, wherein the node characteristics are determined by the CPU frequency S(Ci) of the node, the memory capacity S(Mi) and the network bandwidth S(Ni); and the node load is determined by the CPU utilization rate L(Ci) of the node, the memory utilization rate L(Mi) and the network bandwidth utilization rate L(Ni);

[0011] S11. Determine the comprehensive score based on the node resource remaining available score and the balance score, wherein the balance score is determined by the CPU frequency balance degree, the memory capacity balance degree and the network bandwidth balance degree of the node; the CPU frequency balance degree is determined by the CPU frequency utilization rate average Lavg(C), the memory capacity balance degree is determined by the memory capacity utilization rate average Lavg(M), and the network bandwidth balance degree is determined by the network bandwidth utilization rate average Lavg(N);

[0012] S12. Assign a scheduling strategy based on the comprehensive score.

[0013] The beneficial effects of the container cloud scheduling method based on the intermittent network environment provided by the application are the same as those of the container cloud scheduling system based on the intermittent network environment provided by the first aspect of the application and / or any one of the implementation manners of the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings described below only illustrate some of the embodiments of the present application, and other drawings can be obtained by those of ordinary skill in the art without any creative effort based on these drawings.

[0015] Figure 1 The mechanism of the container cloud technical solution provided by the embodiments of the present application;

[0016] Figure 2 The structure diagram of the container cloud scheduling system based on an intermittent network environment provided by the embodiments of the present application;

[0017] Figure 3 The multi-level hierarchical structure model provided by the embodiments of the present application;

[0018] Figure 4 The flowchart of the container cloud scheduling method based on an intermittent network environment provided by the embodiments of the present application. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without any creative effort belong to the scope of protection of the present application.

[0020] In a first aspect, referring to Figures 1 to 3 The purpose of the present application is to provide a container cloud scheduling system based on an intermittent network environment, which comprises a node resource evaluation module, the node resource evaluation module determines the node resource remaining available score based on the node characteristics and the node load. The node characteristics are determined by the CPU frequency S(Ci), the memory capacity S(Mi) and the network bandwidth S(Ni) of the node. The node load is determined by the CPU utilization L(Ci), the memory utilization L(Mi) and the network bandwidth utilization L(Ni) of the node. A node balance factor module, the node balance factor module determines the comprehensive score based on the node resource remaining available score and the balance score. The balance score is determined by the CPU frequency balance degree, the memory capacity balance degree and the network bandwidth balance degree of the node. The CPU frequency balance degree is determined by the CPU frequency utilization average Lavg(C), the memory capacity balance degree is determined by the memory capacity utilization average Lavg(M), and the network bandwidth balance degree is determined by the network bandwidth utilization average Lavg(N). A scheduling decision module, the scheduling decision module specifies the scheduling strategy based on the comprehensive score.

[0021] In order to reasonably schedule tasks to improve the load balancing efficiency of the whole cluster, reduce the influence on the smooth completion of tasks due to the failure of a single node. At present, network factors have become an index that cannot be ignored in the combat scene.

[0022] The characteristics of the node and the load of the node must be considered in the scheduling process to achieve the effect of task cluster load balancing and full play of node resources. The characteristics of the node, i.e. the static characteristics of the node, indicate the information computing and processing capacity of the node, and the value does not change with the growth of time. The characteristics S(ni) of node ni are calculated as follows:

[0023] S(n i )=k1*m*S(C i )+k2*S(M i )+k3*S(N i ) (1-1)

[0024] Where m represents the number of CPU cores of node ni, S(Ci) represents the CPU frequency of node ni, S(Mi) represents the memory capacity of node ni, and S(Ni) represents the network bandwidth of node ni. K1, K2, and K3 are weights.

[0025] The load of the node is a dynamic attribute of the node, which describes the resource utilization of the node and will change with the type and duration of the task. The comprehensive load statistics rule of the node includes:

[0026] L(n i )=k1*L(C i )+k2*L(M i )+k3*L(N i ) (1-2)

[0027] Where L(Ci) represents the CPU utilization of node ni, L(Mi) represents the memory utilization of node ni, and L(Ni) represents the network bandwidth utilization of node ni. K1, K2, and K3 are weights.

[0028] The remaining available score of the node resource of node n is calculated as follows:

[0029]

[0030] The weights K1, K2, and K3 are determined by the analytic hierarchy process, and the specific steps are as follows:

[0031] S100. Construct a fuzzy pair comparison matrix, form a multi-level hierarchical structure modeling by considering the mutual relationship between various factors in the determination system and all elements of the decision problem, and the CPU, memory and network bandwidth of the cluster are the factors affecting the container application scheduling decision.

[0032] As Figure 2 shown, the above multi-level hierarchical structure model can be divided into three layers, wherein the target layer is the target node that meets the scheduling requirements after filtering through the budget stage. The criterion layer is the factor that affects scheduling, including node CPU, memory and network bandwidth. The scheme layer is the node that meets the container resource requirements, and finally the node with the highest score is selected as the optimal scheduling node through sorting.

[0033] In the fuzzy analytic hierarchy process, the importance relationship table is shown in Table 1-1. The relative importance of the factors affecting the decision is compared pairwise and represented by 1-9. The larger the value, the greater the influence of the factor on the decision target relative to another factor, and the higher the importance. The fuzzy pair comparison matrix is shown in (1-12), and the element value is shown in (1-13).

[0034]

[0035]

[0036] Table 1-1 Importance Relationship Table

[0037]

[0038]

[0039] S101. Construct an alpha-cut set matrix and convert it to a judgment matrix, construct a fuzzy pair comparison matrix, and convert the fuzzy numbers to triangular fuzzy numbers of three-tuples. Given alpha, convert the three-tuples to alpha-cut set M α , thereby constructing an alpha-cut set matrix P α , the elements in the cut set matrix are

[0040]

[0041] Given an optimization parameter representing the degree of optimization, the binary tuple cut set matrix can be converted to a judgment matrix A, which is a square matrix. Then solve the eigenvalues and eigenvectors of the judgment matrix. Given an optimization parameter λ ∈ [0, 1], λ reflects the upper and lower bounds after the fuzzy given alpha, λ = 1 takes the upper bound value, and λ = 0 takes the lower bound value. Convert the upper and lower bound ranges in the cut set matrix to single values in the judgment matrix, and the elements in the judgment matrix are By giving alpha, the binary tuple cut set matrix is obtained. By giving the cut set matrix an optimization parameter λ, the final judgment matrix A can be obtained.

[0042]

[0043] S102. Determine the maximum eigenvalue and eigenvector of the matrix

[0044] The solution of the eigenvalue problem of the judgment matrix level variable is a pure mathematical problem. The main methods for solving the eigenvalue and eigenvector problem of a square matrix include the geometric mean method (square root method), the arithmetic mean method (sum method), the least square method (power method), and the eigenvector method. The eigenvalues obtained by the above methods are approximately equal, but there are slight differences, which may affect the actual production. Generally, the geometric mean method and the arithmetic mean method have poor accuracy, and the least square method has high computational complexity. Therefore, the eigenvector method is used to solve the eigenvalue and eigenvector of the judgment matrix.

[0045] S103. The consistency of the matrix is checked and the weight is normalized. The uniformity of the fuzzy judgment matrix is particularly important when establishing the fuzzy judgment matrix, which reflects the uniformity of people's thinking and judgment of things. However, in actual practice, due to the complexity of objective things and the one-sidedness of people's cognition, the judgment matrix constructed by people often does not have uniformity. At this time, it is necessary to adjust it to meet the requirements of uniformity. We can check the uniformity of the matrix through CI, RI and CR, where CI is the uniformity index, and the greater the CI value, the more serious the inconsistency of the matrix A, thereby proving that CI = 0 is a sufficient and necessary condition for the matrix A to be a uniform matrix.

[0046]

[0047] where λ max is the maximum eigenvalue of the judgment matrix, n is the order of the judgment matrix, and is the total amount of influence factors. In addition, the random statistical consistency index RI is used to check whether the judgment matrix A has satisfactory consistency. The calculation method of RI is as follows:

[0048]

[0049] The random consistency index can be obtained by looking up the table, as given in Table 1-2.

[0050] Table 1-2 Random matrix of different orders and average consistency index RI value

[0051] n 1 2 3 4 5 6 7 8 RI 0 0 0.52 0.89 1.11 1.25 1.35 1.40

[0052] CR is the random consistency ratio. When CR < 0.1, it is generally assumed that the inconsistency degree of the matrix A is within the allowable range, and the eigenvector can be used as the weight vector.

[0053]

[0054] Finally, the judgment random consistency ratio CR is used to determine whether it meets the requirement of being less than 0.1, and the maximum characteristic vector normalization is used to obtain the weight coefficient.

[0055] In the node balance factor module, the resources of each dimension are scored according to their utilization and the importance of the resources, which may lead to the occurrence of unbalanced utilization of resources of each dimension. When the bandwidth utilization of a certain node is relatively large, and the utilization of the other two aspects of computation and memory is relatively low, although the weight factor of the network bandwidth will be reduced, the overall score of the node is still relatively high. If the task with high bandwidth utilization is continuously scheduled to this node, the performance of the node will decrease, and the phenomenon of unbalanced resource consumption will occur. Therefore, the node balance factor module is used to reflect the balance of each dimension of each node, so as to avoid the situation that the score is high although the utilization of a single dimension is high.

[0056] The CPU frequency utilization rate average, the memory capacity utilization rate average, and the network bandwidth utilization rate average are determined in the following manner:

[0057]

[0058]

[0059]

[0060] The CPU frequency balance degree, the memory capacity balance degree, and the network bandwidth balance degree are determined in the following manner:

[0061] (1-|L(C i )-L avg (C)|) (1-14)

[0062] (1-|L(M i )-L avg (M)|) (1-15)

[0063] (1-|L(N i )-L avg (N)|) (1-16)

[0064] The larger the value is, the more balanced the utilization of resources of each dimension is, and the higher the score of the node is. The final score of the node balance module is:

[0065]

[0066] The entire scoring algorithm comprehensively considers the CPU, memory, and network bandwidth factors, and the score is composed of the remaining available score of the node resources and the balance score. The final score of a single node is as follows.

[0067] score(i) = 10 * [available(i) + balance(i)] (1-18) Load balance degree LBD is a key technical index reflecting the current load balancing state of the network cluster, and the LBD evaluation system is constructed according to the load evaluation measurement parameters of each node and the projection state thereof on the phase structure, so as to reflect the current load condition of the network cluster by evaluating the mapping state of the load evaluation measurement parameters (CPU usage rate, storage efficiency and network bandwidth usage rate) of the node on the phase space and the concentration degree of the cluster center of gravity on the phase space.

[0068] The number of nodes in the cluster is defined as n, and the number of node load evaluation parameters is m. At t, the node set projected by the node load evaluation parameters in the m-dimensional phase space is U = {x i1 ,x i2 ,…,x im}, wherein 1≤i≤n. The center position of the node load projection is defined as G (X1, X2, …, X m ), and the corresponding center calculation formula is:

[0069]

[0070] The center position reflects the total load condition of the cluster at t, and the average value of the load projection of each node to the center distance can be determined by measurement, so as to determine the LBD of the cluster at t, and the calculation formula is as follows:

[0071]

[0072] When LBD = 0, if the total load of all working nodes in the group is equal, the load projection points will be gathered in a point, and the overall load balancing is the most ideal. When LBD = 1, since the full load and empty load working nodes in the group each occupy half, and the load projection is farthest from the middle position, therefore, the group is in a poor load balancing state. Therefore, the smaller the value is, the better the load balancing of the group is.

[0073] The container cloud scheduling mechanism based on the intermittent network environment is verified by experiment test, and compared with the conventional container cloud, the effect is better, and the load balancing performance is better.

[0074] Referring to Figure 4 , in a second aspect, the application provides a container cloud scheduling method based on an intermittent network environment, comprising the following steps:

[0075] S10. determining a node resource remaining available score based on node characteristics and node load, wherein the node characteristics are determined by CPU frequency S(Ci), memory capacity S(Mi) and network bandwidth S(Ni) of the node; and the node load is determined by CPU utilization L(Ci), memory utilization L(Mi) and network bandwidth utilization L(Ni) of the node;

[0076] S11. determining a comprehensive score based on the node resource remaining available score and a balance score, wherein the balance score is determined by CPU frequency balance degree, memory capacity balance degree and network bandwidth balance degree of the node; and the CPU frequency balance degree is determined by CPU frequency utilization average Lavg(C), the memory capacity balance degree is determined by memory capacity utilization average Lavg(M), and the network bandwidth balance degree is determined by network bandwidth utilization average Lavg(N);

[0077] S12. assigning a scheduling strategy based on the comprehensive score.

[0078] The method has the same beneficial effects as the scheduling system of the first aspect and / or any one of the implementation manners of the first aspect of the present application, and thus will not be described here.

[0079] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other.

[0080] The principles and implementation manners of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation manner and application range of the present application can be changed according to the idea of the present application. In conclusion, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A container cloud scheduling system based on an intermittent network environment, characterized in that, include: A node resource evaluation module determines the remaining usable score of node resources based on node characteristics and node load; wherein, the node characteristics are determined in the following way: S(n i )=k1*m*S(C i )+k2*S(M i )+k3*S(N i ); where m represents the number of CPU cores in node ni, S(Ci) represents the CPU frequency of node ni, S(Mi) represents the memory capacity of node ni, S(Ni) represents the network bandwidth of node ni, and K1, K2, and K3 are weights; the node load is determined in the following way: L(n i )=k1*L(C i )+k2*L(M i )+k3*L(N i Where L(Ci) represents the CPU utilization of node ni, L(Mi) represents the memory utilization of node ni, L(Ni) represents the network bandwidth utilization of node ni, and K1, K2, and K3 are weights; the remaining usable score of node resources is determined in the following way: The node balancing factor module determines a comprehensive score based on the node's remaining usable resource score and balancing score. The balancing score is determined by the node's CPU frequency balancing, memory capacity balancing, and network bandwidth balancing. The CPU frequency balancing is determined by the average CPU frequency utilization L. avg (C) It is determined that the memory capacity balance is determined by the average memory capacity utilization rate L. avg (M) is determined by the average network bandwidth utilization rate L. avg (N) Determined; the average CPU frequency utilization, average memory capacity utilization, and average network bandwidth utilization are determined in the following ways: The CPU frequency balance, memory capacity balance, and network bandwidth balance are determined in the following way: (1-|L(C i )-L avg (C)|), (1-|L(M) i )-L avg (M)|), (1-|L(N) i )-L avg (N)|); The balance score is determined in the following way: The overall score is determined as follows: score(i) = 10 * [available(i) + balance(i)]; The scheduling decision module specifies a scheduling strategy based on the comprehensive score.

2. The container cloud scheduling system based on an intermittent network environment according to claim 1, characterized in that, Determine the weights K1, K2, and K3 using the analytic hierarchy process: S100. Construct a fuzzy contrast matrix by modeling the interrelationships between various factors in the system and forming a multi-level hierarchical structure for all elements of the decision problem. The CPU, memory, and network bandwidth of the nodes in the cluster are considered as factors influencing the scheduling decision of container applications. The fuzzy contrast matrix is ​​as follows: S101. Construct an α-cut set matrix and convert it into a judgment matrix. After constructing a fuzzy contrast matrix, convert the fuzzy numbers into triangular fuzzy numbers of triples. Given α, convert the triples into α-cut sets Mα, thus constructing an α-cut set matrix Pα. The elements of the cut set matrix are... Given an optimization parameter representing the degree of optimization, the binary cut-set matrix can be transformed into a judgment matrix A, which is a square matrix. Then, the eigenvalues ​​and eigenvectors of the judgment matrix are solved. Let the given optimization parameter λ∈[0,1], where λ reflects the upper and lower bounds after a fuzzy assignment of α. λ=1 represents the upper bound, and λ=0 represents the lower bound. The upper and lower bound ranges in the cut-set matrix are transformed into single values ​​in the judgment matrix. The elements of the judgment matrix... Given α, we obtain the cut set matrix of the binary tuple. Given an optimization parameter λ to the cut set matrix, we can obtain the final judgment matrix A. S102. Determine the largest eigenvalue and eigenvector of the matrix by calculation, and use the eigenvector method to solve for the largest eigenvalue and eigenvector of the judgment matrix A; S103. Perform consistency checks and normalize weights on the matrix. The consistency of the matrix is ​​checked by CI, RI and CR, where CI represents the consistency index, RI represents the random statistical consistency index and CR represents the random consistency ratio.

3. A container cloud scheduling method based on an intermittent network environment, implemented according to the container cloud scheduling system based on an intermittent network environment as described in claim 1, characterized in that, Includes the following steps: S10. Determine the remaining usable score of node resources based on node characteristics and node load, wherein the node characteristics are determined by the node's CPU frequency S(Ci), memory capacity S(Mi), and network bandwidth S(Ni); and the node load is determined by the node's CPU utilization L(Ci), memory utilization L(Mi), and network bandwidth utilization L(Ni). S11. Determine a comprehensive score based on the remaining usable resource score and the balance score of the node, wherein the balance score is determined by the CPU frequency balance, memory capacity balance, and network bandwidth balance of the node; and the CPU frequency balance is determined by the average CPU frequency utilization L. avg (C) It is determined that the memory capacity balance is determined by the average memory capacity utilization rate L. avg (M) is determined by the average network bandwidth utilization rate L. avg (N) Determined; S12. Specify the scheduling strategy based on the comprehensive score.

Images