A service management method and platform for a service management platform Kubernetes

Abstract

The application relates to the technical field of cloud computing service management, and particularly discloses a service management method for a service management platform Kubernetes, which comprises the following steps: determining service resources configured by each node and resource usage information of each kind of service resource in each node used by each pod; respectively determining first resource occupation information of each pod needing to occupy each kind of service resource in the node based on the running requirement of a service running process; determining a target node by using an analytic hierarchy process according to the type of the service resource, the resource usage information and the first resource occupation information; and after each pod is deployed to the target node to realize service deployment, entering the service running process. The service management method provided by the embodiment of the application enables the Kubernetes to ensure that the pods are distributed to optimal nodes while meeting application requirements and the service is optimally deployed according to multiple factors such as service resource requirement conditions, node resources and node load conditions when the service is deployed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cloud computing service management, and particularly relates to a service management method for a service management platform Kubernetes and the management platform Kubernetes. BACKGROUND

[0002] Kubernetes is a popular service management platform, and a node in Kubernetes is the smallest computing hardware unit in Kubernetes, which is a representation of a single machine in a cluster. Kubernetes does not directly run containers, but encapsulates one or more containers into a Pod. Service management technology is a key technology of the platform, and its goal is to reasonably deploy Pods in the system and allocate node resources to various Pods for use. Therefore, service management technology plays a decisive role in the quality of services carried by the system.

[0003] The native service management system of Kubernetes cannot well guarantee the quality of service. For example, only CPU and memory resources are considered when deploying services, without considering other resource requirements such as network bandwidth; secondly, the scheduling system usually ignores the requirements of services for specific indicators (for example, low delay requirements); the system only determines whether the load of a node is too high according to memory usage, and when the load of a node is too high, only measures are taken to close the Pod that uses more resources and then redeploy it; in addition, when the quality of service decreases, the system also has no measures to control the number of replicas of the service.

[0004] It can be seen that the existing Kubernetes is limited to the influence of CPU and memory resources when deploying services, and cannot ensure that the Pods are allocated to the optimal nodes while meeting the application requirements according to various factors such as service resource demand, node resources, and node load, so as to realize the optimized deployment of services. The service running process cannot adjust the quality of service, node pressure and system resource consumption according to the actual changes of various factors. SUMMARY

[0005] The embodiments of the present application provide a service management method for a service management platform Kubernetes and the management platform Kubernetes, so as to at least solve the problem that the existing Kubernetes cannot realize the optimized deployment of services.

[0006] In a first aspect, a service management method for a service management platform Kubernetes is provided, comprising:

[0007] determining service resource configurations of each node and resource usage information of each service resource in each node used by each pod, the types of the service resources including CPU, memory and network bandwidth;

[0008] determining first resource occupation information of each pod required to occupy each service resource in each node based on operation requirements of a service operation process;

[0009] determining a target node by using an analytic hierarchy process according to the types of the service resources, the resource usage information and the first resource occupation information;

[0010] deploying each pod to the target node to implement service deployment, and then entering the service operation process.

[0011] Optionally, when the resource usage information is resource idle occupation information of each service resource in each node not occupied by each pod, the target node is determined by using the analytic hierarchy process according to the types of the service resources, the resource usage information and the first resource occupation information, including:

[0012] determining relative importance parameters between each service resource in each node according to the first resource occupation information of each pod required to occupy each service resource in each node, the relative importance parameters being ratios of the first resource occupation information of different service resources;

[0013] constructing a first index layer judgment matrix according to the types of the service resources and the relative importance parameters to obtain resource importance weights corresponding to each service resource;

[0014] constructing a first scheme layer judgment matrix according to the resource idle occupation information of each service resource in each node to obtain idle occupation weights of each service resource in each node;

[0015] constructing a first target layer weight matrix according to the resource importance weights and the idle occupation weights to obtain target weight values of each node, and determining a node with a maximum target weight value as the target node.

[0016] Optionally, in the service operation process, the service management method further includes:

[0017] real-time monitoring and storing second resource occupation information of each pod occupying each service resource in each node in the service operation process, and dynamically updating the first index layer judgment matrix according to the second resource occupation information before the pod is deployed to the target node determined before next service deployment.

[0018] Optionally, the Pod types include resource-limited Pods and non-resource-limited Pods; and the resource usage information is resource occupation information of each service resource in the node occupied by each pod; in the service running process, the service management method further comprises:

[0019] In a preset time interval, SLA violation information of the service is continuously obtained, and meanwhile, load information of each node is continuously obtained according to the resource occupation information of each service resource in the node.

[0020] According to the Pod type, the load information, the SLA violation information and the second resource occupation information of each pod to each service resource in the node, it is determined whether the Pod on the node needs to be migrated or the replica is scaled.

[0021] Optionally, the service management method further comprises:

[0022] The pre-warning upper limit value of the resource occupation information is determined.

[0023] The load information of each node is continuously obtained according to the resource occupation information of each service resource in the node, comprising:

[0024] In a preset time interval, the resource occupation information of each service resource in each node is sequentially traversed, and it is determined whether the resource occupation information of any service resource in each node is greater than the pre-warning upper limit value; if yes, the node with the resource occupation information greater than the pre-warning upper limit value is determined as a node with excessive load.

[0025] Optionally, before the determination of whether the Pod on the node needs to be migrated or the replica is scaled, a rated violation threshold of the SLA violation information is determined; and:

[0026] According to the Pod type, the load information, the SLA violation information and the second resource occupation information of each pod to each service resource in the node, it is determined whether the Pod on the node needs to be migrated or the replica is scaled, comprising:

[0027] When the SLA violation information of the service exceeds the rated violation threshold, and the node where the service is located is a node with excessive load, the service resource type, the resource occupation information of each service resource in the node and the second resource occupation information of each pod to each service resource in the node are used to select a pod to be processed by using an analytic hierarchy process.

[0028] According to the Pod type, the load information, the SLA violation information and the second resource occupation information of each pod to each service resource in the node, it is determined whether the Pod on the node needs to be migrated or the replica is scaled.

[0029] Optionally, the service management method further comprises:

[0030] According to the resource occupation information of each service resource in the node, the resource shortage degree parameters between the service resources are determined, and the resource shortage degree parameters are the ratios of the resource occupation information of different service resources in each node.

[0031] According to the type of the service resource and the resource shortage degree parameter, a second index layer judgment matrix is constructed to obtain the resource shortage degree weight corresponding to each service resource in the node.

[0032] According to the second resource occupation information of each pod to each service resource in the node, a second scheme layer judgment matrix is constructed to obtain the resource occupation weight of each pod.

[0033] According to the resource shortage degree weight and the resource occupation weight, a second target layer weight is constructed to obtain the target weight value of each pod, and the pod with the maximum target weight value is determined as the pod to be processed.

[0034] Optionally, the service management method further comprises:

[0035] The second resource occupation information of the pod to be processed to each service resource in the node is determined, and the occupation early warning value is determined.

[0036] According to the type of the pod to be processed and the second resource occupation information of the pod to each service resource in the node, it is determined whether the pod on the node needs to be migrated or the replica is scaled.

[0037] If the pod to be processed is a non-resource-limited pod, and the second resource occupation information of the pod to at least one service resource in the node is greater than the occupation early warning value, the pod to be processed is subjected to replica expansion processing, and the expanded replica is deployed to the target node determined before the next service deployment.

[0038] In the second aspect of the present application, a service management platform Kubernetes is provided, comprising:

[0039] A first determination module is configured to determine the service resources configured by each node and the resource usage information of each service resource in each node used by each pod, and the type of the service resource at least includes CPU, memory and network bandwidth.

[0040] The second determining module is configured to determine first resource occupation information of each pod needing to occupy each service resource in the node based on operation requirements of the service operation process.

[0041] The node screening module is configured to determine the target node by using an analytic hierarchy process according to the type of the service resource, the resource usage information and the first resource occupation information.

[0042] The deployment module is configured to deploy each pod to the target node to implement service deployment, and then enter the service operation process.

[0043] The embodiment of the application has the following beneficial effects:

[0044] Firstly, when the application optimizes service deployment of the pod by using the analytic hierarchy process, the use of the node resource and the demand of the pod for various resources are fully considered, the weight of each type of resource is determined according to the importance of the resource, and finally the target node is screened. The resource weight matrix can be dynamically updated during the system operation process, and the pod is deployed more reasonably. Secondly, the quality of the service is maximally guaranteed during the service operation process, the node load is relieved from two aspects of pod migration and pod replica expansion, the quality of the service is improved, and the waste of resources is reduced by appropriately reducing the number of replicas of the service when the current node load is too light. Therefore, the service management method provided by the embodiment of the application enables the Kubernetes to ensure that the pod is allocated to the optimal node to realize the optimized deployment of the service according to the service resource demand situation, the node resource, the node load situation and other factors when the service is deployed, and to continuously monitor the service quality, the node resource usage situation and other factors during the service operation process, so as to reasonably migrate or control the replica of the pod according to the actual change situation of each factor, so as to achieve the purpose of improving the quality of the service, relieving the node pressure and reducing the consumption of system resources.

[0045] The above description is only a summary of the technical scheme of the application, in order to more clearly understand the technical means of the application, the specific embodiments of the application can be implemented according to the content of the specification, and in order to make the above and other purposes, characteristics and advantages of the application more obvious and easy to understand, the following specific embodiments of the application are described. BRIEF DESCRIPTION OF DRAWINGS

[0046] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not meant to limit the present application. Furthermore, the same reference numerals are used throughout the several drawings to represent similar components. In the drawings:

[0047] Figure 1A flowchart of a service management method for a service management platform Kubernetes is provided for the first embodiment of the present application.

[0048] Figure 2 For Figure 1 A flowchart of implementing step S13 in the first embodiment of the present application is shown in the figure.

[0049] Figure 3 A flowchart of implementing the service running process of the service management method provided for the first embodiment of the present application is shown in the figure.

[0050] Figure 4 For Figure 3 A flowchart of implementing step S22 in the first embodiment of the present application is shown in the figure.

[0051] Figure 5 For Figure 4 A flowchart of implementing step S221 of screening the pods to be processed in the first embodiment of the present application is shown in the figure.

[0052] Figure 6 A structural diagram of a service management platform Kubernetes is provided for the second embodiment of the present application.

[0053] Figure 7 A flowchart of a service management method for a service management platform Kubernetes is provided for the third embodiment of the present application.

[0054] Figure 8 A modeling structural diagram of determining the target node by using the analytic hierarchy process in the third embodiment of the present application is shown in the figure.

[0055] Figure 9 A modeling structural diagram of screening the pods to be processed by using the analytic hierarchy process in the third embodiment of the present application is shown in the figure.

[0056] Figure 10 A structural model diagram of a service management platform is provided for the third embodiment of the present application.

[0057] Figure 11 A custom scheduling structural diagram is provided for the service management method based on a service management platform Kubernetes of the third embodiment of the present application.

[0058] Figure 12 A structural diagram of a Qos controller configured for the service management platform Kubernetes is provided for the third embodiment of the present application.

[0059] In the figure: 1 - first determination module, 2 - second determination module, 3 - node screening module, 4 - deployment module, 5 - pod supervision module, 6 - resource calculation module, 61 - pod screening module, 62 - resource scheduling module, 7 - running supervision module. DETAILED DESCRIPTION

[0060] Exemplary embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be accurately conveyed to those skilled in the art.

[0061] In the first embodiment of the present application, a service management method for a service management platform Kubernetes is provided, comprising the following steps:

[0062] Step S11: Determine the service resources configured by each node and the resource usage information of each service resource in each node used by each pod. The types of service resources include CPU, memory and network bandwidth. To ensure that the service deployment considers not only the demand for CPU and memory resources, but also the influence of network bandwidth resources, and better guarantees the quality of service;

[0063] Step S12: Based on the running requirements of the service running process, determine the first resource occupation information of each pod occupying each service resource in the node, i.e. the resource demand of the pod;

[0064] Step S13: According to the type of service resource, resource usage information and first resource occupation information, determine the target node by using the analytic hierarchy process;

[0065] Step S14: After deploying each pod to the target node to implement service deployment, enter the service running process.

[0066] Therefore, through the above service management process, Kubernetes can ensure that the pod is allocated to the optimal node while meeting the application demand according to the service resource demand, node resources and other factors, realize the optimized deployment of the service, and improve the quality of the service.

[0067] In the embodiment of the present application, when the resource usage information is the resource idle occupation information of each service resource in the node not occupied by each pod, please refer to Figure 2 , in step S13, according to the type of service resource, resource usage information and first resource occupation information, the target node is determined by using the analytic hierarchy process, which comprises the following steps:

[0068] Step S131: According to the first resource occupation information of each pod needing to occupy each service resource in the node, the relative importance parameters between each service resource in each node are determined, and the relative importance parameters are the ratio of the first resource occupation information of different service resources, for example, the CPU resource demand of a certain pod is slightly higher than the importance of memory, and the relative importance parameter value can be set to 2;

[0069] Step S132: According to the type of service resource and the relative importance parameter, a first index layer judgment matrix is constructed to obtain the resource importance weight corresponding to each service resource;

[0070] Step S133: According to the resource idle occupation information of each service resource in each node, a first scheme layer judgment matrix is constructed to obtain the idle occupation weight of each service resource in each node;

[0071] Step S134: According to the resource importance weight and the idle occupation weight, a first target layer weight matrix is constructed to obtain the target weight value of each node, and the node with the maximum target weight value is determined as the target node.

[0072] Specifically, in step S132 of the embodiment of the application, the horizontal and vertical coordinates of the first index layer judgment matrix are set as each service resource, and the matrix element takes the relative importance parameter of the horizontal coordinate representing the resource relative to the vertical coordinate representing the resource. Then the maximum characteristic vector of the first index layer judgment matrix is calculated, and after normalization processing, the resource importance weight corresponding to each service resource is obtained.

[0073] In a preferred embodiment of the application, in step S133, according to the resource idle occupation information of each service resource in each node, the first scheme layer judgment matrix is constructed, and each service resource corresponds to a matrix D, for example, the scheme layer matrix of CPU resource is the horizontal and vertical coordinates of each node (Node), and the matrix element Dij is the CPU idle occupation rate of Nodei / the CPU idle occupation rate of Nodej. Then the maximum characteristic vector of the scheme layer judgment matrix of CPU resource is calculated, and the weight value (i.e. CPU idle occupation weight) of CPU idle occupation in each node Node is obtained by normalization processing. Similarly, the memory matrix, network bandwidth matrix (including uplink bandwidth matrix and downlink bandwidth matrix) are constructed, and the idle occupation weight of each Node for different service resources is calculated.

[0074] The target weight value of each node is obtained by the following formula: Nodei weight value = CPU resource importance weight * Nodei CPU idle occupation weight + memory resource importance weight * Nodei memory idle occupation weight + uplink bandwidth resource importance weight * Nodei uplink bandwidth idle occupation weight + downlink bandwidth resource importance weight * Nodei downlink bandwidth idle occupation weight, wherein the symbol "*" represents multiplication. The target weight values of all nodes are obtained by analogy, and then the node with the maximum score (i.e., the node with the maximum target weight value) is selected as the target node, and each pod is deployed to the target node, so that the optimized deployment of the service is realized. In the conventional AHP, a consistency check is required when the comparison matrix is constructed, but the embodiment of the present application does not have the consistency problem because the real data is used for comparison.

[0075] In a preferred embodiment of the present application, in the service running process, the service management method further comprises the following steps: real-time monitoring and storing second resource occupation information of each pod in the service running process on each service resource in the node, and dynamically updating the first index layer judgment matrix according to the second resource occupation information before the pod is deployed to the target node determined before the next service deployment. That is, the resource usage of the pod is recorded during the system running process, the resource occupation data (i.e., the second resource occupation information) of the pod is monitored in real time, and the resource occupation data of the pod is stored by using the original database Etcd of Kubernetes. When the subsequent pod needs to be scheduled, the first index layer judgment matrix in step S132 is dynamically updated according to the latest resource usage of the pod, so that the pod is deployed more reasonably. For example, the latest resource usage in the recorded data shows that the CPU occupation of the pod is 30% (quantized as 3), and the memory occupation is 20% (quantized as 2), and the relative importance parameter of CPU to memory in the first index layer judgment matrix is 3 / 2, which is rounded up to 2.

[0076] In the embodiment of the present application, the types of the pod include resource-limited pods and non-resource-limited pods. When the resource usage information is the resource occupation information of each service resource in the node occupied by each pod, please refer to Figure 3 In the service running process, the service management method further comprises:

[0077] Step S21: continuously acquiring SLA violation information of the service in a preset time interval, and continuously acquiring load information of each node according to the resource occupation information of each service resource in each node;

[0078] Step S22: determining whether the pods on the node need to be migrated or the replica scaling processing is needed according to the type of the pods, the load information, the SLA violation information and the second resource occupation information of each pod to each service resource in the node.

[0079] Optionally, the service management method provided by the embodiment of the present application further comprises determining a pre-warning upper limit value of the resource occupation information; and in step S21, the load information of each node is continuously acquired according to the resource occupation information of each service resource in each node, and specifically comprising:

[0080] In a preset time interval, the resource occupation information of each service resource in each node is sequentially traversed, and it is determined whether the resource occupation information of the service resource in each node is greater than the pre-warning upper limit value; if yes, the node with the resource occupation information greater than the pre-warning upper limit value is determined as the node with excessive load. For example, the pre-warning upper limit value is set to 90% (the value can be adjusted according to the actual operation, which is not uniquely limited in the present application), and if the resource occupation information of the node is greater than 90%, the node is regarded as the node with excessive load.

[0081] In an optional embodiment of the present application, before step S22 determines whether the pods on the node need to be migrated or the replica scaling processing is needed, the rated violation threshold of the SLA violation information is further determined; and please refer to Figure 4 , step S22 determines whether the pods on the node need to be migrated or the replica scaling processing is needed according to the type of the pods, the load information, the SLA violation information and the second resource occupation information of each pod to each service resource in the node, and specifically comprising:

[0082] Step S221: when the SLA violation information of the service exceeds the rated violation threshold, and the node where the service is located is the node with excessive load, the pods to be processed are screened out by using the analytic hierarchy process according to the type of the service resource, the resource occupation information of each service resource in the node and the second resource occupation information of each pod to each service resource in the node.

[0083] Step S222: determining whether the pods on the node need to be migrated or the replica scaling processing is needed according to the type of the pods to be processed and the second resource occupation information of each pod to each service resource in the node.

[0084] Please refer to Figure 5 , in step S221, the pods to be processed are screened out by using the analytic hierarchy process according to the type of the service resource, the resource occupation information of each service resource in the node and the second resource occupation information of each pod to each service resource in the node, and specifically comprising:

[0085] Step S2211: determining a resource shortage degree parameter between each service resource according to the resource occupied information of each service resource in the node, the resource shortage degree parameter being a ratio of the resource occupied information of different service resources in each node;

[0086] Step S2212: constructing a second index layer judgment matrix according to the type of the service resource and the resource shortage degree parameter, to obtain a resource shortage degree weight corresponding to each service resource in the node;

[0087] Step S2213: constructing a second scheme layer judgment matrix according to the second resource occupied information of each pod to each service resource in the node, to obtain a resource occupied weight of each pod to each service resource in the node;

[0088] Step S2214: constructing a second target layer weight according to the resource shortage degree weight and the resource occupied weight, to obtain a target weight value of each pod, and determining a pod with the largest target weight value as the pod to be processed.

[0089] Specifically, when the resource occupied information is the resource occupation rate of each service resource in the node, referring to the setting method of the matrix when the target node is screened, in step S2212, the horizontal and vertical coordinates of the second index layer judgment matrix are set as each type of service resource in the node Node, and each element in the matrix is the ratio of the resource occupation rate between the corresponding nodes Node. The horizontal and vertical coordinates of the second scheme layer judgment matrix are each Pod, and each matrix element value is the ratio of the use rate of the resource between the corresponding Pods (i.e., the ratio of the second resource occupied information). The resource shortage degree weight and the resource occupied weight are respectively obtained by finding the maximum eigenvector of each judgment matrix and performing normalization processing. Finally, the second target matrix is calculated, and the Pod to be processed is selected according to the weighted sum. The weight calculation formula of the Pod refers to the formula for finding the weight value of each node when the first target weight matrix is constructed in step S134. For example, Podi target weight value = CPU resource shortage degree weight * Podi CPU resource occupied weight + memory resource shortage degree weight * Podi memory resource occupied weight + uplink bandwidth resource shortage degree weight * Podi uplink bandwidth resource occupied weight + downlink bandwidth resource shortage degree weight * Podi downlink bandwidth resource occupied weight, wherein the * symbol represents the product. In this way, the target weight values of all Pods are obtained, and then the Pod with the largest score (i.e., the Pod with the largest target weight value) is selected as the pod to be processed. In this way, the Pod with a large resource occupation rate is selected, and then the load of the node Node is relieved by migrating or replicating the operation of the Pod.

[0090] In the embodiment of the present application, the service management method further comprises determining an occupation early warning value of the second resource occupation information of each service resource in the node for the to-be-processed pod; and the step S222 of determining whether the pod on the node needs to be migrated or scaled according to the type of the to-be-processed pod and the second resource occupation information of each service resource in the node, comprises: if the to-be-processed pod is a non-resource-limited pod and the second resource occupation information of at least one service resource in the node is greater than the occupation early warning value, then the to-be-processed pod is scaled out, and the scaled pod is deployed into the target node determined before the next service deployment, so as to continuously optimize the deployment and improve the service quality.

[0091] In the optional embodiment of the present application, the step S222 of determining whether the pod on the node needs to be migrated or scaled according to the type of the to-be-processed pod and the second resource occupation information of each service resource in the node, further comprises:

[0092] If the to-be-processed pod is a resource-limited pod, or the to-be-processed pod is a non-resource-limited pod and the second resource occupation information of each service resource in the node is less than the occupation early warning value, then the to-be-processed pod is closed and pod stop information is generated, and then the corresponding to-be-processed pod is deployed into the target node determined before the next service deployment according to the pod stop information.

[0093] When the type of the pod is a resource-limited pod, a first early warning lower limit value of the resource occupation information is determined when determining the early warning upper limit value of the resource occupation information, and after sequentially traversing the resource occupation information of each service resource in each node within a preset time interval, if it is judged that the resource occupation information of each service resource in each node is greater than the early warning upper limit value, then it is judged whether the resource occupation information of any service resource in each node is less than the first early warning lower limit value, and if yes, then the node with the resource occupation information less than the first early warning lower limit value is determined as a node with too light load.

[0094] In the step S22, the type of the pod, the load information, the SLA violation information, and the second resource occupation information of each pod to each service resource in the node are used to determine whether the pod on the node needs to be migrated or scaled, and the step further comprises: when the SLA violation information of the service exceeds the rated violation threshold, but the node where the service is located is neither a node with too heavy load nor a node with too light load, then the resource-limited pod of the service is scaled out. For the resource-limited pod, the number of resources used by the pod is limited when the pod is initially defined, and when the user request volume increases sharply, the service quality may be reduced due to insufficient allocated resources, and at this time, the system performs scaling processing on the service where the pod is located, so that the number of replicas is increased by one, and more resources are allocated to the service to improve the service quality.

[0095] Secondly, when the first early warning lower limit value of the resource occupation information is determined, a second early warning lower limit value of the resource occupation information is also determined, and the second early warning lower limit value is greater than the first early warning lower limit value. Then, the determination of whether the Pod on the node needs to be migrated or the replica scaling processing in step S22 also includes:

[0096] If the node of the service in which the resource-limited Pod is located is a lightly loaded node, the replica of the Pod on the node is scaled down, and it is ensured that the resource occupation information of the service resource after the scaling down processing is less than the second early warning lower limit value. Assuming that service 1 has four Pod replicas, and the memory allocation of each replica is 400m, but the average memory actually used by the four replicas in 30 minutes is 200m, at this time, there is a large amount of idle memory resource (that is, the first early warning lower limit value can be determined as 50% in an embodiment of the application), but other services cannot use these resources, resulting in waste of resources, at this time, the replica of the service can be considered to be scaled down, if the resource usage after scaling down: 200*4 / (400*(4-1))<80% (that is, the second early warning lower limit value), the number of Pod replicas can be reduced by one to realize the scaling down processing. That is, when the service is lightly loaded, the number of replicas is reduced, and the node resource usage rate is kept around 80% to reduce resource waste.

[0097] In the service deployment of the Pod by using the analytic hierarchy process in the embodiment of the application, the node resource usage and the demand of the Pod for various resources are fully considered, the weights of various resources are determined according to the importance of the resources, and the target node is finally screened out. The resource weight matrix can be dynamically updated in the system running process, and the Pod is deployed more reasonably. Secondly, the quality of the service is maximally guaranteed in the service running process, the node load is relieved from two aspects of Pod migration and Pod replica expansion, the quality of the service is improved, and the number of replicas of the service can be appropriately reduced to reduce resource waste when the current node load is too light. Therefore, the service management method provided in the embodiment of the application enables the Kubernetes to deploy the service according to various factors such as service resource demand, node resource, node load and the like, to ensure that the Pod is allocated to the optimal node while meeting the application demand, to realize the optimized deployment of the service, and to continuously monitor the service quality, node resource usage and the like in the service running process, so as to reasonably migrate or control the replica of the Pod according to the actual change of each factor, to achieve the purpose of improving the quality of the service, relieving the node pressure and reducing the system resource consumption.

[0098] Please refer to Figure 6 In the second embodiment of the application, a service management platform Kubernetes is provided, which comprises:

[0099] The first determining module 1 is configured to determine service resources configured by each node and resource usage information of each service resource in each node used by each pod, and the types of the service resources at least include CPU, memory and network bandwidth;

[0100] The second determining module 2 is configured to determine first resource occupation information of each pod required to occupy each service resource in each node based on operation requirements of a service operation process;

[0101] The node screening module 3 is configured to determine a target node by using an analytic hierarchy process according to the types of the service resources, the resource usage information and the first resource occupation information.

[0102] The deployment module 4 is configured to deploy each pod to the target node to implement service deployment after the service operation process.

[0103] When the resource usage information is resource idle occupation information of each service resource in each node not occupied by each pod, the node screening module 3 determines the target node by using the analytic hierarchy process according to the types of the service resources, the resource usage information and the first resource occupation information, and specifically performs the following steps of the method:

[0104] Step S131: Determine relative importance parameters between each service resource in each node according to the first resource occupation information of each pod required to occupy each service resource in each node, and the relative importance parameters are ratios of the first resource occupation information of different service resources, for example, the relative importance parameter value can be set to 2 because the CPU resource requirement of a certain pod is slightly higher than the importance of the memory;

[0105] Step S132: Construct a first index layer judgment matrix according to the types of the service resources and the relative importance parameters to obtain resource importance weights corresponding to each service resource;

[0106] Step S133: Construct a first scheme layer judgment matrix according to the resource idle occupation information of each service resource in each node to obtain idle occupation weights of each service resource in each node;

[0107] Step S134: Construct a first target layer weight matrix according to the resource importance weights and the idle occupation weights to obtain target weight values of each node, and determine a node with the largest target weight value as the target node.

[0108] The service management platform Kubernetes further includes:

[0109] The pod supervision module 5 is configured to monitor and store second resource occupation information of each pod in the service running process in real time, and dynamically update the first index layer judgment matrix of the node screening module 3 according to the second resource occupation information before the pod is deployed to the target node determined in advance before the next service deployment.

[0110] The types of the pods include resource-limited pods and non-resource-limited pods; and when the resource occupation information is the resource occupation information of each service resource in the node occupied by each pod, the service management platform Kubernetes further includes:

[0111] The running supervision module 7 is configured to continuously acquire SLA violation information of the service in a preset time interval, and continuously acquire load information of each node according to the resource occupation information of each service resource in the node.

[0112] The resource calculation module 6 is configured to determine whether the pods on the node need to be migrated or the replica of the pods need to be scaled according to the types of the pods, the load information, the SLA violation information and the second resource occupation information of each pod to each service resource in the node.

[0113] The first determination module 1 is further configured to determine a pre-warning upper limit value of the resource occupation information; and when the running supervision module 7 continuously acquires the load information of each node according to the resource occupation information of each service resource in the node, the following steps are specifically performed: the resource occupation information of each service resource in each node is sequentially traversed in a preset time interval, and it is determined whether the resource occupation information of any service resource in each node is greater than the pre-warning upper limit value determined by the first determination module 1; if yes, the node with the resource occupation information greater than the pre-warning upper limit value is determined as a node with excessive load.

[0114] Before the resource calculation module 6 determines whether the pods on the node need to be migrated or the replica of the pods need to be scaled, the resource calculation module 6 further determines a rated violation threshold of the SLA violation information; and the resource calculation module 6 includes:

[0115] The pod screening module 61 is configured to, when the SLA violation information of the service exceeds the rated violation threshold and the node where the service is located is the node with excessive load, screen out a pod to be processed according to the type of the service resource, the resource occupation information of each service resource in the node and the second resource occupation information of each pod to each service resource in the node by using the analytic hierarchy process.

[0116] The resource scheduling module 62 is configured to determine whether the pods on the node need to be migrated or the replica of the pods need to be scaled according to the types of the pods to be processed and the second resource occupation information of the pods to each service resource in the node.

[0117] The pod screening module 61 screens the pod to be processed according to the type of service resource, the resource occupation information of each service resource in the node, and the second resource occupation information of each pod to each service resource in the node, and adopts the analytic hierarchy process to screen the pod to be processed, and the specific steps of the method are as follows:

[0118] Step S2211: determining the resource shortage degree parameters between each service resource according to the resource occupation information of each service resource in the node, and the resource shortage degree parameters being the ratio of the resource occupation information of different service resources in each node;

[0119] Step S2212: constructing a second index layer judgment matrix according to the type of service resource and the resource shortage degree parameters to obtain the resource shortage degree weight corresponding to each service resource in the node;

[0120] Step S2213: constructing a second scheme layer judgment matrix according to the second resource occupation information of each pod to each service resource in the node to obtain the resource occupation weight of each pod to each service resource in the node;

[0121] Step S2214: constructing a second target layer weight according to the resource shortage degree weight and the resource occupation weight to obtain the target weight value of each pod, and determining the pod with the largest target weight value as the pod to be processed.

[0122] The pod monitoring module 5 is also used to determine the occupation early warning value of the second resource occupation information of the pod to be processed to each service resource in the node; and the resource scheduling module 62 determines whether the pod on the node needs to be migrated or the replica scaling processing according to the type of the pod to be processed and the second resource occupation information of the pod to be processed to each service resource in the node, and the specific steps of the method are as follows: if the pod to be processed is a non-resource-limited pod, and the second resource occupation information of the pod to be processed to at least one service resource in the node is greater than the occupation early warning value determined by the pod monitoring module 5, then the pod to be processed is subjected to replica expansion processing, and the expanded replica is deployed to the target node determined before the next service deployment.

[0123] The resource scheduling module 62 also performs the following steps of the method when determining whether the pod on the node needs to be migrated or the replica scaling processing according to the type of the pod to be processed and the second resource occupation information of the pod to be processed to each service resource in the node: if the pod to be processed is a resource-limited pod, or the pod to be processed is a non-resource-limited pod and the second resource occupation information of the pod to be processed to each service resource in the node is less than the occupation early warning value, then the pod to be processed is closed and the pod stop information is generated, and the corresponding pod to be processed is deployed to the target node determined by the node screening module 3 before the next service deployment according to the pod stop information.

[0124] Optionally, when the type of the Pod is a resource-limited Pod, the first determining module 1 determines a first lower limit of the pre-warning of the resource occupation information when determining the upper limit of the pre-warning of the resource occupation information, and the running supervision module 7 traverses the resource occupation information of each service resource in each node in a preset time interval, and if it is determined that the resource occupation information of no service resource in each node is greater than the upper limit of the pre-warning, it is determined whether the resource occupation information of any service resource in each node is less than the first lower limit of the pre-warning determined by the first determining module 1, and if so, the node with the resource occupation information less than the first lower limit of the pre-warning is determined as the node with too light load.

[0125] When the resource calculation module 6 determines whether the Pod on the node needs to be migrated or the replica scaling processing is needed according to the type of the Pod, the load information, the SLA violation information and the second resource occupation information of each pod to each service resource in the node, the resource scheduling module 62 further executes the following steps of the method: when the SLA violation information of the service exceeds the rated violation threshold, but the node where the service is located is neither a node with too heavy load nor a node with too light load, the resource-limited Pod of the service is subjected to replica expansion processing.

[0126] Optionally, the first determining module 1 determines a second lower limit of the pre-warning of the resource occupation information when determining the first lower limit of the pre-warning of the resource occupation information, and the second lower limit of the pre-warning is greater than the first lower limit of the pre-warning; and the resource calculation module 6 further executes the following steps of the method by using the resource scheduling module 71:

[0127] If the node of the resource-limited Pod is a node with too light load, the replica of the Pod on the node is subjected to scaling processing, and it is ensured that the resource occupation information of the service resource after the scaling processing is less than the second lower limit of the pre-warning.

[0128] Finally, the application discloses a Kubernetes-oriented service management method, which can be used for service scheduling management in a cluster system composed of nodes of the same type (identical in resource type and quantity), ensures that a Pod is allocated to an optimal node when a service is deployed, and dynamically adjusts the service according to node load and service quality when the service is running to enable users to obtain high-quality services. The following points are mainly implemented: 1) the Pods are divided into two categories: one is a resource-limited Pod, and the other is a non-resource-limited Pod, and the two types of Pods are uniformly managed when the service is deployed and scheduled; 2) in combination with various factors such as node resources, load conditions, and service quality requirements, the analytic hierarchy process is used to realize optimal deployment of the service, and the Pod is allocated to the optimal node while meeting the application requirements; 3) during the service running process, the service quality and node resource usage are continuously monitored, and the analytic hierarchy process is used to select appropriate Pods for migration; 4) during the service running process, the number of service replicas is controlled to reduce SLA violations and system resource waste. The method disclosed by the application can realize reasonable use of node physical resources based on service characteristics and system resource usage, and improve user satisfaction during service running.

[0129] In the third embodiment of the application, the embodiment of the application is based on the two above-mentioned embodiments, and the attached Figure 7-12 An application example of the application is introduced.

[0130] The embodiment of the application provides a Kubernetes-oriented service management method, and mainly solves the following two problems: 1) service optimal deployment: various factors such as node resources, load conditions, and service resource requirements are comprehensively considered to ensure that a Pod is allocated to an optimal node while meeting application requirements, and optimal deployment of the service is realized; 2) service scheduling oriented to service quality: the service quality and node resource usage are continuously monitored, the Pod is reasonably migrated, and the number of replicas is controlled to relieve node pressure and improve service quality.

[0131] I. To achieve the above purpose, please refer to Figure 11 The embodiment of the application registers a self-defined algorithm (analytic hierarchy process) as a scheduling plug-in by using Scheduler-extender, and realizes the proposed service management strategy in cooperation with the original scheduler of Kubernetes, and the following technical scheme is specifically adopted to realize the same:

[0132] First step: when deploying services, the analytic hierarchy process in the multi-criteria decision method is used to select a target node, and the Deployment controller is applied to deploy the Pod. The specific steps include the following (in the embodiment of the application, in order to ensure unified management of the resource-limited Pod and the non-resource-limited Pod, the resource occupied by the Pod = Max (actual resource usage quantity of the Pod, resource quantity allocated to the Pod)) :

[0133] The service resources of the node are divided into four types: CPU, memory, uplink bandwidth, and downlink bandwidth. According to the resource demand of the Pod (i.e., the first resource occupation information mentioned in the first embodiment), the relative importance degree between the resources is obtained, and a judgment matrix of the index layer is constructed. The horizontal and vertical coordinates of the matrix are each service resource, and the element value is the importance parameter of the horizontal coordinate representing resource relative to the vertical coordinate representing resource. For example, since the CPU resource demand of a certain Pod is slightly higher than the importance of the memory, the corresponding element value can be set to 2. The maximum characteristic vector is calculated for the judgment matrix of the index layer, and normalization processing is performed to obtain the value of the resource importance weight of each type of service resource.

[0134] When the resource usage information is the resource idle occupation rate of each type of service resource in the node that is not occupied by the Pod, a judgment matrix of the scheme layer is constructed, and each type of resource corresponds to a matrix D. For example, the horizontal and vertical coordinates of the CPU matrix are each node (Node), and the matrix element Dij is the CPU idle occupation rate of Nodei / the CPU idle occupation rate of Nodej. Then, the weight value of the CPU idle occupation in each Node is obtained according to the above steps. Similarly, the memory matrix, the uplink bandwidth matrix, and the downlink bandwidth matrix are constructed, and the weight of each Node for different resources is calculated.

[0135] (4) A maximum weight matrix is constructed, and the weight value of each node is calculated. The weight value of Nodei = CPU resource weight * weight of the idle CPU resource of Nodei + memory resource weight * weight of the idle memory resource of Nodei + uplink bandwidth resource weight * weight of the idle uplink bandwidth resource of Nodei + downlink bandwidth resource weight * weight of the idle downlink bandwidth resource of Nodei.

[0136] (5) The Node with the maximum score is selected as the target node, and the Pod is deployed to the node. In the conventional analytic hierarchy process, a consistency judgment is required when constructing the comparison matrix. Since the real data is used for comparison in the embodiment of the application, the consistency problem does not occur.

[0137] Second step: record the resource usage of the Pod during the system running. Real-time monitor the resource occupation data of the Pod (equivalent to the second resource occupation information in the first embodiment), and store the resource occupation data of the Pod in the original database Etcd of Kubernetes. When the Pod needs to be scheduled later, dynamically update the judgment matrix of the target layer in the first step according to the latest resource usage of the Pod, for example: the latest resource usage in the recorded data shows that the CPU occupation of the Pod is 30% (quantified as 3), and the memory occupation is 20% (quantified as 2), so the importance of CPU to memory in the target layer matrix is 3 / 2, and the upward rounding of 3 / 2 is 2.

[0138] Third step: dynamically adjust the service during the system running. When the monitoring finds that the service response time is too long and the load of the node where the service is located is too heavy, the Pod on the node is migrated or the replica of the Pod is expanded to reduce the load of the node and improve the service quality. If the node does not appear to be too heavy, the replica of the service with too long response time needs to be expanded at this time to ensure the service quality. The specific steps include the following steps:

[0139] Node load monitoring. Use Pormetheus to monitor the resources of the node, and the Qos controller will traverse each node every minute. If the resource occupation is greater than 90%, the node is considered to be too heavy.

[0140] When the node is too heavy, the analytic hierarchy process is also used to construct the index layer judgment matrix according to the resource occupation of the node, and the scheme layer judgment matrix according to the resource usage of each Pod. Each resource corresponds to a matrix, and the horizontal and vertical coordinates are each Pod. Finally, the decision matrix, i.e. the weight value of each Pod, is calculated.

[0141] Get the Pod with the highest score. If the resource occupation of the Pod is less than 40%, the Qos controller will check whether there is enough free resource in the cluster. If there is free resource, the Pod will be killed (closed), and the Deployment controller will receive the signal that the Pod is killed and will be deployed again through the scheduler. If there is no available resource, the migration is skipped and the monitoring continues until there is available resource for migration.

[0142] (4) The Pod with the highest score is a non-resource-limited type and its resource occupation rate is greater than 40%, so the replica of the Pod is expanded, i.e. the number of replicas is increased by one. The new replica will be deployed to a suitable node through the scheduler by applying the analytic hierarchy process again.

[0143] (5) When the node does not appear to be overloaded, but there is still a large service delay, the Qos controller will expand the replicas of the service due to insufficient resources allocated to the service corresponding Pod, and increase the total resource occupation of the service.

[0144] Fourth step: for resource-limited Pods, when the service that the Pod belongs to occupies too few resources, that is, resource usage number * replica number (30min average value) / (single Pod resource application number * (replica number) - 1) < 4 / 5, at this time, the replica number is reduced by one to reduce resource waste, * represents multiplication.

[0145] II. For the above self-defined scheduling process based on AHP, the following is the process:

[0146] 1. Adopt AHP to model the problem into the structure shown in the figure, and divide the target of decision, the factors considered (decision criteria) and the decision objects into the highest layer, the middle layer and the lowest layer according to their mutual relationship: Figure 8

[0147] (1) The highest layer (target layer): the purpose of decision and the problem to be solved;

[0148] (2) The middle layer (criterion layer or index layer): the factors considered, the criteria of decision;

[0149] (3) The lowest layer (scheme layer): the alternative schemes for decision;

[0150] 2. Index comparison quantification specification:

[0151]

[0152] In order to facilitate comparison between resources, the following provisions are adopted (utilization rate of each type of resource - first resource occupation information required by pod): 0-10% is recorded as 1; 10%-20% is recorded as 2; 20%-30% is recorded as 3; 30%-40% is recorded as 4; 40%-50% is recorded as 5; 50%-60% is recorded as 6; 60%-70% is recorded as 7; 70%-80% is recorded as 8; 80%-100% is recorded as 9.

[0153] 3. Build index layer judgment matrix:

[0154] 1) According to the importance of resources, construct the judgment matrix.

[0155] The index layer judgment matrix is constructed according to the general service resource demand situation, that is, CPU is more important than memory, memory is more important than bandwidth, and uplink bandwidth and downlink bandwidth are equally important.

[0156] Z A1 CPU A2 Memory A3 Uplink Bandwidth A4 Downlink Bandwidth A1 CPU 1 2 3 3 A2 Memory 1 / 2 1 2 2 A3 Uplink Bandwidth 1 / 3 1 / 2 1 1 A4 Downlink Bandwidth 1 / 3 1 / 2 1 1 sum 2.166 4 7 7

[0157] ​2) Arithmetic mean method (sum-product method)

[0158] Column-wise normalization:

[0159] Z A1 CPU A2 Memory A3 Uplink Bandwidth A4 Downlink Bandwidth ω A1 CPU 0.462 0.5 0.429 0.429 0.455 A2 Memory 0.231 0.25 0.286 0.286 0.263 A4 Bandwidth 0.154 0.125 0.143 0.143 0.141 A6 Latency 0.154 0.125 0.143 0.143 0.141

[0160] Scheme layer judgment matrix construction

[0161] If CPU(Nodei) > CPU(Nodej);

[0162] Then A1ij = CPU(Nodei) / CPU(Nodej), (round up) Aji = 1 / A1ij;

[0163] Otherwise A1ji = CPU(Nodej) / CPU(Nodei), (round up) Aij = 1 / A1ji;

[0164] CPU(Nodei): The idle CPU occupancy rate of the ith node;

[0165] CPU: Matrix

[0166] A1 CPU Node1 Node2 Node3 Node1 Node2 Node3 ω Node1 1 1 / 4 2 0.1818 0.1818 0.1818 0.1818 Node2 4 1 8 0.7273 0.7273 0.7273 0.7273 Node3 1 / 2 1 / 8 1 0.0909 0.0909 0.0909 0.0909 Sum 5.5000 1.3750 11

[0167] If Mem(Nodei) > Mem(Nodej);

[0168] Then A2ij = Mem(Nodei) / Mem(Nodej), (round up) A2ji = 1 / A2ij;

[0169] Otherwise A2ji = Mem(Nodej) / Mem(Nodei), (round up) A2ij = 1 / A2ji;

[0170] Mem(Nodei): The idle memory occupancy rate of the ith node;

[0171] Memory matrix:

[0172] A2 Memory Node1 Node2 Node3 Node1 Node2 Node3 ω Node1 1 5 2 0.5882 0.6250 0.5714 0.5949 Node2 1 / 5 1 1 / 2 0.1176 0.1250 0.1429 0.1285 Node3 1 / 2 2 1 0.0909 0.0909 0.0909 0.2766 Sum 1.7000 8.0000 3.5000

[0173] If B1(Nodei) > B1(Nodej)

[0174] Then A3ij = B1(Nodei) / B1(Nodej), (round up) A3ji = 1 / A3ij;

[0175] Otherwise A3ji = B1(Nodej) / B1(Nodei), (round up) A3ij = 1 / A3ji;

[0176] b1(Nodei): The idle uplink bandwidth occupancy rate of the ith node;

[0177] Uplink bandwidth matrix:

[0178] A3 Node1 Node2 Node3 Node1 Node2 Node3 ω Node1 1 5 7 0.7447 0.7692 0.7000 0.7380 Node2 1 / 5 1 2 0.1489 0.1538 0.2000 0.1676 Node3 1 / 7 1 / 2 1 0.1064 0.0769 0.1000 0.0944 Sum 1.3429 6.5000 10.0000

[0179] If B2(Nodei) > B2(Nodej);

[0180] Then A4ij = B2(Nodei) / B2(Nodej), (upward rounding) A4ji = 1 / A4ij;

[0181] Otherwise A4ji = B2(Nodej) / B2(Nodei), (upward rounding) A4ij = 1 / A4ji;

[0182] b2(Nodei): the i-th node free downlink bandwidth occupancy rate;

[0183] Downlink bandwidth matrix:

[0184] A4 Node1 Node2 Node3 Node1 Node2 Node3 ω Node1 1 3 5 0.6521 0.6667 0.6250 0.6279 Node2 1 / 3 1 2 0.2174 0.2222 0.2500 0.2299 Node3 1 / 5 1 / 2 1 0.1304 0.1111 0.1250 0.1222 Sum 1.5333 4.5000 8.0000

[0185] 5. Constructing the maximum weight matrix

[0186] Z ω Node1 Node2 Node3 A1 CPU 0.455 0.1818 0.7273 0.0909 A2 Memory 0.263 0.5949 0.1285 0.2766 A3 Uplink Bandwidth 0.141 0.7380 0.1676 0.0944 A4 Downlink Bandwidth 0.141 0.6279 0.2299 0.1222 Score 0.4299516 0.4134915 0.1437369

[0187] According to the maximum weight matrix, Node1 obtains the maximum weight value, that is, the selected target node, and then deploys the pod into the target node.

[0188] 5. Scheduler-extender realizes custom scheduling:

[0189] Schduler extender is a Kubernetes external extension method, which can independently build a scheduling service according to the needs, and realize the corresponding remote calling interface (http). Scheduler will make a remote call according to the user-defined resources and interfaces in the corresponding stage of scheduling. The corresponding service makes decisions according to its own resource data and the intermediate scheduling results passed by Scheduler.

[0190] Scheduler-extender only needs to implement the interface of the corresponding plug-in and write a yaml file to register the corresponding service interface, so as to realize the extension of Scheduler. Without modifying any code of the scheduler, the plug-in of the scheduling plug-in can be realized.

[0191] Please refer to Figure 11 , Scheduler-extender provides two interfaces (Filter and Prioritize):

[0192] Filter interface: Filter is mainly used to call extender for secondary filtering after the pre-selection stage is completed. In this stage, the embodiment of the application polls all Node nodes, filters out nodes whose certain resources are greater than 90%, and adds the passed to canSchedule and the failed to canNotSchedule. The returned result is in ExtenderFilterResult.

[0193] Prioritize interface: Prioritize is mainly used to score each node in the preferred stage. In this stage, the embodiment of the application scores the nodes by using the above analytic hierarchy process and transmits the result to the scheduler through http.

[0194] The Filter interface extension pseudo code is as follows:

[0195]

[0196] The Prioritize interface extension pseudo code is as follows:

[0197]

[0198] Three, Qos controller based on service quality:

[0199] 1. As shown in Figure 10 , Kubernetes provides a Client-go library for developers to develop it again. The patent uses the official Client-go library to design a Qos controller to ensure service quality. The Qos controller mainly completes the following functions:

[0200] 1) The Qos controller continuously obtains the load of each Node and the SLA violation of the service at a fixed time interval through the configured Prometheus (service monitoring component).

[0201] 2) When it is monitored that a Node is overloaded, the overloaded Node is screened out, and the Pod (which consumes more resources) that needs to be processed is calculated. The Qos controller obtains the resource occupation of the Pod, decides to migrate or replicate according to the number of Pod resource occupation, and reduces the load of the Node. The migrated Pod or the newly added Pod replica is deployed to a suitable node through the scheduling system again.

[0202] 3) When the number of SLA violations of a service is too large (i.e. the SLA violation information of the service exceeds the rated violation threshold, the specific value of which is set according to the service requirements), but the Node where the service is located is not overloaded, the Qos controller will expand the number of replicas of the service considering the resource shortage caused by the insufficient number of replicas. And when the ratio of the resources occupied by the Pod to the resources allocated to it is too low, the number of replicas is reduced to reduce the waste of resources.

[0203] 2. The design of the Qos controller includes the following contents:

[0204] 1) Node overload judgment.

[0205] Node resource occupancy rate greater than 90% (i.e. resource occupancy information greater than the warning upper limit value 90%) is defined as overloaded.

[0206] 11) Node overload processing.

[0207] When the Node is overloaded, first select the Pod with large resource occupancy according to the resource shortage degree of the Node, and then relieve the pressure of the Node by migrating or expanding the number of replicas. The selection of the Pod also uses the analytic hierarchy process. The horizontal and vertical coordinates of the index layer matrix are set as the resources in the Node, and the elements in the matrix are the ratios of the resource occupancy between the corresponding Nodes; the scheme layer is constructed according to different judgment matrices of each type of resource, and the horizontal and vertical coordinates of each type of resource are each Pod, and the element values of each matrix are the usage ratios of the corresponding Pods. Finally, the target matrix is calculated, and the Pod to be processed is selected according to the weighted sum.

[0208] 12) When selecting the Pod to be processed, the analytic hierarchy process is used to model the problem into the structure shown in the figure. Figure 9 The target of decision, the factors considered (decision criteria) and the decision objects are divided into the highest layer, the middle layer and the lowest layer according to their mutual relationship:

[0209] (1) The highest layer (target layer): the purpose of decision and the problem to be solved;

[0210] (2) The middle layer (criterion layer or index layer): the factors considered, the criteria of decision;

[0211] (3) The lowest layer (scheme layer): the alternative schemes for decision;

[0212] a. Quantitative definition of index comparison:

[0213]

[0214]

[0215] ​For the convenience of comparison between resources, the following provisions are adopted (the utilization rate of each type of resource - the data of the resource occupation information when the resource occupation rate is specific): 0-10% is recorded as 1; 10%-20% is recorded as 2; 20%-30% is recorded as 3; 30%-40% is recorded as 4; 40%-50% is recorded as 5; 50%-60% is recorded as 6; 60%-70% is recorded as 7; 70%-80% is recorded as 8; 80%-100% is recorded as 9.

[0216] b. Construct the index layer judgment matrix:

[0217] According to the resource shortage degree, the judgment matrix is constructed.

[0218] According to the node resource occupation situation, the index layer judgment matrix is constructed, such as: Node1 appears overloading, at this time, the occupation rate of each resource is: CPU: 90% (quantized as 9), memory: 80% (quantized as 8), uplink bandwidth: 40% (quantized as 5), and downlink bandwidth: 30% (quantized as 3). Z12=90 / 80 (rounded up) =2, then Z21=1 / 2; Z13=90 / 40 (rounded up) =3, then Z31=1 / 3, and the other elements of the matrix are the same.

[0219] Z A1 CPU A2 Memory A3 Uplink Bandwidth A4 Downlink Bandwidth A1 CPU 1 2 3 3 A2 Memory 1 / 2 1 2 3 A3 Uplink Bandwidth 1 / 3 1 / 2 1 2 A4 Downlink Bandwidth 1 / 3 1 / 3 1 / 2 1 sum 2.166 3.833 6.5 9

[0220] c. Arithmetic mean method (sum product method)

[0221] Column normalization:

[0222] Z A1 CPU A2 Memory A3 Uplink Bandwidth A4 Downlink Bandwidth ω A1 CPU 0.462 0.522 0.462 0.333 0.445 A2 Memory 0.231 0.261 0.308 0.333 0.345 A4 Bandwidth 0.154 0.13 0.154 0.222 0.165 A6 Latency 0.154 0.087 0.077 0.111 0.107

[0223] Scheme layer judgment matrix construction

[0224] If CPU(Podi)>CPU(Podj);

[0225] Then A1ij=CPU(Podi) / CPU(Podj), (rounded up) Aji=1 / A1ij;

[0226] Otherwise A1ji=CPU(Podj) / CPU(Podi), (rounded up) Aij=1 / A1ji;

[0227] CPU(Podi): CPU occupation rate of the ith Pod;

[0228] CPU: Matrix

[0229] A1 CPU Pod1 Pod2 Pod3 Pod1 Pod2 Pod3 ω Pod1 1 1 / 4 2 0.1818 0.1818 0.1818 0.1818 Pod2 4 1 8 0.7273 0.7273 0.7273 0.7273 Pod3 1 / 2 1 / 8 1 0.0909 0.0909 0.0909 0.0909 Sum 5.5000 1.3750 11

[0230] If Mem(Podi)>Mem(Podj), Mem represents the memory resource;

[0231] A2ji = Mem(Podj) / Mem(Podi), (round up) A2ij = 1 / A2ji;

[0232] A2ji = Mem(Podj) / Mem(Podi), (round up) A2ij = 1 / A2ji;

[0233] Mem(Podi): Memory occupancy of the ith Pod;

[0234] Memory matrix:

[0235] A2 Memory Pod1 Pod2 Pod3 Pod1 Pod2 Pod3 ω Pod1 1 5 2 0.5882 0.6250 0.5714 0.5949 Pod2 1 / 5 1 1 / 2 0.1176 0.1250 0.1429 0.1285 Pod3 1 / 2 2 1 0.0909 0.0909 0.0909 0.2766 Sum 1.7000 8.0000 3.5000

[0236] B1(Podi) > B1(Podj),

[0237] A3ji = B1(Podj) / B1(Podi) (round up) A3ij = 1 / A3ji;

[0238] A3ji = B1(Podj) / B1(Podi) (round up) A3ij = 1 / A3ji;

[0239] B1(Podi): Uplink bandwidth occupancy of the ith Pod;

[0240] Uplink bandwidth matrix:

[0241] A3 Bandwidth Node1 Node2 Node3 Node1 Node2 Node3 ω Node1 1 5 7 0.7447 0.7692 0.7000 0.7380 Node2 1 / 5 1 2 0.1489 0.1538 0.2000 0.1676 Node3 1 / 7 1 / 2 1 0.1064 0.0769 0.1000 0.0944 Sum 1.3429 6.5000 10.0000

[0242] B2(Podi) > B2(Podj);

[0243] A4ji = B2(Podj) / B2(Podi) (round up) A4ij = 1 / A4ji;

[0244] A4ji = B2(Podj) / B2(Podi) (round up) A4ij = 1 / A4ji;

[0245] B2(Podi): Downlink bandwidth occupancy of the ith Pod;

[0246] Downlink bandwidth matrix:

[0247] A4 Latency Node1 Node2 Node3 Node1 Node2 Node3 ω Node1 1 3 5 0.6521 0.6667 0.6250 0.6279 Node2 1 / 3 1 2 0.2174 0.2222 0.2500 0.2299 Node3 1 / 5 1 / 2 1 0.1304 0.1111 0.1250 0.1222 Sum 1.5333 4.5000 8.0000

[0248] d. Constructing the maximum weight matrix

[0249] Z ω Node1 Node2 Node3 A1 CPU 0.445 0.1818 0.7273 0.0909 A2 Memory 0.345 0.5949 0.1285 0.2766 A3 Uplink Bandwidth 0.165 0.7380 0.1676 0.0944 A4 Downlink Bandwidth 0.107 0.6279 0.2299 0.1222 Score 0.4750968 0.4202343 0.16452

[0250] According to the maximum weight matrix, Pod1 obtains the maximum weight value, which is the selected target node.

[0251] 13) Replica expansion

[0252] When the Pod to be processed is obtained, first determine its resource occupancy (i.e., the second resource occupancy information). If the Pod is a non-resource-limited Pod and the resource occupancy of a certain service resource is greater than 40% (the occupancy warning value), it will be migrated to another node Node, and the resource usage after migration will also excessively occupy resources. Therefore, the best method for such a Pod is to perform scaling processing. The system modifies the replicas field of the Deployment in which the Pod is located by calling the client-go interface to reduce the number of replicas by one.

[0253] 14) Pod migration

[0254] When the Pod to be processed is a resource-limited Pod, or the Pod is a non-resource-limited Pod and each resource occupancy is less than 40% (the occupancy warning value), the system closes the Pod through the client-go interface. After the Pod stops, the Deployment receives the Pod stop information and redeploy the Pod. The Pod is deployed to a node Node with relatively small resource competition through a self-defined scheduling algorithm.

[0255] 5. Resource-limited Pod resource shortage leading to low service quality processing

[0256] For resource-limited Pods, when the user request volume increases, the allocated resources may not be enough to cause the service quality to decrease. At this time, the system will perform scaling processing on the service in which the Pod is located. The Replicas method of the Deployment in the client-go interface is called to increase the number of replicas by one, and more resources are allocated to the service to improve the service quality.

[0257] 6. Resource-limited Pod resource recycling processing

[0258] Non-limited Pods use resources according to actual conditions. When the user request volume is low, resource occupancy will be low, and this stage can be ignored. For resource-limited Pods, the following situation may occur: Assume that service 1 has four Pod replicas, and each replica is allocated 400m of memory. However, the average memory actually used by the four replicas in 30 minutes is 200m. At this time, there is a large amount of idle memory, but other services cannot use these resources, resulting in resource waste. At this time, the number of replicas of the service can be scaled down. If the resource usage after scaling down is: 200*4 / (400*(4-1))<80% (the second lower warning value), the number of replicas can be reduced by one. Since each replica itself occupies some resources, the calculated resource usage will be lower than the actual usage.

[0259] The Qos controller pseudo code is as follows:

[0260]

[0261]

[0262] The Kubernetes-oriented service management method has the following advantages and effects:

[0263] The analytic hierarchy process is used to optimize service deployment, fully considers the use of node resources and the demand of the Pod for various resources, determines the weights of various resources according to the importance of the resources, and finally screens the target node. The resource weight matrix can be dynamically updated during system operation, and the Pod is more reasonably deployed.

[0264] In the service running process, the quality of the service is maximized, the node load is relieved from the aspects of Pod migration and Pod replica expansion, and the service quality is improved. At the same time, if the current node load is too low, the number of service replicas is appropriately reduced to reduce resource waste.

[0265] It should be noted that in this article, the terms "include", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0266] The above-mentioned embodiment numbers of the application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0267] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and necessary general hardware platform, of course, also can pass through hardware, but in many cases, the former is the better implementation mode. Based on such understanding, the technical solutions of the present application can be embodied in the form of software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a plurality of instructions for controlling a terminal (which can be a mobile phone, computer, server, air conditioner, or network equipment, etc.) to execute the method described in each embodiment of the present application.

[0268] The embodiments of the present application are described above with reference to the accompanying drawings, but the present application is not limited to the above-described specific embodiments, and the above-described specific embodiments are merely illustrative, but not restrictive, and a person of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims, and these all belong to the protection of the present application.

Claims

1. A service management method for a service management platform Kubernetes, characterized in that, The method comprises the following steps: determining service resources of each node configuration and resource usage information of each service resource in each node used by each Pod, the types of the service resources including CPU, memory and network bandwidth; determining first resource occupation information of each Pod required to occupy each service resource in each node based on the operation requirements of a service operation process; determining a target node by using an analytic hierarchy process according to the types of the service resources, the resource usage information and the first resource occupation information; after deploying each Pod to the target node to implement service deployment, entering the service operation process; when the resource usage information is idle resource occupation information of each service resource in each node not occupied by each Pod, the step of determining the target node by using the analytic hierarchy process according to the types of the service resources, the resource usage information and the first resource occupation information comprises the following steps: determining relative importance parameters between each service resource in each node according to the first resource occupation information of each Pod required to occupy each service resource in each node, the relative importance parameters being ratios of the first resource occupation information of different service resources; constructing a first index layer judgment matrix according to the types of the service resources and the relative importance parameters to obtain resource importance weights corresponding to each service resource; constructing a first scheme layer judgment matrix according to the idle resource occupation information of each service resource in each node to obtain idle occupation weights of each service resource in each node; constructing a first target layer weight matrix according to the resource importance weights and the idle occupation weights to obtain target weight values of each node, and determining a node with the largest target weight value as the target node. 2.The service management method of the service-oriented management platform Kubernetes according to claim 1, wherein, In the service operation process, the service management method further comprises the following steps: real-time monitoring and storing second resource occupation information of each Pod occupying each service resource in each node in the service operation process, and dynamically updating the first index layer judgment matrix according to the second resource occupation information before deploying the Pod to the target node determined before next service deployment. 3.The service management method of the service-oriented management platform Kubernetes according to claim 2, wherein, When the types of the Pods include resource-limited Pods and non-resource-limited Pods, and the resource usage information is resource occupation information of each service resource in each node occupied by each Pod, in the service operation process, the service management method further comprises the following steps: continuously obtaining SLA violation information of the service in a preset time interval, and continuously obtaining load information of each node according to the resource occupation information of each service resource in each node; determining whether to perform Pod migration or replica scaling processing on the Pods on the node according to the types of the Pods, the load information, the SLA violation information and the second resource occupation information of each Pod occupying each service resource in each node. 4.The service management method of the service-oriented management platform Kubernetes according to claim 3, wherein, The service management method further comprises the following steps: determining a pre-warning upper limit value of the resource occupation information; the step of continuously obtaining load information of each node according to the resource occupation information of each service resource in each node comprises the following steps: Within a preset time interval, the resource occupancy information of each service resource in each node is sequentially traversed, and it is determined whether there is a service resource in each node whose resource occupancy information is greater than the warning upper limit. If so, the node with the resource occupancy information greater than the warning upper limit is identified as an overloaded node. 5.The service management method of the service-oriented management platform Kubernetes according to claim 4, wherein, Before determining whether Pods on a node need to be migrated or replicas scaled, the rated violation threshold for the SLA violation information is also determined; then: The step of determining whether to migrate or scale replicas of Pods on a node based on the type of Pod, the load information, the SLA violation information, and the second resource usage information of each Pod for each type of service resource on the node includes: When the SLA violation information of a service exceeds the rated violation threshold, and the node where the service is located is an overloaded node, the Pods to be processed are selected by the analytic hierarchy process based on the type of the service resource, the resource occupancy information of each service resource in the node, and the second resource occupancy information of each Pod for each service resource in the node. Based on the type of Pod to be processed and its secondary resource usage information for each service resource in the node, determine whether the Pod on the node needs to be migrated or its replicas scaled. 6.The service management method of the service-oriented management platform Kubernetes according to claim 5, wherein, The step of filtering out Pods to be processed using the analytic hierarchy process (AHP) based on the type of the service resource, the resource occupancy information of each service resource in the node, and the second resource occupancy information of each Pod for each service resource in the node includes: The resource scarcity parameter between service resources is determined based on the resource occupancy information of each service resource in the node. The resource scarcity parameter is the ratio of the resource occupancy information of different service resources in each node. Based on the type of the service resource and the resource scarcity parameter, a second indicator layer judgment matrix is ​​constructed to obtain the resource scarcity weight corresponding to each type of service resource in the node. Based on the second resource usage information of each Pod for each type of service resource in the node, a second scheme layer judgment matrix is ​​constructed to obtain the resource usage weight of each Pod. Based on the resource scarcity weight and the resource occupancy weight, a second target layer weight is constructed to obtain the target weight value of each Pod, and the Pod with the largest target weight value is determined as the Pod to be processed. 7.The service-oriented management platform Kubernetes service management method of claim 6, wherein, The service management method also includes: If the Pod to be processed determines the resource occupancy warning value for the second resource occupancy information of each service resource in the node, then: The process of determining whether to migrate or scale replicas of Pods on a node based on the type of Pod to be processed and its second resource occupancy information for each service resource on the node includes: If the Pod to be processed is a non-resource-limited Pod, and its second resource occupancy information for at least one service resource in the node is greater than the occupancy warning value, then the Pod to be processed will be expanded by replicas, and the expanded replicas will be deployed to the target node determined before the next service deployment.

8. A service management platform Kubernetes, characterized in that, include: The first determining module is configured to determine service resources configured by each node and resource usage information of each service resource in each node used by each Pod, and types of the service resources at least include CPU, memory and network bandwidth; The second determining module is configured to determine first resource occupation information of each Pod required to occupy each service resource in the node based on operation requirements of a service operation process; The node screening module is configured to determine a target node by using an analytic hierarchy process according to the types of the service resources, the resource usage information and the first resource occupation information; The deployment module is configured to deploy each Pod to the target node to implement service deployment and then enter the service operation process; When the resource usage information is resource idle occupation information of each service resource in the node not occupied by each Pod, the target node is determined by using the analytic hierarchy process according to the types of the service resources, the resource usage information and the first resource occupation information, including: According to the first resource occupation information of each Pod required to occupy each service resource in the node, a relative importance parameter between each service resource in each node is determined, and the relative importance parameter is a ratio of the first resource occupation information of different service resources; According to the types of the service resources and the relative importance parameter, a first index layer judgment matrix is constructed to obtain a resource importance weight corresponding to each service resource; According to the resource idle occupation information of each service resource in each node, a first scheme layer judgment matrix is constructed to obtain an idle occupation weight of each service resource in each node; According to the resource importance weight and the idle occupation weight, a first target layer weight matrix is constructed to obtain a target weight value of each node, and a node with the largest target weight value is determined as the target node.

Images