Mesh microservice resource control method based on adaptive processing rate in Kubernetes

Abstract

The present invention discloses a method for controlling mesh microservice resources based on adaptive processing rate in Kubernetes, which is oriented to mesh microservice systems in the cloud based on Kubernetes, and takes into account the negative impact of synchronous calls on container performance. The main works of the present invention are: A) Proposing a queuing model with adaptive processing rate to cope with the impact of proportional changes in requests of different business types on container processing rate; B) Proposing a feedforward control method based on prediction; C) Proposing a feedback control method based on queue length perception; Considering the queue length of each microservice, a Jackson queuing network is constructed to accurately calculate the impact of bottleneck services on other services; D) A virtual machine provisioning strategy based on linear programming to maximize resource utilization. The present invention is applied to a mesh microservice system deployed based on Kubernetes. Through the above method, the system service quality can be greatly improved.

Description

Technical Field

[0001] The present invention belongs to the field of cloud computing resource supply, and in particular relates to a meshed microservice resource control method based on adaptive processing rate in Kubernetes. Background Art

[0002] Microservices-based architecture is conducive to building flexible software systems. Since containers are lighter and more convenient than virtual machines (VMs), deploying microservices in containers rented from public cloud providers can provide elastic processing capabilities. Kubernetes (K8s for short) is a popular container application orchestration and management system. Many public cloud vendors provide K8s-based container application running platforms, such as Amazon Elastic Kubernetes Service (Amazon EKS) and Alibaba Cloud Kubernetes (ACK).

[0003] When applications are decoupled into multiple microservices and deployed in a K8s cluster, how to provide appropriate resources for each microservice is a key issue. Most container scheduling algorithm studies based on the K8s platform focus on resource scheduling methods for single batch tasks or mixed loads of batch tasks and Web systems, dynamically applying for and releasing Pods to reduce resource costs. In these works, they assume that batch jobs and Web systems consume a fixed amount of resources. However, for Web systems, the resources consumed by the system change with changes in load. Many studies have proposed some resource scaling strategies for single microservices. The built-in HPA (Horizontal Pod Autoscaler) of K8s is a threshold-based resource scaling strategy. HPA adjusts the number of pods based on CPU or memory utilization. It responds slowly to performance changes. Toka et al. proposed HPA+ (Toka L, Dobreff G, Fodor B, et al. Adaptive AI-based auto-scaling for Kubernetes [C] / / 2020 20th IEEE / ACM International Symposium on Cluster, Cloud and Internet Computing (CCGRID). IEEE, 2020: 599-608.), which links CPU consumption to the number of user requests. It uses LSTM (Long Short-Term Memory) to predict the number of requests to improve the system's ability to handle changing loads. However, purely threshold-based methods always result in control delays. By dynamically adjusting the number of allocated containers, the hybrid method of queuing theory and control theory (Cai Z, Buyya R. Inverse Queuing Model based Feedback Control for Elastic Container Provisioning of Web Systems in Kubernetes [J]. IEEE Transactions on Computers, 2021: 1. DOI: 10.1109 / TC.2021.3049598.) can almost make the average response time follow the given reference time. However, these methods are designed for single-layer systems and do not consider the impact between meshed microservices.

[0004] Only a few existing works consider mesh service systems, and most of them consider resource allocation or migration without considering the calling relationship between services. DAI et al. (DAI Zhiming, ZHOU Mingtuo, YANGYang, LI Jian, LIU Jun. Fog computing resource scheduling in intelligent factories [J]. Journal of University of Chinese Academy of Sciences, 2021, 38 (5): 702-711.) used the Socks Shop mesh microservice container system to conduct simulation experiments, and based on the genetic algorithm IDGSA to guide the allocation of resources for containers for processing tasks and schedule the location of containers. Xu et al. (Xu M, Buyya R. Brownout Con: A software system based on brownout and containers for energy-efficient cloud computing [J]. Journal of Systems and Software, 2019, 155: 91-103.) combined the sliding window strategy to scale the container resources of each microservice in Socks Shop, but the main consideration was to use the disconnection strategy when the load increased to ensure system performance and save energy consumption. Jackson queuing network (JQN) is widely used to model the impact between meshed multi-service queues. For example, the JPRM proposed by Lei et al. (Lei Y, Cai Z, Wu H, et al. Cloud Resource Provisioning and Bottleneck Eliminating for Meshed Web Systems [C] / / 2020 IEEE 13th International Conference on Cloud Computing (CLOUD). IEEE, 2020: 512-516.) uses the Holt-Winter prediction model plus JQN to predict the arrival rate of each layer and the impact of the bottleneck layer on other layers, and uses the M / M / N queuing model to estimate the number of leased virtual machines. However, JPRM does not consider the queue length of each layer when calculating the impact of the bottleneck layer on other layers. At the same time, in a real production environment, synchronous calls to other services will occupy some server threads and connections. Synchronous calls complicate the estimation of container processing capacity and the identification of system bottlenecks. Summary of the invention

[0005] The purpose of the present invention is to provide a mesh microservice resource control method based on adaptive processing rate in Kubernetes, which is aimed at a mesh microservice system in a cloud based on Kubernetes, and takes into account the negative impact of synchronous calls on container performance; a queue length-aware Jackson queuing network is used to quantify the impact of the bottleneck layer on other microservices.

[0006] The technical solution to achieve the purpose of the present invention is: a mesh microservice resource control method based on adaptive processing rate in Kubernetes, comprising the following steps:

[0007] A) Establish a queuing model with adaptive processing rate for the microservice cluster based on the proportion of synchronous calls generated by the service;

[0008] B) performing prediction-based feedforward control according to the pricing interval of the virtual machine to generate a call probability matrix between microservices;

[0009] C) performing feedback control based on queue length perception according to the control interval;

[0010] D) Virtual machine provisioning strategy based on linear programming: Linear programming is used to calculate the number of virtual machines that can just meet the container demand; the strategy requires the immediate leasing of the required virtual machines, and the release of virtual machines needs to wait until the next pricing interval.

[0011] Further, step A) is as follows:

[0012] Given a system: S = {S i |i∈{1,2,3...,n}}, where S is the set of services in the system; B={b l |l∈{1,2,3...,L}}, where B is the set of services in the system; b l =(S l,1 ,...,S l,k ,...), S l,k ∈S is b l The kth service accessed in the query; synchronously calling other services by a service will reduce its own request processing rate; a queuing model with adaptive processing rate is established according to the different services accessed by users.

[0013] Further, step B) performs a prediction-based feedforward control method according to the pricing interval of the virtual machine, and the process is as follows:

[0014] Statistics on historical user access data are collected, and the user request arrival rate for the next interval is predicted using the load prediction method; the appropriate number of containers is leased and supplied based on the predicted request arrival rate and the queuing model in step A.

[0015] Furthermore, in step C), the feedback control method is executed between the feedforward control method to detect the system bottleneck service; and the call probability matrix Ψ between services is generated according to the user access business ratio, and the influence between services is quantified using the queue length-aware JQN to calculate the final request arrival rate; and the appropriate number of containers is supplied according to the queuing model in step A to ensure the stability of the system.

[0016] Further, in step D), according to the number of containers in steps B) and C), N is the sum of Pods required by all services in the system. i For Service i The number of Pods required; linear programming is used to calculate the number of virtual machines that can just meet the container requirements; the strategy requires that the required virtual machines be leased immediately, and the release of virtual machines needs to wait until the next pricing interval.

[0017] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for controlling meshed microservice resources based on adaptive processing rate in Kubernetes is implemented.

[0018] Compared with the existing technology, the advantages of the present invention are: (1) the adaptive processing rate calculation method considering the change of business type ratio is helpful to improve the accuracy of the queuing model; (2) the prediction-based feedforward control method can prevent the negative impact of load changes; (3) the queue length-aware JQN feedback control method can improve system stability by considering the impact of newly arrived requests and accumulated requests at the same time; (4) the resource supply strategy combining containers and virtual machines can control the rental cost while ensuring the quality of service.

[0019] The present invention is further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a diagram of the resource management architecture based on Kubernetes.

[0021] Figure 2 This is a schematic diagram of examples of users accessing different services.

[0022] Figure 3 It is the activity diagram of the feedforward control method based on the prediction method.

[0023] Figure 4 It is the activity diagram of the feedback control method based on queue length perception. DETAILED DESCRIPTION

[0024] The present invention improves the hybrid resource configuration architecture of JPRM with feedforward control and feedback control. In the test based on the actual production platform, the processing capacity of the container for requests under different business types varies. To this end, the present invention proposes a queuing model based on adaptive processing rate, which dynamically adjusts the processing rate of the container according to the proportion of synchronous calls. In the feedforward control algorithm part, the request arrival rate of each service is predicted by the time series analysis method. Then the minimum number of containers required for each service is obtained by the adaptive queuing model. When the resources of some services become bottlenecks due to insufficient supply or other reasons, the requests to call other services through the service will be limited by the total processing rate of the service (less than the arrival rate). Many requests will be blocked in the service request queue. JPRM assumes that the increased rate of calling other services is equal to the arrival rate minus the original processing rate, which ignores that there are many tasks in the bottleneck layer queue, and these tasks will also generate calls to other layers. Therefore, the present invention will consider the impact of queued tasks. In the feedback control algorithm part, JQN is used to consider the impact of the bottleneck layer (service) on other layers, and the bottleneck is quickly eliminated. Compared with other methods, the service quality can be improved.

[0025] The present invention will be further described below in conjunction with specific embodiments and accompanying drawings.

[0026] Figure 1 The Kubernetes resource management architecture in the present invention is described. The Master node is responsible for managing the K8s cluster. Each container is usually embedded in a Pod, which is the smallest scheduling unit in K8s. The Pod needs to be deployed in the node, and each node is a virtual machine rented from a cloud service provider. The container of each service may come from different Worker nodes. Traefik is a front-end load balancer responsible for distributing requests to access services in the system to back-end containers. The present invention builds a Kubernetes Controller resource controller, in which the Resource Auto-scaler component connects to the K8s cluster through an API to dynamically adjust the number of nodes or Pods according to the algorithm; the Log Manager collects and analyzes the logs of the front-end load balancer to obtain information such as the average response time and queue length of each service.

[0027] The present invention proposes a mesh microservice resource control method based on adaptive processing rate in Kubernetes, which is oriented to the mesh microservice system in the cloud and takes into account the negative impact of synchronous calls on container performance. The queue length-aware Jackson Queuing Network (JQN) is used to quantify the impact of the bottleneck layer on other microservices.

[0028] The Kubernetes Controller, a resource controller for mesh microservice systems, dynamically adjusts resources based on user load, including the following steps:

[0029] Step 1, such as Figure 2 An example of a user accessing a service in the system is shown. Different users have different service access paths when calling system services. Based on the proportion of synchronous calls generated by the service, a queuing model with an adaptive processing rate is established for the microservice cluster. The process is as follows:

[0030] Step 1.1, assuming μ i is the original processing rate when there is no synchronous call, ρ i It is the processing rate reduction rate when the service generates synchronous calls. Therefore, in business type b l Next S i The processing rate is calculated as μ i,l =μ i ×α l ,b l ∈B i , B i It contains S i A collection of businesses, α l is the coefficient; when business b l S i When a synchronous call is made, α l =ρ i , otherwise α l =1; therefore, business b l Medium S i The average processing time is Simultaneous service S i In b l The request arrival rate in the business is λ i,l , S i The total request arrival rate is λ i Therefore S i The response time expectations for all requests are:

[0031]

[0032] S i The container handling rate is updated to:

[0033]

[0034] Step 1.2: Based on the M / M / N queuing model, service S i The probability that there is no request in

[0035]

[0036] The expected response time is

[0037]

[0038] Step 1.3 Based on the above formula, a minimum response time is given S i The minimum number of containers required is

[0039]

[0040] Where Z + is the set of positive integers.

[0041] Step 2, such as Figure 3 The activity diagram of the prediction-based feedforward control method is shown. Execution of the prediction-based feedforward control method at pricing intervals includes the following steps:

[0042] Data input: original processing rate μ i , the processing rate decreases ρ i , reference response time

[0043] Step 2.1: Collect the total arrival rate λ of each service i and the service arrival rate λ i,l ;

[0044] Step 2.2: Use the Holt-Winter model to predict the arrival rate of the next time interval

[0045] Step 2.3, calculate the real-time processing rate of the service container based on the current combination of different business types according to the adaptive processing rate queuing model

[0046] Step 2.4: Based on the set reference response time, the predicted arrival rate and processing rate Substituting into the queuing model, we can get the number of containers N required for each service. i ;

[0047] In step 2.5, if the predicted processing rate is lower than the current processing rate, use the current arrival rate to calculate the number of containers required.

[0048] Step 2.6, based on the number of containers N obtained i To decide whether to add or release a container.

[0049] Step 3, such as Figure 4 The activity diagram of the feedback control method based on queue length perception is shown. The specific steps of the feedback control method based on queue length perception according to the control interval include:

[0050] Data Input: Processing Rate Response time limit W Si , the number of containers currently present

[0051] Step 3.1: Collect the total arrival rate λ of each service i and the service arrival rate λ i,l , the queue length q i ;

[0052] Step 3.2, when service S i The average response time is greater than the response time limit S i Determined to be the bottleneck layer;

[0053] Step 3.3, calculate the redundant calls to other layers (increased request rate) generated by the bottleneck layer by the following formula;

[0054]

[0055] Step 3.4, construct the matrix Ψ according to the load pattern of users calling different services, that is,

[0056]

[0057] where ψ ij (i,j≠0) is S i Call S j The ratio of; Ψ changes with the change of the proportion of various services in the system; therefore, in each feedforward control cycle, the matrix Ψ can be updated according to the current business mode;

[0058] Step 3.5, based on Ψ and Δλ, the final arrival rate λ′ i for:

[0059]

[0060] Step 3.6, each service S i The number of containers required N i It can be calculated through the queuing model;

[0061] Step 3.7. When N i Greater than the current number of containers Additional containers will be allocated to service S only when i .

[0062] Step 4: The specific steps of the virtual machine provisioning strategy based on linear programming are:

[0063] Step 4.1, H m,m∈{1,2,...,M} is the number of Pods (containers) that the mth virtual machine can accommodate; The sum of the number of Pods currently allocated to all services. Pods should be added; establish the integer linear programming (ILP) model shown below to solve the optimal number of virtual machines;

[0064] minΔC pod

[0065] stΔC pod >gap

[0066]

[0067] in is the number of virtual machines of type m that need to be leased.

[0068] Step 4.2, when Pods need to be released. The ILP model is as follows:

[0069] minΔC pod

[0070] stΔC pod <gap

[0071]

[0072] in is the number of released virtual machines of type m. Since virtual machines are generally priced by the hour, no matter when a virtual machine is decided to be released, the release operation will not be performed until the next pricing interval.

[0073] The present invention is compared with two HPAs built into K8s, namely HPA-CPU (using the CPU utilization threshold as the scaling basis) and HPA-Request (using the request arrival rate threshold as the scaling basis). The Kubernetes system includes Figure 2 The mesh microservice and business system shown. JMeter is used as a load generator, and requests are sent using user access data based on Wikipedia. The experiment lasts for 1200 minutes, the scheduling interval of the feedforward control method is 720 seconds, and the scheduling interval of the feedback control method is 180 seconds. By comparing the percentage of SLA (service level agreement) violation intervals of different controllers, the present invention achieves the best performance and ensures the service quality of the system.

Claims

1. A mesh microservice resource control method based on adaptive processing rate in Kubernetes, characterized in that: The following steps are involved: A) Based on the proportion of synchronous calls generated by the service, a queuing model with adaptive processing rate is established for the microservice cluster; the details are as follows: Given a system: S = {S i |i∈{1, 2, 3..., n}}, where s is the set of services in the system; B={b l |l∈{1, 2, 3..., L}}, where B is the set of services in the system; b l =(S i，1， ..., S l，k， ...), S i，k ∈S is b l The kth service accessed in the query; synchronously calling other services will reduce its own request processing rate; Establish a queuing model with adaptive processing rate according to different user access system services; Establish a queue model with adaptive processing rate. The specific steps include: A1. The queuing model is used to model the relationship between the processing rate μ, the number of servers N, the request arrival rate λ and the expected response time; assuming μ i is the original processing rate when there is no synchronous call, ρ i It is the processing rate reduction rate when the service generates synchronous calls; therefore, in business type b l Next S i The processing rate is calculated as μ i，l =μ i ×α l , b l ∈B i , B i It contains S i A collection of businesses, α l is the coefficient; when business b l S i When a synchronous call is made, α l =ρ i , otherwise α l =1; therefore, business b l Medium S i The average processing time is Simultaneous service S i In b l The request arrival rate in the business is λ i，l , S i The total request arrival rate is λ i ; Therefore S i The response time expectations for all requests are: S i The container handling rate is updated to: A2. Based on the M / M / N queuing model, service S i The probability that there is no request is: The expected response time is: A3. Based on the above formula, a minimum response time is given S i The minimum number of containers required is Where Z + is the set of positive integers; B) performing prediction-based feedforward control according to the pricing interval of the virtual machine to generate a call probability matrix between microservices; C) Performing feedback control based on queue length perception according to the control interval; the feedback control method is executed between the feedforward control method to detect the system bottleneck service; and generating the call probability matrix Ψ between services according to the user access service ratio, using the queue length-aware JQN to quantify the impact between services, and calculate the final request arrival rate; and supplying the appropriate number of containers according to the queue model in step A to ensure the stability of the system; the specific steps include: Data Input: Processing Rate Response time limit The number of containers currently present C1. Collect the total arrival rate λ of each service i , service arrival rate λ i，l And queue request length q i ; C2. When service S i The average response time of S is greater than the response time limit. i Determined to be the bottleneck layer; C3. The redundant calls to other layers generated by the bottleneck layer are calculated by the following formula; C4. Construct the matrix Ψ according to the load pattern of users calling different services, that is, where ψ ij YesS i Call S j The ratio of i, j≠0; Ψ changes with the change of the proportion of various services in the system; therefore, in each feedforward control cycle, the matrix Ψ can be updated according to the current business mode; C5. Based on Ψ and Δλ, the final arrival rate λ′ i It can be obtained by the following formula; C6. Each service S i The number of containers required N i Calculated by the queue model with adaptive processing rate; C7.When N i Greater than the current number of containers Additional containers will be allocated to service S only when i ; D) Virtual machine provisioning strategy based on linear programming: Linear programming is used to calculate the number of virtual machines that can just meet the container demand; the strategy requires the immediate leasing of the required virtual machines, and the release of virtual machines needs to wait until the next pricing interval.

2. According to claim 1, the mesh microservice resource control method based on adaptive processing rate in Kubernetes is characterized in that: Step B) Execute the prediction-based feedforward control method according to the pricing interval of the virtual machine, the process is as follows: Statistics on historical user access data are collected, and the user request arrival rate for the next interval is predicted using the load prediction method; the appropriate number of containers is leased and supplied based on the predicted request arrival rate and the queuing model in step A.

3. The method for controlling meshed microservice resources based on adaptive processing rate in Kubernetes according to claim 2, characterized in that: Executing the forecast-based feedforward control method at pricing intervals includes the following steps: Data input: original processing rate μ i , the processing rate decreases ρ i , reference response time B1. Collect the total arrival rate λ of each service i and the service arrival rate λ i，l ; B2. Use the Holt-Winter model to predict the arrival rate of the next time interval B3. Calculate the real-time processing rate of containers for each service based on the current combination of different business types B4. Based on the set reference response time, the predicted arrival rate and processing rate Get the number of containers N required for each service i ; B5. If the predicted processing rate is lower than the current processing rate, use the current arrival rate to calculate the number of containers required; B6. According to the number of containers N obtained i To decide whether to add or release a container.

4. The method for controlling meshed microservice resources based on adaptive processing rate in Kubernetes according to claim 1, characterized in that: In step D), according to the number of containers in steps B) and C), N is the sum of Pods required by all services in the system. i For Service i The number of Pods required; linear programming is used to calculate the number of virtual machines that can just meet the container requirements; the strategy requires that the required virtual machines be leased immediately, and the release of virtual machines needs to wait until the next pricing interval.

5. According to claim 4, the method for controlling meshed microservice resources based on adaptive processing rate in Kubernetes is characterized in that: The virtual machine provisioning strategy based on linear programming has the following specific steps: D1.H m , m∈{1,2,...,M} is the number of Pods that the mth virtual machine can accommodate; The sum of the number of Pods currently allocated to all services. Pods should be added; an integer linear programming model is established as shown below to solve the optimal number of virtual machines; min ΔC pod s.t.ΔC pod >gap in is the number of virtual machines of type m that need to be leased; D2. When Pods need to be released; the ILP model is as follows: min ΔC pod s.t.ΔC pod <gap in is the number of virtual machines of type m released; no matter when a virtual machine is decided to be released, the release operation will not be performed until the next pricing interval.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method for controlling meshed microservice resources based on adaptive processing rate in Kubernetes is implemented.

Images