Microservice resource allocation method based on combined neural network

Abstract

The present invention discloses a microservice resource allocation method based on a combined neural network. The present invention identifies the dependencies of microservices and the dynamic characteristics of a cloud environment based on a combined neural network, predicts and locates microservices that cause performance defaults, and locks their restricted bottleneck resources, and allocates resources for bottleneck resources, so as to maximize the resource utilization rate of a data center while reducing performance defaults. The microservice resource allocation method based on a combined neural network provided by the present invention can well capture the drastic changes in workload characteristics and performance interference patterns in a cloud environment, accurately predict and locate microservices that are about to cause performance defaults, and allocate their bottleneck resources, so as to maximize the utilization rate of resources while reducing the default rate of online services.

Description

Technical Field

[0001] The present invention relates to a microservice resource allocation method, in particular to a microservice resource allocation method based on a combined neural network. Background Art

[0002] Cloud computing allows tenants to rent resources to deploy their applications. Both public and private cloud services are hosted on multiple servers in cloud data centers. Data shows that the resource utilization rate of most cloud service data centers is only 10% to 30%. In order to improve server utilization, some studies have proposed to deploy latency-sensitive online services and offline jobs in a mixed manner. Offline jobs are generally based on computing logic and are not sensitive to latency. They mainly include data analysis jobs related to distributed computing models such as MapReduce, data warehouse jobs such as Hive, and jobs related to machine learning model training. Although colocation technology can improve the utilization of data centers, it also brings huge challenges. During the simultaneous operation of colocation loads, offline jobs on the underlying servers compete for resources and cause interference, resulting in a decrease in online service performance or even a default.

[0003] At the same time, with the popularity of microservices, online services are gradually transitioning from single applications to microservice architecture. Microservice architecture provides many benefits, such as the ability to independently deploy, scale, and develop individual services in different programming languages, making applications more modular, easier to understand, develop, and test. However, due to the variability of dynamic workloads, the cascading effects that may result from complex interactions between microservices, etc., it becomes more difficult to predict performance degradation and adjust resources. Currently, the commonly used solutions mainly use machine learning models to link observable resource usage indicators or resource allocation indicators with application performance. However, these models are difficult to adapt to dynamic changes in cloud environments, such as drastic changes in workload characteristics and performance interference patterns. Summary of the invention

[0004] Since the current resource allocation method based on the mixed deployment of microservices cannot quickly adjust the performance prediction model to adapt to the dramatic changes in workload characteristics and performance interference patterns. Therefore, the present invention uses a combined neural network for prediction, uses CNN (convolutional neural network) to determine the dependencies between microservices, and uses LSTM (long short-term memory artificial neural network) to identify dynamic timing features, so as to achieve good prediction results, accurately locate the microservices that are about to cause defaults and allocate resources for bottleneck resources, so as to reduce the default rate of online services and improve the utilization of resources as much as possible.

[0005] The specific steps of the method of the present invention are:

[0006] Step 1. Use a distributed tracing system to track user requests to access online services, and identify the microservice request path graph that makes up the online service based on the system call requests and call timestamps.

[0007] Step 2. Collect data, collect online service defaults, record the latency, number of unprocessed requests, and throughput of each microservice. And for each microservice, collect hardware indicators of its CPU, memory, IO, memory bandwidth, three-level cache, and other resources, and finally store the data in the database after preprocessing.

[0008] Step 3. Use the collected data to train a combined neural network (CNN+LSTM) online to locate the microservices that are about to cause a breach.

[0009] The combined neural network uses a 4-layer CNN to process high-dimensional data, determine the dependencies between microservices, screen out key microservices and key features related to performance defaults, and uses a 3-layer LSTM to identify dynamic timing features in the cloud environment. The combined neural network inputs the latency, number of unprocessed requests, throughput, and collected hardware indicators of each microservice, and finally outputs the probability of each microservice causing performance default. At the same time, the probability factor is used to perform gradient descent updates during the back propagation process of network training to improve the versatility of the model. When a microservice is replaced, the network is split into a base layer and an update layer to reuse the incremental update method of the base layer weight coefficient and the update layer weight coefficient to reduce training time.

[0010] Step 4. Determine the prediction accuracy of the combined neural network. If the prediction accuracy does not meet the set threshold requirement, use the proxy model, i.e., the threshold control model, to replace the combined neural network for prediction, to make up for the defect of long online training time of the combined neural network. If the prediction accuracy of the combined neural network meets the requirement, the prediction result is directly output to locate the i-th microservice S that is about to cause default. i .

[0011] Step 5. After locating the microservice that is about to cause a default, cluster the non-default samples in the hardware indicator state space of this microservice through the binary k-means method. Traverse all samples of the same category as the current hardware indicator sample, find the sample closest to the current hardware indicator sample based on the Mahalanobis distance d between samples, and accurately identify the bottleneck resource R = (R1,…,R j ,…,R l ).

[0012] Step 6. Allocate resources for the bottleneck resources identified in step 5, starting from the allocation degree A i Minimal batch job C i Start freeing up resources for use by microservices.

[0013] Step 7. Repeat steps 1 to 6 periodically.

[0014] Beneficial effects of the present invention: The microservice resource allocation method based on combined neural networks provided by the present invention can well capture the drastic changes in workload characteristics and performance interference patterns in the cloud environment, accurately predict and locate microservices that are about to cause performance defaults, and allocate their bottleneck resources, thereby maximizing resource utilization while reducing the online service default rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is the overall architecture diagram of the present invention;

[0016] Figure 2 This is an example of a microservice request path graph;

[0017] Figure 3 is a combined neural network graph. DETAILED DESCRIPTION

[0018] The present invention tracks user requests through a distributed tracking system, identifies a request path graph based on microservices, and records indicators related to online service performance. In the present invention, for each microservice, it is necessary to record its delay, number of unprocessed requests, throughput, and a series of hardware indicators. The combined neural network is trained with these data to predict the microservices that are about to cause default. Since the online training time of the combined neural network is long, the present invention proposes a proxy model based on threshold control to replace its work during the combined neural network training phase. After the microservice is located, the samples that have not defaulted in the state space of the hardware indicator of the microservice can be clustered, and the Mahalanobis distance can be used to identify the samples in the same class that are closest to the current hardware indicator samples. The bottleneck resources are identified by comparison, and the resources are allocated based on the allocation degree.

[0019] The microservice resource allocation method based on combined neural network proposed in the present invention has the following steps:

[0020] Step 1. Use a distributed tracing system to track user requests to access online services, and identify the microservice request path graph that makes up the online service based on the system call requests and call timestamps.

[0021] Step 2. Collect data, collect online service defaults, record the latency, number of unprocessed requests, and throughput of each microservice. And for each microservice, collect hardware indicators of its CPU, memory, IO, memory bandwidth, three-level cache, and other resources, and finally store the data in the database after preprocessing.

[0022] Step 3. Use the above collected data to train the combined neural network (CNN+LSTM) online to locate the microservices that are about to cause defaults. The combined neural network uses a 4-layer CNN to process high-dimensional data, determine the dependencies between microservices, screen out key microservices and key features related to performance defaults, and uses a 3-layer LSTM to identify dynamic timing features in the cloud environment. The input of the combined neural network is the latency, number of unprocessed requests, throughput, and collected hardware indicators of each microservice, and finally outputs the probability of each microservice causing performance default. At the same time, the probability factor is used for gradient descent update during the back propagation process of network training to improve the versatility of the model. When the microservice is replaced, the network is split into a base layer and an update layer to reuse the incremental update method of the base layer weight coefficient and the update layer weight coefficient to reduce training time.

[0023] Step 4. Determine the prediction accuracy of the combined neural network. If the prediction accuracy does not meet the set threshold requirement, use the proxy model, i.e., the threshold control model, to replace the combined neural network for prediction, to make up for the defect of long online training time of the combined neural network. If the prediction accuracy of the combined neural network meets the requirement, the prediction result is directly output to locate the i-th microservice S that is about to cause default. i .

[0024] Step 5. After locating the microservice that is about to cause a default, cluster the non-default samples in the hardware indicator state space of this microservice through the binary k-means method. Traverse all samples of the same category as the current hardware indicator sample, find the sample closest to the current hardware indicator sample based on the Mahalanobis distance d between samples, and accurately identify the bottleneck resource R = (R1,…,R j ,…,R l ).

[0025] Step 6. Allocate resources for the bottleneck resources identified in step 5, starting from the allocation degree A i Minimal batch job C i Start freeing up resources for use by microservices.

[0026] Step 7. Repeat steps 1 to 6 periodically.

[0027] The present invention is further described below in conjunction with an embodiment. This embodiment involves four modules, namely: a request tracking and data collection module, a microservice default prediction module, a bottleneck resource location module, and a bottleneck resource adjustment module.

[0028] Request tracking and data collection module: Tracks user requests, identifies the request path of online services through microservices, and collects various performance-related indicators. After preprocessing, they are stored in the database for subsequent use, such as Figure 1 As shown in ①.

[0029] Microservice default prediction module: It uses a combined neural network to predict and locate microservices that cause performance defaults. It does not require prior information about the application and is completely data-driven. During its online training, a threshold-controlled proxy model is proposed to replace its work, thereby making up for the long training time of the combined neural network. Figure 1 As shown in ②.

[0030] Bottleneck resource location module: Cluster the non-violated samples in the state space of the underlying indicators of the located microservice, thereby narrowing the search space. Find the most similar samples in the class to which the current sample belongs, and after comparison, the bottleneck resource can be locked, such as Figure 1 As shown in ③.

[0031] Bottleneck resource adjustment module: Use the corresponding hardware technology to allocate bottleneck resources based on the allocation degree. The time and resource coefficients can be adjusted, such as Figure 1 As shown in ④.

[0032] In this embodiment, it is assumed that there are 1…i…n microservices, S i represents the i-th microservice; each microservice uses 1…i…m types of resources, R i represents the i-th resource; each microservice is co-located with 1…i…q offline jobs, C i represents the i-th batch job, A i represents the allocation degree of the i-th batch processing job, and the steps of this embodiment are as follows:

[0033] (1) Step 1

[0034] Use a distributed tracing system to track user requests to access online services, and identify the request path graph of the microservices that make up the online service through information such as the message identifiers and timestamps sent and received, such as Figure 2 shown.

[0035] (2) Step 2

[0036] This step collects data. According to the above tracking system, the latency and throughput of each microservice are recorded in sequence. Since the request queue length is highly correlated with performance and performance default, this step also collects the number of unfinished requests in each microservice. For each microservice, it is also necessary to collect hardware indicators of resources such as CPU, memory, IO, memory bandwidth, and three-level cache for use in subsequent steps. Finally, the collected data is preprocessed and stored in the database.

[0037] (3) Step 3

[0038] The CNN+LSTM combined neural network is trained online using the above collected data to locate the microservices that are about to cause default. Locating the microservices that are about to cause default requires both identifying the dependencies between microservices and using time series features to use past defaults to predict future defaults. Since CNN can handle high-dimensional data sets well and can perform automatic feature extraction, and LSTM has a long-term memory function and has great advantages in identifying sequence rules, this embodiment combines CNN and LSTM to construct a combined neural network to achieve better prediction results. Specifically:

[0039] First, a 4-layer CNN is used to filter out microservices and non-critical indicators that will not affect service performance, and then a 3-layer LSTM is used to capture dynamic features in the cloud environment. The overall network structure is as follows: Figure 3 As shown, the input of the combined neural network is the latency, throughput, number of unprocessed requests and a series of hardware indicators of each microservice, and finally outputs the probability of each microservice causing performance default. For CNN, each layer contains two functions: convolution and activation. The convolution function uses convolution kernels for feature extraction and feature mapping. The excitation function mainly performs nonlinear mapping on the output of the convolution, and the excitation function used is the ReLu function with a faster convergence speed. In addition, the size and number of convolution kernels can be adaptively adjusted during the training process. For LSTM, each layer of LSTM includes multiple LSTM units, and each unit consists of three control switches: forget gate, input gate, and output gate. Assume that at time t, the unit has three inputs: the input value x at the current moment t , the output value h at the previous moment t-1 , and the cell state c at the previous moment t-1 ; The unit has two outputs: the current output value h t , and the current cell state c t .

[0040] The calculation formula of the forget gate is as follows, where f t is the forgetting factor, W f is the weight matrix of the forget gate, [h t-1 ,x t ] means connecting two vectors into a longer vector, b f is the bias term of the forget gate, and σ is the sigmoid function:

[0041] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0042] The calculation formula of the input gate is as follows, where W iis the weight matrix of the input gate, b i is the bias term of the input gate, W c is the weight matrix for calculating the unit state, b c is the bias term for computing the cell state, i t is the update factor, For the new candidate unit state:

[0043] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0044]

[0045]

[0046] The calculation formula of the output gate is as follows, where W o is the weight matrix of the output gate, b o is the bias term of the output gate, o t is the output factor:

[0047] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0048] h t =o t ·tanh(c t )

[0049] At the same time, this embodiment proposes probability factors and incremental updates to further improve the versatility of the combined neural network:

[0050] 1) Probability Factor

[0051] There is a back-propagation process in the training of the combined neural network, but sometimes only a local optimal solution can be achieved by updating through gradient descent. Therefore, this embodiment proposes a probability factor ∈, and performs gradient descent updates with a certain probability ∈, otherwise a random update is performed, so as to improve the availability and versatility of the prediction model.

[0052] 2) Incremental Update

[0053] When microservices are replaced, the prediction accuracy of the combined deep network will decrease, so the network needs to be retrained. In order to significantly reduce the network training time, the network weight parameters trained each time can be saved in the database. Figure 3As shown in , CNN and LSTM can be split into a base layer and an update layer respectively. When the network needs to be retrained, the network weight parameters of the base layer can be obtained from the database and used directly, and only the network weight coefficients of the update layer need to be retrained.

[0054] (4) Step 4

[0055] Since the online training time of the combined neural network is relatively long, it cannot be used directly. Therefore, this embodiment proposes a proxy model to replace the combined neural network during training. After the accuracy of the combined neural network meets the set threshold requirements, the proxy model automatically transfers the prediction work. The proxy model itself is a threshold control model that can set the default threshold of each microservice. The proxy model calculates the current value of each microservice based on its current latency, number of unprocessed requests, and throughput. Once the current value of a microservice exceeds its default threshold, it is considered that the microservice will cause a performance default of the online service.

[0056] (5) Step 5

[0057] After accurately locating the microservice that is about to cause a default, it is necessary to analyze the resource interference it is experiencing to identify the bottleneck resources. Since the request tracking and data collection module collects a series of hardware indicators, a hardware indicator state space can be maintained for each microservice. Each microservice uses 1…i…m types of resources, R i represents the i-th resource, then the currently observed microservice indicator sample is

[0058] Cluster all non-default samples in the hardware indicator state space. In this embodiment, the binary k-means method is used as the clustering method. It uses SSE (Sum of Squared Error) as an indicator to measure the clustering effect. First, all data points are regarded as a cluster, and then the cluster is divided into two. Then one of the clusters is selected to continue the division. The selection of which cluster to divide is based on the principle of minimizing SSE. Secondly, among all samples belonging to the same class as the current sample, the distance between samples is used to find the sample closest to the current sample. Compare the values ​​of each resource in the two samples. The resources whose values ​​differ by more than the specified resource threshold are bottleneck resources. The resource threshold can be customized. In this embodiment, the Mahalanobis distance is used as the distance between samples, as shown in the following formula. The distance d between samples x and y is defined as follows, and S is the covariance matrix:

[0059]

[0060] (6) Step 6

[0061] For the above samples x and y, assume that the current bottleneck resource identified is R = (R1,…,Rj ,…,R l ). Then the actual amount of resources required by the current microservice is

[0062] Step 6.1 For each bottleneck resource, the bottleneck resource adjustment module first detects whether there are excess idle resources to be allocated. If there are, they are directly allocated to the microservice. At this time, the required resources can be updated to For the existing idle resources.

[0063] Step 6.2 If the microservice resources are still not satisfied ( If there is a value greater than 0 in , the batch processing job currently co-located with the microservice needs to give up resources. At this time, collecting the batch processing job indicators co-located with the microservice is different from continuously collecting all the co-location job data. After locating the microservice that caused the breach and determining the resources it needs, collecting the co-location job indicators can greatly reduce the cost of data collection. The indicators collected here are the same as the hardware indicators of the microservice, including CPU, memory, IO, memory bandwidth, three-level cache and other resources, a total of m types.

[0064] Step 6.3 After collecting the indicators of the batch processing jobs in the hybrid deployment, you need to select the batch processing jobs that release resources. When selecting, first measure whether the resources released by the batch processing job match the resource values ​​required by the microservice, and then consider the waiting time for the batch processing job (giving priority to releasing the resources of batch processing jobs with longer waiting times). Therefore, α is defined as the resource coefficient and β is the time coefficient. These two coefficients are adjustable. The resource usage indicator of the i-th batch processing job is The time to be completed is T i , define the allocation degree A i for:

[0065]

[0066]

[0067] in express, indicates, s indicates

[0068] Based on the distribution degree A i Allocate resources and distribute the batch processing jobs of the hybrid deployment according to the allocation degree A. i Sort in ascending order and release batch job resources in order until the resource requirements are met.

[0069] (7) Step 7

[0070] Repeat steps 1 to 6 periodically.

Claims

1. A microservice resource allocation method based on a combined neural network, characterized in that The method comprises the following steps: Step 1. Use a distributed tracing system to track user requests to access online services, and identify the microservice request path graph that constitutes the online service based on system call requests and call timestamps; Step 2. Record the latency, number of unprocessed requests, and throughput of each microservice, and collect hardware metrics for each microservice. Step 3. Online training of the combined neural network to locate the microservices that are about to cause a breach; The combined neural network uses a four-layer CNN network to process high-dimensional data, determine the dependencies between microservices, and screen out key microservices and key features related to performance violations; Use a three-layer LSTM network to identify dynamic timing features in cloud environments; The combined neural network input is the latency, number of unprocessed requests, throughput and collected hardware indicators of each microservice; Output the probability of causing performance violation for each microservice; Step 4. Determine the prediction accuracy of the combined neural network; If the prediction accuracy does not meet the set threshold, a proxy model, i.e., a threshold control model, is used to make predictions instead of the combined neural network; If the prediction accuracy meets the requirement, the prediction result is directly output to locate the microservice that is about to cause default; Step 5. After locating the microservice that is about to cause a default, cluster the non-default samples of the hardware indicator state space of this microservice using the binary k-means method; Traverse all samples of the same category as the current hardware indicator sample, find the sample closest to the current hardware indicator sample based on the Mahalanobis distance d between samples, and accurately identify the bottleneck resource by comparing the two samples; Step 6. Allocate resources for the bottleneck resources identified in step 5, and release resources for microservices starting from the batch job with the smallest allocation degree; Step 7. Repeat steps 1 to 6 periodically.

2. The microservice resource allocation method based on combined neural network according to claim 1 is characterized in that: In step 1, the request path graph of the microservices that make up the online service is identified through the identifiers and timestamps of the sent and received messages.

3. The microservice resource allocation method based on combined neural network according to claim 1 is characterized in that: In step 3, the probability factor is used for gradient descent update during the back propagation process of network training.

4. The microservice resource allocation method based on combined neural network according to claim 1 is characterized in that: In step 3, when a microservice is replaced, each network is split into a base layer and an update layer, so as to reduce the training time by reusing the base layer weight coefficient and updating the update layer weight coefficient in an incremental update manner.

5. The microservice resource allocation method based on combined neural network according to claim 1 is characterized in that: In step 6, assign degree A i Defined as Among them, α is the resource coefficient, β is the time coefficient, R for Microservices j The demand, R j is the jth resource, is the R owned by the i-th batch job j resource quantity, f(s) is the resource matching function, s is the independent variable, T i is the time to be completed for the i-th batch job.

Images