Load-aware multi-micro-service replica elastic scaling method in computing power network

Abstract

The invention discloses a load-aware multi-micro-service replica elastic scaling method in a computing power network, and aims to cope with a dynamic online working load environment, ensure that end-to-end response time delay meets a service level agreement (SLA) and improve the utilization rate of micro-service resources at the same time. According to the method, firstly, historical performance data of all micro-services in a Kubernetes cluster are collected through plug-ins such as Prometheus, cAdvisor and Istio, the data are stored in a time sequence database, and a training data set is constructed; then, a graph data structure is constructed according to the calling relation between the micro-services and the micro-service features in the data set, the multi-micro-service CPU utilization rate and a P90 response delay predictor are trained through a graph attention network, and therefore an intelligent agent interactive simulation environment is formed. Based on the simulation environment, a deep context multi-arm bandit model is designed and trained, and an optimal copy adjustment strategy is selected in a large state space and a complex nonlinear relation in combination with an upper confidence bound (UCB) strategy, so that automatic horizontal stretching of the multi-micro service system is realized. And finally, deploying the trained model in an actual environment, regularly monitoring and updating context information of the micro-service through Prometheus, performing real-time copy adjustment, reducing an SLA violation risk and improving a resource utilization rate.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to an automatic horizontal scaling algorithm for multi-microservice applications with complex business logic intertwined in computing network scenarios. In this environment, workloads usually arrive online and are unpredictable and time-varying. The algorithm is designed to handle such complex and dynamically changing workloads, ensuring that the end-to-end request response time meets the service level agreement (SLA) requirements while improving resource utilization and optimizing the system's responsiveness and scalability. Background Art

[0002] With the rapid development of computer networks and cloud computing technologies, cloud applications are gradually evolving from traditional monolithic architectures to loosely coupled microservice architectures. Microservice architectures achieve higher scalability, flexibility, and rapid deployment and iteration capabilities by splitting large single applications into multiple independent, functionally clear microservice modules. Each microservice can be developed, deployed, and expanded independently, greatly improving the overall efficiency and flexibility of the system. However, the advantages of this architecture also bring complexity to resource management and performance optimization.

[0003] Currently, the online workload of microservice applications is highly dynamic and unpredictable, which makes the reasonable allocation of computing resources a huge challenge. As the workload changes, the application system needs to dynamically adjust the computing resource allocation of each microservice to reduce costs and improve resource utilization while ensuring that the end-to-end response latency meets the service quality (Service Level Agreement, SLA) requirements. At present, the microservice automatic scaling mechanism in the Kubernetes (K8S) cluster mainly relies on the CPU or memory utilization threshold strategy based on a single microservice. When the utilization exceeds the set threshold, it is expanded by adding container replicas. However, this independent expansion strategy fails to fully consider the interrelationships between multiple microservices, and the setting of the utilization threshold also relies on experience, which makes it difficult to accurately optimize according to the set SLA level.

[0004] Existing reinforcement learning (RL) methods have been widely used in dynamic decision-making problems, especially in scenarios where strategies need to be constantly adjusted according to environmental changes, such as the dynamic scaling of a single microservice. However, for multi-microservice scenarios with complex dependencies, existing methods have major deficiencies in capturing the coupling relationship between microservices and training efficiency. Existing RL methods usually rely on frequent interactions with the real environment to explore the optimal strategy, which is particularly difficult in multi-microservice applications. Since the horizontal scaling of multi-microservices involves a huge state and action space, action exploration in the cluster requires repeated exploration of the combination of scaling of multi-microservice replicas, and constantly creating and destroying replicas in the system, which causes unacceptable time overhead. In addition, frequent adjustments of microservice replicas can lead to significant response time fluctuations, which in turn affects the performance of the system. Therefore, in such a scenario with a huge state space and high operation cost, existing online exploration reinforcement learning methods are not only costly, but also difficult to effectively respond to changes in dynamic loads in real time.

[0005] In order to cope with the dynamic workload changes in multi-microservice architectures and improve resource utilization while ensuring that the end-to-end response latency meets the SLA, an efficient RL-based microservice automatic scaling method is urgently needed. This method should be able to effectively capture the coupling relationship between microservices and build an accurate simulated interaction environment, thereby improving training efficiency without directly interacting with the real environment. In addition, this method should also have the ability to handle large state and action spaces and quickly adapt to online workload changes. Summary of the invention

[0006] In response to the problems existing in the above-mentioned background technology, the present invention discloses a load-aware elastic scaling method for multiple microservice replicas in a computing power network, which improves the utilization rate of microservice resources while maximally ensuring that the end-to-end response delay meets the SLA requirements according to the continuous changes in the user's online request load.

[0007] Step 1: Based on the non-intrusive service mesh technology, design a historical indicator collector based on plug-ins such as Prometheus and Istio in the K8S cluster, sample the cluster historical data and use it as the subsequent graph attention network training dataset.

[0008] 1) Deploy Prometheus components and integrate cAdvisor and Istio. Deploy Prometheus in the Kubernetes cluster to monitor and collect performance indicators of each microservice, and integrate cAdvisor and Istio into Prometheus. cAdvisor is responsible for monitoring the CPU consumption of each microservice instance, while Istio provides traffic information and call chain data in the service grid, capturing the call relationship and latency distribution between microservices.

[0009] 2) Cluster microservice indicator data collection. Use the Locust tool to simulate time-varying workload requests with different numbers of users and multiple modes, and based on the random changes in the number of replicas, use the Prometheus query language to regularly collect comprehensive performance data. The set of microservices that meet the DAG dependency relationship is denoted as M = {m1, m2, m3...m N}, where N is the number of microservices, and the integrated Prometheus component is used to collect and classify the indicator data of each microservice. Since the creation and destruction of microservice replicas have a certain time overhead, and the sampling method of Prometheus is based on a specific time window, the sampling interval is set to 30 seconds. At each timestamp t, the data sequence sampled for each microservice can be expressed as in They represent the request arrival rate (QPS), CPU utilization, P90 delay and number of replicas of microservice m at time t respectively. We take P90 delay as the response delay of a single microservice. All collected indicator data will be stored in the time series database.

[0010] Step 2: Based on the time series database obtained in step 1, simulate the impact of changes in microservice workload and number of replicas in the real environment on microservice CPU utilization and P90 latency, train microservice CPU utilization and P90 latency predictors based on graph attention networks, and build a simulation environment that interacts with deep contextual multi-armed bandit machines.

[0011] 1) Construct a directed graph G = (M, E) based on the complex call relationship between multiple microservices, where the node set M corresponds to all microservices, and the edge set E represents the call relationship between microservices. The microservice indicator data of each timestamp t-1 is extracted from the time series database collected in step 1 to form the node feature vector at time t-1. The clear task is to use the currently known features of each node at time t-1 And the partial features of each node after the change at time t To train the model to obtain the remaining features of each graph node at time t Combine the features of each node to get the complete feature vector of each node's adjacent timestamps:

[0012]

[0013] 2) A two-layer graph attention network model is used to capture the complex dependencies between microservices. The call relationship between microservices is used to dynamically adjust the influence of neighbor nodes on the target node. The representation of each node is updated by aggregating the features of neighbor nodes, thereby accurately capturing the interaction between microservices. In the first layer of GAT, the node input features are linearly transformed and mapped to a high-dimensional hidden space. The attention weights between nodes are calculated to measure the influence of neighbor nodes. After normalization, the weighted sum of neighbor node features is used to obtain the updated node representation and a nonlinear activation function is applied. At the same time, a multi-head attention mechanism is used to enhance the model learning ability. The second layer of GAT further extracts high-order feature representations, aggregates neighbor node information by recalculating attention weights, and introduces residual connections between the two layers to alleviate the gradient vanishing problem. Batch normalization and Dropout operations are used after each layer to enhance model stability.

[0014] 3) Use the updated node representation for prediction and design two independent but identical networks. The output layer consists of a fully connected layer and a Sigmoid activation function to predict the microservice CPU utilization and P90 latency, respectively.

[0015] Step 3: Based on the model trained in step 2, a deep contextual multi-armed bandit algorithm is designed to interact with the model instead of the online environment, and efficient offline training is performed for the specific context information of each timestamp t to obtain a multi-microservice automatic scaling model that adapts to different load changes.

[0016] 1) Define the context features, action space, and reward function of multi-microservice applications.

[0017] First, the context feature S t Defined as:

[0018] S t =(W t , W t-1 , R t-1 )

[0019] Where W t , W t-1 represents the QPS of each microservice at time t and time t-1, R t-1 Represents the number of replicas of each microservice at time t-1. All three are N-dimensional vectors, and the value of the i-th dimension (i∈[1,N]) represents the value of the relevant performance indicator in the i-th microservice.

[0020] Then, the action space A t Defined as:

[0021]

[0022] in Indicates the microservice mi The scaling operation at the current moment can have a value of 1, -1, or 0, which corresponds to +1, -1, or unchanged number of replicas. Each action selection selects a scaling strategy for each microservice.

[0023] Next, the reward function is defined as:

[0024]

[0025] The specific value of SLA is the upper limit of the response delay value that meets the service level agreement. α and β represent custom weights, which measure the importance of improving the average CPU utilization of microservices and reducing the SLA violation rate. C (C>0) represents the penalty coefficient when violating the SLO. t It indicates the end-to-end P90 response delay of the microservice system. In multi-application microservices, it is manifested as the P90 delay of the front-end microservice.

[0026] The optimization goal is to select an appropriate replica scaling strategy for all microservices given the current context state, so as to maximize the average resource utilization of microservices while ensuring the SLA violation rate.

[0027] 2) Design a deep contextual multi-armed bandit model architecture. The microservice system consists of multiple services. Based on the contextual features designed above, the action space also expands rapidly with the increase in the number of microservices and optional actions due to its high spatial dimension. In order to adapt to the changes in different load patterns and enhance the generalization ability of the model, a deep neural network is introduced in CMAB to approximate the state-action value function Q(s, a; θ) to solve the problem of large state and action space.

[0028] 3) The microservice CPU utilization and P90 latency prediction model trained based on the graph attention mechanism in step 2 is used as the model environment for interacting with the deep context multi-armed bandit to quickly train the model, thereby optimizing the resource management of the microservice system. In step 2, the prediction of the key performance indicators of the microservice system is realized through the graph attention network (GAT). The model can predict the request arrival rate of each microservice at time t. And the number of copies after the change And the known time t-1 To predict the CPU utilization of each microservice and P90 response delay The DCMAB continuously repeats the training process, observing the current state, selecting actions, predicting performance, calculating rewards, and optimizing strategies until the rewards converge.

[0029] Step 4: Inject online time-varying workloads into the system, put the trained DCMAB into the actual environment, use the Prometheus plug-in to detect changes in microservice indicators at time intervals t and continuously update context information, input the real-time detected context information into the trained DCMAB, and prompt the model to make replica horizontal scaling decisions for each microservice at each timestamp t. After the set time T, the final application SLA violation and microservice resource utilization are obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Fig. 1 is a flow chart of the present invention;

[0031] FIG. 2 is a system architecture diagram of the present invention. DETAILED DESCRIPTION

[0032] As shown in FIG2 , the specific steps of the technical solution of the present invention are:

[0033] Step 1: Based on the non-intrusive service mesh technology, design a historical indicator collector based on Prometheus and Istio plug-ins in the K8S cluster, sample the cluster historical data and use it as the subsequent graph attention network training data set. The specific steps are as follows:

[0034] 1) Deploy Prometheus components and integrate cAdvisor and Istio. Deploy Prometheus in the Kubernetes cluster to monitor and collect performance indicators of each microservice, and integrate cAdvisor and Istio into Prometheus. cAdvisor is responsible for monitoring the CPU consumption of each microservice instance, while Istio provides traffic information and call chain data in the service grid, capturing the call relationship and latency distribution between microservices.

[0035] 2) Cluster microservice indicator data collection. Use the Locust tool to simulate time-varying workload requests with different numbers of users and multiple modes, and based on the random changes in the number of replicas, use the Prometheus query language to regularly collect comprehensive performance data. The set of microservices that meet the DAG dependency relationship is denoted as M = {m1, m2, m3...m N}, where N is the number of microservices, and the integrated Prometheus component is used to collect and classify the indicator data of each microservice. Since the creation and destruction of microservice replicas have a certain time overhead, and the sampling method of Prometheus is based on a specific time window, the sampling interval is set to 30 seconds. At each timestamp t, the data sequence sampled for each microservice can be expressed as in They represent the request arrival rate (QPS), CPU utilization, P90 latency, and number of replicas of microservice m at time t respectively. We take the P90 latency as the latency of a single microservice. All collected indicator data will be stored in the time series database.

[0036] Step 2: Based on the time series database obtained in step 1, simulate the impact of changes in microservice workload and number of replicas in the real environment on microservice CPU utilization and P90 latency, train microservice CPU utilization and P90 latency predictors based on graph attention networks, and build a simulation environment that interacts with deep contextual multi-armed bandits. The specific steps are as follows:

[0037] 1) According to the complex calling relationship between multiple microservices, a directed graph G = (M, E) is constructed, where the node set M corresponds to all microservices, and the edge set E represents the calling relationship between microservices. i Call m j , then there exists a directed edge (m i , m j ). Based on the time series database collected in step 1, the microservice indicator data of each timestamp t-1 is extracted to form the node feature vector of the previous moment Since the online load is constantly changing in the real environment, In order to build a simulation environment that interacts with the multi-armed bandit, we need to Simulate changing the number of replicas Contribution to immediate rewards, and the reward function and CPU utilization and P90 delay Therefore, the problem is transformed into: How to use the currently known features of each node at time t-1 And the partial features of each node known at time t To train the model to obtain the remaining features of each graph node at time t All the above features can be obtained from the time series database in step 1. Combine the features of each node to obtain the complete feature vector of adjacent timestamps of each node:

[0038]

[0039] By aggregating complete node features and call relationships, a complete graph data structure is formed for model training.

[0040] 2) A two-layer graph attention network model is used to capture the complex dependencies between microservices. The calling relationship between microservices is utilized to dynamically adjust the influence of neighbor nodes on the target node, and the representation of each node is updated by aggregating the features of neighbor nodes, thereby accurately capturing the interactions between microservices.

[0041] In the first layer of GAT, the input features of each node are Apply a linear transformation to map it into a higher dimensional latent space:

[0042]

[0043] in, is a trainable weight matrix used to map the input feature dimension d to the hidden dimension d′. Then, for each node m i and its neighbor node m j ∈N(m i ) Calculate the attention weight e ij , used to measure the neighbor node m j For node m i Impact:

[0044]

[0045] in, is a trainable attention vector, || represents a vector concatenation operation, and LeakyReLU is a nonlinear activation function used to introduce nonlinear features. Then, the attention weight is normalized by the softmax function:

[0046]

[0047] Normalized attention weight α ij Used to represent neighbor node m j For the target node m i The sum of all weights is 1, so that each node can dynamically adjust the influence of its neighboring nodes on itself. Next, the normalized attention weights are used to weight the sum of the features of the neighboring nodes, and the nonlinear activation function σ is applied to obtain the updated node representation:

[0048]

[0049] To improve the expressiveness of the model, a multi-head attention mechanism is used, that is, K independent attention heads are used to calculate feature representations at the same time. This can capture different types of dependencies between microservices, enhance the learning ability of the model, and make the feature representation more robust. These features are spliced ​​together:

[0050]

[0051] Node representation generated by the first layer of GAT It is further input into the second layer GAT to extract higher-order feature representation. The second layer GAT uses only one attention head and no longer performs concatenation:

[0052]

[0053] in, is the weight matrix of the second layer, α′ ij is the recalculated attention weight. In this way, the second layer of GAT can further aggregate the information of neighboring nodes to form a more comprehensive node representation.

[0054] In order to alleviate the gradient vanishing problem, a residual connection is introduced between the two layers of GAT. Specifically, the input feature After linear transformation, it is added to the output of the second layer GAT:

[0055]

[0056] in, is a linear transformation matrix. At the same time, in order to further enhance the stability of the model, batch normalization and dropout (randomly discarding a part of neurons) are used after each layer to prevent overfitting and improve generalization ability.

[0057] 3) Using the updated node representation for prediction, two independent but identical networks are designed to predict the CPU utilization and P90 latency of each microservice. Each network output layer consists of a fully connected layer and a Sigmoid activation function. The P90 latency is normalized. The CPU utilization and P90 latency predictions are expressed as:

[0058]

[0059] in and Both are trainable weight matrices. The independent design of the two networks allows them to focus on different objectives while sharing the same GAT structure, thereby improving the model's predictive power while optimizing training efficiency.

[0060] Step 3: Based on the model trained in step 2, a deep contextual multi-armed bandit algorithm is designed to interact with the model instead of the online environment, and efficient offline training is performed for the specific context information of each timestamp t to obtain a multi-microservice automatic scaling model that adapts to different load changes. The specific steps are as follows:

[0061] 1) Define the context features, action space, and reward function of multi-microservice applications.

[0062] First, the context feature St Defined as:

[0063] S t =(W t , W t-1 , R t-1 )

[0064] Where W t , W t-1 represents the QPS of each microservice at time t and time t-1, R t-1 Represents the number of replicas of each microservice at time t-1. All three are N-dimensional vectors, and the value of the i-th dimension (i∈[1,N]) represents the value of the relevant performance indicator in the i-th microservice. The significance of designing the context feature in this way is that it is necessary to decide how to scale each microservice replica at time t based on the microservice load at time t-1 and the microservice replica settings and the dynamically changed load at time t to maximize the immediate reward.

[0065] Then, the action space A t Defined as:

[0066]

[0067] in Indicates the microservice m i The scaling operation at the current moment can have a value of 1, -1, or 0, which corresponds to +1, -1, or unchanged number of replicas. Each action selection selects a scaling strategy for each microservice.

[0068] Next, the reward function is defined as:

[0069]

[0070] The specific value of SLA is the upper limit of the response delay value that meets the service level agreement. α and β represent custom weights, which measure the importance of improving the average CPU utilization of microservices and reducing the SLA violation rate. C (C>0) represents the penalty coefficient when violating the SLO. t It indicates the end-to-end P90 response delay of the microservice system. In multi-application microservices, it is manifested as the P90 delay of the front-end microservice.

[0071] The optimization goal is to select an appropriate replica scaling strategy for all microservices given the current context state, so as to maximize the average resource utilization of microservices while ensuring the SLA violation rate.

[0072] 2) Design a deep contextual multi-armed bandit model architecture. A microservice system consists of multiple services. Based on the contextual features designed above, the action space also expands rapidly with the increase in the number of microservices and optional actions due to its high spatial dimension. In addition, the relationship between system performance and state and action space is usually nonlinear, and traditional contextual multi-armed bandit (CMAB) is difficult to effectively cope with these complexities. In order to adapt to the changes in different load patterns and enhance the generalization ability of the model, a deep neural network is introduced in CMAB to approximate the state-action value function Q(s, a; θ) to solve the problem of large state and action space.

[0073] 3) The microservice CPU utilization and P90 latency prediction model trained based on the graph attention mechanism in step 2 is used as the model environment for interacting with the deep context multi-armed bandit to quickly train the model, thereby optimizing the resource management of the microservice system. In step 2, the prediction of the key performance indicators of the microservice system is achieved through the graph attention network (GAT). The model can predict the request arrival rate of each microservice at the current moment. And the number of copies after the change Predict CPU utilization for each microservice and P90 response delay The changes in the simulation environment are realized by the DCMAB, thus realizing the interaction process between the DCMAB and the simulation environment. The specific interaction process is as follows:

[0074] Action selection strategy: At time t, by observing the current context feature S t , get the W of the previous moment t-1 , R t-1 and the current W t , and then use the UCB strategy to select action A t The basic idea of ​​the UCB strategy is to assign a confidence upper bound to each action to encourage the selection of actions that have not been fully explored, and to select the maximum UCB value each time an action is selected. The specific formula is:

[0075]

[0076] Where Q(S t , A t ; θ) is the current state S t Next action A t The reward estimate of θ represents the parameters of the neural network, c is the exploration parameter, which controls the balance between exploration and utilization, N(A t ) indicates action A t The number of times selected.

[0077] Simulate the environment (model in step 2) interaction: take action A t Then get the new number of replicas R of each microservice at time tt , so by aggregating S t =(W t ,W t-1 ,R t-1 ), R t 、Microservice P90 delay L at time t-1 t-1 、Microservice CPU utilization C at time t-1 t-1 , and input the GAT model trained in step 2, we can get the execution action A t After L t With R t , that is, the response delay and CPU utilization of each microservice at time t, so the reward R after the current action can be calculated t .

[0078] Model update: According to the calculated reward R t and the current state S t , use the deep neural network to update the DCMAB model parameters and minimize the loss function:

[0079]

[0080] By continuously optimizing the neural network parameters θ, the model gradually learns to select the best replica adjustment strategy under different load conditions.

[0081] DCMAB continuously repeats the above process, observing the current state, selecting actions, predicting performance, calculating rewards, and optimizing strategies until the rewards converge.

[0082] Step 4: Inject online time-varying workloads into the system, put the trained DCMAB into the actual environment, use the Prometheus plug-in to detect changes in microservice indicators at time intervals t and continuously update context information, input the real-time detected context information into the trained DCMAB, and prompt the model to make replica horizontal scaling decisions for each microservice at each timestamp t. After the set time T, the final application SLA violation and microservice resource utilization are obtained.

Claims

1. A load-aware elastic scaling method for multiple microservice replicas in a computing network, characterized in that The method comprises at least the following steps: Step 1: Based on the non-intrusive service mesh technology, design a historical indicator collector based on plug-ins such as Prometheus and Istio in the K8S cluster, sample the cluster historical data and use it as the subsequent graph attention network training data set; Step 2: Based on the time series dataset obtained in step 1, according to the changes in microservice workload and number of replicas, train the microservice CPU utilization and P90 latency predictor based on the graph attention network, and build a simulation environment that interacts with the deep context multi-armed bandit; Step 3: Based on the model trained in step 2, a deep contextual multi-armed bandit algorithm is designed to interact with the model, and efficient offline training is performed for each timestamp and specific context information to obtain a multi-microservice automatic scaling model that adapts to different load changes; Step 4: Based on the multi-microservice automatic scaling model obtained in step 3, inject online time-varying workloads into the system, set a certain load injection time, and put the trained model into the real K8S environment to verify the SLA violations of microservice response delays and CPU resource utilization.

2. The load-aware multi-microservice replica elastic scaling method in a computing network according to claim 1 is characterized in that Training the microservice CPU utilization and P90 latency predictor based on the graph attention network also includes the following steps: 1) According to the complex calling relationship between multiple microservices, a directed graph G = (M, E) is constructed, and the features of each node are combined to obtain the complete feature vector of each node: in They represent the workload, CPU utilization, P90 latency, and number of replicas of microservice m at time t-1, respectively. and They represent the workload and number of replicas of microservice m at time t respectively. By aggregating complete node features and call relationships, a complete graph data structure is formed for model training. 2) A two-layer graph attention network model is used to capture the complex dependencies between microservices. The calling relationship between microservices is utilized to dynamically adjust the influence of neighbor nodes on the target node. The representation of each node is updated by aggregating the features of neighbor nodes, thereby accurately modeling the interactions between microservices. 3) Using the updated node representation for prediction, we design two independent but identical networks, one for predicting the workload change of each microservice at time t. and number of copies CPU utilization after and P90 delay This builds a simulation environment for interacting with a deep contextual multi-armed bandit.

3. The load-aware elastic scaling method for multiple microservice replicas in a computing network according to claim 1 is characterized in that The model-based deep contextual multi-armed bandit algorithm also includes at least the following steps: 1) Define the contextual features of multi-microservice applications and construct state space, dynamic space and reward function. State space S t Defined as: S t =(W t ,W t-1 ,R t-1 ) Where W t , W t-1 represents the workload (QPS) of each microservice at time t and time t-1, respectively, and R t-1 It represents the number of replicas of each microservice at time t-1. All three are N-dimensional vectors, and the value of the i-th dimension represents the value of the relevant performance indicator in the i-th microservice. Action Space A t Defined as: in Indicates the microservice m i The scaling operation at the current moment can have a value of 1, -1, or 0, which corresponds to +1, -1, or unchanged number of replicas. Each action selection selects a scaling strategy for each microservice. The reward function is defined as: Where α and β represent custom weights, respectively, measuring the importance of improving the average CPU utilization of microservices and reducing the SLA violation rate, and C (C>0) represents the penalty coefficient when violating the SLO. t Indicates the end-to-end P90 response delay of the microservice system, specifically the P90 delay of the front-end microservice. 2) The microservice CPU utilization and P90 latency prediction model trained based on the graph attention mechanism in step 2 is used as the model environment for interacting with the deep contextual multi-armed bandit to quickly train the model, thereby optimizing the resource management of the microservice system. In order to adapt to different load pattern changes, the model needs to have good generalization ability. A deep neural network with strong representation ability is introduced in CMAB to approximate the state-action value function to solve the problem of large state and action space. At time t, by observing the current context feature S t , get the W of the previous moment t-1 ,R t-1 and the current W t , and then use the UCB strategy to select action A t After taking action, we get a new R t , so by aggregating S t , A t and the remaining L t-1 and R t-1 Input the GAT model to get the execution action A t After L t With R t , that is, the response delay and CPU utilization of each microservice, can be used to calculate the reward R after the current action is executed t According to the calculated reward and current state, the DCMAB model parameters are updated using a deep neural network to minimize the loss function. By continuously optimizing the neural network parameters, the model gradually learns to select the best replica adjustment strategy under different load conditions.

Images