Cluster elastic scaling method based on load time sequence prediction

Through the value-oriented reinforcement learning of the LSTM network and DQN algorithm using sparse attention mechanism in container clusters, the problem of slow response speed, low resource utilization and poor generalization ability of the container cluster elastic scaling method is solved, and more efficient load response and resource management.

CN119065829APending Publication Date: 2024-12-03UESTC (SHENZHEN) ADVANCED RES INST
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202410971077.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-19
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing container cluster elastic scaling method faces the problems of low load change response speed, low resource utilization rate, and poor generalization ability to different types of loads.

Method used

The LSTM codec network based on the sparse attention mechanism is used to predict the load timing, and combined with the value-oriented reinforcement learning of the DQN algorithm, a cluster elastic scaling decision-making framework is built to realize pre-scaling decision-making to improve response speed and resource utilization.

Benefits of technology

It improves the response speed and resource utilization of container clusters when load changes, enhances the generalization ability of different types of loads, and achieves safe and efficient elastic scaling.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119065829A_ABST
    Figure CN119065829A_ABST
Patent Text Reader

Abstract

The invention provides a cluster elastic scaling method based on load time sequence prediction, and belongs to the technical field of computers. Comprising the following steps: introducing a sparse attention mechanism between an input layer and an output layer of an LSTM network, and constructing an LSTM coding and decoding load time sequence prediction module based on the sparse attention mechanism; constructing a cluster elastic scaling decision-making agent based on reinforcement learning; constructing a value-oriented reinforcement learning algorithm based on a DQN algorithm; and constructing a container cluster elastic scaling decision framework based on load time sequence prediction, and outputting a container cluster elastic scaling strategy. According to the method, the sparse attention mechanism is applied to the LSTM coding and decoding network, so that the calculation complexity of the LSTM coding and decoding network is reduced, and the response speed and the calculation precision of the network for processing a long-time sequence are improved; according to the method, load prediction is carried out through historical load time sequence data, the reinforcement learning state space of a decision scene is improved according to the prediction result, and the safe and efficient elastic expansion and contraction capability of a container cluster in a complex load scene is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular relates to a cluster elastic scaling method based on load time series prediction. Background Art

[0002] In recent years, with the continuous development of cloud computing, massive service systems have been deployed in the cloud. When the cloud faces fluctuations in service request load, the container cluster that handles the load can automatically scale up and down to maintain stable service performance and save hardware and energy resources for operators.

[0003] Although container technology is superior to traditional virtual machine technology in terms of lightweight and fast start and stop, the cold start problem in the containerized operating environment still exists because the cold start of the container requires loading microservice function code and dependencies. One of the core competitiveness of cloud computing is to rely on lightweight container technology to achieve rapid supply and recovery of computing resources, which is reflected in the elastic scaling capability of the container cluster.

[0004] So far, there have been many studies on the elastic scaling methods of container clusters in terms of periodic processing of different types of workload requests. However, due to the container cold start and burst traffic problems in cloud computing scenarios, the existing elastic scaling methods of container clusters still have certain limitations, which are mainly manifested in the following three points:

[0005] (1) The response speed to load changes is low. For periodic data, the periodicity of the load is not fully exploited. The existing prediction algorithms are not fast enough and accurate enough to handle long time series, which results in inadequate deployment of container clusters in advance. This makes it difficult to respond quickly when the load flow changes, and the request violation rate may increase when facing sudden traffic.

[0006] (2) Low resource utilization of container clusters. Traditional scaling methods cannot grasp the timing characteristics of loads and have low response speeds. To ensure that they can cope with sudden traffic, resources are reserved, which leads to a decrease in overall resource utilization.

[0007] (3) Poor generalization capabilities for different types of loads. Traditional scaling methods are generally driven by rules or indicators, and resources are adjusted by configuring monitoring thresholds. Since different types of loads have different sensitivities to different resources, traditional scaling methods generally need to be customized for specific scenarios, and lack autonomous learning capabilities and generalization capabilities for different scenarios. Summary of the invention

[0008] The present invention aims to provide a cluster elastic scaling method based on load time series prediction, by predicting the time series of periodic load data. The goal of pre-scaling decision is achieved, thereby improving the decision effect and response speed of elastic scaling of container clusters; the present invention includes a time series prediction module based on a long short-term memory (LSTM) codec network and a scaling decision module based on a value-oriented deep reinforcement learning algorithm (DQN). Aiming at the problem of performance degradation of the original LSTM codec network in processing long time series, the time series prediction module improves the LSTM in combination with the sparse attention mechanism, thereby improving the prediction accuracy of the module in processing long time series; the scaling decision module improves the state space of the DQN decision module in combination with the prediction results of the time series prediction module in order to solve the problem of slow response speed of elastic scaling decision, and achieves the goal of pre-scaling decision, thereby improving the decision effect and response speed of cluster elastic scaling. In order to solve the technical problems of low response speed of the elastic scaling method of container clusters in the prior art to load changes, low resource utilization of container clusters, and poor generalization ability for different types of loads.

[0009] In order to solve the above technical problems, the present invention proposes a cluster elastic scaling method based on load time series prediction, which includes the following steps:

[0010] Step S1: introduce a sparse attention mechanism between the input layer and the output layer of the LSTM network, and construct an LSTM encoding and decoding load timing prediction module based on the sparse attention mechanism;

[0011] Step S2: Construct a cluster elastic scaling decision-making agent based on reinforcement learning;

[0012] Step S3: construct a value-oriented reinforcement learning algorithm based on the DQN algorithm;

[0013] Step S4: Build a container cluster elastic scaling decision framework based on load time series prediction and output the container cluster elastic scaling strategy.

[0014] Furthermore, in step S1, for each prediction time t, the attention weight of the i-th decoder LSTM unit in the output layer is The calculation is as follows:

[0015]

[0016] in, represents the hidden state of the decoder LSTM unit at the previous moment, h i represents the hidden state of the encoder LSTM unit at the current moment, represents the cell state of the decoder LSTM unit at the previous moment; X represents the input workload time series data, is the value vector matrix in the attention mechanism, and W l with U lRepresents the weight matrix in the attention mechanism, corresponding to the attention weight matrix of the hidden state at the previous moment and the attention weight matrix at the current moment, W l and U l is a learnable parameter;

[0017] Encoder Attention weights for the hidden state of the i-th encoder LSTM unit The calculation is as follows:

[0018]

[0019] is the hidden state h of the i-th encoder LSTM unit i The attention weights, and Represents the range of encoder state feature set, F enc Represents the hidden state feature dimension of the encoder LSTM unit;

[0020] Predict the context vector c at time t t is the product and accumulation of the hidden state attention weight calculated according to the attention mechanism and the hidden state of the encoder LSTM unit:

[0021]

[0022] Calculate the context vector c t After that, use and Map it as the input of the LSTM unit of the decoder at time t.

[0023]

[0024] in, The mapping weight matrix and offset matrix representing the encoder LSTM long short-term memory neural network are obtained by fitting the input time series after training, and both are learnable parameters.

[0025] Using input The hidden state at the previous moment and cell status Hidden state of the decoder at time t Update; finally, combine and the context vector c t Generate the predicted value at time t

[0026]

[0027]

[0028] Among them, W y 、b y They represent the mapping weight matrix and offset matrix respectively obtained by fitting the decoder LSTM long short-term memory neural network to the input time series after training; and is a learnable parameter; represents the feature set, F dec Represents the decoder LSTM unit hidden state feature dimension.

[0029] Furthermore, in step S2, the cluster elastic scaling decision agent includes<S,A,P,R,y> The five-tuple is composed of: the agent state space S includes workload information and container cluster resource status information;

[0030] Workload information mainly includes the type of requested load, timestamp, SLO level, and required CPU and memory resources, as shown in the following formula:

[0031] [s1=[type1,time1,slo1,cpu1,mem1],...,[type n ,time n ,slo n ,cpu n ,mem n ]]

[0032] Where n represents the workload information vector dimension;

[0033] The cluster resource status information mainly includes the CPU and memory resources of the current container cluster, as shown in the following formula:

[0034] [s2=[cpu1,mem1],[cpu2,mem2],...,[cpu m ,mem m ]]

[0035] Where m represents the dimension of cluster resource status information vector;

[0036] The state space S is defined as the Cartesian product of all possible workload information vectors and container cluster resource state information vectors, as shown in the following formula:

[0037]

[0038] in, Represents a set of workload information vector dimensions, Represents a set of cluster resource status information vector dimensions.

[0039] Furthermore, the agent action space includes action a1 for cluster elastic expansion operation, action a2 for cluster elastic reduction operation, action a3 for configuration adjustment operation, and action a4 for start / stop operation.

[0040] Furthermore, the agent reward includes the resource utilization reward r resource and SLO violation rate reward r SLO , as shown in the following formula:

[0041] r=r resource +r SLO

[0042]

[0043] Among them, x t is the number of containers in the container cluster at the tth moment, f r (x t ) represents the resource consumption of the working container, f p (x t ) represents the idle container resource consumption;

[0044]

[0045] Among them, SLOV is the service level objective violation rate.

[0046] Furthermore, in step S3, the update formula based on the DQN reinforcement learning method is:

[0047] Q(s t ,a t )=Q(s t ,a t )+a(r t +γmaxQ(s t+1 ,a t+1 )-Q(s t ,a t ))

[0048] Among them, s t Indicates the current state, a t Indicates the action taken, r t Indicates the reward obtained, s t+1 represents the next state, a t+1 Indicates the next state s t+1 All possible actions, Q(s t ,a t ) means in state s t Take action a t The resulting action value function; Q(s t+1 ,a t+1 ) represents the next state s estimated by the target networkt+1 All possible actions a t+1 The action value function of ; γ is the discount factor.

[0049] Furthermore, in step S4, the workload information and container cluster status information are used as input environmental states, and the load timing prediction module performs time series prediction through an improved LSTM encoding and decoding network based on a sparse attention mechanism to obtain load prediction results for future moments; the scaling decision module takes actions based on the load prediction results according to a policy network of a value-oriented reinforcement learning algorithm based on the DQN algorithm, and at the same time, the value network of the value-oriented reinforcement learning algorithm based on the DQN algorithm scores cluster scaling actions based on network feedback, and finally outputs an elastic scaling strategy for the container cluster that accumulates the reward value and is large.

[0050] Compared with the prior art, the present invention has the following beneficial technical effects:

[0051] 1) The present invention applies the sparse attention mechanism to the LSTM codec network, which reduces the computational complexity of the LSTM codec network and improves the response speed and computational accuracy of the network in processing long time series.

[0052] 2) The present invention uses the timing prediction results to improve the state space of the DQN algorithm, and replaces the workload information vector in the state space with the predicted output of the timing prediction module, so that the elastic scaling decision algorithm can perform elastic scaling decision operations in advance, and grasp the actual state space information through the rewards obtained by real-time environmental state calculation, so as to adapt to the upcoming traffic peaks or troughs in advance, and realize efficient elastic scaling of the cluster under complex load scenarios.

[0053] 3) The present invention performs load forecasting through historical load time series data, and improves the reinforcement learning state space of the decision-making scenario based on the forecast results, thereby realizing the safe and efficient elastic scaling capability of the container cluster in complex load scenarios.

[0054] 4) Compared with other scaling decision algorithms, the cluster elastic scaling method based on time series prediction of the present invention reduces the cluster control response time, improves resource utilization while ensuring user availability, and has a better elastic scaling effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0056] Figure 1 This is a schematic diagram of the LSTM encoding and decoding load timing prediction module based on the sparse attention mechanism of the present invention.

[0057] Figure 2 The figure is a schematic diagram of a container cluster elastic scaling decision framework based on load time series prediction according to the present invention.

[0058] Figure 3 The following is a comparison chart of the response delay of three types of loads under different elastic scaling methods. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] The present invention proposes a cluster elastic scaling method based on load time series prediction, which includes the following steps:

[0061] Step S1: Introduce a sparse attention mechanism between the input layer and the output layer of the LSTM network to build an LSTM encoding and decoding load timing prediction module based on the sparse attention mechanism.

[0062] like Figure 1 As shown in the figure, the time series data of the workload is input into the LSTM encoder. For each prediction moment, based on the hidden state of the encoder at all moments and the state of the decoder at the previous moment, a context vector is generated using the temporal attention mechanism to update the hidden state of the decoder at the prediction moment. The vector is then mapped to the decoder to update the hidden state of the decoder at the current moment, and finally the prediction output result is obtained.

[0063] Specifically, the workload time series data X = {X1, ...X i ,…X T}At each moment, the encoder LSTM unit is input to the corresponding LSTM unit of the encoder. The encoder LSTM unit understands the time series data of the input workload and calculates the hidden state. The output of each moment will be used as the input of the next moment until the entire input time series is processed. The attention mechanism participates in the process of the encoder converting the time series data of the input workload into a context vector. It uses its ability to measure the degree of correlation between the hidden state at each moment and the current calculation moment, and discards the decoder LSTM unit with a small degree of correlation in a weighted manner, and only selects the key moments for calculation.

[0064] For each prediction time t(1≤t≤T p), the attention weight of the i-th (1≤i≤T) decoder LSTM unit in the output layer The calculation is as follows:

[0065]

[0066] in, represents the hidden state of the decoder LSTM unit at the previous moment, h i represents the hidden state of the encoder LSTM unit at the current moment, represents the cell state of the decoder LSTM unit at the previous moment; X represents the input workload time series data, is the value vector matrix in the attention mechanism, and W l with U l Represents the weight matrix in the attention mechanism, corresponding to the attention weight matrix of the hidden state at the previous moment and the attention weight matrix at the current moment, W l and U l are learnable parameters.

[0067] Encoder Attention weights for the hidden state of the i-th encoder LSTM unit The calculation is as follows:

[0068]

[0069] is the hidden state h of the i-th encoder LSTM unit i The attention weights, and Represents the range of encoder state feature set, F enc Represents the hidden state feature dimension of the encoder LSTM unit.

[0070] Predict the context vector c at time t t is the product and accumulation of the hidden state attention weight calculated according to the attention mechanism and the hidden state of the encoder LSTM unit:

[0071]

[0072] Calculate the context vector c t After that, use and Map it as the input of the LSTM unit of the decoder at time t.

[0073]

[0074] in, The mapping weight matrix and offset matrix representing the encoder LSTM long short-term memory neural network are obtained by fitting the input time series after training, and both are learnable parameters.

[0075] Using input The hidden state at the previous moment and cell status Hidden state of the decoder at time t Finally, combine and the context vector c t Generate the predicted value at time t

[0076]

[0077] Among them, W y 、b y They represent the mapping weight matrix and offset matrix respectively obtained by fitting the decoder LSTM long short-term memory neural network to the input time series after training; and is a learnable parameter; represents the feature set, F dec Represents the decoder LSTM unit hidden state feature dimension.

[0078] The update process of the LSTM unit involves the forget gate, input gate, updating the cell state, output gate, and updating the hidden state:

[0079] Forget Gate t The update is achieved through the following formula:

[0080]

[0081] Among them, σ represents the sigmoid function, W f represents the weight matrix obtained by forget gate training, b f Represents the bias matrix obtained by forget gate training.

[0082] Input Gate i t The update is achieved through the following formula:

[0083]

[0084] Among them, i t represents the activation value of the input gate, W i represents the weight matrix obtained by input gate training, b i Represents the bias matrix obtained by input gate training.

[0085] Updating the memory unit is achieved through the following formula:

[0086]

[0087] in, represents the candidate memory unit, W C represents the candidate memory unit weight matrix obtained through training, b C Represents the candidate memory unit bias matrix obtained through training.

[0088] Updating the cell state is achieved through the following formula:

[0089]

[0090] in, Represents the memory unit of the previous time step, that is, the cell state, and * represents element-by-element multiplication.

[0091] Output gate o t This is achieved through the following formula:

[0092]

[0093] Among them, W o represents the output gate weight matrix obtained through training, b o Represents the output gate bias matrix obtained through training.

[0094] Updating the hidden state is achieved through the following formula:

[0095] h t =o t *tanh(C t )

[0096] Step S2: Build a cluster elastic scaling decision-making agent based on reinforcement learning.

[0097] The cloud native container cluster needs to control the cluster over a period of time, and the number of cluster containers at different times will change dynamically to adapt to changes in load. Due to the switching between the working state and the idle state of the container, when faced with a large number of load requests, the cluster will start the corresponding container to process; but if the load enters the trough period, the running idle containers will generate additional resource loss. In order to improve the resource utilization of the container cluster, it is necessary to provide cluster elastic scaling capabilities to cope with workloads at different times and determine whether the cloud native container cluster needs to be expanded or contracted and the corresponding number of operating containers.

[0098] In the cluster elastic scaling decision mechanism based on reinforcement learning, the elastic scaling controller that manages the scale of the cloud native container cluster is abstracted as an intelligent agent. The intelligent agent perceives the current cluster resource status information through the monitoring program. By applying the reinforcement learning algorithm, the intelligent agent selects actions in the cluster management decision mechanism based on elastic scaling to meet the requirements of requesting load to configure container resources. The intelligent agent continuously interacts with the resource status information in the scaling decision framework, observes the results of action execution, and evaluates the rationality of task configuration through the reward function. The intelligent agent optimizes the elastic scaling decision strategy with the goal of maximizing the cumulative reward value and continuously improves the quality of the strategy model. The whole process is continuously iterated to achieve the best elastic scaling decision and continuously improve the performance of the system.

[0099] The cluster elastic scaling decision-making agent based on reinforcement learning of the present invention includes<S,A,P,R,y> The purpose of the five-tuple is to connect the cloud-native container cluster workload time series data with elastic scaling decisions to provide a basic operating environment for training.

[0100] Reinforcement Learning Parameter Space Quintuple<S,A,P,R,y> The definitions are as follows:

[0101] State space S: The state space defines all environmental information that the agent can directly obtain, including workload information and container cluster resource status information.

[0102] Workload information mainly includes the type of requested load, timestamp, SLO level, and required CPU and memory resources, as shown in the following formula:

[0103] [s1=[type1,time1,slo1,cpu1,mem1],...,[type n ,time n ,slo n ,cpu n ,mem n ]]

[0104] Where n represents the workload information vector dimension.

[0105] The cluster resource status information mainly includes the CPU and memory resources of the current container cluster, as shown in the following formula:

[0106] [s2=[cpu1,mem1],[cpu2,mem2],...,[cpu m ,mem m ]]

[0107] Wherein, m represents the dimension of the cluster resource status information vector.

[0108] The state space S is defined as the Cartesian product of all possible workload information vectors and container cluster resource state information vectors, as shown in the following formula:

[0109]

[0110] in, Represents a set of workload information vector dimensions, Represents a set of cluster resource status information vector dimensions.

[0111] Action space A: The action space is the main means for the intelligent agent to interact with the reinforcement learning environment. Its design quality will significantly affect the quality of the data generated during the interaction, thereby interfering with the ability of the subsequent decision algorithm to achieve the expected goal. The present invention introduces three actions to achieve the expected goal. The specific introduction is shown in Table 1.

[0112] Table 1 Cluster elastic scaling decision model action space design table

[0113]

[0114] As can be seen from Table 1, action a1 is a cluster elastic capacity expansion operation, which is a direct action. Its purpose is to increase the number of container instances according to the current load situation to cope with the increase in load pressure; action a2 is a cluster elastic capacity reduction operation, which is a direct action. Its purpose is to reduce the number of container instances according to the current load situation to cope with the decrease in load pressure; action a3 is a configuration adjustment operation, which is an indirect action. Its purpose is to adjust the configuration of a container in the current container sequence (such as CPU, memory and other resources) to adapt to different load conditions; action a4 is a start / stop operation, which is a direct action. Its purpose is to start or stop a specific container instance.

[0115] Reward R: In order to enable the elastic scaling controller to automatically generate and execute the long-term optimal solution under real-time load input, the reinforcement learning method needs to obtain rewards from the container cluster environment and evaluate the scaling decision, thereby affecting the next scaling decision. Guided by the reward function, the elastic scaling controller can learn and adapt to different environmental changes and take corresponding optimization actions to meet the load requirements.

[0116] The reward function r designed by the present invention consists of two parts: resource utilization reward r resource and SLO violation rate reward r SLO .

[0117] r=r resource +r SLO

[0118] When a training round ends, the agent will calculate rewards based on the container resource utilization during the round, calculate the SLO violation rate, and calculate rewards based on the violation level.

[0119]

[0120] Among them, x t is the number of containers in the container cluster at the tth moment, f r (x t ) represents the resource consumption of the working container, f p (x t ) indicates the idle container resource consumption.

[0121]

[0122] Among them, SLOV is the service level objective (SLO) violation rate.

[0123] State transition probability function P: In the scenario of elastic scaling of server clusters, the state transition function can describe the possibility of the system transitioning from one state to another after performing a certain action. The present invention adopts a value-oriented reinforcement learning algorithm to build a decision model. The role of the state transition function is implicitly included in the Q value update process and does not require explicit design.

[0124] Discount factor y: The load time series is periodic and time-related, so the action of the current state not only affects the transfer of the current state, but also affects the transfer of the future long-term state, so it is necessary to consider the future long-term rewards. According to the characteristics of the container cluster processing load tasks, a smaller discount factor helps improve the system's ability to optimize real-time performance and response latency, while a larger discount factor can help the system focus more on overall resource utilization optimization and long-term performance.

[0125] Step S3: Construct a value-oriented reinforcement learning algorithm based on the DQN algorithm.

[0126] The cloud-native container cluster elastic scaling decision module obtains the workload information and container cluster status information of the entire environment at each moment, thereby controlling the container cluster to transfer from the previous state to the next state. This transfer process mainly depends on the action decided at the current moment and the cluster state at the previous moment. The decision module expects that the actions taken at each moment can maximize the resource utilization at the current moment and subsequent moments, that is, focus on the cumulative benefits of the actions.

[0127] The state space S is defined as the Cartesian product of all possible workload information vectors and container cluster resource state information vectors. Therefore, the workload information vector is replaced by the predicted output of the timing prediction module, so that the elastic scaling decision algorithm can perform elastic scaling decision operations in advance to adapt to the upcoming traffic peaks or troughs in advance.

[0128] The workload information used to calculate the reward is obtained based on the current state information, so that the decision module can obtain the predicted state space in advance and make the selection and implementation of the action strategy. The value judgment of the action is based on the current environmental state, that is, the actual workload information and container cluster state information. Therefore, the intelligent decision module based on the DQN algorithm makes action decisions through the predicted load changes provided by the timing prediction module, achieving the goal of early scaling decision-making, and grasping the actual state space information through the rewards calculated through real-time environmental state.

[0129] The Q network of the DQN algorithm first uses a convolutional neural network to calculate the state information s at the current time t. t The output of the convolutional neural network is extracted and then mapped to the action space of the reinforcement learning environment by a fully connected network.

[0130] During the model training process, the DQN algorithm uses the "ε-greedy" strategy to select the next action to be executed according to the Q estimate output by the target Q network Q(s,a,θ), that is, the action with the largest Q estimate in the current state is selected with a probability of 1-ε, and an action is randomly selected with a probability of ε to increase the model's "exploration" ability for unknown environments. max Start to decrease linearly to ε mi n, so that the model can have a strong ability to explore unknown space in the early stage of training, prevent falling into the local optimum and affecting the training effect, and stabilize the good strategy model of the current training in the later stage to prevent large fluctuations from affecting the stability of the model. In addition, in order to reduce the temporal correlation of the model training samples, DQN uses the experience replay pool D to randomly sample and store the samples (s, a, r, s′) generated during the training process; where s represents the state, a represents the action, r represents the reward, and s′ represents the state at the next moment. When the number of samples in D reaches the threshold, the network parameters are adjusted and optimized so that it can more accurately estimate the action value under different states, that is, minimize the error function:

[0131]

[0132] Among them, E (s,a,s′,r)∈D represents the expected value of a sample randomly sampled from the experience replay pool, y(s,a) is the current moment environmental reward obtained by observation and Q(s′,a′,θ- ) The network estimates the value of the next action, namely:

[0133]

[0134] Among them, r(s,a) represents the reward discount factor for taking action a in state s, which is used to control the importance of future rewards; γ is the discount factor, Q(s′,a′,θ - ) is the target Q network that uses the same model as Q(s,a,θ). When the algorithm model reaches the set threshold, the online parameters are optimized according to the latest results.

[0135] In summary, the update formula based on the DQN reinforcement learning method is:

[0136] Q * (s t ,a t )=Q(s t ,a t )+a(r t +γmaxQ(s t+1 ,a t+1 )-Q(s t ,a t ))

[0137] Among them, s t Indicates the current state, a t Indicates the action taken, r t Indicates the reward obtained, s t+ 1 indicates the next state, a t+ 1 indicates the next state s t+1 All possible actions, Q(s t ,a t ) means in state s t Take action a t The resulting action value function; Q(s t+1 ,a t+1 ) represents the next state s estimated by the target network t+1 All possible actions a t+1 The action value function of ; γ is the discount factor, and a∈[0,1] is the learning rate.

[0138] Step S4: Build a container cluster elastic scaling decision framework based on load time series prediction and output the container cluster elastic scaling strategy.

[0139] The container cluster elastic scaling decision framework based on load time series prediction is shown in Figure 2. The workload information and container cluster status information are used as input environment states. The load time series prediction module uses the improved LSTM encoder-decoder network based on the sparse attention mechanism to perform time series prediction to obtain the load prediction results at future moments. The scaling decision module takes actions based on the load prediction results according to the policy network of the value-oriented reinforcement learning algorithm based on the DQN algorithm. At the same time, the value network of the value-oriented reinforcement learning algorithm based on the DQN algorithm scores the cluster scaling actions based on network feedback, and finally outputs the container cluster elastic scaling strategy that makes the cumulative reward value as large as possible. The historical time series data of the workload is used as the input of the load time series prediction module, and the load prediction results of the load time series prediction module are used to update the state space of the reinforcement learning environment, so that the algorithm framework can adjust the deployment in advance based on the predicted value, while the value network is based on the real workload data and the container cluster status data. The value judgment is made to evaluate the prediction results. According to the cluster elastic scaling strategy, the exploration mechanism and reward function are redesigned to enhance the exploration and convergence capabilities of the reinforcement learning agent in cluster scaling decisions.

[0140] In order to verify the effectiveness of the present invention, the following elastic scaling methods were selected for experimental comparison. Rule-triggered cluster elastic scaling method (Trigger), Kubernetes horizontal expansion and contraction component (k8s-HPA), cluster scaling method based on elastic network regression (ENR), automatic scaling method based on time domain convolutional network (TCN), elastic scaling method based on particle swarm optimization algorithm (PSO). Three types of loads were selected for comparison. Type 1 and Type 2 loads selected 288 periodic moments, that is, a load quantity sample of one day for experimental display. Type 3 loads do not have periodicity, and 50 sampling moments were selected for experimental display.

[0141] The response delay changes of each comparison method under three different types of workload scenarios are shown in the figure. Figure 3 As shown in the figure. The black curve represents the load quantity at each moment, the red line represents the response delay threshold for the SLO violation of this type of load, and the blue curve represents the response delay of the container cluster processing the load request. Below each workload timing chart is a corresponding response delay line chart in the same column, which indicates the response delay performance of the container cluster when processing this type of load request.

[0142] According to the data set, the SLO violation response delay is set to 0.25s. In the three types of load experiments, the blue response delay curve of the present invention can be maintained below the red SLO violation threshold at most times, showing good decision-making performance and robustness. Among the other five algorithms compared, the Trigger algorithm does not have the ability to make decisions in advance due to its threshold triggering principle, so it performs the worst in the three types of loads; the k8s-HPA algorithm allocates excess containers as hot standby in advance, so when the recovery time is greater than the load cycle, it can have a better response delay performance, such as type 2 load, but when the recovery time is less than the load cycle or there is a burst scene traffic, the k8s-HPA algorithm performs poorly; ENR and TCN algorithms both have load prediction capabilities, which are shown in type 1 and type 2 experiments, and still require a lot of training to improve prediction accuracy; PSO, as a heuristic algorithm, has a certain learning ability. It can be seen from the response delay curve of type 2 that the later cluster scaling decision effect is better, but the performance under burst scene traffic is still insufficient.

[0143] In summary, the present invention uses an LSTM encoding and decoding load timing prediction module based on a sparse attention mechanism to perform workload timing prediction, so that it can have prediction accuracy under workloads with periodic timing characteristics, thereby improving the effect of cluster elastic scaling decisions and ensuring that the response delay of load requests does not violate the SLO; and when facing burst scenario traffic, our method can also use the learning ability of reinforcement learning to adapt to fluctuations as quickly as possible, make correct scaling decisions, and learn the long-term change characteristics of the load.

[0144] The SLO violation rates in each set of experiments are shown in Table 2.

[0145] Table 2 Comparison of SLO violation rates under different scaling decision methods

[0146]

[0147] It can be seen from Table 2 that the SLO violation rate of the present invention is the lowest, which shows that the present invention reduces the SLO violation rate by optimizing the response delay. Specifically, compared with the comparison method, the present invention reduces the SLO violation rate by an average of 51.2%.

[0148] In the two data scenarios of type one and type two with periodic regularities, the ENR, TCN and PSO algorithms using convolutional neural networks and heuristic algorithms can better grasp the elastic scaling rules of the cluster. Compared with the two traditional fixed threshold triggering methods of Trigger and k8s-HPA, the SLO violation rate is guaranteed to be within 10%, thereby meeting the SLO requirements. On this basis, the present invention further optimizes the SLO violation rate and achieves an excellent performance of about 2.5%. In the type three load data scenario without periodicity, although several methods have a large degradation, the present invention still has a lower average violation rate, which proves that the present invention achieves a reduction in the SLO violation rate by optimizing the response delay. Specifically, based on the best performance among the five comparison methods, the present invention reduces the SLO violation rate by an average of 51.2%.

[0149] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A cluster elastic scaling method based on load time series prediction, It is characterized in that The steps include: Step S1: introduce a sparse attention mechanism between the input layer and the output layer of the LSTM network, and construct an LSTM encoding and decoding load timing prediction module based on the sparse attention mechanism; Step S2: Construct a cluster elastic scaling decision-making agent based on reinforcement learning; Step S3: construct a value-oriented reinforcement learning algorithm based on the DQN algorithm; Step S4: Build a container cluster elastic scaling decision framework based on load time series prediction and output the container cluster elastic scaling strategy.

2. According to claim 1, the cluster elastic scaling method based on load time series prediction, It is characterized in that In step S1, for each prediction time t, the attention weight of the i-th decoder LSTM unit in the output layer is The calculation is as follows: in, represents the hidden state of the decoder LSTM unit at the previous moment, h i represents the hidden state of the encoder LSTM unit at the current moment, represents the cell state of the decoder LSTM unit at the previous moment; X represents the input workload time series data, is the value vector matrix in the attention mechanism, and W l with U l Represents the weight matrix in the attention mechanism, corresponding to the attention weight matrix of the hidden state at the previous moment and the attention weight matrix at the current moment, W l and U l is a learnable parameter; Encoder Attention weights for the hidden state of the i-th encoder LSTM unit The calculation is as follows: is the hidden state h of the i-th encoder LSTM unit i The attention weights, and Represents the range of encoder state feature set, F enc Represents the hidden state feature dimension of the encoder LSTM unit; Predict the context vector c at time t t is the product and accumulation of the hidden state attention weight calculated according to the attention mechanism and the hidden state of the encoder LSTM unit: Calculate the context vector c t After that, use and Map it to the input of the LSTM unit of the decoder at time t; in, The mapping weight matrix and offset matrix representing the encoder LSTM long short-term memory neural network are obtained by fitting the input time series after training, and both are learnable parameters; Using input The hidden state at the previous moment and cell status Hidden state of the decoder at time t Update; finally, combine and the context vector c t Generate the predicted value at time t Among them, W y 、b y They represent the mapping weight matrix and offset matrix respectively obtained by fitting the decoder LSTM long short-term memory neural network to the input time series after training; and is a learnable parameter; represents the feature set, F dec Represents the decoder LSTM unit hidden state feature dimension.

3. According to claim 1, the cluster elastic scaling method based on load time series prediction, It is characterized in that In step S2, the cluster elastic scaling decision agent includes<S,A,P,R,y> The five-tuple is composed of: the agent state space S includes workload information and container cluster resource status information; Workload information mainly includes the type of requested load, timestamp, SLO level, and required CPU and memory resources, as shown in the following formula: [s 1 =[type 1 ,time 1 ,slo 1 ,cpu 1 ,mem 1 ],...,[type n ,time n ,slo n ,cpu n ,mem n ]] Where n represents the workload information vector dimension; The cluster resource status information mainly includes the CPU and memory resources of the current container cluster, as shown in the following formula: [s 2 =[cpu 1 ,mem 1 ],[cpu 2 ,mem 2 ],...,[cpu m ,mem m , Where m represents the dimension of cluster resource status information vector; The state space S is defined as the Cartesian product of all possible workload information vectors and container cluster resource state information vectors, as shown in the following formula: in, Represents a set of workload information vector dimensions, Represents a set of cluster resource status information vector dimensions.

4. According to claim 3, the cluster elastic scaling method based on load time series prediction, It is characterized in that The agent action space includes actions a 1 For cluster elastic expansion operation, action a 2 For cluster elastic scaling operation, action a 3 To configure the adjustment operation, action a 4 For start / stop operation.

5. According to claim 4, the cluster elastic scaling method based on load time series prediction, It is characterized in that Agent rewards include resource utilization rewards r resource and SLO violation rate reward r SLO , as shown in the following formula: r=r resource +r SLO Among them, x t is the number of containers in the container cluster at the tth moment, f r (x t ) represents the resource consumption of the working container, f p (x t ) represents the idle container resource consumption; Among them, SLOV is the service level objective violation rate.

6. The cluster elastic scaling method based on load time series prediction according to claim 5, It is characterized in that In step S3, the update formula based on the DQN reinforcement learning method is: Q * (s t ,a t )=Q(s t ,a t )+a(r t +γmaxQ(s t+1 ,a t+1 )-Q(s t ,a t )) Among them, s t Indicates the current state, a t Indicates the action taken, r t Indicates the reward obtained, s t+ 1 indicates the next state, a t+ 1 indicates the next state s t+1 All possible actions, Q(s t ,a t ) means in state s t Take action a t The resulting action value function; Q(s t+1 ,a t+1 ) represents the next state s estimated by the target network t+ 1 All possible actions a t+ 1; γ is the discount factor, and a∈[0,1] is the learning rate.

7. The cluster elastic scaling method based on load time series prediction according to claim 6, It is characterized in that In step S4, the workload information and container cluster status information are used as input environment status, and the load timing prediction module performs time series prediction through an improved LSTM encoding and decoding network based on a sparse attention mechanism to obtain the load prediction result at the future moment; The scaling decision module takes action based on the load prediction results according to the policy network of the value-oriented reinforcement learning algorithm based on the DQN algorithm. At the same time, the value network of the value-oriented reinforcement learning algorithm based on the DQN algorithm scores the cluster scaling actions based on network feedback, and finally outputs the elastic scaling strategy of the container cluster that accumulates the reward value and is large.

Citation Information

Cited By

  • Dynamic video memory allocation method and system based on data characteristics

    CN120315866A

  • Container elastic expansion and contraction method based on asymmetric risk perception and dual-channel spatial-temporal feature fusion

    CN121705035A

  • A container elastic scaling method based on asymmetric risk perception and double-channel space-time feature fusion

    CN121705035B