A cloud resource dynamic scaling method and system

Abstract

The present invention provides a cloud resource dynamic scaling method and system, which obtains the user load on each service node cluster and the corresponding virtual unit of the virtual data center; constructs a load prediction model based on model combination according to historical load information; constructs a decision model based on reinforcement learning, determines the state, action, reward function and state action value function definition of the model; configures a decision algorithm for determining the cloud resource adjustment plan and the probabilistic greedy selection strategy, and determines the action under the future load cycle; constructs a cloud resource dynamic scaling executor to adjust the virtual unit. The present invention can construct an accurate load prediction model in a random and open cloud environment, design an adaptive cloud resource dynamic scaling strategy, solve the problem of reduced service quality and increased supply cost caused by insufficient and wasteful cloud resource supply in large-scale service systems in public cloud environments, and reduce costs and increase efficiency.

Description

Technical Field

[0001] The present invention belongs to the field of cloud computing, and in particular relates to a cloud resource dynamic scaling method and system. Background Art

[0002] With the rapid development of the Internet and modern service industries, service systems have been widely deployed and applied in government, business, education and other fields. Service systems are typical interactive intensive Web applications, which have the characteristics of large user scale, fast data growth and strong load volatility. In order to ensure the performance of the system, service providers need to continuously increase investment in IT infrastructure. However, the expansion of enterprise local data centers has problems such as high operating costs, low resource utilization and difficult maintenance. According to statistics, the resource utilization rate of most enterprise data centers is below 15%, and some are less than 5%. Due to the large load volatility of the service system, there are a large number of surplus resources in the data center during idle time, resulting in a huge waste of energy consumption.

[0003] The emergence of cloud computing provides an opportunity to solve the above problems. Cloud computing is a new computing model based on virtualization technology, which delivers computing, storage, network and other resources to users on a pay-as-you-go basis. Based on the advantages of low cost, high availability and on-demand use of cloud computing, more and more service providers are beginning to migrate their service systems to the public cloud. At present, most cloud providers adopt a tiered payment business model, which requires users to determine the type and quantity of cloud resources to be rented according to their business needs before using cloud services. Due to the volatility of the load of large-scale service systems, their resource requirements are also dynamically changing. Therefore, in actual operations, they are often faced with how to adjust the user's cloud resource usage to ensure that the cloud resource supply is minimized while meeting the user's service quality requirements, so as to bring the greatest benefits to the service provider.

[0004] In response to the current problem of dynamic scaling of cloud resources, although relevant departments and enterprises have studied and adjusted the cloud computing service model, there are still the following three shortcomings:

[0005] (1) The existing application scenarios of dynamic scaling of cloud resources are often small-scale service systems in private cloud environments or public cloud environments. Although the proposed method has good results in the above environments, it is difficult to meet the service quality requirements of large-scale service systems in more complex public cloud environments.

[0006] (2) Many methods have been proposed for load forecasting. However, these forecasting methods only consider the load data of the time period close to the current time period, and ignore the historical load information of different load cycles in the same time period, resulting in low load forecasting accuracy.

[0007] (3) In terms of cloud resource scaling decisions, existing research work mainly focuses on economic models and evaluation methods, but there is relatively little research on key issues such as when to scale and how to scale. Therefore, it is difficult to meet the actual needs of large-scale service systems in terms of dynamic scaling of cloud resources. Summary of the invention

[0008] The present invention provides a cloud resource dynamic scaling method based on load prediction and reinforcement learning, which can construct a prediction model based on model combination and a decision model based on reinforcement learning, use a probability-based greedy decision algorithm to determine the cloud resource adjustment action, formulate the optimal cloud resource dynamic scaling strategy, and realize the intelligent dynamic scaling of cloud resources.

[0009] The method comprises the following steps:

[0010] S1: Obtain each service node cluster of the virtual data center and the user load on the corresponding virtual unit;

[0011] S2: Based on historical load information, a load forecasting model based on model combination is constructed;

[0012] S3: Build a decision-making model based on reinforcement learning, determine the model's state, action, reward function and state-action value function definition;

[0013] S4: configuring a decision algorithm for a probability-based greedy selection strategy for determining cloud resource adjustment plans to determine actions under future load cycles;

[0014] S5: Build a cloud resource dynamic scaling executor to adjust the virtual unit instances.

[0015] It should be further explained that the virtual data center is defined as a cloud resource supply solution consisting of a group of service nodes, service node clusters and the connections between them; a service component is defined as an encapsulation of a group of services and the dependencies between them. In order to meet service requirements, a service component needs to be instantiated into multiple service component instances.

[0016] Define a service node as a virtual unit instance that deploys a service component instance. All virtual unit instances that deploy the same service component are service node clusters. Where S = {s1, s2, …, k} is the service set requested by the user, VU={vu0,vu1,…, m} is a collection of virtual units, vu0 is the virtual unit of the control node, vu j (j=1,…,m) is the virtual unit of the service node, m is the number of service nodes, and the virtual units of different service nodes may be the same; define the user service set S in the virtual unit vu j The load distribution on is: Δj ={λ 1j ,λ 2j ,…,λ kj}, where λ ij (i=1,2,..,k) is service s i In the virtual unit vu j The service cluster The load distribution is Δ={λ1,λ2,…,λ k},in

[0017] It should be further explained that the load forecasting model based on model combination includes a short-term load forecasting model, a long-term load forecasting model, a load residual forecasting model and a combined forecasting model;

[0018] The short-term load forecasting model is: in, are the actual loads in time periods t, t-1, ..., t-m+1 in the load cycle T, respectively, S is the short-term load forecasting method, j is the method f S The order of According to the method f S The predicted load value for period t+1 in the load cycle T.

[0019] The long-term load forecasting model is: in, is the actual load value of period t+1 in load cycle T-1, is the actual load value of time period t+1 in load cycle T-2, ..., is the actual load value of period t+1 in load cycle Ti, f L is the long-term load forecasting method, i is the method f L The order of According to the method f L The predicted load value for period t+1 in the load cycle Ti.

[0020] The load residual prediction model is: in, are the load prediction residuals of time periods t, t-1, …, t-j+1 in the load cycle T, respectively. Here, are the load forecast values ​​of time periods t, t-1, …, t-j+1 in the load cycle T, respectively, E is the load residual prediction method, j is the method f E The order of According to the method f E The load prediction residual value for period t+1 in the predicted load cycle T.

[0021] The combined prediction model is in, is the load forecast value generated by the short-term forecast model, is the load forecast value generated by the long-term forecast model, is the load prediction residual generated by the residual prediction model, F is the combined prediction method, is the load value of time period t+1 in load cycle T predicted according to method F.

[0022] It should be further explained that the decision model based on reinforcement learning is a Markov decision model based on load cycle, and S is defined as c is the state set of service component c. The state of service component c in time period t (t = 0, 1, ..., n) of load cycle T (T = 1, 2, ...) can be defined as: in, The virtual unit vu of the deployment service component c j The number of service components c in state The cloud resource supply capabilities are: in, is the virtual unit vu j For the maximum load capacity of service component c, let s φ =(0,0,…,0) is the termination state, which means stopping the provision of cloud resources; and are the initial state and the final state of the load cycle T, respectively, and That is, the state at the end of a load cycle is also the initial state of the next load cycle.

[0023] Definition A c is the action set of service component c. The action of service component c in time period t (t = 0, 1, ..., n) of load cycle T (T = 1, 2, ...) is defined as: in, For the status The set of all actions that can be performed under Indicates adjustment of virtual unit vu j If the number Indicates adding virtual unit vu j If the number Indicates reducing virtual vu j If the number This means that the virtual unit vu j The number of adjustments is defined as For the status The action to be performed is the action to be performed. After: If t < n, the state changes from to If t = n, the state changes from to

[0024] Define as the load prediction value of service component c at time t + 1 in the load cycle T, θ c as the control range of the load, a φ = {0, 0, …, 0} indicates that the null action of not adjusting the quantity of any virtual unit can generate the state action set

[0025] If it shows that the cloud resource supply capacity of service component c at time t in the load cycle T can meet the load demand at time t + 1. Therefore,

[0026] If it shows that the cloud resource supply capacity of service component c at time t in the load cycle T cannot meet the load demand at time t + 1. Let be the average load increase at time t + 1, then

[0027] If it shows that the load capacity of service component c at time t in the load cycle T exceeds the load demand at time t + 1. Let be the average load decrease at time t + 1, then

[0028] For each service component c, the reward function formula for executing the action in the state is defined as follows:

[0029]

[0030] Among them, is the cloud resource adjustment cost of the action ; If then it is necessary to increase the supply of cloud resources, and the reward function is defined as that is, the higher the cost of increasing cloud resources under the condition of ensuring service quality, the lower the reward, and vice versa, the higher the reward; If then it is necessary to reduce the supply of cloud resources, and the reward function is defined as that is, the higher the cost of reducing cloud resources under the condition of ensuring service quality, the higher the reward, and vice versa, the lower the reward; If there is no need to adjust the supply of cloud resources. Therefore,

[0031] Defining the state-action value function Represents in state Next action The cumulative bonus brought by the same state and action may have different rewards due to different load cycles and time periods; therefore, a strategy evaluation method based on temporal difference is used to adjust The value of the state-action cumulative reward is dynamically updated according to the following formula:

[0032]

[0033] Among them, γ T ∈[0,1] is the discount factor, γ T = 0 means only considering the immediate reward, as γ T tends to 1, indicating that future returns will be taken into account more; η T ∈[0,1] is the learning rate, η T =0 means no longer learning new knowledge, η T =1 means that only the latest information is useful; the value of the discount factor and the learning rate is related to the load cycle. As the load cycle T increases, γ T The value of increases gradually, and T The value of decreases gradually.

[0034] Further, the step S4 specifically includes the following steps:

[0035] S4.1: Set various initialization parameters of the model;

[0036] S4.2: Determine the state of the current period t in the current load cycle T Is it a terminal state? φ , if yes, go to S4.14, otherwise go to S4.3;

[0037] S4.3: Set the load period t to 0;

[0038] S4.4: Determine whether the load period t is less than the total number of periods n. If so, go to S4.5; otherwise, go to S4.13;

[0039] S4.5: Calculate load forecast value based on model combination forecasting method

[0040] S4.6: Determine if the model is in state The following action set Update the action set of service component c

[0041] S4.7: Use dynamic ε T - Greedy strategy to choose actions Update the optimal action set

[0042] S4.8: Observe according to the reward function formula (1) Generate new state

[0043] S4.9: Determine the new state generated Whether it belongs to the state set S of the existing service component c c , if yes, go to S4.11, if no, go to S4.10;

[0044] S4.10: New state Add to the state set S of service component c c middle;

[0045] S4.11: Update according to the cumulative reward formula (2)

[0046] S4.12: Update the current period t value and jump to S4.4;

[0047] S4.13: Update the load cycle T value and jump to S4.2;

[0048] S4.14: The algorithm ends and the output state set S c and the optimal action set a c .

[0049] Further, the step S4.7 specifically includes: dynamic ε T -The greedy strategy selects the action to be performed in each state. Next, with probability ε T exist Randomly select an action from the list, that is, explore, and with probability 1-ε T Choose the best action That is, use it. T Related to the load cycle, as the load cycle T increases, ε T The value of is gradually smaller.

[0050] The present invention also provides a cloud resource dynamic scaling system, the system comprising: a user load information acquisition module, a model building module, a function definition module, an algorithm configuration module and a virtual unit adjustment module;

[0051] The user load information acquisition module is used to obtain the user load on each service node cluster and the corresponding virtual unit of the virtual data center;

[0052] The model building module is used to build a load forecasting model based on model combination according to historical load information;

[0053] The function definition module is used to build a decision-making model based on reinforcement learning, determine the model's state, action, reward function, and state-action value function definition;

[0054] The algorithm configuration module is used to set the decision algorithm based on determining the cloud resource adjustment plan and the probabilistic greedy selection strategy, and determine the actions under the future load cycle;

[0055] The virtual unit adjustment module is used to build a cloud resource dynamic scaling executor and adjust the virtual unit.

[0056] It can be seen from the above technical solutions that the present invention has the following advantages:

[0057] The method for dynamic scaling of cloud resources based on load prediction and reinforcement learning provided by the present invention first establishes a load prediction model based on model combination based on each service node cluster of the virtual data center and the user historical load information on the corresponding virtual unit. Secondly, a decision model based on reinforcement learning is established. Through intelligent decision-making and continuous interaction with the cloud environment, the understanding of its operating rules is gradually deepened, and then the optimal cloud resource adjustment scheme adopted under different states is learned. Then, an efficient solution algorithm is configured, that is, a decision algorithm based on a probability greedy selection strategy, to solve the action sequence and formulate a cloud resource dynamic scaling scheme. Finally, the cloud resource dynamic scaling executor adjusts the cloud resources according to the scheme.

[0058] The present invention introduces a virtual data center to model cloud resources for large-scale service systems in public cloud environments, standardizes the corresponding relationship between service systems and cloud resources, and simplifies the complexity of the problem. For load prediction, the prediction model based on model combination proposed by the present invention generates a combined prediction value by integrating a short-term prediction model based on the same load cycle and a long-term prediction model based on a historical load cycle, and then performs a residual prediction on the combined prediction value and the true value to correct the error of the combined load prediction. This method can solve the volatility and irregularity of the load at a specific time, and further improve the accuracy of the load prediction results. For cloud resource scaling decisions, the decision model based on reinforcement learning provided by the present invention continuously learns the laws of dynamic scaling of cloud resources, formulates the optimal cloud resource adjustment plan under different states, and minimizes the cost while meeting the service quality requirements, which has practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0060] Figure 1 This is a flow chart of a method for dynamic scaling of cloud resources based on load prediction and reinforcement learning;

[0061] Figure 2 is a flow chart of a load prediction method based on model combination of the present invention;

[0062] Figure 3 is a schematic diagram of a decision model based on reinforcement learning of the present invention;

[0063] Figure 4 It is a flow chart of the decision algorithm of the probability-based greedy selection strategy of the present invention. DETAILED DESCRIPTION

[0064] The cloud resource dynamic scaling method based on load prediction and reinforcement learning provided by the present invention is based on cloud computing and virtualization technology, and is a new computing model that delivers computing, storage, network and other resources to users in a pay-as-you-go manner. It can build an accurate load prediction model in a random and open cloud environment and design an adaptive cloud resource dynamic scaling strategy.

[0065] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] See also Figure 1 FIG. 1 is a flowchart of a method for dynamic scaling of cloud resources based on load prediction and reinforcement learning in a specific embodiment. The method includes the following steps:

[0067] S1: Obtain each service node cluster of the virtual data center and the user load on the corresponding virtual unit.

[0068] A virtual data center can be defined as a cloud resource provisioning solution consisting of a group of service nodes, service node clusters, and the connections between them; a service component is defined as an encapsulation of a group of services and their dependencies. To meet service requirements, a service component needs to be instantiated into multiple service component instances.

[0069] A service node is defined as a virtual unit instance that deploys a service component instance. All virtual unit instances that deploy the same service component are a service node cluster. Where S = {s1, s2, …, k} is the service set requested by the user, VU={vu0,vu1,…, m} is a collection of virtual units, vu0 is the virtual unit of the control node, vu j (j=1,…,m) is the virtual unit of the service node, m is the number of service nodes, and the virtual units of different service nodes may be the same; define the user service set S in the virtual unit vu j The load distribution on is: Δ j ={λ 1j ,λ 2j ,…,λ kj}, where λ ij (i=1,2,..,k) is service s i In the virtual unit vu j The service cluster The load distribution is Δ={λ1,λ2,…,λ k},in

[0070] S2: Based on historical load information, a load forecasting model based on model combination is constructed.

[0071] This step specifically includes: Figure 2 The flow chart of the load forecasting method based on model combination is described, and the models include short-term load forecasting model, long-term load forecasting model, load residual forecasting model and combined forecasting model.

[0072] S2.1: Build a short-term load forecasting model

[0073] in, are the actual loads in time periods t, t-1, ..., t-m+1 in the load cycle T, respectively, S is the short-term load forecasting method, j is the method f S The order of According to the method f S The predicted load value for period t+1 in the load cycle T.

[0074] S2.2: Build a long-term load forecasting model in, is the actual load value of period t+1 in load cycle T-1, is the actual load value of time period t+1 in load cycle T-2, ..., is the actual load value of period t+1 in load cycle Ti, fL is the long-term load forecasting method, i is the method f L The order of According to the method f L The predicted load value for period t+1 in the load cycle Ti.

[0075] S2.3: Constructing load residual prediction model

[0076] in, are the load prediction residuals of time periods t, t-1, …, t-j+1 in the load cycle T, respectively. Here, are the load forecast values ​​of time periods t, t-1, …, t-j+1 in the load cycle T, respectively, E is the load residual prediction method, j is the method f E The order of According to the method f E The load prediction residual value for period t+1 in the predicted load cycle T.

[0077] S2.4: Build a combined prediction model

[0078] in, is the load forecast value generated by the short-term forecast model, is the load forecast value generated by the long-term forecast model, is the load prediction residual generated by the residual prediction model, F is the combined prediction method, is the load value of time period t+1 in load cycle T predicted according to method F.

[0079] S3: Build a decision-making model based on reinforcement learning, determine the model's state, action, reward function and state-action value function definition.

[0080] The decision model based on reinforcement learning is a Markov decision model based on the load cycle. Define S c is the state set of service component c. The state of service component c in time period t (t = 0, 1, ..., n) of load cycle T (T = 1, 2, ...) can be defined as: in, The virtual unit vu of the deployment service component c j The number of service components c in state The cloud resource supply capabilities are: in, is the virtual unit vu j For the maximum load capacity of service component c, let s φ =(0,0,…,0) is the termination state, which means stopping the provision of cloud resources; and are the initial state and the termination state of the duty cycle T respectively, and there is That is, the state in the last period of a duty cycle is also the initial state of the next duty cycle.

[0081] Define A c as the action set of the service component c. The action of the service component c at time t (t = 0, 1,..., n) in the duty cycle T (T = 1, 2,...) is defined as: where is the set of all actions that can be executed in the state .

[0082] represents adjusting the number of virtual units vu j .

[0083] If represents increasing the number of virtual units vu j .

[0084] If represents decreasing the number of virtual vu j .

[0085] If then it means not adjusting the number of virtual units vu j .

[0086] Define as the action executed in the state . After executing the action : If t < n, the state changes from to If t = n, the state changes from to

[0087] Define as the load prediction value of the service component c at time t + 1 in the duty cycle T, θ c is the control range of the load, a φ = {0, 0,..., 0} represents the null action of not adjusting the number of any virtual units. The action set of generating the state can be as follows

[0088] If it shows that the cloud resource supply capacity of the service component c at time t in the duty cycle T can meet the load demand at time t + 1. Therefore

[0089] If This means that the cloud resource supply capacity of service component c in period t of load cycle T cannot meet the load demand of period t+1. is the average load increase in period t+1, then

[0090] if This indicates that the load capacity of service component c in period t of load cycle T exceeds the load demand in period t+1. is the average load reduction in period t+1, then

[0091] For each service component c, in state Next action The reward function formula is defined as follows:

[0092]

[0093] in, For Action The cloud resource adjustment cost; if Then it is necessary to increase the supply of cloud resources, and the reward function is defined as That is, under the condition of ensuring service quality, the higher the cost of cloud resources, the lower the reward, and vice versa; if Then it is necessary to reduce the supply of cloud resources, and the reward function is defined as That is, under the condition of ensuring service quality, the higher the cost of reducing cloud resources, the higher the reward, and vice versa; if There is no need to adjust the supply of cloud resources, so

[0094] Defining the state-action value function Represents in state Next action The cumulative bonus brought by the same state and action may have different rewards due to different load cycles and time periods; therefore, a strategy evaluation method based on temporal difference is used to adjust The value of the state-action cumulative reward is dynamically updated according to the following formula:

[0095]

[0096] Among them, γ T ∈[0,1] is the discount factor, γ T = 0 means only considering the immediate reward, as γ T tends to 1, indicating that future returns will be taken into account more; η T ∈[0,1] is the learning rate, η T=0 means no longer learning new knowledge, η T =1 means that only the latest information is useful; the value of the discount factor and the learning rate is related to the load cycle. As the load cycle T increases, γ T The value of increases gradually, and T The value of decreases gradually.

[0097] The present invention is attached Figure 3 The schematic diagram of the decision model based on reinforcement learning is described. The model determines the cloud resource scaling strategy for each service component c∈C in each load period t(t=0,1,…,n) of each load cycle T(T=1,2,…). Through continuous interaction with the cloud environment, the model will gradually deepen its understanding of its operating rules and learn the optimal cloud resource adjustment plan.

[0098] S4: Configure a decision algorithm for determining a probability-based greedy selection strategy for cloud resource adjustment plans to determine actions under future load cycles.

[0099] The present invention is attached Figure 4 The decision algorithm flow of the probability-based greedy selection strategy is described. S4 also includes the following steps:

[0100] S4.1: Set the initialization parameters of the model.

[0101] S4.2: Determine the state of the current period t in the current load cycle T Is it a terminal state? φ , if yes, go to S4.14, otherwise go to S4.3.

[0102] S4.3: Set the load period t to 0.

[0103] S4.4: Determine whether the load period t is less than the total number of periods n. If so, go to S4.5; otherwise, go to S4.13.

[0104] S4.5: Calculate load forecast value based on model combination forecasting method

[0105] S4.6: Determine if the model is in state The following action set Update the action set of service component c

[0106] S4.7: Use dynamic ε T - Greedy strategy to choose actions Update the optimal action set

[0107] In this step, the dynamic ε T-The greedy strategy selects the action to be performed in each state. Next, with probability ε T exist Randomly select an action from the list, that is, explore, and with probability 1-ε T Choose the best action That is, use it. T Related to the load cycle, as the load cycle T increases, ε T The value of is gradually smaller.

[0108] S4.8: Observe according to the reward function formula (1) Generate new state

[0109] S4.9: Determine the new state generated Whether it belongs to the state set S of the existing service component c c , if yes, go to S4.11, if no, go to S4.10.

[0110] S4.10: New state Add to the state set S of service component c c middle.

[0111] S4.11: Update according to the cumulative reward formula (2)

[0112] S4.12: Update the current period t value and jump to S4.4.

[0113] S4.13: Update the load cycle T value and jump to S4.2.

[0114] S4.14: The algorithm ends and the output state set S c and the optimal action set a c .

[0115] S5: Build a cloud resource dynamic scaling executor to adjust the virtual unit instance. Here, the tools provided by the public cloud environment are used to adjust the virtual unit instance and complete the dynamic scaling of cloud resources.

[0116] The following is an embodiment of a cloud resource dynamic scaling system provided by an embodiment of the present disclosure. The system and the cloud resource dynamic scaling method of the above embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the cloud resource dynamic scaling system, reference can be made to the embodiment of the above cloud resource dynamic scaling method.

[0117] The system includes: a user load information acquisition module, a model building module, a function definition module, an algorithm configuration module and a virtual unit adjustment module;

[0118] The user load information acquisition module is used to obtain the user load on each service node cluster and the corresponding virtual unit of the virtual data center;

[0119] The model building module is used to build a load forecasting model based on model combination according to historical load information;

[0120] The function definition module is used to build a decision-making model based on reinforcement learning, determine the model's state, action, reward function, and state-action value function definition;

[0121] The algorithm configuration module is used to set the decision algorithm based on determining the cloud resource adjustment plan and the probabilistic greedy selection strategy, and determine the actions under the future load cycle;

[0122] The virtual unit adjustment module is used to build a cloud resource dynamic scaling executor and adjust the virtual unit.

[0123] The units and algorithm steps of each example described in the embodiments disclosed in the cloud resource dynamic scaling method and system involved in the present invention can be implemented by electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0124] The flowchart of the cloud resource dynamic scaling method involved in the present invention illustrates the possible implementation architecture, functions and operations of the devices, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a portion of a code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. Exemplarily speaking, two boxes represented in succession can actually be executed substantially in parallel, and they may sometimes be executed in the opposite order, depending on the functions involved.

[0125] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0126] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A cloud resource dynamic scaling method, characterized in that: Methods include: S1: Obtain the user load on each service node cluster and the corresponding virtual unit in the virtual data center; The virtual data center is defined as a cloud resource provisioning solution consisting of a service node, a service node cluster, and a connection between the service node and the service node cluster; A service component is defined as the encapsulation of a set of services and their dependencies, and a service component is instantiated into multiple service component instances; A service node is defined as a virtual unit instance that deploys a service component instance. All virtual unit instances that deploy the same service component are a service node cluster. Where S is the user service set, S = {s1, s2, ..., s k }; VU is a set of virtual units, VU = {vu0, vu1, ..., vu m }; vu0 is the virtual unit of the control node, vu j is the virtual unit of the service node, j=1,…,m; m is the number of service nodes, and the virtual units of different service nodes are the same; Define the user service set S in the virtual unit vu j superior, Virtual unit vu j The load distribution is: Δ j ={λ 1j ,λ 2j ,…,λ kj }; where λ ij For services i In the virtual unit vu j The load on; i = 1, 2, .., k; Service Node Cluster The load distribution is Δ={λ1,λ2,…,λ k },in S2: Based on the historical load information, a load forecasting model based on model combination is constructed, including: short-term load forecasting model, long-term load forecasting model, load residual forecasting model and combined forecasting model; The short-term load forecasting model is: in, are the actual loads in time periods t, t-1, …, t-j+1 in the load cycle T, respectively, and f S is the short-term load forecasting method, j is the short-term load forecasting method f S The order of According to the short-term load forecasting method f S The predicted load value of time period t+1 in the load cycle T; The long-term load forecasting model is: in, is the actual load value of period t+1 in load cycle T-1, is the actual load value of period t+1 in load cycle T-2; is the actual load value of period t+1 in load cycle Ti, f L is the long-term load forecasting method, i is the method f L The order of According to the method f L The predicted load value of time period t+1 in the load cycle T; The load residual prediction model is: in, are the load prediction residuals of time periods t, t-1,…, t-j+1 in the load cycle T respectively; are the load forecast values ​​of time periods t, t-1, and t-j+1 in the load cycle T, respectively, E is the load residual prediction method, j is the method f E The order of According to the method f E The load prediction residual value of the period t+1 in the predicted load cycle T; The combined prediction model is in, is the load forecast value generated by the short-term forecast model, is the load forecast value generated by the long-term forecast model, is the load prediction residual generated by the residual prediction model, F is the combined prediction method, is the load value of time period t+1 in load cycle T predicted by combined prediction method F; S3: Build a decision-making model based on reinforcement learning, determine the state and action of the model, and define the reward function and state-action value function; The decision model based on reinforcement learning in step S3 is a Markov decision model based on load cycle, and S is defined as c is the state set of service component c; The state of service component c in time period t of load cycle T is defined as: Among them, T=1,2,……,t=0,1,……n, Virtual unit vu for deploying service component c j the number of Service component c is in state The cloud resource supply capabilities are: in, is the virtual unit vu j For the maximum load capacity of service component c; Assume φ =(0,0,…,0) is the termination state, which means stopping the provision of cloud resources; and are the initial state and the final state of the load cycle T, respectively, and That is, the state at the end of a load cycle is also the initial state of the next load cycle; In the method, define A c is the action set of service component c. The action of service component c in time period t of load cycle T is defined as: Where, T=1,2,…; t=0,1,…,n; For the status The set of all actions performed under Indicates adjustment of virtual unit vu j The number of; j = 0, 1, ..., m; if Indicates adding virtual unit vu j the number of if Indicates the reduction of virtual unit vu j the number of if This means that the virtual unit vu j The quantity is adjusted; In the method, we also define For the status The action to be performed is the action to be performed. back: If t < n, the state transitions from to If t = n, the state changes from Transform to definition is the load prediction value of service component c in period t+1 of load cycle T, θ c is the load control range, a φ ={0,0,…,0} represents an empty action that does not adjust the number of any virtual units. The state can be generated according to the following rules Action set if This means that the cloud resource supply capacity of service component c in period t of load cycle T can meet the load demand of period t+1, so if This means that the cloud resource supply capacity of service component c in period t of load cycle T cannot meet the load demand of period t+1. is the average load increase in period t+1, then if This indicates that the load capacity of service component c in period t of load cycle T exceeds the load demand in period t+1. is the average load reduction in period t+1, then In the method, for each service component c, in state Next action The reward function formula is defined as follows: in, For Action The cost of adjusting cloud resources; if Then increase the supply of cloud resources, and the reward function is defined as if Then reduce the supply of cloud resources, and the reward function is defined as if The supply of cloud resources is not adjusted, so Defining the state-action value function Represents in state Next action The cumulative rewards brought about; Adopting the strategy evaluation method based on temporal difference to adjust The value of the state-action cumulative reward is dynamically updated according to the following formula: Among them, γ T ∈[0,1] is the discount factor, γ T = 0 means immediate reward, with γ T If it tends to 1, it means that future returns will be taken into account; η T ∈[0,1] is the learning rate, η T =0 means no longer learning new knowledge, η T =1 means that only the latest information is useful; the value of the discount factor and the learning rate is related to the load cycle. As the load cycle T increases, γ T The value of gradually increases, and η T The value of decreases gradually; S4: configure the decision algorithm for determining the cloud resource adjustment plan and the probabilistic greedy selection strategy, and determine the actions under the future load cycle; S4.1: Set various initialization parameters of the model; S4.2: Determine the state of the current period t in the current load cycle T Is it a terminal state? φ , if yes, go to S4.14, otherwise go to S4.3; S4.3: Set the load period t to 0; S4.4: Determine whether the load period t is less than the total number of periods n. If so, go to S4.5; otherwise, go to S4.13; S4.5: Calculate load forecast value based on model combination forecasting method S4.6: Determine if the model is in state The following action set Update the action set of service component c S4.7: Use dynamic ε T - Greedy strategy to choose actions Update the optimal action set Dynamic ε T -The greedy strategy also includes: selecting the action to be performed in each state; In Status Next, with probability ε T exist Randomly select an action from the list, explore, and with probability 1-ε T Choose the best action to make use of; ε T Related to the load cycle, as the load cycle T increases, ε T The value of is gradually smaller; S4.8: Observe according to the reward function formula (1) Generate new state S4.9: Determine the new state generated Whether it belongs to the state set S of the existing service component c c , if yes, go to S4.11, if no, go to S4.10; S4.10: New state Add to the state set S of service component c c middle; S4.11: Update according to the cumulative reward formula (2) S4.12: Update the current period t value and jump to S4.4; S4.13: Update the load cycle T value and jump to S4.2; S4.14: The algorithm ends and the state set S is output c and the optimal action set a c ; S5: Build a cloud resource dynamic scaling executor to adjust the virtual unit.

2. A cloud resource dynamic scaling system, characterized in that: The system adopts the cloud resource dynamic scaling method as claimed in claim 1; the system comprises: a user load information acquisition module, a model building module, a function definition module, an algorithm configuration module and a virtual unit adjustment module; The user load information acquisition module is used to obtain the user load on each service node cluster and the corresponding virtual unit of the virtual data center; The model building module is used to build a load forecasting model based on model combination according to historical load information; The function definition module is used to build a decision-making model based on reinforcement learning, determine the model's state, action, reward function, and state-action value function definition; The algorithm configuration module is used to set the decision algorithm based on determining the cloud resource adjustment plan and the probabilistic greedy selection strategy, and determine the actions under the future load cycle; The virtual unit adjustment module is used to build a cloud resource dynamic scaling executor and adjust the virtual unit.

Images