Cloud resource allocation method in cloud computing environment, electronic device and storage medium

Abstract

This application discloses a cloud resource allocation method in a cloud computing environment, the cloud computing environment including a host and multiple virtual machines / containers, the multiple virtual machines / containers being used to deploy and run cloud computing applications. The method includes: obtaining system metrics of the host and the multiple virtual machines / containers when the cloud computing application runs on the multiple virtual machines / containers; using the system metrics of the host and the system metrics of the multiple virtual machines / containers to perform reinforcement learning to obtain resource allocation strategies for the multiple virtual machines / containers, the resource allocation strategies for the multiple virtual machines / containers being used to represent the resource allocation of the respective virtual machines / containers. This application also discloses electronic devices and storage media. This application improves the resource management performance of dynamic cloud computing.

Description

Technical Field

[0001] The disclosed embodiments of this application relate to the field of artificial intelligence technology, and more specifically, to a cloud resource allocation method, electronic device, and storage medium in a cloud computing environment. Background Technology

[0002] Currently, reinforcement learning (RL) is a powerful machine learning method that is widely used in the field of cloud computing resource allocation. However, the cloud computing environment is affected by various factors such as seasonal changes, market trends, and sudden events. When the environment changes rapidly, traditional reinforcement learning algorithms cannot adapt quickly, leading to a decrease in the effectiveness of resource allocation strategies. Furthermore, as the scale of cloud computing expands, problems such as high algorithm complexity and long training time arise. Summary of the Invention

[0003] According to embodiments of this application, this application proposes a cloud resource allocation method, electronic device, and storage medium in a cloud computing environment to solve the above-mentioned problems.

[0004] The first aspect of this application discloses a cloud resource allocation method in a cloud computing environment, the cloud computing environment including a host and multiple virtual machines / containers, the multiple virtual machines / containers being used to deploy and run cloud computing applications, the method comprising: obtaining system metrics of the host and system metrics of the multiple virtual machines / containers when the cloud computing application is running on the multiple virtual machines / containers; and performing reinforcement learning using the system metrics of the host and the system metrics of the multiple virtual machines / containers to obtain resource allocation strategies for the multiple virtual machines / containers, the resource allocation strategies for the multiple virtual machines / containers being used to represent the resource allocation of the respective virtual machines / containers.

[0005] In some embodiments, obtaining system metrics of the plurality of virtual machines / containers includes: determining a polling order of the plurality of virtual machines / containers; polling the plurality of virtual machines / containers according to the polling order to obtain the system metrics of the current virtual machine / container in the plurality of virtual machines / containers within a polling cycle; and performing reinforcement learning using the system metrics of the host and the system metrics of the plurality of virtual machines / containers includes: performing reinforcement learning using the system metrics of the host and the system metrics of the current virtual machine / container to obtain a resource allocation strategy for the current virtual machine / container, wherein the resource allocation strategy of the current virtual machine / container is used to represent the resource allocation of the current virtual machine / container.

[0006] In some embodiments, reinforcement learning is performed using the system metrics of the host and the system metrics of the current virtual machine / container to obtain the resource allocation strategy of the current virtual machine / container. This includes: extracting features from the system metrics of the host and the system metrics of the current virtual machine / container to obtain system indication feature data; performing LSTM-based time series prediction on the system indication feature data to obtain service quality perception results; and learning based on the service quality perception results to obtain the resource allocation strategy of the current virtual machine / container.

[0007] In some embodiments, learning is performed based on the service quality perception results to obtain the resource allocation strategy for the current virtual machine / container, including: obtaining the current system metrics of the host and the current system metrics of the current virtual machine / container; and using a meta-hierarchical network to learn the resource allocation strategy for the current virtual machine / container based on the current system metrics of the host, the current system metrics of the current virtual machine / container, and the service quality perception results.

[0008] In some embodiments, the meta-hierarchical network includes a main policy network and multiple sub-policy networks. Learning using the meta-hierarchical network based on the current system metrics of the host, the current system metrics of the current virtual machine / container, and the service quality perception result includes: using the main policy network to determine a target sub-policy network from the multiple sub-policy networks based on the current system metrics of the host and the current system metrics of the current virtual machine / container; and using the target sub-policy network to learn based on the current system metrics of the host, the current system metrics of the current virtual machine / container, and the service quality perception result.

[0009] In some embodiments, when learning is performed using the target sub-policy network, the main policy network is used as part of the learning environment, and the target sub-policy network learns using a preset active inference algorithm.

[0010] In some embodiments, the service quality perception result is used to learn a reward function using the target sub-policy network.

[0011] In some embodiments, the goal of learning using a preset active inference algorithm is to minimize the free energy, which includes a first value and a second value, wherein the first value represents information obtained from executing a specific strategy, and the second value represents the information divergence between the predicted future observation sequence and the preferred observation sequence.

[0012] In some embodiments, the LSTM comprises two layers, and the number of historical steps / length is configured to be any one of 0, 1, 3, and 5.

[0013] The second aspect of this application discloses an electronic device including a memory and a processor coupled to each other, the processor being configured to execute program instructions stored in the memory to implement the cloud resource allocation method in the cloud computing environment described in the first aspect.

[0014] The third aspect of this application discloses a non-volatile computer-readable storage medium storing program instructions thereon, which, when executed by a processor, implement the cloud resource allocation method in the cloud computing environment described in the first aspect.

[0015] The beneficial effects of this application are as follows: It obtains system metrics of the host and multiple virtual machines / containers when a cloud computing application is running on multiple virtual machines / containers; it uses the host's system metrics and the system metrics of multiple virtual machines / containers for reinforcement learning to obtain resource allocation strategies for multiple virtual machines / containers; the resource allocation strategies of multiple virtual machines / containers are used to represent the resource allocation of the corresponding virtual machines / containers; and by using the host's system metrics and the system metrics of multiple virtual machines / containers for reinforcement learning, it analyzes the current cloud computing environment and obtains corresponding resource allocation strategies, which can quickly adapt to different scenarios and improve the resource management performance of dynamic cloud computing. Attached Figure Description

[0016] The present application will be further described below with reference to the accompanying drawings and embodiments. In the drawings:

[0017] Figure 1 This is a schematic diagram of a cloud computing environment according to an embodiment of this application;

[0018] Figure 2 This is a flowchart illustrating a cloud resource allocation method in a cloud computing environment according to an embodiment of this application.

[0019] Figure 3 This is a polling diagram of a cloud resource allocation method in a cloud computing environment according to an embodiment of this application;

[0020] Figure 4 This is a schematic diagram of the meta-hierarchical network in an embodiment of this application;

[0021] Figure 5 This is a comparative schematic diagram of cloud resource allocation in a cloud computing environment according to an embodiment of this application;

[0022] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application;

[0023] Figure 7 This is a schematic diagram of the structure of a non-volatile computer-readable storage medium according to an embodiment of this application. Detailed Implementation

[0024] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0025] In this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Additionally, the character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, "many" in this application means two or more. Moreover, the term "at least one" in this application means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. Furthermore, the terms "first," "second," and "third" in this application are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features.

[0026] Resource allocation in cloud computing environments has always been a research hotspot, with its main technical challenges stemming from the highly dynamic nature of the environment, low resource utilization, and limitations of predictive models. Cloud computing environments are influenced by various factors such as seasonal changes, market trends, and unforeseen events, leading to highly dynamic and unpredictable resource demands from tenants. For example, retail businesses often face surges in traffic during holidays, requiring increased computing resources and additional storage. Similarly, major social events may temporarily increase demand for specific services. Furthermore, the vast differences in business characteristics, workloads, and Quality of Service (QoS) requirements among different tenants result in diverse resource demands. This high dynamism makes traditional static resource allocation strategies inadequate. To ensure QoS, traditional resource allocation strategies often allocate resources exceeding tenants' average needs to cope with peak loads. However, this leads to significant resource waste and increases the operating costs for cloud service providers. Traditional predictive models struggle to accurately capture the complexity and variability of tenant demands, resulting in inaccurate resource allocation decisions.

[0027] Reinforcement learning (RL) is applied to cloud resource allocation. Traditional RL algorithms need to find a balance between exploring new state-action pairs and utilizing existing knowledge to improve long-term returns. Traditional RL methods are often customized for specific Markov Decision Processes (MDPs), and when new environments (e.g., new MDP scenarios) arise, retraining is often required, leading to inefficiency and resource waste. Furthermore, the overemphasis on exploration in traditional RL algorithms can result in low resource utilization, while over-reliance on existing knowledge may miss better allocation strategies. They typically assume a static or slowly changing environment, which is inconsistent with the highly dynamic nature of cloud environments. That is, when the environment changes rapidly, such as a sudden increase or decrease in workload, traditional RL algorithms often struggle to adapt quickly, leading to a decline in the effectiveness of resource allocation strategies. Simultaneously, as cloud computing scales up, the dimensionality of the state and action spaces that reinforcement learning algorithms need to consider increases dramatically. High-dimensional state and action spaces pose significant challenges to algorithm training and convergence, leading to increased algorithm complexity and longer training times.

[0028] To address these issues, this application proposes a cloud resource allocation method, electronic device, and storage medium in a cloud computing environment.

[0029] To enable those skilled in the art to better understand the technical solution of this application, the technical solution of this application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] Please see Figure 1 , Figure 1This is a schematic diagram of a cloud computing environment according to an embodiment of this application. The cloud computing environment 10 includes a host 110 and multiple virtual machines (VMs) / containers 120 to provide services to clients via a network. The host 110 can be a physical machine equipped with cloud service resources (e.g., CPU, memory, disk, and network interface card (NIC)). The multiple VMs / containers 120 are used to deploy and run cloud computing applications. Each VM / container 120 runs one application, which may be the same, different, or partially the same. Cloud computing applications include Cassandra, Etcd, Hadoop, FFmpeg, Kafka, MongoDB, MySQL, RabbitMQ, Redis, and Spark. The cloud resource allocation method in the cloud computing environment of this embodiment is applicable to cloud resource allocation in dynamic environments. This can be a single changing environment or multiple changing environments with different scenarios. Examples include the surge in traffic often faced by retail businesses during holidays, the rapid increase in demand for specific streaming media services due to large sporting events, or the significant differences in business characteristics, workloads, and service quality requirements among different tenants in the same cloud computer room.

[0031] Please see Figure 2 , Figure 2 This is a flowchart illustrating a cloud resource allocation method in a cloud computing environment according to an embodiment of this application. The execution entity of this method can be an electronic device with computing capabilities, used to schedule and allocate cloud computing resources, such as a cloud computing scheduling engine, a microcomputer, a server, etc.

[0032] It should be noted that if substantially the same result is obtained, the method of this application is not based on... Figure 2 The sequence of processes shown is limited.

[0033] In some possible implementations, this method can be implemented by the processor calling computer-readable instructions stored in memory, such as... Figure 2 As shown, the method may include the following steps:

[0034] S21: Obtain system metrics of the host and multiple virtual machines / containers when the cloud computing application is running on multiple virtual machines / containers.

[0035] During the operation of a cloud computing application, the system metrics H of the corresponding host and the system metrics V of multiple virtual machines / containers are obtained. The system metrics H of the host includes parameters such as host CPU, host memory, host disk, and host network. The system metrics V of multiple virtual machines / containers include parameters such as virtual CPU, virtual memory, virtual disk, and virtual network.

[0036] S22: Using system metrics of the host and system metrics of multiple virtual machines / containers, reinforcement learning is performed to obtain resource allocation strategies for multiple virtual machines / containers. The resource allocation strategies for multiple virtual machines / containers are used to represent the resource allocation of the corresponding virtual machines / containers.

[0037] Reinforcement learning is performed using system metrics from the host machine and multiple virtual machines / containers to derive resource allocation strategies for each virtual machine / container. These strategies represent the resource allocation for their respective virtual machines / containers. For example, the acquired system metrics H and V are used as input state information for reinforcement learning training to output corresponding resource allocation strategies, thereby allocating appropriate CPU shares.

[0038] In this embodiment, system metrics of the host and multiple virtual machines / containers are obtained when the cloud computing application runs on multiple virtual machines / containers. Reinforcement learning is performed using the system metrics of the host and multiple virtual machines / containers to obtain resource allocation strategies for multiple virtual machines / containers. The resource allocation strategies of multiple virtual machines / containers are used to represent the resource allocation of the corresponding virtual machines / containers. By using the system metrics of the host and multiple virtual machines / containers to perform reinforcement learning, the current cloud computing environment can be analyzed to obtain the corresponding resource allocation strategies. This can quickly adapt to different scenarios and improve the resource management performance of dynamic cloud computing.

[0039] In some embodiments, obtaining system metrics of multiple virtual machines / containers includes: determining a polling order for the multiple virtual machines / containers; polling the multiple virtual machines / containers according to the polling order to obtain the system metrics of the current virtual machine / container within a polling cycle; and performing reinforcement learning using the system metrics of the host and the system metrics of the multiple virtual machines / containers, including: performing reinforcement learning using the system metrics of the host and the system metrics of the current virtual machine / container to obtain a resource allocation strategy for the current virtual machine / container, wherein the resource allocation strategy of the current virtual machine / container is used to represent the resource allocation of the current virtual machine / container.

[0040] For example, in a certain scenario, to obtain system metrics for M virtual machines / containers, such as... Figure 3 As shown, Figure 3 This is a schematic diagram of the polling method for cloud resource allocation in a cloud computing environment according to an embodiment of this application. The polling order of multiple virtual machines / containers is determined, for example, a polling queue Q1 for M virtual machines / containers is determined as {virtual machine / container 1, virtual machine / container 2, ..., virtual machine / container M}, where polling queue Q1 represents the polling order. Further, the M virtual machines / containers are polled according to polling queue Q1, for example, switching from virtual machine / container 1 to virtual machine / container 2 within a unit of time.

[0041] Within a polling cycle, the system metrics of the current virtual machine / container in M ​​virtual machines / containers are obtained. For example, if the current virtual machine / container is virtual machine / container 1, the agent (cloud computing scheduling engine) obtains the system metrics of the polled virtual machine / container 1 and the host as input for state information, performs reinforcement learning, and obtains the resource allocation strategy of virtual machine / container 1. The resource allocation strategy of virtual machine / container 1 is used to represent the resource allocation of virtual machine / container 1, such as the CPU share required for virtual machine / container 1 to run.

[0042] In this embodiment, multiple virtual machines / containers are polled in a polling order to obtain the system metrics of the current virtual machine / container within a polling cycle. Polling each virtual machine / container in sequence ensures fair and timely access to CPU resources, thereby maintaining the fairness and quality of service of the entire cloud computing environment. In other words, the container polling scheduling framework effectively reduces the dimensionality of the state and action space, enabling this method to run efficiently in larger-scale and more complex and dynamic cloud environments.

[0043] In some embodiments, reinforcement learning is performed using system metrics of the host and system metrics of the current virtual machine / container to obtain the resource allocation strategy of the current virtual machine / container. This includes: extracting features from the system metrics of the host and system metrics of the current virtual machine / container to obtain system indicator feature data; performing time series prediction based on LSTM (Long Short-Term Memory) on the system indicator feature data to obtain service quality perception results; and learning based on the service quality perception results to obtain the resource allocation strategy of the current virtual machine / container.

[0044] Feature extraction is performed on the system metrics of the host and the current virtual machine / container to obtain system indicator feature data. The system indicator feature data includes information on the CPU, memory, network and disk of the host and the current virtual machine / container, such as CPU utilization, CPU time waiting for I / O, memory utilization, disk I / O inbound and outbound speed, network I / O inbound and outbound speed, etc.

[0045] LSTM-based time series prediction is performed on the system indicator feature data to identify long-term dependencies in the time series data, thereby obtaining initial service quality perception results, which are represented as follows:

[0046]

[0047] Where i is the current virtual machine / container number, TPS represents the number of tasks processed per second, and Latency represents the average time delay for task completion.

[0048] For applications deployed in the cloud, task throughput and task response time are important metrics for measuring service quality; higher task throughput and shorter task response times generally indicate better service quality. Since service quality metrics vary across different application loads, the initial service quality perception results are normalized to obtain the service quality perception results, as shown below:

[0049]

[0050] Among them, QoS max This indicates the QoS that the load can achieve under ideal conditions, i.e., the maximum QoS that can be achieved.

[0051] Furthermore, reinforcement learning is performed based on the service quality perception results to obtain the current resource allocation strategy for virtual machines / containers. Understandably, in this embodiment, using the service quality perception results as the basis for resource allocation ensures that the cloud computing environment can accurately and dynamically allocate resources according to the actual needs of different applications to improve performance. Additionally, the novel container round-robin scheduling framework combined with LSTM ensures learning fairness, and accuracy can be improved by using LSTM to learn the historical behavior of virtual machines / containers.

[0052] In some embodiments, the LSTM comprises two layers, and the number of historical steps / length is configured to be any one of 0, 1, 3, and 5.

[0053] Time series prediction is performed on system indicator feature data of hosts and virtual machines / containers based on LSTM. The LSTM layer can be configured to have 2 layers, and the number of history steps / length can be set to any of 0, 1, 3, and 5. In this embodiment, the performance of the relevant algorithm can be improved with the increase of history length, but a longer history length may complicate training and increase the risk of overfitting. Therefore, the history length is selected based on a balance between performance and complexity.

[0054] In some embodiments, learning is performed based on the service quality perception results to obtain the resource allocation strategy for the current virtual machine / container, including: obtaining the current system metrics of the host and the current system metrics of the current virtual machine / container; and using a meta-level network to learn the resource allocation strategy for the current virtual machine / container based on the current system metrics of the host, the current system metrics of the current virtual machine / container, and the service quality perception results.

[0055] For example, obtaining the current system metrics of the host and the current system metrics of the current virtual machine / container, i.e., obtaining the current state S. t , means as follows:

[0056]

[0057] Among them, S t This indicates the state at time t, where i is the virtual machine / container number. Its sub-items represent the system indicator characteristic data of the host and the virtual machine / container at time t, including CPU, memory, network, and disk characteristic information.

[0058] Furthermore, based on the current system metrics and service quality perception results of the host and the current virtual machine / container, the resource allocation strategy of the current virtual machine / container can be obtained by learning using a meta-hierarchical network. In this case, learning using a meta-hierarchical network means introducing a meta-hierarchical reinforcement learning (MHRL) algorithm for training.

[0059] During the learning process, the action space A can be represented as follows:

[0060] A = {c|c∈[0,C}} r ]},

[0061] Where c represents the CPU share allocated by the agent to virtual machine / container i, C r This indicates the remaining CPU share available for allocation when the system makes a decision.

[0062] In some embodiments, the meta-hierarchical network includes a main policy network and multiple sub-policy networks. For example... Figure 4 As shown, Figure 4 This is a schematic diagram of a meta-hierarchical network according to an embodiment of this application. The meta-hierarchical network includes a master policy network with network parameter ψ and a set of networks with network parameters ω1, ω2, ... ω3. y A network of y sub-policy networks.

[0063] Accordingly, using the master policy network, based on the host's current system metrics and the current virtual machine / container's current system metrics, the target sub-policy network is determined from multiple sub-policy networks. That is, after receiving a resource allocation request, the agent uses the master policy network based on the current state S. t Designate a sub-policy network as the target sub-policy network, where the main policy network is determined based on the current state S. t The process of specifying a sub-policy network is as follows:

[0064] sub k ~π(sub) k |s t ), 1≤k≤y,

[0065] Where π represents the main strategy generation based on S t Sub-strategy k The probability density, where k represents the index of the sub-policy.

[0066] Furthermore, the parameters of the main policy network are updated using the gradient method as follows:

[0067]

[0068] Where μ represents the learning rate, r H The master policy network is represented by a slave in state S. t Sub-policy generation k The reward that can be obtained, β represents the decay factor, and π ψ This indicates that the main policy network is in a given state S. t In the case of generating sub-policy Sub k The probability, R H This represents the main policy reward function.

[0069] The master policy network determines the current state S based on the main policy network. t Specify Sub-Policy Network k After defining the target sub-policy network, the target sub-policy network is used to learn based on the current system metrics of the host, the current system metrics of the current virtual machine / container, and the service quality perception results.

[0070] Furthermore, in some embodiments, when learning is performed using a target sub-policy network, the main policy network is used as part of the learning environment, and the target sub-policy network learns using a preset active inference algorithm.

[0071] In learning using the target sub-policy network, the main policy network is used as part of the learning environment. That is, after selecting the sub-policy network, the algorithm transitions to a specific resource scheduling phase, and the main policy network is regarded as part of the environment, and the active inference algorithm is used as the sub-policy network.

[0072] The objective of learning using the pre-defined active inference algorithm is to minimize the free energy, denoted as:

[0073]

[0074] The free energy F includes a first value and a second value. The first value represents the information gained from executing a specific policy, i.e., a quantification of the amount of information the agent expects to gain from executing a specific policy, which can be called information gain to improve prediction accuracy. The second value represents the information divergence (KL divergence) between the predicted future observation sequence and the preferred observation sequence. The future observation sequence can be a sequence of states predicted based on LSTM.

[0075] In the embodiments of this application, the active reasoning algorithm is combined to alleviate the performance instability caused by the imbalance between exploration and utilization. The agent can no longer make decisions by passively receiving environmental information and rewards. In other words, when resource demand changes rapidly due to multiple factors, it can quickly adapt to such changes by combining learned meta-knowledge with active reasoning.

[0076] In some embodiments, the service quality perception result is used for a reward function learned using the target sub-policy network.

[0077] Given the perceived service quality, it is represented as follows:

[0078]

[0079] At this point, the average service quality perception result of the entire cloud computing environment is used as the standard for evaluating performance, thus obtaining the reward function used for learning with the target sub-policy network:

[0080]

[0081] Where, N vm This represents the number of virtual machines in the system, and α is the coefficient for reward scaling adjustment.

[0082] To facilitate understanding, examples are provided to illustrate the application scenarios of the cloud resource allocation method in the cloud computing environment of this application embodiment.

[0083] For example, in a cloud computing environment, a physical machine with 60 CPU cores, 120GB of RAM, and running Ubuntu 18.04 is used as the virtualization method, with each Docker container configured with 6 virtual CPUs and 4GB of RAM, running on CentOS 7.2. Furthermore, each container deploys a different application, including but not limited to Cassandra, Etcd, Hadoop, FFmpeg, Kafka, MongoDB, MySQL, RabbitMQ, Redis, and Spark.

[0084] The baseline algorithm can be the MHRL algorithm. For example, for the MHRL algorithm, a uniform discount rate of 0.99 and a batch size of 128 are set. Both the main policy network and the sub-policy network have two hidden layers, each with 256 units. The learning rate of the main policy network is 0.0003, the learning rate of the sub-policy network is 0.001, the LSTM layer is configured with two layers, and the number of historical steps is set to 5.

[0085] At this point, the highly dynamic cloud environment includes Scenario 1, Scenario 2, and Scenario 3. Each scenario uses the same physical devices and containers, but differs in the patterns of load size variation. For example, the load sizes of Scenario 1, Scenario 2, and Scenario 3 are divided into three levels: [1, 2, 3], corresponding to low, medium, and high loads, respectively. The load levels in these scenarios fluctuate according to specific state transition probability matrices. The state transition probabilities of Scenario 1, Scenario 2, and Scenario 3 are represented by matrices T1, T2, and T3, respectively.

[0086]

[0087] T represents the probability of transitioning from load level i to load level j within scenario p. In other words, the stability of each scenario can be distinguished by a matrix, and their average load size can be indicated.

[0088] like Figure 5 As shown, Figure 5 This is a comparative schematic diagram of cloud resource allocation in a cloud computing environment according to an embodiment of this application. It can be seen that in scenarios 1, 2 and 3, the average quality of service (QoS) that can be achieved using the MHRL algorithm, and the average QoS of the MHRL algorithm, steadily increase with the increase of the number of scenarios, showing considerable performance in various scenarios. In order to better demonstrate that the method still maintains a high performance level under dynamic scenarios and increased load levels, the PPO algorithm and the System scheduling algorithm are used as baseline algorithms for comparison.

[0089] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0090] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. The electronic device 60 includes a memory 61 and a processor 62 coupled to each other. The processor 62 is used to execute program instructions stored in the memory 61 to implement the steps of the cloud resource allocation method embodiment in the cloud computing environment described above. In a specific implementation scenario, the electronic device 60 may include, but is not limited to, a microcomputer or a server.

[0091] Specifically, processor 62 controls itself and memory 61 to implement the steps of the cloud resource allocation method embodiment in the cloud computing environment described above. Processor 62 can also be called a CPU (Central Processing Unit), and it may be an integrated circuit chip with signal processing capabilities. Processor 62 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 62 can be implemented using integrated circuit chips.

[0092] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a non-volatile computer-readable storage medium according to an embodiment of this application. The non-volatile computer-readable storage medium 70 is used to store a computer program 701. When the computer program 701 is executed by a processor, for example by the aforementioned... Figure 6 When the processor 62 in the embodiment is executed, it is used to implement the steps of the above-described embodiment of the cloud resource allocation method for a cloud computing environment.

[0093] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0094] In the several embodiments provided in this application, it should be understood that the disclosed methods and related devices can be implemented in other ways. For example, the related device implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication disconnection shown or discussed may be indirect coupling or communication disconnection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0095] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0096] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0097] Those skilled in the art will readily recognize that numerous modifications and variations can be made to the apparatus and method while maintaining the teachings of this application. Therefore, the above disclosure should be considered limited only by the scope of the appended claims.

Claims

1. A method for allocating cloud resources in a cloud computing environment, characterized in that, The cloud computing environment includes a host and multiple virtual machines / containers, the multiple virtual machines / containers being used to deploy and run cloud computing applications, and the method comprising: Obtain the system metrics of the host and the system metrics of the multiple virtual machines / containers when the cloud computing application is running on the multiple virtual machines / containers; Reinforcement learning is performed using the system metrics of the host and the system metrics of the multiple virtual machines / containers to obtain the resource allocation strategies of the multiple virtual machines / containers. The resource allocation strategies of the multiple virtual machines / containers are used to represent the resource allocation of the corresponding virtual machines / containers.

2. The method according to claim 1, characterized in that, Obtain system metrics for the multiple virtual machines / containers, including: Determine the polling order of the plurality of virtual machines / containers; The plurality of virtual machines / containers are polled according to the polling order to obtain the system metrics of the current virtual machine / container in the plurality of virtual machines / containers within a polling cycle; The reinforcement learning process using the system metrics of the host and the system metrics of the multiple virtual machines / containers includes: Reinforcement learning is performed using the system metrics of the host and the system metrics of the current virtual machine / container to obtain the resource allocation strategy of the current virtual machine / container, which is used to represent the resource allocation of the current virtual machine / container.

3. The method according to claim 2, characterized in that, Using the system metrics of the host and the system metrics of the current virtual machine / container, reinforcement learning is performed to obtain the resource allocation strategy for the current virtual machine / container, including: Feature extraction is performed on the system metrics of the host and the system metrics of the current virtual machine / container to obtain system indicator feature data; The system indicator feature data is subjected to LSTM-based time series prediction to obtain service quality perception results; Based on the service quality perception results, learning is performed to obtain the resource allocation strategy for the current virtual machine / container.

4. The method according to claim 3, characterized in that, Based on the service quality perception results, learning is performed to obtain the current virtual machine / container resource allocation strategy, including: Obtain the current system metrics of the host and the current system metrics of the virtual machine / container; Based on the current system metrics of the host and the current system metrics of the virtual machine / container, as well as the service quality perception results, the resource allocation strategy of the current virtual machine / container is obtained by learning using a meta-hierarchical network.

5. The method according to claim 4, characterized in that, The meta-hierarchical network includes a main policy network and multiple sub-policy networks; Based on the current system metrics of the host and the current system metrics of the virtual machine / container, as well as the service quality perception results, learning is performed using a meta-hierarchical network, including: Using the main policy network, a target sub-policy network is determined from the plurality of sub-policy networks based on the current system metrics of the host and the current system metrics of the virtual machine / container. The target sub-policy network is used to learn based on the current system metrics of the host, the current system metrics of the virtual machine / container, and the service quality perception results.

6. The method according to claim 5, characterized in that, In the learning process using the target sub-policy network, the main policy network serves as part of the learning environment, and the target sub-policy network learns using a preset active inference algorithm.

7. The method according to claim 5 or 6, characterized in that, The service quality perception result is used as the reward function learned by the target sub-policy network.

8. The method according to claim 6, characterized in that, The goal of learning using a pre-defined active reasoning algorithm is to minimize the free energy, which includes a first value and a second value, wherein the first value represents the information obtained from executing a specific strategy, and the second value represents the information divergence between the predicted future observation sequence and the preferred observation sequence.

9. The method according to claim 3, characterized in that, The LSTM consists of two layers, and the number of historical steps / length is configured to be any one of 0, 1, 3, and 5.

10. An electronic device, characterized in that, It includes a memory and a processor coupled to each other, the processor being used to execute program instructions stored in the memory to implement the cloud resource allocation method in a cloud computing environment as described in any one of claims 1 to 9.

11. A non-volatile computer-readable storage medium storing program instructions thereon, characterized in that, When the program instructions are executed by the processor, they implement the cloud resource allocation method in the cloud computing environment as described in any one of claims 1 to 9.

Images