Virtual machine cooperative scheduling method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of virtual machine cooperative scheduling, and discloses a virtual machine cooperative scheduling method and device, electronic equipment and a storage medium, wherein the method comprises the following steps: collecting state information of a virtual machine and a host, and forming a preliminary scheduling scheme based on the state information; then, through regular inspection and dynamic state monitoring, when a detection result does not meet preset requirements, automatically adjusting a scheduling strategy, and optimizing an adjustment action set by using reinforcement learning. The application has the beneficial effects that: through active adaptation of an agent, resource overload and idle phenomena are significantly reduced, resource utilization efficiency and service quality are improved, smooth operation can be ensured in high-load and low-load stages, and the dynamic scheduling mechanism will help a cloud data center to maximize resource utilization rate and optimize user experience when dealing with complex and changeable business demands.

Description

Technical Field

[0001] This invention relates to the field of virtual machine collaborative scheduling technology, and in particular to a virtual machine collaborative scheduling method, apparatus, electronic device and storage medium. Background Technology

[0002] In modern cloud data center environments, virtual machine scheduling is a crucial step in achieving efficient resource management. With the rapid growth in demand for cloud computing services, the operational modes of various online and offline services are becoming increasingly complex, placing higher performance demands on virtual machine scheduling algorithms. Traditional resource scheduling methods primarily rely on instantaneous resource utilization to make virtual machine placement or migration decisions, often failing to fully consider the periodic patterns of service load. For example, many online applications face high traffic demands during peak daytime hours, while offline computing tasks are mostly executed at night, resulting in significant tidal resource demand patterns. If current scheduling algorithms cannot effectively identify and utilize these complementary load patterns, resource utilization will inevitably become unbalanced at different times, with resource overload during some periods and high resource vacancy rates during others. Therefore, there is an urgent need for a scheduling method that can accurately identify periodic load changes and adjust resource allocation in real time to achieve more efficient cloud resource utilization and service quality. Summary of the Invention

[0003] Based on this, it is necessary to propose a virtual machine cooperative scheduling method, device, electronic device and storage medium to address the existing virtual machine cooperative scheduling problem.

[0004] A virtual machine cooperative scheduling method, the method comprising:

[0005] Obtain first state information of each target virtual machine in a specified virtual machine cluster, and obtain second state information of each target host in a specified host cluster; wherein, the host cluster is the execution carrier of the specified virtual machine cluster;

[0006] Based on a preset initial scheduling method, an initial scheduling scheme is formed according to each of the first state information and the second state information;

[0007] The initial scheduling scheme is executed to allocate the target virtual machines to each of the target hosts, and the specified host cluster is inspected.

[0008] When the inspection results do not meet the preset inspection requirements, the first state vector of each target virtual machine and the second state vector of the target host are obtained.

[0009] The first state vector and the second state vector are input into the initial agent to generate the target action set;

[0010] The target action set is optimized by using a preset reinforcement learning strategy to obtain an optimized scheduling scheme, and each target virtual machine is rescheduled to each target host according to the optimized scheduling scheme.

[0011] Furthermore, the step of obtaining the first state vector of each target virtual machine and the second state vector of the target host when the inspection results do not meet the preset inspection requirements includes:

[0012] Obtain the CPU utilization, CPU resource demand vector, current host location, virtual machine type, and historical CPU utilization mean and standard deviation of each target virtual machine, and obtain the host type, host CPU utilization, and remaining available resource capacity of the target host.

[0013] A first state vector is formed based on the CPU utilization of the target virtual machine, the CPU resource demand vector, the current host location, the virtual machine type, and the mean and standard deviation of historical CPU utilization. A second state vector is formed based on the host type of the target host, the host CPU utilization, and the remaining available resource capacity.

[0014] Further, in the steps of obtaining the CPU utilization, CPU resource demand vector, current host location, virtual machine type, and historical mean and standard deviation of CPU utilization for each of the target virtual machines, and obtaining the host type, host CPU utilization, and remaining available resource capacity of the target host, the step of obtaining the virtual machine type of each of the target virtual machines includes:

[0015] Periodically collect the CPU utilization vector of the target virtual machine;

[0016] Calculate the first day's average utilization, the first night's average utilization, and the utilization variance of the target virtual machine based on the CPU utilization vector.

[0017] The virtual machine type is determined based on the first day's average utilization, the first night's average utilization, and the utilization variance of the target virtual machine.

[0018] Further, in the steps of obtaining the CPU utilization, CPU resource demand vector, current host location, virtual machine type, and historical mean and standard deviation of CPU utilization for each of the target virtual machines, and obtaining the host type, host CPU utilization, and remaining available resource capacity of the target host, the step of obtaining the host type of each of the target hosts includes:

[0019] Periodically collect the CPU utilization of the target host to obtain a set of host CPU utilization.

[0020] Calculate the second day average utilization and second night average utilization of the target host based on the host CPU utilization set;

[0021] The host type of the target host is determined based on the average utilization rate of the target host during the second day and the average utilization rate during the second night.

[0022] Further, the step of inputting the first state vector and the second state vector into the initial agent to generate the target action set includes:

[0023] The second state vector is input into the initial agent to construct constraints, thereby obtaining the target agent;

[0024] Generate an initial action set based on the target agent;

[0025] The initial action set is optimized according to preset action rules to obtain the target action set.

[0026] Further, the step of optimizing the initial action set according to preset action rules to obtain the target action set includes:

[0027] Obtain the average utilization rate of each of the target hosts;

[0028] Calculate the penalty index for each target host based on the average utilization rate;

[0029] The initial action set is optimized based on the penalty index to obtain the target action set.

[0030] Further, the step of optimizing the target action set using a preset reinforcement learning strategy to obtain an optimized scheduling scheme, and then rescheduling each of the target virtual machines to each of the target hosts according to the optimized scheduling scheme, includes:

[0031] Obtain the strategy optimization objective;

[0032] Based on the optimization objective of the strategy, the objective function is set using a preset SAC algorithm;

[0033] The objective function guides the optimization process of the initial agent to optimize the target action set and obtain an optimized scheduling scheme.

[0034] According to the optimized scheduling scheme, each of the target virtual machines is rescheduled to each of the target hosts.

[0035] A virtual machine cooperative scheduling device, the device comprising:

[0036] The first state information acquisition module is used to acquire the first state information of each target virtual machine in a specified virtual machine cluster, and to acquire the second state information of each target host in a specified host cluster; wherein, the host cluster is the execution carrier of the specified virtual machine cluster.

[0037] An initial scheduling scheme forming module is used to form an initial scheduling scheme based on a preset initial scheduling method and according to each of the first state information and the second state information;

[0038] The target virtual machine allocation module is used to execute the initial scheduling scheme, allocate the target virtual machines to each of the target hosts, and perform inspections on the specified host cluster;

[0039] The first state vector acquisition module is used to acquire the first state vector of each target virtual machine and the second state vector of the target host when the inspection result does not meet the preset inspection requirements.

[0040] The target action set generation module is used to input the first state vector and the second state vector into the initial agent to generate a target action set;

[0041] The optimized scheduling scheme acquisition module is used to optimize the target action set through a preset reinforcement learning strategy to obtain an optimized scheduling scheme, and to reschedule each of the target virtual machines to each of the target hosts according to the optimized scheduling scheme.

[0042] An electronic device includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the following steps:

[0043] Obtain first state information of each target virtual machine in a specified virtual machine cluster, and obtain second state information of each target host in a specified host cluster; wherein, the host cluster is the execution carrier of the specified virtual machine cluster;

[0044] Based on a preset initial scheduling method, an initial scheduling scheme is formed according to each of the first state information and the second state information;

[0045] The initial scheduling scheme is executed to allocate the target virtual machines to each of the target hosts, and the specified host cluster is inspected.

[0046] When the inspection results do not meet the preset inspection requirements, the first state vector of each target virtual machine and the second state vector of the target host are obtained.

[0047] The first state vector and the second state vector are input into the initial agent to generate the target action set;

[0048] The target action set is optimized by using a preset reinforcement learning strategy to obtain an optimized scheduling scheme, and each target virtual machine is rescheduled to each target host according to the optimized scheduling scheme.

[0049] A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the following steps:

[0050] Obtain first state information of each target virtual machine in a specified virtual machine cluster, and obtain second state information of each target host in a specified host cluster; wherein, the host cluster is the execution carrier of the specified virtual machine cluster;

[0051] Based on a preset initial scheduling method, an initial scheduling scheme is formed according to each of the first state information and the second state information;

[0052] The initial scheduling scheme is executed to allocate the target virtual machines to each of the target hosts, and the specified host cluster is inspected.

[0053] When the inspection results do not meet the preset inspection requirements, the first state vector of each target virtual machine and the second state vector of the target host are obtained.

[0054] The first state vector and the second state vector are input into the initial agent to generate the target action set;

[0055] The target action set is optimized by using a preset reinforcement learning strategy to obtain an optimized scheduling scheme, and each target virtual machine is rescheduled to each target host according to the optimized scheduling scheme.

[0056] The beneficial effects of this invention are as follows: By collecting the status information of virtual machines and hosts, a preliminary scheduling scheme is formed based on this information. Subsequently, through regular inspections and dynamic status monitoring, when the detection results do not meet the preset requirements, the scheduling strategy is automatically adjusted. By using reinforcement learning to optimize the set of adjustment actions, the periodic patterns of service load can be effectively discovered and utilized. Through the active adaptation of the agent to changes, resource overload and idle phenomena are significantly reduced, thereby improving resource utilization efficiency and service quality. This ensures stable operation during both high and low load phases. This dynamic scheduling mechanism will help cloud data centers maximize resource utilization and optimize user experience when dealing with complex and ever-changing business needs. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] in:

[0059] Figure 1 This is an application environment diagram of the virtual machine cooperative scheduling method in one embodiment;

[0060] Figure 2 Here is a flowchart of a virtual machine cooperative scheduling method in one embodiment;

[0061] Figure 3 This is a structural block diagram of a virtual machine cooperative scheduling device in one embodiment;

[0062] Figure 4 This is a structural block diagram of an electronic device in one embodiment;

[0063] Figure 5 This is a block diagram of the agent in one embodiment. Detailed Implementation

[0064] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0065] Figure 1 This is a diagram of a virtual machine collaborative scheduling application environment in one embodiment. (Refer to...) Figure 1 This virtual machine collaborative scheduling method is applied to a virtual machine collaborative scheduling system. The system includes a terminal 110 and a server 120. The terminal 110 and server 120 are connected via a network. The terminal 110 can be a desktop terminal or a mobile terminal; a mobile terminal can be at least one of a mobile phone, tablet, or laptop. The server 120 can be a standalone server or a server cluster consisting of multiple servers. The terminal 110 is used to obtain status information, and the server 120 is used to generate optimized scheduling schemes.

[0066] like Figure 2As shown, in one embodiment, a virtual machine cooperative scheduling method is provided. This method can be applied to both terminals and servers; this embodiment uses terminal application as an example. The virtual machine cooperative scheduling method specifically includes the following steps:

[0067] S1: Obtain the first state information of each target virtual machine in the specified virtual machine cluster, and obtain the second state information of each target host in the specified host cluster; wherein, the host cluster is the execution carrier of the specified virtual machine cluster;

[0068] S2: Based on a preset initial scheduling method, an initial scheduling scheme is formed according to each of the first state information and the second state information;

[0069] S3: Execute the initial scheduling scheme, allocate the target virtual machines to each of the target hosts, and perform inspections on the designated host cluster;

[0070] S4: When the inspection results do not meet the preset inspection requirements, the first state vector of each target virtual machine and the second state vector of the target host are obtained.

[0071] S5: Input the first state vector and the second state vector into the initial agent to generate the target action set;

[0072] S6: The target action set is optimized and solved by a preset reinforcement learning strategy to obtain an optimized scheduling scheme, and each target virtual machine is rescheduled to each target host according to the optimized scheduling scheme.

[0073] As described in step S1 above, the first state information of each target virtual machine in the specified virtual machine cluster and the second state information of each target host in the specified host cluster are obtained. The first state information typically includes performance indicators such as virtual machine CPU utilization, memory usage, I / O performance, and network bandwidth. This information directly reflects the virtual machine's running status and helps the scheduling system understand the performance of each virtual machine at that moment. Secondly, the second state information involves host performance indicators in the host cluster, such as host CPU load, memory usage, storage availability, and network connection status. This information can be collected in real time using preset monitoring tools (such as performance monitoring software).

[0074] Specifically, the system collects the CPU utilization vector of the physical host cluster at each scheduling time t to obtain the second state information, in the following form: ,in, Indicates host CPU utilization at time t.

[0075] Receive the set of newly arrived virtual machine requests in the system to obtain the first state information. Each request It has known resource requirements, priority parameters, and historical utilization curves. This provides real-time, accurate resource input for subsequent intelligent scheduling, ensuring the effectiveness of scheduling decisions.

[0076] As described in step S2 above, after obtaining the first and second state information, an initial scheduling scheme is formed based on a preset initial scheduling method. Based on the existing resource state, preliminary virtual machine allocation and scheduling decisions are made. The initial scheduling method can be based on a predetermined rule or allocation algorithm, such as round-robin scheduling, least-load-first scheduling, or other heuristic algorithms. When forming the scheduling scheme, the system considers the resource requirements of each virtual machine and the host's carrying capacity to achieve a certain degree of load balancing in the initial allocation. Furthermore, the scheduling scheme also needs to consider the service quality requirements of the business, such as different requirements for response time, availability, and throughput. The formed initial scheduling scheme provides a baseline for subsequent resource management and optimization, enabling the system to further dynamically adjust and optimize based on the initial scheduling.

[0077] As described in step S3 above, after the initial scheduling scheme is formed, the system executes the scheme to allocate target virtual machines to target hosts. The purpose is to distribute computing resources to suitable hosts, achieving initial load balancing and resource utilization optimization. Simultaneously, after scheduling, the system performs inspections on the designated host cluster. The inspection cycle is not limited, for example, every 5 minutes, to ensure the effectiveness of the initial scheduling scheme and the health status of the hosts. The inspection includes various checks, such as monitoring host performance metrics, network connectivity, storage status, and the operational health status of virtual machines. Through inspections, the system can promptly identify potential problems, such as resource bottlenecks, virtual machine anomalies, and excessive host load. This inspection mechanism not only helps to promptly discover and resolve problems that arise during the scheduling process but also provides necessary feedback information for subsequent dynamic adjustments, ensuring the system continues to operate in its optimal state.

[0078] As described in step S4 above, during the inspection phase, if the inspection results are found to fail to meet the preset performance requirements, measures will be taken to obtain the first state vector of each target virtual machine and the second state vector of each target host. The state vectors typically include more detailed performance measurement results, reflecting the current performance of the virtual machines and hosts in real time. For effective adjustments, the system needs to analyze the multi-dimensional information contained in the first and second state vectors. This information includes not only basic usage indicators such as CPU, memory, storage, and network, but also complex indicators such as communication latency between the virtual machines and hosts, and job queuing. These state vectors will provide comprehensive data support for subsequent dynamic decision-making, enabling subsequent optimized scheduling schemes to better meet the dynamic needs of actual operation. Furthermore, if the inspection results are found to meet the preset inspection requirements, it is determined that the current initial scheduling scheme will continue to be executed.

[0079] As described in step S5 above, after obtaining the first and second state vectors, the system inputs data into the initial agent in this stage to generate a set of target actions. The agent is built based on a deep learning or reinforcement learning model, and its main task is to automatically generate the action set by learning previous interaction processes. Specifically, the agent can be pre-trained using historical load data on a simulated cloud platform and then fine-tuned online. These action sets contain a series of possible decisions for the current state, which may include virtual machine migration, resource reallocation, and other operations. During the generation of the action set, the agent utilizes information from the state vectors for pattern recognition and decision-making. Through analysis of the state information, the agent can identify problems and bottlenecks in previous scheduling decisions and propose possible optimized scheduling schemes based on the learned strategies. The advantage of this method is that it can dynamically adapt to environmental changes and continuously optimize scheduling strategies, rather than relying on static rules, thereby improving resource utilization efficiency and system performance.

[0080] As described in step S6 above, the target action set is optimized and solved using a preset reinforcement learning strategy to obtain an optimized scheduling scheme. The potential effects of each generated action are evaluated, and a comparison of their merits is made based on historical data and real-time state information. The action set is then optimized using a reinforcement learning algorithm to achieve the best resource scheduling effect. The optimized scheduling scheme will reschedule virtual machines based on the key information extracted by the agent. During this process, the system will comprehensively consider multiple indicators such as load balancing, response time, and resource utilization to ensure that the final scheduling scheme can not only effectively cope with the current load situation but also adapt to possible future changes. The dynamism and real-time nature of this process enable cloud data centers to maintain good responsiveness when facing peak loads and maintain efficient resource utilization when under low loads, thereby improving the quality and stability of the entire cloud service. The specific implementation of this method in cloud data center hardware resource management includes changes in physical resource allocation caused by virtual machine migration. Through dynamic scheduling, the energy consumption and overheating risk of physical hosts are reduced.

[0081] In one embodiment, step S4, which involves obtaining the first state vector of each target virtual machine and the second state vector of the target host when the inspection results do not meet the preset inspection requirements, includes:

[0082] S401: Obtain the CPU utilization, CPU resource demand vector, current host location, virtual machine type, and historical mean and standard deviation of CPU utilization for each of the target virtual machines, and obtain the host type, host CPU utilization, and remaining available resource capacity of the target host.

[0083] S402: A first state vector is formed based on the CPU utilization of the target virtual machine, the CPU resource demand vector, the current host location, the virtual machine type, and the mean and standard deviation of historical CPU utilization. A second state vector is formed based on the host type of the target host, the host CPU utilization, and the remaining available resource capacity.

[0084] As described in step S401 above, to obtain multiple performance metrics of each target virtual machine and the status information of the target host, the system first needs to collect the following information for each target virtual machine: CPU utilization: This is an important indicator for measuring the current computing load of the virtual machine, representing the proportion of CPU resources actually used by the virtual machine to its maximum available CPU resources. High CPU utilization may indicate that the virtual machine is running intensive tasks, while low utilization may indicate that resources are idle. This information helps to understand the workload of the virtual machine. CPU resource demand vector: This is a detailed description of the CPU resource demand of the virtual machine at different time periods, including peak and average demand. This vector can help the scheduling algorithm predict future load changes and achieve more accurate scheduling. Current host location: The host information where the virtual machine is located is crucial for scheduling decisions. Understanding the current location of the virtual machine can help determine the necessity of migration. Virtual machine type: Different types of virtual machines may have different performance characteristics and resource requirements. For example, web servers and database servers differ in their CPU, memory, and storage requirements. Historical CPU utilization mean and standard deviation: Historical data provides important references. The mean reflects the virtual machine load size under normal conditions, while the standard deviation indicates the degree of load fluctuation. By analyzing these records, the system can better predict future resource needs. For the target host, the system needs to obtain the following information: Host type: The host type determines its performance standards and resource configuration, significantly impacting virtual machine allocation strategies. Host CPU utilization: Similar to virtual machine CPU utilization, host CPU utilization reflects the host's own load status, helping to ensure that it is not overloaded during peak load periods. Remaining available resource capacity: Understanding the current remaining resource capacity of the host is crucial for the rational allocation of virtual machines, ensuring that new resource allocation does not lead to insufficient host resources. Data collection provides necessary raw information for subsequent state vector generation, facilitating more refined scheduling.

[0085] As described in step S402 above, the acquired information is used to construct a first state vector and a second state vector. This process integrates multiple collected performance metrics to form a characteristic data representation, facilitating subsequent scheduling decisions. First, the system constructs a first state vector based on relevant information about the target virtual machine. This vector includes: CPU utilization: As an important performance indicator, it reflects the current load status of the virtual machine. CPU resource demand vector: Indicates the virtual machine's demand for CPU resources, revealing changes in demand over different time periods. Current host location: The specific physical or virtual location, which can be represented as a unique identifier for the host. Virtual machine type: Type information can be transformed into features in the vector using one-hot encoding or other methods. Historical CPU utilization mean and standard deviation: These two statistics summarize historical performance, indicating the stability and volatility of the virtual machine load. Next, when forming the second state vector, the system considers information related to the target host, including: Host type: Similarly, one-hot encoding or other methods can be used to transform the type features into a vector. Host CPU utilization: This directly reflects the host's load status. Remaining available resource capacity: This information indicates how many additional virtual machines the host can currently support, becoming an important parameter for dynamic scheduling.

[0086] Specifically, the first state vector is defined as follows:

[0087]

[0088] in: This represents the CPU utilization of the virtual machine (v) at time (t); and These represent the mean and standard deviation of the virtual machine's historical CPU utilization, respectively, used to characterize load stability; This is the CPU resource requirement vector for the virtual machine; Label the virtual machine type (daytime, nighttime, stable, or sensitive); The target host's current CPU utilization; This represents the remaining available resource capacity of the host. Assuming the number of virtual machines is n and the number of hosts is m, then... The first five items are the virtual machine's configuration information, so their length is (5×n). The last two items are the host's configuration information, with a length of (2×m). Finally... The length is (5×n+2×m).

[0089] Because the traditional one-dimensional state space stretches all information to one dimension, when using reinforcement learning networks to solve it, the input layer size grows linearly with the number of users and edge nodes, while the output layer grows exponentially, eventually leading to an explosion in network size. Therefore, this invention will use a two-dimensional state space, namely:

[0090]

[0091] in Replaced This represents the current host location of the virtual machine. This replacement method is because changes in the deployment location of all virtual machines are the root cause of changes in host load; using the most fundamental variable as the state is more beneficial for subsequent reinforcement learning training. The length is (6×n). This state representation simultaneously reflects the load characteristics of the virtual machine to be migrated, the host resource status, and the overall system resource distribution, providing complete input information for subsequent decisions.

[0092] If there are a large number of virtual machines, such as more than 10, then PCA (Principal Component Analysis) or an autoencoder can be used for dimensionality reduction.

[0093] In one embodiment, the step S401 of obtaining the CPU utilization, CPU resource demand vector, current host location, virtual machine type, and historical mean and standard deviation of CPU utilization for each of the target virtual machines, and obtaining the host type, host CPU utilization, and remaining available resource capacity of the target host, includes the step of obtaining the virtual machine type of each of the target virtual machines, which further includes:

[0094] S4011: Periodically collect the CPU utilization vector of the target virtual machine;

[0095] S4012: Calculate the first day average utilization, the first night average utilization, and the utilization variance of the target virtual machine based on the CPU utilization vector;

[0096] S4013: Determine the virtual machine type based on the first day's average utilization, the first night's average utilization, and the utilization variance of the target virtual machine.

[0097] As described in step S4011 above, the CPU utilization of the target virtual machine is monitored and data is collected periodically. This process includes real-time collection of the CPU usage of the target virtual machine and storing the data in vector form. The CPU utilization vector is usually composed of the utilization rate at a series of time points, reflecting the CPU resource occupation of the virtual machine in different time periods. This periodic monitoring can capture the changing patterns of the virtual machine load.

[0098] The reason for periodically collecting CPU utilization vectors is that workloads in cloud service environments often have obvious periodicity and volatility. For example, online services usually have higher traffic during the day on weekdays, while traffic may be relatively lower at night or on weekends. By continuously monitoring the CPU utilization of virtual machines, the system can obtain historical data over a period of time. This data will be used to analyze overall trends and identify potential load peaks or troughs. The collected utilization vectors can also be used for subsequent feature extraction, data analysis, and decision optimization. Through a deeper understanding of this information, scheduling algorithms can more effectively manage and schedule virtual machines.

[0099] As described in step S4012 above, the day-night average utilization and utilization variance of the target virtual machine are calculated. Statistical analysis is performed based on the collected CPU utilization vector to calculate the first day's average utilization, the first night's average utilization, and the variance of the utilization of the target virtual machine. First, the daily CPU utilization data needs to be divided into "daytime" and "nighttime" periods, typically based on predetermined time intervals, such as defining daytime as 08:00 to 20:00 and nighttime as 20:00 to 08:00 the next day. Next, by calculating the average CPU utilization over these two time periods, a clear view of the virtual machine's load at different times is provided to the system. The first day's average utilization reflects the virtual machine's operational capacity during high-load periods, while the first night's average utilization reveals its performance during low-load periods. Furthermore, calculating the utilization variance helps assess the stability of the virtual machine's load. A larger variance indicates greater fluctuations in CPU utilization, meaning higher uncertainty in the virtual machine's resource usage. Combining these statistical data provides a solid foundation for determining the virtual machine type in subsequent stages, ensuring that the scheduling system can make more accurate decisions.

[0100] As described in step S4013 above, the virtual machine type is determined based on the average utilization and variance. The type of the target virtual machine is determined using the calculated average utilization of the first day, the average utilization of the first night, and the utilization variance. The definition of the virtual machine type can be based on its performance characteristics and workload features.

[0101] In one specific embodiment, virtual machines are divided into four categories: daytime, nighttime, stable, and sensitive, and the load patterns of the hosts are labeled as similar.

[0102] Calculate the average utilization and variance of virtual machines within a day / night cycle; for each virtual machine Define its time interval (Daytime) and The CPU utilization sequence during (nighttime) is as follows:

[0103]

[0104] ;

[0105] Calculate the average utilization rate and variance within the diurnal cycle:

[0106] ;

[0107] ;

[0108] ;

[0109] in The overall average utilization rate of virtual machines is defined as:

[0110] ;

[0111] Based on statistical characteristics, virtual machine types are classified, and the criteria for determining the four load types are as follows:

[0112] Day-dominant VMs: VMs whose daytime utilization is significantly higher than nighttime utilization. ;

[0113] in The empirical threshold (usually 0.5 to 1.0) represents the degree of significance of diurnal fluctuations.

[0114] These types of virtual machines have significantly higher resource requirements during the daytime, making them suitable for co-location with nighttime virtual machines.

[0115] Night-dominant VMs: VMs whose utilization is significantly higher at night than during the day.

[0116] ;

[0117] This indicates that the virtual machine's computing needs increase at night, making it suitable for deployment in conjunction with a daytime virtual machine.

[0118] Stable virtual machines: those with small day-night utilization differences and low variance.

[0119]

[0120] in The threshold for day-night balance. This represents the load fluctuation threshold. Stable virtual machines have stable resource consumption and are suitable for deployment on core compute nodes to maintain a stable system load.

[0121] Sensitive VMs: VMs whose utilization fluctuates drastically and exhibits irregular day-night variations.

[0122] These types of virtual machines exhibit high-frequency fluctuations or uncertain load patterns, making them suitable for placement on nodes with elastic resources to ensure QoS.

[0123] In one embodiment, the step S401 of obtaining the CPU utilization, CPU resource demand vector, current host location, virtual machine type, and historical mean and standard deviation of CPU utilization for each of the target virtual machines, and obtaining the host type, host CPU utilization, and remaining available resource capacity of the target host, includes the step of obtaining the host type of each of the target hosts, which further includes:

[0124] S4111: Periodically collect the host CPU utilization of the target host to obtain a set of host CPU utilization;

[0125] S4112: Calculate the second day average utilization and the second night average utilization of the target host based on the host CPU utilization set;

[0126] S4113: Determine the host type of the target host based on the second day average utilization rate and the second night average utilization rate of the target host.

[0127] As described in step S4111 above, the CPU utilization of the target host is collected periodically. The main purpose of periodically collecting the CPU utilization data of the target host and forming a utilization dataset is to monitor the operating status and load of the target host in order to better understand its resource utilization. Periodic collection means that the system records the CPU utilization of the target host at predetermined time intervals (e.g., every minute or every hour).

[0128] As described in step S4112 above, the average utilization of the target host during the day and night is calculated. Using the collected host CPU utilization data, the average utilization of the target host during the second day and the average utilization during the second night are calculated. The purpose of this process is to deeply analyze the performance of the target host at different time periods, thereby gaining a more comprehensive understanding of the host's resource usage. First, the system divides time into two periods: "daytime" and "nighttime." Typically, the daytime period can be set to 08:00 to 20:00 on weekdays, and the nighttime period to 20:00 to 08:00 the next day. Next, the system groups the collected CPU utilization data and calculates the average utilization within the two periods. The average utilization during the second day reflects the host's performance during high-load periods, while the average utilization during the second night reveals the virtual machine's operation during low-load periods.

[0129] As described in step S4113 above, the host type of the target host is determined. The host type is determined based on the calculated average utilization rate of the second day and the average utilization rate of the second night. A set of standards or thresholds can be predefined to categorize host types into multiple categories. For example, host types may include high-performance computing (HPC), standard operating hosts, low-power hosts, etc. Hosts with high average daytime utilization and low average nighttime utilization may be defined as "high-load" types, suitable for handling high-concurrency requests or intensive computing tasks; while host types with low average daytime and nighttime utilization may be considered "low-load" types, more suitable for long-running batch processing tasks or other services that do not require real-time response.

[0130] Specifically, for the host Calculate its average utilization rate and time distribution characteristics:

[0131]

[0132] ;

[0133] like , If it is a pre-set constant, it is marked as "Daytime Reload Host"; if If it is, then mark it as "Nighttime Heavy Load Host"; otherwise, mark it as "Stable Host".

[0134] The system also records the host's idle periods:

[0135] ;

[0136] in Define a threshold for the idle state of a node (e.g., 20%).

[0137] In one embodiment, step S5, which inputs the first state vector and the second state vector into the initial agent to generate the target action set, includes:

[0138] S501: Input the second state vector into the initial agent to construct constraints and obtain the target agent;

[0139] S502: Generate an initial action set based on the target agent;

[0140] S503: Optimize the initial action set according to the preset action rules to obtain the target action set.

[0141] As described in step S501 above, the system inputs the previously generated second state vector into the initial agent to construct the constraints of the target agent. The second state vector contains key information about the target host, such as the host's CPU utilization, remaining available resource capacity, host type, etc. The purpose of constructing the constraints is to provide clear restrictions and guidance for the agent's decision-making process. By inputting the second state vector into the agent, the agent can fully understand the current resource status of the host so that it can prioritize resource utilization efficiency and service quality when making decisions.

[0142] As described in step S502 above, based on the constructed target agent, the system begins to generate an initial action set. The initial action set includes a series of scheduling actions that the agent can perform in its current state, such as migrating, starting, or shutting down virtual machines. During this process, the agent will utilize its built-in policies to analyze feasible actions from given constraints. For example, the agent might assess the current host load and choose to migrate some high-load virtual machines to hosts with lower loads to achieve a more balanced resource allocation. At this time, the agent will also utilize previously collected historical data to help predict which actions are most likely to yield higher rewards. In this process, the agent uses state information and built-in decision-making strategies, such as Q-learning and policy gradient reinforcement learning methods, to generate the action set. It is worth noting that the initial action set is not required to be optimal, but rather provides a foundation for subsequent optimization.

[0143] As described in step S503 above, the initial action set is optimized according to preset action rules. The optimization process is like filtering and improving the initial actions to ensure that the final decision prioritizes meeting constraints and business needs. The preset action rules may include various criteria, such as priority rules to prioritize high-load virtual machines in scheduling, or limiting the execution frequency of certain actions to avoid excessive system migration. Rules may also include optimal resource utilization and ensuring service quality stability. The formulation of these rules needs to consider specific application scenarios, business requirements, and previously collected state information. In this optimization phase, the AI ​​evaluates the possible effects and potential risks of each action based on feedback information from the reinforcement learning model. By continuously adjusting the action set, it achieves the dual goals of maximizing resource utilization and ensuring service quality. By applying optimization algorithms (such as genetic algorithms, simulated annealing, and policy optimization in reinforcement learning), the system can ensure that the final target action set is reasonable and effective, thus raising the effective allocation of resources and system stability to a new level.

[0144] In one embodiment, step S503, which optimizes the initial action set according to preset action rules to obtain the target action set, includes:

[0145] S5031: Obtain the average utilization rate of each of the target hosts;

[0146] S5032: Calculate the penalty index for each of the target hosts based on the average utilization rate;

[0147] S5033: Optimize the initial action set based on the penalty index to obtain the target action set.

[0148] As described in step S5031 above, the average utilization rate of each target host is obtained. Average utilization rate refers to the proportion of a target host's resources (such as CPU, memory, and storage) that are continuously used within a certain period. Obtaining the average utilization rate of each target host typically involves real-time monitoring of its resource usage over a period of time. This process can be achieved through performance monitoring tools. The system will periodically record the resource usage data of each host at different time periods. The utilization rate data for these time periods can be further statistically averaged to reflect the overall usage of the host. For example, calculating the average CPU utilization rate of a host over eight hours in a workday can accurately show the host's resource carrying capacity during peak hours and other potential bottlenecks. By comparing the utilization rates of multiple hosts, the system can identify which hosts may be at risk of overload, thus providing a basis for priority scheduling.

[0149] As described in step S5032 above, a penalty index is calculated for each target host based on the average utilization rate. The penalty index is an important indicator for measuring host load levels, used to assess the rationality of current resource usage and provide a basis for subsequent priority scheduling strategies. The calculation of the penalty index typically involves analyzing and judging the host's average utilization rate. Generally, if the average utilization rate of a host exceeds a certain preset threshold (e.g., 80%), it is considered to be in a high-load state. In this case, the system may consider allocating fewer new virtual machines to that host, or even migrate some heavily loaded virtual machines to other hosts to ensure service quality stability. Conversely, if the host's average utilization rate is low (e.g., below 30%), a negative penalty value can be set to incentivize the scheduling strategy to migrate new or lightly loaded virtual machines to this host. This calculation and setting of the penalty index allows the scheduling algorithm to better consider the current load state of the host when making decisions. The penalty index is not just an absolute value; it influences the direction of host scheduling decisions, making scheduling strategies more intelligent, helping to avoid the risk of overloading a particular host, and achieving optimized load allocation.

[0150] As described in step S5033 above, the initial action set is optimized based on the penalty index. Combining the penalty mechanism with action optimization ensures that the final scheduling decision is both efficient and reasonable. Based on the penalty index, the scheduling algorithm evaluates each action in the initial action set, considering their impact on the target host. For example, actions may include options to migrate some virtual machines to high-load hosts or move them to low-load hosts. During evaluation, the system prioritizes scheduling schemes that reduce the pressure on high-load hosts, minimizing host overload risk and improving overall resource utilization efficiency. Furthermore, the penalty index can be used to adjust action priorities. For example, hosts under high load should be given a higher "negative penalty" to incentivize the system to more actively migrate virtual machines to lightly loaded hosts. Conversely, for low-load hosts, the scheduling strategy can be encouraged to migrate more virtual machines to these hosts, thereby improving resource utilization.

[0151] Specifically, to guide the agent to maintain a balance in resource utilization globally, this invention designs a piecewise penalty function based on the host load state. At each time step t, the system's global reward is defined as:

[0152]

[0153] Among them, the penalty function The definition is as follows:

[0154] ;

[0155] This function imposes an exponential penalty on high-utilization hosts, thereby prompting agents to actively avoid hotspot concentration; at the same time, it does not penalize low-utilization hosts, in order to encourage migration to idle nodes.

[0156] In one embodiment, step S6, which involves optimizing the target action set using a preset reinforcement learning strategy to obtain an optimized scheduling scheme, and then rescheduling each target virtual machine to each target host according to the optimized scheduling scheme, includes:

[0157] S601: Obtain the strategy optimization target;

[0158] S602: Based on the optimization objective of the strategy, set the objective function using the preset SAC algorithm;

[0159] S603: The initial agent is guided by the objective function to optimize the target action set and obtain an optimized scheduling scheme;

[0160] S604: Reschedule each of the target virtual machines to each of the target hosts according to the optimized scheduling scheme.

[0161] As described in step S601 above, the strategy optimization objective is obtained. The strategy optimization objective defines the specific direction and outcome that the agent pursues when performing resource scheduling and decision-making. In the virtual machine scheduling scenario of a cloud data center, these objectives may involve multiple aspects, such as improving resource utilization, reducing energy consumption, optimizing response time, and ensuring service quality. Specifically, the strategy optimization objective can be a set of one or more weights, representing the priority among different objectives. For example, if resource utilization is considered the most important objective, it needs to be given a higher weight in the objective setting. Similarly, if service quality is equally important, consideration of parameters such as service latency needs to be reflected in the objective function. Through such objective setting, the system clarifies the key performance indicators that the agent should focus on during the scheduling optimization process. Furthermore, obtaining the strategy optimization objective also requires adjustments based on business needs and actual conditions to ensure that the decision-making process remains consistent with the actual needs of business operations.

[0162] As described in step S602 above, based on the determined policy optimization objective, a target function is set using a pre-defined SAC (Soft Actor-Critic) algorithm. The target function is a mathematical expression used to evaluate the effectiveness of policy execution, typically including different performance metrics related to resource scheduling. In the SAC algorithm, the target function usually consists of the expected reward value, representing the reward obtained by the agent after performing a certain action. Specifically, the target function may include multiple components such as virtual machine migration success rate, resource utilization, energy consumption, and user satisfaction. For example, the reward function can be designed to assign a positive reward for successful migration and a negative reward for failed migration or overload; additional positive rewards can also be added for arrangements with high resource utilization. By reasonably designing the target function, the agent can learn which operations will yield higher rewards during the optimization process, thereby selecting these better actions in actual scheduling. The setting of the target function is crucial to the quality of the optimization results; different designs will lead to different decision-making tendencies in the agent. Therefore, this step is not only a mathematical expression of the objective but also a key step in ensuring that the reinforcement learning process achieves effective results. The initial agent is implemented using the SAC algorithm; therefore, it can also be called an SAC agent or a virtual machine scheduling model based on SAC. The network structure of this initial agent includes:

[0163] Policy Network (Actor Network):

[0164] The network consists of 3 to 4 fully connected layers, each containing 256 to 512 neurons.

[0165] Use the ReLU activation function, and the output layer uses either tanh or softmax (depending on the action space type).

[0166] Q-value network (Critic Network):

[0167] It typically consists of two independent Q-value networks with a structure similar to that of policy networks (3 to 4 layers, 256 to 512 neurons per layer, with ReLU activation function).

[0168] The output is the Q-value of the corresponding state-action pair.

[0169] Value Network:

[0170] It also consists of 3 to 4 fully connected layers, each with 256 to 512 neurons, using the ReLU activation function.

[0171] LSTM layer:

[0172] An LSTM layer can be added before the backbone network to process temporal features.

[0173] The number of hidden units in an LSTM layer is typically between 128 and 256.

[0174] The training parameters are as follows:

[0175] Learning rate: Use approximately 3e-4 or 1e-3, typically in conjunction with the Adam optimizer.

[0176] Batch size: Set to 256 or 512.

[0177] Discount factor (gamma): Usually set to 0.99.

[0178] Soft update parameter (tau): usually set to 0.005.

[0179] LSTM layers are used to process time-series data, taking the states of multiple time steps as sequential inputs to enhance the understanding of dynamic environmental changes. The outputs of the LSTM (hidden states) are then connected to subsequent fully connected layers to improve the agent's decision-making capabilities.

[0180] The solutions to the network scaling problem include: reducing network size through pruning by removing redundant neurons and connections; optimizing input feature selection by retaining only important features to reduce input dimensionality; and implementing parameter sharing by sharing some of the same parameters across multiple networks to reduce the total number of parameters.

[0181] The initial agent can be trained using reinforcement learning, which aims to learn the optimal policy through interaction with the environment. The core of reinforcement learning is the four-tuple (S, A, R, P), namely state, action, reward, and transition probability. Its four-tuple components are designed as follows:

[0182] State S: The first state vector with a fixed-length time series is taken as input.

[0183] S={s t0 ,s t1 ,s t2 ,s t3 ,s t4 ,s t5};

[0184] Among them, s t[…] It represents state information over continuous time.

[0185] Action A: The action set is the policy output in reinforcement learning. By performing actions, one can interact with the environment.

[0186] A = {a1, a2, a3, a4, a5, a6}, specifically the probabilities of 6 fault identification results (drift, bias, noise, impact, constant value, performance degradation).

[0187] Reward R: The reward is used to evaluate the quality of the strategy and measure the merits of the actions, thereby guiding the agent to learn the optimal strategy. Specifically, the reward is 1 for a correct recognition result and -1 for an incorrect recognition result.

[0188] Transition probability P: In the sequence data samples targeted by this invention, the states are obtained sequentially from the experience pool, and the transition probability is fixed, taking a value of 1. Specifically, it refers to data that are sequentially connected at equal intervals.

[0189] After backpropagation training of the optimized SAC deep reinforcement learning model, in order to evaluate the model's recognition ability, the model's precision, recall, and F-score were compared and weighed to evaluate its recognition ability.

[0190] As described in step S603 above, the objective function is input into the initial agent to guide the optimization solution. The agent will conduct multiple trials and iterative learning based on the objective function. By combining the policy evaluation and policy optimization steps in the SAC algorithm, the agent continuously adjusts its policy and gradually approaches the optimal solution. Specifically, the agent will perform simulations to predict scheduling performance under different conditions and update its policy based on the reward value calculated from the objective function. For example, the agent may evaluate the reward of each action in multiple states and use this data to calculate how good the current policy is. Then, a reinforcement learning policy is applied to optimize it. Through this solution process, the agent can adjust its action selection so that the final target action set meets the preset policy optimization objective in terms of performance, thereby achieving efficient resource scheduling. Finally, the system will generate an optimized scheduling scheme that is significantly better than the previous initial action set in terms of performance indicators, realizing optimized resource allocation to support more efficient virtual machine scheduling.

[0191] As described in step S604 above, the target virtual machine is rescheduled to the target host according to the optimized scheduling scheme. Based on the optimized scheduling scheme, the system identifies the optimal matching relationship between virtual machines and hosts. For example, the system will schedule virtual machines with heavy computing loads to hosts with higher resource availability, while shutting down or migrating corresponding virtual machines during low load periods, ensuring effective utilization of host resources. The core of this step is to guide the dynamic redeployment of virtual machines using optimized information, ensuring that resource configuration not only meets current operational needs but also prepares for future load changes. During the rescheduling process, the system may need to consider requirements such as service quality assurance and business continuity to reduce uncertainties or service interruptions during virtual machine migration or reconfiguration. This rescheduling implementation accelerates system response capabilities, improves overall resource utilization efficiency, and enables cloud data centers to more flexibly respond to ever-changing load demands.

[0192] In a traditional SAC architecture, the Q network and target Q network are typically identical to the policy network. When the number of hosts is m and the number of virtual machines is n, the output dimension of the policy network grows exponentially. This leads to an explosion in the number of network parameters. As the cluster size increases, this problem severely impacts learning efficiency and can even cause memory overflow, making training impossible. To address this, this invention proposes an improved SAC network structure, whose agent structure is as follows: Figure 5 As shown, the intelligent agent includes the following modules:

[0193] Policy Network Module:

[0194] The policy network module generates service migration decisions based on the observed state of the current system. Its core structure consists of a state mining network and fully connected (FC) layers. This module is designed to enable agents to extract time-related features from complex and dynamic cluster environments, achieving long-term optimization of migration strategies.

[0195] To address the dynamic and temporally correlated nature of resource states in virtual machine scheduling environments, this invention designs a state mining network within the policy network. Traditional SAC structures rely solely on single-moment state vectors for decision-making, failing to fully capture the temporal evolution of cluster load, leading to policy updates being sensitive to instantaneous fluctuations and unstable learning. The proposed state mining network, by combining a fully connected layer with a Long Short-Term Memory (LSTM) network, effectively extracts implicit temporal features from multi-moment observations, thereby enhancing the policy network's ability to perceive and predict system state change trends.

[0196] Specifically, as shown in the figure, the state mining network first uses a fully connected layer (FC) to perform feature mapping on the current observation (i.e., the state vector) input, compressing and extracting key resource utilization patterns and virtual machine load characteristics. Subsequently, an LSTM layer receives this feature sequence, utilizes its internal memory mechanism to maintain a certain length of historical state information, learns the temporal evolution of system load from it, and outputs a hidden state vector reflecting global dynamic characteristics. This hidden state, as a high-dimensional semantic representation of the network, reflects the resource dependencies and evolutionary relationships between the host and virtual machines. Based on this, the hidden state vector is input to the fully connected layer, and the action probability distribution is calculated using the Softmax function, outputting the migration probability of each virtual machine on different target hosts. Finally, the system obtains migration decisions by sampling row by row, realizing the mapping from state features to migration actions.

[0197] Q network module:

[0198] The Q-network module is used to evaluate the value relationship between the current system state and transition actions, thereby providing optimization directions for updating the policy network. Its core function is to characterize the expected value of the system's long-term reward after performing a specific transition action by learning the Q-value function of the state-action pair.

[0199] In this invention, the Q-network takes the system's two-dimensional state matrix and the action vector output by the policy network as input, and outputs the corresponding Q-value. By minimizing the mean square error between the predicted Q-value and the target Q-value, the Q-network can continuously adjust its parameters, thereby accurately estimating the value of different transfer actions. To improve training stability and expressive power, this invention adopts the same structure in the Q-network design as the policy network.

[0200] In the SAC algorithm framework, the parameter updates of the policy network depend on the gradient feedback provided by the Q network. Therefore, the estimation accuracy of the Q network directly affects the policy convergence speed and performance. The Q network used in this invention can accurately reflect the nonlinear relationship between virtual machine scheduling cost and system benefits, thereby improving the global optimality of scheduling decisions and training convergence efficiency.

[0201] Target Q network module:

[0202] The target Q-network module is used to stabilize the training process of the SAC algorithm, preventing policy divergence caused by oscillations in Q-value estimation or overfitting. Its main function is to provide a smooth and stable target reference value for the Q-network, thereby improving the training robustness and convergence performance of the entire system.

[0203] During training, the parameters of the target Q-network are not learned directly from the data. Instead, they are updated by performing an exponential moving average (EMA) on the parameters of the main Q-network, resulting in smoother changes. When calculating the target value, the SAC algorithm uses the Q-value output by the target Q-network as a reference, effectively suppressing learning noise caused by fluctuations in training data or unstable gradients.

[0204] Specifically, in each update round, the parameters of the target Q network are updated as follows:

[0205] ;

[0206] in, Main Q network parameters, For the target Q network parameters, This is a smoothing coefficient (typically between 0.005 and 0.01). This mechanism ensures that the target network updates slowly, thus acting as a "stabilizer" to make policy optimization more stable and reliable.

[0207] By introducing an improved SAC reinforcement learning structure, adaptive optimization and long-term optimal control of the transfer policy are achieved. The temporal feature extraction mechanism based on the state mining network enables the agent to perceive the dynamic trend of cluster load changes, significantly improving the stability and generalization ability of the policy. At the same time, the improved network structure effectively suppresses the parameter explosion problem caused by the exponential growth of action dimensions, improving the training efficiency and scalability in large-scale cluster environments.

[0208] Reference Figure 3 The present invention also provides a virtual machine cooperative scheduling device, the device comprising:

[0209] The first state information acquisition module 902 is used to acquire the first state information of each target virtual machine in a specified virtual machine cluster, and to acquire the second state information of each target host in a specified host cluster; wherein, the host cluster is the execution carrier of the specified virtual machine cluster.

[0210] The initial scheduling scheme forming module 904 is used to form an initial scheduling scheme based on a preset initial scheduling method and according to each of the first state information and the second state information;

[0211] The target virtual machine allocation module 906 is used to execute the initial scheduling scheme, allocate the target virtual machines to each of the target hosts, and perform inspections on the designated host cluster;

[0212] The first state vector acquisition module 908 is used to acquire the first state vector of each target virtual machine and the second state vector of the target host when the inspection result does not meet the preset inspection requirements.

[0213] The target action set generation module 910 is used to input the first state vector and the second state vector into the initial agent to generate a target action set;

[0214] The optimized scheduling scheme acquisition module 912 is used to optimize the target action set by a preset reinforcement learning strategy to obtain an optimized scheduling scheme, and to reschedule each of the target virtual machines to each of the target hosts according to the optimized scheduling scheme.

[0215] In one embodiment, the first state vector acquisition module 908 includes:

[0216] The host type acquisition submodule is used to acquire the CPU utilization, CPU resource demand vector, current host location, virtual machine type, and historical mean and standard deviation of CPU utilization for each target virtual machine, and to acquire the host type, host CPU utilization, and remaining available resource capacity of the target host.

[0217] The second state vector acquisition submodule is used to form a first state vector based on the CPU utilization, CPU resource demand vector, current host location, virtual machine type, and the mean and standard deviation of historical CPU utilization of the target virtual machine, and to form a second state vector based on the host type, host CPU utilization, and remaining available resource capacity of the target host.

[0218] In one embodiment, the host type acquisition submodule includes:

[0219] The CPU utilization vector collection unit is used to periodically collect the CPU utilization vector of the target virtual machine.

[0220] The utilization variance calculation unit is used to calculate the first day average utilization, the first night average utilization, and the utilization variance of the target virtual machine based on the CPU utilization vector.

[0221] The virtual machine type determination unit is used to determine the virtual machine type based on the first day average utilization, the first night average utilization, and the utilization variance of the target virtual machine.

[0222] In one embodiment, the host type acquisition submodule includes:

[0223] The host CPU utilization set acquisition unit is used to periodically collect the host CPU utilization of the target host to obtain the host CPU utilization set;

[0224] The average utilization calculation unit is used to calculate the second day average utilization and the second night average utilization of the target host based on the host CPU utilization set.

[0225] The host type determination unit is used to determine the host type of the target host based on the second day average utilization rate and the second night average utilization rate of the target host.

[0226] In one embodiment, the target action set generation module 910 includes:

[0227] The constraint construction submodule is used to input the second state vector into the initial agent to construct constraints and obtain the target agent;

[0228] An initial action set generation submodule is used to generate an initial action set based on the target intelligent agent;

[0229] The target action set acquisition submodule is used to optimize the initial action set according to preset action rules, thereby obtaining the target action set.

[0230] In one embodiment, the target action set acquisition submodule includes:

[0231] An average utilization acquisition unit is used to acquire the average utilization of each of the target hosts;

[0232] A penalty index calculation unit is used to calculate a penalty index for each of the target hosts based on the average utilization rate.

[0233] The target action set acquisition unit is used to optimize the initial action set based on the penalty index to obtain the target action set.

[0234] In one embodiment, the optimized scheduling scheme acquisition module 912 includes:

[0235] The strategy optimization target acquisition submodule is used to acquire strategy optimization targets;

[0236] The objective function setting unit is used to set the objective function according to the optimization objective of the strategy by using a preset SAC algorithm;

[0237] The solution submodule is used to guide the optimization process of the initial agent through the objective function, so as to optimize the target action set and obtain an optimized scheduling scheme;

[0238] The scheduling submodule is used to reschedule each of the target virtual machines to each of the target hosts according to the optimized scheduling scheme.

[0239] Figure 4 An internal structural diagram of an electronic device in one embodiment is shown. This electronic device can specifically be a terminal or a server, and more specifically, a computer device. Figure 4 As shown, the electronic device includes a processor, memory, and network interface connected via a system bus. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and may also store a computer program. When executed by the processor, this computer program enables the processor to implement a virtual machine cooperative scheduling method. The internal memory may also store a computer program, which, when executed by the processor, enables the processor to implement a virtual machine cooperative scheduling method. Those skilled in the art will understand that… Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0240] In one embodiment, an electronic device is provided, including a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the following steps:

[0241] Obtain first state information of each target virtual machine in a specified virtual machine cluster, and obtain second state information of each target host in a specified host cluster; wherein, the host cluster is the execution carrier of the specified virtual machine cluster;

[0242] Based on a preset initial scheduling method, an initial scheduling scheme is formed according to each of the first state information and the second state information;

[0243] The initial scheduling scheme is executed to allocate the target virtual machines to each of the target hosts, and the specified host cluster is inspected.

[0244] When the inspection results do not meet the preset inspection requirements, the first state vector of each target virtual machine and the second state vector of the target host are obtained.

[0245] The first state vector and the second state vector are input into the initial agent to generate the target action set;

[0246] The target action set is optimized by using a preset reinforcement learning strategy to obtain an optimized scheduling scheme, and each target virtual machine is rescheduled to each target host according to the optimized scheduling scheme.

[0247] By collecting status information from virtual machines and hosts, a preliminary scheduling scheme is formed. Subsequently, through regular inspections and dynamic status monitoring, the scheduling strategy is automatically adjusted when the detection results do not meet the preset requirements. Reinforcement learning is used to optimize the set of adjustment actions, which can effectively discover and utilize the periodic patterns of service load. Through the agent's proactive adaptation to changes, resource overload and idle phenomena are significantly reduced, thereby improving resource utilization efficiency and service quality. This ensures stable operation during both high and low load phases. This dynamic scheduling mechanism will help cloud data centers maximize resource utilization and optimize user experience when dealing with complex and ever-changing business needs.

[0248] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, causes the processor to perform the following steps:

[0249] Obtain first state information of each target virtual machine in a specified virtual machine cluster, and obtain second state information of each target host in a specified host cluster; wherein, the host cluster is the execution carrier of the specified virtual machine cluster;

[0250] Based on a preset initial scheduling method, an initial scheduling scheme is formed according to each of the first state information and the second state information;

[0251] The initial scheduling scheme is executed to allocate the target virtual machines to each of the target hosts, and the specified host cluster is inspected.

[0252] When the inspection results do not meet the preset inspection requirements, the first state vector of each target virtual machine and the second state vector of the target host are obtained.

[0253] The first state vector and the second state vector are input into the initial agent to generate the target action set;

[0254] The target action set is optimized by using a preset reinforcement learning strategy to obtain an optimized scheduling scheme, and each target virtual machine is rescheduled to each target host according to the optimized scheduling scheme.

[0255] By collecting status information from virtual machines and hosts, a preliminary scheduling scheme is formed. Subsequently, through regular inspections and dynamic status monitoring, the scheduling strategy is automatically adjusted when the detection results do not meet the preset requirements. Reinforcement learning is used to optimize the set of adjustment actions, which can effectively discover and utilize the periodic patterns of service load. Through the agent's proactive adaptation to changes, resource overload and idle phenomena are significantly reduced, thereby improving resource utilization efficiency and service quality. This ensures stable operation during both high and low load phases. This dynamic scheduling mechanism will help cloud data centers maximize resource utilization and optimize user experience when dealing with complex and ever-changing business needs.

[0256] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0257] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0258] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A virtual machine cooperative scheduling method, characterized in that, The method includes: Obtain first state information of each target virtual machine in a specified virtual machine cluster, and obtain second state information of each target host in a specified host cluster; wherein, the host cluster is the execution carrier of the specified virtual machine cluster; Based on a preset initial scheduling method, an initial scheduling scheme is formed according to each of the first state information and the second state information; The initial scheduling scheme is executed to allocate the target virtual machines to each of the target hosts, and the specified host cluster is inspected. When the inspection results do not meet the preset inspection requirements, the first state vector of each target virtual machine and the second state vector of the target host are obtained. Specifically, this includes: obtaining the CPU utilization, CPU resource demand vector, current host location, virtual machine type, and the mean and standard deviation of historical CPU utilization of each target virtual machine; and obtaining the host type, host CPU utilization, and remaining available resource capacity of the target host. The first state vector is formed based on the CPU utilization, CPU resource demand vector, current host location, virtual machine type, and the mean and standard deviation of historical CPU utilization of the target virtual machine; and the second state vector is formed based on the host type, host CPU utilization, and remaining available resource capacity of the target host. The process of inputting the first state vector and the second state vector into an initial agent to generate a target action set includes: inputting the first state vector and the second state vector into the initial agent to construct constraints, thereby obtaining a target agent; generating an initial action set based on the target agent; and optimizing the initial action set according to preset action rules to obtain the target action set. Specifically, this includes: obtaining the average utilization rate of each target host; calculating a penalty index for each target host based on the average utilization rate; and optimizing the initial action set based on the penalty index to obtain the target action set. The target action set is optimized by using a preset reinforcement learning strategy to obtain an optimized scheduling scheme, and each target virtual machine is rescheduled to each target host according to the optimized scheduling scheme.

2. The virtual machine cooperative scheduling method according to claim 1, characterized in that, The step of obtaining the virtual machine type of each target virtual machine in the steps of obtaining the CPU utilization, CPU resource demand vector, current host location, virtual machine type, and historical mean and standard deviation of CPU utilization of each target virtual machine, and obtaining the host type, host CPU utilization, and remaining available resource capacity of the target host, includes: Periodically collect the CPU utilization vector of the target virtual machine; Calculate the first day's average utilization, the first night's average utilization, and the utilization variance of the target virtual machine based on the CPU utilization vector. The virtual machine type is determined based on the first day's average utilization, the first night's average utilization, and the utilization variance of the target virtual machine.

3. The virtual machine cooperative scheduling method according to claim 1, characterized in that, The step of obtaining the CPU utilization, CPU resource demand vector, current host location, virtual machine type, and historical mean and standard deviation of CPU utilization for each of the target virtual machines, and obtaining the host type, host CPU utilization, and remaining available resource capacity of the target host, includes the step of obtaining the host type of each target host, which further includes: Periodically collect the CPU utilization of the target host to obtain a set of host CPU utilization data; Calculate the second day average utilization and the second night average utilization of the target host based on the host CPU utilization set; The host type of the target host is determined based on the average utilization rate of the target host during the second day and the average utilization rate during the second night.

4. The virtual machine cooperative scheduling method according to claim 1, characterized in that, The step of optimizing the target action set using a preset reinforcement learning strategy to obtain an optimized scheduling scheme, and then rescheduling each of the target virtual machines to each of the target hosts according to the optimized scheduling scheme, includes: Obtain the strategy optimization objective; Based on the optimization objective of the strategy, the objective function is set using a preset SAC algorithm; The objective function guides the optimization process of the initial agent to optimize the target action set and obtain an optimized scheduling scheme. According to the optimized scheduling scheme, each of the target virtual machines is rescheduled to each of the target hosts.

5. A virtual machine collaborative scheduling device, characterized in that, The device includes: The first state information acquisition module is used to acquire the first state information of each target virtual machine in a specified virtual machine cluster, and to acquire the second state information of each target host in a specified host cluster; wherein, the host cluster is the execution carrier of the specified virtual machine cluster. An initial scheduling scheme forming module is used to form an initial scheduling scheme based on a preset initial scheduling method and according to each of the first state information and the second state information; The target virtual machine allocation module is used to execute the initial scheduling scheme, allocate the target virtual machines to each of the target hosts, and perform inspections on the specified host cluster; The first state vector acquisition module is used to acquire the first state vector of each target virtual machine and the second state vector of the target host when the inspection result does not meet the preset inspection requirements. Specifically, it includes: acquiring the CPU utilization, CPU resource demand vector, current host location, virtual machine type, and the mean and standard deviation of historical CPU utilization of each target virtual machine; acquiring the host type, host CPU utilization, and remaining available resource capacity of the target host; forming the first state vector based on the CPU utilization, CPU resource demand vector, current host location, virtual machine type, and the mean and standard deviation of historical CPU utilization of the target virtual machine; and forming the second state vector based on the host type, host CPU utilization, and remaining available resource capacity of the target host. A target action set generation module is used to input the first state vector and the second state vector into an initial agent to generate a target action set. Specifically, this includes: inputting the first state vector and the second state vector into the initial agent to construct constraints, obtaining a target agent; generating an initial action set based on the target agent; and optimizing the initial action set according to preset action rules to obtain the target action set. Specifically, this includes: obtaining the average utilization rate of each target host; calculating a penalty index for each target host based on the average utilization rate; and optimizing the initial action set based on the penalty index to obtain the target action set. The optimized scheduling scheme acquisition module is used to optimize the target action set through a preset reinforcement learning strategy to obtain an optimized scheduling scheme, and to reschedule each of the target virtual machines to each of the target hosts according to the optimized scheduling scheme.

6. The apparatus according to claim 5, characterized in that, When the first state vector acquisition module acquires the virtual machine type of each of the target virtual machines, it includes: Periodically collect the CPU utilization vector of the target virtual machine; Calculate the first day's average utilization, the first night's average utilization, and the utilization variance of the target virtual machine based on the CPU utilization vector. The virtual machine type is determined based on the first day's average utilization, the first night's average utilization, and the utilization variance of the target virtual machine.

7. The apparatus according to claim 5, characterized in that, When the first state vector acquisition module acquires the host type of each of the target hosts, it includes: Periodically collect the CPU utilization of the target host to obtain a set of host CPU utilization data; Calculate the second day average utilization and the second night average utilization of the target host based on the host CPU utilization set; The host type of the target host is determined based on the average utilization rate of the target host during the second day and the average utilization rate during the second night.

8. The apparatus according to claim 5, characterized in that, When the optimized scheduling scheme acquisition module optimizes the target action set using a preset reinforcement learning strategy to obtain an optimized scheduling scheme, and reschedules each of the target virtual machines to each of the target hosts according to the optimized scheduling scheme, the module includes: Obtain the strategy optimization objective; Based on the optimization objective of the strategy, the objective function is set using a preset SAC algorithm; The objective function guides the optimization process of the initial agent to optimize the target action set and obtain an optimized scheduling scheme. According to the optimized scheduling scheme, each of the target virtual machines is rescheduled to each of the target hosts.

9. A computer-readable storage medium, characterized in that, The system stores a computer program that, when executed by a processor, causes the processor to perform the steps of the virtual machine cooperative scheduling method as described in any one of claims 1 to 4.

10. An electronic device, characterized in that, The device includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the virtual machine cooperative scheduling method as described in any one of claims 1 to 4.

Images