Resource management method and device of cloud platform, electronic equipment and storage medium

Abstract

The application discloses a resource management method and device of a cloud platform, electronic equipment and a storage medium, and relates to the technical field of computers. According to the application, the future failure probability is accurately calculated and the health level is divided based on physical host running state data and a time sequence prediction model, the resource reservation demand is dynamically determined in combination with the resource specification and the failure probability of the virtual machine on the target health level host, and the corresponding computing and memory resources are reserved only in the host with the qualified health level, so that the accurate matching of the resource reservation and the actual failure risk is realized, the secondary failure probability of the cloud computing platform is reduced, the rationality and accuracy of the resource reservation are optimized, the resource utilization efficiency is improved, and the technical effects of strengthening the business continuity guarantee capability of the cloud platform and the reliability of large-scale resource scheduling management are achieved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to cloud platform resource management methods, apparatus, electronic devices and storage media. Background Technology

[0002] As the core infrastructure of modern data centers, cloud computing platforms are widely used for enterprise-level business continuity assurance and large-scale resource scheduling and management. Among related technologies, a high-availability assurance system based on host connectivity checks has been built through the collaborative operation of hardware monitoring, performance analysis, and log parsing.

[0003] Existing high availability assurance methods directly adopt fixed-ratio resource reservation or static host isolation strategies, without dynamically adjusting resource allocation based on actual failure risks. This significantly increases the risk of secondary failures when unhealthy host resources are misused. Summary of the Invention

[0004] This application provides a resource management method, apparatus, electronic device, and storage medium for cloud platforms, in order to at least address the problem that the risk of secondary failures increases significantly when the resources of a sub-healthy host are misused in related technologies.

[0005] This application provides a resource management method for a cloud platform, including: Obtain the running status data of multiple physical hosts in the cloud platform; Based on the aforementioned operational status data and time-series prediction model, the failure probability of each physical host in future time periods is obtained; The physical host is classified into different health levels based on the failure probability. Based on the resource specifications of virtual machines deployed on physical hosts at the target health level and their corresponding failure probabilities, determine the reserved resource requirements for each resource specification. Based on the reserved resource requirements, computing and memory resources are reserved in hosts with a health level higher than or equal to the target health level, and the resource reservation information is updated.

[0006] Optionally, the operational status data includes at least two of the following: hardware monitoring data, performance index data, and system log data.

[0007] Optionally, classifying the physical host into different health levels based on the failure probability includes: When the probability of failure exceeds the first threshold, it is judged to be in an unhealthy state; When the probability of failure is below the second threshold, it is judged to be in a healthy state; When the failure probability is between the first threshold and the second threshold, it is determined to be an intermediate healthy state; Wherein, the first threshold is greater than the second threshold.

[0008] Optionally, determining the reserved resource requirements for each resource specification based on the resource specifications of virtual machines deployed on physical hosts at the target health level and their corresponding failure probabilities includes: For each resource specification, the number of virtual machines of each resource specification in each intermediate health level host is accumulated, and then multiplied by the failure probability of the corresponding host to obtain the weighted number. The weighted quantity is rounded down to obtain the reserved resource requirement for each of the resource specifications.

[0009] Optionally, reserving computing and memory resources in hosts with a health level higher than or equal to the target health level based on the reserved resource requirement includes: The resources are sorted according to their resource usage, and resources are reserved for each resource specification in turn based on the sorting results. Among them, the reservation operation is performed on the host with the highest health level that has enough remaining resources.

[0010] Optionally, the method further includes: When a virtual machine scheduling request is a fault evacuation request, resources should be allocated from the reserved resources first. When the scheduling request is a new creation or migration request, resources are allocated from non-reserved resources first.

[0011] This application also provides a resource management device for a cloud platform, comprising: The acquisition unit is used to acquire the running status data of multiple physical hosts in the cloud platform; The calculation unit is used to obtain the failure probability of each physical host in a future time period based on the operating status data and time series prediction model; A partitioning unit is used to divide the physical host into different health levels based on the failure probability; The determination unit is used to determine the reserved resource requirements for each resource specification based on the resource specifications of virtual machines deployed on physical hosts at the target health level and their corresponding failure probabilities. The reservation unit is used to reserve computing and memory resources in hosts with a health level higher than or equal to the target health level according to the reserved resource demand, and to update the resource reservation information.

[0012] Optionally, the operational status data includes at least two of the following: hardware monitoring data, performance index data, and system log data.

[0013] Optionally, the partitioning unit is further configured to: When the probability of failure exceeds the first threshold, it is judged to be in an unhealthy state; When the probability of failure is below the second threshold, it is judged to be in a healthy state; When the failure probability is between the first threshold and the second threshold, it is determined to be an intermediate healthy state; Wherein, the first threshold is greater than the second threshold.

[0014] Optionally, the determining unit is further configured to: For each resource specification, the number of virtual machines of each resource specification in each intermediate health level host is accumulated, and then multiplied by the failure probability of the corresponding host to obtain the weighted number. The weighted quantity is rounded down to obtain the reserved resource requirement for each of the resource specifications.

[0015] Optionally, the reserved unit is further used for: The resources are sorted according to their resource usage, and resources are reserved for each resource specification in turn based on the sorting results. Among them, the reservation operation is performed on the host with the highest health level that has enough remaining resources.

[0016] Optionally, the device further includes: The allocation unit is used to allocate resources from the reserved resources first when the virtual machine scheduling request is a fault evacuation request. Allocation unit. Also used to prioritize allocating resources from non-reserved resources when the scheduling request is a new creation or migration request.

[0017] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the resource management method of any of the cloud platforms described above.

[0018] This application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of any of the above-described cloud platform resource management methods.

[0019] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described cloud platform resource management methods.

[0020] This application achieves a precise match between resource reservation and actual failure risk by accurately calculating future failure probabilities and classifying health levels based on physical host operating status data and time-series prediction models. It dynamically determines the required reserved resources by combining the resource specifications and failure probabilities of virtual machines on hosts with the target health level, reserving corresponding computing and memory resources only on hosts that meet the health level criteria. Therefore, it solves the technical problem in existing high availability assurance methods where fixed-ratio resource reservations or static host isolation strategies fail to dynamically adjust resource allocation based on actual failure risks, leading to the misuse of resources on sub-healthy hosts and increasing the risk of secondary failures. This results in reducing the probability of secondary failures on cloud computing platforms, optimizing the rationality and accuracy of resource reservations, improving resource utilization efficiency, and ultimately strengthening the business continuity assurance capabilities and the reliability of large-scale resource scheduling and management of cloud platforms. Attached Figure Description

[0021] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a resource management method for a cloud platform provided in an embodiment of this application; Figure 2 A schematic diagram of the structure of a resource management device for a cloud platform provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of another cloud platform resource management device provided in an embodiment of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0024] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0025] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] The embodiments of this application provide a resource management method for a cloud platform. The method is described in detail below in conjunction with the execution flow of the resource management method for a cloud platform. Figure 1 This is a flowchart illustrating a cloud platform resource management method provided in an embodiment of this application.

[0027] like Figure 1 As shown, the method includes the following steps: Step 101: Obtain the running status data of multiple physical hosts in the cloud platform; Operational status data includes three main categories: hardware health metrics, system performance metrics, and log event metrics. Hardware health metrics originate from the self-monitoring functions of physical hardware, such as SMART test information for disks, including the number of remapped sectors and temperature parameters, as well as temperature sensor readings for the CPU and memory, and ECC error counts for memory, etc.

[0028] System performance metrics are obtained through the operating system and monitoring tools, such as CPU utilization, memory utilization, disk I / O latency, and network packet loss rate. These metrics reflect the host's real-time load and resource utilization. Log event metrics originate from system log files, recording error events and alarm information during software and hardware operation. To obtain this data, dedicated data collectors need to be deployed on each physical host. These collectors can be categorized based on data type: hardware metric collectors, performance metric collectors, and log collectors. The collectors run at predetermined time intervals, such as once per minute, collecting raw data from the host system.

[0029] During the data collection process, the collector may need to perform preliminary data cleaning, such as removing invalid values ​​or formatting timestamps, to ensure data consistency. The collected runtime status data is then transmitted to the central monitoring module of the cloud platform via a secure network channel. In the monitoring module, the data is temporarily stored and prepared for subsequent analysis. To ensure the efficiency and reliability of data collection, the collector is typically designed as a lightweight daemon process to minimize the impact on host performance. Furthermore, the collection configuration can be dynamically adjusted to adapt to the characteristics of different hosts and network conditions. The acquisition of runtime status data is a fundamental aspect of cloud platform management, relying on the integration of multiple data sources.

[0030] Hardware metric collectors interact directly with hardware interfaces, such as by reading SMART attributes from the disk or sensor data. Performance metric collectors invoke performance counters or monitoring tools provided by the operating system, such as the Proc file system. Log collectors parse system log files in real time to extract key events. These collectors can run independently or be integrated into an agent. The data collection cycle can be adjusted according to operational strategies, such as shortening the collection interval during high-load periods to obtain more granular data. The collected data is typically organized in time-series format, with each data point containing a timestamp, host identifier, and metric value.

[0031] During transmission, data may be compressed or encrypted to ensure efficiency and security. After receiving the data, the monitoring module performs preliminary verification, such as checking the data format and range, to ensure data validity. Furthermore, acquiring operational status data involves collaboration with other components of the cloud platform, such as synchronizing host status with the resource database. By continuously and comprehensively acquiring operational status data, the cloud platform can establish a historical record of host health, providing a solid data foundation for subsequent fault prediction and high-availability decisions.

[0032] Step 102: Based on the operating status data and the time series prediction model, obtain the failure probability of each physical host in the future time period; By leveraging historical and current operational status data, a time-series prediction model is used to quantify the probability of each physical host failing within a specific future time window. The time-series prediction model is a machine learning model specifically designed for processing time-series data, capable of capturing dynamic patterns and dependencies in the data over time. Specifically, the model takes as input the hardware health metric sequence, system performance metric sequence, and log event metric sequence obtained in the previous stage.

[0033] The raw indicator data first needs to be preprocessed, including data cleaning to handle possible missing or outlier values, and normalization to eliminate the influence of different indicator units, scaling the values ​​to a uniform range. The preprocessed data is then constructed into a standard time series format, i.e., a three-dimensional data structure containing time steps and feature dimensions.

[0034] Each time step represents a specific historical sampling moment, such as data points per minute over the past 24 hours, while the feature dimensions correspond to different types of monitoring metrics. The time-series prediction model learns from this chronologically ordered data to identify potential patterns and early signs leading to host failures. During the training phase, the model uses a large amount of historical data, including both normal operating state sequences and sequences of subsequent failures, to learn the data evolution patterns before failures occur. In the prediction phase, the model receives the latest operating state data sequences from the current moment and a past time window, performs internal calculations and feature extraction, and finally outputs a value between zero and one, which is the predicted failure probability.

[0035] The closer the failure probability value is to one, the higher the risk of the physical host failing in the future; conversely, the closer it is to zero, the lower the risk. This prediction is performed periodically, for example, every five minutes or hour, to ensure real-time assessment of host health. The model's output will serve as a key basis for subsequent high-availability decisions. The entire prediction process is automated within the cloud platform's prediction module, which loads pre-trained model parameters and receives real-time data streams from the monitoring module, thereby efficiently and continuously quantifying the failure risk of all hosts in the cluster.

[0036] Step 103: Divide the physical host into different health levels according to the failure probability; Based on the predicted failure probability values, all physical hosts in the cluster are classified into health status categories, thus providing a clear basis for subsequent differentiated resource management and high availability strategy implementation. The classification of health levels is achieved by setting predefined probability threshold ranges, which map continuous failure probability values ​​to discrete health status indicators.

[0037] The system defines multiple health levels, such as healthy, sub-healthy, and unhealthy, each corresponding to a specific range of failure probabilities. When the failure probability value of a physical host falls into a low threshold range, for example, very close to zero, it indicates that the risk of failure for that host during the predicted period is extremely low, and the system marks it as healthy. Hosts marked as healthy are considered stable and reliable resource providers, can take priority in undertaking new virtual machine deployment tasks, and can serve as the primary bearer of reserved resources.

[0038] When the failure probability value falls within the threshold range of the middle range, it indicates that the host has some potential risk, but has not yet reached the level of urgency requiring immediate action. The system will mark it as being in a sub-healthy state. Hosts in a sub-healthy state can usually continue to run existing services, but the virtual machines running on them are identified as objects that may require backup resource protection. Therefore, these hosts will become the primary targets of dynamic resource reservation calculations.

[0039] When the failure probability value falls into a high threshold range, such as being very close to one, it indicates a high risk of host failure, and the system will mark it as unhealthy. Hosts in an unhealthy state are considered high-risk nodes about to fail, requiring proactive preventative measures. After completing the health level classification, the system immediately updates the host's health status attributes and synchronizes this status change to core components of the cloud platform, such as the scheduler and resource database. The scheduler will adjust its scheduling strategy based on the host's latest health status, for example, temporarily placing unhealthy hosts into an unschedulable state to prevent new virtual machines from being deployed on them.

[0040] Changes in health status may also trigger corresponding alarm notifications to alert operations and maintenance personnel. Through this hierarchical classification mechanism, the cloud platform can transform abstract failure probability prediction results into specific and actionable system status instructions, achieving a tiered and refined risk perception.

[0041] Step 104: Determine the reserved resource requirements for each resource specification based on the resource specifications of the virtual machines deployed on the physical host at the target health level and their corresponding failure probabilities. The system needs to determine the target health level, which typically refers to a set of physical hosts marked as being in a sub-healthy state. These hosts pose a certain risk of failure, therefore the virtual machines running on them require backup resources to ensure business continuity. The system retrieves a list of all hosts in the cluster that are in a sub-healthy state. For each sub-healthy host in the list, the system further analyzes all currently deployed virtual machine instances.

[0042] Each virtual machine instance is associated with a predefined resource specification, which explicitly defines the amount of computing resources the virtual machine uses, such as the number of virtual CPU cores and memory capacity. The system counts the number of virtual machines corresponding to each resource specification running on the sub-healthy host. Next, the calculation process incorporates the host's own failure probability as a weighting factor. For each resource specification, the system iterates through all sub-healthy hosts, multiplies the number of virtual machines of that specification on all sub-healthy hosts by the failure probability of each host, and then sums all the products. This weighted summation formula is: multiply the number of virtual machines of a certain specification on each sub-healthy host by the failure probability of that host, and then sum the results for all hosts.

[0043] The summation result is a numerical value reflecting the total virtual machine demand based on risk weighting. Since reserved resources need to exist as whole units, this value is typically rounded up to obtain the final theoretical reserved resource requirement for that resource specification—that is, the number of complete resource replicas that need to be reserved on healthy hosts. The system repeats the above calculation process for all existing resource specifications in the cluster.

[0044] It outputs a detailed mapping between resource specifications and reserved requirements. For example, specification A requires N units to be reserved, and specification B requires M units. This calculation process is executed periodically or triggered dynamically, updating the reserved requirements based on changes in host health and virtual machine deployment. Through this method, resource reservation is no longer a static or experience-based configuration, but rather a precise quantification mechanism based on real-time risk assessment and actual business load. This ensures high availability while minimizing resource idle waste.

[0045] Step 105: Based on the reserved resource requirement, reserve computing and memory resources in hosts with a health level higher than or equal to the target health level, and update the resource reservation information.

[0046] Step 105 involves reserving computing and memory resources on hosts with a health level higher than or equal to the target health level based on the reserved resource requirements, and updating the resource reservation information. This step is a crucial execution process that transforms theoretical reservation requirements into actual system resource reservations, with the goal of physically allocating spare resources within the cluster for high availability assurance. The execution process begins by obtaining the list of reserved resource requirements for each resource specification calculated in the previous step.

[0047] The system first determines the range of hosts that can be used to carry reserved resources, namely all hosts with a health level not lower than the target health level, typically including all physical hosts marked as healthy or sub-healthy. To allocate reserved resources efficiently and reasonably, the system sorts these qualified hosts according to some resource availability, such as sorting them in descending order of remaining available memory capacity, prioritizing hosts with more abundant resources to undertake the reservation task. Subsequently, the system iterates through each resource specification that needs to be reserved and its corresponding requirement. For each specification, the system searches the sorted list of qualified hosts for hosts that can meet the resource requirements according to its specified computing and memory resource requirements.

[0048] The specific search process involves checking each host one by one, starting from the top of the list, to determine if the available computing and memory resources that the host currently has that are neither reserved nor occupied are greater than or equal to the amount of resources required by the current specification. Once a host that meets the criteria is found, the system performs a reservation operation on that host. The reservation operation is reflected in updating the record for that host in the cloud platform resource database, specifically increasing the total amount of computing and memory resources reserved for high availability on that host, that is, accumulating the amount of computing and memory resources required by the specification into the corresponding reserved resource attribute fields.

[0049] To accurately reflect the reduction in resources currently available for normal scheduling on a host, the values ​​of total available computing resources and total available memory resources in the host's record need to be reduced simultaneously. After reserving one set of resources on a host, the count of reserved resources allocated for that specification is incremented by one, and the search continues for the next host of the same specification to allocate the next set of resources, until the total reserved requirement for that specification is met, or all qualified hosts are unable to provide a complete set of the required resources.

[0050] The system will repeat this allocation process for all resource specifications. During the allocation process, if a specification fails to allocate all the required number of reserved resources, the system will generate a corresponding resource shortage alarm and notify the cloud platform administrator to intervene, such as expanding cluster resources. After all reservation operations are completed, the resource reservation status of the relevant hosts in the cluster is determined, and the reservation information field in the resource database is updated in real time, providing accurate resource availability information for subsequent scheduling operations such as virtual machine evacuation.

[0051] In some embodiments, the operational status data includes at least two of hardware monitoring data, performance indicator data, and system log data.

[0052] Hardware monitoring data originates directly from the monitoring mechanisms and sensors of the physical hardware itself, reflecting the basic health status and potential defects of hardware components. This type of data may include, but is not limited to, information provided by SMART self-monitoring analysis reporting technology for disk drives, such as remapped sector counts, read error rates, boot cycle counts, and temperature; it also includes temperature sensor readings for the central processing unit and memory modules, as well as correctable error check and correction counts in the memory subsystem.

[0053] These metrics are often early indicators of hardware failure. Performance metrics reflect the host's load and efficiency at the software runtime level, collected by the operating system or monitoring agents. This type of data covers the utilization of computing resources, such as CPU utilization and memory utilization; it also includes input / output performance data, such as disk read / write latency and throughput; and network communication quality metrics, such as network packet loss rate and bandwidth usage. These metrics can reveal signs of system instability caused by resource overload or performance bottlenecks. System log data is a record of events extracted from the operating system and critical service log files, containing information on errors, warnings, and critical state changes generated during program execution.

[0054] By analyzing these time-series log events, failure modes related to software anomalies, driver issues, or configuration conflicts can be identified. The advantage of fusing at least two of these data types is that it overcomes the limitations and biases of a single data source. For example, relying solely on performance metrics may fail to detect potential hardware degradation, while analyzing only logs may miss early warning signs of performance-related failures caused by resource contention. By combining monitoring data characterizing the underlying hardware, performance data reflecting real-time load, and log data recording discrete events, time-series predictive models can obtain richer feature inputs, thereby learning more complex failure patterns and more subtle anomaly patterns. After collection, these different types of data are uniformly reported to the cloud platform's monitoring module and aligned according to timestamps, collectively forming a complete runtime status time-series sequence with multiple feature channels for model training and prediction.

[0055] In some embodiments, classifying the physical host into different health levels based on the failure probability includes: When the probability of failure exceeds the first threshold, it is judged to be in an unhealthy state; When the probability of failure is below the second threshold, it is judged to be in a healthy state; When the failure probability is between the first threshold and the second threshold, it is determined to be an intermediate healthy state; Wherein, the first threshold is greater than the second threshold.

[0056] The step of classifying the physical host into different health levels based on the failure probability includes determining it as unhealthy when the failure probability is higher than a first threshold, determining it as healthy when the failure probability is lower than a second threshold, and determining it as an intermediate healthy state when the failure probability is between the first threshold and the second threshold, wherein the first threshold is greater than the second threshold.

[0057] This feature clarifies the specific rules for mapping continuous failure probability values ​​to discrete health levels. By setting two key probability thresholds, a three-level state classification is achieved, providing a clear decision boundary for implementing differentiated operation and maintenance strategies. The first threshold is a high probability threshold used to identify high-risk states. When the predicted failure probability value of a physical host exceeds this first threshold, it indicates that the likelihood of failure in the future is extremely high, and the system will classify it as unhealthy. Hosts in an unhealthy state will trigger the highest level of early warning and handling procedures.

[0058] The second threshold is a lower probability threshold used to identify a low-risk state. When the failure probability value of a host is below this second threshold, it indicates that its operation is stable and reliable with extremely low risk, and the system will classify it as a healthy state. Hosts in a healthy state are the first choice for carrying critical business and reserving resources. When the failure probability value is between the first and second thresholds—that is, neither reaching the high-risk threshold nor exceeding the low-risk safety line—the system classifies it as an intermediate healthy state, also known as a sub-healthy state. Such hosts have some potential risks, but have not yet reached the urgency level requiring immediate withdrawal of business operations.

[0059] The specific values ​​of the first and second thresholds can be set and adjusted based on historical fault data statistical analysis results or domain operation and maintenance experience. For example, the first threshold can be set to 0.8 and the second threshold can be set to 0.3. This three-level division mechanism constructs a gradient risk response framework: unhealthy state corresponds to immediate proactive handling; intermediate healthy state corresponds to preventive resource planning; and healthy state corresponds to normal scheduling and operation.

[0060] In some embodiments, determining the reserved resource requirements for each resource specification based on the resource specifications of virtual machines deployed on physical hosts at the target health level and their corresponding failure probabilities includes: For each resource specification, the number of virtual machines of each resource specification in each intermediate health level host is accumulated, and then multiplied by the failure probability of the corresponding host to obtain the weighted number. The weighted quantity is rounded down to obtain the reserved resource requirement for each of the resource specifications.

[0061] The determination of the reserved resource requirement for each resource specification based on the resource specifications of virtual machines deployed on physical hosts at the target health level and their corresponding failure probabilities includes, for each resource specification, accumulating the number of virtual machines of each resource specification in each intermediate health level host, multiplying them by the failure probability of the corresponding host to obtain a weighted quantity; and rounding the weighted quantity to obtain the reserved resource requirement for each resource specification.

[0062] The core of accurately calculating the reserved demand for high-availability resources lies in integrating the probability of host failure risk into resource demand assessment, thereby achieving risk-weighted dynamic resource planning. Specifically, the system first identifies all hosts in the intermediate health level, which are considered to have a moderate risk of failure. For each resource specification defined in the cloud platform, the system needs to iterate through these hosts in the intermediate health level. For each such host, the system counts the number of virtual machine instances currently running on it that belong to that specific resource specification.

[0063] This quantity is multiplied by the host's predicted failure probability. This multiplication assigns a risk weight to the quantity; the higher the failure probability of a host, the greater the contribution of its virtual machines to the demand for reserved resources, because the host is more likely to fail, requiring the evacuation and restart of its virtual machines. Next, the system performs the above calculation on all hosts with intermediate health levels, summing the products obtained for each host. This sum represents the weighted total virtual machine demand for this resource specification across all risky hosts, reflecting the scale of backup resources needed to ensure business continuity after considering the differences in the probability of failure among different hosts.

[0064] Since actual resource reservation operations must be performed on a complete resource unit basis—for example, resources required for half a virtual machine cannot be reserved—the calculated weighted quantity is usually a decimal and needs to be rounded down. The rounding rule is generally set to round up to ensure that the number of reserved resource units at least meets the weighted requirement, avoiding insufficient reservation due to discarding decimal parts.

[0065] The rounded integer value is determined as the reserved resource requirement for this resource specification, which is the number of complete resource copies that need to be reserved in advance in the cluster. This calculation process is executed automatically and periodically, responding in real time to changes in host health and virtual machine deployment migrations, thereby ensuring the accuracy and timeliness of the reserved requirement.

[0066] This approach transforms resource reservation from a traditional, static configuration based on fixed ratios or experience into a dynamic, data-driven intelligent decision-making process. It effectively balances resource utilization efficiency with high availability assurance, avoiding resource waste due to over-reservation and preventing the inability to successfully evacuate virtual machines in the event of host failure due to insufficient reservation. The calculated reserved resource requirements directly guide subsequent actual resource reservation operations, providing crucial data input for the high availability management of the cloud platform.

[0067] During the accumulation process, the system maintains a mapping structure to record each resource specification and its corresponding cumulative weighted value. For each host in the intermediate healthy host list, its failure probability value is given by the prediction model and is a value between zero and one. When counting virtual machines, the system needs to query the virtual machine deployment information of that host, which is usually stored in the cloud platform's resource database. The multiplication operation reflects the risk weighting concept; virtual machines running on hosts with high failure probabilities are considered to have a higher probability of needing to be migrated, and therefore need to be allocated a larger share of backup resources. The accumulation operation ensures that the demand across all risky hosts is aggregated, thus obtaining the overall demand at the cluster level. Rounding is a practical requirement for resource reservation because resource allocation must be based on the smallest allocation unit, such as a virtual CPU core or a gigabyte of memory.

[0068] Rounding up ensures a conservative estimate of demand, providing a buffer for potential calculation errors or unforeseen circumstances, and further enhancing the system's robustness. Furthermore, this calculation process can consider the priority of resource specifications; for example, specifications with high resource consumption can be calculated first to ensure that critical operations have sufficient backup resources.

[0069] In some embodiments, reserving computing and memory resources in hosts with a health level higher than or equal to the target health level based on the reserved resource requirement includes: The resources are sorted according to their resource usage, and resources are reserved for each resource specification in turn based on the sorting results. Among them, the reservation operation is performed on the host with the highest health level that has enough remaining resources.

[0070] The system prioritizes reserving resources from hosts with high health levels, selecting those with sufficient remaining resources. This feature specifies the execution strategy and order of actual resource reservation, aiming to ensure the efficiency, stability, and rational utilization of resources through optimized allocation logic. First, the system sorts all resource specifications requiring reservation in descending order of their resource consumption, with specifications requiring larger resources appearing at the top of the list. This sorting strategy prioritizes specifications consuming more computing and memory resources, as the successful reservation of large-scale resources has a greater impact on the overall cluster resource planning and is more susceptible to resource fragmentation. Prioritizing the reservation of large-scale resources improves the overall reservation success rate and avoids the problem of large-scale resources not finding sufficient contiguous resource blocks due to the initial allocation of smaller-scale resources.

[0071] The sorting is typically based on the sum of the number of virtual CPU cores and memory capacity defined in the resource specification, or their combined weight. When processing each resource specification in the sorted order, the system allocates the specific number of shares required for its reserved resource needs. The process of selecting a host for each share of resource follows explicit priority rules. The system first searches for suitable hosts from the group of hosts with the highest health levels—that is, the list of hosts deemed healthy. The system iterates through the sorted list of healthy hosts, checking whether the current remaining available computing and memory resources of each host are greater than or equal to the amount of resources required for one share of the current resource specification. The remaining available resources referred to here are the host's total resources minus the allocated resources and the reserved resources.

[0072] Once a host meeting the criteria is found in the healthy host list, it is immediately selected and reserved. If a suitable host cannot be found for the current number of resource units after traversing the healthy host list, the system will turn to the next highest health level host group, i.e., the list of hosts judged to be in an intermediate health state, and perform the same resource satisfaction check. This strategy of prioritizing hosts with higher health levels aims to place valuable reserved resources on more stable and reliable physical hosts, thereby minimizing the secondary risk of reserved resources becoming unavailable due to the failure of the reserved host itself, and ensuring the availability of the high-availability backup resource pool itself.

[0073] Once a host is selected, the system performs a resource reservation operation, updating the host's record in the resource database, increasing its reserved resource attribute value, and correspondingly decreasing its total available resource attribute value to reflect successful resource reservation. The system repeats this host selection and resource update process for the current specification until all copies of that specification are reserved, or until all qualified hosts are unable to meet the resource requirements. The system then moves on to the next resource specification in the sorted list. This reservation method, which combines specification sorting with host health priority, not only achieves the physical allocation of reserved resources but also optimizes the quality and security of resource allocation, making the construction of the entire high-availability resource pool more scientific and reliable.

[0074] In some embodiments, the method further includes: When a virtual machine scheduling request is a fault evacuation request, resources should be allocated from the reserved resources first. When the scheduling request is a new creation or migration request, resources are allocated from non-reserved resources first.

[0075] This feature establishes the core strategy followed by the cloud platform scheduler when handling different types of resource requests. It aims to achieve fine-grained management and effective isolation of reserved and non-reserved resources by differentiating between the urgency and business purpose of the request, thereby optimizing overall resource utilization while ensuring high availability. When the scheduler receives a virtual machine scheduling request, it first needs to determine the type of the request. If the request is identified as a fault evacuation request—that is, a situation where the virtual machine needs to be urgently migrated to another host due to a failure of the source host or a predicted high risk—the scheduler will initiate a priority allocation process for reserved resources.

[0076] The scheduler filters hosts in the cluster that meet the health requirements and whose reserved resource attributes (computing and memory resources) can meet the virtual machine's needs. Reserved resources are backup resources pre-allocated for high availability purposes, specifically designed to handle such failure scenarios. The scheduler selects one eligible candidate host, typically based on strategies such as load balancing or proximity, then deducts the required resources from that host's reserved resource quota and updates the corresponding reserved resource attribute and used resource attribute values ​​in the host's resource database.

[0077] This mechanism ensures that fault evacuation requests can quickly and reliably obtain resources, greatly improving the certainty and speed of business recovery and meeting the stringent requirements for recovery time targets in high-availability scenarios. Conversely, when scheduling requests are for creating new virtual machines or regular migration requests, these requests are planned or non-urgent business operations.

[0078] The scheduler prioritizes allocation from the host's non-reserved resources, i.e., ordinary available resources. The scheduler selects hosts whose total available resources minus used resources and reserved resources still have enough remaining to meet the virtual machine's needs. Prioritizing the use of non-reserved resources ensures that the dedicated resource pool reserved for fault evacuation scenarios is not arbitrarily occupied by daily business operations, thus maintaining a certain capacity of high-availability emergency resources at all times.

[0079] Only when non-reserved resources are insufficient can the scheduler consider other resource sources based on more complex strategies. This request-type-based differentiated scheduling strategy constructs a clear multi-level resource allocation system. Logically dividing the resource pool into a safeguard portion for emergency fault recovery and a general portion for daily business expansion achieves an effective balance between resource assurance and resource efficiency. The scheduler executes this strategy by querying the host's resource reservation and health status in real time, enabling the entire resource scheduling process to adaptively support the cloud platform's high availability goals and daily operational needs.

[0080] The following example illustrates a cloud platform resource management method provided in the embodiments of this application.

[0081] The specific implementation process of this invention is as follows: Metrics collection: Hardware metrics, performance metrics, and log collectors are deployed on each host in the cluster to collect the corresponding metrics at 1-minute intervals and upload them to the cloud platform monitoring module.

[0082] Model Training: The LSTM model was trained using three years of historical monitoring and host failure data. The core of the model training lies in the preparation and preprocessing of the data, and the detailed process is as follows: Data collection: The monitoring and fault data from the past 3 years were divided into different samples with a 24-hour interval. The samples included key indicators and fault records within 24 hours (1 for fault occurrence and 0 for normal operation).

[0083] Data processing: For each sample, missing values ​​are removed, and then normalization is performed. Finally, the sequence required by the LSTM is constructed according to the format (number of samples, time step, feature dimension), where the time step is 1440 (predicting the next moment based on data from the past 24 hours), and the feature dimension is 12. Finally, the label of each sample is mapped to whether a fault has occurred at the next moment of the last time step.

[0084] Fault prediction The cloud platform monitoring module preprocesses the monitoring metrics from the past 24 hours to generate a sequence of metrics across three dimensions: hardware metrics, system metrics, and log metrics.

[0085] The three index sequences are used as three LSTM channels and are processed by the feature fusion layer and the fully connected layer, respectively.

[0086] The sigmoid activation function is used to output the failure probability of all hosts in the cluster, and the result is denoted as... ( N, where N is the number of hosts in the cluster.

[0087] High-availability execution process Based on the probability of failure, the host health is marked, and the specific strategy is as follows: if 0.8, mark the host health as unhealthy, mark this host as unschedulable in the scheduler, and report the host unhealthy alarm to the monitoring alarm module.

[0088] If 0.3 0.8 indicates the host's health is in a sub-healthy state.

[0089] if A value less than 0.3 indicates that the host's health is healthy.

[0090] Actively evacuate virtual machines from unhealthy hosts: For unhealthy hosts in the cluster, migrate virtual machines to other hosts using a hot migration method.

[0091] High availability resource reservation (the cloud platform's compute_nodes table adds two attributes, vcpus_reserved and mem_reserved, to record the amount of CPU and memory reserved for high availability on the host, respectively), the specific process is as follows: Retrieve the list of sub-healthy hosts in the cluster, denoted as [ , ,.., ].

[0092] Calculate the sub-health host separately ( The number of virtual machines of virtual machine specification falvor_j on K) is .

[0093] The number of virtual machines of different specifications on all hosts is calculated and denoted as {flavor_j:} }( M). Among them. = (Round the result up to the nearest integer).

[0094] Sort the specification list flavor_1, flavor_2, ..., flavor_M in descending order of resource size, denoted as […]. , ,…, .

[0095] Reserve CPU and memory resources from healthy or sub-healthy hosts.

[0096] Sort the list of healthy hosts in the cluster in descending order of remaining memory, denoted as . , ,…, .

[0097] Sort the list of hosts in the cluster that are in poor health from largest to smallest based on their remaining memory, and denote it as follows: , ,…, .

[0098] From the specifications list , ,…, In the middle, select one specification flavor_j in sequence (let its CPU be C_j and its memory size be M_j), and perform the following operations to reserve one set of resources for this specification.

[0099] If the number of resources already reserved for specification flavor_j is less than If yes, then execute ii.; otherwise, no more resources will be reserved for this specification.

[0100] from ,…, , ,…, Select hosts that meet the specifications in terms of remaining memory and CPU, let's say host_k, and update the resource attributes of the host's compute_nodes table: compute_nodes[host_k].vcpus_reserved += C_j compute_nodes[host_k].mem_reserved += M_j compute_nodes[host_k].vcpus -= C_j compute_nodes[host_k].mem_mb -= M_j Repeat step c) until enough resources are reserved for each specification, or there are not enough resources to reserve.

[0101] If a specification does not have enough reserved resources, an alarm will be reported to the monitoring and alarm module, prompting the administrator to expand the resources.

[0102] Scheduling module execution strategy When scheduling virtual machines (assuming the virtual machine's CPU is CPU and its memory size is MEM), it is necessary to determine the type of resource request.

[0103] Select a suitable host from the list of healthy hosts in the cluster, based on whether the host vcpus_reserved and mem_reserved meet the virtual machine requirements.

[0104] Assuming the selected host is host_l, update the properties of the corresponding database table: compute_nodes[host_l].vcpus_reserved -= CPU compute_nodes[host_l].mem_reserved -= MEM compute_nodes[host_l].vcpus += CPU compute_nodes[host_l].memeory_mb += MEM compute_nodes[host_l].vcpus_used += CPU compute_nodes[host_l].memeory_mb_used += MEM From the list of unhealthy hosts in the cluster, select a suitable host based on whether the host's vcpus_reserved and mem_reserved meet the virtual machine requirements. If a suitable host is found, update the database attributes; otherwise, scheduling fails.

[0105] From the list of healthy hosts in the cluster, select a suitable host based on whether the candidate hosts vcpus - vcpus_used and memeory_mb - memeory_mb_used meet the virtual machine requirements.

[0106] Assuming the host is selected as host_l, update the properties of the corresponding database table: compute_nodes[host_l].vcpus_used += CPU compute_nodes[host_l].memeory_mb_used += MEM From the list of unhealthy hosts in the cluster, select a suitable host based on whether the host vcpus - vcpus_used and memory_mb - memory_mb_used meet the virtual machine requirements. If a suitable host is found, update the database attributes; otherwise, scheduling fails.

[0107] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0108] Embodiments of this application also provide a resource management device for a cloud platform, such as... Figure 2 As shown, it includes: Acquisition unit 21 is used to acquire the running status data of multiple physical hosts in the cloud platform; The calculation unit 22 is used to obtain the failure probability of each physical host in a future time period based on the operating status data and the time series prediction model. The partitioning unit 23 is used to divide the physical host into different health levels according to the failure probability; The determining unit 24 is used to determine the reserved resource requirements for each resource specification based on the resource specifications of the virtual machines deployed on the physical host at the target health level and their corresponding failure probabilities. The reservation unit 25 is used to reserve computing and memory resources in hosts with a health level higher than or equal to the target health level according to the reserved resource demand, and update the resource reservation information.

[0109] Furthermore, in one possible implementation of this application embodiment, the operating status data includes at least two of hardware monitoring data, performance index data, and system log data.

[0110] Furthermore, in one possible implementation of this application embodiment, the partitioning unit 23 is further used for: When the probability of failure exceeds the first threshold, it is judged to be in an unhealthy state; When the probability of failure is below the second threshold, it is judged to be in a healthy state; When the failure probability is between the first threshold and the second threshold, it is determined to be an intermediate healthy state; Wherein, the first threshold is greater than the second threshold.

[0111] Furthermore, in one possible implementation of this application embodiment, the determining unit 24 is further configured to: For each resource specification, the number of virtual machines of each resource specification in each intermediate health level host is accumulated, and then multiplied by the failure probability of the corresponding host to obtain the weighted number. The weighted quantity is rounded down to obtain the reserved resource requirement for each of the resource specifications.

[0112] Furthermore, in one possible implementation of this application embodiment, the reserved unit 25 is further used for: The resources are sorted according to their resource usage, and resources are reserved for each resource specification in turn based on the sorting results. Among them, the reservation operation is performed on the host with the highest health level that has enough remaining resources.

[0113] Furthermore, in one possible implementation of the embodiments of this application, such as Figure 3 The apparatus shown also includes: Allocation unit 26 is used to allocate resources from the reserved resources first when the virtual machine scheduling request is a fault evacuation request; Allocation unit 26. It is also used to allocate resources from non-reserved resources first when the scheduling request is a new creation or migration request.

[0114] For a description of the features in the embodiment corresponding to the resource management device of the cloud platform, please refer to the relevant description of the embodiment corresponding to the resource management method of the cloud platform, which will not be repeated here.

[0115] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above-described cloud platform resource management method embodiments.

[0116] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described cloud platform resource management method embodiments when running.

[0117] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0118] The embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described cloud platform resource management method embodiments.

[0119] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described cloud platform resource management method embodiments.

[0120] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0121] The resource management method, apparatus, electronic device, and storage medium for a cloud platform provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A resource management method for a cloud platform, characterized in that, include: Obtain the running status data of multiple physical hosts in the cloud platform; Based on the aforementioned operational status data and time-series prediction model, the failure probability of each physical host in future time periods is obtained; The physical host is classified into different health levels based on the failure probability. Based on the resource specifications of virtual machines deployed on physical hosts at the target health level and their corresponding failure probabilities, determine the reserved resource requirements for each resource specification. Based on the reserved resource requirements, computing and memory resources are reserved in hosts with a health level higher than or equal to the target health level, and the resource reservation information is updated.

2. The method according to claim 1, characterized in that, The operational status data includes at least two of the following: hardware monitoring data, performance index data, and system log data.

3. The method according to claim 1, characterized in that, The step of classifying the physical host into different health levels based on the failure probability includes: When the probability of failure exceeds the first threshold, it is judged to be in an unhealthy state; When the probability of failure is below the second threshold, it is judged to be in a healthy state; When the failure probability is between the first threshold and the second threshold, it is determined to be an intermediate healthy state; Wherein, the first threshold is greater than the second threshold.

4. The method according to claim 1, characterized in that, The determination of the reserved resource requirements for each resource specification based on the resource specifications of virtual machines deployed on physical hosts at the target health level and their corresponding failure probabilities includes: For each resource specification, the number of virtual machines of each resource specification in each intermediate health level host is accumulated, and then multiplied by the failure probability of the corresponding host to obtain the weighted number. The weighted quantity is rounded down to obtain the reserved resource requirement for each of the resource specifications.

5. The method according to claim 1, characterized in that, The step of reserving computing and memory resources in hosts with a health level higher than or equal to the target health level based on the reserved resource requirement includes: The resources are sorted according to their resource usage, and resources are reserved for each resource specification in turn based on the sorting results. Among them, the reservation operation is performed on the host with the highest health level that has enough remaining resources.

6. The method according to claim 1, characterized in that, The method further includes: When a virtual machine scheduling request is a fault evacuation request, resources should be allocated from the reserved resources first. When the scheduling request is a new creation or migration request, resources are allocated from non-reserved resources first.

7. A resource management device for a cloud platform, characterized in that, include: The acquisition unit is used to acquire the running status data of multiple physical hosts in the cloud platform; The calculation unit is used to obtain the failure probability of each physical host in a future time period based on the operating status data and time series prediction model; A partitioning unit is used to divide the physical host into different health levels based on the failure probability; The determination unit is used to determine the reserved resource requirements for each resource specification based on the resource specifications of virtual machines deployed on physical hosts at the target health level and their corresponding failure probabilities. The reservation unit is used to reserve computing and memory resources in hosts with a health level higher than or equal to the target health level according to the reserved resource demand, and to update the resource reservation information.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the resource management method of the cloud platform as described in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the resource management method of the cloud platform as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the resource management method of the cloud platform as described in any one of claims 1 to 6.

Images