Cloud resource dynamic scheduling method, system and device and storage medium

Abstract

The invention discloses a cloud resource dynamic scheduling method, system and device and a storage medium, relates to the technical field of cloud computing resource management, and provides basic data support for resource demand prediction by collecting multi-dimensional core resource real-time data. The input feature weight is dynamically adjusted by means of an LSTM neural network model fused with a dynamic feature selection mechanism, and the short-term fluctuation and long-term trend of the resource demand are respectively captured through an LSTM structure, so that the time sequence analysis better fits the multi-scale characteristics of the load, and the accuracy of resource demand prediction is improved; a prediction result and a current resource configuration state are integrated through a benefit function, overall consideration of economic constraints and service quality requirements is realized, iterative solution is carried out in combination with an optimization algorithm, and a resource scheduling scheme can balance multiple objectives; meanwhile, the model design naturally has adaptive capacity to complex load changes, the deviation between resource configuration and actual requirements is reduced, and accurate prediction and efficient dynamic configuration of cloud resource requirements are achieved.

Description

Technical Field

[0001] This application relates to the field of cloud computing resource management technology, and in particular to a method, system, device and storage medium for dynamic scheduling of cloud resources. Background Technology

[0002] As a core support for digital transformation, cloud computing's resource scheduling capabilities directly determine the balance between service stability, resource utilization, and operating costs. The core objective of cloud resource scheduling is to dynamically allocate resources such as CPU, memory, disk, and network based on business load, achieving efficient resource utilization and cost optimization while meeting Service Level Agreements (SLAs).

[0003] With the diversification of cloud service scenarios, user load exhibits complex characteristics of both short-term sudden fluctuations and long-term cyclical trends. Traditional cloud resource scheduling solutions are gradually revealing their limitations in adapting to these changes. On the one hand, existing prediction models mostly employ a single LSTM or RNN structure, lacking a dynamic feature adjustment mechanism. This makes them unable to accurately capture multi-scale load change patterns, and the commonly used mean squared error (MSE) loss function is sensitive to outliers such as sudden load spikes, resulting in insufficient prediction robustness and a tendency for under- or over-allocation of resources. On the other hand, traditional solutions lack effective constraint mechanisms for resource over-allocation, leading to significant resource idleness and waste. Furthermore, they fail to balance economic constraints with SLA requirements, making it difficult to achieve dual optimization of cost and service quality. In addition, static feature selection strategies limit model adaptability, preventing dynamic adjustment of feature weights based on load fluctuations, further reducing prediction accuracy in complex scenarios. These problems make it difficult for existing scheduling solutions to accurately predict and efficiently allocate resource demands when facing dynamic and complex cloud loads, hindering the refined operation and cost optimization of cloud computing. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a method, system, device, and storage medium for dynamic scheduling of cloud resources, including the following:

[0005] Firstly, this application provides a method for dynamic scheduling of cloud resources, the method comprising:

[0006] Obtain real-time data from the cloud management system, including at least CPU resource usage data, memory resource usage data, disk storage resource usage data, network resource usage data, application resource usage data, and virtualization resource usage data;

[0007] After the real-time data is preprocessed, it is input into a pre-trained resource scheduling prediction model, which is a dual long short-term memory LSTM neural network model that integrates a dynamic feature selection mechanism.

[0008] The resource scheduling prediction model is used to perform time series analysis on the preprocessed real-time data and output the prediction results of cloud resource demand.

[0009] The prediction results are input into the benefit function, and the resource allocation benefits are calculated in combination with the current resource allocation status of the cloud management system. The dynamic scheduling scheme of cloud resources is obtained by solving the optimization algorithm.

[0010] Optionally, the step of performing time series analysis on the preprocessed real-time data using the resource scheduling prediction model includes:

[0011] After the input features are dynamically selected and linearly transformed by a dynamic feature selector, the feature weights are calculated and the features are weighted using the sigmoid activation function.

[0012] The weighted features are fed into an LSTM unit, which includes a short-term LSTM and a long-term LSTM, used to capture short-term load fluctuations and long-term trends, respectively.

[0013] The output results of LSTM are fused by a fusion layer. The fusion layer determines the weight ratio of short-term LSTM output and long-term LSTM output based on the current load volatility. The weight coefficient of short-term LSTM output is α, and the weight coefficient of long-term LSTM output is 1-α. After weighted fusion, the prediction result of cloud resource demand is obtained.

[0014] Optionally, the resource scheduling prediction model is trained based on a combined loss function, which includes a Huber loss function term, a resource excess penalty loss function term, and an L2 regularization term. When outputting cloud resource demand prediction results, the resource scheduling prediction model constrains the prediction behavior through the design logic of the combined loss function: prioritizing ensuring that the deviation between the predicted value and the actual value is within a preset threshold to avoid extreme errors; and suppressing over-configuration where the predicted value exceeds the actual value to balance prediction accuracy and resource utilization efficiency.

[0015] Optionally, inputting the prediction result into the benefit function and calculating the resource allocation benefit in conjunction with the current resource allocation status of the cloud management system includes:

[0016] Construct an input parameter set, which includes a predicted resource demand vector, a current configuration vector, an SLA contract vector, and a price sensitivity vector;

[0017] Substitute the input parameter set into the benefit function formula to calculate the comprehensive benefit value. The calculation formula is as follows: Where β is the economic weighting coefficient, γ is the service quality weighting coefficient, and C actual C represents the actual cost consumed. maxFor the maximum permissible cost, the SLA is the service level agreement satisfaction level, and R... min For minimum acceptable resources, R alloc To allocate resources;

[0018] Based on the comprehensive benefit value, a three-dimensional benefit vector is generated and output. The three-dimensional benefit vector includes economic benefit score, performance guarantee score, and resilience score.

[0019] Optionally, the optimization algorithm is a search direction optimization algorithm, and solving the dynamic scheduling scheme for cloud resources using the search direction optimization algorithm includes:

[0020] The initial search direction is set as multiple preset ratio values ​​of the prediction results, including the downward ratio, the baseline ratio, and the upward ratio of the prediction results.

[0021] Iteratively calculate the resource allocation efficiency for each search direction, select the direction with the highest efficiency as the current optimal direction, and then apply the formula new... d =d+0.5×(d[best idx ]-d) Update the search direction; where new d The updated search direction is d, where d is the original search direction, and best is the best search direction. idx The index for the optimal direction;

[0022] If the difference between the updated search direction and the original direction is less than the threshold, or if the number of iterations reaches the preset number of iterations, then the iteration stops and a dynamic scheduling scheme for cloud resources is output.

[0023] Optionally, the training process of the resource scheduling prediction model includes:

[0024] After acquiring multi-dimensional historical resource data from the cloud management system and completing the structured storage of the data according to timestamps or time windows, preprocessing operations such as time window formatting and sliding window Z-score standardization are performed.

[0025] The Huber loss function and resource excess penalty term are introduced. First, the Huber loss function and the resource excess penalty loss function are constructed separately. Then, the combined loss function containing the Huber loss function term, the resource excess penalty loss function term and the L2 regularization term are constructed by combining the L2 regularization term. The loss calculation threshold of the Huber loss function is determined based on the standard deviation of historical resource data.

[0026] The preprocessed historical resource data is input into the LSTM model, and the loss deviation between the predicted value and the true value is calculated iteratively through the combined loss function. When the predicted value output by the LSTM is less than the true value, the gradient of the Huber loss function dominates the loss calculation. When the predicted value output by the LSTM is greater than the true value, the gradient of the resource excess penalty loss function and the gradient of the Huber loss function are superimposed to calculate the loss. When the prediction error exceeds the loss calculation threshold, the gradient magnitude of the loss calculation is constrained by the linear gradient of the Huber loss function.

[0027] Based on the loss calculation results, backpropagation is used to optimize the model parameters, and the model is continuously trained until it converges, resulting in a well-trained resource scheduling prediction model.

[0028] Secondly, this application provides a cloud resource dynamic scheduling system, which includes:

[0029] The acquisition unit is used to acquire real-time data of the cloud management system. The real-time data includes at least CPU resource usage data, memory resource usage data, disk storage resource usage data, network resource usage data, application resource usage data, and virtualization resource usage data.

[0030] The first data processing unit is used to preprocess the real-time data and then input it into a pre-trained resource scheduling prediction model. The resource scheduling prediction model is a dual long short-term memory LSTM neural network model that integrates a dynamic feature selection mechanism.

[0031] The second data processing unit is used to perform time series analysis on the preprocessed real-time data through the resource scheduling prediction model and output the prediction results of cloud resource demand.

[0032] The resource scheduling optimization unit is used to input the prediction results into the benefit function, calculate the resource allocation benefits in combination with the current resource configuration status of the cloud management system, and obtain the dynamic scheduling scheme of cloud resources through the optimization algorithm.

[0033] Optionally, the second data processing unit is specifically used to perform dynamic feature selection and linear transformation on the input features through a dynamic feature selector, and then calculate the feature weights and complete the feature weighting using the sigmoid activation function.

[0034] The weighted features are fed into an LSTM unit, which includes a short-term LSTM and a long-term LSTM, used to capture short-term load fluctuations and long-term trends, respectively.

[0035] The output results of LSTM are fused by a fusion layer. The fusion layer determines the weight ratio of short-term LSTM output and long-term LSTM output based on the current load volatility. The weight coefficient of short-term LSTM output is α, and the weight coefficient of long-term LSTM output is 1-α. After weighted fusion, the prediction result of cloud resource demand is obtained.

[0036] Optionally, the resource scheduling prediction model is trained based on a combined loss function, which includes a Huber loss function term, a resource excess penalty loss function term, and an L2 regularization term. When outputting cloud resource demand prediction results, the resource scheduling prediction model constrains the prediction behavior through the design logic of the combined loss function: prioritizing ensuring that the deviation between the predicted value and the actual value is within a preset threshold to avoid extreme errors; and suppressing over-configuration where the predicted value exceeds the actual value to balance prediction accuracy and resource utilization efficiency.

[0037] Optionally, the resource scheduling optimization unit is specifically used to construct an input parameter set, which includes a predicted resource demand vector, a current configuration vector, an SLA contract vector, and a price sensitivity vector.

[0038] Substitute the input parameter set into the benefit function formula to calculate the comprehensive benefit value. The calculation formula is as follows: Where β is the economic weighting coefficient, γ is the service quality weighting coefficient, and C actual C represents the actual cost consumed. max For the maximum permissible cost, the SLA is the service level agreement satisfaction level, and R... min For minimum acceptable resources, R alloc To allocate resources;

[0039] Based on the comprehensive benefit value, a three-dimensional benefit vector is generated and output. The three-dimensional benefit vector includes economic benefit score, performance guarantee score, and resilience score.

[0040] Optionally, the optimization algorithm is a search direction optimization algorithm, and the resource scheduling optimization unit solves the cloud resource dynamic scheduling scheme through the search direction optimization algorithm, including:

[0041] The initial search direction is set as multiple preset ratio values ​​of the prediction results, including the downward ratio, the baseline ratio, and the upward ratio of the prediction results.

[0042] Iteratively calculate the resource allocation efficiency for each search direction, select the direction with the highest efficiency as the current optimal direction, and then apply the formula new... d =d+0.5×(d[best idx ]-d) Update the search direction; where new dThe updated search direction is d, where d is the original search direction, and best is the best search direction. idx The index for the optimal direction;

[0043] If the difference between the updated search direction and the original direction is less than the threshold, or if the number of iterations reaches the preset number of iterations, then the iteration stops and a dynamic scheduling scheme for cloud resources is output.

[0044] Optionally, the system further includes a training unit, specifically used for:

[0045] After acquiring multi-dimensional historical resource data from the cloud management system and completing the structured storage of the data according to timestamps or time windows, preprocessing operations such as time window formatting and sliding window Z-score standardization are performed.

[0046] The Huber loss function and resource excess penalty term are introduced. First, the Huber loss function and the resource excess penalty loss function are constructed separately. Then, the combined loss function containing the Huber loss function term, the resource excess penalty loss function term and the L2 regularization term are constructed by combining the L2 regularization term. The loss calculation threshold of the Huber loss function is determined based on the standard deviation of historical resource data.

[0047] The preprocessed historical resource data is input into the LSTM model, and the loss deviation between the predicted value and the true value is calculated iteratively through the combined loss function. When the predicted value output by the LSTM is less than the true value, the gradient of the Huber loss function dominates the loss calculation. When the predicted value output by the LSTM is greater than the true value, the gradient of the resource excess penalty loss function and the gradient of the Huber loss function are superimposed to calculate the loss. When the prediction error exceeds the loss calculation threshold, the gradient magnitude of the loss calculation is constrained by the linear gradient of the Huber loss function.

[0048] Based on the loss calculation results, backpropagation is used to optimize the model parameters, and the model is continuously trained until it converges, resulting in a well-trained resource scheduling prediction model.

[0049] Thirdly, this application provides an apparatus comprising a memory and a processor, the memory for storing instructions or code, and the processor for executing the instructions or code to cause the apparatus to perform the method described in any of the implementations of the first aspect.

[0050] Fourthly, this application provides a computer-readable storage medium storing code, wherein when the code is executed, a device executing the code implements the method described in any of the implementations of the first aspect.

[0051] This application provides a method for dynamic scheduling of cloud resources. When executing the method, firstly, real-time data from the cloud management system is acquired. This real-time data includes at least CPU resource usage data, memory resource usage data, disk storage resource usage data, network resource usage data, application resource usage data, and virtualization resource usage data. Then, the real-time data is preprocessed and input into a pre-trained resource scheduling prediction model. This resource scheduling prediction model is a dual Long Short-Term Memory (LSTM) neural network model incorporating a dynamic feature selection mechanism. Next, the preprocessed real-time data is subjected to time series analysis by the resource scheduling prediction model, and the predicted cloud resource demand is output. Finally, the predicted result is input into a benefit function, and the resource allocation benefit is calculated in conjunction with the current resource configuration status of the cloud management system. An optimization algorithm is then used to obtain a dynamic cloud resource scheduling scheme.

[0052] In this way, by collecting real-time data on core resources from multiple dimensions, comprehensive and realistic data support is provided for resource demand forecasting, ensuring the integrity of the forecast input. Utilizing an LSTM neural network model with a dynamic feature selection mechanism, the weights of input features can be dynamically adjusted. Simultaneously, the LSTM structure captures both short-term fluctuations and long-term trends in resource demand, making time series analysis more closely aligned with the multi-scale characteristics of the load and effectively improving the accuracy of resource demand forecasting. By integrating the forecast results with the current resource allocation status through a payoff function, a holistic consideration of economic constraints and service quality requirements is achieved. Combined with iterative optimization algorithms, the resource scheduling scheme can balance multiple objectives. Furthermore, the model design inherently possesses adaptability to complex load changes, reducing the deviation between resource allocation and actual demand. Thus, accurate forecasting and efficient dynamic allocation of cloud resource demand can be achieved, ensuring that resource allocation meets Service Level Agreement (SLA) requirements while avoiding under- or over-allocation, improving resource utilization, reducing operating costs, ensuring stable operation of the cloud management system under different load conditions, and adapting to diverse cloud business scenarios. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings used in the description of the embodiment or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 A flowchart illustrating a dynamic cloud resource scheduling method provided in this application embodiment;

[0055] Figure 2A flowchart of an LSTM neural network and dynamic feature selection process provided in this application embodiment;

[0056] Figure 3 A flowchart illustrating the processing of an LSTM unit provided in an embodiment of this application;

[0057] Figure 4 This is a schematic diagram of the structure of a cloud resource dynamic scheduling system provided in an embodiment of this application. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0059] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0060] Figure 1 This is a flowchart illustrating a dynamic cloud resource scheduling method provided in an embodiment of this application. (Combined with...) Figure 1 As shown, the cloud resource dynamic scheduling method provided in this application embodiment may include:

[0061] S101. Obtain real-time data from the cloud management system, wherein the real-time data includes at least CPU resource usage data, memory resource usage data, disk storage resource usage data, network resource usage data, application resource usage data, and virtualization resource usage data.

[0062] Among them, the cloud management system refers to the core platform used to centrally manage, monitor and schedule various resources in the cloud computing environment. It can integrate computing, storage, network and other resources and achieve unified management and control. The integrity and accuracy of its real-time data directly affect the effect of subsequent resource scheduling.

[0063] The real-time data covers at least six categories of core resource usage data, specifically as follows: CPU resource usage data includes the total CPU utilization rate per unit time, the utilization rate of each CPU core, the system load within a specific time window, the CPU idle time ratio per unit time, the number of context switches per unit time, and the number of interrupt requests processed by the CPU per unit time. This data comprehensively reflects the CPU's operating status and load intensity; memory resource usage data includes the usage of all physical memory in the system. This includes total memory usage, the amount of memory used by the system (used memory), the amount of available memory that the system can allocate (available memory), the memory usage in the swap partition (memory swap ratio), the amount of memory in the system cache (memory cache size), and the number of memory paging operations (memory paging count), which can accurately reflect the occupation and idle status of memory resources. Disk storage resource usage data involves the total capacity of the hard drive or storage device (total disk capacity), the amount of disk space used (used disk space), the amount of unused disk space (remaining disk space), and the disk I / O usage (disk I / O utilization rate, usually expressed as read / write operation rate, the number of disk read / write operations per unit time). Disk read / write operations (DLO) and the average latency of disk I / O requests (DIR) effectively reflect disk storage utilization efficiency and remaining capacity. Network resource usage data includes total network traffic per unit time, upload traffic per unit time, download traffic per unit time, average data transmission latency, packet loss rate, and network bandwidth utilization as a percentage, providing a comprehensive understanding of network transmission quality and load. Application resource usage data includes the percentage of CPU resources used by a single application. The data includes resource usage, memory size of a single application (application memory usage), disk storage space of a single application (application disk usage), and network traffic generated by a single application (application network traffic), providing a targeted understanding of each application's resource consumption. Virtualization resource usage data includes CPU usage of each virtual machine (virtual machine CPU utilization), memory size of each virtual machine (virtual machine memory usage), disk usage of each virtual machine (virtual machine disk usage), network traffic of the virtual machine (virtual machine network bandwidth), and the number of disk and network I / O operations of the virtual machine (virtual machine I / O operations), adapting to the resource management needs of virtualized cloud environments.

[0064] In addition to the core resource data mentioned above, the real-time data also includes timestamp and time window data, system performance and health status data, user behavior and usage pattern data, and other related resource data. The timestamp refers to the point in time for each data record, and the time window is the period of data collection, such as by hour, day, or week, providing time dimension support for subsequent time series analysis. System performance and health status data includes the overall system load, encompassing CPU, memory, disk, and network loads; system crash logs recording system crashes or anomalies; and device health status data, such as hard drive health status, temperature monitoring, and fan speed, providing system-level feasibility constraints for resource scheduling. User behavior and usage pattern data includes user login and logout times, user activity frequency and activity level, and the frequency of user resource requests, enabling resource scheduling to better align with actual user needs. Other relevant resource data includes GPU usage in the system (GPU utilization rate if the system has GPU resources), the number of database connections and their growth trend, and, for systems using Storage Area Networks (SANs), SAN usage, expanding the adaptability of this method to specific resource scenarios.

[0065] The multi-dimensional data acquired during the above process comprehensively reflects the operational status and resource demand characteristics of the cloud management system from different perspectives, providing sufficient feature support for the dynamic feature selection and multi-scale prediction of the subsequent LSTM model, ensuring that the model accurately captures short-term load fluctuations and long-term trends. Table 1 selects core and representative indicators for structured storage. These indicators are key factors affecting resource scheduling decisions, satisfying the core needs of model training and prediction while avoiding storage and computational pressure caused by data redundancy, achieving a balance between data usability and processing efficiency. Furthermore, all the above real-time data are collected according to timestamps or time windows and stored in a database or log system. The data can be presented in the form of a data table, as shown in Table 1 below. Table 1 is a record table of multi-dimensional resource real-time data of the cloud management system. Through this data table, the subsequent LSTM model can easily process and learn the resource demand change patterns at different points in time, providing a solid data foundation for accurate resource prediction and optimized configuration schemes, effectively avoiding prediction bias caused by missing data dimensions or non-standard formats, thereby improving the rationality and efficiency of cloud resource scheduling.

[0066] Table 1: Real-time Data Recording Table of Multi-dimensional Resources in Cloud Management System

[0067]

[0068] Using the data in Table 1, the LSTM model can process and learn resource demand changes at different points in time, providing users with more accurate resource prediction and optimization schemes, and providing a well-structured input data source for subsequent preprocessing operations. Through this structured data table, subsequent preprocessing and model training can easily extract the patterns of resource demand changes at different points in time, effectively avoiding prediction biases caused by missing data dimensions or non-standard formats.

[0069] S102. After preprocessing the real-time data, it is input into a pre-trained resource scheduling prediction model, which is a dual long short-term memory LSTM neural network model that integrates a dynamic feature selection mechanism.

[0070] After data acquisition, the data preprocessing stage begins. In this embodiment, preprocessing operations include time window data formatting and sliding window Z-score normalization, where the sliding window size is set to 24 hours. The core purpose of time window data formatting is to convert continuous time-series resource data into an input format that the resource scheduling prediction model can directly process. This is specifically achieved by constructing an input feature matrix, the expression of which is: The meanings of each parameter are as follows: X t The input feature matrix represents the current time point t; CPU%(tn) represents the CPU utilization rate n times before the current time point t (n ranges from 0 to 15); Mem%(tn) represents the memory utilization rate n times before the current time point t; DiskIO(tn) represents the disk I / O data n times before the current time point t; "t-15" to "t" represent the time series span, that is, integrating resource data from 15 historical time points to the current time point to form a complete time series feature input, ensuring that the model can capture the resource change patterns in a short period of time.

[0071] Sliding window Z-score standardization addresses the dimensional differences in various resource metrics (such as CPU utilization as a percentage and memory usage in GB). It calculates the mean (μ) and standard deviation (σ) of the data within each 24-hour sliding window and transforms the original data into standardized data using the formula Z=(x-μ) / σ. This operation effectively eliminates the interference of dimensional differences on model training, ensuring that different types of resource data are on the same numerical scale. This improves the stability of model training and accelerates the convergence of model parameters, laying the foundation for improved prediction accuracy.

[0072] The preprocessed feature data will be input into a pre-trained resource scheduling prediction model, which is a dual long short-term memory LSTM neural network model that integrates dynamic feature selection mechanism (hereinafter referred to as Dy-LSTM model). Its core is to improve the accuracy and adaptability of resource demand prediction by combining dynamic feature selection with multi-scale prediction structure.

[0073] S103. Perform time series analysis on the preprocessed real-time data using the resource scheduling prediction model, and output the prediction results of cloud resource demand.

[0074] Figure 2 This application provides a flowchart of an LSTM neural network and dynamic feature selection process. The resource scheduling prediction model used in this solution is a dual long short-term memory LSTM neural network model that integrates a dynamic feature selection mechanism. This model models and learns from historical time-series data through LSTM units to accurately predict the resource needs of users in the future. At the same time, it innovatively introduces a dynamic feature selection mechanism, which allows the model to automatically adjust the weights of input features according to load changes at different time points, effectively improving prediction accuracy and scene adaptability.

[0075] Specifically, the process of performing time series analysis on preprocessed real-time data using the resource scheduling prediction model is as follows: First, preprocessed multi-dimensional data such as user historical resource usage, current load, and system status are input features into the model. These input features cover time series data of various core resources such as CPU, memory, disk, and network, providing comprehensive support for prediction. Then, the input features enter the Dynamic Feature Selector. First, a linear transformation is performed through the nn.Linear layer to map the features to the dimensions suitable for model training. Then, the dynamic weights of each input feature are calculated using the Sigmoid activation function. The output value of the Sigmoid activation function is between 0 and 1, which can intuitively reflect the importance of each feature. For example, in a sudden load scenario, the weights of features strongly correlated with short-term load, such as CPU utilization and network traffic, will automatically increase. In a stable load scenario, the weights of features such as memory cache size and disk idle rate will be adjusted accordingly. Through this dynamic weighting method, the model can adaptively focus on key features and reduce interference from irrelevant features.

[0076] After feature weighting, the weighted features are fed into an LSTM unit for temporal modeling. The LSTM unit adopts a dual-LSTM parallel structure with multi-scale prediction, setting up a short-term LSTM with 6 time steps and a long-term LSTM with 24 time steps. The short-term LSTM focuses on capturing short-term fluctuations in resource demand, enabling rapid response to scenarios with sudden business requests and instantaneous load peaks. The long-term LSTM is used to extract long-term periodic trends in resource demand, adapting to long-term load variation patterns such as weekdays and holidays, and business peaks and troughs. The two LSTM structures work in parallel to achieve comprehensive capture of load features at different time scales.

[0077] Next, a fusion layer weights and fuses the outputs of the short-term LSTM and the long-term LSTM. The fusion process dynamically adjusts the weight coefficient α based on the current load volatility. The fusion formula is y_pred = α × short_term_out + (1-α) × long_term_out, where α is the weight coefficient of the short-term LSTM output, 1-α is the weight coefficient of the long-term LSTM output, and their sum is 1. Load volatility is calculated by measuring the variance of resource utilization per unit time. Higher volatility indicates more drastic short-term load changes; in this case, a larger value of α indicates a higher weight of the short-term LSTM output in the fusion result, ensuring the model can respond promptly to sudden load spikes. Lower volatility indicates a load that better conforms to long-term cyclical patterns; in this case, a larger value of 1-α indicates that the long-term LSTM output dominates, ensuring the stability of the prediction results.

[0078] The innovative design of this scheme in terms of parallel dual LSTM structure and dynamic weight fusion brings significant technical benefits: the parallel dual LSTM structure can simultaneously mine feature information of short-term fluctuations and long-term trends, avoiding the limitation of a single LSTM structure that cannot handle multi-scale loads; dynamic weight fusion allows the model to adaptively adjust the fusion ratio according to the load characteristics, which is more flexible than fixed weight fusion. Figure 3 The flowchart of an LSTM unit provided in this application embodiment clearly shows the overall computation process of short-term LSTM, long-term LSTM and fusion layer. Through this structure, the model can more comprehensively understand the different characteristics of time series data. In complex and ever-changing cloud load scenarios, it ensures both the prediction sensitivity of sudden loads and the prediction accuracy of long-term loads. The final output cloud resource demand prediction results are more in line with actual business needs, providing accurate and reliable decision-making basis for subsequent resource allocation optimization.

[0079] The resource scheduling prediction model in this embodiment is trained based on a combined loss function, which includes a Huber loss function term, a resource excess penalty loss function term, and an L2 regularization term. When outputting cloud resource demand prediction results, the resource scheduling prediction model constrains the prediction behavior through the design logic of the combined loss function: prioritizing ensuring that the deviation between the predicted value and the actual value is within a preset threshold to avoid extreme errors; and simultaneously suppressing over-configuration where the predicted value exceeds the actual value, balancing prediction accuracy and resource utilization efficiency. For details regarding the specific construction method, parameter definition, and gradient adjustment strategy during the training phase of this combined loss function, please refer to the subsequent description of the model training process in this embodiment.

[0080] S104. Input the prediction result into the benefit function, calculate the resource allocation benefit in combination with the current resource allocation status of the cloud management system, and obtain the cloud resource dynamic scheduling scheme through optimization algorithm.

[0081] The benefit function is used to clarify the optimization objective of resource allocation. Its core is to combine user needs with resource supply to quantify the comprehensive benefits of different resource allocation schemes, providing a quantitative basis for subsequent optimization decisions. Within each time step, the resource allocation of the cloud management system is presented in vector form. Each dimension of the vector corresponds to the allocation of a type of resource, such as the number of CPU cores, memory capacity, and storage IOPS, ensuring a comprehensive consideration of resources across multiple dimensions.

[0082] Constructing an input parameter set is a prerequisite for calculating the efficiency of resource allocation. This input parameter set includes four core vectors: a predicted resource demand vector, a current configuration vector, an SLA contract vector, and a price sensitivity vector. The predicted resource demand vector is [number of CPU cores, GB of memory, storage IOPS], directly inheriting the output of the resource scheduling prediction model described earlier. The current configuration vector records the actual quantity of various resources currently allocated by the cloud management system. The SLA contract vector is a quantitative value of the service level agreement satisfaction, ranging from 0 to 1; a higher value indicates that the service quality better meets the agreed standards. The core of the price sensitivity vector is the user cost preference coefficient β, ranging from 0 to 1, used to reflect the user's emphasis on cost control.

[0083] Substituting the above set of input parameters into the benefit function formula, the comprehensive benefit value can be calculated. The formula for calculating the benefit function is as follows:

[0084] ;

[0085] Where β is the economic weighting coefficient, used to adjust the relative importance of economic efficiency and service quality in the overall benefits. In this scheme, the value is taken as 0.6 or 0.7. When the budget is tight, β can be increased to prioritize cost control, and when service quality is prioritized, β can be decreased to ensure SLA satisfaction; Economic Part , The actual cost incurred is related to resource usage and service consumption. This is the maximum allowable cost, the upper limit of the budget. Lower actual costs mean better economic efficiency, and vice versa. Strive to ensure that actual costs are lower than the maximum cost to ensure efficient resource utilization, reducing costs through energy-saving measures or optimized resource scheduling. (Service quality section) SLA (Service Level Agreement) is a key indicator for measuring service quality, and it takes a value between 0 and 1. It is the minimum acceptable resource, representing the minimum resources required to run the service. Resource allocation is typically related to system configuration or service scale. Proper resource allocation ensures that SLAs meet user expectations and that resource distribution achieves a reasonable balance, especially when resources are limited. The value of can determine the relative importance of SLA and resource allocation. The service quality weighting factor determines the weight of SLA and resource allocation in service quality. In this scheme, the value is taken as 0.8 or 0.7, with the higher value preferred. This indicates that Service Level Agreement (SLA) accounts for a larger share of total benefit, and is lower. This focuses more on the rational allocation of resources. Adjustments are made based on actual needs. If customer experience is of paramount importance, it can be improved. Prioritize service quality; if system resource allocation is a bottleneck, reduce... It places more emphasis on the effective use of resources.

[0086] Based on the calculated comprehensive benefit value, a three-dimensional benefit vector containing economic benefit score, performance guarantee score, and resilience score is generated and output. The three-dimensional vector comprehensively reflects the overall performance of resource allocation from different dimensions, providing multi-dimensional reference for optimization decision-making.

[0087] The optimization algorithm is a search direction optimization algorithm. Solving the dynamic scheduling scheme for cloud resources using this algorithm includes: setting an initial search direction, which is set as multiple preset ratio values ​​of the prediction results, including a downward ratio, a base ratio, and an upward ratio of the prediction results; iteratively calculating the resource allocation benefits corresponding to each search direction; selecting the direction with the highest benefit as the current optimal direction; and applying the formula new... d =d+0.5×(d[best idx ]-d) Update the search direction; where new d The updated search direction is d, where d is the original search direction, and best is the best search direction. idx The index for the optimal direction;

[0088] If the difference between the updated search direction and the original direction is less than the threshold, or if the number of iterations reaches the preset number of iterations, then the iteration stops and a dynamic scheduling scheme for cloud resources is output.

[0089] Specifically, this solution uses a search direction optimization algorithm to find the optimal dynamic scheduling scheme for cloud resources. The core idea of ​​this algorithm is to fine-tune the predicted resource demand value and select the scheme with the best overall benefits from multiple candidate configuration schemes. The specific execution process is as follows: First, the initial search direction is set. Based on the prediction result output by the resource scheduling prediction model, three preset ratio values ​​are set as the initial search direction, which are 80%, 100%, and 120% of the prediction result, corresponding to the downward, benchmark, and upward ratios of the prediction result, to ensure that the initial search range can cover a reasonable range of resource configuration fluctuations.

[0090] After entering the iterative optimization phase, each iteration calculates the resource allocation benefits corresponding to the three search directions. That is, it calculates the comprehensive benefit value for each candidate direction using the aforementioned benefit function, and then selects the direction with the largest comprehensive benefit value as the current optimal direction. According to the formula new... d =d+0.5×(d[best idx ]-d) Update the search direction, where new d The updated search direction is d, where d is the original search direction, and best is the best search direction. idx The index represents the current optimal direction. This update strategy gradually narrows the search range and approaches the optimal solution by moving towards the optimal direction. A dual termination condition is set during the iteration process: if the difference between the updated search direction and the original direction is less than a preset threshold (e.g., 0.001), it indicates that the search direction has stabilized, and further iteration will not significantly improve efficiency; if the number of iterations reaches the preset maximum number of iterations (10 in this scheme), the iteration stops to ensure real-time scheduling. After either termination condition is met, the resource configuration scheme corresponding to the current optimal direction is output, which is the dynamic cloud resource scheduling scheme.

[0091] The specific implementation logic of this search direction optimization algorithm can be embodied as follows: After initializing the search direction vector, the algorithm iteratively calculates the benefits of each direction within the maximum number of iterations, selects the optimal direction, updates the search direction, and determines the convergence state, finally outputting the optimal solution. Its advantages lie in its simple process, fast convergence speed, and ability to quickly adapt to dynamic changes in resource requirements. Simultaneously, multi-directional searching ensures that no optimal configuration scheme is overlooked. By flexibly adjusting the predicted value, current state, and adjustment factor, the algorithm can adapt to the optimization needs of different scenarios, ensuring both optimal overall resource allocation benefits and considering the real-time nature and practicality of scheduling. It effectively balances service quality and cost control, avoiding resource waste and insufficient configuration.

[0092] The above embodiments describe a cloud resource dynamic scheduling method according to this application, which uses a pre-trained resource scheduling prediction model. The training process of the pre-trained resource scheduling prediction model in this application is described below with reference to specific embodiments:

[0093] The training process of the pre-trained resource scheduling prediction model in this application will be described in detail below with reference to specific embodiments. This model is a dual long short-term memory LSTM neural network model that integrates a dynamic feature selection mechanism, and its training process is as follows:

[0094] First, multi-dimensional historical resource data of the cloud management system is acquired. This multi-dimensional historical resource data covers core resource dimensions such as CPU resource usage data, memory resource usage data, disk storage resource usage data, network resource usage data, application resource usage data, and virtualization resource usage data. It also includes auxiliary data such as timestamps, time window data, system performance and health status data, and user behavior and usage pattern data, comprehensively covering various resource-related information during the operation of the cloud management system. This historical resource data is then structured and stored according to timestamps or preset time windows to form a well-organized time-series dataset. Preprocessing operations are then performed on this dataset, specifically including time window formatting and sliding window Z-score normalization. The data preprocessing process is the same as step S102 described above.

[0095] Next, the Huber loss function and a resource excess penalty term are introduced to construct a combined loss function that balances prediction accuracy, robustness to outliers, and resource utilization efficiency. First, the Huber loss function and the resource excess penalty loss function are constructed separately. Then, they are integrated with the L2 regularization term to form the complete combined loss function. The Huber loss function is a commonly used error metric in regression problems. Its core advantage lies in its ability to balance squared loss under small error scenarios and linear loss under large error scenarios, exhibiting strong robustness to outliers in the data and preventing model instability due to sudden loads or other outliers. Its mathematical expression is:

[0096] ;

[0097] In the above formula, y i The actual resource utilization rate is obtained from real-time resource metrics data collected by the open-source system monitoring and time-series data acquisition tool Prometheus, which can truly reflect the workload of the cloud management system. δ represents the model's predicted value, i.e., the future resource demand prediction result output by the aforementioned Dy-LSTM model; δ is the error threshold, determined based on historical data analysis. Statistical analysis shows that 85% of resource fluctuations are within ±1.2σ (σ is the standard deviation of historical data), therefore, in this embodiment, δ is set to 1.2σ. When the prediction error... When the Huber loss function uses mean squared error (L2 loss), it can obtain a smooth gradient, helping the model to more accurately fit the data distribution under normal load; when the prediction error When the function is switched to absolute error (L1 loss), the model training divergence caused by sudden loads and other outliers is avoided by limiting the gradient magnitude, which significantly enhances the model's resistance to extreme errors.

[0098] The resource over-allocation penalty loss function is specifically designed to constrain resource over-allocation behavior and avoid resource waste caused by predicted values ​​exceeding actual needs. Its mathematical expression is:

[0099] ;

[0100] In the formula, λ is the penalty coefficient, used to control the intensity of the over-configuration penalty, and its value is λ. base ×C unit The specific explanation is as follows:

[0101] λ base The base penalty coefficient is determined based on the results of multiple A / B tests, with different λ values ​​used in the test. base The value is a variable. Under the same model structure, data preprocessing process, and evaluation criteria, the impact of different values ​​on the reduction of prediction accuracy loss and resource waste is compared. Ultimately, it is determined that when λbase is 0.25, resource waste can be reduced while ensuring that the prediction accuracy loss is less than 3%, achieving a balance between penalty severity and business needs; C unit The resource unit price coefficient is set according to the cost differences of different resources. The values ​​are 0.8, 1.0 and 1.2 for CPU, memory and storage respectively, which impose stronger over-configuration constraints on high-cost resources and further optimize the cost control effect.

[0102] This penalty item was passed. The function implements conditional activation, only when the predicted value... Greater than the actual value y i Penalties are only incurred at certain times, effectively preventing the suppression of reasonable expansion needs and ensuring that the model controls waste without affecting service quality.

[0103] Based on the two loss functions mentioned above, this embodiment further incorporates an L2 regularization term to construct a complete combined loss function, the mathematical expression of which is:

[0104] ;in, The Huber loss component is used to accurately measure the error between the predicted value and the actual value, ensuring the prediction accuracy under normal and abnormal loads. The portion addressing resource excess penalties specifically targets and constrains over-allocation behavior, aiming to reduce resource waste and operating costs. If the predicted value... Exceeding actual demand y i This will result in a penalty. λ is the penalty coefficient, which controls the impact of this part on the total loss. ϵ‖θ‖² is the L2 regularization term, where θ represents all trainable parameters of the model, and ϵ is the regularization coefficient. It is used to penalize the L2 norm of the model parameters, effectively preventing overfitting due to excessive parameter complexity and improving the model's generalization ability on new data.

[0105] During model training, this combined loss function dynamically adjusts the gradient update strategy based on the relationship between the predicted and true values: when the predicted value output by the model is less than the true value, the gradient of the Huber loss function dominates the model parameter update, helping the model quickly improve prediction accuracy through smooth gradients and ensuring that resource allocation meets actual needs; when the predicted value is greater than the true value, the gradient of the resource excess penalty loss function is superimposed with the gradient of the Huber loss function to form a stronger gradient signal, pushing the model parameters to adjust in the direction of reducing over-prediction and effectively suppressing resource waste; when the prediction error exceeds a preset threshold δ, the linear gradient of the Huber loss function comes into play, limiting the excessive growth of the gradient magnitude and preventing the model from experiencing gradient explosion or divergence due to extreme errors, thus ensuring the stability of the training process.

[0106] In practical applications, the core role of this loss function is indirectly reflected through the trained and fixed model parameters. The resource scheduling prediction model deployed after training incorporates the design goals of the combined loss function (resistance to outliers and prevention of over-provisioning) into its prediction logic: When facing normal load scenarios, the model outputs predictions with minimal deviation from actual demand, aligning with the Huber loss's accurate fitting characteristic for small errors; when encountering abnormal situations such as sudden load spikes, the model avoids significant deviations in predictions, demonstrating the Huber loss's ability to suppress extreme errors; when handling resource demand predictions, the model naturally avoids situations where predictions significantly exceed actual demand, fulfilling the constraint objective of the resource over-provisioning penalty term and reducing waste caused by resource over-provisioning from the source.

[0107] Subsequently, the preprocessed historical resource data is input into the LSTM model to initiate the model training process. During training, the loss deviation between the model's predicted value and the true value is iteratively calculated using the aforementioned combined loss function. The true value is real-time resource indicator data collected by the open-source system monitoring and time-series data acquisition tool Prometheus, which accurately reflects the actual workload of the cloud management system. Based on the relationship between the predicted and true values, the loss calculation method is dynamically adjusted: when the predicted value output by the LSTM is less than the true value, the gradient of the Huber loss function dominates the loss calculation, smoothing the gradient to help the model quickly improve prediction accuracy and ensure that the trained model can accurately capture resource demand gaps; when the predicted value is greater than the true value, the gradient of the resource excess penalty loss function is superimposed with the gradient of the Huber loss function to calculate the loss, forming a stronger gradient signal that drives the model parameters to adjust towards reducing over-prediction, thus suppressing resource waste at its source; when the prediction error exceeds the preset loss calculation threshold δ, the linear gradient of the Huber loss function is used to constrain the gradient magnitude of the loss calculation, preventing gradient explosion or divergence due to extreme errors and ensuring the stability of the training process.

[0108] Finally, based on the loss calculation results of each iteration, the trainable parameters of the model are optimized using the backpropagation algorithm, including the weight coefficients of the dynamic feature selector, the hidden layer weights and bias terms of the dual LSTM units, etc. Training continues iteratively until the model converges. The convergence criterion is that the combined loss function value tends to stabilize within a preset number of consecutive rounds and no longer decreases significantly. At this point, the trained resource scheduling prediction model is obtained. This training process, through scientific data preprocessing, customized combined loss function design, and dynamic gradient adjustment strategies, ensures that the trained model has both high accuracy in predicting resource demand and effectively avoids over-allocation of resources. It also exhibits strong adaptability to abnormal load scenarios, providing reliable support for the accurate generation of subsequent dynamic cloud resource scheduling solutions.

[0109] The above are some specific implementations of a cloud resource dynamic scheduling method provided in the embodiments of this application. Based on this, this application also provides a corresponding system. The system provided in the embodiments of this application will be described below from the perspective of functional modularization.

[0110] Figure 4 This is a schematic diagram of the structure of a cloud resource dynamic scheduling system provided in an embodiment of this application. (Combined with...) Figure 4 As shown in the embodiment of this application, the cloud resource dynamic scheduling system 400 includes:

[0111] The acquisition unit 410 is used to acquire real-time data of the cloud management system. The real-time data includes at least CPU resource usage data, memory resource usage data, disk storage resource usage data, network resource usage data, application resource usage data, and virtualization resource usage data.

[0112] The first data processing unit 420 is used to preprocess the real-time data and then input it into a pre-trained resource scheduling prediction model. The resource scheduling prediction model is a dual long short-term memory LSTM neural network model that integrates a dynamic feature selection mechanism.

[0113] The second data processing unit 430 is used to perform time series analysis on the preprocessed real-time data through the resource scheduling prediction model and output the prediction results of cloud resource demand.

[0114] The resource scheduling optimization unit 440 is used to input the prediction result into the benefit function, calculate the resource allocation benefit in combination with the current resource configuration status of the cloud management system, and obtain the dynamic scheduling scheme of cloud resources through the optimization algorithm.

[0115] In one implementation of this application, the second data processing unit is specifically used to perform dynamic feature selection and linear transformation processing on the input features through a dynamic feature selector, and then calculate the feature weights using the sigmoid activation function and complete the feature weighting.

[0116] The weighted features are fed into an LSTM unit, which includes a short-term LSTM and a long-term LSTM, used to capture short-term load fluctuations and long-term trends, respectively.

[0117] The output results of LSTM are fused by a fusion layer. The fusion layer determines the weight ratio of short-term LSTM output and long-term LSTM output based on the current load volatility. The weight coefficient of short-term LSTM output is α, and the weight coefficient of long-term LSTM output is 1-α. After weighted fusion, the prediction result of cloud resource demand is obtained.

[0118] In one implementation of this application, the resource scheduling prediction model is trained based on a combined loss function, which includes a Huber loss function term, a resource excess penalty loss function term, and an L2 regularization term. When outputting cloud resource demand prediction results, the resource scheduling prediction model constrains the prediction behavior through the design logic of the combined loss function: prioritizing ensuring that the deviation between the predicted value and the actual value is within a preset threshold to avoid extreme errors; and suppressing over-configuration where the predicted value exceeds the actual value to balance prediction accuracy and resource utilization efficiency.

[0119] In one implementation of this application, the resource scheduling optimization unit is specifically used to construct an input parameter set, which includes a predicted resource demand vector, a current configuration vector, an SLA contract vector, and a price sensitivity vector.

[0120] Substitute the input parameter set into the benefit function formula to calculate the comprehensive benefit value. The calculation formula is as follows: Where β is the economic weighting coefficient, γ is the service quality weighting coefficient, and C actual C represents the actual cost consumed. max For the maximum permissible cost, the SLA is the service level agreement satisfaction level, and R... min For minimum acceptable resources, R alloc To allocate resources;

[0121] Based on the comprehensive benefit value, a three-dimensional benefit vector is generated and output. The three-dimensional benefit vector includes economic benefit score, performance guarantee score, and resilience score.

[0122] In one implementation of this application, the optimization algorithm is a search direction optimization algorithm, and the resource scheduling optimization unit solves the cloud resource dynamic scheduling scheme using the search direction optimization algorithm, including:

[0123] The initial search direction is set as multiple preset ratio values ​​of the prediction results, including the downward ratio, the baseline ratio, and the upward ratio of the prediction results.

[0124] Iteratively calculate the resource allocation efficiency for each search direction, select the direction with the highest efficiency as the current optimal direction, and then apply the formula new... d =d+0.5×(d[best idx ]-d) Update the search direction; where new d The updated search direction is d, where d is the original search direction, and best is the best search direction. idx The index for the optimal direction;

[0125] If the difference between the updated search direction and the original direction is less than the threshold, or if the number of iterations reaches the preset number of iterations, then the iteration stops and a dynamic scheduling scheme for cloud resources is output.

[0126] In one implementation of this application, the system further includes a training unit, specifically used for:

[0127] After acquiring multi-dimensional historical resource data from the cloud management system and completing the structured storage of the data according to timestamps or time windows, preprocessing operations such as time window formatting and sliding window Z-score standardization are performed.

[0128] The Huber loss function and resource excess penalty term are introduced. First, the Huber loss function and the resource excess penalty loss function are constructed separately. Then, the combined loss function containing the Huber loss function term, the resource excess penalty loss function term and the L2 regularization term are constructed by combining the L2 regularization term. The loss calculation threshold of the Huber loss function is determined based on the standard deviation of historical resource data.

[0129] The preprocessed historical resource data is input into the LSTM model, and the loss deviation between the predicted value and the true value is calculated iteratively through the combined loss function. When the predicted value output by the LSTM is less than the true value, the gradient of the Huber loss function dominates the loss calculation. When the predicted value output by the LSTM is greater than the true value, the gradient of the resource excess penalty loss function and the gradient of the Huber loss function are superimposed to calculate the loss. When the prediction error exceeds the loss calculation threshold, the gradient magnitude of the loss calculation is constrained by the linear gradient of the Huber loss function.

[0130] Based on the loss calculation results, backpropagation is used to optimize the model parameters, and the model is continuously trained until it converges, resulting in a well-trained resource scheduling prediction model.

[0131] This application also provides corresponding devices and computer storage media for implementing the solutions provided in this application.

[0132] The device includes a memory and a processor. The memory stores instructions or code, and the processor executes the instructions or code to cause the device to perform the method described in any embodiment of this application.

[0133] The computer storage medium stores code, and when the code is run, the device running the code implements the method described in any embodiment of this application.

[0134] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus a general-purpose hardware platform. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as a read-only memory (ROM) / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a router) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0135] It is understood that in the specific embodiments of this application, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved need to obtain user permission or consent when the above embodiments of this application are applied to specific products or technologies, and the collection, use and processing of related data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0136] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0137] It should also be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0138] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A cloud resource dynamic scheduling method, characterized in that, The method comprises: obtaining real-time data of the cloud management system, the real-time data comprising at least CPU resource usage data, memory resource usage data, disk storage resource usage data, network resource usage data, application resource usage data and virtualization resource usage data; after preprocessing the real-time data, inputting the preprocessed real-time data into a resource scheduling prediction model pre-trained, the resource scheduling prediction model being a long short-term memory (LSTM) neural network model with a dynamic feature selection mechanism; performing time series analysis on the preprocessed real-time data by using the resource scheduling prediction model, and outputting a prediction result of cloud resource demand; inputting the prediction result into a benefit function, calculating resource configuration benefit in combination with a current resource configuration state of the cloud management system, and solving a cloud resource dynamic scheduling scheme by using an optimization algorithm.

2. The method of claim 1, wherein, The time series analysis on the preprocessed real-time data by using the resource scheduling prediction model comprises: after dynamic feature selection and linear transformation processing of input features by using a dynamic feature selector, calculating feature weights by using a sigmoid activation function and completing feature weighting; inputting the weighted features into an LSTM unit, the LSTM unit comprising a short-term LSTM and a long-term LSTM, which are respectively used for capturing short-term load fluctuation and long-term trend; performing fusion processing on the output result of the LSTM by using a fusion layer, the fusion layer determining the weight proportion of the output of the short-term LSTM and the output of the long-term LSTM according to a current load fluctuation rate, wherein the weight coefficient of the output of the short-term LSTM is α, the weight coefficient of the output of the long-term LSTM is 1-α, and the prediction result of cloud resource demand is obtained after weighted fusion.

3. The method of claim 2, wherein, The resource scheduling prediction model is trained based on a combined loss function, the combined loss function comprising a Huber loss function item, a resource surplus penalty loss function item and an L2 regularization item; when outputting the prediction result of cloud resource demand, the prediction behavior is constrained by the design logic of the combined loss function: the deviation between the prediction value and the true value is preferentially ensured to be within a preset threshold to avoid extreme error; meanwhile, the over-provisioning condition of the prediction value exceeding the true value is inhibited to balance the prediction accuracy and resource utilization efficiency.

4. The method of claim 1, wherein, The input parameter set comprises a predicted resource demand vector, a current configuration vector, an SLA contract vector and a price sensitivity vector; based on the comprehensive benefit value, generating a three-dimensional benefit vector and outputting the three-dimensional benefit vector, the three-dimensional benefit vector comprising an economic benefit score, a performance guarantee score and an elasticity score. The input parameter set is substituted into a benefit function formula to calculate a comprehensive benefit value, and the calculation formula is: wherein β is an economic weight coefficient, γ is a service quality weight coefficient, C actual is an actual consumption cost, C max is a maximum allowable cost, SLA is a service level agreement satisfaction degree, R min is a minimum acceptable resource, R alloc is an allocated resource amount; The optimization algorithm is a search direction optimization algorithm, and solving the cloud resource dynamic scheduling scheme by using the search direction optimization algorithm comprises:

5. The method of claim 1, wherein, setting an initial search direction, the initial search direction being set as a plurality of preset proportion values of the prediction result, comprising a lower floating proportion, a benchmark proportion and an upper floating proportion of the prediction result; ​ Iterative calculation of each search direction corresponding resource configuration benefit, select the benefit of the largest direction as the current optimal direction, and according to the formula new d =d+0.5×(d[best idx ]-d) update search direction; wherein new d The updated search direction, d is the original search direction, best idx The index of the optimal direction; If the difference between the updated search direction and the original direction is less than a threshold value, or the number of iterations reaches a preset number of iterations, the iteration is stopped and the cloud resource dynamic scheduling scheme is output.

6. The method of claim 1, wherein, The training process of the resource scheduling prediction model comprises: After obtaining multi-dimensional historical resource data of a cloud management system and performing structured storage of the data according to a timestamp or a time window, a time window formatting and a sliding window Z-score standardization preprocessing operation is performed; A Huber loss function and a resource surplus penalty term are introduced, a Huber loss function and a resource surplus penalty loss function are respectively constructed, and a combined loss function comprising a Huber loss function term, a resource surplus penalty loss function term and an L2 regularization term is constructed by combining an L2 regularization term, wherein a loss calculation threshold value of the Huber loss function is determined based on a standard deviation of historical resource data; The preprocessed historical resource data is input into an LSTM model, and the loss deviation between the predicted value and the true value is iteratively calculated by the combined loss function; when the predicted value output by the LSTM is less than the true value, the gradient of the Huber loss function dominates the loss calculation; when the predicted value output by the LSTM is greater than the true value, the gradient of the resource surplus penalty loss function and the gradient of the Huber loss function are superimposed to calculate the loss; when the prediction error exceeds the loss calculation threshold value, the linear gradient of the Huber loss function is used to constrain the gradient amplitude of the loss calculation; Based on the loss calculation result, the model parameters are optimized by back propagation, and the training is continued until the model converges, to obtain a trained resource scheduling prediction model.

7. A cloud resource dynamic scheduling system, characterized in that, The system comprises: An acquisition unit configured to acquire real-time data of a cloud management system, the real-time data comprising at least CPU resource usage data, memory resource usage data, disk storage resource usage data, network resource usage data, application resource usage data and virtualization resource usage data; A first data processing unit configured to input the preprocessed real-time data into a pre-trained resource scheduling prediction model after preprocessing the real-time data, the resource scheduling prediction model being a double long short-term memory (LSTM) neural network model fused with a dynamic feature selection mechanism; A second data processing unit configured to perform time series analysis on the preprocessed real-time data by using the resource scheduling prediction model, and output a prediction result of cloud resource demand; A resource scheduling optimization unit configured to input the prediction result into a benefit function, calculate resource configuration benefits in combination with a current resource configuration state of the cloud management system, and obtain a cloud resource dynamic scheduling scheme by solving the optimization algorithm.

8. The system of claim 7, wherein, The system further comprises a training unit configured to: After obtaining multi-dimensional historical resource data of a cloud management system and performing structured storage of the data according to a timestamp or a time window, a time window formatting and a sliding window Z-score standardization preprocessing operation is performed; The Huber loss function and the resource surplus penalty term are introduced, the Huber loss function and the resource surplus penalty loss function are respectively constructed, and then the combination loss function including the Huber loss function term, the resource surplus penalty loss function term and the L2 regularization term is constructed by combining the L2 regularization term, wherein the loss calculation threshold of the Huber loss function is determined based on the standard deviation of the historical resource data; The preprocessed historical resource data is input into the LSTM model, and the loss deviation between the predicted value and the true value is iteratively calculated by the combination loss function; when the predicted value output by the LSTM is less than the true value, the gradient of the Huber loss function dominates the loss calculation; when the predicted value output by the LSTM is greater than the true value, the gradient of the resource surplus penalty loss function and the gradient of the Huber loss function are superimposed to calculate the loss; when the prediction error exceeds the loss calculation threshold, the linear gradient of the Huber loss function is used to constrain the gradient amplitude of the loss calculation; Based on the loss calculation result, the model parameters are optimized by back propagation, and the model is continuously trained until the model converges, so as to obtain the trained resource scheduling prediction model.

9. A computing device, comprising: The computing device comprises a memory and a processor; The memory is used to store a computer program; The processor is used to implement the steps of the method according to any one of claims 1 to 6 when the computer program is executed.

10. A computer-readable storage medium, characterized in that, The computer program is stored on the computer readable storage medium, and the steps of the method according to any one of claims 1 to 6 are implemented when the computer program is executed by the processor.

Images