Cloud computing resource optimization method based on intelligent scheduling

Abstract

The invention discloses a cloud computing resource optimization method based on intelligent scheduling, and belongs to the technical field of cloud computing resource processing. The method comprises the steps of obtaining real-time operation data of target data in a data optimization detection range, collecting historical resource scheduling records and task execution logs, and constructing a multi-dimensional resource state data set; according to the method, multi-objective optimization, simulation verification and reinforcement learning feedback in the step S5 are carried out, a perception-prediction-scheduling-monitoring-optimization closed-loop mechanism is constructed, the resource utilization rate, the response time and the energy consumption cost of a multi-objective optimization function are balanced, and a particle swarm optimization algorithm is combined with simulation verification to generate a global optimal strategy; and reinforcement learning dynamically adjusts model parameters by taking the execution deviation as a reward signal, continuously updates a resource perception dimension and a prediction model, realizes continuous iterative upgrade of a resource optimization effect, and performs optimization processing on cloud computing resource optimization based on intelligent scheduling.

Description

Technical Field

[0001] This invention relates to the field of cloud computing resource processing technology, specifically to a cloud computing resource optimization method based on intelligent scheduling. Background Technology

[0002] Intelligent scheduling refers to the use of artificial intelligence and big data analytics technologies. As a core technology for cloud computing resource management, it integrates AI and big data analytics capabilities to build a resource coordination mechanism with real-time perception and dynamic decision-making capabilities. This enables precise control over various cloud computing resources, such as physical servers, virtual nodes, storage resources, and network bandwidth. Its core objective is to improve resource utilization, reduce energy consumption, and ensure service quality under dynamic load environments. Cloud computing resources constitute the core of modern IT infrastructure, encompassing physical / virtual servers, storage devices, network bandwidth, and various service-oriented components. These resources possess characteristics such as elastic scaling, on-demand provisioning, and distributed deployment, providing dynamic support for diverse applications.

[0003] Traditional cloud computing resource scheduling relies heavily on rule-based or heuristic algorithms, such as genetic algorithms and simulated annealing. While these algorithms can be effective in handling static or simple load scenarios, they are prone to getting stuck in local optima due to over-reliance on pre-defined rules when facing complex dynamic loads, leading to resource allocation imbalances. At the same time, traditional algorithms are slow to respond to changes in resource status and struggle to capture task load fluctuations in real time. This often results in some nodes being overloaded while others are idle, causing a loss of population diversity and a decline in global search capabilities. Consequently, they cannot meet the high elasticity and high reliability requirements of modern cloud computing scenarios.

[0004] To address the aforementioned technical shortcomings, a solution is proposed. Summary of the Invention

[0005] The purpose of this invention is to provide a cloud computing resource optimization method based on intelligent scheduling to solve the problems mentioned above.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a cloud computing resource optimization method based on intelligent scheduling, which determines the data optimization detection range covered by the cloud computing resource cluster, and performs targeted data processing based on this range, including the following steps;

[0007] S1. Obtain real-time running data of target data within the data optimization detection range, and collect historical resource scheduling records and task execution logs to construct a multi-dimensional resource status dataset;

[0008] S2. Obtain the data of the current tasks to be executed and historical tasks, extract features, analyze the key features that are related to each other, and combine time series analysis and trend prediction models to generate a task resource demand prediction dataset for the future preset time period.

[0009] S3. Obtain a multi-dimensional resource status dataset and a task resource demand prediction dataset, construct a resource-task matching model, balance resource utilization, task execution efficiency and energy consumption cost through a multi-objective optimization algorithm, generate an initial scheduling policy signal, and then perform iterative optimization of the policy through simulation verification to obtain the optimal scheduling policy.

[0010] S4. Obtain the initial scheduling policy signal, optimize and generate the optimal scheduling policy, deploy it to the cloud computing resource cluster, monitor the resource allocation, task running status and cloud computing resource cluster load changes in real time during the policy execution process, and obtain the policy execution monitoring dataset.

[0011] S5. Retrieve preset evaluation indicators and strategy execution monitoring datasets for joint analysis to identify deviations and deficiencies in the optimal scheduling strategy. Dynamically adjust the scheduling model parameters through reinforcement learning mechanism, update the multi-dimensional resource status dataset of step S1 and the task resource demand prediction dataset of step S2, and achieve continuous optimization of the scheduling strategy.

[0012] Furthermore, the process of constructing the multi-dimensional resource status dataset in step S1 is as follows:

[0013] Acquire real-time running data of target data within the data optimization detection range, and extract all data of the target data running within the data optimization detection range from the start point to the end point of the latest collection cycle. This includes collecting virtual machine CPU quota utilization, memory allocation utilization, and virtual network bandwidth usage data for virtual nodes, which are marked as time core datasets; collecting storage space utilization, read / write response time, and data redundancy indicators for storage devices, which are marked as time storage datasets; and collecting bandwidth utilization, packet loss rate, and transmission latency data for network links, which are marked as time network datasets.

[0014] Furthermore, the real-time operational data is preprocessed, including the following steps: Step 1: Retrieve the pre-stored 3σ rule, identify outliers in the real-time operational data, fill continuous outliers with the mean of adjacent data, and remove discrete outliers; Step 2: Retrieve the data type standard table, and standardize and differentiate the real-time operational data according to the data type standard table. Percentage data retains its original proportion, rate data is converted to a unit-time average value based on the normalized dimensionless measurement value recorded in the data type standard table, and time data is uniformly converted to millisecond units; Step 3: Align the preprocessed real-time operational data with historical resource scheduling records and historical task execution logs by timestamp to generate a multi-dimensional resource status dataset containing real-time status, historical trends, and resource identifiers.

[0015] Furthermore, the processing procedure for the task resource demand prediction dataset in step S2 is as follows:

[0016] Detailed data of the currently pending task is obtained, and historical task execution logs are retrieved from the multi-dimensional resource status dataset. The two sets of data are used to jointly extract sensitive data features, covering features such as task identification features, time features, initiating user features, and priority. The script of the currently pending task is parsed to obtain the hard requirement data of the number of CPU cores, memory capacity, storage quota, and GPU resources required.

[0017] Furthermore, historical task data is clustered and grouped according to resource demand characteristics, covering the average execution time, resource demand fluctuation range, and failure rate characteristic parameters of different types of tasks, including compute-intensive, memory-intensive, and I / O-intensive tasks. The pre-stored time series model is retrieved, and the historical task data in S1 is used as training samples. Combined with the current distribution characteristics of peak load periods of the cloud computing resource cluster, the number of tasks, total resource demand, and peak occurrence time in different time periods of the next 1 hour, 4 hours, and 24 hours are predicted to generate a task resource demand prediction dataset containing predicted values ​​and confidence intervals.

[0018] Furthermore, the data processing procedure in step S3 is as follows:

[0019] Real-time available resource data is extracted from a multi-dimensional resource status dataset and a resource capability matrix is ​​formed according to type and performance classification. Task demand data is extracted from a task resource demand prediction dataset and converted into a task resource demand vector. A multi-objective optimization function is jointly constructed using the resource capability matrix and the task resource demand vector.

[0020] In the multi-objective optimization function, the core objective is to maximize resource utilization, the constraint is to minimize task response time, and the auxiliary objective is to minimize energy consumption cost. The initial scheduling strategy is generated by matching the resource capability matrix and the task requirement vector through the particle swarm optimization algorithm.

[0021] Furthermore, the pre-stored simulation environment is retrieved, the initial scheduling strategy is imported into the simulation environment for execution, and the indicators of resource conflict rate and task queuing time are statistically analyzed. Based on the simulation results, the weight coefficients of the optimization function are adjusted, multiple candidate strategies are obtained iteratively, and the optimal scheduling strategy is selected through comprehensive scoring.

[0022] Furthermore, the data processing procedure in step S4 is as follows:

[0023] The system obtains the optimal scheduling strategy and extracts the specific resource allocation instructions it contains, including virtual machine creation / migration instructions, storage volume mounting instructions, and network path planning instructions. These instructions are then distributed to the cloud computing resource cluster management nodes in order of priority. The system also collects the instruction execution status in real time through the data optimization detection range of the cloud computing resource cluster, recording the instruction reception time, execution start time, execution completion time, and execution result. Furthermore, it performs high-frequency sampling of the running status of currently pending tasks, recording real-time running data such as task CPU utilization, memory usage changes, and network connection status.

[0024] Furthermore, the collected current task status data and pre-stored task execution data are aggregated, along with cluster macro indicators such as average cloud resource cluster utilization, load balancing, and the number of hot nodes, to form a policy execution monitoring dataset that includes execution progress, task status, and cluster load.

[0025] Furthermore, the data processing procedure in step S5 is as follows:

[0026] The strategy execution monitoring dataset is acquired, and feature data such as resource utilization, task response time, and energy consumption cost are extracted from it. These feature data are then marked as core indicator data. Pre-stored preset evaluation indicators are retrieved and compared with the core indicator data for joint analysis. Deviation data such as cloud computing resource cluster utilization deviation rate, response time exceedance rate, and energy consumption optimization insufficiency rate are obtained. Based on this, the causes of deviation are analyzed: if it is caused by untimely resource status perception, the sampling frequency of the corresponding resource in step S1 is adjusted; if it is caused by inaccurate prediction of task resource demand dataset, the truncation length of historical task data in the prediction model in step S2 is optimized; the deviation data is used as a reward signal for reinforcement learning to update the weight parameters of the resource-task matching model in step S3, and the scheduling strategy generation model is retrained to obtain a strategy update scheme, which dynamically optimizes steps S1 to S3.

[0027] The beneficial effects of this invention are:

[0028] 1. This invention constructs a multi-dimensional resource status dataset to achieve real-time operational data collection of all types of resources, including virtual nodes, storage devices, and network links. Combined with 3σ rule outlier handling, data type standardization, and historical data alignment, it forms a complete data optimization process encompassing real-time status, historical trends, and resource identifiers. This effectively avoids the limitations of traditional methods in terms of certain resource indicators, providing comprehensive and accurate data support for subsequent scheduling strategy generation. It eliminates scheduling imbalances caused by missing resource information, ensures the completeness and timeliness of resource status perception, and achieves full-dimensional resource perception and data integration, solving the problem of the one-sided nature of traditional resource status perception.

[0029] 2. This invention generates a task resource demand prediction dataset containing multiple hours of future demand by clustering and grouping task data and using time series model prediction based on historical task characteristics and cluster load peak patterns. It identifies computationally intensive and memory-intensive task types and assigns differentiated prediction weights to different subtypes to achieve accurate prediction of task demand. Compared with traditional static scheduling, it can detect load fluctuations in advance, reduce resource supply and demand mismatch, improve task execution stability, and effectively alleviate the problem of delayed response to load fluctuations in traditional algorithms. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a flowchart of the method of the present invention;

[0032] Figure 2 This is a sample diagram illustrating the standard table for the data type of this invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Example 1: Please refer to Figure 1 - Figure 2As shown, this embodiment is a cloud computing resource optimization method based on intelligent scheduling. It determines the data optimization detection range covered by the cloud computing resource cluster and performs targeted data processing based on this range, including the following steps;

[0035] S1. Obtain real-time operational data of target data within the data optimization detection range, and collect historical resource scheduling records and task execution logs to construct a multi-dimensional resource status dataset. The construction process of the multi-dimensional resource status dataset is as follows:

[0036] The system acquires real-time operational data of target data within the data optimization detection range. It extracts all data from the start to the end of the latest collection cycle for the target data within the data optimization detection range. This includes data on virtual machine CPU quota utilization, memory allocation utilization, and virtual network bandwidth usage collected from virtual nodes, labeled as the time core dataset; data on storage space utilization, read / write response time, and data redundancy indicators collected from storage devices, labeled as the time storage dataset; and data on network links, including bandwidth utilization, packet loss rate, and transmission latency, labeled as the time network dataset. It should be noted that the real-time operational data sampling frequency is set as follows: CPU utilization and memory usage-related core parameters once per second; disk I / O and network throughput parameters once per 10 seconds.

[0037] Preprocessing of real-time runtime data includes the following steps:

[0038] Step 1: Retrieve the pre-stored 3σ rule to identify outliers in real-time running data. Continuous outliers are filled with the mean of adjacent data, while discrete outliers are removed. It should be noted that the 3σ rule is based on the normal distribution characteristics of the data. The specific steps are as follows: Step 1: Calculate the mean μ and standard deviation σ of the real-time running data, such as CPU utilization and memory usage; Step 2: Set the judgment threshold to μ±3σ, that is, when the data value x satisfies x<μ-3σ or x>μ+3σ, it is judged as an outlier; Step 3: For continuous outliers, such as sudden fluctuations in network throughput, fill them with the mean of the three normal data before and after the data; for discrete outliers, such as invalid values ​​of 0 caused by storage device offline, directly remove the data. For example, if the historical average of physical server CPU utilization is 60% and the standard deviation is 8%, then when a CPU utilization > 84% (60%+3×8%) or < 36% (60%-3×8%) is detected, it is judged as an outlier.

[0039] Step 2: Retrieve the data type standard table and standardize the real-time data according to the table. Percentage data retains its original proportions. Rate data is converted to a unit-time average based on the normalized dimensionless values ​​recorded in the data type standard table. Time data is uniformly converted to millisecond units. It should be noted that the data type standard table is a pre-stored structured data table containing three columns: data type, processing rules, and unit conversion standard. Details can be found as follows... Figure 2 As shown;

[0040] In the third phase, the preprocessed real-time running data is linked and aligned with historical resource scheduling records and historical task execution logs by timestamp to generate a multi-dimensional resource status dataset that includes real-time status, historical trends, and resource identifiers.

[0041] The historical resource scheduling record association processing includes: extracting historical resource operation data from the multi-dimensional resource status dataset for the past 30 days, associating it with the resource scheduling records of the same period according to the scheduling event timestamp, and marking the changes in resource status before and after each scheduling; associating historical task execution logs with historical resource scheduling records according to task ID, statistically analyzing the execution effect of different task types under different resource states, forming a historical association dataset containing resource status-scheduling behavior-task effect. By analyzing the correspondence between high utilization scheduling strategies and task types, the historical association dataset provides data support for the task clustering weight allocation of S2 and the optimization function weight setting of S3, enhancing the coherence and traceability of the data.

[0042] S2. Obtain the current task to be executed and historical task data for feature extraction, analyze the key features that are correlated with each other, and combine time series analysis and trend prediction models to generate a task resource demand prediction dataset for a future preset time period. The processing procedure for the relevant data of the task resource demand prediction dataset is as follows:

[0043] Detailed data of the currently pending tasks is obtained, and historical task execution logs from the multi-dimensional resource status dataset are retrieved. Sensitive data features are extracted by combining the two sets of data, covering features such as task identifier features, time features, initiating user features, and priority. The script of the currently pending task is parsed to obtain the hard requirement data of the number of CPU cores, memory capacity, storage quota, and GPU resources required. It should be noted that the data source is the historical task logs of the past 30 days from the multi-dimensional resource status dataset generated by S1 and the queue of the currently pending tasks.

[0044] Historical task data is clustered and grouped according to resource demand characteristics, covering average execution time, resource demand fluctuation range, and failure rate characteristics of different types of tasks, including compute-intensive, memory-intensive, and I / O-intensive tasks. A pre-stored time series model is retrieved, and historical task data from S1 is used as training samples. Combined with the current distribution characteristics of peak load periods in the cloud computing resource cluster, the number of tasks, total resource demand, and peak occurrence time for different time periods in the next 1 hour, 4 hours, and 24 hours are predicted. This generates a task resource demand prediction dataset containing predicted values ​​and confidence intervals. It should be noted that the prediction model uses an LSTM time series model, the historical task data window is set to 30 days, and the predicted outputs for the number of tasks and total resource demand for 1 hour, 4 hours, and 24 hours are provided, along with ±5% confidence intervals. A peak task demand warning signal is generated, such as for weekdays from 9:00 to 11:00, providing an accurate task resource demand prediction dataset for S3.

[0045] Clustering and grouping processing includes: constructing sensitive data feature vectors based on the type and quantity of resources required by the current task to be executed; and using the K-means clustering algorithm to divide the historical task data in step S1 into computationally intensive, memory-intensive, and I / O-intensive tasks. The K-means clustering algorithm is categorized into computationally intensive and network-intensive types. The elbow rule is used to determine the K value, which can be 5. The clustering results are manually verified and the boundaries adjusted. Tasks are then categorized into different types: computationally intensive (e.g., data modeling) and memory-intensive (e.g., caching service). Examples are as follows: Example 1: Multiple clustering experiments are conducted with K values ​​ranging from 2 to 8. Example 2: The sum of squared distances (SSE) for each K value is calculated, where SSE represents the sum of squared distances from each sample to its cluster center. Example 3: A K-SSE curve is plotted. The K value corresponding to the elbow is the optimal value, and the elbow represents the point where the SSE decrease rate abruptly changes. When K=5, the SSE decreases from 1200 (K=4) to 850, while at K=6, the SSE only decreases to 780, indicating a significantly slower rate of decrease. Therefore, K=5 is determined to be the optimal number of clusters, corresponding to the categories of computationally intensive, memory-intensive, I / O-intensive, network-intensive, and hybrid.

[0046] For each type of task, calculate the mean, standard deviation, maximum and minimum of resource requirements to generate category feature statistics. Combine user priority tags and task time limits to further subdivide similar tasks, identify subtypes of urgent tasks, batch tasks and long-cycle tasks, and assign differentiated prediction weights to different subtypes. This makes the generated task resource requirement prediction dataset more in line with actual business scenarios and improves the accuracy of strategy generation in step S3.

[0047] S3. Obtain a multi-dimensional resource status dataset and a task resource demand prediction dataset, construct a resource-task matching model, balance resource utilization, task execution efficiency and energy consumption cost through a multi-objective optimization algorithm, generate an initial scheduling policy signal, and then perform iterative optimization of the policy through simulation verification to obtain the optimal scheduling policy.

[0048] The data processing procedure in step S3 is as follows:

[0049] Real-time available resource data is extracted from the multi-dimensional resource status dataset and a resource capability matrix is ​​formed according to type and performance classification. Task demand data is extracted from the task resource demand prediction dataset and converted into a task resource demand vector. A multi-objective optimization function is jointly constructed using the resource capability matrix and the task resource demand vector. It should be noted that the resource capability matrix can be constructed by combining the number of idle CPU cores and memory capacity of 8 physical servers with real-time available resource data. Based on the task resource demand prediction dataset of step S2, the task demand for the next 4 hours is extracted to form a demand vector, such as 10 compute-intensive tasks and 5 memory-intensive tasks.

[0050] In the multi-objective optimization function, maximizing resource utilization is the core objective, minimizing task response time is the constraint, and minimizing energy cost is the auxiliary objective. The particle swarm optimization algorithm is used to match and calculate the resource capability matrix and task demand vector to generate an initial scheduling strategy. It should be noted that: the weights for resource utilization, response time, and energy cost in the multi-objective optimization function can be set to 0.4, 0.3, and 0.3 respectively. The initial strategy is generated using the particle swarm optimization algorithm, with a population size of 50, meaning 50 resource-task matching schemes are optimized simultaneously; the maximum number of iterations T can be set to 100; the initial value of the inertia weight ω can be set to 0.9, and it decays linearly to 0.4 with each iteration, using the formula: ω = 0.9 - 0.5 × t / T, where t is the current iteration number; the upper limit of particle velocity Vmax = 0.1 × the maximum value of the resource capability matrix to avoid excessive particle jumping. At the 50th iteration, the inertia weight ω = 0.9 - 0.5 × 50 / 100 = 0.65, with the particle velocity update focusing more on guiding the global optimal solution.

[0051] The simulation environment is built on Docker to create a simulated cloud computing resource cluster with 10 physical machines and 50 virtual nodes. The feasibility threshold is set to 80 points. The strategy with the highest comprehensive score is taken as the optimal output, generating a resource allocation list and scheduling sequence table to provide a solution for the execution of step S4.

[0052] The pre-stored simulation environment is retrieved, the initial scheduling strategy is imported into the simulation environment for execution, and the indicators of resource conflict rate and task queuing time are statistically analyzed. Based on the simulation results, the weight coefficients of the optimization function are adjusted, multiple candidate strategies are obtained iteratively, and the optimal scheduling strategy is selected through comprehensive scoring.

[0053] The simulation verification process includes: building a simulation environment consistent with the actual cloud computing resource cluster configuration based on the multi-dimensional resource status dataset from step S1. The simulation environment includes simulation models of physical servers, virtual nodes, storage devices, and network links, with parameter settings consistent with the actual cloud computing resource cluster; importing multiple candidate scheduling strategies into the simulation model to simulate the task submission and resource allocation process over the next 24 hours, collecting data on the number of resource conflicts, task waiting time, and node overload frequency during the simulation process, and calculating the strategy feasibility score; for strategies with a feasibility score higher than 80 (out of 100), further calculating performance indicators such as cloud computing resource cluster utilization, energy consumption cost, and task completion rate, generating a comprehensive score through weighted summation using the analytic hierarchy process (AHP), and selecting the strategy with the highest comprehensive score as the optimal scheduling strategy to ensure that the strategy executed in step S4 has high feasibility and optimization effect.

[0054] Example 2:

[0055] S4. Obtain the initial scheduling policy signal, optimize and generate the optimal scheduling policy, deploy it to the cloud computing resource cluster, and monitor the resource allocation, task running status, and cloud computing resource cluster load changes in real time during policy execution to obtain the policy execution monitoring dataset. The data processing process is as follows:

[0056] Obtain the optimal scheduling strategy and extract the specific resource allocation instructions contained within it, including virtual machine creation / migration instructions, storage volume mounting instructions, and network path planning instructions, and distribute them to the cloud computing resource cluster management nodes in order of priority;

[0057] By optimizing the detection range of data from the cloud computing resource cluster, the execution status of instructions is collected in real time, and data such as instruction reception time, execution start time, execution completion time and execution result are recorded. The running status of the current task to be executed is sampled at high frequency, and real-time running data such as task CPU utilization, memory usage changes and network connection status are recorded. It should be noted that after the optimal scheduling strategy issues the instruction, the execution status is fed back to the control center in real time through the cloud computing resource cluster event bus.

[0058] The system aggregates the collected current task status data and pre-stored task execution data, along with macro-level indicators of the cloud resource cluster such as average utilization, load balancing, and number of hot nodes. This forms a policy execution monitoring dataset that includes execution progress, task status, and cloud resource cluster load. Note that the task execution status sampling frequency can be once every 5 seconds, and the cloud resource cluster load indicators can be aggregated every minute. The processing results include a policy execution progress bar, task success / failure indicators, and a node load heatmap. Real-time alarm signals for resource overload and task timeout are generated. The policy execution progress bar can show 80% of instructions completed; the task success / failure indicator can show 15 successful tasks and 0 failures; the node load heatmap shows resource overload, which can be indicated by node CPU utilization exceeding 95%; and task timeout can be indicated by task execution exceeding 30 minutes. This forms the policy execution monitoring dataset, providing data for S5 feedback.

[0059] S5. Retrieve preset evaluation indicators and strategy execution monitoring datasets for joint analysis. The preset evaluation indicators are set as follows: resource utilization rate can be ≥85%. Based on historical data statistics, when the resource utilization rate of the cloud computing resource cluster is between 85% and 90%, it can avoid resource idleness and reserve 5%-10% buffer to cope with sudden loads; average task response time can be ≤500ms. Referring to industry SLA standards, the response time of cloud computing tasks should be controlled within 500ms to ensure user experience; energy consumption reduction rate can be ≥10%. Compared with the energy consumption data of traditional scheduling algorithms, intelligent scheduling can achieve more than 10% energy consumption optimization, which is an achievable industry target and meets the requirements of green computing.

[0060] The deviations and shortcomings of the optimal scheduling strategy are identified. The scheduling model parameters are dynamically adjusted using a reinforcement learning mechanism. The multi-dimensional resource status dataset in step S1 and the task resource demand prediction dataset in step S2 are updated to achieve continuous optimization of the scheduling strategy. The data processing procedure is as follows:

[0061] Obtain the policy execution monitoring dataset and extract feature data such as resource utilization, task response time, and energy consumption cost. Mark these as core indicator data. Retrieve pre-stored preset evaluation indicators and perform joint comparative analysis with the core indicator data to obtain deviation data for cloud computing resource cluster utilization deviation rate, response time exceedance rate, and energy consumption optimization insufficiency rate. Analyze the causes of these deviations based on this data.

[0062] If the issue is caused by untimely resource status awareness, adjust the sampling frequency of the corresponding resource in step S1.

[0063] If the task resource requirement prediction dataset is inaccurate, optimize the historical task data truncation length of the prediction model in step S2.

[0064] Using the bias data as a reward signal for reinforcement learning, the weight parameters of the resource-task matching model in step S3 are updated, the scheduling policy generation model is retrained, and a policy update scheme is obtained, which dynamically optimizes steps S1 to S3.

[0065] Combining Examples 1 and 2, a continuous 72-hour test showed that: based on the multi-dimensional resource status dataset of S1 and the task resource demand prediction dataset of S2, the resource utilization rate of the strategy generated by S3 after execution by S4 was significantly improved compared with the traditional method, the average task response time was reduced from 800ms to 420ms, and the energy consumption of the cloud computing resource cluster was effectively controlled and reduced, verifying the data consistency and the effectiveness of the method.

[0066] Multi-objective optimization, simulation verification, and reinforcement learning feedback in the S5 steps construct a closed-loop mechanism of perception-prediction-scheduling-monitoring-optimization. The multi-objective optimization function balances resource utilization, response time, and energy consumption cost. The particle swarm optimization algorithm, combined with simulation verification, generates a globally optimal strategy. Reinforcement learning dynamically adjusts model parameters using execution deviation as a reward signal, continuously updating the resource perception dimension and prediction model. This avoids the shortcomings of traditional heuristic algorithms that are prone to getting trapped in local optima, improves the adaptability of the scheduling strategy to complex dynamic scenarios, and achieves continuous iterative upgrades in resource optimization effects, constituting an optimization process for cloud computing resources based on intelligent scheduling.

[0067] The above description is merely an example and illustration of the structure of the present invention. Those skilled in the art can make various modifications or additions to the specific examples described, or use similar methods to replace them, as long as they do not deviate from the structure of the invention or exceed the scope defined in the claims, and all such modifications and additions should fall within the protection scope of the present invention.

[0068] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0069] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for optimizing cloud computing resources based on intelligent scheduling, characterized in that, Determine the data optimization detection range covered by the cloud computing resource cluster, and perform targeted data processing within this range, Comprise the following steps; S1, obtain the real-time running data of the target data in the data optimization detection range, and collect historical resource scheduling records and task execution logs to construct a multi-dimensional resource state data set; S2, obtain the current to-be-executed task and historical task data for feature extraction, analyze the key features associated between the features, combine time series analysis and trend prediction model to generate task resource demand prediction data set in future preset period; S3, obtain the multi-dimensional resource state data set and the task resource demand prediction data set, construct a resource-task matching model, balance the resource utilization rate, task execution efficiency and energy consumption cost through multi-objective optimization algorithm, generate initial scheduling strategy signal, and then optimize the strategy through simulation verification to obtain the optimal scheduling strategy; S4, obtain the optimal scheduling strategy generated by optimizing the initial scheduling strategy signal, and deploy it to the cloud computing resource cluster, real-time monitor the resource allocation, task running state and cloud computing resource cluster load change during the strategy execution process, and obtain the strategy execution monitoring data set; S5, call the preset evaluation index and the strategy execution monitoring data set for joint analysis, identify the deviation and deficiency of the optimal scheduling strategy, dynamically adjust the scheduling model parameters through reinforcement learning mechanism, update the multi-dimensional resource state data set of S1 step and the task resource demand prediction data set of S2 step, and realize the continuous optimization of the scheduling strategy. 2.The method of claim 1, wherein, The construction process of the multi-dimensional resource state data set in the S1 step is as follows: Obtain the real-time running data of the target data in the data optimization detection range, intercept all the data of the target data in the data optimization detection range from the start point to the end point of the latest collection period, cover the virtual machine CPU quota usage rate, memory allocation utilization rate, and virtual network bandwidth occupation data collected from the virtual node, and mark it as the time core data set, collect the storage space usage rate, read-write response time, and data redundancy index from the storage device, mark it as the time storage data set, and collect the bandwidth utilization rate, data packet loss rate, and transmission delay data from the network link, mark it as the time network data set. 3.The method of claim 2, wherein, The pre-processing of real-time running data includes the following progress: progress one, call the pre-stored 3σ rule to identify abnormal values of real-time running data, fill in continuous abnormal values with adjacent data mean, and eliminate discrete abnormal values; Progress two, call the data type standard table, standardize the real-time running data according to the data type standard table, and distinguish between percentage data and rate data; Progress three, align the pre-processed real-time running data, historical resource scheduling records and historical task execution logs according to the time stamp to generate a multi-dimensional resource state data set containing real-time state, historical trend and resource identifier. 4.The method of claim 1, wherein, The processing process of the task resource demand prediction data set related data in the S2 step is as follows: Obtain detailed data of the current task to be executed, call historical task execution logs in the multi-dimensional resource state data set, and jointly implement sensitive data feature extraction with the two sets of data, covering task identification features, time features, initiating user features, priority, and other features. Analyze the current task script to obtain the hard requirement data of CPU core number, memory capacity, storage quota, and GPU resources. 5.The intelligent scheduling based cloud computing resource optimization method of claim 4, wherein, Cluster the historical task data according to resource demand features, covering average execution time, resource demand fluctuation range, and failure rate feature parameters of different types of tasks such as compute-intensive, memory-intensive, and I / O-intensive. Call the pre-stored time series model, use the historical task data in S1 as training samples, combine the current cloud computing resource cluster load peak period distribution characteristics, and predict the task quantity, resource demand total amount, and peak occurrence time in different time periods such as 1 hour, 4 hours, and 24 hours in the future. Generate a task resource demand prediction data set containing prediction values and confidence intervals.

6. The method of claim 1, wherein, The data processing process in the S3 step is as follows: Extract real-time available resource data from the multi-dimensional resource state data set, form a resource capability matrix according to type and performance classification, extract task demand data from the task resource demand prediction data set, and convert it into a task resource demand vector. Jointly construct a multi-objective optimization function with the resource capability matrix and the task resource demand vector. In the multi-objective optimization function, maximize resource utilization, minimize task response time, and minimize energy cost as auxiliary targets. Use the particle swarm optimization algorithm to match and calculate the resource capability matrix and the task demand vector to generate an initial scheduling strategy. 7.The method of claim 6, wherein, Call the pre-stored simulation environment, import the initial scheduling strategy into the simulation environment for simulation execution, and calculate the resource conflict rate and task queuing time. Based on the simulation results, adjust the optimization function weight coefficients, and iteratively obtain multiple candidate strategies. Select the optimal scheduling strategy through comprehensive scoring. 8.The method of claim 1, wherein, The data processing process in the S4 step is as follows: Obtain the optimal scheduling strategy and extract the specific resource allocation instructions it contains, including virtual machine creation / migration instructions, storage volume mounting instructions, and network path planning instructions. Send them to the cloud computing resource cluster management node in order of priority; Collect the instruction execution status in real time through the data optimization detection range of the cloud computing resource cluster, record the instruction reception time, execution start time, execution completion time, and execution result data, and perform high-frequency sampling on the current task running state. Record real-time running data such as task CPU usage, memory usage changes, and network connection status. 9.The intelligent scheduling based cloud computing resource optimization method of claim 8, wherein, Collect the current task state data collected and the pre-stored task running data, and form a strategy execution monitoring data set containing execution progress, task state, and cluster load based on cloud computing resource cluster usage rate, load balancing degree, and hot node quantity. 10.The method of claim 1, wherein, The data processing process in the S5 step is as follows: The strategy execution monitoring data set is acquired and feature data of resource utilization, task response time and energy consumption cost is extracted therefrom, which is marked as core indicator data, pre-stored preset evaluation indicators are called and compared with the core indicator data to obtain deviation data of cloud computing resource cluster utilization deviation rate, response time exceeding rate and energy consumption optimization deficiency rate, and the deviation causes are analyzed accordingly: if the resource state perception is not timely, the sampling frequency of the corresponding resource in S1 step is adjusted; if the task resource demand prediction data set is not accurate, the historical task data interception length of the prediction model in S2 step is optimized; the deviation data is used as a reward signal for reinforcement learning, the weight parameters of the resource-task matching model in S3 step are updated, the scheduling strategy generation model is retrained, and a strategy updating scheme is obtained, and S1 to S3 steps are dynamically optimized.

Images