Data center resource scheduling optimization method based on artificial intelligence
By using an AI-based hybrid intelligent model, the problems of inaccurate load prediction, weak multi-constraint optimization capabilities, and poor dynamic adaptability in data center resource scheduling are solved, achieving efficient resource utilization and energy consumption optimization, and ensuring the service quality and energy efficiency of the data center.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING ALPHA RISK CONTROL TECH CO LTD
- Filing Date
- 2026-01-09
- Publication Date
- 2026-05-08
AI Technical Summary
Existing technologies in data center resource scheduling suffer from problems such as insufficient load prediction accuracy, weak multi-constraint collaborative optimization capability, poor dynamic adaptability, and an imbalance between migration overhead and scheduling efficiency, resulting in low resource utilization, high SLA default rate, and excessive energy consumption.
By adopting an AI-based hybrid intelligent model, combining multi-dimensional data collection, feature engineering, and hybrid intelligent model construction, and through an improved Transformer network and reinforcement learning algorithm, we can achieve multi-objective collaborative optimization of resource utilization, SLA compliance rate, and energy consumption, thereby improving dynamic adaptability.
It achieves high-precision load forecasting, multi-objective collaborative optimization, strong dynamic adaptability, resource utilization rate increased to over 85%, SLA default rate reduced to below 0.1%, unit business energy consumption reduced by 15%~25%, migration overhead is controllable, and service continuity is guaranteed.
Smart Images
Figure CN121996352A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of cloud computing and data center operation and maintenance technology, and specifically to an artificial intelligence-based data center resource scheduling optimization method. Background Technology
[0002] With the rapid development of cloud computing, big data, and artificial intelligence technologies, data centers are continuously expanding in scale and becoming increasingly complex in business types. The number of virtual resources such as virtual machines / containers is growing exponentially, placing higher demands on the real-time performance, reliability, and energy efficiency of resource scheduling. Traditional data center resource scheduling methods mainly rely on static rule scheduling (such as round-robin scheduling and load threshold-triggered scheduling) or single algorithm scheduling (such as scheduling based on genetic algorithms and particle swarm optimization), which have the following significant drawbacks: Insufficient accuracy in load forecasting: Traditional methods rely on historical statistical data for simple trend forecasting, which cannot effectively capture the non-linear fluctuation characteristics of resource load and the sudden change patterns of business concurrency. This results in scheduling schemes lagging behind actual load changes, which can easily lead to server overload or resource idleness. Weak ability to coordinate optimization across multiple constraints: Data center resource scheduling needs to simultaneously meet resource capacity constraints, SLA (Service Level Agreement) constraints, and energy consumption constraints. However, traditional methods often focus on single-objective optimization (such as pursuing only the maximization of resource utilization), leading to increased SLA default rates or excessive energy consumption, making it difficult to achieve a balance between multiple objectives. Poor dynamic adaptability: When faced with complex scenarios such as heterogeneous server clusters and dynamic adjustment of business priorities, traditional static rules or single algorithms have insufficient generalization ability, cannot dynamically adjust scheduling strategies according to real-time running status, and model updates rely on offline retraining, which cannot adapt to the dynamic changes in data centers. Imbalance between migration overhead and scheduling efficiency: Virtual machine / container migration is an important means of achieving load balancing, but traditional scheduling methods do not fully consider the overhead of network bandwidth occupation and time consumption during the migration process, which can easily lead to business interruption or response delay, affecting service quality. Therefore, there is an urgent need for a resource scheduling optimization method that can accurately predict load changes, collaboratively satisfy multiple constraints, and has dynamic adaptive capabilities, in order to solve problems such as low resource utilization, high SLA default rate, and excessive energy consumption in existing technologies. Summary of the Invention
[0003] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide a data center resource scheduling optimization method based on artificial intelligence.
[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a data center resource scheduling optimization method driven by a hybrid intelligent model based on artificial intelligence. Through multi-dimensional data collection, feature engineering optimization, hybrid intelligent model construction and closed-loop iterative optimization, it achieves multi-objective collaborative optimization of resource utilization, SLA compliance rate and energy consumption, thereby improving the intelligence level and dynamic adaptability of data center resource scheduling. The technical solution of this invention is as follows: A data center resource scheduling optimization method based on artificial intelligence, comprising the following steps: S0. Model Initialization Configuration Import nearly 6 months of historical operational data from the data center (including physical resource data, virtual resource data, business service data, and environmental perception data) in advance to pre-train the hybrid intelligent scheduling model. Set the number of network layers (6-12 layers for Transformer encoder, 3-5 layers for PPO algorithm network hidden layers), the number of neurons in the hidden layer (512-1024), the number of training iterations (1000-5000 times), and the constraint thresholds (such as SLA response time threshold and energy consumption limit) to obtain the initial training model. S1. Multi-dimensional data collection Collect multi-dimensional operational data from the data center to form the raw scheduling dataset, specifically including: Physical resource data: server CPU utilization, memory usage, remaining storage capacity, network bandwidth utilization, and hardware health status parameters (such as fan speed and motherboard temperature). Virtual resource data: number of virtual machines / containers, resource requests (number of CPU cores, memory size, etc.), current load intensity, migration history and lifecycle status (running, paused, destroyed). Business service data: business type (e.g., compute-intensive, storage-intensive, network-intensive), SLA parameters (response time threshold, availability requirements), business priority coefficient (level 1 to 5, with level 1 being the highest), request response time threshold, and business concurrency; Environmental sensing data: temperature in the computer room area, air conditioning operating power, total power consumption, and cooling system energy consumption. S2. Data Preprocessing The preprocessing and feature engineering of the original scheduling dataset involves the following steps: Noise filtering: The median filtering algorithm is used to remove outliers in the data collected by the hardware sensors to avoid extreme data from affecting model training; Missing value completion: For missing time series data, linear interpolation is used to complete the missing data to ensure data continuity; Normalization: Map all feature data to the [0,1] interval to eliminate the influence of dimensional differences on the model; Feature engineering: The mean, variance, and trend slope of resource utilization for the past 10 time slices were extracted using the sliding window method (window size N=10) to generate time series features of resource utilization. By decomposing the load data using wavelet transform, the load fluctuation frequency, amplitude, and abrupt change characteristics are extracted to form load fluctuation features. The service QoS requirements are quantified into response time weight (0.4), availability weight (0.3), and reliability weight (0.3), and a service QoS requirement feature vector is constructed. Based on the mapping relationship between server energy consumption and resource utilization (obtained through polynomial fitting), energy consumption constraint features are generated. S3. Construction and Training of Hybrid Intelligent Scheduling Model A hybrid intelligent model integrating a load prediction sub-model and a reinforcement learning decision-making sub-model is constructed. The specific implementation method is as follows: Load prediction sub-model: An improved Transformer network is adopted, and an attention mechanism optimization module (introducing load fluctuation weight coefficient) is added to the encoder. The input is the time series characteristics of resource utilization and the characteristics of business concurrency, and the output is the resource load prediction value for the next T=5 time slices. This improved Transformer network improves the prediction accuracy under nonlinear fluctuation scenarios by paying attention to the load characteristics of different time slices differently. Reinforcement learning decision sub-model: Employing the proximal policy optimization (PPO) algorithm, defined as: State space: load forecast, real-time resource status (physical / virtual resource utilization), and service priority coefficient; Action space: resource allocation ratio (CPU / memory / storage / bandwidth allocation ratio), virtual machine / container migration path (target node for cross-server migration), server wake-up / sleep control; Reward function: Where U is the resource utilization rate (target ≥ 85%), E is the unit business energy consumption (target ≤ preset threshold), and S is the SLA default rate (target ≤ 0.1%). Model constraints: Resource capacity constraint: Total resources allocated to virtual resources ≤ Maximum available resources of the physical server; SLA constraints: Response time ≤ 500ms, service availability ≥ 99.9%, data transmission reliability ≥ 99.99%; Energy consumption constraints: Total energy consumption of the data center ≤ preset upper limit, energy consumption of a single server ≤ 85% of rated power consumption; Migration overhead constraints: Migration bandwidth usage ≤ 30%, migration time ≤ 500ms; Model training: Using the model input feature set as training data, the load prediction sub-model and the reinforcement learning decision sub-model are trained alternately, and the model parameters are iteratively optimized until the reward function value converges (convergence threshold is ±0.01). S4. Real-time scheduling decision generation and verification The real-time operational status data of the data center (collected every minute) is input into the trained hybrid intelligent model, and combined with the current business priority weights, an optimized scheduling scheme is generated, including: Physical resource allocation scheme: clearly define the number of CPU cores, memory size, storage partitions, and network bandwidth quotas for each virtual machine / container; Virtual resource scheduling scheme: Determine the timing of virtual machine / container creation / destruction, migration path, and load balancing target node; Server control scheme: Develop hibernation / wake-up policies for idle servers and load migration ratio (≥30%) for overloaded servers. Scheduling scheme verification: A data center simulation environment is constructed using digital twin technology to verify the constraint satisfaction of the generated scheduling scheme. If there is a constraint violation (such as exceeding the SLA default rate), the scheduling scheme is regenerated through the action space adjustment mechanism of the PPO algorithm until all constraints are satisfied. S5. Scheduling Execution and Closed-Loop Optimization Scheduling and execution: The validated optimized scheduling scheme is distributed to the data center resource management system to perform resource allocation, virtual machine / container migration and server control operations; Metrics collection: Collect operational metrics after scheduling, including resource utilization, SLA compliance rate, unit business energy consumption, and migration overhead; Closed-loop optimization: An improved online gradient descent (OGD) algorithm (forgetting factor λ=0.9) is used to update the parameters of the hybrid intelligent model online. The model parameters are fine-tuned only based on newly collected index samples to avoid catastrophic forgetting and achieve dynamic iterative optimization of the scheduling strategy.
[0005] The beneficial effects of this invention are reflected in: High prediction accuracy: By using the improved Transformer network load prediction sub-model and combining it with the attention mechanism optimization module, it can accurately capture the nonlinear fluctuations and sudden changes in load, reducing the load prediction error to below 5%, and providing a reliable basis for scheduling decisions. Multi-objective collaborative optimization: By designing a reward function that integrates resource utilization, energy consumption, and SLA, collaborative optimization under multiple constraints is achieved, increasing the resource utilization compliance rate to over 85%, reducing the SLA default rate to below 0.1%, and reducing unit business energy consumption by 15%~25%. Strong dynamic adaptability: Based on the incremental learning algorithm, the model is updated online, which can quickly adapt to the dynamic changes in data center resource status, business type and load characteristics without offline retraining, thus improving scheduling flexibility; Controllable migration overhead: By optimizing migration paths and setting overhead constraints, the migration time of virtual machines / containers can be controlled within 500ms, and the migration bandwidth usage can be ≤30%, avoiding business interruption or response delay and ensuring service continuity. Attached Figure Description
[0006] In the attached diagram: Figure 1 This is a flowchart of the method steps of the present invention; Figure 2 This is a diagram of the hybrid intelligent scheduling model architecture of the present invention; Figure 3 This is a data processing flowchart of the present invention; Figure 4 This is a flowchart illustrating the implementation verification process of the present invention. Detailed Implementation
[0007] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only a part of the embodiments of the invention, and not all of them. Unless otherwise specified, the embodiments and features described in this application can be combined with each other. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.
[0008] Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of a person skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the invention.
[0009] Please refer to the instruction manual appendix. Figures 1-4 This invention provides an artificial intelligence-based data center resource scheduling optimization method. The application environment of this embodiment is a large cloud data center, which contains 1,000 heterogeneous servers (CPU model is Intel Xeon Gold 6330, memory capacity is 256GB, storage capacity is 10TB, network bandwidth is 10Gbps), running 5,000 virtual machines and 20,000 containers, and supporting services including e-commerce transactions, live video streaming, data analysis, etc. (covering compute-intensive, storage-intensive and network-intensive services). Implementation steps Step 1: Model Initialization Configuration Import nearly 6 months of historical operational data (approximately 10TB) from this data center to pre-train the hybrid intelligence model: Improved Transformer network: encoder layers = 8 layers, hidden layer neurons = 768, attention heads = 12; PPO algorithm network: Hidden layers = 4 layers, number of neurons = 512, number of training iterations = 3000; Constraint thresholds: SLA response time threshold = 500ms, total energy consumption limit of data center = 500kW, energy consumption threshold of single server = 425W (85% of rated power consumption of 500W). Step 2: Multi-dimensional data collection Multi-dimensional data is collected through data center monitoring systems (such as Zabbix and Prometheus) at a frequency of once per minute, specifically including: Physical resource data: CPU utilization (real-time collection), memory usage, remaining storage capacity, network bandwidth utilization, and motherboard temperature for each server; Virtual resource data: resource request amount, current load intensity (CPU / memory utilization), migration history and lifecycle status for each virtual machine / container; Business service data: business type tags, SLA response time thresholds (300ms for e-commerce transactions, 500ms for live video streaming), business priority coefficients (level 5 for e-commerce transactions, level 4 for live video streaming), and business concurrency. Environmental sensing data: temperature in various areas of the computer room (acquisition accuracy ±0.1℃), air conditioning operating power, total power consumption and cooling system energy consumption. Step 3: Data Preprocessing The median filtering algorithm is used to filter out outliers in server temperature and bandwidth utilization (such as bandwidth data that suddenly spikes to 100%). Missing memory usage data (missing rate ≤ 5%) was completed using linear interpolation. Min-Max normalization maps all feature data to the [0,1] interval; Feature engineering: The sliding window method (N=10) was used to extract the mean, variance, and trend slope of CPU utilization over the past 10 minutes. Wavelet transform (db4 wavelet basis) decomposes the load data to extract the fluctuation frequency (0.1~1Hz), amplitude and abrupt change points; Business QoS feature vectors: e-commerce transactions (0.4, 0.3, 0.3), live video streaming (0.3, 0.4, 0.3). Energy consumption constraint characteristics: Based on polynomial fitting, the mapping relationship between server energy consumption and CPU utilization is obtained: E=0.02U²+0.1U+50 (E is energy consumption, U is CPU utilization). Step 4: Training the Hybrid Intelligent Model Load prediction sub-model training: Input resource utilization time-series characteristics and business concurrency characteristics, train an improved Transformer network, and output the CPU / memory load prediction value for the next 5 minutes. The prediction error after training is 4.2%. Reinforcement learning decision sub-model training: The decision model is trained based on the PPO algorithm with load prediction, real-time resource status and business priority as input. The reward function is R=0.4U-0.3E-0.3S. The training continues until the reward function converges (fluctuation range ±0.008). Model validation: Offline validation was performed using historical data. The results showed that the resource utilization rate met the target of 86.5%, the SLA default rate was 0.08%, and the unit business energy consumption was reduced by 22%, thus satisfying the constraints. Step 5: Real-time scheduling decision and execution Real-time data input: Input the real-time running data from the data center into the trained model every minute; Scheduling scheme generation: The model allocates more CPU resources to e-commerce transaction business (high priority) (the allocation ratio is increased by 10%), generates hibernation instructions for idle servers (utilization ≤30%), and generates virtual machine migration schemes for overloaded servers (CPU utilization ≥90%) (migrating to servers with a load rate of 60%). Solution verification: Verified in a digital twin simulation environment, the migration time was 320ms, bandwidth usage was 25%, and there were no constraint violations. Scheduling and execution: The plan is sent to the resource management system to perform resource allocation, virtual machine migration, and server hibernation operations. Step 6: Closed-loop optimization After collecting scheduling metrics every hour, the improved OGD algorithm (λ=0.9) is used to update model parameters. When business concurrency surges (such as e-commerce promotional activities), the model quickly adjusts the scheduling strategy through online updates, maintaining resource utilization compliance rate above 85% and SLA default rate below 0.1%. Implementation effect This embodiment optimizes data center resource scheduling using the above method. Statistical results after three months of operation show: Resource utilization rate: increased by an average of 23% (from 65% to 88%). SLA compliance rate: increased from 98.5% to 99.92% (default rate 0.08%). Unit business energy consumption: reduced by 22% (from 0.85kWh / GB to 0.66kWh / GB); Migration overhead: The average migration time was 310ms, bandwidth usage was 24%, and there was no service interruption. The above implementation results show that the method of the present invention can effectively improve the intelligence level and overall performance of data center resource scheduling, and has significant practical value.
[0010] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
[0011] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.
[0012] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.
Claims
1. A data center resource scheduling optimization method based on artificial intelligence, characterized in that, Includes the following steps: S1. Multi-dimensional data collection: Collect physical resource data, virtual resource data, business service data, and environmental perception data of the data center to form the original scheduling dataset; S2. Data preprocessing: The original scheduling dataset is subjected to noise filtering, missing value completion and normalization. Resource utilization time series features, load fluctuation features, service QoS requirement features and energy consumption constraint features are extracted through feature engineering to obtain the model input feature set. S3. Construction and training of hybrid intelligent scheduling model: Construct a hybrid intelligent model that integrates load prediction sub-model and reinforcement learning decision-making sub-model. Use the input feature set of the model as training data, set resource capacity constraints, SLA constraints and energy consumption constraints as model constraints, and train the model to output the optimal resource scheduling strategy. S4. Real-time scheduling decision generation: Input the real-time operation status data of the data center into the trained hybrid intelligent scheduling model, and combine it with the current business priority weights to generate an optimized scheduling scheme for virtual machine / container resource allocation, server load balancing and dynamic resource adjustment; S5. Scheduling Execution and Closed-Loop Optimization: Execute the optimized scheduling scheme, collect the resource utilization rate, SLA compliance rate, unit business energy consumption and migration overhead indicators after scheduling, and use the incremental learning algorithm to update the parameters of the hybrid intelligent scheduling model online based on the indicators to realize the dynamic iterative optimization of the scheduling strategy.
2. The data center resource scheduling optimization method based on artificial intelligence according to claim 1, characterized in that, The multi-dimensional data mentioned in step S1 specifically includes: Physical resource data: server CPU utilization, memory usage, remaining storage capacity, network bandwidth utilization, and hardware health status parameters; Virtual resource data: number of virtual machines / containers, resource requests, current load intensity, migration history, and lifecycle status; Business service data: business type, service level agreement (SLA) parameters, business priority coefficient, request response time threshold, and business concurrency; Environmental sensing data: temperature in the computer room area, air conditioning operating power, total power consumption, and cooling system energy consumption.
3. The data center resource scheduling optimization method based on artificial intelligence according to claim 1, characterized in that, The feature engineering described in step S2 is implemented as follows: The mean, variance, and trend slope of resource utilization over the past N time slices are extracted using the sliding window method to generate time series features of resource utilization, where N is a preset positive integer from 5 to 30. By decomposing the load data using wavelet transform, the load fluctuation frequency, amplitude, and abrupt change characteristics are extracted to form load fluctuation features. The service QoS requirements are quantified into response time weight, availability weight, and reliability weight, and a service QoS requirement feature vector is constructed. Based on the mapping relationship between server energy consumption and resource utilization, energy consumption constraint features are generated.
4. The data center resource scheduling optimization method based on artificial intelligence according to claim 1, characterized in that, The construction method of the hybrid intelligent scheduling model mentioned in step S3 is as follows: The load prediction sub-model adopts an improved Transformer network. By adding an attention mechanism optimization module to the encoder, it takes the time-series characteristics of resource utilization and the characteristics of business concurrency as input, and outputs the resource load prediction values for the next T time slices, where T is a preset positive integer from 3 to 10. The reinforcement learning decision sub-model adopts the proximal policy optimization (PPO) algorithm. The state space is the load prediction value, real-time resource status and business priority coefficient. The action space is the resource allocation ratio, virtual machine / container migration path and server wake-up / sleep control. The reward function is constructed with the goal of maximizing the resource utilization compliance rate, minimizing the SLA default rate and unit business energy consumption. The decision model is trained to obtain the model. The formula for calculating the reward function R is as follows: Where U is the resource utilization compliance rate, E is the unit business energy consumption, S is the SLA default rate, and α, β, and γ are preset weighting coefficients. .
5. The data center resource scheduling optimization method based on artificial intelligence according to claim 1, characterized in that, The constraints mentioned in step S3 are specifically as follows: Resource capacity constraints: The total number of CPU cores, memory capacity, storage space, and network bandwidth allocated to all virtual resources shall not exceed the maximum available resources of the corresponding physical server. SLA constraints: Business request response time ≤ preset response threshold, service availability ≥ 99.9%, data transmission reliability ≥ 99.99%; Energy consumption constraints: Total energy consumption of the data center ≤ preset energy consumption limit, and energy consumption of a single server ≤ 85% of its rated power consumption; Migration overhead constraints: Network bandwidth usage during virtual machine / container migration ≤ 30% of total bandwidth, migration time ≤ 500ms.
6. The data center resource scheduling optimization method based on artificial intelligence according to claim 1, characterized in that, The optimized scheduling scheme described in step S4 specifically includes: Physical resource allocation scheme: clearly define the number of CPU cores, memory size, storage partition and network bandwidth quota for each virtual machine / container; Virtual resource scheduling scheme: Determine the timing of virtual machine / container creation / destruction, cross-server migration path, and load balancing target node; Server control scheme: Develop hibernation / wake-up policies for idle servers, load migration ratios for overloaded servers, and resource allocation priorities for heterogeneous servers.
7. The data center resource scheduling optimization method based on artificial intelligence according to claim 1, characterized in that, The incremental learning algorithm described in step S5 is an improved online gradient descent (OGD) algorithm. By setting the historical parameter forgetting factor λ (λ∈[0.8,0.95]), the model parameters are fine-tuned only for newly collected running index samples, preserving the historical training effect of the model and avoiding catastrophic forgetting.
8. The data center resource scheduling optimization method based on artificial intelligence according to claim 1, characterized in that, It also includes step S0: model initialization configuration, pre-importing nearly 6 months of historical operation data from the data center, pre-training the hybrid intelligent scheduling model, setting the number of network layers, the number of hidden layer neurons, the number of training iterations and constraint thresholds to obtain the initial training model.
9. The data center resource scheduling optimization method based on artificial intelligence according to claim 1, characterized in that, Step S4 also includes a scheduling scheme verification process: a data center simulation environment is constructed using digital twin technology, and the constraint satisfaction of the generated optimized scheduling scheme is verified. If there is a constraint violation, the scheduling scheme is regenerated through the action space adjustment mechanism of the reinforcement learning decision sub-model until all constraints are satisfied.
10. The data center resource scheduling optimization method based on artificial intelligence according to claim 4, characterized in that, The improved Transformer network's attention mechanism optimization module improves the accuracy of load prediction by introducing load fluctuation weight coefficients to differentiate the resource utilization characteristics of different time slices.