An artificial intelligence-based computing power center dynamic resource scheduling method
By combining distributed data acquisition and federated learning frameworks with hierarchical dynamic scheduling strategies, the problems of data privacy leakage, low collaborative efficiency, and uneven energy consumption in computing centers have been solved, achieving coordinated optimization of resource utilization and energy consumption and ensuring the stable operation of computing centers.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- DONGGUAN GUAN YIN TECH
- Filing Date
- 2026-05-09
- Publication Date
- 2026-06-12
AI Technical Summary
The existing centralized resource scheduling methods of computing centers have problems such as high risk of data privacy leakage, low collaborative efficiency, difficulty in balancing resource utilization and energy consumption, and poor model training adaptability.
A federated learning framework based on FedAvg is constructed using distributed data acquisition and preprocessing. Lightweight sub-models are trained through distributed nodes to generate hierarchical dynamic scheduling strategies, which are then fed back and adaptively adjusted in real time to achieve coordinated optimization of resources and energy consumption.
It achieves data privacy protection, efficient collaboration of distributed nodes, improved resource utilization and optimized energy consumption, shortens the model training cycle, and ensures the stable operation of the computing center.
Smart Images
Figure CN122195679A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computing center resource scheduling technology, and particularly relates to a dynamic resource scheduling method for computing centers based on artificial intelligence. Background Technology
[0002] With the rapid development of technologies such as artificial intelligence, big data, and cloud computing, computing centers are expanding in scale, increasing in the number of nodes, and becoming increasingly complex in task types, placing higher demands on the efficiency, privacy, security, and energy economy of resource scheduling. Existing computing centers mostly employ centralized resource scheduling methods, where all computing nodes upload their raw operational data (including sensitive information such as load, energy consumption, and task queues) to a central node, which then uniformly predicts resource demand and formulates scheduling strategies.
[0003] However, this centralized scheduling method has obvious technical drawbacks: First, it poses a high risk of data privacy leakage. The raw operating data of each computing node contains a large amount of sensitive information, and centralized uploading can easily lead to data leakage, which does not meet the requirements of data security protection. Second, the efficiency of multi-node collaboration is low. The central node needs to process the raw data of all nodes, resulting in high data transmission and processing pressure, which increases the delay in issuing scheduling instructions and makes it difficult to adapt to the collaborative scheduling needs of distributed computing centers. Third, it is difficult to balance resource utilization and energy consumption. Traditional scheduling strategies focus on improving resource utilization while neglecting energy consumption optimization, or vice versa, and cannot achieve the optimal synergy between the two. Fourth, the model training cycle is long. When new nodes are added or the task type changes, the global model needs to be retrained, resulting in poor adaptability.
[0004] Therefore, how to achieve efficient collaborative scheduling of distributed computing nodes, improve resource utilization, and reduce energy consumption while ensuring data privacy and security has become an urgent technical problem to be solved in the field of computing center resource scheduling. Summary of the Invention
[0005] The purpose of this invention is to provide a dynamic resource scheduling method for computing centers based on artificial intelligence. This method achieves multiple objectives, including privacy protection, efficient collaboration, energy consumption optimization, and adaptive adaptation, thereby improving the overall operating efficiency and stability of computing centers.
[0006] To achieve the above objectives, the present invention provides the following technical solution: A dynamic resource scheduling method for computing power centers based on artificial intelligence includes the following steps: S1. Distributed Data Acquisition and Preprocessing: Each computing node acts as a distributed node, collecting its own operational data in real time. The operational data includes CPU load, memory usage, storage utilization, energy consumption parameters, and task queue information. The collected operational data is preprocessed locally to obtain standardized feature data. The standardized feature data is stored only locally on the distributed nodes and is not transmitted to the central aggregation node, thus ensuring data privacy from the source. S2. Federated Learning Model Training and Iteration: Construct a Federated Learning framework based on FedAvg, which includes distributed nodes and a central aggregation node. Each distributed node trains a lightweight sub-model based on local standardized feature data to predict local future resource demand and energy consumption changes. Each distributed node encrypts the trained model parameters and uploads them to the central aggregation node. The central aggregation node uses a federated averaging algorithm to aggregate the model parameters to generate a global prediction model, and then distributes the global model parameters to each distributed node. Each distributed node fine-tunes the model by combining local real-time data to complete the model iteration. S3. Generation of hierarchical dynamic scheduling strategy: Based on the resource demand and energy consumption prediction results of each distributed node output by the global prediction model, the central aggregation node generates a hierarchical dynamic scheduling strategy and distributes it to each distributed node for execution, combining the task classification results and node running status. The hierarchical dynamic scheduling strategy includes hierarchical task scheduling, dynamic allocation and migration of virtual machines, and dynamic energy consumption optimization, so as to achieve optimal resource utilization and energy consumption. S4. Real-time feedback and adaptive adjustment of scheduling effect: Each distributed node collects the running data after scheduling in real time, and uploads it to the central aggregation node after statistical processing and encryption. Each distributed node optimizes its local sub-model based on the feedback data, and the central aggregation node adjusts the global scheduling strategy weight based on the feedback data of all distributed nodes to achieve adaptive optimization of the scheduling strategy. At the same time, an abnormal threshold is set to trigger emergency scheduling to ensure system stability.
[0007] Furthermore, in step S1, the frequency of collecting the running data is once every 10 seconds, and the specific collection range includes: CPU load, memory usage, storage utilization, energy consumption parameters, and task queue information. The standardized feature data is stored in a lightweight SQLite database with a storage period of 7 days, and is automatically deleted after the storage period expires.
[0008] Furthermore, in step S1, the local preprocessing adopts a three-level processing flow, consisting of anomaly detection, cleaning, and normalization. The anomaly detection uses the isolated forest algorithm to remove abnormal data; the data cleaning uses linear interpolation to fill in missing data with a missing rate of ≤5%; the normalization uses the Min-Max normalization algorithm to map the feature data to the [0,1] interval, and the normalization formula is: x_norm=(x-x_min) / (x_max-x_min), where x is the original data, and x_min and x_max are the historical minimum and maximum values of the feature.
[0009] Furthermore, in step S2, the lightweight sub-model is an improved LSTM sub-model with the following structure: input layer, hidden layer, and output layer; the model training parameters are: learning rate = 0.001, batch size = 32, training epochs = 50, optimizer is Adam, loss function is mean squared error, model size ≤ 100MB, and it is adapted to the lightweight training requirements of the node end.
[0010] Furthermore, in step S2, the model parameters are uploaded using AES-256 encryption, and the upload cycle is once every 30 minutes; the parameter aggregation formula of the central aggregation node is: W_global = (1 / N)×ΣW_local, where N is the number of distributed nodes, W_global is the global model parameter, and W_local is the model parameter of each distributed node; the learning rate of model fine-tuning is 0.0005, the number of fine-tuning rounds is 10, and the iteration cycle is 30 minutes, to ensure continuous optimization of model prediction accuracy.
[0011] Furthermore, in step S3, the task hierarchical scheduling adopts the K-means clustering algorithm, which combines task real-time performance and computing power requirements for classification. The classification thresholds are: high real-time tasks, medium real-time tasks, and low real-time tasks; CPU-intensive tasks, memory-intensive tasks, and storage-intensive tasks. The scheduling priority is: high-priority real-time tasks > high-priority non-real-time tasks > medium-priority tasks > low-priority tasks. The allocation algorithm adopts a greedy algorithm, which prioritizes allocation to nodes with a load of 30%-70% and an energy efficiency ≥1.2.
[0012] Furthermore, in step S3, the load thresholds for the dynamic allocation and migration of virtual machines are: high load ≥ 80%, low load ≤ 30%; virtual machine migration adopts Live Migration technology, combined with an AI dynamic bandwidth allocation algorithm based on Q-learning (bandwidth allocation range = 100Mbps-1000Mbps), migration latency ≤ 500ms, and task interruption rate ≤ 0.1%; virtual machine allocation adopts a dynamic partitioning algorithm, with each partition having ≥ 2 CPU cores and ≥ 4GB of memory.
[0013] Furthermore, in step S3, the dynamic energy consumption optimization adopts dynamic voltage and frequency adjustment technology, with the CPU operating frequency adjustment range being 1.0GHz-2.5GHz and the adjustment step size being 0.1GHz; combined with dynamic adjustment of fan speed based on PID control (speed range being 1000r / min-3000r / min), the idle energy consumption is reduced by ≥20%, achieving a balance between energy consumption and task efficiency.
[0014] Furthermore, in step S4, the feedback data is collected every 5 seconds, and the collected content includes resource utilization, energy consumption, task execution success rate, and task delay. The feedback data is encapsulated in JSON format and uploaded in an encrypted manner. The local sub-model is updated every hour through an online incremental learning algorithm, and the central aggregation node adjusts the global scheduling strategy weight every hour. When the task delay exceeds the standard or the energy consumption exceeds 90% of the rated energy consumption, emergency scheduling is automatically triggered.
[0015] Furthermore, the distributed nodes and the central aggregation node use the TLS 1.3 encrypted communication protocol to achieve data interaction.
[0016] In summary, the beneficial technical effects of the present invention are as follows: 1. Distributed data acquisition and preprocessing are adopted. Raw data and standardized feature data are stored locally and are not uploaded to the central node. Only encrypted model parameters and scheduling instructions are transmitted, which avoids data privacy leakage from the source and meets data security protection requirements.
[0017] 2. Based on the federated learning framework, each distributed node autonomously completes local model training, while the central node is only responsible for parameter aggregation and policy distribution. This reduces the amount of data transmission and the processing pressure on the central node. The delay in issuing scheduling instructions is ≤100ms, which significantly improves the collaborative scheduling efficiency of distributed computing nodes.
[0018] 3. Through the hierarchical dynamic scheduling strategy, the organic combination of hierarchical task scheduling, dynamic allocation and migration of virtual machines, and dynamic optimization of energy consumption is achieved. Under the premise of ensuring efficient task execution, the resource utilization rate is kept stable at 30%-70%, and the overall energy consumption is reduced by ≥12%, achieving the dual goals of maximizing resource utilization and minimizing energy consumption.
[0019] 4. Through model iteration and adaptive adjustment of scheduling strategies, it can adapt to scenarios such as the addition of nodes and changes in task types, without the need to retrain the global model, and the model training cycle is greatly shortened; at the same time, through emergency scheduling mechanisms and implementation verification standards, it ensures the long-term stable operation of the computing center, with a system failure rate of ≤0.05%. Attached Figure Description
[0020] The accompanying drawings are provided to further illustrate the invention and form part of the specification, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating the steps of a dynamic resource scheduling method for a computing center based on artificial intelligence, as described in this embodiment. Detailed Implementation
[0021] The present invention will be further described in detail below with reference to the accompanying drawings.
[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0023] Please see Figure 1 This invention provides a technical solution: a dynamic resource scheduling method for computing power centers based on artificial intelligence, comprising the following steps: S1: Distributed Data Acquisition and Preprocessing Each computing node, acting as a distributed node, collects its own real-time operational data at a preset frequency. The collection scope covers CPU load, memory usage, storage utilization, energy consumption parameters, and task queue information, ensuring the comprehensiveness and real-time nature of the data. The collection frequency is set to once every 10 seconds, and the specific collection content and accuracy requirements are as follows: CPU load (unit: %, precision to one decimal place), memory usage (unit: GB, precision to one decimal place), storage utilization (unit: %, precision to one decimal place), energy consumption parameters (voltage: V, precision 0.01V; current: A, precision 0.01A; heat dissipation energy consumption: W, precision 1W), and task queue information (task type: CPU-intensive / memory-intensive / storage-intensive, encoded with 01 / 02 / 03; priority: high / medium / low, encoded with 1 / 2 / 3; estimated execution time: s, precision 1s).
[0024] The collected raw operational data underwent local preprocessing using a three-level processing flow of "anomaly detection-cleaning-normalization" to ensure data quality: ① Anomaly detection: The isolated forest algorithm was used, with 100 isolated trees and an anomaly threshold of 0.8, to remove outliers such as sensor malfunctions and data mutations; ② Data cleaning: Linear interpolation was used to fill in missing data with a missing rate ≤5% to ensure data integrity; ③ Normalization: The Min-Max normalization algorithm was used to map all feature data to the [0,1] interval. The normalization formula is: x_norm=(x-x_min) / (x_max-x_min), where x is the original data, and x_min and x_max are the historical minimum and maximum values of the feature, facilitating subsequent model training.
[0025] The preprocessed standardized feature data is stored only locally on the distributed nodes, using a lightweight SQLite database with a storage period of 7 days. Data is automatically deleted after the storage period expires. The original data or standardized feature data is never uploaded to the central aggregation node, thus preventing data privacy leaks from the source.
[0026] S2: Federated Learning Model Training and Iteration A federated learning framework based on FedAvg is constructed, comprising distributed nodes and a central aggregation node. The two nodes interact via a TLS 1.3 encrypted communication protocol, transmitting only non-privacy data such as model parameters and scheduling instructions to ensure secure communication. The distributed nodes are configured with at least 8 CPU cores and at least 16GB of memory, supporting local lightweight model training. The central aggregation node is configured with at least 16 CPU cores and at least 64GB of memory, possessing large-scale parameter aggregation and parallel computing capabilities, ensuring real-time issuance of scheduling instructions with a latency of ≤100ms.
[0027] Each distributed node trains an improved lightweight LSTM sub-model based on locally preprocessed standardized feature data to predict local resource demand and energy consumption changes over the next 1-6 hours. The specific structure of this improved LSTM sub-model is as follows: input layer (feature dimension = 12, corresponding to 12 classes of collected data), hidden layer (2 layers, 64 neurons per layer, ReLU activation function), and output layer (2 dimensions, outputting predicted future resource demand and predicted energy consumption changes respectively). The model training parameters are set as follows: learning rate = 0.001, batch size = 32, training epochs = 50, using the Adam optimizer, and the mean squared error loss function to ensure a lightweight model (model size ≤ 100MB) suitable for node computing power.
[0028] Model initialization: On each distributed node, an improved lightweight LSTM sub-model is built based on the TensorFlow framework. The model structure strictly follows the preset configuration: input layer (12-dimensional, corresponding to 12 types of collected data), hidden layer (2 layers, 64 neurons per layer, activation function is ReLU), output layer (2-dimensional, outputting the predicted values of resource demand and energy consumption changes for the next 1-6 hours respectively); the model training parameters are set as follows: learning rate 0.001, batch size 32, training epochs 50, optimizer is Adam, loss function is mean squared error. Through model compression technology, the model size is ensured to be ≤100MB to adapt to the computing resources of the distributed nodes.
[0029] Local model training: Each distributed node calls the standardized feature data in the local SQLite database as model training samples to start the local training program. After training is completed, the model parameters are calculated and encrypted using the AES-256 encryption algorithm to prevent leakage during parameter transmission.
[0030] After model training is complete, each distributed node uploads its trained model parameters, encrypted using AES-256, to the central aggregation node every 30 minutes. The central aggregation node uses a federated averaging algorithm to aggregate the model parameters uploaded by all distributed nodes. The aggregation formula is: W_global = (1 / N)×ΣW_local, where N is the number of distributed nodes, W_global is the global model parameter, and W_local is the model parameter of each distributed node. After aggregation, the central aggregation node distributes the global model parameters to each distributed node. Each distributed node performs 10 rounds of fine-tuning based on the new global parameters and local real-time data, with a fine-tuning learning rate of 0.0005, completing one model iteration. The iteration cycle is 30 minutes, ensuring continuous optimization of model prediction accuracy, with a prediction accuracy ≥90% (resource demand prediction error ≤10%, energy consumption prediction error ≤8%).
[0031] S3: Generation of hierarchical dynamic scheduling strategies The central aggregation node, based on the resource demand and energy consumption predictions of each distributed node output by the global prediction model, combines task classification results with node operating status to generate a hierarchical dynamic scheduling strategy, which is then distributed to each distributed node for execution, achieving optimal synergy between resource utilization and energy consumption. This hierarchical dynamic scheduling strategy comprises three core modules: (1) Task hierarchical scheduling: K-means clustering algorithm (number of clusters = 3) is used to classify tasks based on their real-time performance and computing power requirements. The classification thresholds are clearly defined: high real-time tasks (response latency requirement ≤ 50ms), medium real-time tasks (response latency requirement ≤ 100ms), and low real-time tasks (response latency requirement ≤ 500ms); CPU-intensive tasks (CPU usage ≥ 60%), memory-intensive tasks (memory usage ≥ 50%), and storage-intensive tasks (storage read / write rate ≥ 100MB / s). The scheduling priority is set as follows: high priority real-time tasks > high priority non-real-time tasks > medium priority tasks > low priority tasks. A greedy algorithm is used for task allocation, prioritizing the allocation of tasks to nodes with a load of 30%-70% and optimal energy efficiency (energy efficiency = computing power output / energy input, ratio ≥ 1.2) to ensure optimal resource matching.
[0032] (2) Dynamic allocation and migration of virtual machines: The load threshold is set as high load ≥80% and low load ≤30%. When the load of a distributed node exceeds the high load threshold, some virtual machines are automatically migrated to nodes with loads lower than the low load threshold. The virtual machine migration adopts Live Migration technology, combined with an AI dynamic bandwidth allocation algorithm based on Q-learning algorithm. The bandwidth allocation range is 100Mbps-1000Mbps. The bandwidth is dynamically adjusted according to the amount of migration data to ensure that the migration latency is controlled within ≤500ms. A task caching mechanism is adopted during the migration process to avoid task interruption, with an interruption rate of ≤0.1%. The virtual machine allocation adopts a dynamic partitioning algorithm, which divides the node memory and CPU resources into fixed partitions (each partition has ≥2 CPU cores and ≥4GB of memory). The corresponding partitions are allocated according to the task requirements to avoid resource fragmentation.
[0033] (3) Dynamic Energy Consumption Optimization: The criterion for judging nodes with excessive energy consumption is that the energy consumption is greater than 80% of the node's rated energy consumption. For nodes with excessive energy consumption, Dynamic Voltage Frequency Scaling (DVFS) technology is used to adjust the node's CPU operating frequency. The adjustment range is 1.0GHz-2.5GHz, and the adjustment step size is 0.1GHz. The target is to reduce idle energy consumption by ≥20%. During the adjustment process, the task execution delay is monitored in real time. If the delay exceeds the corresponding task threshold, the frequency adjustment is stopped immediately to ensure that the task execution efficiency is not affected. At the same time, a dynamic fan speed adjustment algorithm based on PID control is adopted. The fan speed adjustment range is 1000r / min-3000r / min. Combined with node energy consumption optimization, a closed-loop control of "prediction-scheduling-energy consumption optimization" is realized.
[0034] S4: Real-time feedback and adaptive adjustment of scheduling effects Each distributed node collects operational data after scheduling every 5 seconds. The collected data includes resource utilization (CPU, memory, storage), energy consumption (real-time energy consumption, cumulative energy consumption), task execution success rate (≥99.9% is considered acceptable), and task latency (statistics by task type). The feedback data is encapsulated in JSON format, encrypted, and then uploaded to the central aggregation node. Only statistical data is uploaded, not the original details, to further protect data privacy.
[0035] Dual adaptive optimization based on feedback data: ① Local sub-model optimization: Each distributed node inputs feedback data into its local sub-model, employing an online incremental learning algorithm to update model parameters every hour, continuously optimizing prediction accuracy to achieve a target accuracy ≥92%; ② Global scheduling strategy optimization: The central aggregation node adjusts the weights of the global scheduling strategy using a weighted average method based on feedback data from all distributed nodes. The weight adjustment cycle is 1 hour, with the following rules: During peak computing power demand periods (e.g., 9:00-21:00), resource utilization weight = 0.6, energy consumption weight = 0.4; during off-peak energy-saving periods (e.g., 21:00-9:00 the next day), resource utilization weight = 0.4, energy consumption weight = 0.6. Simultaneously, a scheduling anomaly threshold is set. When task execution delay exceeds the limit (exceeding the corresponding task threshold) or energy consumption abnormally increases (exceeding 90% of rated energy consumption), emergency scheduling is automatically triggered, migrating the task to a backup node to ensure the stable operation of the computing center.
[0036] To ensure the effectiveness of the scheduling method, the following preset verification standards are used for testing. Implementation is considered successful if all standards are met: ① Privacy Protection: Communication data between nodes and the central node is detected using packet capture tools to confirm that no raw operational data is transmitted, only encrypted data such as model parameters and scheduling instructions exist; ② Prediction Accuracy: After 72 hours of continuous monitoring, the prediction accuracy of both the local sub-model and the global model is ≥90%, resource demand prediction error is ≤10%, and energy consumption prediction error is ≤8%; ③ Scheduling Performance: Resource utilization is stable at 30%-70%, high-priority task latency is ≤50ms, virtual machine migration latency is ≤500ms, and task execution success rate is ≥99.9%; ④ Energy Consumption Optimization: Idle energy consumption during off-peak hours is reduced by ≥20%, and overall energy consumption is reduced by ≥12%; ⑤ Stability: After 7 days of continuous operation, there are no scheduling anomalies or task interruptions, and the system failure rate is ≤0.05%.
[0037] Example 1 Distributed nodes: Servers equipped with Intel Xeon E5-2690 CPUs (8 cores), 16GB of memory, and 1TB of storage are used to deploy a lightweight SQLite database, supporting local model training; Central aggregation nodes: Management servers equipped with Intel Xeon Gold6248 CPUs (16 cores), 64GB of memory, and 4TB of storage are used to enable large-scale parameter aggregation, with a scheduling instruction issuance latency of ≤100ms; Distributed nodes and central aggregation nodes communicate with each other via the TLS 1.3 encrypted communication protocol.
[0038] The central aggregation node monitors the energy consumption data of each distributed node in real time. When the energy consumption of a node exceeds 80% of the rated energy consumption, an energy consumption optimization command is issued. The node then activates Dynamic Voltage Frequency Scaling (DVFS) technology to adjust the CPU operating frequency (adjustment range 1.0GHz-2.5GHz, adjustment step size 0.1GHz). At the same time, it activates dynamic fan speed adjustment based on PID control to adjust the fan speed to 1000r / min-3000r / min, ensuring that the idle energy consumption is reduced by ≥20%. During the adjustment process, the task execution delay is monitored in real time. If the delay exceeds the real-time threshold of the corresponding task, the frequency adjustment is immediately stopped and the state is restored to the state before adjustment, ensuring that the task execution efficiency is not affected, and realizing closed-loop control of "prediction-scheduling-energy consumption optimization".
[0039] Model and Strategy Optimization: ① Local Sub-model Optimization: Each distributed node inputs feedback data into its local improved LSTM sub-model, employing an online incremental learning algorithm to update model parameters every hour, continuously optimizing prediction accuracy to achieve a target of ≥92%; ② Global Scheduling Strategy Optimization: The central aggregation node adjusts the weights of the global scheduling strategy based on feedback data from all distributed nodes using a weighted average method. The adjustment cycle is one hour, and the adjustment rules strictly adhere to presets: During peak computing power demand periods (9:00-21:00), resource utilization weight = 0.6, energy consumption weight = 0.4, prioritizing scheduling efficiency; during off-peak energy-saving periods (21:00-9:00 the next day), resource utilization weight = 0.4, energy consumption weight = 0.6, prioritizing energy consumption optimization. Simultaneously, a scheduling anomaly threshold is set. When task execution delay exceeds the corresponding task's real-time threshold, or node energy consumption exceeds 90% of the rated energy consumption, emergency scheduling is automatically triggered, migrating tasks on the abnormal node to a backup node to ensure stable operation of the computing center and prevent task interruptions.
[0040] Feedback Data Collection and Upload: Each distributed node collects post-scheduling operational data every 5 seconds. The collected data includes: CPU, memory, and storage resource utilization rates; real-time energy consumption; cumulative energy consumption; task execution success rate (statistics by task type); and task latency (statistics by task type). The collected feedback data is encapsulated in JSON format, for example: {"Node ID":"Node01","CPU Utilization":56.8,"Memory Utilization":45.2,"Real-time Energy Consumption":320W,"Task Success Rate":99.95%,"High-Priority Task Latency":35ms}. After encapsulation, the data is encrypted using AES-256 and uploaded to the central aggregation node. Only statistical data is uploaded; raw detailed data is not uploaded to further protect data privacy.
[0041] The implementation effect verification procedure was initiated and run continuously for 7 days. Each item was tested according to the preset verification standards. The verification process and results are as follows: ① Prediction accuracy verification: The prediction data of the local sub-model and the global model were monitored continuously for 72 hours. The actual operating data were compared, and the prediction accuracy, resource demand prediction error, and energy consumption prediction error were calculated. The results showed that the global model prediction accuracy was 93.2%, the local sub-model average prediction accuracy was 92.5%, the resource demand prediction error was 8.7%, and the energy consumption prediction error was 7.3%, all of which met the preset standards (prediction accuracy ≥90%, resource demand prediction error ≤10%, energy consumption prediction error ≤8%). ② Privacy protection verification: Using the network packet capture tool Wireshark, the communication data between the distributed nodes and the central aggregation node was monitored in real time for 72 hours to confirm that there was no original operation data transmission, only encrypted model parameters, scheduling instructions, and feedback statistics. The privacy protection meets the requirements. ③ Energy consumption optimization verification: The idle energy consumption during the off-peak period from 21:00 to 9:00 the next day was statistically analyzed. Compared with before optimization, the idle energy consumption decreased by an average of 22.3% (≥20%). The overall energy consumption over 7 days was statistically analyzed. Compared with when this scheduling method was not used, the overall energy consumption decreased by 13.1% (≥12%). The energy consumption optimization effect met the standard. ④ Scheduling performance verification: Resource utilization, task latency, virtual machine migration latency, and task execution success rate were statistically analyzed over 7 days. The results show that the resource utilization of each distributed node is stable between 38% and 65%, which is within the optimal range of 30% to 70%. The average latency of high-priority tasks is 38ms (≤50ms), the average latency of virtual machine migration is 420ms (≤500ms), and the task execution success rate is 99.96% (≥99.9%). The scheduling performance meets the requirements. ⑤ Stability test: After running continuously for 7 days and monitoring the system's operating status in real time, no scheduling anomalies, task interruptions, or node crashes occurred. The system failure rate was 0.03% (≤0.05%), and the stability met the requirements.
[0042] In summary, the scheduling method in this embodiment passes all verification standards, can operate stably and efficiently, and can achieve multiple goals such as privacy protection, efficient collaboration, and energy consumption optimization. It can be directly applied to the engineering deployment of distributed computing centers.
[0043] Example 2 This scheduling method is applied to a distributed computing center containing 20 distributed nodes, specifically 5 edge computing nodes and 15 core computing nodes. The edge nodes mainly undertake data collection and lightweight tasks for terminal devices, while the core nodes undertake large-scale computing tasks. The specific implementation differences are as follows, and the remaining steps are the same as in Example 1.
[0044] Hardware configuration adaptation: The edge computing nodes adopt industrial-grade servers with Intel Core i7-12700 CPU (8 cores), 16GB of memory, and 512GB of storage to meet the lightweight requirements of edge scenarios; the core computing nodes adopt servers with Intel Xeon Gold 6348 CPU (24 cores), 128GB of memory, and 2TB of storage to improve large-scale computing power; the hardware configuration of the central aggregation node is upgraded to CPU ≥ 24 cores and memory ≥ 128GB to ensure the efficiency of parameter aggregation and policy distribution of 20 nodes.
[0045] The improved LSTM sub-model for edge nodes is further compressed, with a model size of ≤50MB, to adapt to the computing power limitations of edge nodes; the sub-model for core nodes maintains its original configuration and improves prediction accuracy; in the scheduling strategy, terminal-related tasks are prioritized to edge nodes, while large-scale computing tasks are allocated to core nodes to optimize node load balancing; the bandwidth allocation range for virtual machine migration between edge nodes and core nodes is adjusted to 500Mbps-2000Mbps to ensure migration efficiency.
[0046] Each distributed node trains an improved LSTM sub-model with the following structure: input layer (12-dimensional), hidden layer (2 layers, 64 neurons / layer, ReLU activation function), and output layer (2-dimensional). Training parameters: learning rate 0.001, batch size 32, training epochs 50, Adam optimizer, MSE loss function, model size ≤100MB. This model is used to predict resource demand and energy consumption changes in the next 1-6 hours.
[0047] Every 30 minutes, each distributed node uploads the trained model parameters to the central aggregation node after encryption using AES-256. The central aggregation node aggregates the parameters using a federated averaging algorithm (W_global = (1 / 10)×ΣW_local), generates a global model, and distributes it to each distributed node. Each distributed node performs 10 rounds of fine-tuning (with a learning rate of 0.0005) based on the global parameters and local real-time data to complete the iteration. After continuous monitoring for 72 hours, the prediction accuracy is ≥90%, the resource demand prediction error is ≤10%, and the energy consumption prediction error is ≤8%.
[0048] Implementation effect verification: Validation was conducted for 7 consecutive days, and the results are as follows: ① Privacy protection: No raw data was transmitted; only encrypted parameters and instructions were transmitted. ② Prediction accuracy: Both global and local model accuracy were ≥90%, resource error ≤10%, and energy consumption error ≤8%. ③ Scheduling performance: Resource utilization was stable at 30%-70%, high-priority task latency ≤50ms, migration latency ≤500ms, and task success rate ≥99.9%. ④ Energy consumption optimization: Low-peak idle energy consumption was reduced by 22%, and overall energy consumption was reduced by 13%. ⑤ Stability: No scheduling anomalies were observed, and the system failure rate was 0.03%, meeting all verification standards and indicating successful implementation.
[0049] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.
[0050] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A dynamic resource scheduling method for computing centers based on artificial intelligence, characterized in that, Includes the following steps: S1. Distributed Data Acquisition and Preprocessing: Each computing node acts as a distributed node, collecting its own operational data in real time. The operational data includes CPU load, memory usage, storage utilization, energy consumption parameters, and task queue information. The collected operational data is preprocessed locally to obtain standardized feature data. The standardized feature data is stored only locally on the distributed nodes and is not transmitted to the central aggregation node. S2. Federated Learning Model Training and Iteration: Construct a Federated Learning framework based on FedAvg, which includes distributed nodes and a central aggregation node. Each distributed node trains a lightweight sub-model based on local standardized feature data to predict local future resource demand and energy consumption changes. Each distributed node encrypts the trained model parameters and uploads them to the central aggregation node. The central aggregation node uses a federated averaging algorithm to aggregate the model parameters to generate a global prediction model, and then distributes the global model parameters to each distributed node. Each distributed node fine-tunes the model by combining local real-time data to complete the model iteration. S3. Generation of hierarchical dynamic scheduling strategy: Based on the resource demand and energy consumption prediction results of each distributed node output by the global prediction model, the central aggregation node generates a hierarchical dynamic scheduling strategy and distributes it to each distributed node for execution. The hierarchical dynamic scheduling strategy includes hierarchical task scheduling, dynamic allocation and migration of virtual machines, and dynamic energy consumption optimization. S4. Real-time feedback and adaptive adjustment of scheduling effect: Each distributed node collects the running data after scheduling in real time, and uploads it to the central aggregation node after statistical processing and encryption; each distributed node optimizes its local sub-model based on the feedback data, and the central aggregation node adjusts the global scheduling strategy weight based on the feedback data of all distributed nodes.
2. The dynamic resource scheduling method for computing centers based on artificial intelligence according to claim 1, characterized in that, In step S1, the frequency of the running data collection is once every 10 seconds. The specific collection range includes: CPU load, memory usage, storage utilization, energy consumption parameters, and task queue information. The standardized feature data is stored in a lightweight SQLite database with a storage period of 7 days. The data is automatically deleted after the storage period expires.
3. The method for dynamic resource scheduling of computing centers based on artificial intelligence according to claim 1, characterized in that, In step S1, the local preprocessing adopts a three-level processing flow, consisting of anomaly detection, cleaning, and normalization. The anomaly detection uses the isolated forest algorithm to remove abnormal data. Data cleaning uses linear interpolation to fill in missing data with a missing rate of ≤5%; normalization uses the Min-Max normalization algorithm to map the feature data to the [0,1] interval. The normalization formula is: x_norm=(x-x_min) / (x_max-x_min), where x is the original data, and x_min and x_max are the historical minimum and maximum values of the feature.
4. The method for dynamic resource scheduling of computing centers based on artificial intelligence according to claim 1, characterized in that, In step S2, the lightweight sub-model is an improved LSTM sub-model with the following structure: input layer, hidden layer, and output layer. The model training parameters are: learning rate = 0.001, batch size = 32, training epochs = 50, optimizer is Adam, loss function is mean squared error, model size ≤ 100MB, and it is suitable for lightweight training requirements at the node end.
5. The dynamic resource scheduling method for computing centers based on artificial intelligence according to claim 4, characterized in that, In step S2, the model parameters are uploaded using AES-256 encryption, and the upload cycle is once every 30 minutes. The parameter aggregation formula of the central aggregation node is: W_global = (1 / N)×ΣW_local, where N is the number of distributed nodes, W_global is the global model parameter, and W_local is the model parameter of each distributed node. The learning rate of model fine-tuning is 0.0005, the number of fine-tuning rounds is 10, and the iteration cycle is 30 minutes to ensure continuous optimization of model prediction accuracy.
6. The method for dynamic resource scheduling of computing centers based on artificial intelligence according to claim 1, characterized in that, In step S3, the task hierarchical scheduling adopts the K-means clustering algorithm, which combines task real-time performance and computing power requirements for classification. The classification thresholds are: high real-time tasks, medium real-time tasks, and low real-time tasks; CPU-intensive tasks, memory-intensive tasks, and storage-intensive tasks. The scheduling priority is: high-priority real-time tasks > high-priority non-real-time tasks > medium-priority tasks > low-priority tasks. The allocation algorithm adopts a greedy algorithm, which prioritizes allocation to nodes with a load of 30%-70% and an energy efficiency ≥1.
2.
7. The method for dynamic resource scheduling of computing centers based on artificial intelligence according to claim 1, characterized in that, In step S3, the load thresholds for the dynamic allocation and migration of virtual machines are: high load ≥ 80%, low load ≤ 30%; virtual machine migration adopts Live Migration technology, combined with an AI dynamic bandwidth allocation algorithm based on Q-learning (bandwidth allocation range = 100Mbps-1000Mbps), migration latency ≤ 500ms, and task interruption rate ≤ 0.1%; virtual machine allocation adopts a dynamic partitioning algorithm, with each partition having ≥ 2 CPU cores and ≥ 4GB of memory.
8. The method for dynamic resource scheduling of computing centers based on artificial intelligence according to claim 1, characterized in that, In step S3, the dynamic energy consumption optimization adopts dynamic voltage and frequency adjustment technology, with the CPU operating frequency adjustment range being 1.0GHz-2.5GHz and the adjustment step size being 0.1GHz; combined with dynamic fan speed adjustment based on PID control (speed range being 1000r / min-3000r / min), the idle energy consumption is reduced by ≥20%, achieving a balance between energy consumption and task efficiency.
9. The method for dynamic resource scheduling of computing centers based on artificial intelligence according to claim 1, characterized in that, In step S4, the feedback data is collected every 5 seconds, and the collected data includes resource utilization, energy consumption, task execution success rate, and task delay. The feedback data is encapsulated in JSON format and uploaded in an encrypted manner. The local sub-model is updated every hour through an online incremental learning algorithm, and the central aggregation node adjusts the global scheduling strategy weight every hour. When the task delay exceeds the standard or the energy consumption exceeds 90% of the rated energy consumption, emergency scheduling is automatically triggered.
10. The method for dynamic resource scheduling of computing centers based on artificial intelligence according to claim 1, characterized in that, The distributed nodes and the central aggregation node use the TLS 1.3 encrypted communication protocol to achieve data interaction.