Data center dynamic optimization method based on deep reinforcement learning

Abstract

The invention relates to the technical field of data dynamic optimization, and discloses a data center dynamic optimization method based on deep reinforcement learning, comprising the following modules and components: an offline deployment and DRL model initialization module, an online dynamic optimization execution module, and a closed loop iteration and abnormity emergency module. The offline deployment and DRL model initialization module comprises a multi-source data acquisition submodule, a data standardization processing subunit, a DRL model training submodule and an optimization threshold configuration subunit; the online dynamic optimization execution module comprises a real-time data perception sub-module, a load prediction and decision generation sub-module, an optimization execution and adaptation sub-module and a feedback closed loop sub-unit; the closed-loop iteration and abnormity emergency module comprises a real-time monitoring sub-module, a model iteration optimization sub-module and an abnormity early warning and emergency sub-unit; the objective of the invention is to solve the problem that the existing optimization method has the limitation of single target guidance and always focuses on the optimization demand of single dimension.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data dynamic optimization, and in particular to a data center dynamic optimization method based on deep reinforcement learning. BACKGROUND

[0002] Data center dynamic optimization refers to an intelligent management process that dynamically adjusts management strategies and execution actions in the core links of resource allocation, environment control, and business scheduling, in combination with real-time running states, load change trends, and business demand differences, to achieve the intelligent management of multi-objective collaborative balance such as energy consumption control, performance guarantee, and resource efficient utilization, and the core lies in replacing the traditional static management mode by continuously adapting to complex and changeable running environments and business scenarios, and improving the flexibility and accuracy of data center operation. The existing optimization methods have the limitation of single target orientation, often focus on single-dimensional optimization requirements, are difficult to consider multiple core demands, are prone to the situation of losing one to gain another, generally rely on preset rules or static models, lack adaptive ability to dynamic load fluctuations, business type changes, and hardware running state evolution, cannot flexibly adjust optimization strategies according to real-time scenarios, need to rely on frequent manual intervention for parameter correction and strategy adjustment, not only increase the operation complexity and labor cost, but also may cause optimization to be not timely due to response lag, and are difficult to match the dynamic change of data center running requirements. SUMMARY

[0003] The present application aims to provide a data center dynamic optimization method based on deep reinforcement learning, to solve the problem of the limitation of single target orientation of the existing optimization methods, often focusing on single-dimensional optimization requirements, being difficult to consider multiple core demands, being prone to the situation of losing one to gain another, generally relying on preset rules or static models, lacking adaptive ability to dynamic load fluctuations, business type changes, and hardware running state evolution, being unable to flexibly adjust optimization strategies according to real-time scenarios, needing to rely on frequent manual intervention for parameter correction and strategy adjustment, not only increasing the operation complexity and labor cost, but also causing optimization to be not timely due to response lag, and being difficult to match the dynamic change of data center running requirements.

[0004] In order to achieve the above-mentioned purpose, the present application adopts the following technical scheme: A data center dynamic optimization method based on deep reinforcement learning, comprising the following modules and components: Offline deployment and DRL model initialization module, online dynamic optimization execution module, closed-loop iteration and abnormal emergency module; The offline deployment and DRL model initialization module comprises a multi-source data acquisition sub-module, a data standardization processing sub-unit, a DRL model training sub-module, and an optimization threshold configuration sub-unit. The online dynamic optimization execution module includes: a real-time data perception submodule, a load prediction and decision generation submodule, an optimization execution and adaptation submodule, and a feedback closed-loop subunit; The closed-loop iteration and emergency response module includes: a real-time monitoring submodule, a model iteration optimization submodule, and an anomaly warning and emergency response subunit.

[0005] As a further improvement to this technical solution: the multi-source data acquisition submodule includes: a sensor cluster, an API interface cluster, and a data transmission triggering unit; the data standardization processing submodule includes: an outlier removal tool, a normalization processing terminal, and a data caching module; the DRL model training submodule includes: a model architecture design unit, a PPO algorithm training platform, a historical data import tool, and a training effect verification terminal; the optimization threshold configuration submodule includes: a threshold setting terminal and a configuration file storage module.

[0006] As a further improvement to this technical solution: the sensor cluster includes: data acquisition hardware, energy consumption, and business data; the API interface cluster includes: interfaces with cloud platforms, servers, and business monitoring systems; the data transmission triggering unit includes: data acquisition triggered by set frequencies and load surges; the outlier removal tool uses 3... The principle is to identify and replace outliers; the data caching module temporarily stores preprocessed data.

[0007] As a further improvement to this technical solution: the model architecture design unit includes: a state space S containing seven dimensions: server cluster CPU, memory, network utilization, real-time total energy consumption, cooling system set temperature, average service response time, and high-priority service ratio; and an action space A containing server resource allocation, cooling temperature adjustment, and virtual machine migration actions; the reward function R is formulated as follows: , =0.4、 =0.5、 =0.1, migration cost coefficient = migration data volume × transmission delay / 1000; the PPO algorithm training platform: initial learning rate 0.001, hidden layer node number 256, iteration 500 rounds, convergence is determined by the average reward fluctuation ≤5% for 10 consecutive rounds; the historical data import tool: imports preprocessed data from multiple scenarios over the past 6 months; the training effect verification terminal: calculates energy consumption reduction rate and response time compliance rate on the test set; the threshold setting terminal configures core, ordinary business response time, CPU utilization, cooling temperature, and energy consumption target thresholds; the configuration file storage module stores threshold configuration files.

[0008] As a further improvement to this technical solution: the real-time data sensing submodule includes: a data synchronization engine and a state integration unit; the data synchronization engine synchronizes sensor and API interface data every 10 seconds; the state integration unit integrates the data into a state vector S_t recognizable by the DRL model; the load prediction and decision generation submodule includes: an LSTM load prediction submodel, a DRL agent, and a decision optimization unit; the LSTM load prediction submodel takes nearly 30 minutes of load data as input and uses a sliding window formula: , The load fluctuation rate over the past 10 minutes is used to predict the load peak and confidence level for the next 15 minutes; the DRL agent receives the current state S_t and the load prediction result, and outputs the optimal action A_t; the decision optimization unit verifies the feasibility of the action.

[0009] As a further improvement to this technical solution: the optimized execution and adaptation submodule includes: a resource scheduling execution engine, a server cluster control interface, a cooling equipment linkage unit, and a cross-node data transmission encryption unit; the resource scheduling execution engine issues resource adjustment instructions and executes virtual machine migration; the server cluster control interface interfaces with server hardware; the cooling equipment linkage unit uses a PID algorithm to adjust the cooling system output; the cross-node data transmission encryption unit uses AES-256 encryption and data verification; the feedback closed-loop subunit includes: an effect acquisition terminal and a reward calculation module; the effect acquisition terminal collects energy consumption, response time, and resource utilization after execution; the reward calculation module calculates the actual reward value R_t according to the reward function and feeds it back.

[0010] As a further improvement to this technical solution: the real-time monitoring submodule includes: an optimization effect monitoring panel and a data visualization tool; the model iteration optimization submodule includes: a data archiving server, a parameter iteration tool, and an optimization effect verification subunit; the anomaly warning and emergency response subunit includes: an anomaly detection module, a tiered warning terminal, and an emergency decision-making unit; the optimization effect monitoring panel displays energy consumption reduction rate, response time compliance rate, resource utilization balance, and model decision accuracy; the data visualization tool presents indicator trends using line charts and bar charts; the data archiving server archives running data, model decision logs, and optimization effect data monthly; the parameter iteration tool uses the gradient descent method iteration formula: , For model parameters, The dynamic adjustment range is 0.0001-0.001, and the loss function is the mean square error between the predicted and actual reward values. A full iteration is performed monthly. The optimization effect verification subunits A and B test and compare the indicators before and after the iteration. The anomaly detection module monitors business response time, server energy consumption, and node failures. The tiered early warning terminal includes a first-level audible and visual alarm and administrator mobile app push notifications, and a second-level system pop-up and SMS reminder. The emergency decision-making unit has a built-in emergency action library, and the first-level early warning executes preset decisions.

[0011] A method for dynamic optimization of data centers based on deep reinforcement learning further includes the following steps: Step 1: Preliminary Preparation and Environment Deployment Phase; Step 2: Data preprocessing and model training initialization phase; Step 3: Online dynamic optimization operation phase; Step 4: Closed-loop iteration and maintenance phase.

[0012] As a further improvement to this technical solution: the preliminary preparation and environment deployment stage includes: environment adaptation confirmation, hardware installation and deployment, software system deployment, permission configuration, and data interface debugging; the data preprocessing and model training initialization stage includes: data acquisition startup, data preprocessing execution, data verification, model parameter configuration, historical data import, model training startup, model convergence verification, and optimization threshold input; the online dynamic optimization operation stage includes: system startup, real-time data acquisition and integration, load prediction, optimization decision generation, optimization action execution, execution effect collection, feedback and model update, and real-time monitoring display; the closed-loop iteration and maintenance stage includes: daily monitoring and inspection, data archiving, monthly model iteration, periodic threshold review, anomaly handling, and equipment maintenance.

[0013] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention achieves a precise balance between data center energy consumption control and business performance assurance through the integrated application of deep reinforcement learning and time series prediction technologies. It effectively solves the limitations of traditional optimization methods that are single-objective-oriented. Relying on multi-dimensional state perception and dynamic decision-making mechanisms, the system can automatically adapt to load fluctuations and business changes, generate resource allocation, environmental control and virtual machine migration strategies that meet actual needs, significantly reduce manual intervention costs, avoid resource idleness and excessive consumption, improve the overall operating efficiency of the data center, and make the optimization process more intelligent and targeted.

[0014] 2. The closed-loop mechanism of perception, decision-making, execution, feedback, and iteration constructed in this invention endows the system with the ability to continuously adapt to dynamic environments. Through regular model iteration and threshold optimization, it ensures that the optimization effect steadily improves in long-term operation and avoids performance decline caused by model rigidity. At the same time, the comprehensive anomaly warning and emergency response mechanism can quickly identify and deal with operational risks, ensuring business continuity. The standardized data processing and encrypted transmission design not only improves data reliability but also meets compliance requirements, providing comprehensive support for the stable, efficient, and secure operation of the data center.

[0015] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it according to the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Specific embodiments of the present invention are given in detail below with reference to the accompanying drawings. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Fig. 1 This is a system diagram of a data center dynamic optimization method based on deep reinforcement learning; Fig. 2 This is a schematic diagram of the steps involved in a dynamic optimization method for data centers based on deep reinforcement learning. Detailed Implementation

[0017] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are for illustrative purposes only and are not intended to limit the scope of the invention. The invention is described more specifically in the following paragraphs by way of example with reference to the accompanying drawings. It should be noted that the drawings are in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the present invention.

[0018] Please see Figs. 1-2 In this embodiment of the invention, a method for dynamic optimization of a data center based on deep reinforcement learning includes the following modules and components: Offline deployment and DRL model initialization module, online dynamic optimization execution module, closed-loop iteration and exception handling module; The offline deployment and DRL model initialization module includes: a multi-source data acquisition submodule, a data standardization processing subunit, a DRL model training submodule, and an optimized threshold configuration subunit; The online dynamic optimization execution module includes: a real-time data perception submodule, a load prediction and decision generation submodule, an optimization execution and adaptation submodule, and a feedback closed-loop subunit; The closed-loop iteration and emergency response module includes: a real-time monitoring submodule, a model iteration optimization submodule, and an anomaly warning and emergency response subunit; Specifically, the offline deployment and DRL model initialization module is responsible for the initial hardware and software deployment, data preprocessing, reinforcement learning model training and threshold configuration, providing basic support for subsequent optimization. Online dynamic optimization execution module: Real-time perception of data center status, generation and execution of optimization decisions, forming a closed loop of perception, decision-making, execution and feedback; Closed-loop iteration and emergency response module: monitors optimization results, continuously iterates the model, handles abnormal situations, and ensures optimization continuity.

[0019] The multi-source data acquisition submodule includes: sensor cluster, API interface cluster, and data transmission triggering unit; the data standardization processing submodule includes: outlier removal tool, normalization processing terminal, and data caching module; the DRL model training submodule includes: model architecture design unit, PPO algorithm training platform, historical data import tool, and training effect verification terminal; the optimized threshold configuration submodule includes: threshold setting terminal and configuration file storage module. Specifically, the multi-source data acquisition submodule includes a sensor cluster, an API interface cluster, and a data transmission triggering unit, used to comprehensively collect various types of data from the data center; Data standardization processing subunit: includes outlier removal tools, normalization processing terminal, and data caching module, used to clean and standardize raw data; DRL Model Training Submodule: Includes model architecture design unit, PPO algorithm training platform, historical data import tool, and training effect verification terminal, used to build and train reinforcement learning models; The optimized threshold configuration subunit includes a threshold setting terminal and a configuration file storage module, used to set optimization goals and constraints.

[0020] The sensor cluster includes: data acquisition hardware, energy consumption, and business data; the API interface cluster includes: interfaces with cloud platforms, servers, and business monitoring systems; the data transmission triggering unit includes: data acquisition triggered by set frequencies and load surges; the outlier removal tool uses 3D... The principle is to identify and replace outliers; the data caching module temporarily stores preprocessed data. Specifically, the sensor cluster collects hardware operating metrics, including CPU, memory, network utilization, energy consumption data, server and cooling system power consumption, business metrics, response time, and throughput, covering core data dimensions of the data center. API interface cluster: Connects to cloud platform management system, server BMC management interface, and business monitoring system to obtain structured data such as virtual machine relationship and business priority; Data transmission triggering unit: By default, it collects data at a set frequency of 10 seconds / time. When the load fluctuation exceeds 20%, it triggers real-time collection to balance data integrity and transmission efficiency. Outlier removal tool: using 3 In principle, if data X satisfies | - |>3 μ is the average value over the past 30 days. Extreme outliers were identified based on the standard deviation over the past 30 days and replaced using linear interpolation to avoid interference from outlier data. Normalization of the terminal: using the Z-score formula: Standardize data units and eliminate differences in magnitude between different indicators; Data caching module: Temporarily stores preprocessed standardized data, supports batch import of model training platforms, and improves data reading efficiency.

[0021] Model architecture design unit: The state space S contains seven dimensions: server cluster CPU, memory, network utilization, real-time total energy consumption, cooling system set temperature, average service response time, and proportion of high-priority services; the action space A contains server resource allocation, cooling temperature adjustment, and virtual machine migration actions; the reward function R formula is: , =0.4、 =0.5、 =0.1, migration cost coefficient = migration data volume × transmission delay / 1000; PPO algorithm training platform: initial learning rate 0.001, hidden layer node number 256, iteration 500 rounds, convergence is determined by the mean fluctuation of reward ≤5% for 10 consecutive rounds; historical data import tool: import preprocessed data from multiple scenarios in the past 6 months; training effect verification terminal: test set calculation energy consumption reduction rate, response time compliance rate; threshold setting terminal configuration core, ordinary business response time, CPU utilization, cooling temperature, energy consumption target threshold; configuration file storage module stores threshold configuration files; Specifically, the model architecture design unit, State Space S, contains 7 core dimensions: server cluster CPU utilization, memory utilization, network utilization, real-time total energy consumption, cooling system set temperature, average business response time, and high-priority business ratio. It comprehensively covers the environmental and business information required for data center optimization, ensuring that the agent accurately perceives the state. Action Space A: Includes three types of executable actions: First, server resource allocation: CPU core count adjustment from 2 to 32 cores, memory quota adjustment from 4 to 128GB; second, cooling temperature adjustment: 18-24℃, step size 0.5℃; and third, virtual machine migration actions: whether migration is triggered, selection of migration target node, taking into account the coordinated optimization of resource scheduling and environment control. Reward function R: Formula: Formula annotation: =0.4 (Energy optimization weight). =0.5 (performance guarantee weight). =0.1 (migration cost weight), the baseline energy consumption is the average energy consumption of the 30 days before optimization; the target response time is set according to the business level (core business ≤300ms, ordinary business ≤500ms); the migration cost coefficient = migration data volume × transmission delay / 1000, which is used to quantify the additional overhead brought by virtual machine migration. The formula's function is to achieve a balance between energy consumption reduction and business performance protection through weight allocation, avoid the loss of one side due to optimization of a single goal, and introduce migration cost constraints to prevent resource waste caused by frequent migrations. PPO Algorithm Training Platform: Core parameters: initial learning rate 0.001, number of hidden layer nodes 256, number of iterations 500 rounds; convergence criterion is that the average reward fluctuation of 10 consecutive rounds is ≤5%, ensuring that the model training is sufficient and stable; function: to train agents using the PPO algorithm (proximal policy optimization), which has both training stability and exploratory nature, and is suitable for the complex environment of dynamic changes in data centers; Historical data import tool: Import preprocessed data from the past 6 months, covering multiple scenarios such as peak load, off-peak load, and fault recovery, to ensure that the model has strong generalization ability; Training effect verification terminal: The training effect of the model is verified by calculating two core indicators, namely energy consumption reduction rate and response time achievement rate, through the test set. Threshold setting terminal: Configure optimization constraint thresholds, including core business response time ≤300ms, ordinary business response time ≤500ms, reasonable CPU utilization range of 30%-70%, safe cooling temperature range of 18-24℃, and energy consumption reduction target ≥15%; Configuration file storage module: Stores the above threshold configuration parameters, providing constraints for online optimization, and supports the retrieval and modification of parameters at any time.

[0022] The real-time data sensing submodule includes: a data synchronization engine and a state integration unit; the data synchronization engine synchronizes sensor and API interface data every 10 seconds; the state integration unit integrates the data into a state vector S_t that can be recognized by the DRL model; the load prediction and decision generation submodule includes: an LSTM load prediction sub-model, a DRL agent, and a decision optimization unit; the LSTM load prediction sub-model takes nearly 30 minutes of load data as input and uses a sliding window formula: , Given the load fluctuation rate over the past 10 minutes, predict the load peak and confidence level for the next 15 minutes; DRL agent: receives the current state S_t and the load prediction result, and outputs the optimal action A_t; Decision optimization unit: verifies the feasibility of the action; Specifically, the real-time data perception submodule includes: a data synchronization engine that synchronizes sensor and API interface data at a frequency of 10 seconds per cycle to ensure data timeliness; and a state integration unit that converts synchronized multi-dimensional data into a state vector S_t recognizable by the DRL model, achieving data format adaptation. Load Prediction and Decision Generation Submodule: LSTM Load Prediction Submodel: Input Data: Load time-series data for the past 30 minutes (CPU utilization, business request volume), sliding window formula: Formula annotation: This represents the load volatility over the past 10 minutes (quantifying the severity of load fluctuations). The floor sign is used to ensure that the window size is an integer; T is the sliding window size (unit: seconds); the formula's function is to adaptively adjust the window size according to load fluctuations. The larger the load fluctuation, the larger the window, which improves the model's prediction accuracy for dynamic loads. The output results are the load peak and confidence level (≥90%) for the next 15 minutes, providing forward-looking support for agent decision-making. DRL agent: Receives the current state vector S_t and LSTM load prediction results, calls the trained PPO model to output the optimal action A_t (resource allocation, temperature adjustment, virtual machine migration instructions). Decision optimization unit: Verify the feasibility of the optimal action A_t, including whether the target node resources are sufficient, whether the migration path is unobstructed, and whether the temperature adjustment is within a safe range, to avoid ineffective decision execution.

[0023] The optimization and adaptation submodule includes: a resource scheduling execution engine, a server cluster control interface, a cooling equipment linkage unit, and a cross-node data transmission encryption unit. The resource scheduling execution engine issues resource adjustment commands and executes virtual machine migration. The server cluster control interface interfaces with server hardware. The cooling equipment linkage unit uses a PID algorithm to adjust the cooling system output. The cross-node data transmission encryption unit uses AES-256 encryption and data verification. The feedback closed-loop submodule includes: an effect acquisition terminal and a reward calculation module. The effect acquisition terminal collects energy consumption, response time, and resource utilization after execution. The reward calculation module calculates the actual reward value R_t according to the reward function and feeds it back. Specifically, the optimization execution and adaptation submodule: resource scheduling execution engine: receives resource adjustment instructions in the optimal action A_t, issues CPU and memory quota adjustment commands through the server cluster control interface, and calls the virtual machine migration tool to complete the migration according to the optimal path planned by Dijkstra's algorithm; Server cluster control interface: Serves as a hardware interface channel, enabling the interaction between software commands and server hardware; Refrigeration equipment linkage unit: Receives temperature adjustment commands and uses a PID control algorithm. , =0.8, =0.8 seconds, =2 seconds, precisely adjust the output of the cooling system to ensure that the temperature control deviation is ≤0.5℃; Cross-node data transmission encryption unit: The AES-256 encryption algorithm is used to encrypt virtual machine migration data, and a data verification subunit is used to verify data integrity to prevent data leakage or tampering. Feedback closed-loop subunit: Effect acquisition terminal: Within 5 minutes after the execution of the optimal action A_t, it collects energy consumption data, business response time, and resource utilization data to provide a basis for reward calculation; Reward calculation module: Calculates the actual reward value R_t according to the reward function R set in weight 4, and feeds it back to the DRL agent to update the model policy network parameters, forming a closed-loop optimization.

[0024] The real-time monitoring submodule includes: an optimization effect monitoring panel and data visualization tools; the model iteration optimization submodule includes: a data archiving server, parameter iteration tools, and an optimization effect verification subunit; the anomaly early warning and emergency response subunit includes: an anomaly detection module, a tiered early warning terminal, and an emergency decision-making unit; the optimization effect monitoring panel displays energy consumption reduction rate, response time compliance rate, resource utilization balance, and model decision accuracy; the data visualization tools present indicator trends using line charts and bar charts; the data archiving server archives monthly operational data, model decision logs, and optimization effect data; the parameter iteration tool uses the gradient descent method iteration formula. , For model parameters, The dynamic adjustment range is 0.0001-0.001, and the loss function is the mean square error between the predicted and actual reward values. A full iteration is performed monthly. The optimization effect verification subunits A and B test and compare metrics before and after iteration. The anomaly detection module monitors business response time, server energy consumption, and node failures. The tiered early warning terminal includes a first-level audible and visual alarm and administrator mobile app push notifications, and a second-level system pop-up and SMS reminder. The emergency decision-making unit has a built-in emergency action library; the first-level early warning executes preset decisions. Specifically, the real-time monitoring submodule includes: an optimization effect monitoring panel that displays core indicators in real time, including energy consumption reduction rate, response time compliance rate, resource utilization balance, and model decision accuracy, making it easy for managers to understand the optimization status; and a data visualization tool that presents indicator trends in line charts and bar charts, supporting switching between real-time / hourly / day views. Model Iteration and Optimization Submodule: Data Archiving Server: Archives data center operation data, model decision logs, and optimization effect data monthly. It adopts a distributed storage architecture, supports multi-dimensional retrieval, and provides data support for model iteration; Parameter Iteration Tool: Iteration Formula: Formula annotation: These are the model parameters (including policy network weights and bias terms). These are the new parameters after iteration; These are the old parameters before the iteration; The learning rate is dynamically adjusted from 0.0001 to 0.001 (adaptively based on the rate of decrease of the loss function). The gradient of the loss function is given. The loss function is defined as the mean squared error between the predicted reward value and the actual reward value. The iteration frequency is: one full iteration is performed every month to optimize the model parameters based on the data from the past month and avoid model rigidity. Optimization effect verification sub-unit: compare the energy consumption reduction rate and response time achievement rate before and after iteration through A and B tests. If the target is not achieved, iterate again. Anomaly Warning and Emergency Response Subunit: Anomaly Detection Module: Real-time monitoring of business response time; exceeding the threshold by 50% triggers a Level 1 warning; a sudden increase in server power consumption of 20% triggers a Level 2 warning; node failure and connectivity interruption trigger a Level 1 warning; Tiered Warning Terminal: Level 1 warning triggers audible and visual alarms (sound intensity ≥ 80dB, flashing red light, and administrator mobile APP push notification); Level 2 warning triggers system pop-ups and SMS reminders; Emergency Decision Unit: Built-in emergency action library; automatically executes preset emergency decisions during Level 1 warnings, including temporarily expanding core business resources, activating redundant nodes, and isolating faulty nodes, with a response time ≤ 10 seconds.

[0025] A method for dynamic optimization of data centers based on deep reinforcement learning further includes the following steps: Step 1: Preliminary Preparation and Environment Deployment Phase; Step 2: Data preprocessing and model training initialization phase; Step 3: Online dynamic optimization operation phase; Step 4: Closed-loop iteration and maintenance phase; The preliminary preparation and environment deployment phase includes: environment compatibility confirmation, hardware installation and deployment, software system deployment, permission configuration, and data interface debugging; the data preprocessing and model training initialization phase includes: data acquisition initiation, data preprocessing execution, data verification, model parameter configuration, historical data import, model training initiation, model convergence verification, and optimization threshold input; the online dynamic optimization and operation phase includes: system startup, real-time data acquisition and integration, load prediction, optimization decision generation, optimization action execution, execution effect collection, feedback and model update, and real-time monitoring display; the closed-loop iteration and maintenance phase includes: daily monitoring and inspection, data archiving, monthly model iteration, periodic threshold review, anomaly handling, and equipment maintenance; Specifically, Step 1: Preliminary preparation and environment deployment stage: complete hardware installation, software deployment, permission configuration, interface debugging, and build the system operating environment; Step 2: Data Preprocessing and Model Training Initialization Phase: Complete data collection, preprocessing, model training, and threshold configuration to prepare for online optimization; Step 3: Online Dynamic Optimization Operation Phase: The system runs automatically, realizing real-time perception, load prediction, decision generation, action execution, and feedback updates; Step 4: Closed-loop iteration and maintenance phase: Regularly conduct model iteration, threshold verification, anomaly handling, and equipment maintenance to ensure long-term stable operation of the system; Preliminary Preparation and Environment Deployment Phase: Environment Compatibility Confirmation: Verify server and cooling equipment models and interface compatibility; Hardware Installation and Deployment: Install sensor clusters and API interface clusters, fix device locations and connect to the network; Software System Deployment: Install core software, configure network parameters, and ensure smooth communication between modules; Permission Configuration: Assign operation permissions, enable operation logs and data encryption functions; Data Interface Debugging: Test interface connection stability and data synchronization success rate (≥99.9%). Data preprocessing and model training initialization phase: Data acquisition startup: Start the multi-source data acquisition submodule and collect data at the set frequency; Data preprocessing execution: Automatically run outlier removal and normalization processing; Data verification: Check the quality of preprocessed data (outlier data percentage ≤ 0.1%); Model parameter configuration: Configure state space, action space, and reward function parameters; Historical data import: Import preprocessed data from multiple scenarios over the past 6 months; Model training startup: Set training parameters and start iterative training; Model convergence verification: Confirm that the model meets the convergence conditions; if it does not converge, adjust the parameters and retrain; Optimization threshold entry: Enter and store the optimization constraint threshold; Online dynamic optimization operation phase: System startup: Start the online dynamic optimization execution module; Real-time data acquisition and integration: Synchronize data and generate state vector S_t; Load prediction: Run the LSTM sub-model to output load peak and confidence level; Optimization decision generation: The DRL agent outputs the optimal action A_t and verifies its feasibility; Optimization action execution: Issue resource adjustment, temperature control, and virtual machine migration instructions; Execution effect collection: Collect core indicator data after the action is executed; Feedback and model update: Calculate the reward value R_t and feed it back to the agent to update parameters; Real-time monitoring and display: The monitoring panel displays indicators and trend charts; Closed-loop iteration and maintenance phase: Daily monitoring and inspection: Review core indicators and operation logs daily; Data archiving: The system automatically archives various types of data monthly; Monthly model iteration: Activate parameter iteration tools to complete model optimization and verification; Periodic threshold review: Adjust and optimize thresholds quarterly to adapt to business changes; Anomaly handling: Receive alerts and execute emergency decisions, and record the handling process after anomalies are resolved; Equipment maintenance: Inspect hardware equipment every six months and clean up redundant data.

[0026] The method of use and working principle of this invention are as follows: Usage: First, complete the initial environment deployment, including hardware installation, software system configuration, permission allocation, and data interface debugging to build a stable operating foundation. Then, perform data preprocessing and model initialization, initiate standardized data collection and cleaning processes, configure core model parameters, import historical data for training, and set optimization constraint thresholds. Once in the online operation phase, the system will automatically conduct real-time data collection and status integration, generate and execute optimization actions through load prediction and intelligent decision-making, and collect feedback on execution results to dynamically update the model. Later, regular daily monitoring and checks, monthly model iterations, threshold verification, and equipment maintenance are required. In case of anomalies, the system will respond with warnings and execute emergency decisions to ensure long-term stable operation of the solution.

[0027] Working Principle: Operating with a closed-loop logic of perception, decision-making, execution, feedback, and iteration, the core relies on deep reinforcement learning and time-series prediction technologies for dynamic optimization. First, a multi-source data acquisition and perception module acquires comprehensive data center status information, which is then input into the system after standardization. An LSTM load prediction sub-model predicts future load trends. The DRL agent combines the current state with the prediction results, generating optimal resource allocation, environmental control, and virtual machine migration decisions based on a pre-set dual-objective reward function. The optimization execution module implements various actions according to the decision instructions, while the feedback module collects the execution results and calculates the reward value, updating the DRL model parameters in reverse. Simultaneously, a real-time monitoring module tracks the optimization status, an anomaly emergency module quickly responds to risks, and an iterative optimization module periodically adjusts model parameters and thresholds to continuously adapt to dynamic changes in the data center, achieving a dynamic balance between energy consumption and performance.

[0028] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Those skilled in the art can readily implement the present invention based on the description and drawings above. However, any modifications, alterations, and variations made by those skilled in the art without departing from the scope of the present invention using the disclosed technical content are equivalent embodiments of the present invention. Furthermore, any modifications, alterations, and variations made to the above embodiments based on the essential technology of the present invention are still within the protection scope of the present invention.

Claims

1. A method for dynamic optimization of a data center based on deep reinforcement learning, characterized in that, Comprise the following modules and components: Offline deployment and DRL model initialization module, online dynamic optimization execution module, closed loop iteration and abnormal emergency module; The offline deployment and DRL model initialization module comprises: multi-source data acquisition submodule, data standardization processing subunit, DRL model training submodule, optimization threshold configuration subunit; The online dynamic optimization execution module comprises: real-time data sensing submodule, load prediction and decision generation submodule, optimization execution and adaptation submodule, feedback closed loop subunit; The closed loop iteration and abnormal emergency module comprises: real-time monitoring submodule, model iteration optimization submodule, abnormal early warning and emergency submodule. 2.The data center dynamic optimization method based on deep reinforcement learning of claim 1, wherein, The multi-source data acquisition submodule comprises: sensor cluster, API interface cluster, data transmission trigger unit; The data standardization processing subunit comprises: outlier rejection tool, normalization processing terminal, data cache module; The DRL model training submodule comprises: model architecture design unit, PPO algorithm training platform, historical data import tool, training effect verification terminal; The optimization threshold configuration subunit comprises: threshold setting terminal, configuration file storage module. 3.The method of claim 2, wherein, The sensor cluster includes: acquisition hardware, energy consumption, business data; the API interface cluster includes: docking cloud platform, server, business monitoring system; the data transmission triggering unit includes: triggering acquisition according to set frequency and load mutation; the outlier elimination tool adopts 3 Principle identification replacement outlier; the data cache module temporarily stores preprocessed data.

4. The method of claim 3, wherein, The model architecture design unit: the state space S contains 7 dimensions of server cluster CPU, memory, network utilization, real-time total energy consumption, refrigeration system set temperature, business average response time, and high priority business proportion; the action space A contains server resource allocation, refrigeration temperature adjustment, and virtual machine migration action; the reward function R formula is: , = 0.4, = 0.5, = 0.1, and the migration cost coefficient = migration data volume x transmission delay / 1000; the PPO algorithm training platform: the initial learning rate is 0.001, the number of hidden layer nodes is 256, the iteration is 500 rounds, and the convergence is determined when the average reward fluctuation of 10 consecutive rounds is less than or equal to 5%; the historical data import tool: import preprocessed data of multiple scenes in the past 6 months. The training effect verification terminal: test set energy consumption reduction rate, response time compliance rate; The threshold setting terminal configuration core, general business response time, CPU utilization, refrigeration temperature, energy consumption target threshold; The configuration file storage module stores threshold configuration file. 5.The data center dynamic optimization method based on deep reinforcement learning of claim 1, wherein, The real-time data sensing submodule includes: a data synchronization engine and a state integration unit; the data synchronization engine synchronizes sensor and API interface data every 10 seconds; the state integration unit integrates the data into a state vector S_t recognizable by the DRL model; the load prediction and decision generation submodule includes: an LSTM load prediction submodel, a DRL agent, and a decision optimization unit; the LSTM load prediction submodel takes nearly 30 minutes of load data as input, and uses a sliding window formula: , The load fluctuation rate over the past 10 minutes is used to predict the load peak and confidence level for the next 15 minutes; the DRL agent receives the current state S_t and the load prediction result, and outputs the optimal action A_t; the decision optimization unit verifies the feasibility of the action. 6.The method of claim 1, wherein, The optimization execution and adaptation submodule comprises: resource scheduling execution engine, server cluster control interface, refrigeration equipment linkage unit, cross-node data transmission encryption unit; The resource scheduling execution engine issues resource adjustment instructions and executes virtual machine migration; The server cluster control interface interfaces with server hardware; The refrigeration equipment linkage unit PID algorithm adjusts the output of the refrigeration system; The cross-node data transmission encryption unit AES-256 encryption and data verification; The feedback closed loop subunit comprises: effect acquisition terminal, reward calculation module; The effect acquisition terminal acquires energy consumption, response time and resource utilization after execution; The reward calculation module calculates the actual reward value R_t according to the reward function and feeds back.

7. The method of claim 1, wherein, The real-time monitoring submodule includes: an optimization effect monitoring panel and a data visualization tool; the model iteration optimization submodule includes: a data archiving server, a parameter iteration tool, and an optimization effect verification subunit; the anomaly early warning and emergency response subunit includes: an anomaly detection module, a tiered early warning terminal, and an emergency decision-making unit; the optimization effect monitoring panel displays energy consumption reduction rate, response time compliance rate, resource utilization balance, and model decision accuracy; the data visualization tool presents indicator trends using line charts and bar charts; the data archiving server archives monthly operational data, model decision logs, and optimization effect data; the parameter iteration tool uses the gradient descent method iteration formula: , For model parameters, The dynamic adjustment range is 0.0001-0.001, and the loss function is the mean square error between the predicted and actual reward values. A full iteration is performed monthly. The optimization effect verification subunits A and B test and compare the indicators before and after the iteration. The anomaly detection module monitors business response time, server energy consumption, and node failures. The tiered early warning terminal includes a first-level audible and visual alarm and administrator mobile app push notifications, and a second-level system pop-up and SMS reminder. The emergency decision-making unit has a built-in emergency action library, and the first-level early warning executes preset decisions.

8. A method for deep reinforcement learning based data center dynamic optimization, applied to the modules and components of the method for deep reinforcement learning based data center dynamic optimization of any of claims 1-7, characterized in that, Further comprising the following steps: Step one: preliminary preparation and environment deployment phase; Step two: data preprocessing and model training initialization phase; Step three: online dynamic optimization running phase; Step four: closed loop iteration and maintenance phase. 9.The method of claim 8, wherein, The pre-preparation and environment deployment stage includes: environment adaptation confirmation, hardware installation deployment, software system deployment, permission configuration, data interface debugging; the data preprocessing and model training initialization stage includes: data acquisition start, data preprocessing execution, data verification, model parameter configuration, historical data import, model training start, model convergence verification, optimization threshold entry; the online dynamic optimization running stage includes: system start, real-time data acquisition and integration, load prediction, optimization decision generation, optimization action execution, execution effect acquisition, feedback and model update, real-time monitoring display; the closed-loop iteration and maintenance stage includes: daily monitoring and inspection, data archiving, model monthly iteration, threshold periodic review, exception handling, equipment maintenance.

Images