Machine room adaptive energy saving method and system based on deep reinforcement learning

Abstract

The invention provides a machine room self-adaptive energy-saving method and system based on deep reinforcement learning, and the method comprises the steps: constructing a machine room self-adaptive regulation and control model comprising an environment analysis sub-module and a strategy generation sub-module, carrying out the pre-training through employing multi-target element reinforcement learning, carrying out the analysis of real-time machine room environment data based on the machine room self-adaptive regulation and control model, and carrying out the analysis of the real-time machine room environment data; the method comprises the following steps: acquiring an initial machine room self-adaptive regulation and control strategy, performing simulation verification on the initial machine room self-adaptive regulation and control strategy by adopting a preset simulation verifier, acquiring real-time performance data of the initial machine room self-adaptive regulation and control strategy, and performing performance fluctuation root cause reasoning based on the real-time performance data. And optimizing the initial machine room self-adaptive regulation and control strategy according to a root cause reasoning result, so as to realize self-adaptive regulation and control of the machine room.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically, to a data center adaptive energy-saving method and system based on deep reinforcement learning. Background Technology

[0002] With the development of the digital economy, the energy consumption problem of data centers and various computer rooms has become increasingly prominent. First, most traditional computer room energy-saving control methods adopt rule-based control strategies based on fixed thresholds. However, this method often cannot adapt to dynamic changes in load and climate, as well as the degradation of equipment status, which easily leads to low energy efficiency in the computer room. Second, the strategies generated by traditional methods often fail to achieve dynamic trade-offs among multiple objectives such as energy saving, security, cost, and equipment lifespan, resulting in low strategy efficiency. In addition, most of the strategies generated by traditional methods are fixed and unchanging. When the strategies are run for a long time, their performance is prone to degradation, and most of them lack the ability to deeply diagnose the causes of strategy failure, resulting in inaccurate and slow control of the computer room and difficulty in achieving adaptive control of the computer room. Summary of the Invention

[0003] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide a data center adaptive energy-saving method based on deep reinforcement learning, the method comprising: An adaptive control model for the computer room, comprising an environmental analysis submodule and a strategy generation submodule, is constructed and pre-trained using multi-objective meta-reinforcement learning. Based on the adaptive control model of the data center, real-time data center environment data is analyzed to obtain the initial adaptive control strategy of the data center; The initial adaptive control strategy for the computer room was simulated and verified using a preset simulation verifier, and real-time performance data of the initial adaptive control strategy for the computer room was obtained. Root cause inference of performance fluctuations is performed based on real-time performance data, and the initial adaptive control strategy for the data center is optimized based on the results of the root cause inference.

[0004] As a further aspect of the present invention, a data center adaptive control model comprising an environmental analysis submodule and a strategy generation submodule is constructed, and pre-trained using multi-objective reinforcement learning, including: A digital twin is constructed for the data center based on its historical operating data and a set of preset security constraint rules. An environmental analysis submodule is constructed based on a graph neural network, and a self-supervised learning strategy is adopted to simultaneously combine historical operation data of the computer room for pre-training. During the training process of the environmental analysis submodule, a baseline map of the computer room operation is constructed simultaneously. The data center operation baseline map is used to store the correlation between the environmental characteristics, preset behavioral rules, and performance of the data center under different environments; A strategy generation submodule is constructed using multi-objective meta-reinforcement learning, and pre-trained using a curriculum-guided multi-objective evolutionary algorithm and meta-learning.

[0005] As a further aspect of the present invention, a strategy generation submodule is constructed using multi-objective meta-reinforcement learning, and pre-trained using a curriculum-guided multi-objective evolutionary algorithm and meta-learning, including: Based on historical data from the data center, an evolutionary algorithm is used to generate candidate strategies, and a pre-set multi-objective reward function is used for performance evaluation. The goal of pre-training is to maximize the cumulative reward. The preset multi-objective reward function includes at least energy efficiency reward, thermal safety reward, operating cost reward, and equipment lifespan reward; Based on the data generated during the operation of the evolutionary algorithm, a meta-learning task is constructed, and parallel training is performed with the goal of minimizing the preset meta-loss function. The meta-learning task includes at least the data center environment data and the weight ratio of the multi-objective reward function during the execution of the evolutionary algorithm. During the training process of the strategy generation submodule, a benchmark strategy library is constructed in parallel. The benchmark strategy library is used to store benchmark strategies under different data center environments and requirements.

[0006] As a further aspect of the present invention, real-time data center environment data is analyzed based on a data center adaptive control model to obtain an initial data center adaptive control strategy, including: Acquire real-time data of the data center environment, which includes at least data center temperature distribution, equipment operating status and data center energy consumption data. After real-time data room environment data is input into the adaptive control model of the data room, the environmental analysis submodule extracts features from the real-time data room environment data and generates a data room environment feature vector. Call upon the baseline map and baseline strategy library of the data center operation generated during the training of the data center adaptive control model; The environmental analysis submodule performs rule matching from the data center operation baseline map based on the data center environmental feature vector to obtain a set of policy constraint rules. The strategy generation submodule performs strategy matching from the benchmark strategy library based on the data center environment feature vector and strategy constraint rule set to generate an initial adaptive control strategy for the data center.

[0007] As a further aspect of the present invention, a preset simulation verifier is used to simulate and verify the initial adaptive control strategy of the data center, and real-time performance data of the initial adaptive control strategy of the data center is obtained, including: The initial adaptive control strategy for the computer room is injected into the preset simulation calibrator for simulation, and the performance is evaluated based on the simulation results to generate performance evaluation results. If the performance evaluation results do not meet the preset performance conditions, the initial adaptive control strategy of the data center will be optimized based on the adaptive control model of the data center, and the simulation and performance evaluation will be carried out again. If the performance evaluation results meet the preset performance conditions, the initial data center adaptive control strategy will be executed, and the corresponding real-time performance data will be obtained synchronously. The real-time performance data includes at least real-time data center temperature distribution, real-time equipment operating status, and real-time data center energy consumption data.

[0008] As a further aspect of the present invention, performance fluctuation root cause inference is performed based on real-time performance data, and the initial data center adaptive control strategy is optimized based on the results of the root cause inference, including: Compare the real-time performance data with the expected performance data of the initial adaptive control strategy for the data center; If the comparison results do not meet the preset performance fluctuation conditions, the performance is determined to be stable and no action is taken. If the comparison results meet the preset performance fluctuation conditions, it is determined to be a performance fluctuation; For the initial adaptive control strategy of the data center that exhibits performance fluctuations, root cause inference of performance fluctuations is performed to obtain the root cause inference results; Based on the root cause reasoning results, the adaptive control model for the data center optimizes the initial adaptive control strategy for the data center.

[0009] As a further aspect of the present invention, the performance fluctuation root cause inference is performed on the initial adaptive control strategy of the data center that exhibits performance fluctuations, and the root cause inference results are obtained, including: Inverse reinforcement learning is used to perform inverse reasoning on real-time performance data to obtain the actual policy network parameter set corresponding to the current initial adaptive control policy of the data center; Obtain the original policy network parameter set corresponding to the current initial adaptive control policy for the computer room; Attribution analysis is performed based on the actual policy network parameter set and the original policy network parameter set to generate root cause inference results. The root cause inference results include at least performance fluctuation type labels, strategy behavior offset parameters, data center environment change parameters, and fluctuation attribution parameter sets.

[0010] Furthermore, embodiments of the present invention also provide a data center adaptive energy-saving system based on deep reinforcement learning, comprising: The model building module is used to build an adaptive control model for the computer room and pre-train it using multi-objective meta-reinforcement learning. The adaptive control model for the computer room includes an environmental analysis submodule and a strategy generation submodule. The data analysis module is used to analyze real-time data center environment data according to the data center adaptive control model and generate an initial data center adaptive control strategy. The simulation verification module is used to perform simulation verification of the initial adaptive control strategy of the computer room according to a preset simulation verifier. The performance monitoring module is used to monitor and acquire real-time performance data corresponding to the initial adaptive control strategy of the computer room in real time. The root cause reasoning module is used to perform root cause reasoning on performance fluctuations and optimize the initial adaptive control strategy for the data center based on the results of the root cause reasoning.

[0011] Compared with the prior art, the present invention has the following beneficial effects: By employing a progressive learning strategy to alternately train the constructed adaptive control model of the computer room through multi-objective evolution and meta-learning, the model learns the characteristic patterns of various complex operating conditions from steady state to fault in a pre-constructed digital twin environment. During the model training process, an environmental map and a benchmark strategy library are constructed, laying a data foundation for subsequent intelligent analysis and strategy generation. By analyzing the data center status in real time through graph neural networks and simultaneously combining meta-policies to quickly retrieve and assemble strategies from the benchmark strategy library, a customized strategy that takes into account both operational goals and historical experience can be generated in a short time, thereby improving the efficiency and applicability of strategy generation. By continuously monitoring the performance fluctuations of the execution strategy and conducting root cause analysis on the performance fluctuation strategy, the applicability of the current execution strategy can be accurately controlled. The execution strategy can be optimized and adjusted based on the results of the root cause analysis, thereby achieving adaptive control of the data center. Attached Figure Description

[0012] Figure 1 This is a flowchart of the steps of an adaptive energy-saving method for computer rooms based on deep reinforcement learning, as described in this invention. Figure 2 This is a schematic diagram of an adaptive energy-saving system for computer rooms based on deep reinforcement learning, according to the present invention. Detailed Implementation

[0013] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0014] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 This is a flowchart of the steps of an adaptive energy-saving method for computer rooms based on deep reinforcement learning according to the present invention. The following is a detailed introduction to this adaptive energy-saving method for computer rooms based on deep reinforcement learning.

[0015] Step S1: Construct an adaptive control model for the computer room that includes an environmental analysis submodule and a strategy generation submodule, and pre-train it using multi-objective meta-reinforcement learning.

[0016] It should be noted that the model training process takes place within a pre-constructed digital twin. This digital twin is a high-fidelity digital twin built for the data center based on historical operating data and a pre-defined set of security constraint rules. Specifically, firstly, the physical framework of the digital twin is constructed by integrating geometric structures from the Building Information Modeling (BIM), equipment parameters of the HVAC system such as chiller COP curves, pump and fan characteristic curves, and duct resistance coefficients, power and heat dissipation models of server racks, and local outdoor meteorological data from multiple years. Then, a reduced-order model or node network model is used to train the data center. Thermal fluid modeling is used to simulate airflow, temperature distribution, and hotspot formation within the computer room. For example, the computer room space is discretized into multiple control volumes, and the heat exchange between each control volume and adjacent control volumes, air conditioning supply, and server heat generation is calculated. Subsequently, the parameters of the digital twin are adjusted based on historical operating data of the computer room to simulate the dynamic response characteristics, delays, and energy consumption of actuators such as chillers, water pumps, cooling towers, fan inverters, and air valves during operation. For example, the action time required for a chilled water valve to change from 10% to 50% opening and the nonlinear changes in flow rate during this process are simulated.

[0017] Then, based on the historical operating data of the data center, various disturbance inputs are defined. The disturbance types include at least: IT load, i.e., the baseline load and random fluctuations of the data center equipment; equipment failures, such as the shutdown of some air conditioning terminal fans, the efficiency reduction of chiller units, and sensor reading drift; and external energy consumption information, such as time-of-use electricity pricing information. Finally, after completing the above operations, the digital twin is constructed. By adjusting the corresponding parameters of the digital twin, diverse training scenarios can be generated. For example, a scenario can be generated where the outdoor dry-bulb temperature is 35°C, the wet-bulb temperature is 28°C, the IT load rate in the data center reaches 85% with fluctuations of ±5%, and the efficiency of a precision air conditioning fan in the northeast corner of the data center decreases by 20%. This scenario can be used to simulate the state of the data center under high-temperature conditions in summer. By executing training decisions within this digital twin and monitoring the chain reactions that conform to physical laws caused by these decisions, such as adjusting the chilled water valve from 10% opening to 50% opening, the corresponding area will experience a temperature drop and an increase in the total power consumption of the water pump, thereby training the model to learn the corresponding evolution patterns and characteristics of the data center.

[0018] Specifically, an environmental analysis submodule is constructed based on a graph neural network, and a self-supervised learning strategy is adopted to simultaneously combine historical operating data of the computer room for pre-training.

[0019] In one possible embodiment, a graph neural network is used to model the data center as a heterogeneous graph, in which each node represents a physical entity, such as a temperature sensor, an air conditioning terminal, or a rack group. Node attributes include their real-time measurements or status, such as temperature, power consumption, and opening degree. Edge connections between nodes are used to characterize known physical connections or strong correlation information between entities, such as the sensor being located downstream of an air conditioning terminal or the information that two racks are adjacent.

[0020] The pre-training process of the environmental analysis submodule is divided into two stages: self-supervised training and supervised graph construction. Specifically, in the self-supervised training stage, firstly, a large number of environmental state maps at different times are randomly sampled from a preset digital twin weight, such as the state maps of the sensor network at different times, and these environmental state maps are used as the training dataset. Next, 15% to 30% of the features of the nodes in the heterogeneous graph are randomly masked, such as setting their temperature value to 0. The graph neural network is required to predict the masked feature values ​​based on the masked graph structure. The loss function in this stage is the mean square error between the predicted value and the true value. Immediately afterwards, two random data augmentations are applied to the same environmental state map in the training dataset, such as adding small noise and randomly discarding a small number of edges, to obtain a positive sample pair. At the same time, the environmental state maps of the corresponding nodes in the environmental state map at different times or in different scenarios relative to the current state are obtained and used as negative sample pairs. The encoder of the graph neural network is trained based on the positive and negative sample pairs. The encoder is trained to make the distance between positive samples in the feature space as close as possible and the distance between negative samples as far as possible. The loss function in this stage is the InfoNCE loss.

[0021] For example, a sensor map of a computer room representing "local overheating" is randomly masked. Let's say the masked area is area A, where sensor temperatures are generally high. The masked map is then input into a graph neural network. The graph neural network learns to infer that the temperature in area A may be high based on the temperatures of adjacent areas B and C and the air conditioning operation, even with the readings of several key sensors in area A masked. It then marks area A as "local overheating," thereby training the graph neural network's ability to extract environmental features from the raw data.

[0022] In the supervised graph construction phase, this phase utilizes the interaction data generated by the strategy generation submodule during the execution of the evolutionary algorithm. This interaction data is organized in the form of quadruplets, namely (environment state graph, environment type label, list of effective behavior rules, performance index vector). The labeled quadruplet data is input into the graph neural network, and the encoder based on the graph neural network extracts environmental features. Weighted fusion is performed according to the preset multi-task learning heads and corresponding loss functions, and backpropagation is performed based on the calculated loss. The weight ratio of the loss function of each learning head is a fixed value obtained from historical data analysis, thereby optimizing the corresponding parameters of the environment analysis submodule.

[0023] It should be noted that the multi-task learning head includes: an environment classification head, which consists of a fully connected layer and is used to generate the probability distribution of environment types. Its loss function is classification cross-entropy loss, and the environment types are predefined environment category labels, such as labels for high temperature and high humidity steady state, local overheating burst, load spike, and normal fluctuation; a behavior association prediction head, which consists of a multi-label classification layer and is used to predict which rules in the preset behavior rule library are effective in the current environment. Each rule corresponds to a binary classification. For example, the preset behavior rule library includes "start the backup chiller when the return air temperature is 2°C higher than the set value" and "use natural cooling mode when the outdoor wet-bulb temperature is lower than 15°C". This task head uses binary cross-entropy loss as its loss function; and a performance regression head, which is used to predict the values ​​that key performance indicators such as PUE, maximum hot spot temperature, and total power consumption can be expected to reach if the most effective combination of behavior rules is adopted in the current environment. Its loss function is mean squared error loss.

[0024] Understandably, during the training process of the environmental analysis submodule, a baseline map of the computer room operation is constructed simultaneously. This baseline map stores the correlation between environmental characteristics, preset behavioral rules, and performance of the computer room under different environments. Specifically, during training, whenever the graph neural network makes a high-confidence accurate prediction for a training sample, a knowledge record is generated simultaneously. For example, for the sample environment map E, the graph neural network classifies it as "high temperature and high humidity" with 95% confidence, and associates behavioral rules R1 and R3 with 85% and 91% confidence respectively, and predicts the value of performance index P. Then, an environment state node is created based on the feature vector of the sample environment map E. The high temperature and high humidity environmental label is used as an attribute of this node. The nodes corresponding to behavioral rules R1 and R3 are linked by edges with correlation relationships, and the normalized value of the confidence between nodes is used as the edge weight of the corresponding edge connection. At the same time, behavioral rule nodes R1 and R3 are linked to performance node P. As the training process progresses, an environment map is finally formed.

[0025] For example, based on the analysis of the sensor map of the computer room corresponding to the local overheating in area A, the environment classification head will classify it as "local overheating outbreak". The behavior association prediction head will retrieve two rules from the rule base that are highly relevant to this environment: "increase the air supply speed of the air conditioning terminal in the corresponding overheated area" and "slightly reduce the cold water supply temperature". The performance regression head will predict that if these rules are executed, the hot spot temperature in area A can be reduced by 2°C within 5 minutes, while the power consumption will increase by 8%. This generates the knowledge that "in the environment of local overheating outbreak, taking the combined action of increasing air supply and reducing water temperature is expected to quickly cool down the area at a certain energy cost", and this knowledge node and edge are solidified into the environment map.

[0026] It should be noted that the nodes within the environmental graph include: environmental state nodes, which store an environmental feature vector encoded by a graph neural network. This environmental feature vector represents a specific environmental situation and is labeled with an environmental type, such as "high temperature and high load at midday" or "sudden local overheating." Its node attributes at least include the historical frequency and duration of this state; policy behavior rule nodes, which describe a specific control rule in natural language, as well as the triggering conditions and applicable parameter range, such as "when the temperature in a certain area exceeds the set value by 2°C, prioritize increasing the air supply volume of the corresponding air conditioning terminal"; performance nodes, whose node attributes include a set of specific performance indicators, which at least include energy efficiency value, thermal safety violation duration, economic cost, and equipment wear; and policy entity nodes, which represent pointers to specific policies in the benchmark policy library and are used to store the unique identifier of the benchmark policy and the path to obtain its policy network parameters.

[0027] Nodes within the environmental graph are connected by the following edges: Causal Influence Edges, connecting two nodes with causal relationships, such as from the "Outdoor Wet Bulb Temperature" node to the "Natural Cooling Tower Efficiency" node, with edge attributes including the average causal effect strength coefficient calculated from historical data, the uncertainty range of the effect, and the data index used to calculate the effect; Triggering Edges, pointing from a specific environmental state node to a policy behavior rule node, indicating that this policy behavior rule is recommended to be enabled in this environmental state, with edge attributes including the confidence level of the trigger, the number of historical activations, and typical environmental parameters at the time of triggering; Guiding Edges, pointing from a policy behavior rule node to a performance node, indicating the expected performance result after taking this action, with edge attributes including the vector of expected performance values, the variance of the actual result, and the reliability score of the relationship.

[0028] Furthermore, based on historical operating data of the data center, an evolutionary algorithm is used to generate candidate strategies, and a preset multi-objective reward function is used for performance evaluation. The strategy generation sub-module is pre-trained with the goal of maximizing the cumulative reward. The preset multi-objective reward function includes at least energy efficiency reward, thermal safety reward, operating cost reward, and equipment lifespan reward.

[0029] A meta-learning task is constructed based on the data generated during the operation of the evolutionary algorithm. Parallel training is performed with the goal of minimizing the preset meta-loss function. The meta-learning task includes at least the data center environment data and the weight ratio of the multi-objective reward function corresponding to the operation of the evolutionary algorithm.

[0030] It should be noted that the strategy generation submodule is represented as a meta-controller, which is used to quickly match and fine-tune a control strategy that fits the target preferences from the benchmark strategy library given a new environmental feature and a set of target preferences. The benchmark strategy library is constructed synchronously during the training process of the strategy generation submodule. A series of training courses with increasing difficulty are designed using a progressive course learning strategy, such as Course 1: constant load and outdoor temperature; Course 2: adding day and night temperature and humidity changes; Course 3: adding random load fluctuations; Course 4: simulating partial equipment failure, etc., and within each training course, multiple rounds of multi-objective evolution and meta-learning are performed alternately.

[0031] The cycle of alternating multi-round multi-objective evolution and meta-learning is represented as follows: starting from course 1, a policy population is randomly initialized, and a multi-objective evolutionary algorithm is run in the environment of course 1. For example, it is run for 20 generations to obtain a Pareto optimal policy set. Representative policies are selected from this set as the baseline policies in the baseline policy library. Based on all the data generated during this evolution process, a large number of meta-learning tasks are constructed. The policy generation submodule is trained with multi-round model-independent meta-learning based on the meta-learning tasks. After entering course 2, the excellent policy population obtained at the end of course 1 is used as part of the initial population. New random individuals are introduced through crossover and mutation. The operations in course 1 are repeated until all the preset courses are completed.

[0032] For example, in the evolutionary stage of Course 2, strategy A emerges in the population: it aggressively uses natural cooling during low nighttime temperatures, resulting in excellent energy savings, but switching is slightly slower during daytime temperature rises; strategy B always retains some mechanical cooling as a buffer, resulting in slightly less energy savings but more stable temperatures throughout the day. Assuming both are solutions on the Pareto front, they are stored in the strategy library, with strategy A being labeled as energy-saving and strategy B as safety-oriented. In the subsequent meta-learning stage, the meta-task generated based on this evolutionary process is: the environment is "the warming period transitioning from dawn to daytime," and the objective weights are [energy saving: 0.6, safety: 0.4]. During the meta-learning training stage, by learning this meta-task, it is determined that when facing similar situations, the optimal initial strategy should be between A and B. That is, when encountering similar situations online, the meta-policy may learn to initialize a strategy network whose parameters are close to strategy A in natural cooling control logic and close to strategy B in mechanical cooling buffer logic. This strategy network can then be fine-tuned to reach its optimum, generating the initial adaptive control strategy for the data center.

[0033] In one possible implementation, at the beginning of each round of the loop, a policy population is initialized, where each individual is an executable policy neural network, such as the policy network of a deep deterministic policy gradient algorithm. The evolutionary objective is to find a Pareto optimal policy set on multiple competing objectives under the current course-defined environmental distribution. It should be noted that in this embodiment, four objectives are taken as examples: energy efficiency E, thermal safety S, operating cost C, and equipment life L.

[0034] Specifically, each strategy will be simulated and run under different environmental instances in the current course, such as different initial temperatures, different load curve segments, and different objective weight preferences, such as emphasizing energy saving. Its long-term reward will be calculated based on a multi-objective reward function, which can be expressed as: , in, Represented as the total reward value, These are the weighting coefficients for four objectives: energy efficiency (E), thermal safety (S), operating cost (C), and equipment lifespan (L). These are the reward values ​​corresponding to the four objectives: energy efficiency (E), thermal safety (S), operating cost (C), and equipment lifespan (L).

[0035] At this point, each strategy in the strategy population will receive a set of reward scalars under different (environment, weight) conditions, as well as a cumulative value vector on each original objective (E, S, C, L). The NSGA-II algorithm is used for non-dominated ranking and elite selection. That is, the strategy is non-dominated based on its performance on each original objective. Strategies that are not superior to any other strategy on all objectives are ranked on the Pareto front. Then, within the same non-dominated layer, the crowding distance is calculated to select individuals that are sparsely distributed in the objective space and can increase population diversity. Through tournament selection, simulated crossover, and mutation, a new generation of the population is generated. After the evolutionary process continues for dozens of generations, the population will converge to a series of Pareto optimal strategies in the current course environment. From the final Pareto front, N strategies covering different trade-offs are selected, namely, strategies that prefer energy saving and take into account both energy saving and thermal safety. The selected strategies are used as the benchmark strategies. The benchmark strategies include at least its complete neural network parameters, the environmental feature vector corresponding to the test scenario, the performance vector under each test scenario, and the reward weight preference vector obtained by back-inferring its behavioral trajectory through anti-reinforcement learning.

[0036] For example, suppose the policy network corresponding to the baseline policy is a neural network with three fully connected layers. Then its neural network parameters are the numerical set of weight matrices and bias vectors between all layers. Suppose that the current baseline policy focuses on energy consumption and thermal violation duration as performance evaluation indicators. Then its performance vector can be represented as {Scenario 1: [Energy consumption: 1500 kWh, thermal violation duration: 0.5 hours], Scenario 2: [Energy consumption: 1800 kWh, thermal violation duration: 2 hours], ...}. The reward weight preference vector is represented by reasoning about behavioral trajectories such as state sequences, action sequences, and reward sequences during the training process through methods such as maximum entropy inverse reinforcement learning. The corresponding multi-objective reward function weight vector is derived. Suppose that the weight vector is [0.6, 0.3, 0.05, 0.05]. This indicates that the behavior preference of the baseline policy is 60% focused on energy saving, 30% focused on thermal safety, and does not pay attention to operating costs and equipment wear and tear.

[0037] Data is sampled from the data pool generated by the evolutionary algorithm, and training tasks for meta-learning are constructed based on the collected data. For example, a task can be represented as: under the high temperature and high load environment of summer daytime, the weight vector of the multi-objective reward function is set to [0.6, 0.3, 0.1, 0.0] to evaluate the policy, generating a meta-task sequence with K tasks, i.e., {T_i|i=1, 2, ..., K}. For each task T_i, it consists of an environment instance and the corresponding multi-objective reward function weight vector, where the environment instance represents all the initial states and dynamic parameters involved in the digital twin during the evolution process, such as the environment vector corresponding to the policy. It should be noted that for each task T_i, those policies that perform well on the meta-task are selected from the data pool generated by the evolutionary algorithm, and a portion of the trajectory is sampled from the interaction history of these policies as the training support set and validation query set for the task. The training support set is used for fast adaptation in the inner loop, and the validation query set is used for meta-parameter update evaluation in the outer loop.

[0038] The meta-learning training phase employs a model-independent meta-learning algorithm. This phase of training is iterative, with each iteration containing an inner loop and an outer loop. Specifically, in the inner loop, for each task T_i in the current batch, starting with the current global parameters of the meta-policy model, the training support set data for task T_i is input into the model, and the policy gradient loss is used to calculate the model's loss on that task. Gradient updates are performed using a policy gradient update function, which can be expressed as follows: , where represents the updated global parameters. It is the learning rate of the inner loop. The loss function is a global parameter of the current meta-policy model. The gradient.

[0039] For example, suppose task T_1 is "to handle the localized overheating issue in the east rack of the data center, and the operator is currently prioritizing security." In the inner loop, the meta-policy parameter... This is used to initialize a control policy, which is trained using the training support set T_1. Assuming the training support set consists of several control sequences from past successful policies handling similar local overheating problems, the meta-policy parameters become... (The text abruptly ends here, so the translation stops as well.) At this point, the strategy represented by this new meta-strategy parameter has been initially optimized for the specific task of "handling local overheating on the east side with a focus on safety".

[0040] The outer loop is used to evaluate the performance of the parameters after policy gradient updates on their respective tasks, that is, to input the query set data of each task T_i into the loop. In a parameterized model, its value is calculated. Losses The training objective is to minimize the sum of these validation losses across all tasks; that is, the meta-loss function of the outer loop is... It should be noted that, due to Through Therefore, it is obtained by gradient update. Ultimately, it can be viewed as a function of the initial elementary parameter θ, and the elementary loss can be calculated. Regarding the original meta-parameters gradient ,based on right To perform an update, thereby achieving the adjustment of the initial meta-parameters. Starting from there, each task is quickly adapted through an inner loop. They can achieve lower validation loss on their respective tasks, enabling efficient adjustments to address various task requirements with just a few gradient updates.

[0041] For example, the adaptation parameters for task T_1 The validation query set corresponding to T_1, i.e., another unseen portion of data about handling local overheating, is used to test the results. The performance of the parameter generation strategy is evaluated to obtain a validation loss. Similarly, this operation is performed on all sampling tasks such as T_2 and T_3. Then, all the obtained validation losses are summed to obtain a corresponding meta-loss. The outer loop then updates the policy gradient based on this meta-loss to adjust the meta-initial parameters. .

[0042] By iterating through the above internal and external loop processes, the strategy generation submodule is forced to learn how to quickly fine-tune a general baseline strategy into an expert strategy specifically designed to deal with the current situation, based on the current environmental characteristics and target preferences.

[0043] Step S2: Analyze real-time data center environment data based on the data center adaptive control model to obtain the initial data center adaptive control strategy.

[0044] Specifically, real-time data center environmental data is acquired, including at least the data center temperature distribution, equipment operating status, and data center energy consumption data. After the real-time data center environmental data is input into the data center adaptive control model, the environmental analysis submodule extracts features from the real-time data center environmental data to generate a data center environmental feature vector. For example, it collects multi-source heterogeneous data such as the temperature distribution of various areas in the current data center, IT equipment load, air conditioning unit operating status, cooling water system parameters, outdoor meteorological data, and current electricity price signals, and preprocesses the above multi-source heterogeneous data. Based on the preprocessed multi-source data, the environmental analysis submodule generates a data center environmental feature vector.

[0045] Furthermore, the system calls upon the baseline map and baseline strategy library of the data center operation generated during the training of the adaptive control model. The environment analysis submodule performs rule matching from the baseline map of the data center operation based on the data center environment feature vector to obtain the set of strategy constraint rules. The strategy generation submodule performs strategy matching from the baseline strategy library based on the data center environment feature vector and the set of strategy constraint rules to generate the initial adaptive control strategy for the data center.

[0046] In one possible embodiment, the data center environment feature vector and the environment graph are interactively queried and similarity matched. Specifically, the cosine similarity between the current data center environment feature vector and the feature vectors of all "environment prototype" nodes in the environment graph is calculated, and the top M environmental prototypes with the lowest cosine similarity are selected. These environmental prototype nodes are used to represent typical operating conditions summarized in history, such as "high temperature and high load," "sudden local overheating," "light load and high natural cooling potential," and "seasonal fluctuations." Starting from the selected environmental prototype nodes, the corresponding associations are obtained by traversing the relational edges in the environment graph. For example, starting from the "high temperature and high load" prototype node, through the "association" edge, we can retrieve a summary node R1 of multiple policy behavior rules that have frequently taken effect in such environments in the past. R1 is "when the temperature in a certain area is continuously higher than the set value by 2°C and the load exceeds the threshold, we will prioritize increasing the fan speed of the corresponding air conditioning terminal instead of immediately reducing the global air supply temperature". At the same time, through the "cause" edge, we can retrieve the expected performance nodes associated with rule R1. The expected performance nodes record the typical PUE improvement range, the time required to eliminate hotspots, and the percentage increase in energy consumption after applying such rules.

[0047] The strategy generation submodule receives the data center environment feature vector generated by the environment analysis submodule, the main classification tags such as local overheating, the retrieved behavioral rule set, and the corresponding expected performance. Based on the above structured information, the strategy generation submodule generates a strategy. Specifically, the strategy generation submodule combines the current data center environment feature vector and the target preference vector set by the operator into a query vector. The target preference vector is represented as a multi-objective reward function weight vector. For example, the default preference is set as [energy efficiency: 0.4, thermal safety: 0.3, operating cost: 0.2C, equipment lifespan: 0.1]. The target preference in the query vector is matched with the multi-objective reward function weight vector corresponding to the benchmark strategy in the benchmark strategy library using cosine similarity. The normalized value of the cosine similarity is used as the preference matching score. Simultaneously, the data center environment feature vector is matched with the environment feature vector corresponding to the benchmark strategy using cosine similarity. The normalized value of the cosine similarity is used as the environment matching score. The preference matching score and the environment matching score are merged using a weighted average method to obtain a comprehensive score. The top 5 benchmark strategies with the comprehensive scores are selected as the candidate strategy set.

[0048] The meta-policy network of the policy generation sub-network takes the current data center environment feature vector and target preference vector as input and outputs a set of neural network weights. Based on these neural network weights, the policy network parameters of the candidate policies in the candidate policy set are weighted and fused to generate an original data center adaptive control policy. The retrieved set of behavioral rules and their corresponding expected performance are formalized into corresponding constraint terms, and these constraint terms are injected into the meta-loss function of the outer loop. Short-term simulation is performed by combining the historical operating data of the current data center in the most recent time window, thereby updating the policy network parameters of the original data center adaptive control policy online, and finally generating the initial data center adaptive control policy.

[0049] Step S3: Use a preset simulation verifier to simulate and verify the initial adaptive control strategy of the computer room, and obtain the real-time performance data of the initial adaptive control strategy of the computer room.

[0050] Specifically, the initial adaptive control strategy for the computer room is injected into a preset simulation calibrator for simulation, and the performance is evaluated based on the simulation results to generate performance evaluation results.

[0051] Understandably, the performance evaluation focuses on the following three dimensions: energy efficiency performance dimension. This dimension is used to evaluate the power consumption and cooling efficiency of the initial adaptive control strategy for the data center under specific operating conditions. Specifically, after the strategy is executed, the total energy consumption at each moment in the digital twin is recorded. The total energy consumption includes at least the energy consumption of IT equipment, chillers, cooling towers, water pumps, and all air conditioning terminal fans. The Power Usage Effectiveness (PUE) is calculated, and the above two indicators are compared with the corresponding thresholds to complete the performance evaluation of this dimension. For example, if the average PUE of a certain strategy is 1.48 and the total power consumption is 1850 kWh, and assuming that the threshold in the current environment is PUE not higher than 1.55 and total power consumption not exceeding 2000 kWh based on historical data analysis, the strategy is deemed to have passed the performance evaluation of the energy efficiency performance dimension because the corresponding performance of this strategy meets the corresponding conditions.

[0052] The thermal safety compliance dimension is used to check for the risk of hotspot temperatures exceeding limits or uneven temperature distribution under all simulation conditions. Specifically, it monitors the highest temperature value recorded by all sensors during the simulation period, the cumulative time for any sensor temperature to exceed the safety limit, and the standard deviation of sensor temperatures in key areas. For example, a strategy is used to solve the problem of excessively high local temperatures in the East Zone rack cluster. Suppose that the simulation results of this strategy show that within 15 minutes after the strategy is implemented, the inlet air temperature of rack A15 drops from 32°C to 27.5°C and remains below 27°C thereafter. At the same time, none of the temperature sensor readings exceed the preset warning threshold of 27.5°C, and the standard deviation of the inlet air temperature of the East Zone rack cluster decreases from the initial 2.5°C to 1.2°C, which is below the threshold of 1.5°C. Then, the strategy is judged to have passed the performance evaluation of the thermal safety compliance dimension.

[0053] The control stability and equipment friendliness dimension is used to evaluate whether the control commands of the initial adaptive control strategy for the computer room are smooth and whether there are any behaviors that may damage the equipment lifespan, such as frequent start-stop operations. Specifically, it monitors the change amplitude of control commands per unit time. For large equipment such as chillers and water pumps, it counts the number of start-stop operations within the simulation cycle and observes whether key controlled variables such as temperature and pressure exhibit unnecessary and continuous periodic fluctuations near the set values. For example, after a certain strategy is implemented, the process of increasing the fan frequency from 45Hz to 50Hz is completed gradually in 6 steps within 3 minutes, and the maximum step change amplitude is 1Hz / 30 seconds. Assuming that the allowable change amplitude threshold of the equipment is 5Hz / minute, it indicates that the strategy is smooth enough. At the same time, during the entire simulation, both chillers maintain stable operation without any unnecessary start-stop operations, and the temperature of the main areas remains stable near the set value after adjustment. In this case, the strategy is judged to pass the performance evaluation of the control stability and equipment friendliness dimension.

[0054] The initial adaptive control strategy for the data center will only be executed if the preset energy efficiency, safety and stability thresholds are met in all three performance evaluation dimensions mentioned above. The corresponding real-time performance data will be acquired simultaneously. The real-time performance data includes at least the real-time data center temperature distribution, real-time equipment operating status and real-time data center energy consumption data.

[0055] If any performance evaluation dimension fails to meet the preset performance conditions, the initial data center adaptive control strategy will be optimized based on the data center adaptive control model. For example, if the preset performance conditions are not met in the thermal safety compliance dimension, a feedback signal will be generated. This signal will be used to guide the increase of the weight ratio of the thermal safety reward item in the target preference vector, and the simulation and performance evaluation will be carried out again.

[0056] Step S4: Perform root cause inference of performance fluctuations based on real-time performance data, and optimize the initial adaptive control strategy for the data center based on the results of the root cause inference.

[0057] Specifically, the real-time performance data is compared with the expected performance data of the initial adaptive control strategy for the data center. If the comparison result does not meet the preset performance fluctuation conditions, it is determined that the performance is stable and no action is taken; if the comparison result meets the preset performance fluctuation conditions, it is determined that the performance is fluctuating.

[0058] Specifically, based on real sensor data and multi-objective reward functions in real-time performance data, the actual reward value obtained by the initial data center adaptive control strategy in the current period is obtained, the expected reward value predicted by the digital twin in the same period is obtained simultaneously, the actual reward value and the expected reward value are continuously monitored, and the actual reward value sequence and the expected reward value sequence are generated respectively.

[0059] A sequential probability ratio test is used to test the hypothesis between the actual reward value sequence and the expected reward value sequence. Specifically, two hypotheses are set: a null hypothesis and an alternative hypothesis. The null hypothesis indicates that the strategy performance has not degraded, that is, the distributions of the actual reward value sequence and the expected reward value sequence are consistent. The alternative hypothesis indicates that the strategy performance has substantially degraded, that is, the mean of the actual reward value sequence is lower than the mean of the expected reward value sequence. When the test results reject the null hypothesis with high confidence and accept the alternative hypothesis, it is determined that the currently implemented initial data center adaptive control strategy has degraded in performance. At this time, the comparative data including the time when the performance degradation started, the relevant state sequence, the action sequence, and the expected reward value sequence are encapsulated into a degradation data package.

[0060] Furthermore, inverse reinforcement learning is used to perform inverse reasoning on the real-time performance data corresponding to the initial adaptive control strategy of the data center that exhibits performance fluctuations, to obtain the actual strategy network parameter set corresponding to the current initial adaptive control strategy of the data center, and to obtain the original strategy network parameter set corresponding to the current initial adaptive control strategy of the data center. Attribution analysis is performed based on the actual strategy network parameter set and the original strategy network parameter set to generate root cause reasoning results.

[0061] In one possible embodiment, the corresponding data in the degraded data packet is injected into the environmental analysis submodule, and a series of data center environmental feature vectors are output. Then, each feature vector is used as a query to match the environmental state node corresponding to the environmental prototype with the highest similarity in the environmental map. For example, the data center environmental feature vector E1 matches the "mild weather steady-state operation node" in the environmental map with a cosine similarity of 0.85, and the data center environmental feature vector E2 matches the "high temperature and high humidity natural cooling failure node" in the environmental map with a cosine similarity of 0.88.

[0062] The maximum entropy inverse reinforcement learning algorithm is used to analyze the corresponding data in the degraded data packets to obtain the actual weight vector implied by the actions executed by the current policy during the degradation period. For example, the weight vector inferred by inverse reinforcement learning is [0.75, 0.10, 0.10, 0.05]. This weight vector indicates that during the degradation period, the policy pursues a reward function that prioritizes energy saving and almost ignores equipment lifespan. Simultaneously, the initial weight vector corresponding to the initial adaptive control policy of the data center before its execution is obtained. The weight offset vector between the actual weight vector and the initial weight vector is calculated, and statistical significance is verified for each dimension of the weight offset vector. A significant offset is considered to have occurred only when the offset corresponding to a dimension exceeds the confidence interval. For example, if the weight offset vector is [+0.25, -0.10, -0.10, -0.05], and the confidence interval obtained through significance verification is that an offset of not less than 0.1 is considered a significant offset, then [+0.25, -0.10, -0.10, 0] is used as the policy behavior offset parameter. Since the offset corresponding to the device lifetime dimension does not exceed the confidence interval, it is directly set to 0 here, indicating that when performing root cause reasoning in the future, only the three dimensions of energy efficiency, thermal safety, and operating cost need to be considered, and the device lifetime dimension does not need to be considered.

[0063] Based on the magnitude of the offset of each dimension in the weight offset vector, starting from the node matched from the environmental graph, reverse reasoning is performed along the corresponding causal influence edge to trace the causal path that caused the offset of each dimension. Taking the analysis of the thermal safety dimension offset in the above weight offset vector as an example, assuming that the hot spot temperature rises due to neglecting the thermal safety dimension, the "high temperature and high humidity natural cooling failure node" is matched from the environmental graph by performing cosine similarity matching. Starting from this node, tracing and reasoning are performed in the reverse direction along the causal influence edge in the environmental graph. Assuming that the causal chain of tracing is: the direct cause of the hot spot temperature rise is insufficient rack cooling; the insufficient cooling is because the chiller unit capacity has reached saturation and cannot provide additional cooling capacity; the root cause of the unit capacity saturation is the sharp drop in the efficiency of the natural cooling tower; finally, the root cause at the very top is the abnormal surge in outdoor wet-bulb temperature.

[0064] Assuming the causal link traces from the outdoor wet-bulb temperature node to the natural cooling tower efficiency node, a dual machine learning approach is used to isolate the net impact of the outdoor wet-bulb temperature causal variable on the natural cooling tower efficiency outcome variable, while controlling for confounding factors such as server load and air conditioning fan speed. This allows for the estimation of the current causal effect strength. Assuming the current estimated causal effect strength is -0.9, this means that for every degree Celsius increase in temperature, cooling efficiency decreases by 0.9 units, and this effect strength is 80% stronger than the historical average. Simultaneously, the confidence level of the triggering edge from the "natural cooling tower efficiency node" to the "adopting enhanced local airflow behavior rule node" has plummeted from 0.9 under normal conditions to 0.2. These two pieces of information indicate a loss of cooling capacity and the ineffectiveness of the original strategy. The failure in the current environment is the cause of the hotspot temperature rise. Therefore, it is inferred that the performance degradation of the current execution strategy is due to its reliance on localized airflow, which has failed in the new data center environment, preventing the achievement of the target. This performance degradation is ultimately labeled as a performance fluctuation type caused by environmental mutation, and {natural cooling tower efficiency drop: 0.8, strategy failure: 0.7} is used as the fluctuation attribution parameter set. The core data center environmental change parameter, outdoor wet-bulb temperature, is obtained. It is assumed that within 45 minutes, the outdoor wet-bulb temperature rapidly rises from 22.0℃ to 26.2℃, with an absolute change of +4.2℃ and a change rate of 5.6℃ / h (this is just an example of the general process of attribution analysis; specific verification is required based on actual conditions).

[0065] It should be noted that in this example, the real-time quantification of confidence is represented by the static confidence corresponding to the edge attribute as the prior probability. When the system is in a specific environmental state, the actual effect after adopting a certain behavior rule is continuously monitored to see the degree of consistency between the actual effect and the performance node pointed to by the rule recorded in the graph. If the rule is expected to improve efficiency or security, but the real-time data shows that the relevant indicators deteriorate or have no significant improvement, then a predefined likelihood function will be used to reduce the confidence of the rule in the current environment.

[0066] Based on the root cause reasoning results, the adaptive control model for the data center optimizes the initial adaptive control strategy. For example, in the above case, the reason for the degradation of the strategy performance is an unforeseen environmental change rather than an internal strategy failure. Therefore, fine-tuning the parameters based on the current strategy is unlikely to achieve the expected results. In this case, new strategy behavior rules and benchmark strategies will be matched with the environmental graph and the benchmark strategy library to regenerate and execute a new strategy. Alternatively, the environmental graph can be searched to see if there are similar cases. If there are, only the strategy used in the case needs to be fine-tuned. If there are no similar cases, a new strategy still needs to be regenerated.

[0067] It should be understood that, in the embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0068] Figure 2 This is a schematic diagram of an adaptive energy-saving system for computer rooms based on deep reinforcement learning, according to the present invention.

[0069] Specifically, a data center adaptive energy-saving system based on deep reinforcement learning includes: The model building module is used to build an adaptive control model for the computer room and pre-train it using multi-objective meta-reinforcement learning. The adaptive control model for the computer room includes an environmental analysis submodule and a strategy generation submodule.

[0070] The data analysis module is used to analyze real-time data center environment data according to the data center adaptive control model and generate an initial data center adaptive control strategy.

[0071] The simulation verification module is used to perform simulation verification of the initial adaptive control strategy of the computer room according to the preset simulation verifier.

[0072] The performance monitoring module is used to monitor and acquire real-time performance data corresponding to the initial adaptive control strategy of the data center.

[0073] The root cause reasoning module is used to perform root cause reasoning on performance fluctuations and optimize the initial adaptive control strategy for the data center based on the results of the root cause reasoning.

[0074] An electronic device, comprising: At least one processor; and at least one memory communicatively connected to the processor; wherein the memory stores instructions executable by at least one processor, the instructions being executed by at least one processor to enable at least one processor to perform the method proposed in Embodiment 1 of the present invention.

[0075] The following is a detailed introduction to the various components of the electronic device: In this context, the processor is the control center of the electronic device. It can be a single processor or a collective term for multiple processing elements. For example, a processor can be one or more central processing units (CPUs), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement Embodiment 1 of this invention, such as one or more digital signal processors (DSPs) or one or more field-programmable gate arrays (FPGAs).

[0076] The processor can perform various functions of an electronic device by running or executing software programs stored in memory and by calling data stored in memory.

[0077] The memory is used to store the software program that executes the solution of the present invention, and the execution is controlled by the processor. For specific implementation methods, please refer to the above method embodiments, which will not be repeated here.

[0078] The memory can be a real-only memory (ROM) or other type of static storage device capable of storing static information and instructions, a random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only (CD-ROM), or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. The memory can be integrated with the processor or exist independently and coupled to the processor through an interface circuit of an electronic device; this embodiment of the invention does not specifically limit this.

[0079] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via limited means (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0080] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A data center adaptive energy-saving method based on deep reinforcement learning, characterized in that, It includes the following steps: An adaptive control model for the computer room, comprising an environmental analysis submodule and a strategy generation submodule, is constructed and pre-trained using multi-objective meta-reinforcement learning. Based on the adaptive control model of the data center, real-time data center environment data is analyzed to obtain the initial adaptive control strategy of the data center; The initial adaptive control strategy for the computer room was simulated and verified using a preset simulation verifier, and real-time performance data of the initial adaptive control strategy for the computer room was obtained. Root cause inference of performance fluctuations is performed based on real-time performance data, and the initial adaptive control strategy for the data center is optimized based on the results of the root cause inference.

2. The adaptive energy-saving method for computer rooms based on deep reinforcement learning according to claim 1, characterized in that, An adaptive control model for the computer room, comprising an environmental analysis submodule and a strategy generation submodule, is constructed and pre-trained using multi-objective meta-reinforcement learning, including: A digital twin is constructed for the data center based on its historical operating data and a set of preset security constraint rules. An environmental analysis submodule is constructed based on a graph neural network, and a self-supervised learning strategy is adopted to simultaneously combine historical operation data of the computer room for pre-training. During the training process of the environmental analysis submodule, a baseline map of the computer room operation is constructed simultaneously. The data center operation baseline map is used to store the correlation between the environmental characteristics, preset behavioral rules, and performance of the data center under different environments; A strategy generation submodule is constructed using multi-objective meta-reinforcement learning, and pre-trained using a curriculum-guided multi-objective evolutionary algorithm and meta-learning.

3. The adaptive energy-saving method for computer rooms based on deep reinforcement learning according to claim 2, characterized in that, A strategy generation submodule is constructed using multi-objective meta-reinforcement learning, and pre-trained using a curriculum-guided multi-objective evolutionary algorithm and meta-learning, including: Based on historical data from the data center, an evolutionary algorithm is used to generate candidate strategies, and a pre-set multi-objective reward function is used for performance evaluation. The goal of pre-training is to maximize the cumulative reward. The preset multi-objective reward function includes at least energy efficiency reward, thermal safety reward, operating cost reward, and equipment lifespan reward; Based on the data generated during the operation of the evolutionary algorithm, a meta-learning task is constructed, and parallel training is performed with the goal of minimizing the preset meta-loss function. The meta-learning task includes at least the data center environment data and the weight ratio of the multi-objective reward function during the execution of the evolutionary algorithm. During the training process of the strategy generation submodule, a benchmark strategy library is constructed in parallel. The benchmark strategy library is used to store benchmark strategies under different data center environments and requirements.

4. The adaptive energy-saving method for computer rooms based on deep reinforcement learning according to claim 1, characterized in that, Based on the adaptive control model of the data center, real-time data center environment data is analyzed to obtain the initial adaptive control strategy for the data center, including: Acquire real-time data of the data center environment, which includes at least data center temperature distribution, equipment operating status and data center energy consumption data. After real-time data room environment data is input into the adaptive control model of the data room, the environmental analysis submodule extracts features from the real-time data room environment data and generates a data room environment feature vector. Call upon the baseline map and baseline strategy library of the data center operation generated during the training of the data center adaptive control model; The environmental analysis submodule performs rule matching from the data center operation baseline map based on the data center environmental feature vector to obtain a set of policy constraint rules. The strategy generation submodule performs strategy matching from the benchmark strategy library based on the data center environment feature vector and strategy constraint rule set to generate an initial adaptive control strategy for the data center.

5. The adaptive energy-saving method for computer rooms based on deep reinforcement learning according to claim 1, characterized in that, A preset simulation verifier is used to simulate and verify the initial adaptive control strategy for the data center, and real-time performance data of the initial adaptive control strategy for the data center is obtained, including: The initial adaptive control strategy for the computer room is injected into the preset simulation calibrator for simulation, and the performance is evaluated based on the simulation results to generate performance evaluation results. If the performance evaluation results do not meet the preset performance conditions, the initial adaptive control strategy of the data center will be optimized based on the adaptive control model of the data center, and the simulation and performance evaluation will be carried out again. If the performance evaluation results meet the preset performance conditions, the initial data center adaptive control strategy will be executed, and the corresponding real-time performance data will be obtained synchronously. The real-time performance data includes at least real-time data center temperature distribution, real-time equipment operating status, and real-time data center energy consumption data.

6. The adaptive energy-saving method for computer rooms based on deep reinforcement learning according to claim 1, characterized in that, Root cause inference of performance fluctuations is performed based on real-time performance data, and the initial adaptive control strategy for the data center is optimized based on the results of the root cause inference, including: Compare the real-time performance data with the expected performance data of the initial adaptive control strategy for the data center; If the comparison results do not meet the preset performance fluctuation conditions, the performance is determined to be stable and no action is taken. If the comparison results meet the preset performance fluctuation conditions, it is determined to be a performance fluctuation; For the initial adaptive control strategy of the data center that exhibits performance fluctuations, root cause inference of performance fluctuations is performed to obtain the root cause inference results; Based on the root cause reasoning results, the adaptive control model for the data center optimizes the initial adaptive control strategy for the data center.

7. The adaptive energy-saving method for computer rooms based on deep reinforcement learning according to claim 6, characterized in that, For the initial adaptive control strategy of the data center that exhibits performance fluctuations, root cause inference of performance fluctuations is performed, and the root cause inference results are obtained, including: Inverse reinforcement learning is used to perform inverse reasoning on real-time performance data to obtain the actual policy network parameter set corresponding to the current initial adaptive control policy of the data center; Obtain the original policy network parameter set corresponding to the current initial adaptive control policy for the computer room; Attribution analysis is performed based on the actual policy network parameter set and the original policy network parameter set to generate root cause inference results. The root cause inference results include at least performance fluctuation type labels, strategy behavior offset parameters, data center environment change parameters, and fluctuation attribution parameter sets.

8. A data center adaptive energy-saving system based on deep reinforcement learning, used to implement the method described in any one of claims 1 to 7, characterized in that, include: The model building module is used to build an adaptive control model for the computer room and pre-train it using multi-objective meta-reinforcement learning. The adaptive control model for the computer room includes an environmental analysis submodule and a strategy generation submodule. The data analysis module is used to analyze real-time data center environment data according to the data center adaptive control model and generate an initial data center adaptive control strategy. The simulation verification module is used to perform simulation verification of the initial adaptive control strategy of the computer room according to a preset simulation verifier. The performance monitoring module is used to monitor and acquire real-time performance data corresponding to the initial adaptive control strategy of the computer room in real time. The root cause reasoning module is used to perform root cause reasoning on performance fluctuations and optimize the initial adaptive control strategy for the data center based on the results of the root cause reasoning.

Images