A knowledge graph-based cross-data center heating ventilation air conditioning control method

Abstract

The application discloses a kind of based on knowledge graph's across data center warm air conditioning control method. Including: first, the knowledge graph of corresponding entity, relationship and rule of source data center is built, and it is embedded into physical information neural network model;Again, the interactive training of reinforcement learning model is carried out, and the basic reinforcement learning model is obtained;Then, according to the physical topological structure difference of source-target data center, the physical information neural network model of source data center is adjusted after adaptation, and then the fine-tuning training of basic reinforcement learning model is carried out, the warm air conditioning control strategy of target data center is obtained and deployed to target data center, and the warm air conditioning control of target data center is realized.The convergence speed and prediction accuracy of the physical information neural network model are significantly improved, and the model's explainability and cross-scene generalization ability are enhanced, which can quickly adapt to different climate conditions, cabinet layout and cooling architecture of data center scene.

Description

Technical Field

[0001] This invention belongs to the field of data center thermal management technology, and in particular relates to a cross-data center HVAC control method based on knowledge graphs. Background Technology

[0002] For data centers, heating, ventilation, and air conditioning (HVAC) systems typically consume up to 50% of the total power in a single facility's energy budget, exceeding server power consumption and power conversion costs.

[0003] The thermal dynamics in GPU racks differ fundamentally from those in traditional CPU server rooms. Accelerator racks exhibit millisecond-level power fluctuations driven by random batch scheduling and memory-intensive cores, resulting in steep, spatially localized heat bursts. To prevent hotspot formation, terminal maintenance still relies on conservative overcooling margins, which offsets the server efficiency gains from advanced silicon and exacerbates the energy footprint of AI clusters.

[0004] Existing thermal management paradigms fall short in handling the rapid, high-density workloads imposed by accelerator-dominated AI data centers. While traditional computational fluid dynamics (CFD) workflows combined with proportional-integral-derivative (PID) or linear model predictive control provide stringent spatial resolution, CFD solvers are computationally prohibitive for real-time inference, and PID requires assumptions of quasi-steady-state dynamics. Therefore, the performance of combining CFD and PID approaches degrades when facing the highly nonlinear transients characteristic of burst GPU loads.

[0005] Purely data-driven alternative models and standard reinforcement learning (RL) agents offer faster inference and support adaptive policy synthesis. However, they require large, stable corpora; they generalize poorly when boundary conditions drift and offer only weak safety guarantees under strict thermal constraints. Existing hybrid RL-MPC schemes attempt to combine optimized structures with learning-based adaptability, but still face challenges in balancing sample efficiency with robust, sub-second-level driving requirements.

[0006] Therefore, existing methods cannot simultaneously provide physical consistency, real-time responsiveness, and data efficiency under the high-frequency load fluctuations characteristic of AI workloads, and it is difficult to achieve a balance between prediction accuracy, convergence speed, and cross-scenario generalization ability. In addition, existing HVAC control models are usually trained for specific data centers and lack the ability to migrate across data centers, resulting in the need for extensive retraining in new environments and making it difficult to adapt to data center scenarios with different climate conditions, rack layouts, and cooling architectures. Summary of the Invention

[0007] To address the problems of existing technologies, this invention provides a cross-data center HVAC control method based on knowledge graphs.

[0008] In a first aspect, the present invention provides a cross-data center HVAC control method based on knowledge graphs, the method comprising:

[0009] S1: Based on the physical topology of the source data center, construct a data center knowledge graph containing entities, relationships, and rules;

[0010] S2: Embed the data center knowledge graph into the physical information neural network model to obtain the basic physical information neural network model (KG-PINN); Combine the operational data of the source data center to train the basic physical information neural network model to obtain the trained basic physical information neural network model;

[0011] S3: Combining the pre-trained basic physical information neural network model, the reinforcement learning model (DDPG) is interactively trained to obtain the basic reinforcement learning model. The basic physical information neural network model serves as an environmental dynamics surrogate model, used to predict changes in the thermal state of the data center based on control actions. The physical information neural network model and the reinforcement learning model interact through a Functional Mock-up Unit (FMU) interface to form a closed-loop control architecture.

[0012] S4: Based on the differences in physical topology between the source data center and the target data center, the basic physical information neural network model is adapted and adjusted to obtain a fine-tuned physical information neural network model.

[0013] S5: Combine the fine-tuned physical information neural network model to fine-tune the basic reinforcement learning model, obtain the fine-tuned basic reinforcement learning model, and deploy it to the target data center to realize the HVAC control of the target data center.

[0014] Optionally, in S1, the entities in the data center knowledge graph are used to represent devices, spatial areas, and sensor objects in the data center, the relationships are used to represent the structural connection relationships, airflow relationships, and heat transfer relationships between the entities, and the rules are used to encode the thermodynamic behavior of the data center.

[0015] Furthermore, the rules include cold aisle temperature evolution rules, hot aisle temperature evolution rules, and return air zone temperature evolution rules.

[0016] Optionally, S2 includes:

[0017] The entity embedding vectors corresponding to the entities in the data center knowledge graph are generated by the embedding layer. The input feature vectors of the physical information neural network model are constructed by combining the entity embedding vectors and the parameters obtained by the rule estimation of the data center knowledge graph. The output of the physical information neural network model is the temperature field prediction result for each spatiotemporal location.

[0018] Furthermore, the input feature vector includes the physical spatial coordinates, timestamp, ambient temperature, entity embedding vector, and cold aisle reference temperature, hot aisle reference temperature, and return air area reference temperature of each spatiotemporal sampling point, as estimated by rules from the data center knowledge graph.

[0019] Optionally, the reward function during the training process of the reinforcement learning model is used to simultaneously optimize the temperature control effect and energy consumption index.

[0020] Optionally, in the reinforcement learning model, the samples stored in the experience replay buffer contain a time delay marker to indicate the time lag steps of each state transition relative to the current state. Such an experience replay buffer is referred to as a time-series hierarchical experience replay buffer.

[0021] Furthermore, in the reinforcement learning model, the loss function of the Critic network includes a multi-window weighted loss, which is calculated based on the time delay labels of samples in the empirical replay buffer.

[0022] Secondly, the present invention provides a knowledge graph-based cross-data center HVAC control system, the system comprising:

[0023] The knowledge graph construction module is used to build a data center knowledge graph containing entities, relationships, and rules based on the physical topology and operational data of the data center.

[0024] The modeling module is used to embed the data center knowledge graph into the physical information neural network model to obtain the basic physical information neural network model; combined with the operating data of the source data center, the basic physical information neural network model is trained to obtain the trained basic physical information neural network model.

[0025] The first training module is used to interactively train the reinforcement learning model by combining the trained basic physical information neural network model to obtain the basic reinforcement learning model.

[0026] The model fine-tuning module is used to adapt and adjust the basic physical information neural network model according to the differences in physical topology between the source data center and the target data center, so as to obtain a fine-tuned physical information neural network model.

[0027] The second training module is used to fine-tune the basic reinforcement learning model by combining the fine-tuned physical information neural network model, so as to obtain the HVAC control strategy of the target data center and deploy it to the target data center.

[0028] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in the first aspect.

[0029] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in the first aspect.

[0030] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0031] 1) By constructing a data center knowledge graph, physical topology, thermodynamic mechanisms, and expert knowledge are embedded into the physical information neural network model in a structured manner, significantly improving the convergence speed and prediction accuracy of the physical information neural network model, while also enhancing the model's interpretability and cross-scenario generalization ability. Experimental results show that KG-PINN improves prediction accuracy by approximately 45% and convergence speed by approximately 8-14 times compared to traditional PINN.

[0032] 2) A knowledge graph-based cross-data center model migration mechanism was proposed. By adapting the knowledge graph structure and fine-tuning the parameters of KG-PINN, the reinforcement learning model (i.e., the HVAC control model) was rapidly migrated between different data centers, avoiding the large amount of data and computational costs required for training from scratch. This enabled the model to quickly adapt to data center scenarios with different climate conditions, rack layouts, and cooling architectures.

[0033] 3) A closed-loop control architecture based on DDPG and KG-PINN was designed. Seamless integration of the KG-PINN model and the DDPG controller was achieved through the FMU interface. KG-PINN was used as the environment model for policy learning and online optimization, realizing sub-second response adaptive HVAC control, significantly reducing energy consumption while ensuring thermal stability. Experimental results show that compared with traditional PID control, this invention can achieve 29.21% energy savings and reduce temperature fluctuation by approximately 48%.

[0034] 4) A time-series hierarchical experience replay buffer mechanism was introduced. By explicitly encoding the diversity of delays, the Critic network can learn the causal relationship of delays, which significantly improves the training stability and control robustness of the reinforcement learning controller when dealing with thermal inertial systems. Attached Figure Description

[0035] Figure 1This is a flowchart of a method provided in an embodiment of the present invention.

[0036] Figure 2 A flowchart illustrating the knowledge graph construction and embedding process provided in this embodiment of the invention.

[0037] Figure 3 A schematic diagram of the KG-PINN and DDPG closed-loop control architecture provided in an embodiment of the present invention.

[0038] Figure 4 A flowchart illustrating the model fine-tuning process for a target data center, as provided in an embodiment of the present invention.

[0039] Figure 5 A flowchart illustrating the model workflow of the target data center provided in this embodiment of the invention.

[0040] The accompanying drawings illustrate specific embodiments of the invention, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0041] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0042] The components of the embodiments of the invention described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0043] In the following, the terms "comprising," "having," and their cognates, which may be used in various embodiments of the invention, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as excluding, firstly, the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more combinations thereof.

[0044] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of the invention pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be interpreted as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of the invention.

[0045] This invention provides a knowledge graph-based cross-datacenter HVAC control method that utilizes knowledge graph embedding and transfer learning to achieve rapid migration and adaptive deployment of HVAC control models across different data centers. The implementation process of this method will be described in detail below with reference to the accompanying drawings and through several embodiments.

[0046] like Figure 1 As shown, the cross-data center HVAC control method based on knowledge graph proposed in this invention may include the following steps:

[0047] S1: Based on the physical topology of the source data center, construct a data center knowledge graph containing entities, relationships, and rules;

[0048] Figure 2 A flowchart for knowledge graph construction and embedding is provided. In data center thermal management scenarios, to effectively capture and represent structured knowledge in physical systems, it is necessary to construct a data center knowledge graph that can uniformly describe physical entities, interaction relationships, and thermodynamic rules. This knowledge graph serves as the prior knowledge carrier for subsequent Physical-Informed Neural Network (PINN) models, constraining the model learning process and enabling it to acquire latent space representations and temperature field prediction capabilities that conform to physical mechanisms.

[0049] In this embodiment, the data center knowledge graph Defined as:

[0050]

[0051] in, This represents an Entity Set, used to represent devices, spatial regions, and sensor objects within a data center. Represents an edge set; Represents a set of relation types. Represents the set of physical rules. (Edge set) With relation type set Used to represent the structural connections, airflow relationships, and heat transfer relationships between entities. A set of physical rules. It is used to encode prior knowledge in fields such as heat transfer, airflow, and energy conservation.

[0052] In one feasible implementation, the physical topology of the source data center includes, but is not limited to, rack distribution information, air conditioning distribution information, supply and return air path information, and sensor deployment information.

[0053] Rack distribution information is used to represent the spatial coordinates and load capacity of each server rack, including the coordinates of the rack's center location. Rated power of the cabinet Real-time operating power of the server rack The hot and cold aisle number of the server rack.

[0054] Air conditioning distribution information is used to indicate the installation location and cooling capacity of each precision air conditioning (CRAH / CRAC) unit, including the coordinates of the air conditioning equipment location. Air supply coverage area Rated cooling capacity Supply air temperature setpoint Fan speed and air volume .

[0055] The supply and return air path information is the airflow organization path of the computer room established based on the equipment layout diagram or engineering experience rules. It includes the path of cold air flowing from the air conditioning equipment to the cold aisle, the path of cold aisle air being heated after flowing through the server rack, and the path of hot air flowing back to the hot aisle and return air area.

[0056] Sensor deployment information is used to indicate the layout of sensors for thermal environment monitoring, including the locations of temperature sensors. Humidity sensor location, power monitoring point location, sensor type identification .

[0057] The aforementioned physical topology information collectively forms the basis of entities and relationships in a knowledge graph.

[0058] For entity sets Each entity A corresponding entity node in the knowledge graph is obtained by abstracting the physical components, thermal environment areas, and monitoring devices in the data center.

[0059] In one feasible implementation, the entity types include, but are not limited to, server racks, CRAH units, ambient zone sensors, cold aisle sensors, hot aisle sensors, and return ducts.

[0060] A server rack represents a heat source for IT equipment, and its attributes include spatial coordinates. Rated power Real-time power GPU / CPU load rate .

[0061] A CRAH unit represents an air conditioning terminal refrigeration device, and its attributes include spatial coordinates. Supply air temperature range Fan speed range Rated cooling capacity , This indicates the minimum and maximum values ​​of the supply air temperature. This indicates the minimum and maximum speed of the fan.

[0062] An ambient zone sensor represents a monitoring point for the overall temperature and humidity of the computer room. Its attributes include the coordinates of the measuring point and the ambient temperature value. Ambient humidity value .

[0063] A cold aisle sensor is a temperature monitoring point in the cold aisle area, and its attributes include the cold aisle temperature. .

[0064] A hot aisle sensor represents a temperature monitoring point in a hot aisle area, and its attributes include the hot aisle temperature. .

[0065] Return duct refers to the path or area for hot air to return, and its attributes include the average temperature of the return air area. Return air flow .

[0066] In some implementations, each entity It can be further represented as a structured object containing category information and attribute information:

[0067]

[0068] in, Indicates the entity category; Represents the entity attribute vector.

[0069] edge set and set of relation types It is obtained by establishing directed relationships between entities based on physical topological connections, airflow organization patterns, and heat exchange paths.

[0070] Any relation edge It can be represented as:

[0071]

[0072] in, Representing entities With entity Relationship types .

[0073] Relation type set This includes, but is not limited to, supply-air, transfer-energy, dissipates-heat, and returns-flow relationships. Supply-air relationships indicate that the air conditioning equipment delivers cold air to the cold aisle; transfer-energy relationships indicate that the cold energy in the cold aisle is absorbed by the server rack; dissipates-heat relationships indicate that the server rack releases heat to the hot aisle; and returns-flow relationships indicate that hot air flows from the hot aisle to the return air area or return air duct.

[0074] In some implementations, the relationship may also include weights to characterize the coupling strength between entities, such as the intensity of airflow influence, heat exchange intensity, or spatial proximity. In this case, the relationship edge can be expanded to represent:

[0075]

[0076] in, This indicates the association weight between entities.

[0077] physical rule set It is based on thermodynamic principles, the law of conservation of energy, and experience in airflow organization.

[0078] In this embodiment, a three-zone lumped thermal zone model is used to describe the thermal coupling relationship between the cold aisle, hot aisle, and return air zone. The cold aisle zone is denoted as Zone-1, the hot aisle zone as Zone-2, and the return air zone as Zone-3. The physical rule set... ,in, This indicates the rules governing the evolution of cold aisle temperatures. This indicates the rules governing the temperature evolution of the thermal aisle; This indicates the rules governing the temperature evolution in the return air region.

[0079] Specifically:

[0080] The cold aisle temperature is jointly determined by the ambient temperature, the air conditioning supply temperature, and the heat dissipation of IT equipment. Its dynamic equation is the cold aisle temperature kinetic equation, which can be expressed as:

[0081]

[0082] in, Let be the average temperature of the cold aisle at time t; Set the air conditioning supply temperature; The ambient temperature; This refers to the mass flow rate of the supplied air. The specific heat capacity of air at constant pressure; The convective heat transfer coefficient between the environment and the cold aisle; The equivalent heat transfer area between the environment and the cold aisle; Total power consumption of IT equipment; This is the equivalent proportionality factor for IT heat leakage into the cold aisle; This is the equivalent heat capacity of the cold aisle.

[0083] The hot aisle temperature is mainly formed by the heat generated by the server rack and the heating of the cold aisle airflow. Its dynamic equation is the hot aisle temperature kinetic equation, which can be expressed as:

[0084]

[0085] in, For a moment Average temperature of the hot passage; The equivalent mass flow rate of the cold aisle after passing through the server rack; The convective heat transfer coefficient between the environment and the thermal channel; The equivalent heat transfer area between the environment and the thermal passage; The equivalent thermal gain coefficient for the conversion of power consumption of IT equipment into the hot channel; This is the equivalent heat capacity of the hot channel.

[0086] The temperature in the return air zone is determined by both the average temperature of the hot passage and environmental conditions. Its dynamic equation is the temperature kinetic equation for the return air zone, which can be expressed as:

[0087]

[0088] in, For a moment The average temperature of the return air area; This is the equivalent air mass flow rate from the hot aisle to the return air zone; The heat transfer coefficient between the ambient area and the return air zone; The equivalent heat exchange area between the ambient air and the return air zone; This is the equivalent heat capacity of the return air area.

[0089] This invention incorporates the aforementioned rules into a knowledge graph. Therefore, the knowledge graph constructed in this invention not only represents physical structural relationships but also encodes mechanistic constraints within the thermal management system, thereby providing physically consistent prior knowledge for subsequent model training.

[0090] S2: Embed the data center knowledge graph into a Physical Information Neural Network (PINN) model to obtain a basic Physical Information Neural Network (KG-PINN). Combine this with operational data from the source data center to train the KG-PINN, resulting in a trained KG-PINN. This model is a knowledge graph-enhanced Physical Information Neural Network model used to learn the spatiotemporal distribution patterns of the data center temperature field. While satisfying physical constraints, it integrates topological structure information, equipment attribute information, and operational status information to improve the model's ability to fit complex thermal fields and its cross-scenario transfer capabilities.

[0091] Specifically as follows:

[0092] In order for knowledge graphs to be processed by neural networks, each entity in the graph is first mapped to a continuous vector space to obtain an entity embedding representation.

[0093] For any entity Its embedding vector is represented as:

[0094]

[0095] in, Indicates entity embedding function; Representing entities The embedding vector; This indicates the embedding dimension, which can typically be set to 50–200 dimensions.

[0096] In some implementations, entity embedding vectors may contain not only entity category information but also the entity's physical attributes and upstream / downstream relationship context. Therefore, This can be further expressed as:

[0097]

[0098] in, Indicates the entity type; This represents the entity attribute vector, i.e., the relation aggregation vector; Representing entities In a knowledge graph, the adjacency context is a local topological context feature.

[0099] Through the above embedding process, the entity embedding vector can simultaneously encode physical location features, equipment functional features, thermal management semantic features, and airflow and heat propagation path information.

[0100] In this embodiment, the input feature vector of KG-PINN is composed of physical state features and knowledge graph enhancement features.

[0101] For any spatiotemporal sampling point, its input feature vector Recorded as:

[0102]

[0103] in, t represents the physical space coordinates; t represents the timestamp. The ambient temperature; The cold aisle reference temperature is estimated by physical rules; The reference temperature of the thermal channel is estimated by physical rules; The reference temperature for the return air area is estimated by physical rules; This refers to the entity embedding vector associated with the corresponding spatial location or device. The mechanistic reference value can be derived from the aforementioned set of physical rules. The three-region thermodynamic equations are obtained by solving or approximating them, and are used to provide a mechanism-based prior reference for neural networks.

[0104] Therefore, the input feature vector can be divided into physical spatiotemporal input feature vectors, rule-enhanced input feature vectors, and graph semantic input feature vectors. Physical spatiotemporal input features include spatial location. Time variables and environmental state variables (such as The rule-enhanced input feature vector includes a reference temperature value calculated based on thermodynamic rules and local thermal influence information derived from device connection relationships. The graph semantic input feature vector includes entity embedding vectors, and may also include relation aggregation vectors and local topological context features.

[0105] To improve the stability and convergence speed of model training, the input variables are normalized or standardized before being input into the KG-PINN network.

[0106] For any input variable Minimum-maximum normalization or standardization can be used. After normalization, all input feature vectors are fed into the KG-PINN model (i.e., the neural network mapping function) for joint modeling to obtain the temperature field prediction results at the corresponding spatiotemporal location.

[0107] The hidden state representation of KG-PINN can be written as:

[0108]

[0109] in, The parameter is Neural network mapping function; It represents a physical state vector composed of spatiotemporal coordinates, environmental state, and rule reference values; This represents the knowledge graph embedding representation obtained from the knowledge graph; This represents the latent space representation that integrates physical state information and prior information from the graph.

[0110] Furthermore, the predicted temperature field output of KG-PINN is given by the following formula:

[0111]

[0112] in, Indicates the predicted output; For parameters The neural network mapping function is used to construct the input feature vector. The nonlinear mapping relationship between the target temperature field variables and the target temperature field variables.

[0113] In this embodiment, the neural network mapping function Composed of a multi-layer feedforward neural network, it achieves high-precision fitting of complex thermal field distribution in data centers by jointly modeling physical state features and knowledge graph embedding features.

[0114] In this embodiment, This may include predicting cold aisle temperatures. Predicting the temperature of the hot passage and predicting return air area temperature .

[0115] In some implementations, the output can also be extended to local temperature field distribution, device-level temperature response, or thermal state prediction results at future times.

[0116] The loss function in the KG-PINN training process includes a physical constraint loss, which comprises residual terms from the energy conservation equation, continuity equation, and Navier-Stokes equation based on the three-region lumped model. The KG-PINN model is trained by minimizing the weighted sum of the empirical loss and the physical constraint loss.

[0117] S3: Combine the trained basic physical information neural network model with interactive training of the reinforcement learning model to obtain the basic reinforcement learning model;

[0118] In this embodiment, to achieve intelligent control of the data center cooling system, the present invention constructs a reinforcement learning controller based on the Deep Deterministic Policy Gradient (DDPG) algorithm, and couples the controller with the KG-PINN model through a functional model unit interface, thereby forming a closed-loop control system, such as... Figure 3 As shown in the diagram, the basic physical information neural network model serves as an environmental dynamics proxy model, used to predict changes in the thermal state of the data center based on control actions. The DDPG controller employs a continuous control strategy based on state feedback output to achieve dynamic optimization control of the cooling system.

[0119] Specifically as follows:

[0120] S3.1: Construct the Actor-Critic architecture for DDPG.

[0121] The DDPG controller includes an Actor network for outputting a deterministic control policy. and the Critic network used to evaluate the action value function under the current policy. .

[0122] In this embodiment, system state Represented as:

[0123]

[0124] in, This refers to the cold aisle temperature. This refers to the temperature of the hot passage. Return air temperature; For IT equipment load power; The ambient temperature; For embedding representation of knowledge graphs.

[0125] Control Action The output is directly from the Actor network and is a continuous control variable. :

[0126]

[0127] in, Set the air conditioner's supply air temperature setting; This refers to the control variables for fan speed or air volume.

[0128] The optimization objective of the Actor network is to maximize the long-term cumulative return, and its gradient update formula is:

[0129]

[0130] in, Let the policy objective function be... This serves as a buffer for experience replay. The action value function output by the Critic network; This represents the gradient operator with respect to the Actor network parameters θ, used to guide the update direction of the policy parameters; Indicates the current strategy The continuous control actions generated below; This represents the gradient with respect to action 'a', used to characterize the effect of action change on the action value function. The extent of the impact; This indicates that the Actor network is in state A deterministic control strategy for output.

[0131] Through the aforementioned strategy gradient update mechanism, the Actor network can continuously adjust the control strategy based on the value assessment information provided by the Critic network, enabling the system to achieve the control objective of optimal energy consumption or optimal overall performance while meeting temperature constraints.

[0132] The Critic network is trained by minimizing the temporal difference (TD) error, and its loss function is:

[0133]

[0134]

[0135] in, Let be the reward value at time t; Let be the loss function of the Critic network; For TD objectives, Discount factor; , These are the parameters for two target networks.

[0136] S3.2: Construct the reward function.

[0137] In this embodiment, the reward function is used to simultaneously optimize temperature control performance and energy consumption indicators, and is defined as:

[0138]

[0139] in, The target cold aisle temperature; Energy consumption of the refrigeration system; These are two weighting coefficients used to balance the trade-off between temperature control accuracy and energy consumption optimization;

[0140] S3.3: Implement the interaction between DDPG and KG-PINN based on the FMU interface.

[0141] In this embodiment, the KG-PINN model is encapsulated as an FMU component to achieve standardized interaction between the reinforcement learning controller and the environment model.

[0142] At each control moment The interaction flow is as follows:

[0143] First, the Actor network outputs control actions. Next, the control actions are transmitted via the FMU interface. Input the KG-PINN model; then, the KG-PINN model calculates and outputs the thermal state at the next time step. Then calculate the reward based on the change in status. Finally, the transferred sample Store in the experience replay buffer.

[0144] The above method enables closed-loop interaction between the controller and the physical agent model.

[0145] In one feasible implementation, considering the significant time lag characteristics of data center thermal systems (such as the delay in the effect of air supply changes on temperature), this invention constructs a time-series hierarchical experience replay buffer.

[0146] For each transferred sample, a time delay marker is introduced. , This indicates the number of steps the state transition occurs relative to the current state. For different time lags, a multi-step TD objective is defined:

[0147]

[0148] in, This is a multi-step temporal difference target corresponding to a time lag τ, used to characterize the long-term return estimation under the effect of delay; Let be the accumulated reward value under the condition of time lag τ, representing the stage reward from time t to t+τ; τ is the discount factor γ raised to the power of τ, used to apply decay weighting to future returns, where γ∈(0,1); This represents the system state obtained after τ time steps following the execution of the action. For the target Actor network in state The target strategy action to be output.

[0149] Furthermore, the Critic network employs multi-delay window weighted training objectives, thereby constructing a multi-window weighted loss, as shown in the following formula:

[0150]

[0151] in, This represents the multi-window weighted loss value; For the time delay window weight; This represents the maximum time delay step.

[0152] This invention introduces a multi-delay training mechanism, which can explicitly model the delay effect of control actions on the system, thereby improving the stability and convergence performance of policy learning.

[0153] S4: Based on the differences in physical topology between the source and target data centers, the basic physical information neural network model is adapted and adjusted to obtain a fine-tuned physical information neural network model, so as to achieve efficient migration across data centers.

[0154] like Figure 4 As shown, it specifically includes:

[0155] Obtain the physical topology information of the target data center, including rack distribution and power information, air conditioning equipment distribution and cooling capacity, airflow path and supply and return air structure, and sensor deployment location.

[0156] Based on the physical structure of the source and target data centers, a data center knowledge graph is constructed. After structural adaptation, the target knowledge graph is obtained. :

[0157]

[0158] Structural adaptation is achieved in the following ways:

[0159] For new entities Initialize entity embedding vectors based on their spatial location and adjacency relationships:

[0160]

[0161] in, For the set of adjacent entities; For normalized weights; This is the entity embedding vector corresponding to the newly added entity.

[0162] For data center knowledge graphs The target knowledge graph exists in the middle. The missing entities in the graph are supplemented with their thermodynamic effects through graph reasoning or rules, as shown in the following formula:

[0163]

[0164] in, For entities in the target data center The estimated temperature value or thermal state variable at the corresponding location is used to characterize the thermodynamic effects of the entity under the current topology. For entities In the target knowledge graph, the set of adjacent entities represents the set of entities that have airflow, heat transfer, or control relationships with that entity; It is an inference function built on a set of physical rules, used to derive the thermal state estimate of the target entity based on the state information of adjacent entities and predefined thermodynamic relationships.

[0165] When the topology changes (e.g., from single-sided air supply to double-sided air supply), the relationship set is updated, and the corresponding physical rules are also updated, as shown in the following formula:

[0166]

[0167]

[0168] in, For the target knowledge graph The set of relationships in the Context describes the connections between entities and their physical interactions. For data center knowledge graph The set of relations in; This is a set of relationship increments introduced due to changes in topology, used to characterize newly added or adjusted entity connection relationships; This is the set of physical rules corresponding to the target data center, used to describe the physical constraints between entities, such as heat transfer, airflow, and energy exchange. The set of physical rules for the source data center; It is a set of incremental physical rules caused by changes in topology, used to update or supplement thermodynamic and fluid dynamic relationships under the new topology.

[0169] After completing the knowledge graph adaptation, the parameters of the KG-PINN model are fine-tuned using the operational data from the target data center:

[0170]

[0171] in, These are the parameters of the source model; For the target model parameters; The learning rate; This is the loss function for the target domain.

[0172] S5: Combine the fine-tuned physical information neural network model to fine-tune the basic reinforcement learning model, obtain the fine-tuned basic reinforcement learning model, and deploy it to the target data center to realize the HVAC control of the target data center (i.e., dynamic optimization and adjustment of the thermal environment of the target data center).

[0173] like Figure 5 As shown, the details are as follows:

[0174] S5.1: Combine the fine-tuned physical information neural network model with the basic reinforcement learning model for fine-tuning training to obtain the fine-tuned basic reinforcement learning model. During the fine-tuning process, the underlying network parameters of the KG-PINN model are frozen to preserve the trainability of the knowledge graph embedding layer and prediction layer, while the physical loss weights are dynamically adjusted to adapt to the climate conditions and cooling architecture characteristics of the target data center.

[0175] The fine-tuned KG-PINN model and the trained DDPG controller are then deployed to the control system of the target data center. Specifically, the fine-tuned KG-PINN model is encapsulated as an FMU component to describe the dynamic evolution of the data center's thermal environment; the trained DDPG controller is deployed as an online decision-making module to generate continuous control actions based on the current thermal state; and the FMU component and the online decision-making module are coupled through a unified data interface to form an online control link.

[0176] In this embodiment, the online control system of the target data center can be represented as:

[0177]

[0178] in, The knowledge graph adapted for the target data center; The KG-PINN model is fine-tuned for the target domain; The control strategy for DDPG Actor after deployment; This is a component for model interaction interfaces.

[0179] S5.2: Collect real-time operational status data of the target data center through the Building Automation System (BAS) or data center monitoring system, and write the data into the adapted target data center knowledge graph to update the current physical status representation of the system.

[0180] The data collected in real time includes, but is not limited to, cold aisle temperature. Hot channel temperature Return air temperature Air conditioning supply temperature Fan speed IT equipment power Ambient temperature Other device-level and area-level sensor readings.

[0181] Timekeeping Real-time sampling data For a moment The real-time observation status is given by the following formula:

[0182]

[0183] In this implementation, real-time sampled data is mapped to dynamic attribute updates in the knowledge graph:

[0184]

[0185] in, Indicates time Dynamic knowledge graph; This represents the function for updating dynamic attributes of a knowledge graph.

[0186] The updated knowledge graph can further infer the structured physical state representation of the current system:

[0187]

[0188] in, This represents the reasoning and embedding extraction functions of knowledge graphs; Indicates time The structured representation obtained from knowledge graph reasoning.

[0189] S5.3: After obtaining the real-time observation data and knowledge graph inference results at the current moment, use the fine-tuned KG-PINN model to predict the thermal state evolution trend of the target data center in the future.

[0190] The input state at the current control moment is constructed using the following formula:

[0191]

[0192] Based on the input state, the KG-PINN model predicts the future thermal state evolution, as shown in the following formula:

[0193]

[0194] in, Represents the future prediction time domain The internal thermal state sequence; Indicates the current control action; This represents the KG-PINN prediction model after fine-tuning the target domain.

[0195] In some implementations, the predicted output includes the future cold aisle temperature trajectory. Future thermal channel temperature trajectory Future return air temperature trajectory Furthermore, it can also include the local temperature field distribution.

[0196] S5.4: The DDPG controller selects the optimal continuous control action based on the current thermal state and the predicted state, and sends it to the HVAC actuator through the FMU interface.

[0197] In this embodiment, the control action is output by the Actor network:

[0198]

[0199]

[0200] in, This is the optimal supply air temperature setpoint; This is the optimal fan speed or air volume setting.

[0201] Control actions are sent to the HVAC actuator (such as the CRAH unit) via the FMU interface or control bus:

[0202]

[0203] in, This indicates the actual control commands issued; This indicates the function for issuing control commands.

[0204] In some implementations, boundary constraints can be applied to the control actions to ensure that they meet engineering constraints:

[0205]

[0206]

[0207] in, These are the minimum and maximum allowable supply air temperatures, respectively. These are the minimum and maximum permissible fan speeds, respectively.

[0208] S5.5: After the control action is executed, continuously collect system feedback data for closed-loop control evaluation, knowledge graph updates, and continuous model optimization.

[0209] Record feedback observations after control execution for:

[0210]

[0211] in, Energy consumption of the refrigeration system;

[0212] Based on the feedback observation data, the following processing can be performed:

[0213] (1) Update dynamic attributes in the knowledge graph. For example, update the rack real-time power, current cold / hot aisle temperature, current return air temperature, and current air conditioning operating status. The update can be represented as:

[0214]

[0215] (2) Evaluate the thermal safety and energy-saving effect of the control strategy based on feedback data. For example, define online performance indicators:

[0216]

[0217] in, For online operating performance metrics; The target cold aisle temperature; These are the weighting coefficients.

[0218] (3) Since the operating environment of the target data center may drift slowly over time (e.g., changes in IT load patterns, equipment aging, changes in airflow organization), the KG-PINN model can be periodically fine-tuned based on accumulated feedback data:

[0219]

[0220] in, These are the current model parameters; These are the updated model parameters; This is a loss function constructed based on online feedback data.

[0221] (4) In some implementations, the newly collected feedback data can also be used to retrain the DDPG controller online or periodically update it offline to adapt to the changes in operating conditions during the long-term operation of the target data center.

[0222] At this point, the new transferred sample can be Add an experience replay buffer and use it for subsequent control strategy optimization.

[0223] This invention provides a knowledge graph-based cross-data center HVAC control system, comprising:

[0224] The knowledge graph construction module is used to build a data center knowledge graph containing entities, relationships, and rules based on the physical topology and operational data of the data center.

[0225] The modeling module is used to embed the data center knowledge graph into the physical information neural network model to obtain the basic physical information neural network model; combined with the operating data of the source data center, the basic physical information neural network model is trained to obtain the trained basic physical information neural network model.

[0226] The first training module is used to interactively train the reinforcement learning model by combining the trained basic physical information neural network model to obtain the basic reinforcement learning model.

[0227] The model fine-tuning module is used to adapt and adjust the basic physical information neural network model according to the differences in physical topology between the source data center and the target data center, so as to obtain a fine-tuned physical information neural network model.

[0228] The second training module is used to fine-tune the basic reinforcement learning model by combining the fine-tuned physical information neural network model, so as to obtain the HVAC control strategy of the target data center and deploy it to the target data center.

Claims

1. A cross-datacenter HVAC control method based on knowledge graphs, characterized in that, Includes the following steps: S1: Based on the physical topology of the source data center, construct a data center knowledge graph containing entities, relationships, and rules; S2: Embed the data center knowledge graph into the physical information neural network model to obtain the basic physical information neural network model; combine the operational data of the source data center to train the basic physical information neural network model to obtain the trained basic physical information neural network model. S3: Combine the trained basic physical information neural network model with interactive training of the reinforcement learning model to obtain the basic reinforcement learning model; S4: Based on the differences in physical topology between the source data center and the target data center, the basic physical information neural network model is adapted and adjusted to obtain a fine-tuned physical information neural network model. Based on the physical topology of the source data center and the target data center, the target knowledge graph is obtained after structural adaptation processing of the data center knowledge graph. The structural adaptation process is achieved through the following methods: For new entities Initialize entity embedding vectors based on their spatial location and adjacency relationships: in, For the set of adjacent entities; For normalized weights; This is the entity embedding vector corresponding to the newly added entity; For entities The embedding vector; For entities that exist in the data center knowledge graph but are missing in the target knowledge graph, their thermodynamic effects are supplemented through graph reasoning or rules. When the topology changes, update the set of relationships and the corresponding physical rules. After completing the knowledge graph adaptation, the parameters of the basic physical information neural network model are fine-tuned using the operational data from the target data center: in, These are the parameters of the source model; For the target model parameters; The learning rate; The target domain loss function; S5: Combine the fine-tuned physical information neural network model to fine-tune the basic reinforcement learning model, obtain the fine-tuned basic reinforcement learning model, and deploy it to the target data center to realize the HVAC control of the target data center.

2. The knowledge graph-based cross-data center HVAC control method according to claim 1, characterized in that, In S1, entities in the data center knowledge graph represent devices, spatial areas, and sensor objects in the data center; relationships represent structural connections, airflow relationships, and heat transfer relationships between entities; and rules encode the thermodynamic behavior of the data center.

3. The knowledge graph-based cross-data center HVAC control method according to claim 2, characterized in that, The rules include cold aisle temperature evolution rules, hot aisle temperature evolution rules, and return air zone temperature evolution rules.

4. The knowledge graph-based cross-data center HVAC control method according to claim 1, characterized in that, S2 includes: Generate entity embedding vectors corresponding to entities in the data center knowledge graph, and construct the input feature vector of the physical information neural network model by combining the entity embedding vectors and the parameters obtained by rule estimation of the data center knowledge graph.

5. The knowledge graph-based cross-data center HVAC control method according to claim 4, characterized in that, The input feature vector includes the physical space coordinates, timestamp, ambient temperature, entity embedding vector, and cold aisle reference temperature, hot aisle reference temperature, and return air area reference temperature estimated by rules from the data center knowledge graph for each spatiotemporal sampling point.

6. The knowledge graph-based cross-data center HVAC control method according to claim 1, characterized in that, In the reinforcement learning model, the samples stored in the experience replay buffer contain time delay markers to indicate the number of time lag steps of each state transition relative to the current state.

7. The knowledge graph-based cross-data center HVAC control method according to claim 6, characterized in that, In the reinforcement learning model, the loss function of the Critic network includes a multi-window weighted loss, which is calculated based on the time delay labels of samples in the empirical replay buffer.

8. A knowledge graph-based cross-data center HVAC control system, characterized in that, include: The knowledge graph construction module is used to build a data center knowledge graph containing entities, relationships, and rules based on the physical topology and operational data of the data center. The modeling module is used to embed the data center knowledge graph into the physical information neural network model to obtain the basic physical information neural network model; combined with the operating data of the source data center, the basic physical information neural network model is trained to obtain the trained basic physical information neural network model. The first training module is used to interactively train the reinforcement learning model by combining the trained basic physical information neural network model to obtain the basic reinforcement learning model. The model fine-tuning module is used to adapt and adjust the basic physical information neural network model according to the differences in physical topology between the source data center and the target data center, so as to obtain a fine-tuned physical information neural network model. Based on the physical topology of the source data center and the target data center, the target knowledge graph is obtained after structural adaptation processing of the data center knowledge graph. The structural adaptation process is achieved through the following methods: For new entities Initialize entity embedding vectors based on their spatial location and adjacency relationships: in, For the set of adjacent entities; For normalized weights; This is the entity embedding vector corresponding to the newly added entity; For entities The embedding vector; For entities that exist in the data center knowledge graph but are missing in the target knowledge graph, their thermodynamic effects are supplemented through graph reasoning or rules. When the topology changes, update the set of relationships and the corresponding physical rules. After completing the knowledge graph adaptation, the parameters of the basic physical information neural network model are fine-tuned using the operational data from the target data center: in, These are the parameters of the source model; For the target model parameters; The learning rate; The target domain loss function; The second training module is used to fine-tune the basic reinforcement learning model by combining the fine-tuned physical information neural network model, so as to obtain the HVAC control strategy of the target data center and deploy it to the target data center.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the knowledge graph-based cross-data center HVAC control method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the knowledge graph-based cross-data center HVAC control method according to any one of claims 1 to 7.

Images