Method and system for mapping control policy security layers based on a proxy model

Abstract

The application discloses a control strategy security layer mapping method and system based on a proxy model. The method comprises the following steps: obtaining an original control action from an upstream host controller; obtaining a current real-time state; performing forward prediction on the original control action and the current real-time state based on a proxy model to obtain prediction results of all spatial points; comparing the prediction results of all spatial points with preset safety thresholds to determine whether the prediction results of all spatial points are lower than the safety thresholds; if not, determining a correction action that meets the requirements and satisfies safety constraint conditions by using a real-time optimization problem to obtain a final control action; and if yes, determining the original control action as the final control action. The method of the application can overcome the technical problem that an existing advanced control strategy cannot actively and accurately ensure process safety while pursuing an optimization target, and can actively prevent safety violations and accurately reflect efficient calculation of complex system characteristics.

Description

Technical Field

[0001] This invention relates to a data center thermal management method, and more specifically to a control strategy security layer mapping method and system based on an agent model. Background Technology

[0002] Data centers, as critical information infrastructure, are among the most energy-intensive facilities. Cooling systems account for a significant proportion of their electricity consumption, making improving energy efficiency a crucial research area. To address this issue, both academia and industry are actively exploring the application of advanced control strategies such as Deep Reinforcement Learning (DRL) to replace traditional control methods (e.g., PID controllers). Through interaction with the environment and trial-and-error learning, DRL can gradually develop a long-term optimal control strategy aimed at energy conservation.

[0003] However, during the learning process of DRL, the "exploration" phase may lead to unsafe actions, such as excessively reducing cooling output, which could cause localized server overheating and potentially trigger downtime. To address this challenge, two main types of security technologies exist: passive security methods and active security methods based on simple models. Passive security methods typically rely on "post-incident punishment," where the DRL agent receives a negative reward to learn and avoid the behavior when the system actually enters an unsafe state. This approach is clearly unsuitable for mission-critical environments because it requires the system to first experience a dangerous situation. Active security methods based on simple models attempt to prevent unsafe behaviors by predicting the consequences of actions. However, these methods often rely on simplified data center models (such as linear or node models), which cannot accurately capture the highly nonlinear and non-uniform temperature distribution within a data center. This means that even if the "average" temperature is within a safe range, certain areas (such as the top of a rack) may still experience localized high temperatures, posing a security risk.

[0004] In summary, existing security technologies are insufficient in addressing the non-uniform spatial characteristics and real-time decision-making requirements of complex systems. Therefore, it is necessary to design a new method to overcome the technical challenge of existing advanced control strategies failing to proactively and accurately guarantee process safety while pursuing optimization objectives. This method should be capable of both proactively preventing safety violations and accurately reflecting the characteristics of complex systems through efficient computation. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a control strategy security layer mapping method and system based on the agent model.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a control policy security layer mapping method based on a proxy model, comprising:

[0007] Obtain raw control actions from the upstream master controller;

[0008] Get the current real-time status;

[0009] Based on the original control action and the current real-time state, a proxy model is used to perform forward prediction in order to obtain the prediction results for all spatial points.

[0010] Compare the prediction results of all spatial points with a preset safety threshold to determine whether the prediction results of all spatial points are lower than the safety threshold.

[0011] If the prediction results for not all spatial points are lower than the safety threshold, then a real-time optimization problem is used to determine the corrective action that meets the requirements and safety constraints, so as to obtain the final control action.

[0012] If the prediction results for all spatial points are lower than the safety threshold, then the original control action is determined as the final control action.

[0013] The further technical solution is as follows: Based on the original control action and the current real-time state, a proxy model is used for forward prediction to obtain the prediction results for all spatial points, including:

[0014] Based on the original control action and the current real-time state, a proxy model is used to predict the corresponding POD coefficient.

[0015] For each spatial point, the temperature field distribution is reconstructed by the surrogate model using the POD coefficients and pre-calculated spatial basis functions to obtain the prediction results for all spatial points.

[0016] The further technical solution is as follows: the prediction of the corresponding POD coefficient based on the original control action and the current real-time state using a proxy model includes:

[0017] use Predict the corresponding POD coefficient, where, For the proxy model, The POD coefficient; This is the current real-time status. For the original control action, These are the parameters for the proxy model.

[0018] Its further technical solution is: the temperature field distribution is expressed as .

[0019] The further technical solution is as follows: the method of using real-time optimization to determine corrective actions that meet the requirements and satisfy safety constraints to obtain the final control action includes:

[0020] The real-time optimization problem is used to determine the corrected action that minimizes the difference from the original control action and satisfies the safety constraints, so as to obtain the final control action.

[0021] The further technical solution is as follows: the optimization objective of the real-time optimization problem is... ,in, To correct the action; This refers to the original control action.

[0022] Its further technical solution is: the security constraint condition is ,in, The estimated temperature for performing the modified action, For safety temperature constraints, the provisions shall be given by the user agreement or relevant standards.

[0023] The further technical solution is as follows: The training process of the proxy model is as follows:

[0024] Latin hypercube sampling is performed on the boundary conditions of the original CFD model to obtain the sampled boundary conditions;

[0025] CFD simulations are performed based on the sampled boundary conditions to generate diverse temperature field results, and the corresponding POD coefficients and spatial basis functions are calculated.

[0026] A multilayer perceptron is used as the learning model. During the training phase, the optimal parameters are found through an optimization process based on the POD coefficients and spatial basis functions to obtain the surrogate model.

[0027] This invention also provides a control policy security layer mapping system based on an agent model, comprising:

[0028] The raw control action acquisition unit is used to acquire raw control actions from the upstream master controller;

[0029] Real-time status acquisition unit, used to acquire the current real-time status;

[0030] The prediction unit is used to perform forward prediction based on the original control action and the current real-time state using a proxy model to obtain the prediction results for all spatial points.

[0031] The comparison unit is used to compare the prediction results of all spatial points with a preset safety threshold to determine whether the prediction results of all spatial points are lower than the safety threshold.

[0032] The correction unit is used to determine, in real-time optimization, a correction action that meets the requirements and satisfies the safety constraints if the prediction results of not all spatial points are lower than the safety threshold, so as to obtain the final control action.

[0033] The final action determination unit is used to determine the original control action as the final control action if the prediction results of all spatial points are lower than the safety threshold.

[0034] Its further technical solution is as follows: the prediction unit includes:

[0035] The coefficient prediction subunit is used to predict the corresponding POD coefficient based on the original control action and the current real-time state using a proxy model.

[0036] The temperature field reconstruction subunit is used to reconstruct the temperature field distribution for each spatial point using the surrogate model with POD coefficients and pre-calculated spatial basis functions to obtain the prediction results for all spatial points.

[0037] The advantages of this invention compared to existing technologies are as follows: By integrating real-time state acquisition, forward prediction, and real-time optimization correction mechanisms, this invention proactively and accurately ensures process safety while pursuing optimization goals. The method first receives the original control actions from the upstream master controller and, combined with the current real-time state of the system, analyzes the prediction results of spatial points using a highly efficient surrogate model. By comparing these prediction results with preset safety thresholds, it determines whether safety conditions are met. If not, a corrective action that approximates the original optimization intent while ensuring system safety is determined by solving a real-time optimization problem. This method fundamentally overcomes the problem that existing advanced control strategies struggle to balance safety during optimization, achieving pre-emptive prediction and correction of potential safety hazards. It not only effectively prevents safety violations but also accurately captures complex system characteristics, ensuring a dual improvement in processing efficiency and accuracy, thus providing a control solution that is both efficient and safe.

[0038] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A flowchart illustrating the control strategy security layer mapping method based on the proxy model provided in this embodiment of the invention;

[0041] Figure 2 A schematic diagram of a sub-process of the control strategy security layer mapping method based on the proxy model provided in an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of a control strategy security layer mapping method based on a proxy model provided in an embodiment of the present invention;

[0043] Figure 4 A schematic block diagram of a control strategy security layer mapping system based on an agent model provided in an embodiment of the present invention;

[0044] Figure 5 A schematic block diagram of a computer device provided for an embodiment of the present invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0047] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0048] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0049] Please see Figure 1 , Figure 1This is a flowchart illustrating the control strategy security layer mapping method based on a surrogate model provided in this invention. This method is applied to a server. By integrating real-time state monitoring, forward prediction, and real-time optimization adjustment, it overcomes the technical challenge of existing advanced control strategies failing to proactively and accurately guarantee process safety while pursuing optimization goals. Specifically, the method first constructs a surrogate model using Latin hypercube sampling and CFD simulation to efficiently and accurately predict the system temperature field distribution under different control actions. Then, based on the current state and the original control action, forward prediction is performed, and the prediction results are compared with safety thresholds to determine whether correction is needed. If correction is required, a real-time optimization problem is solved to minimize the difference from the original control action while ensuring that safety constraints at all spatial points are met, thereby achieving the goal of both proactively preventing safety violations and accurately reflecting the characteristics of complex systems. This method not only improves system security but also reduces the demand for computing resources through an efficient computational model, making it suitable for scenarios requiring rapid response and real-time control.

[0050] Figure 1 This is a flowchart illustrating the control policy security layer mapping method based on a proxy model provided in an embodiment of the present invention. Figure 1 As shown, the method includes the following steps S110 to S160.

[0051] S110: Obtain the raw control actions from the upstream master controller.

[0052] In this embodiment, the initial control action refers to the preliminary operation instructions (such as adjusting the air supply temperature and air volume) generated by the upstream master controller (such as the DRL agent) to optimize a specific goal.

[0053] like Figure 3 As shown, these are action instructions received from the upstream master controller (e.g., a deep reinforcement learning (DRL) agent) aimed at optimizing specific objectives, such as power usage efficiency (PUE). These actions typically include, but are not limited to, adjusting parameters of the data center cooling system, such as supply air temperature (T). in ) and air volume (m in The DRL agent generates an optimized control action based on its trained model and the current policy. This action aims to minimize energy consumption or achieve other preset optimization objectives. However, since the DRL model may not fully consider all safety constraints, direct execution... This could cause certain points in the system to exceed safe operating limits, such as server inlet temperatures exceeding the ASHRAE-recommended safe threshold of 27°C. Therefore, further evaluation and possible modification of this original control action are needed to ensure the safe operation of the system.

[0054] S120, Get the current real-time status.

[0055] In this embodiment, the current real-time state refers to the system's current operating conditions and environmental parameters (such as IT load and temperature data at each monitoring point), which, together with the original control action, serve as inputs for evaluating and correcting the control action to ensure the system's safe and optimized operation.

[0056] The system's current operating conditions and environmental parameters, i.e. It reflects the specific state of the system at a particular point in time. This includes, but is not limited to, IT load (P... IT This includes the current temperature distribution, humidity levels, and other factors that may affect system performance and security. In the specific example of a data center, real-time status can include the computing load of each rack, temperature data monitored by various sensors, and the operating status of the cooling system. Accurately capturing real-time status is fundamental for effective forward prediction and risk identification, as the surrogate model needs to predict future physical field changes (such as temperature field) under different control actions based on the latest system state. Furthermore, the accuracy of real-time status directly affects the effectiveness and safety of corrective actions, ensuring that the final output control action satisfies both optimization objectives and guarantees safe system operation.

[0057] S130. Based on the original control action and the current real-time state, a proxy model is used to perform forward prediction to obtain the prediction results for all spatial points.

[0058] In this embodiment, the prediction results for all spatial points refer to the temperature predictions for each monitoring point within the data center using a surrogate model. These results not only include the average temperature distribution across the entire space but also pay special attention to potential local hotspots, thereby ensuring the overall security and stability of the system. This process is achieved through a comprehensive analysis of the system's current operating conditions and environmental parameters (i.e., the "current real-time state") and the preliminary operating instructions (i.e., the "original control actions") generated by the upstream main controller.

[0059] In one embodiment, please refer to Figure 2 The above-mentioned step S130 may include steps S131 to S132.

[0060] S131. Based on the original control action and the current real-time state, a proxy model is used to predict the corresponding POD coefficient.

[0061] In this embodiment, the POD coefficients refer to a set of values ​​calculated through a surrogate model, which represent the main characteristic modes of the physical field under specific conditions.

[0062] use Predict the corresponding POD coefficient, where, For the proxy model, The POD coefficient; This is the current real-time status. For the original control action, These are the parameters for the proxy model.

[0063] S132. For each spatial point, the temperature field distribution is reconstructed using the surrogate model with the POD coefficients and pre-calculated spatial basis functions to obtain the prediction results for all spatial points.

[0064] The temperature field distribution is expressed as: .

[0065] in It is a spatial basis function extracted using the POD technique, which captures the main variation patterns of the system's temperature field. In this expression, T_avg typically refers to the average value of the entire temperature field, while... This describes the portion of the temperature field change predicted based on the current control actions and system state. Thus, by applying this formula to each spatial point, a detailed temperature field distribution can be reconstructed, providing precise information about potential safety risks (such as localized overheating).

[0066] The method described in this embodiment not only improves the speed and accuracy of prediction, but also ensures that the system can proactively identify and correct any potential security threats without affecting its optimization objectives. This method is particularly important for complex systems such as data centers that require high-precision temperature control, as it can effectively prevent hardware damage and performance degradation caused by temperature runaway.

[0067] In one embodiment, the training process of the above-described proxy model is as follows:

[0068] Latin hypercube sampling is performed on the boundary conditions of the original CFD model to obtain the sampled boundary conditions.

[0069] CFD simulations are performed based on the sampled boundary conditions to generate diverse temperature field results, and the corresponding POD coefficients and spatial basis functions are calculated.

[0070] A multilayer perceptron is used as the learning model. During the training phase, the optimal parameters are found through an optimization process based on the POD coefficients and spatial basis functions to obtain the surrogate model.

[0071] In this embodiment, to ensure the training dataset has good representativeness and coverage, the Latin Hypercube Sampling (LHS) method is used to sample the boundary conditions of the original CFD model. LHS is a stratified random sampling method that can uniformly distribute sample points in a multidimensional space, thereby improving the efficiency and quality of sampling.

[0072] Define the system's control actions (e.g., supply air temperature T) in Air volume (m) in ) and system status (e.g., IT load P) IT ( ) are used as boundary conditions, and a series of different combinations of boundary conditions are generated using the LHS technique. These combinations will be used for subsequent CFD simulations.

[0073] High-precision physical simulation tools (such as CFD) are used to run simulations under different boundary conditions to obtain "physical field snapshots" under various operating conditions. These snapshots are crucial for building efficient and accurate surrogate models.

[0074] For each set of sampled boundary conditions, a CFD simulation is run once to generate the corresponding 3D temperature field of the data center. Detailed temperature distribution information is extracted from each temperature field to form a high-dimensional vector representing that temperature field. Proper Orthogonal Decomposition (POD) technology is applied to extract the main feature modes (i.e., POD modes or spatial basis functions) from these high-dimensional vectors. This step significantly reduces data dimensionality while retaining most of the original information. Calculate the POD coefficient for the temperature field under each set of boundary conditions. It reflects the changes in the temperature field under specific conditions.

[0075] Establish an efficient mapping function F such that, given any set of boundary conditions μ and system state x, the corresponding POD coefficients can be quickly predicted. We chose to use a multilayer perceptron (MLP) here because it has strong nonlinear fitting capabilities and is suitable for handling complex relationships.

[0076] The POD coefficients obtained in the previous steps and their corresponding boundary conditions and system status The input and output pairs constitute the training dataset. The goal of training an MLP model is to find the optimal parameters that minimize the error between the predicted and true values. Commonly used optimization algorithms include gradient descent and its variants (such as the Adam optimizer). During training, the model performance is continuously optimized by adjusting parameters such as network structure (number of layers, number of neurons per layer), activation function type, and regularization strategy until a satisfactory level of accuracy is achieved. Finally, the fully trained MLP model becomes the desired surrogate model F, which can efficiently predict the POD coefficients based on the input boundary conditions, thereby reconstructing the complete temperature field distribution.

[0077] This two-stage approach not only ensures the surrogate model's advantage in prediction speed but also guarantees its reliability in accuracy, enabling it to effectively support real-time secure mapping tasks in practical applications.

[0078] S140. Compare the prediction results of all spatial points with the preset safety threshold to determine whether the prediction results of all spatial points are lower than the safety threshold.

[0079] S150. If the prediction results of not all spatial points are lower than the safety threshold, then a real-time optimization problem is used to determine the corrective action that meets the requirements and safety constraints, so as to obtain the final control action.

[0080] In this embodiment, a real-time optimization problem is used to determine the corrected action that minimizes the difference from the original control action and satisfies the safety constraints, so as to obtain the final control action.

[0081] The optimization objective of the real-time optimization problem is ,in, To correct the action; This refers to the original control action.

[0082] Safety constraints are ,in, The estimated temperature for performing the modified action, For safety temperature constraints, they are given by the user agreement or relevant standards (such as the ASHARE standard).

[0083] Upon receiving the original control action and the current state of the system Then, the offline-trained surrogate model is used for forward prediction to obtain refined prediction results of the space physics field (e.g., temperature field). Then, the predicted temperature value for each spatial point is compared one by one with a preset safety threshold (e.g., the server air inlet temperature must not exceed 27°C). This process identifies any potential safety violations, especially localized hotspots. If the predicted temperature at all spatial points is below the safety threshold, the control action is considered safe; otherwise, corrections are needed.

[0084] A correction procedure is initiated when the predicted temperature at at least one spatial point exceeds a safety threshold. The core of this correction procedure is a real-time optimization problem aimed at finding a corrective action. This allows the corrective action to retain the original control action to the greatest extent possible. It achieves the optimization goal while ensuring the safety of all spatial points.

[0085] Optimization objective: Minimize the difference between the corrected action and the original control action, i.e.: The objective function here aims to find a modified action that is as close as possible to the original control action in order to maintain the original optimization effect.

[0086] Safety constraints: The modified control actions must fully meet the safety requirements, i.e. ,in, This indicates the temperature field prediction result based on the correction action and the current system state. This is the preset upper limit of the safe temperature.

[0087] Define the objective function and constraints for the optimization problem described above. Solve this problem using an appropriate optimization algorithm (such as gradient descent or a more advanced optimizer) to find the optimal correction action. The obtained correction action is output as the final control action and executed by the physical system.

[0088] S160. If the prediction results of all spatial points are lower than the safety threshold, then the original control action is determined as the final control action.

[0089] If, after verification in step S140, the predicted temperature at all spatial points is found to be below the set safety threshold, this means that the original control action will not cause any safety hazards. Therefore, the original control action can be directly... This final control action is output to the physical system for execution without requiring correction. This step ensures efficient system operation and fully utilizes the optimization benefits of advanced control strategies such as DRL without safety risks.

[0090] Through the three steps described above (S140 to S160), the technical solution of this embodiment can effectively prevent and eliminate potential security risks while maintaining energy efficiency, achieving proactive security protection for complex systems such as data centers. This method not only improves system security but also ensures the effectiveness and reliability of control strategies.

[0091] The method in this embodiment aims to address the technical problem that existing advanced control strategies (such as Deep Reinforcement Learning, DRL) cannot proactively and accurately guarantee process safety when pursuing optimization goals (such as energy saving). Specifically, this invention provides a security layer mapping technology based on a Surrogate Model, which can complete the following three steps before the execution of control actions: Proactive prediction: Using a high-fidelity surrogate model, it predicts in real time the refined spatial physical field (e.g., temperature distribution of thousands of points in a data center) that will result from actions recommended by the DRL agent. Risk identification: Based on the predicted complete physical field, it accurately identifies any potential local security violations (e.g., local hotspots). Security mapping: If a risk is predicted, the technology can automatically map (or correct) the original "unsafe action" into a "corrected action" that retains the original action's optimization intent to the greatest extent while ensuring that all spatial points in the system are in a safe state.

[0092] like Figure 3 As shown, a security layer is inserted between the main controller (e.g., a DRL agent) and the physical system (e.g., a data center). First, a computationally efficient and high-fidelity system proxy model is created. This proxy model can quickly predict the complete spatial physical field (e.g., temperature field) of the system based on control actions (e.g., air supply temperature and airflow of CRAC units) and system states (e.g., IT load). First, a "physical field snapshot" dataset of the system is obtained under various operating conditions (different control actions and IT loads) through high-precision physical simulations (e.g., computational fluid dynamics, CFD) or historical data collected from real systems. For example, 100 sets of 3D temperature field data for the data center under different operating conditions can be obtained (each temperature field contains 300,000 spatial points). Next, the "Proper Orthogonal Decomposition" (POD) technique is applied to these "physical field snapshots." POD is a mathematical method used to extract the most dominant feature patterns (i.e., "POD modes" or "spatial basis functions" Φ) from high-dimensional data. For example, it might only require five POD modes to reconstruct over 99.9% of the original temperature field characteristics. Any temperature physics field T... z All of these can be approximated as a linear combination of these POD modes: ,in These are the "POD coefficients" corresponding to each mode. These POD coefficients β are determined by the system's boundary conditions, i.e., the control actions. (For example: air supply temperature) Air volume ) and system status x (e.g., IT load) The purpose of this step is to train a mapping function F such that... This F is the surrogate model. F can be a linear regression model or a nonlinear model, such as a multilayer perceptron (MLP), to achieve higher accuracy. To train the surrogate model, the boundary conditions of the original CFD model need to be Latin hypercube sampled first, and the CFD model is run based on these sampled boundary conditions to generate different temperature field results and their corresponding POD coefficients. And spatial basis functions. Taking a multilayer perceptron (MLP) as an example, the training phase involves finding the optimal model parameters. This involves learning the POD coefficients mapped under different boundary conditions. The amount of training data required is related to the size of the CFD model; for small to medium-sized data centers (<600 square meters), approximately 100 sets of CFD simulation results typically need to be generated offline. The output of this stage is a set of POD modes (basis functions). And a trained surrogate model F. Together, these two constitute a complete and computationally efficient physics predictor.

[0093] Next, the process is executed in real time during system operation, for example, at the beginning of each control cycle (e.g., every 5 minutes), as shown in the appendix. Figure 2 The security layer receives a "raw control action" from the upstream master controller (such as a DRL agent). This action is designed to optimize a specific objective (such as PUE). When the original action is deemed unsafe by function F (e.g., a cold aisle sensor reading exceeds the ASHRAE recommended 27°C threshold or another user-defined threshold), the safety layer intercepts the original recommended instruction. This prevents it from being sent directly to the physical system. Simultaneously, the security layer obtains the system's current real-time status. (For example, the current IT load). Utilizing the proxy model built in the first phase, the security layer performs a forward prediction within milliseconds. Subsequently, Each point in the data is compared to a preset safety threshold (e.g., the server inlet temperature must not exceed 27°C). If If the temperature at all points is below the safety threshold, the action is deemed safe, and the safety layer outputs the "final control action". Conversely, if If the temperature at at least one point exceeds a safe threshold (e.g., a localized hotspot is predicted), the action is deemed unsafe, and the safety layer initiates a correction procedure. This correction procedure needs to solve a real-time optimization problem to find a "corrective action." The optimization objective is to approximate the original action as closely as possible. In order to preserve the optimization intent of DRL to the greatest extent possible; while safety constraints require This ensures that the predicted results of the corrective actions in the surrogate model fully satisfy the safety constraints. Finally, the safety layer outputs the "final control action". In this way, the system can achieve the energy-saving optimization effects brought by DRL while ensuring that it does not perform any actions that may lead to safety violations.

[0094] Figure 3 The physical systems (such as data centers) in the system are responsible for performing... And provide feedback on the new status. .

[0095] This embodiment innovatively transforms the traditional passive "post-punishment" mechanism into an active "pre-prediction and correction" mechanism, eliminating potential unsafe conditions before they occur. Unlike methods that rely on node average temperature, the POD-based surrogate model can accurately predict non-uniform spatial physics fields, effectively identify and eliminate local hotspots, and provide more refined and reliable safety assurance. The surrogate model (POD-ROM) has a computation speed far exceeding traditional full-order physics simulations (such as CFD). It simplifies the solution of complex partial differential equations into simple algebraic calculations, making it easy to embed into control loops and meet the requirements of real-time online correction. The safety layer, as an independent module, is completely decoupled from the upper-level main controller (DRL). This allows the main controller to be freely updated or replaced without modifying the safety layer, giving it strong versatility and scalability, enabling it to be used as a "safety plug-in" in various optimized controllers.

[0096] The aforementioned control strategy safety layer mapping method based on a proxy model integrates real-time state acquisition, forward prediction, and real-time optimization correction mechanisms to proactively and accurately ensure process safety while pursuing optimization goals. This method first receives the original control actions from the upstream master controller and, combined with the current real-time state of the system, uses a computationally efficient proxy model to analyze the prediction results of spatial points. By comparing these prediction results with preset safety thresholds, it determines whether safety conditions are met. If not, it determines a corrective action that approximates the original optimization intent while ensuring system safety by solving a real-time optimization problem. This method fundamentally overcomes the problem that existing advanced control strategies struggle to balance safety during optimization, achieving proactive prediction and correction of potential safety hazards. It not only effectively prevents safety violations but also accurately captures complex system characteristics, ensuring a dual improvement in processing efficiency and accuracy, thus providing a highly efficient and safe control solution.

[0097] Figure 4 This is a schematic block diagram of a control strategy security layer mapping system 300 based on a proxy model provided in an embodiment of the present invention. Figure 4As shown, corresponding to the above-described proxy-based control policy security layer mapping method, this invention also provides a proxy-based control policy security layer mapping system 300. This proxy-based control policy security layer mapping system 300 includes a unit for executing the above-described proxy-based control policy security layer mapping method, and the system can be configured in a server. Specifically, please refer to... Figure 4 The agent-based control strategy security layer mapping system 300 includes an original control action acquisition unit 301, a real-time status acquisition unit 302, a prediction unit 303, a comparison unit 304, a correction unit 305, and a final action determination unit 306.

[0098] The system comprises: an initial control action acquisition unit 301 for acquiring initial control actions from the upstream master controller; a real-time status acquisition unit 302 for acquiring the current real-time status; a prediction unit 303 for performing forward prediction using a surrogate model based on the initial control actions and the current real-time status to obtain prediction results for all spatial points; a comparison unit 304 for comparing the prediction results of all spatial points with a preset safety threshold to determine whether the prediction results of all spatial points are lower than the safety threshold; a correction unit 305 for determining, if not all spatial point prediction results are lower than the safety threshold, a correction action that meets the requirements and satisfies the safety constraints using a real-time optimization problem to obtain the final control action; and a final action determination unit 306 for determining the initial control action as the final control action if the prediction results of all spatial points are lower than the safety threshold.

[0099] In one embodiment, the prediction unit 303 includes:

[0100] The coefficient prediction subunit is used to predict the corresponding POD coefficients based on the original control action and the current real-time state using a surrogate model; the temperature field reconstruction subunit is used to reconstruct the temperature field distribution for each spatial point using the surrogate model with the POD coefficients and pre-calculated spatial basis functions, so as to obtain the prediction results for all spatial points.

[0101] In one embodiment, the correction unit 305 is used to determine the correction action that minimizes the difference from the original control action and satisfies the safety constraints by using a real-time optimization problem, so as to obtain the final control action.

[0102] In one embodiment, the system further includes a training unit for performing Latin hypercube sampling on the boundary conditions of the original CFD model to obtain sampled boundary conditions; performing CFD simulation based on the sampled boundary conditions to generate diverse temperature field results and calculating the corresponding POD coefficients and spatial basis functions; and using a multilayer perceptron as the learning model to find the optimal parameters through an optimization process based on the POD coefficients and spatial basis functions during the training phase to obtain a surrogate model.

[0103] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned agent model-based control strategy security layer mapping system 300 and its various units can be found in the corresponding descriptions in the aforementioned method embodiments. For the sake of convenience and brevity, these details will not be repeated here.

[0104] The aforementioned agent-based control strategy security layer mapping system 300 can be implemented as a computer program, which can be used in, for example... Figure 5 It runs on the computer device shown.

[0105] Please see Figure 5 , Figure 5 This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device 500 can be a server, wherein the server can be a standalone server or a server cluster composed of multiple servers.

[0106] See Figure 5 The computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a non-volatile storage medium 503 and internal memory 504.

[0107] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, cause the processor 502 to perform a control policy security layer mapping method based on an agent model.

[0108] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0109] The internal memory 504 provides an environment for the execution of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a control strategy security layer mapping method based on the agent model.

[0110] This network interface 505 is used for network communication with other devices. Those skilled in the art will understand that... Figure 5The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 500 to which the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0111] The processor 502 is used to run a computer program 5032 stored in the memory to implement all the steps of the agent-based control strategy security layer mapping method.

[0112] It should be understood that in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0113] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0114] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein when executed by a processor, the computer program causes the processor to perform all steps of the agent-based control policy security layer mapping method.

[0115] The storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0116] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0117] In the embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of each unit is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0118] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the system of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0119] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0120] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A control strategy security layer mapping method based on a proxy model, characterized in that, include: Acquire raw control actions from the upstream master controller; raw control actions refer to preliminary operation instructions generated by the upstream master controller to optimize specific objectives, including adjusting the supply air temperature and air volume. Get the current real-time status; the current real-time status refers to the system's current operating conditions and environmental parameters. Based on the original control actions and the current real-time state, a surrogate model is used for forward prediction to obtain prediction results for all spatial points; including: Based on the original control action and the current real-time state, a proxy model is used to predict the corresponding POD coefficient. For each spatial point, the temperature field distribution is reconstructed by the surrogate model using the POD coefficients and pre-calculated spatial basis functions to obtain the predicted temperature results for all spatial points; Compare the prediction results of all spatial points with a preset safety threshold to determine whether the prediction results of all spatial points are lower than the safety threshold. If not all predicted spatial points are below the safety threshold, a real-time optimization problem is used to determine corrective actions that meet the requirements and safety constraints, thus obtaining the final control action. Specifically, a real-time optimization problem is used to determine the corrective action that minimizes the difference from the original control action and meets the safety constraints, thus obtaining the final control action. The optimization objective of the real-time optimization problem is... ,in, To correct the action; This refers to the original control action; If the prediction results for all spatial points are lower than the safety threshold, then the original control action is determined as the final control action.

2. The control strategy security layer mapping method based on the agent model according to claim 1, characterized in that, The step of predicting the corresponding POD coefficient using a proxy model based on the original control action and the current real-time state includes: use Predict the corresponding POD coefficient, where, For the proxy model, The POD coefficient; This is the current real-time status. For the original control action, These are the parameters for the proxy model.

3. The control strategy security layer mapping method based on the agent model according to claim 2, characterized in that, The temperature field distribution is expressed as: ; Spatial basis functions extracted using POD technology; T avg This usually refers to the average value of the entire temperature field.

4. The control strategy security layer mapping method based on the agent model according to claim 1, characterized in that, The security constraints are as follows: ,in, The estimated temperature for performing the modified action, For safety temperature constraints, the provisions shall be given by the user agreement or relevant standards.

5. The control strategy security layer mapping method based on the agent model according to claim 1, characterized in that, The training process of the proxy model is as follows: Latin hypercube sampling is performed on the boundary conditions of the original CFD model to obtain the sampled boundary conditions; CFD simulations are performed based on the sampled boundary conditions to generate diverse temperature field results, and the corresponding POD coefficients and spatial basis functions are calculated. A multilayer perceptron is used as the learning model. During the training phase, the optimal parameters are found through an optimization process based on the POD coefficients and spatial basis functions to obtain the surrogate model.

6. A control strategy security layer mapping system based on a proxy model, characterized in that, The system uses the agent-based control policy security layer mapping method as described in any one of claims 1 to 5, including: The raw control action acquisition unit is used to acquire raw control actions from the upstream master controller; Real-time status acquisition unit, used to acquire the current real-time status; The prediction unit is used to perform forward prediction based on the original control action and the current real-time state using a proxy model to obtain the prediction results for all spatial points. The comparison unit is used to compare the prediction results of all spatial points with a preset safety threshold to determine whether the prediction results of all spatial points are lower than the safety threshold. The correction unit is used to determine, in real-time optimization, a correction action that meets the requirements and satisfies the safety constraints if the prediction results of not all spatial points are lower than the safety threshold, so as to obtain the final control action. The final action determination unit is used to determine the original control action as the final control action if the prediction results of all spatial points are lower than the safety threshold. The prediction unit includes: The coefficient prediction subunit is used to predict the corresponding POD coefficient based on the original control action and the current real-time state using a proxy model. The temperature field reconstruction subunit is used to reconstruct the temperature field distribution for each spatial point using the surrogate model with POD coefficients and pre-calculated spatial basis functions, so as to obtain the predicted temperature results for all spatial points.

Images