A refrigeration system energy consumption optimization method, device and readable storage medium

Abstract

The application provides a refrigeration system energy consumption optimization method, device and readable storage medium, relates to the technical field of energy consumption optimization, and the method comprises the following steps: modeling the energy consumption optimization process of a refrigeration system as a Markov decision process, defining the state space and action space of the refrigeration system energy consumption optimization scene; using a neural network to predict the state transition in the Markov decision process according to the measured point data collected by a sensor, and constructing a state transition prediction model; based on the constructed state transition prediction model, designing and training a refrigeration system optimization model based on a proximal policy optimization (PPO) algorithm; deploying the strategy output by the refrigeration system optimization model trained based on the PPO algorithm to an actual refrigeration system for real-time control and optimization. The method, device and medium can solve the problems that the existing refrigeration system energy consumption optimization method lacks adaptability to complex environmental conditions and is difficult to achieve optimal refrigeration effect and energy consumption efficiency under different operating conditions.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of energy consumption optimization, and in particular to a refrigeration system energy consumption optimization method, device and readable storage medium. BACKGROUND

[0002] With the rapid development of cloud computing, big data and artificial intelligence technology, the scale and number of data centers are constantly expanding. A large number of servers, storage devices and network devices in the data center will generate a large amount of heat during operation, so an efficient refrigeration system is crucial to ensure the stable operation of the data center. The energy consumption of the refrigeration system accounts for a large proportion of the total energy consumption of the data center, so how to reduce the energy consumption of the refrigeration system while ensuring the normal operation of the equipment has become a hot research issue.

[0003] At present, the refrigeration system of the data center mainly relies on traditional rule-based or threshold-based control strategies, but these traditional control strategies lack adaptability to complex environmental conditions and are difficult to achieve optimal refrigeration effect and energy consumption efficiency under different operating conditions. At the same time, the existing optimization methods cannot fully utilize the large amount of real-time data collected by sensors in the data center, lack dynamic adaptability and intelligence, and are difficult to achieve optimal energy consumption control effect. SUMMARY

[0004] The technical problem to be solved by the present application is to solve the above-mentioned deficiencies of the prior art, to provide a refrigeration system energy consumption optimization method, device and readable storage medium, to solve the problem that the existing refrigeration system energy consumption optimization method lacks adaptability to complex environmental conditions, is difficult to achieve optimal refrigeration effect and energy consumption efficiency under different operating conditions, and cannot fully utilize the large amount of real-time data collected by sensors, resulting in difficulty in achieving optimal energy consumption control effect.

[0005] In a first aspect, the present application provides a refrigeration system energy consumption optimization method, which comprises:

[0006] Modeling the energy consumption optimization process of the refrigeration system as a Markov decision process, defining the state space and action space of the refrigeration system energy consumption optimization scenario;

[0007] According to the measurement point data collected by the sensor, using a neural network to predict the state transition in the Markov decision process, constructing a state transition prediction model, the input parameters of the state transition prediction model being the environmental state and action at the current time, and the output parameter being the environmental state at the next time;

[0008] Based on the constructed state transition prediction model, designing and training a refrigeration system optimization model based on the proximal policy optimization (PPO) algorithm;

[0009] The strategy output by the PPO algorithm-based refrigeration system optimization model is deployed to an actual refrigeration system for real-time control and optimization.

[0010] Further, the state space includes the temperature of each area in the data center, the humidity of each area, outdoor humidity, outdoor temperature, and the refrigeration set value of the refrigeration system at the previous time;

[0011] The action space is defined as the refrigeration set value of the refrigeration system at the current time.

[0012] Further, the reward function of the Markov decision process is composed of humidity reward, temperature reward, and energy consumption reward.

[0013] The reward function of the temperature reward is R_temp = |T_target - T_actual|, where T_target is the target temperature and T_actual is the actual temperature.

[0014] The reward function of the humidity reward is R_hum = |H_target - H_actual|, where H_target is the target humidity and H_actual is the actual humidity.

[0015] The reward function of the energy consumption reward is R_energy = -E_consumed, where E_consumed is the energy consumption value of the refrigeration system.

[0016] The total reward function is R_total = w1*R_temp + w2*R_hum + w3*R_energy, where w1, w2, and w3 are weighting coefficients used to balance the contributions of different parts.

[0017] Further, the state transition in the Markov decision process is predicted using a neural network based on the sensor-collected measurement point data, and a state transition prediction model is constructed, specifically including:

[0018] The measurement point data of the data center is collected in real time by sensors, and the measurement point data includes the temperature of each area in the data center, the humidity of each area, outdoor humidity, outdoor temperature, and the refrigeration set value and energy consumption data of the refrigeration system.

[0019] The collected measurement point data is preprocessed.

[0020] Using the preprocessed measurement point data, a data-driven method is used to construct the state transition prediction model based on the multilayer perceptron MLP in the deep neural network.

[0021] Further, the state transition prediction model is constructed based on a multilayer perceptron MLP in a deep neural network using a data-driven method, and specifically includes:

[0022] The model architecture of the state transition prediction model is designed, and the architecture includes an input layer, a hidden layer, and an output layer.

[0023] A plurality of triplets including an environment state at a current time, an action at the current time, and an environment state at a next time are obtained from the preprocessed measurement point data, the environment state at the current time and the environment state at the next time each include a temperature of each region in the data center, a humidity of each region, an outdoor humidity, an outdoor temperature, and a refrigeration system refrigeration set value and energy consumption data at the corresponding time; the action at the current time includes a refrigeration system refrigeration set value.

[0024] The law of state transition is learned according to the triplets, and the constructed state transition prediction model is obtained.

[0025] Further, based on the constructed state transition prediction model, a refrigeration system optimization model based on a proximal policy optimization PPO algorithm is designed and trained, and specifically includes:

[0026] Based on the constructed state transition prediction model, the architecture of the Actor network and the Critic network of the PPO algorithm is designed, the input of the Actor network is a state vector at a current time, the output is a mean value and a variance of a set action of the refrigeration system, the input of the Critic network is a state vector at a current time, and the output is a value function of the current state.

[0027] The training strategy of the PPO algorithm is designed, the advantage function is calculated using advantage estimation, the step length of policy update is constrained using clipping Clip method, the Actor network is updated using policy gradient method, and the Critic network is updated using minimum time difference error.

[0028] The refrigeration system optimization model based on the PPO algorithm is trained according to the training strategy.

[0029] Further, the strategy output by the refrigeration system optimization model trained based on the PPO algorithm is deployed to an actual refrigeration system for real-time control and optimization, and the method further includes:

[0030] During the actual operation of the data center, new environment state data and feedback information are continuously collected.

[0031] The refrigeration system optimization model is updated through the new environment state data and feedback information, and the update process adopts an incremental learning strategy.

[0032] In a second aspect, the present application provides a refrigeration system energy consumption optimization device, which comprises:

[0033] An optimization process modeling module is configured to model an energy consumption optimization process of the refrigeration system as a Markov decision process, and define a state space and an action space of a refrigeration system energy consumption optimization scenario.

[0034] A state transition prediction module is connected to the optimization process modeling module, and is configured to use a neural network to predict state transition in the Markov decision process according to sensor-acquired measurement point data, and construct a state transition prediction model, wherein input parameters of the state transition prediction model are an environmental state and an action at a current time, and output parameters are an environmental state at a next time.

[0035] An optimization model construction module is connected to the state transition prediction module, and is configured to design and train a refrigeration system optimization model based on a proximal policy optimization (PPO) algorithm based on the constructed state transition prediction model.

[0036] A real-time control optimization module is connected to the optimization model construction module, and is configured to deploy a strategy output by the refrigeration system optimization model trained based on the PPO algorithm to an actual refrigeration system, and perform real-time control and optimization.

[0037] In a third aspect, the present application provides a refrigeration system energy consumption optimization device, which comprises a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to implement the refrigeration system energy consumption optimization method of the first aspect.

[0038] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the refrigeration system energy consumption optimization method of the first aspect.

[0039] The application provides a refrigeration system energy consumption optimization method, device and readable storage medium. First, the energy consumption optimization process of the refrigeration system is modeled as a Markov decision process, and the state space and action space of the refrigeration system energy consumption optimization scene are defined; then, according to the measurement point data collected by a sensor, a neural network is used to predict the state transition in the Markov decision process, and a state transition prediction model is constructed, wherein the input parameters of the state transition prediction model are the environmental state and action at the current time, and the output parameter is the environmental state at the next time; based on the constructed state transition prediction model, a refrigeration system optimization model based on a proximal policy optimization (PPO) algorithm is designed and trained; finally, the strategy output by the refrigeration system optimization model trained based on the PPO algorithm is deployed to an actual refrigeration system for real-time control and optimization. By modeling the energy consumption optimization process of the refrigeration system as a Markov decision process (MDP) and combining the neural network and the PPO algorithm, the application can not only achieve optimal refrigeration effect and energy consumption efficiency under different operating conditions, but also significantly enhance the adaptability of the model to complex environmental states by using the neural network to accurately predict the state transition and the efficient strategy optimization capability of the PPO algorithm. In addition, the application makes full use of a large amount of real-time data collected by the sensor, and through continuous learning and optimization, can realize intelligent control and optimization of the refrigeration system, effectively reduce energy consumption, and significantly improve the energy utilization efficiency of the data center. The application solves the problems that the existing refrigeration system energy consumption optimization method lacks adaptability to complex environmental states, is difficult to achieve optimal refrigeration effect and energy consumption efficiency under different operating conditions, and cannot fully utilize a large amount of real-time data collected by the sensor, resulting in difficulty in achieving optimal energy consumption control effect. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 A flowchart of a refrigeration system energy consumption optimization method according to Embodiment 1 of the application;

[0041] Figure 2 A structural schematic diagram of a data center refrigeration system energy consumption optimization framework according to the application;

[0042] Figure 3 A structural schematic diagram of an MLP network in a state transition prediction model according to the application;

[0043] Figure 4 A structural schematic diagram of a refrigeration system energy consumption optimization device according to Embodiment 2 of the application;

[0044] Figure 5 A structural schematic diagram of a refrigeration system energy consumption optimization device according to Embodiment 3 of the application. DETAILED DESCRIPTION

[0045] To enable those skilled in the art to better understand the technical solution of the present invention, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0046] It is understood that the specific embodiments and accompanying drawings described herein are merely for explaining the invention and are not intended to limit the invention.

[0047] It is understood that, without conflict, the various embodiments and features in the embodiments of the present invention can be combined with each other.

[0048] It is understood that, for ease of description, only the parts related to the present invention are shown in the accompanying drawings, while the parts unrelated to the present invention are not shown in the drawings.

[0049] It is understood that each unit or module involved in the embodiments of the present invention may correspond to only one entity structure, or may be composed of multiple entity structures, or multiple units or modules may be integrated into one entity structure.

[0050] It is understood that the terms "first," "second," etc., in the embodiments of the present invention are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0051] It is understood that, without conflict, the functions and steps marked in the flowcharts and block diagrams of this invention may occur in a different order than that marked in the accompanying drawings.

[0052] It is understood that the flowcharts and block diagrams of this invention illustrate the possible architecture, functions, and operations of systems, apparatuses, devices, and methods according to various embodiments of this invention. Each block in the flowchart or block diagram may represent a unit, module, program segment, or code, containing executable instructions for implementing the specified function. Furthermore, each block or combination of blocks in the block diagram and flowchart can be implemented using a hardware-based system to achieve the specified function, or using a combination of hardware and computer instructions.

[0053] It is understood that the units and modules involved in the embodiments of the present invention can be implemented by software or by hardware. For example, the units and modules can be located in a processor.

[0054] Example 1:

[0055] This embodiment provides a method for optimizing the energy consumption of a refrigeration system, such as... Figure 1 As shown, the method includes:

[0056] Step S101: Model the energy consumption optimization process of the refrigeration system as a Markov decision process, and define the state space and action space of the energy consumption optimization scenario of the refrigeration system.

[0057] In this embodiment, for a refrigeration system, a Markov model of a refrigeration system energy consumption optimization problem is constructed using self-defined data center environment states, actions, state transitions, and rewards. Among them, the action is the refrigeration set value of the refrigeration system, the state mainly includes the temperature, humidity of each area in the data center and the action of the last time, and the reward includes temperature reward, humidity reward and energy consumption reward. The self-defined state transition refers to how the temperature and humidity of the data center will change when we change the refrigeration set value of the refrigeration system.

[0058] Optionally, the state space includes the temperature of each area in the data center, the humidity of each area, the outdoor humidity, the outdoor temperature, and the refrigeration set value of the refrigeration system at the last time.

[0059] The action space is defined as the refrigeration set value of the refrigeration system at the current time.

[0060] In this embodiment, the state space (State Space) includes the temperature of each area in the data center, the humidity of each area, the outdoor humidity, the outdoor temperature, and the refrigeration set value at the last time. Specifically, the state space can be represented as a multi-dimensional vector, for example, [T1, T2, …, H1, H2, …, Tout, Hout, Cooling_Set_Point_previous], where T represents the temperature of different areas, H represents the humidity of different areas, Tout and Hout represent the outdoor temperature and humidity respectively.

[0061] In this embodiment, the action space (Action Space) is defined as the set value of the refrigeration system at the current time, which is usually a continuous value. This value represents the set temperature or power of the control water chiller or other refrigeration equipment.

[0062] Optionally, the reward function of the Markov decision process is composed of humidity reward, temperature reward, and energy consumption reward.

[0063] The reward function of the temperature reward is: R_temp = -|T_target-T_actual|, where T_target is the target temperature and T_actual is the actual temperature.

[0064] The reward function of the humidity reward is: R_hum = -|H_target-H_actual|, where H_target is the target humidity and H_actual is the actual humidity.

[0065] The reward function of the energy consumption reward is: R_energy = -E_consumed, where E_consumed is the energy consumption value of the refrigeration system.

[0066] The total reward function is: R_total = w1*R_temp + w2*R_hum + w3*R_energy, where w1, w2, w3 are weighting coefficients to balance the contribution of different parts.

[0067] In this embodiment, the reward function (Reward Function) is composed of the following three parts:

[0068] (1) Temperature reward: the smaller the temperature deviation within the target temperature range, the higher the reward. Set the reward function as: R_temp = -|T_target - T_actual|, where T_target is the target temperature.

[0069] Where the target temperature refers to the above refrigeration set value (i.e. temperature set value); T_actual (actual temperature) represents the temperature value collected by the sensor (excluding outdoor); the temperature reward here is calculated for each area.

[0070] (2) Humidity reward: the smaller the humidity deviation within the target humidity range, the higher the reward. Set the reward function as: R_hum = -|H_target - H_actual|.

[0071] It should be noted that if the air conditioner can set the humidity value, then H_target (target humidity) here is the humidity set value of the air conditioner, and H_actual (actual humidity) is the real humidity value collected from the indoor area sensor; if the air conditioner cannot set the humidity value, then the appropriate H_target can be determined according to the demand, and the reinforcement learning will try to make the humidity of each area close to this H_target during the training process.

[0072] (3) Energy consumption reward: the lower the energy consumption of the refrigeration system, the higher the reward. Set the reward function as: R_energy = -E_consumed, where E_consumed is the energy consumption, which can be understood as the energy consumption value of the refrigeration system under the current temperature set value.

[0073] The total reward function can be expressed as: R_total = w1*R_temp + w2*R_hum + w3*R_energy, where w1, w2, w3 are weighting coefficients to balance the contribution of different parts.

[0074] It should be noted that the total reward function is the goal that the agent (i.e. the control algorithm of the refrigeration system) needs to maximize in the decision-making process. By designing a reasonable reward function, the agent can learn a control strategy that meets the temperature and humidity requirements and saves energy as much as possible.

[0075] Step S102: According to the sensor-acquired measurement point data, the state transition in the Markov decision process is predicted using a neural network to construct a state transition prediction model, the input parameters of the state transition prediction model being the environment state and action at the current time, and the output parameter being the environment state at the next time;

[0076] In this embodiment, multiple types of sensors (such as temperature sensors, humidity sensors, etc.) are installed at various key positions in the data center to collect measurement point data (i.e., environment state data) in real time. The distribution of these sensors covers all areas that need to be monitored in the data center to ensure the comprehensiveness and accuracy of the data.

[0077] Optionally, the state transition in the Markov decision process is predicted using a neural network according to the sensor-acquired measurement point data to construct a state transition prediction model, and the method specifically comprises:

[0078] The measurement point data of the data center is collected in real time by sensors, and the measurement point data includes the temperature of each area in the data center, the humidity of each area, the outdoor humidity, the outdoor temperature, and the refrigeration set value and energy consumption data of the refrigeration system;

[0079] The collected measurement point data is preprocessed;

[0080] The preprocessed measurement point data is used to construct the state transition prediction model based on the MLP (Multilayer Perceptron) in the deep neural network using a data-driven method.

[0081] In this embodiment, the preprocessing step can include:

[0082] (1) Data cleaning. Remove outliers and missing values, for example, use interpolation or nearest neighbor method to fill in missing data.

[0083] (2) Data normalization. Normalize the data to the range [0, 1] to adapt to the input requirements of the neural network. The normalization formula is: X_normalized = (X-X_min) / (X_max-X_min).

[0084] (3) Time series processing. Use the sliding window method to convert time series data into model input format.

[0085] In this embodiment, after preprocessing, a data-driven method is used to construct a state transition prediction model using a multilayer perceptron (MLP). In the training process, the model uses historical data of the data center to learn the rules of state transition.

[0086] Optionally, the state transition prediction model is constructed based on a multilayer perceptron (MLP) in a deep neural network using a data-driven method, specifically including:

[0087] The model architecture of the state transition prediction model is designed, and the architecture includes an input layer, a hidden layer, and an output layer.

[0088] A plurality of triplets including an environment state at a current time, an action at the current time, and an environment state at a next time are obtained from the preprocessed measurement point data, the environment state at the current time and the environment state at the next time each including a temperature of each region in the data center, a humidity of each region, an outdoor humidity, an outdoor temperature, and a refrigeration system refrigeration set value and energy consumption data at the corresponding time; the action at the current time includes the refrigeration system refrigeration set value.

[0089] The law of state transition is learned according to the triplets to obtain the constructed state transition prediction model.

[0090] In this embodiment, the state transition prediction model includes an input layer, a hidden layer, and an output layer, the input of the input layer is a combination of the environment state at the current time and the action, and the dimension is [state_dim+action_dim]; the hidden layer is two fully connected layers, each layer has 128 neurons, and the activation function is Leaky ReLU; the output of the output layer is the environment state at the next time, and the dimension is [state_dim].

[0091] In this embodiment, a plurality of (state, action, next_state) triplets are obtained from the preprocessed measurement point data; a mean square error (MSE) is used as a loss function: L = (1 / N) * Σ(y_true-y_pred)^2; an Adam optimizer is used, and the learning rate is set to 0.001. Wherein, state represents the environment state at the current time, action represents the action at the current time, and next_state represents the environment state at the next time.

[0092] In this embodiment, a training set and a validation set are constructed according to the triplets, and the performance of the model is evaluated by the loss change of the training set and the validation set. Early stopping method (Early Stopping) is used to prevent overfitting.

[0093] Step S103: based on the constructed state transition prediction model, a refrigeration system optimization model based on a PPO (Proximal Policy Optimization) algorithm is designed and trained.

[0094] In the embodiment, the PPO algorithm includes an Actor network (policy network) and a Critic network (value network), the Critic Net (value network) and the Actor Net (policy network) in the PPO algorithm can be designed based on a DNN (Deep Nueral Network, deep neural network), and an activation function in the network is Leaky ReLu.

[0095] Optionally, the refrigeration system optimization model based on the PPO algorithm is designed and trained based on the constructed state transition prediction model, and specifically includes the following steps.

[0096] Based on the constructed state transition prediction model, the architecture of the Actor network and the Critic network of the PPO algorithm is designed, the input of the Actor network is a state vector at a current time, the output is a mean value and a variance of a set action of the refrigeration system, the input of the Critic network is a state vector at a current time, and the output is a value function of a current state.

[0097] The training strategy of the PPO algorithm is designed, the advantage function is calculated by using Advantage estimation, the step length of policy update is constrained by using a Clip (clipping) method, the Actor network is updated by using a policy gradient method, and the Critic network is updated by using a minimum time difference error.

[0098] The refrigeration system optimization model based on the PPO algorithm is trained according to the training strategy.

[0099] In the embodiment, the PPO algorithm is used to design and train a refrigeration system energy consumption optimization model, and specifically includes the following steps.

[0100] (1) The Actor-Critic architecture is designed, the input of the Actor network is a state vector [state_dim] at a current time, the output is a mean value and a variance [mean, std_dev] of a set action of the refrigeration system, the network structure is a three-layer fully connected network, the hidden layer size is 256-128, and the activation function is Leaky ReLU. The input of the Critic network is a state vector [state_dim] at a current time, the output is a value function (Value) of a current state, and the network structure is similar to that of the Actor network.

[0101] (2) The PPO algorithm training strategy is designed, the advantage estimation (GAE-Lambda) is used to calculate the advantage function, the step length of the policy update is restricted by using the Clip method to ensure the stability of the policy update, and the loss function includes two parts of the policy loss and the value function loss, the Actor network is updated by using the policy gradient method, and the Critic network is updated by minimizing the time difference error (TD Error).

[0102] (3) In the training process, the Batch Size of each training sample is set to 64, 10 updates are performed in each training period (Epoch), the exploration rate (Exploration Rate) is gradually reduced, starting from 0.2 and gradually reducing to 0.01, and the total training rounds are set to 10000 rounds.

[0103] Step S104: The strategy output by the refrigeration system optimization model trained based on the PPO algorithm is deployed to the actual refrigeration system for real-time control and optimization.

[0104] In this embodiment, the strategy output by the trained Actor-Critic network is deployed to the actual data center refrigeration system for real-time control and optimization.

[0105] Specifically, based on the current state data (such as temperature, humidity, etc.), the Actor network outputs the optimal refrigeration setting value in real time to directly control the power output of the refrigeration equipment.

[0106] Optionally, after the strategy output by the refrigeration system optimization model trained based on the PPO algorithm is deployed to the actual refrigeration system for real-time control and optimization, the method further comprises:

[0107] During the actual operation of the data center, new environmental state data and feedback information are continuously collected;

[0108] The refrigeration system optimization model is updated through the new environmental state data and feedback information, wherein the update process adopts an incremental learning strategy.

[0109] It should be noted that the state transition prediction model can be regarded as a simulation of the real environment. In the real environment, we directly take actions to affect the environmental state, and the environmental state will change accordingly. The state transition prediction model simulates this process, so that our optimization model can be boldly tested on the state transition model, and then we can test and further optimize the preliminary optimization model in the real system, which is safer and more efficient.

[0110] Specifically, during the actual operation of the data center, new environmental state data and feedback information are continuously collected, and the strategy is updated through new data. The update process adopts an incremental learning strategy to prevent model overfitting.

[0111] Among them, the feedback information is mainly the real-time energy consumption value, because it involves the reward accumulation value (return) in the optimization model, if there is a significant decrease, then the optimization model needs to be updated; at the same time, the user's feedback can also guide the update, such as finding that the temperature and humidity deviate greatly from the set value, we can guide the model update by increasing the coefficient of the temperature and humidity reward in the reward function during the training process.

[0112] It should be noted that in order to solve the problems of high energy consumption and inflexible control strategy in the prior art, the present application provides a refrigeration system energy consumption optimization method, which models the energy consumption optimization process of the data center refrigeration system as a Markov decision process (MDP), combines deep neural networks (DNN) and PPO algorithm, and can realize intelligent control and optimization of the refrigeration system, reduce energy consumption, and improve the energy utilization efficiency of the data center.

[0113] In one specific embodiment, the refrigeration system energy consumption optimization method can include the following steps:

[0114] Step 1: Design a data center refrigeration system energy consumption optimization framework, model the optimization process as a Markov decision process, and define the state space and action space of the data center refrigeration system energy consumption optimization scenario.

[0115] Step 1.1: For the refrigeration system, use self-defined data center environmental state, action, state transition and reward to build a Markov model of the refrigeration system energy consumption optimization problem.

[0116] Step 1.2: The action is the refrigeration set value of the data center refrigeration system, the state mainly includes the temperature, humidity of the data center and the action at the last time, and the reward includes temperature reward, humidity reward and energy consumption reward.

[0117] Among them, the self-defined state transition refers to how the temperature and humidity of the data center will change when we change the refrigeration set value of the refrigeration system.

[0118] Step 2: Collect real-time data of key measuring points in the data center through sensors, and preprocess the collected data to ensure the accuracy and stability of the input data.

[0119] Among them, the key measuring point data includes the temperature of each area in the data center, the humidity of each area, the outdoor humidity, the outdoor temperature, and the refrigeration set value and energy consumption data of the refrigeration system.

[0120] Step 3: Based on the data collected by the sensor, a state transition prediction model is constructed.

[0121] Step 3.1: Use a data-driven method to construct an accurate state transition prediction model using a multi-layer perceptron (MLP) in a deep neural network.

[0122] Step 3.2: Select the measurement points composed of the same time state and action combination as the input parameters of the state transition prediction model, the next time state as the output parameter, and the activation function in the MLP network as Leaky ReLu. Based on a large number of fragments of historical running trajectories of the data center, the optimal network parameters are obtained through training.

[0123] It should be noted that the self-defined state transition in step 1 is equivalent to problem modeling, which determines the parameters of state transition and is a basic definition; while the state transition prediction model in this step is a solution method, emphasizing prediction and model, because we use a neural network model to predict state transition.

[0124] Step 4: Based on the constructed state transition prediction model, design and train a refrigeration system optimization model based on the PPO (Proximal Policy Optimization) algorithm.

[0125] Step 5: Strategy deployment and model update: design Critic Net (value network) and Actor Net (policy network) in the PPO algorithm based on DNN (Deep Nueral Network). The activation function in the network is Leaky ReLu. Deploy the strategy output by the trained Actor-Critic network to the actual data center refrigeration system for real-time control and optimization. In actual operation, update the strategy based on new data to further improve the energy efficiency and adaptive ability of the refrigeration system.

[0126] In another specific embodiment, a data center refrigeration system energy consumption optimization method based on the PPO algorithm is provided, which realizes intelligent optimization control of the data center refrigeration system through multi-step modeling, training, deployment and optimization to improve the energy utilization efficiency of the data center and reduce the refrigeration energy consumption. Figure 2 The data center refrigeration system energy consumption optimization framework is shown. The main steps include:

[0127] 1. Model the energy consumption optimization process of the data center refrigeration system as a Markov decision process (MDP). In this process:

[0128] State Space: including the temperature of each area in the data center, the humidity of each area, the outdoor humidity, the outdoor temperature, and the cooling set point of the previous time. Specifically, the state space can be represented as a multi-dimensional vector, for example, [T1, T2, …, H1, H2, …, Tout, Hout, Cooling_Set_Point_previous], where T represents the temperature of different areas, H represents the humidity of different areas, Tout and Hout represent the outdoor temperature and humidity respectively.

[0129] It should be noted that a single machine room can be taken as the optimization unit, and the area here can refer to a single machine room, specifically the temperature and humidity of the air outlet and air return of the air conditioner in each machine room, as well as the outdoor temperature and humidity. If better conditions are available, the temperature and humidity at both ends of the column and the center can be collected.

[0130] Action Space: defined as the set value of the cooling system at the current time, usually a continuous value. This value represents the set temperature or power of the control water chiller or other cooling equipment.

[0131] Reward Function: composed of the following three parts:

[0132] (1) Temperature reward: the smaller the temperature deviation within the target temperature range, the higher the reward. Set the reward function as: R_temp = |T_target - T_actual|, where T_target is the target temperature.

[0133] Where the target temperature refers to the cooling set value (i.e. temperature set value) above; T_actual (actual temperature) represents the temperature value collected by the sensor (excluding outdoor); the temperature reward here is calculated for each area.

[0134] (2) Humidity reward: the smaller the humidity deviation within the target humidity range, the higher the reward. Set the reward function as: R_hum = |H_target - H_actual|.

[0135] It should be noted that if the air conditioner can set the humidity value, then H_target (target humidity) here is the humidity set value of the air conditioner, and H_actual (actual humidity) is the real humidity value collected from the indoor area sensors; if the air conditioner cannot set the humidity value, then the appropriate H_target can be determined according to the needs, and reinforcement learning will try to make the humidity of each area close to this H_target during the training process.

[0136] (3) Energy consumption reward: The lower the energy consumption of the refrigeration system, the higher the reward. Set the reward function as: R_energy = -E_consumed, where E_consumed is the energy consumption, which can be understood as the energy consumption value of the refrigeration system under the current temperature setting.

[0137] It should be noted that the above reward values are negative, which can be understood as penalty values. In the energy consumption reward part, as long as there is energy consumption, it needs to be punished, and the higher the energy consumption, the greater the punishment, so negative numbers are used to represent it. In order to make the calculation logic of each element in the total reward function consistent, we also use negative reward logic for temperature and humidity.

[0138] The total reward function can be expressed as: R_total = w1*R_temp + w2*R_hum + w3*R_energy, where w1, w2, w3 are weighting coefficients used to balance the contributions of different parts.

[0139] It should be noted that the total reward function is the goal that the agent (i.e. the control algorithm of the refrigeration system) needs to maximize in the decision-making process. By designing a reasonable reward function, the agent can learn a control strategy that not only meets the temperature and humidity requirements, but also saves energy as much as possible.

[0140] 2. Real-time collection of environmental state data through multiple types of sensors (such as temperature sensors, humidity sensors, etc.) installed at key locations in the data center. The distribution of these sensors should cover all areas that need to be monitored to ensure the comprehensiveness and accuracy of the data. The collected data includes: temperature and humidity data of each area, outdoor temperature and humidity data, historical refrigeration system refrigeration set value and energy consumption data. The preprocessing steps include:

[0141] (1) Data cleaning. Remove outliers and missing values, for example, use interpolation or nearest neighbor method to fill in missing data.

[0142] (2) Data normalization. Normalize the data to the range [0, 1] to adapt to the input requirements of neural networks. The normalization formula is: X_normalized = (X - X_min) / (X_max - X_min).

[0143] (3) Time series processing. Use sliding window method to convert time series data into model input format.

[0144] 3. Use a data-driven approach to build a state transition prediction model using a multi-layer perceptron (MLP), Figure 3The structural diagram of the MLP network in the state transition prediction model is shown, including the input layer, hidden layer and output layer. During the training process, the model uses the historical data of the data center to learn the rules of state transition. The specific steps are as follows:

[0145] (1) Design the model architecture, the input is the current time environment state and action combination, the dimension is [state_dim+action_dim]; the hidden layer is two fully connected layers, each layer has 128 neurons, and the activation function is Leaky ReLU; the output is the next time environment state, the dimension is [state_dim].

[0146] (2) Sample multiple groups of (state, action, next_state) triples from historical data; use mean square error (MSE) as the loss function: L=(1 / N)*Σ(y_true-y_pred)^2; use Adam optimizer, learning rate is set to 0.001.

[0147] Where, state represents the current time environment state, including the temperature, humidity, and outdoor humidity, temperature of each area, and the current time cooling set value, energy consumption value. next_state represents the next time environment state, including the temperature, humidity, and outdoor humidity, temperature of each area, and the next time cooling set value, energy consumption value.

[0148] (3) Evaluate the performance of the model by the loss change of the training set and the validation set. Use EarlyStopping to prevent overfitting.

[0149] 4. Use PPO algorithm to design and train the cooling system energy consumption optimization model, which includes the following steps:

[0150] (1) Design the Actor-Critic architecture, the input of the Actor network is the current time state vector [state_dim], and the output is the mean and variance of the set action [mean, std_dev]. The network structure is a three-layer fully connected network, the hidden layer size is 256-128, and the activation function is Leaky ReLU. The input of the Critic network is the current time state vector [state_dim], and the output is the value function (Value) of the current state. The network structure is similar to the Actor network.

[0151] (2) Design the PPO algorithm training strategy and use Advantage estimation (GAE-Lambda) to calculate the advantage function; use the Clip method to constrain the step size of policy update to ensure the stability of policy update; the loss function includes two parts: policy loss and value function loss. Use the policy gradient method to update the Actor network and minimize the temporal difference error (TD Error) to update the Critic network.

[0152] (3) During the training process, the batch size for each training sample is set to 64, and 10 updates are performed in each training cycle (Epoch); the exploration rate is set to gradually decrease, starting from 0.2 and gradually decreasing to 0.01; the total number of training rounds is set to 10,000 rounds.

[0153] It's important to note that the state transition prediction model can be viewed as a simulation of a real-world environment. In a real environment, we directly take actions that affect the environmental state, causing it to change. The state transition prediction model simulates this process. This allows us to confidently test our optimized model on the state transition model, and then further refine and test this initial optimized model in a real system. This approach is safer and more efficient in training.

[0154] 5. Deploy the trained Actor-Critic network into the actual cooling system of the data center and optimize its control in the following aspects:

[0155] (1) Real-time strategy application: Based on the current state data (such as temperature, humidity, etc.), the Actor network outputs the optimal cooling setpoint in real time and directly controls the power output of the cooling equipment.

[0156] (2) Online Model Update: During actual operation in the data center, new environmental status data and feedback information are continuously collected, and the strategy is updated based on the new data. The update process adopts an incremental learning strategy to prevent model overfitting.

[0157] It should be noted that PPO algorithm training refers to gaining experience based on the exploration of state transition prediction models by our reinforcement learning agent. However, the optimized model trained in this way is ultimately based on virtual data and is not entirely reliable, therefore it needs to be updated online in a real system.

[0158] Among them, the feedback information is mainly the real-time energy consumption value, because it involves the reward cumulative value (return) in the optimization model, if there is a significant decline, then the optimization model needs to be updated; at the same time, the user's feedback can also guide the update, such as finding that the temperature and humidity deviate greatly from the set value, we can guide the model update by increasing the coefficient of the temperature and humidity reward in the training process reward function.

[0159] It should be noted that the refrigeration system energy consumption optimization method provided by the embodiment of the application can respond to environmental changes in real time, has the characteristics of self-learning ability and efficient intelligent control, and aims to further improve the energy utilization efficiency of the data center and reduce the operating cost.

[0160] In another specific embodiment, the refrigeration system energy consumption optimization method comprises:

[0161] S1, define the state and action of the data center refrigeration system energy consumption optimization scene, and use Markov decision process (MDP) to describe the refrigeration system energy consumption optimization process;

[0162] Among them, the state space of the data center refrigeration system energy consumption optimization scene includes the temperature of each area, the humidity of each area, the outdoor humidity, the outdoor temperature, and the last time refrigeration set value, and the action space includes the current time refrigeration set value.

[0163] Among them, the reward function of the Markov decision process is composed of humidity reward, temperature reward and energy consumption reward.

[0164] S2, real-time data center key data acquisition and preprocessing through sensors;

[0165] S3, use a neural network to predict the state transition in the Markov decision process, and construct a state transition prediction model; the input parameters of the state transition prediction model are the current time environmental state and action, and the output is the next time environmental state.

[0166] Specifically, a data-driven method is used, and a multi-layer perception machine (MLP) in a deep neural network is used to construct the state transition prediction model.

[0167] S4, based on the state transition prediction model, design and train a refrigeration system optimization model based on PPO algorithm;

[0168] S5, deploy the strategy output by the trained Actor-Critic network to the actual data center refrigeration system, and further update the optimization model during operation.

[0169] The refrigeration system energy consumption optimization method provided by the embodiment of the present application firstly models the energy consumption optimization process of the refrigeration system as a Markov decision process, defines the state space and action space of the refrigeration system energy consumption optimization scene; then according to the measured point data collected by the sensor, uses a neural network to predict the state transition in the Markov decision process, constructs a state transition prediction model, the input parameters of the state transition prediction model are the environmental state and action at the current time, and the output parameter is the environmental state at the next time; based on the constructed state transition prediction model, a refrigeration system optimization model based on a proximal policy optimization (PPO) algorithm is designed and trained; finally, the strategy output by the refrigeration system optimization model trained based on the PPO algorithm is deployed to the actual refrigeration system for real-time control and optimization. By modeling the energy consumption optimization process of the refrigeration system as a Markov decision process (MDP) and combining the neural network and the PPO algorithm, the present application can not only achieve the optimal refrigeration effect and energy consumption efficiency under different operating conditions, but also significantly enhance the adaptability of the model to complex environmental states by using the neural network to accurately predict the state transition and the efficient strategy optimization capability of the PPO algorithm. In addition, the present application makes full use of a large amount of real-time data collected by the sensor, and through continuous learning and optimization, can realize intelligent control and optimization of the refrigeration system, effectively reduce energy consumption, and significantly improve the energy utilization efficiency of the data center. The present application solves the problems of the prior art that the existing refrigeration system energy consumption optimization method lacks adaptability to complex environmental states, is difficult to achieve optimal refrigeration effect and energy consumption efficiency under different operating conditions, and cannot fully utilize a large amount of real-time data collected by the sensor, resulting in difficulty in achieving optimal energy consumption control effect.

[0170] Embodiment 2:

[0171] As shown in Figure 4 The present embodiment provides a refrigeration system energy consumption optimization device for executing the above refrigeration system energy consumption optimization method, comprising:

[0172] An optimization process modeling module 11 is configured to model the energy consumption optimization process of the refrigeration system as a Markov decision process, and define the state space and action space of the refrigeration system energy consumption optimization scene;

[0173] A state transition prediction module 12 is connected with the optimization process modeling module 11, configured to use a neural network to predict the state transition in the Markov decision process according to the measured point data collected by the sensor, and construct a state transition prediction model, the input parameters of the state transition prediction model are the environmental state and action at the current time, and the output parameter is the environmental state at the next time;

[0174] The optimization model construction module 13 is connected with the state transition prediction module 12, and is configured to design and train a refrigeration system optimization model based on a proximal policy optimization (PPO) algorithm based on the constructed state transition prediction model.

[0175] The real-time control optimization module 14 is connected with the optimization model construction module 13, and is configured to deploy a strategy output by the refrigeration system optimization model trained based on the PPO algorithm to an actual refrigeration system for real-time control and optimization.

[0176] Optionally, the state space includes temperatures of regions in the data center, humidities of the regions, outdoor humidity, outdoor temperature, and a refrigeration set value of the refrigeration system at a previous time point.

[0177] The action space is defined as the refrigeration set value of the refrigeration system at the current time point.

[0178] Optionally, the reward function of the Markov decision process is composed of a humidity reward, a temperature reward, and an energy consumption reward.

[0179] The reward function of the temperature reward is R_temp = |T_target - T_actual|, where T_target is a target temperature and T_actual is an actual temperature.

[0180] The reward function of the humidity reward is R_hum = |H_target - H_actual|, where H_target is a target humidity and H_actual is an actual humidity.

[0181] The reward function of the energy consumption reward is R_energy = -E_consumed, where E_consumed is an energy consumption value of the refrigeration system.

[0182] The total reward function is R_total = w1*R_temp + w2*R_hum + w3*R_energy, where w1, w2, and w3 are weighting coefficients for balancing contributions of different parts.

[0183] Optionally, the state transition prediction module 12 includes:

[0184] The real-time acquisition unit is configured to acquire, by a sensor, data point data of the data center in real time, the data point data including temperatures of regions in the data center, humidities of the regions, outdoor humidity, outdoor temperature, and a refrigeration set value and energy consumption data of the refrigeration system.

[0185] The preprocessing unit is configured to preprocess the acquired data point data.

[0186] The state transition prediction model construction unit is configured to use a data-driven method to construct the state transition prediction model based on a multilayer perceptron (MLP) in a deep neural network, using the preprocessed measurement point data.

[0187] Optionally, the state transition prediction model construction unit specifically includes:

[0188] The model architecture design unit is configured to design a model architecture of the state transition prediction model, the architecture including an input layer, a hidden layer, and an output layer.

[0189] The triple obtaining unit is configured to obtain a plurality of groups of triples including an environment state at a current time, an action at the current time, and an environment state at a next time from the preprocessed measurement point data, the environment state at the current time and the environment state at the next time each including a temperature of each region in the data center, a humidity of each region, an outdoor humidity, an outdoor temperature, and a refrigeration system refrigeration set value and energy consumption data at a corresponding time; and the action at the current time including the refrigeration system refrigeration set value.

[0190] The first construction unit is configured to learn a rule of state transition according to the triples to obtain the constructed state transition prediction model.

[0191] Optionally, the optimization model construction module 13 includes:

[0192] The network design unit is configured to design an architecture of an Actor network and a Critic network of the PPO algorithm based on the constructed state transition prediction model, an input of the Actor network being a state vector at a current time, and an output of the Actor network being a mean value and a variance of a set action of the refrigeration system, an input of the Critic network being the state vector at the current time, and an output of the Critic network being a value function of the current state.

[0193] The training strategy design unit is configured to design a training strategy of the PPO algorithm, calculate an advantage function using an Advantage estimation, constrain a step length of policy updating using a clipping (Clip) method, update the Actor network using a policy gradient method, and update the Critic network using a minimization of a temporal difference error.

[0194] The model training unit is configured to train the refrigeration system optimization model based on the PPO algorithm according to the training strategy.

[0195] Optionally, the device further includes:

[0196] The data collection module is configured to continuously collect new environment state data and feedback information during actual operation of the data center.

[0197] An optimization model updating module is configured to update the refrigeration system optimization model by using the new environment state data and feedback information, wherein the updating process adopts an incremental learning strategy.

[0198] Embodiment 3

[0199] Reference Figure 5 The embodiment provides a refrigeration system energy consumption optimization device, which comprises a memory 21 and a processor 22. The memory 21 stores a computer program, and the processor 22 is configured to execute the computer program to perform the refrigeration system energy consumption optimization method in the embodiment 1.

[0200] The memory 21 is connected with the processor 22. The memory 21 can be a flash memory or a read-only memory or other memories, and the processor 22 can be a central processing unit or a single-chip microcomputer.

[0201] Embodiment 4

[0202] The embodiment provides a computer readable storage medium, which stores a computer program. When the computer program is executed by a processor, the refrigeration system energy consumption optimization method in the above embodiment 1 is implemented.

[0203] The computer readable storage medium includes volatile or non-volatile, removable or non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, computer program modules or other data. The computer readable storage medium includes but is not limited to RAM (Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable read only memory), flash memory or other memory technologies, CD-ROM (Compact Disc Read-Only Memory), digital versatile disc (DVD) or other optical disc storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer.

[0204] In summary, the refrigeration system energy consumption optimization method, device and readable storage medium provided by the embodiment of the present application first model the energy consumption optimization process of the refrigeration system as a Markov decision process, define the state space and action space of the refrigeration system energy consumption optimization scene; then according to the measurement point data collected by the sensor, use a neural network to predict the state transition in the Markov decision process, construct a state transition prediction model, the input parameters of the state transition prediction model are the environmental state and action at the current time, and the output parameter is the environmental state at the next time; then based on the constructed state transition prediction model, design and train a refrigeration system optimization model based on the proximal policy optimization (PPO) algorithm; finally, deploy the strategy output by the refrigeration system optimization model trained based on the PPO algorithm to the actual refrigeration system for real-time control and optimization. The present application models the energy consumption optimization process of the refrigeration system as a Markov decision process (MDP), and combines a neural network and a PPO algorithm, which not only can achieve optimal refrigeration effect and energy consumption efficiency under different operating conditions, but also significantly enhances the adaptability of the model to complex environmental states through the precise prediction of state transition by the neural network and the efficient strategy optimization capability of the PPO algorithm. In addition, the present application makes full use of a large amount of real-time data collected by the sensor, and through continuous learning and optimization, can realize intelligent control and optimization of the refrigeration system, effectively reduce energy consumption, and significantly improve the energy utilization efficiency of the data center. The present application solves the problems that the existing refrigeration system energy consumption optimization method lacks adaptability to complex environmental states, is difficult to achieve optimal refrigeration effect and energy consumption efficiency under different operating conditions, and cannot fully utilize a large amount of real-time data collected by the sensor, resulting in difficulty in achieving optimal energy consumption control effect.

[0205] It can be understood that the above embodiments are only exemplary embodiments adopted for illustrating the principles of the present application, and the present application is not limited thereto. Various modifications and improvements can be made by those skilled in the art without departing from the spirit and essence of the present application, and these modifications and improvements are also considered to be within the protection scope of the present application.

Claims

1. A method for energy consumption optimization of a refrigeration system, characterized in that, The method comprises: Modeling the energy consumption optimization process of the refrigeration system as a Markov decision process, defining the state space and action space of the refrigeration system energy consumption optimization scene; According to the measured point data collected by the sensor, the state transition in the Markov decision process is predicted using a neural network to construct a state transition prediction model, the input parameters of the state transition prediction model being the environmental state and action at the current time, and the output parameter being the environmental state at the next time; Based on the constructed state transition prediction model, a refrigeration system optimization model based on the proximal policy optimization (PPO) algorithm is designed and trained; The strategy output by the refrigeration system optimization model trained based on the PPO algorithm is deployed to the actual refrigeration system for real-time control and optimization; The state transition prediction model is constructed by using a neural network to predict the state transition in the Markov decision process according to the measured point data collected by the sensor, specifically comprising: Real-time acquisition of measured point data of the data center by a sensor, the measured point data including the temperature of each region in the data center, the humidity of each region, outdoor humidity, outdoor temperature, and refrigeration set value and energy consumption data of the refrigeration system; Preprocessing the collected measured point data; Designing a model architecture of the state transition prediction model, the architecture including an input layer, a hidden layer, and an output layer; Obtaining a plurality of triplets including the environmental state at the current time, the action at the current time, and the environmental state at the next time from the preprocessed measured point data, the environmental state at the current time and the environmental state at the next time both including the temperature of each region in the data center, the humidity of each region, outdoor humidity, outdoor temperature, and refrigeration set value and energy consumption data of the refrigeration system at the corresponding time; the action at the current time including the refrigeration set value of the refrigeration system; Learning the law of state transition according to the triplets to obtain the constructed state transition prediction model.

2. The method of claim 1, wherein, The state space includes the temperature of each region in the data center, the humidity of each region, outdoor humidity, outdoor temperature, and the refrigeration set value of the refrigeration system at the previous time; The action space is defined as the refrigeration set value of the refrigeration system at the current time.

3. The method of claim 1, wherein, The reward function of the Markov decision process is composed of humidity reward, temperature reward, and energy consumption reward; The reward function of the temperature reward is: R_temp = -|T_target - T_actual|, wherein T_target is the target temperature and T_actual is the actual temperature; The reward function of the humidity reward is: R_hum = -|H_target - H_actual|, wherein H_target is the target humidity and H_actual is the actual humidity; The reward function of the energy consumption reward is: R_energy = -E_consumed, wherein E_consumed is the energy consumption value of the refrigeration system; The total reward function is: R_total = w1*R_temp + w2*R_hum + w3*R_energy, wherein w1, w2, and w3 are weighting coefficients for balancing the contributions of different parts.

4. The method of claim 2, wherein, The state transition prediction model is constructed, and a refrigeration system optimization model based on a proximal policy optimization (PPO) algorithm is designed and trained, specifically including: Based on the constructed state transition prediction model, the architecture of the Actor network and the Critic network of the PPO algorithm is designed, the input of the Actor network is the state vector at the current time, and the output is the mean and variance of the set action of the refrigeration system, the input of the Critic network is the state vector at the current time, and the output is the value function of the current state; The training strategy of the PPO algorithm is designed, the advantage function is calculated by using the advantage estimation, the step length of the policy update is constrained by using the clipping method, the Actor network is updated by using the policy gradient method, and the Critic network is updated by using the minimum time difference error; The refrigeration system optimization model based on the PPO algorithm is trained according to the training strategy.

5. The method of claim 2, wherein, After the strategy output by the refrigeration system optimization model trained based on the PPO algorithm is deployed to the actual refrigeration system for real-time control and optimization, the method further includes: During the actual operation of the data center, new environmental state data and feedback information are continuously collected; The refrigeration system optimization model is updated through the new environmental state data and feedback information, wherein the update process adopts an incremental learning strategy.

6. A refrigeration system energy consumption optimization device, characterized by, The device includes: An optimization process modeling module is configured to model the energy consumption optimization process of a refrigeration system as a Markov decision process, and define the state space and action space of the refrigeration system energy consumption optimization scene; A state transition prediction module is connected to the optimization process modeling module, configured to use a neural network to predict the state transition in the Markov decision process according to the measured point data collected by a sensor, and construct a state transition prediction model, wherein the input parameters of the state transition prediction model are the environmental state and action at the current time, and the output parameter is the environmental state at the next time; An optimization model construction module is connected to the state transition prediction module, configured to design and train a refrigeration system optimization model based on a proximal policy optimization (PPO) algorithm based on the constructed state transition prediction model; A real-time control optimization module is connected to the optimization model construction module, configured to deploy the strategy output by the refrigeration system optimization model trained based on the PPO algorithm to the actual refrigeration system for real-time control and optimization. The state transition prediction module includes: A real-time acquisition unit is configured to acquire measured point data of a data center in real time through a sensor, wherein the measured point data includes the temperature of each area in the data center, the humidity of each area, outdoor humidity, outdoor temperature, and refrigeration set value and energy consumption data of the refrigeration system; A preprocessing unit is configured to preprocess the acquired measured point data; A model architecture design unit is configured to design the model architecture of the state transition prediction model, wherein the architecture includes an input layer, a hidden layer, and an output layer; The triple acquisition unit is configured to obtain a plurality of triples from the preprocessed measurement point data, each triple including an environment state at a current time, an action at the current time, and an environment state at a next time, wherein the environment state at the current time and the environment state at the next time each include a temperature of each region in the data center, a humidity of each region, an outdoor humidity, an outdoor temperature, and a refrigeration system refrigeration setting value and energy consumption data at a corresponding time; and the action at the current time includes the refrigeration system refrigeration setting value. The first construction unit is configured to learn a rule of state transition according to the triples to obtain a constructed state transition prediction model.

7. A refrigeration system energy consumption optimization device, characterized by, The computer readable storage medium stores a computer program, and the processor is configured to execute the computer program to implement the refrigeration system energy consumption optimization method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the processor is configured to execute the computer program to implement the refrigeration system energy consumption optimization method according to any one of claims 1-5.

Images