Refrigerating system energy consumption optimization method and device and readable storage medium

Abstract

The invention provides a refrigeration system energy consumption optimization method and device and a readable storage medium, and relates to the technical field of energy consumption optimizing.The method comprises the steps that an energy consumption optimization process of a refrigeration system is modeled into a Markov decision process, and a state space and an action space of an energy consumption optimization scene of the refrigeration system are defined; according to measuring point data collected by a sensor, state transition in the Markov decision process is predicted by using a neural network, and a state transition prediction model is constructed; based on the constructed state transition prediction model, designing and training a refrigeration system optimization model based on a near-end strategy optimization PPO algorithm; and deploying a strategy output by the refrigeration system optimization model trained based on the PPO algorithm into an actual refrigeration system, and performing real-time control and optimization. According to the method, the device and the medium, the problems that an existing refrigeration system energy consumption optimization method lacks adaptive capacity to complex environment states, and the optimal refrigeration effect and energy consumption efficiency are difficult to achieve under different operation conditions can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of energy consumption optimization, and in particular to a method, a device and a readable storage medium for optimizing energy consumption of a refrigeration system. Background Art

[0002] With the rapid development of cloud computing, big data and artificial intelligence technologies, the scale and number of data centers are constantly expanding. A large number of servers, storage devices and network devices in data centers will generate a lot of heat during operation, so an efficient cooling system is essential to ensure the stable operation of data centers. The energy consumption of the cooling system accounts for a large proportion of the total energy consumption of the data center, so how to reduce the energy consumption of the cooling system while ensuring the normal operation of the equipment has become a hot topic in current research.

[0003] At present, the cooling system of data centers mainly relies on traditional rule-based or threshold-based control strategies. However, these traditional control strategies lack the ability to adapt to complex environmental conditions and are difficult to achieve optimal cooling effects and energy efficiency under different operating conditions. At the same time, existing optimization methods cannot fully utilize the large amount of real-time data collected by sensors in data centers, lack dynamic adaptive capabilities and intelligence, and are difficult to achieve optimal energy consumption control effects. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a refrigeration system energy consumption optimization method, device and readable storage medium in response to the above-mentioned deficiencies in the prior art, so as to solve the problems that the existing refrigeration system energy consumption optimization method lacks the ability to adapt to complex environmental conditions, is difficult to achieve the optimal refrigeration effect and energy efficiency under different operating conditions, and cannot make full use of the large amount of real-time data collected by the sensor, resulting in difficulty in achieving the optimal energy consumption control effect.

[0005] In a first aspect, the present invention provides a method for optimizing energy consumption of a refrigeration system, the method comprising:

[0006] The energy consumption optimization process of the refrigeration system is modeled as a Markov decision process, and the state space and action space of the refrigeration system energy consumption optimization scenario are defined;

[0007] According to the measurement point data collected by the sensor, a neural network is used to predict the state transition in the Markov decision process, and a state transition prediction model is constructed, wherein the input parameters of the state transition prediction model are the environmental state and action at the current moment, and the output parameter is the environmental state at the next moment;

[0008] Based on the constructed state transition prediction model, a refrigeration system optimization model based on the proximal strategy optimization PPO algorithm is designed and trained;

[0009] The strategy output by the refrigeration system optimization model trained based on the PPO algorithm is deployed to the actual refrigeration system for real-time control and optimization.

[0010] Further, the state space includes the temperature of each area in the data center, the humidity of each area, the outdoor humidity, the outdoor temperature, and the cooling set value of the cooling system at the last moment;

[0011] The action space is defined as the refrigeration setting value of the refrigeration system at the current moment.

[0012] Furthermore, the reward function of the Markov decision process consists of three parts: humidity reward, temperature reward, and energy consumption reward;

[0013] The reward function of the temperature reward is: R_temp=-|T_target-T_actual|, where T_target is the target temperature and T_actual is the actual temperature;

[0014] The reward function of the humidity reward is: R_hum=-|H_target-H_actual|, where H_target is the target humidity and H_actual is the actual humidity;

[0015] The reward function of the energy consumption reward is: R_energy=-E_consumed, where E_consumed is the energy consumption value of the refrigeration system;

[0016] The total reward function is: R_total = w1*R_temp+w2*R_hum+w3*R_energy, where w1, w2, and w3 are weighted coefficients used to balance the contributions of different parts.

[0017] Furthermore, the state transition in the Markov decision process is predicted using a neural network based on the measurement point data collected by the sensor to construct a state transition prediction model, which specifically includes:

[0018] The sensors collect the data of the data center in real time, including the temperature of each area in the data center, the humidity of each area, the outdoor humidity, the outdoor temperature, and the refrigeration setting value and energy consumption data of the refrigeration system;

[0019] Preprocessing the collected measuring point data;

[0020] The state transfer prediction model is constructed based on the multi-layer perceptron MLP in the deep neural network by using the preprocessed measurement point data and a data-driven method.

[0021] Furthermore, the preprocessed measurement point data is used to construct the state transition prediction model based on the multi-layer perceptron MLP in the deep neural network using a data-driven method, specifically including:

[0022] Designing a model architecture of the state transition prediction model, wherein the architecture includes an input layer, a hidden layer, and an output layer;

[0023] A plurality of triplets including the current environment state, the current action, and the next environment state are obtained from the pre-processed measurement point data, wherein the current environment state and the next environment state both include the temperature of each area in the data center, the humidity of each area, the outdoor humidity, the outdoor temperature, and the refrigeration setting value and energy consumption data of the refrigeration system at the corresponding time; the action at the current time includes the refrigeration setting value of the refrigeration system;

[0024] The law of state transfer is learned according to the triples to obtain the constructed state transfer prediction model.

[0025] Furthermore, based on the constructed state transition prediction model, a refrigeration system optimization model based on the proximal strategy optimization (PPO) algorithm is designed and trained, specifically including:

[0026] Based on the constructed state transition prediction model, the architecture of the Actor network and the Critic network of the PPO algorithm is designed, wherein the input of the Actor network is the state vector at the current moment, and the output is the mean and variance of the set action of the refrigeration system; the input of the Critic network is the state vector at the current moment, and the output is the value function of the current state;

[0027] Design the training strategy of the PPO algorithm, use Advantage estimation to calculate the advantage function, use Clip to constrain the step size of the policy update, use the policy gradient method to update the Actor network, and use the minimization of the temporal difference error to update the Critic network;

[0028] The refrigeration system optimization model based on the PPO algorithm is trained according to the training strategy.

[0029] Furthermore, after deploying the strategy output by the refrigeration system optimization model trained based on the PPO algorithm to the actual refrigeration system for real-time control and optimization, the method further includes:

[0030] During the actual operation of the data center, continuously collect new environmental status data and feedback information;

[0031] The refrigeration system optimization model is updated using the new environmental status data and feedback information, wherein the updating process adopts an incremental learning strategy.

[0032] In a second aspect, the present invention provides a refrigeration system energy consumption optimization device, the device comprising:

[0033] The optimization process modeling module is used to model the energy consumption optimization process of the refrigeration system as a Markov decision process and define the state space and action space of the refrigeration system energy consumption optimization scenario;

[0034] A state transfer prediction module is connected to the optimization process modeling module and is used to predict the state transfer in the Markov decision process using a neural network based on the measurement point data collected by the sensor, and to construct a state transfer prediction model, wherein the input parameters of the state transfer prediction model are the environmental state and action at the current moment, and the output parameter is the environmental state at the next moment;

[0035] An optimization model construction module, connected to the state transfer prediction module, is used to design and train a refrigeration system optimization model based on the proximal strategy optimization PPO algorithm based on the constructed state transfer prediction model;

[0036] The real-time control optimization module is connected to the optimization model building module and is used to deploy the strategy output by the refrigeration system optimization model trained based on the PPO algorithm into the actual refrigeration system for real-time control and optimization.

[0037] In a third aspect, the present invention provides a refrigeration system energy consumption optimization device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the refrigeration system energy consumption optimization method described in the first aspect above.

[0038] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for optimizing energy consumption of a refrigeration system described in the first aspect above is implemented.

[0039] The present invention provides a method, device and readable storage medium for optimizing energy consumption of a refrigeration system. First, the energy consumption optimization process of the refrigeration system is modeled as a Markov decision process, and the state space and action space of the energy consumption optimization scenario of the refrigeration system are defined; then, according to the measurement point data collected by the sensor, a neural network is used to predict the state transition in the Markov decision process, and a state transition prediction model is constructed, wherein the input parameters of the state transition prediction model are the environmental state and action at the current moment, and the output parameter is the environmental state at the next moment; then, based on the constructed state transition prediction model, a refrigeration system optimization model based on the proximal policy optimization PPO algorithm is designed and trained; finally, the strategy output by the refrigeration system optimization model trained based on the PPO algorithm is deployed to the actual refrigeration system for real-time control and optimization. The present invention, by modeling the energy consumption optimization process of the refrigeration system as a Markov decision process (MDP), and combining the neural network and the PPO algorithm, can not only achieve the optimal refrigeration effect and energy efficiency under different operating conditions, but also significantly enhance the model's adaptability to complex environmental conditions by using the neural network to accurately predict state transitions and the efficient policy optimization capability of the PPO algorithm. In addition, the present invention makes full use of the large amount of real-time data collected by the sensor, and through continuous learning and optimization, it can realize intelligent control and optimization of the refrigeration system, effectively reduce energy consumption, and significantly improve the energy efficiency of the data center. It solves the problem that the existing refrigeration system energy consumption optimization method lacks the ability to adapt to complex environmental conditions, is difficult to achieve the optimal refrigeration effect and energy efficiency under different operating conditions, and cannot make full use of the large amount of real-time data collected by the sensor, resulting in difficulty in achieving the optimal energy consumption control effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flow chart of a method for optimizing energy consumption of a refrigeration system according to Embodiment 1 of the present invention;

[0041] Figure 2 A schematic diagram of the structure of a data center refrigeration system energy consumption optimization framework according to an embodiment of the present invention;

[0042] Figure 3 A schematic diagram of the structure of an MLP network in a state transition prediction model according to an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram of the structure of a refrigeration system energy consumption optimization device according to Embodiment 2 of the present invention;

[0044] Figure 5 This is a structural schematic diagram of a refrigeration system energy consumption optimization device according to Example 3 of the present invention. DETAILED DESCRIPTION

[0045] In order to enable those skilled in the art to better understand the technical solution of the present invention, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0046] It should be understood that the specific embodiments and drawings described herein are only used to explain the present invention rather than to limit the present invention.

[0047] It can be understood that, in the absence of conflict, the various embodiments of the present invention and the various features in the embodiments can be combined with each other.

[0048] It can be understood that, for the convenience of description, the drawings of the present invention only show the parts related to the present invention, while the parts irrelevant to the present invention are not shown in the drawings.

[0049] It can be understood that each unit and module involved in the embodiments of the present invention may correspond to only one physical structure, or may be composed of multiple physical structures, or multiple units and modules may be integrated into one physical structure.

[0050] It can be understood that the terms "first", "second", etc. in the embodiments of the present invention are used to distinguish different objects, or to distinguish different processing of the same object, rather than to describe a specific order of objects.

[0051] It can be understood that, without conflict, the functions and steps marked in the flowcharts and block diagrams of the present invention may occur in an order different from that marked in the drawings.

[0052] It is understood that the flowcharts and block diagrams of the present invention illustrate the possible architectures, functions, and operations of the systems, devices, equipment, and methods according to the various embodiments of the present invention. Each box in the flowchart or block diagram may represent a unit, module, program segment, or code, which contains executable instructions for implementing the specified functions. Moreover, each box or combination of boxes in the block diagram and flowchart may be implemented by a hardware-based system that implements the specified functions, or may be implemented by a combination of hardware and computer instructions.

[0053] It can be understood that the units and modules involved in the embodiments of the present invention can be implemented by software or hardware. For example, the units and modules can be located in a processor.

[0054] Embodiment 1:

[0055] This embodiment provides a method for optimizing energy consumption of a refrigeration system. Figure 1 As shown, the method includes:

[0056] Step S101: Modeling the energy consumption optimization process of the refrigeration system as a Markov decision process, and defining the state space and action space of the refrigeration system energy consumption optimization scenario.

[0057] In this embodiment, for the refrigeration system, a Markov model for the energy consumption optimization problem of the refrigeration system is constructed using customized data center environmental states, actions, state transitions, and rewards. Among them, the action is the refrigeration setting value of the refrigeration system, the state mainly includes the temperature, humidity, and action of each area in the data center at the previous moment, and the reward includes temperature reward, humidity reward, and energy consumption reward. The customized state transition refers to how the temperature and humidity of the data center will change when we change the refrigeration setting value of the refrigeration system.

[0058] Optionally, the state space includes the temperature of each area in the data center, the humidity of each area, the outdoor humidity, the outdoor temperature, and the cooling set value of the cooling system at the last moment;

[0059] The action space is defined as the refrigeration setting value of the refrigeration system at the current moment.

[0060] In this embodiment, the state space includes the temperature of each area in the data center, the humidity of each area, the outdoor humidity, the outdoor temperature, and the cooling set point at the previous moment. Specifically, the state space can be represented as a multidimensional vector, for example, [T1, T2, ..., H1, H2, ..., Tout, Hout, Cooling_Set_Point_previous], where T represents the temperature of different areas, H represents the humidity of different areas, and Tout and Hout represent the outdoor temperature and humidity respectively.

[0061] In this embodiment, the action space is defined as the set value of the refrigeration system at the current moment, which is usually a continuous value. The value represents the set temperature or power of the chiller or other refrigeration equipment.

[0062] Optionally, the reward function of the Markov decision process consists of three parts: humidity reward, temperature reward, and energy consumption reward;

[0063] The reward function of the temperature reward is: R_temp=-|T_target-T_actual|, where T_target is the target temperature and T_actual is the actual temperature;

[0064] The reward function of the humidity reward is: R_hum=-|H_target-H_actual|, where H_target is the target humidity and H_actual is the actual humidity;

[0065] The reward function of the energy consumption reward is: R_energy=-E_consumed, where E_consumed is the energy consumption value of the refrigeration system;

[0066] The total reward function is: R_total = w1*R_temp+w2*R_hum+w3*R_energy, where w1, w2, and w3 are weighted coefficients used to balance the contributions of different parts.

[0067] In this embodiment, the reward function consists of the following three parts:

[0068] (1) Temperature reward: The smaller the temperature deviation within the target temperature range, the higher the reward. Set the reward function to: R_temp = -|T_target - T_actual|, where T_target is the target temperature.

[0069] Among them, the target temperature refers to the cooling set value above (that is, the temperature set value); T_actual (actual temperature) refers to the temperature value collected by the sensor (excluding the outdoor); the temperature reward here is calculated for each area.

[0070] (2) Humidity reward: The smaller the humidity deviation within the target humidity range, the higher the reward. Set the reward function to: R_hum = -|H_target - H_actual|.

[0071] It should be noted that if the air conditioner can set the humidity value, then H_target (target humidity) here is the humidity setting value of the air conditioner, and H_actual (actual humidity) is the actual humidity value collected by sensors in various indoor areas; if the air conditioner cannot set the humidity value, then the appropriate H_target can be determined according to needs. Reinforcement learning will try to make the humidity of each area close to this H_target during training.

[0072] (3) Energy consumption reward: The lower the energy consumption of the refrigeration system, the higher the reward. The reward function is set as: R_energy = -E_consumed, where E_consumed is the energy consumption, which can be understood as the energy consumption value of the refrigeration system under the current temperature setting value.

[0073] The total reward function can be expressed as: R_total = w1*R_temp+w2*R_hum+w3*R_energy, where w1, w2, w3 are weighted coefficients used to balance the contributions of different parts.

[0074] It should be noted that the total reward function is the goal that the agent (i.e., the control algorithm of the refrigeration system) needs to maximize in the decision-making process. By designing a reasonable reward function, the agent can be guided to learn a control strategy that not only meets the temperature and humidity requirements but also saves energy as much as possible.

[0075] Step S102: According to the measurement point data collected by the sensor, a neural network is used to predict the state transition in the Markov decision process, and a state transition prediction model is constructed, wherein the input parameters of the state transition prediction model are the environmental state and action at the current moment, and the output parameter is the environmental state at the next moment;

[0076] In this embodiment, various types of sensors (such as temperature sensors, humidity sensors, etc.) are installed at various key locations in the data center to collect measurement point data (i.e., environmental status data) in real time. The distribution of these sensors covers all areas of the data center that need to be monitored to ensure the comprehensiveness and accuracy of the data.

[0077] Optionally, the predicting the state transition in the Markov decision process using a neural network based on the measurement point data collected by the sensor to construct a state transition prediction model specifically includes:

[0078] The sensors collect the data of the data center in real time, including the temperature of each area in the data center, the humidity of each area, the outdoor humidity, the outdoor temperature, and the refrigeration setting value and energy consumption data of the refrigeration system;

[0079] Preprocessing the collected measuring point data;

[0080] The state transfer prediction model is constructed based on the MLP (Multilayer Perceptron) in the deep neural network by using the preprocessed measurement point data and a data-driven method.

[0081] In this embodiment, the preprocessing step may include:

[0082] (1) Data cleaning: Remove outliers and missing values, such as using interpolation or the nearest neighbor method to fill in missing data.

[0083] (2) Data normalization. Normalize the data to the range of [0,1] to meet the input requirements of the neural network. The normalization formula is: X_normalized = (X-X_min) / (X_max-X_min).

[0084] (3) Time series processing: Use the sliding window method to convert time series data into the model input format.

[0085] In this embodiment, after preprocessing, a data-driven approach is adopted to construct a state transition prediction model using a multi-layer perceptron (MLP). During the training process, the model uses historical data from the data center to learn the law of state transition.

[0086] Optionally, the preprocessed measurement point data is used to construct the state transition prediction model based on a multi-layer perceptron MLP in a deep neural network using a data-driven method, specifically including:

[0087] Designing a model architecture of the state transition prediction model, wherein the architecture includes an input layer, a hidden layer, and an output layer;

[0088] A plurality of triplets including the current environment state, the current action, and the next environment state are obtained from the pre-processed measurement point data, wherein the current environment state and the next environment state both include the temperature of each area in the data center, the humidity of each area, the outdoor humidity, the outdoor temperature, and the refrigeration setting value and energy consumption data of the refrigeration system at the corresponding time; the action at the current time includes the refrigeration setting value of the refrigeration system;

[0089] The law of state transfer is learned according to the triples to obtain the constructed state transfer prediction model.

[0090] In this embodiment, the state transfer prediction model includes an input layer, a hidden layer and an output layer. The input of the input layer is the environment state and action combination at the current moment, and the dimension is [state_dim+action_dim]; the hidden layer is two fully connected layers, each layer has 128 neurons, and the activation function is Leaky ReLU; the output of the output layer is the environment state at the next moment, and the dimension is [state_dim].

[0091] In this embodiment, multiple groups of (state, action, next_state) triplets are obtained from the preprocessed measurement point data; the mean square error (MSE) is used as the loss function: L = (1 / N) * Σ (y_true-y_pred)^2; the Adam optimizer is used, and the learning rate is set to 0.001. Among them, state represents the current state of the environment, action represents the action at the current moment, and next_state represents the next state of the environment.

[0092] In this embodiment, a training set and a validation set are constructed based on the triples, and the performance of the model is evaluated by the loss changes of the training set and the validation set. Early stopping is used to prevent overfitting.

[0093] Step S103: Based on the constructed state transition prediction model, a refrigeration system optimization model based on a PPO (Proximal Policy Optimization) algorithm is designed and trained.

[0094] In this embodiment, the PPO algorithm includes an Actor network (policy network) and a Critic network (value network). The Critic Net (value network) and Actor Net (policy network) in the PPO algorithm can be designed based on DNN (Deep Nueral Network), and the activation function in the network is Leaky ReLu.

[0095] Optionally, the state transition prediction model constructed based on the constructed state transition prediction model is designed and trained based on the proximal policy optimization (PPO) algorithm for the refrigeration system optimization model, specifically including:

[0096] Based on the constructed state transition prediction model, the architecture of the Actor network and the Critic network of the PPO algorithm is designed, wherein the input of the Actor network is the state vector at the current moment, and the output is the mean and variance of the set action of the refrigeration system; the input of the Critic network is the state vector at the current moment, and the output is the value function of the current state;

[0097] Design the training strategy of the PPO algorithm, use Advantage estimation to calculate the advantage function, use Clip to constrain the step size of the policy update, use the policy gradient method to update the Actor network, and use the minimization of the temporal difference error to update the Critic network;

[0098] The refrigeration system optimization model based on the PPO algorithm is trained according to the training strategy.

[0099] In this embodiment, the PPO algorithm is used to design and train the refrigeration system energy consumption optimization model, which specifically includes the following steps:

[0100] (1) Design the Actor-Critic architecture. The input of the Actor network is the current state vector [state_dim], and the output is the mean and variance [mean, std_dev] of the set action of the refrigeration system. The network structure is a three-layer fully connected network with a hidden layer size of 256-128 and an activation function of Leaky ReLU. The input of the Critic network is the current state vector [state_dim], and the output is the value function (Value) of the current state. The network structure is similar to that of the Actor network.

[0101] (2) Design a PPO algorithm training strategy and use Advantage estimation (GAE-Lambda) to calculate the advantage function. Use the Clip method to constrain the step size of the policy update to ensure the stability of the policy update. The loss function includes two parts: policy loss and value function loss. Use the policy gradient method to update the Actor network and minimize the temporal difference error (TD Error) to update the Critic network.

[0102] (3) During the training process, the batch size of each training sample is set to 64, and 10 updates are performed in each training cycle (Epoch); the exploration rate is set to gradually decrease from 0.2 to 0.01; the total number of training rounds is set to 10,000 rounds.

[0103] Step S104: deploying the strategy output by the refrigeration system optimization model trained based on the PPO algorithm to the actual refrigeration system for real-time control and optimization.

[0104] In this embodiment, the strategy output by the trained Actor-Critic network is deployed to the actual data center refrigeration system for real-time control and optimization.

[0105] Specifically, based on current status data (such as temperature, humidity, etc.), the Actor network outputs the optimal cooling set point in real time and directly controls the power output of the refrigeration equipment.

[0106] Optionally, after deploying the strategy output by the refrigeration system optimization model trained based on the PPO algorithm to an actual refrigeration system for real-time control and optimization, the method further comprises:

[0107] During the actual operation of the data center, continuously collect new environmental status data and feedback information;

[0108] The refrigeration system optimization model is updated using the new environmental status data and feedback information, wherein the updating process adopts an incremental learning strategy.

[0109] It should be noted that the state transition prediction model can be regarded as a simulation of the real environment. In the real environment, we directly take actions to affect the state of the environment, and the state of the environment will change. The state transition prediction model simulates this process, so that our optimization model can be boldly tested on the state transition model, and then we can test and further optimize this preliminary optimization model in the real system, which is safer and more efficient to train.

[0110] Specifically, during the actual operation of the data center, new environmental status data and feedback information are continuously collected, and the strategy is updated based on the new data. The update process uses an incremental learning strategy to prevent the model from overfitting.

[0111] Among them, the feedback information is mainly the real-time energy consumption value, because it involves the cumulative reward value (return) in the optimization model. If there is a significant decline, the optimization model needs to be updated; at the same time, user feedback can also guide the update. For example, if it is found that the temperature and humidity deviate greatly from the set value, we can guide the model update by increasing the coefficient of the temperature and humidity reward in the reward function of the training process.

[0112] It should be noted that in order to solve the problems of high energy consumption and inflexible control strategy existing in the prior art, the present invention provides a method for optimizing energy consumption of a refrigeration system. This method models the energy consumption optimization process of the data center refrigeration system as a Markov decision process (MDP), combines a deep neural network (DNN) and a PPO algorithm, and can realize intelligent control and optimization of the refrigeration system, reduce energy consumption, and improve the energy utilization efficiency of the data center.

[0113] In a specific embodiment, the refrigeration system energy consumption optimization method may include the following steps:

[0114] Step 1: Design a data center cooling system energy consumption optimization framework, model the optimization process as a Markov decision process, and define the state space and action space of the data center cooling system energy consumption optimization scenario.

[0115] Step 1.1: For the cooling system, a Markov model for the cooling system energy consumption optimization problem is constructed using the customized data center environment states, actions, state transitions, and rewards.

[0116] Step 1.2: The action is the cooling set value of the data center cooling system. The state mainly includes the temperature, humidity and action of the data center at the previous moment. The reward includes temperature reward, humidity reward and energy consumption reward.

[0117] Among them, the customized state transition refers to how the temperature and humidity of the data center will change when we change the cooling set point of the cooling system.

[0118] Step 2: Collect data from key measurement points in the data center in real time through sensors, and pre-process the collected data to ensure the accuracy and stability of the input data.

[0119] Among them, key measurement point data include the temperature of each area in the data center, the humidity of each area, outdoor humidity, outdoor temperature, and the cooling set value and energy consumption data of the cooling system.

[0120] Step 3: Build a state transition prediction model based on the data collected by the sensor.

[0121] Step 3.1: Use a data-driven approach and combine the multi-layer perceptron MLP in the deep neural network to construct an accurate state transition prediction model.

[0122] Step 3.2: Select the measurement points composed of the state and action at the same moment as the input parameters of the state transition prediction model, and the state at the next moment as the output parameter. The activation function in the MLP network is Leaky ReLu. Training is performed based on a large number of fragments of the historical operation trajectory of the data center to obtain better network parameters.

[0123] It should be noted that the customized state transition in step 1 is equivalent to problem modeling, which determines the parameters of state transition and is a basic definition; while the state transition prediction model in this step is a solution, emphasizing prediction and model, because we use a neural network model to predict state transition.

[0124] Step 4: Based on the constructed state transition prediction model, design and train a refrigeration system optimization model based on the PPO (Proximal Policy Optimization) algorithm.

[0125] Step 5: Strategy deployment and model update: Based on DNN (Deep Nueral Network), the Critic Net (value network) and Actor Net (strategy network) in the PPO algorithm are designed, and the activation function in the network is Leaky ReLu. The strategy output by the trained Actor-Critic network is deployed to the actual data center cooling system for real-time control and optimization. During the actual operation, the strategy is updated based on new data to further improve the energy efficiency and adaptability of the cooling system.

[0126] In another specific embodiment, a method for optimizing energy consumption of a data center cooling system based on a PPO algorithm is provided, which implements intelligent optimization control of a data center cooling system through multi-step modeling, training, deployment, and optimization to improve energy efficiency of the data center and reduce cooling energy consumption. Figure 2 The energy consumption optimization framework of the data center cooling system is presented. The corresponding main steps include:

[0127] 1. The energy consumption optimization process of the data center cooling system is modeled as a Markov decision process (MDP). In this process:

[0128] State Space: includes the temperature of each area in the data center, the humidity of each area, the outdoor humidity, the outdoor temperature, and the cooling set point at the previous moment. Specifically, the state space can be represented as a multidimensional vector, for example, [T1, T2, ..., H1, H2, ..., Tout, Hout, Cooling_Set_Point_previous], where T represents the temperature of different areas, H represents the humidity of different areas, and Tout and Hout represent the outdoor temperature and humidity respectively.

[0129] It should be noted that a single computer room can be used as the optimization unit. The area here can refer to a single computer room. Specifically, the temperature and humidity of the air-conditioning outlets and return air outlets in each computer room as well as the outdoor temperature and humidity are collected. If better conditions are available, the temperature and humidity at both ends and the center of the column can be collected.

[0130] Action Space: Defined as the set value of the cooling system at the current moment, usually a continuous value. This value represents the set temperature or power of the chiller or other cooling equipment.

[0131] Reward Function: It consists of the following three parts:

[0132] (1) Temperature reward: The smaller the temperature deviation within the target temperature range, the higher the reward. Set the reward function to: R_temp = -|T_target - T_actual|, where T_target is the target temperature.

[0133] Among them, the target temperature refers to the cooling set value above (that is, the temperature set value); T_actual (actual temperature) refers to the temperature value collected by the sensor (excluding the outdoor); the temperature reward here is calculated for each area.

[0134] (2) Humidity reward: The smaller the humidity deviation within the target humidity range, the higher the reward. Set the reward function to: R_hum = -|H_target - H_actual|.

[0135] It should be noted that if the air conditioner can set the humidity value, then H_target (target humidity) here is the humidity setting value of the air conditioner, and H_actual (actual humidity) is the actual humidity value collected by sensors in various indoor areas; if the air conditioner cannot set the humidity value, then the appropriate H_target can be determined according to needs. Reinforcement learning will try to make the humidity of each area close to this H_target during training.

[0136] (3) Energy consumption reward: The lower the energy consumption of the refrigeration system, the higher the reward. The reward function is set as: R_energy = -E_consumed, where E_consumed is the energy consumption, which can be understood as the energy consumption value of the refrigeration system under the current temperature setting value.

[0137] It should be noted that the above reward values ​​are negative values, which can be understood as penalty values. In the energy consumption reward part, as long as there is energy consumption, a penalty is required. The higher the energy consumption, the greater the penalty, so negative numbers are used to represent it. In order to make the calculation logic of each element in the total reward function consistent, we also use negative reward logic for temperature and humidity.

[0138] The total reward function can be expressed as: R_total = w1*R_temp+w2*R_hum+w3*R_energy, where w1, w2, w3 are weighted coefficients used to balance the contributions of different parts.

[0139] It should be noted that the total reward function is the goal that the agent (i.e., the control algorithm of the refrigeration system) needs to maximize in the decision-making process. By designing a reasonable reward function, the agent can be guided to learn a control strategy that not only meets the temperature and humidity requirements but also saves energy as much as possible.

[0140] 2. Collect environmental status data in real time by installing multiple types of sensors (such as temperature sensors, humidity sensors, etc.) at key locations in the data center. The distribution of these sensors should cover all areas that need to be monitored to ensure the comprehensiveness and accuracy of the data. The collected data includes: temperature and humidity data of each area, outdoor temperature and humidity data, historical refrigeration system cooling set values ​​and energy consumption data. The preprocessing steps include:

[0141] (1) Data cleaning: Remove outliers and missing values, such as using interpolation or the nearest neighbor method to fill in missing data.

[0142] (2) Data normalization. Normalize the data to the range of [0,1] to meet the input requirements of the neural network. The normalization formula is: X_normalized = (X-X_min) / (X_max-X_min).

[0143] (3) Time series processing: Use the sliding window method to convert time series data into the model input format.

[0144] 3. Using a data-driven approach, we use a multi-layer perceptron (MLP) to build a state transition prediction model. Figure 3The schematic diagram of the structure of the MLP network in the state transition prediction model is shown, including the input layer, hidden layer, and output layer. During the training process, the model uses the historical data of the data center to learn the law of state transition. The specific steps are as follows:

[0145] (1) Design the model architecture. The input is the current environment state and action combination, with a dimension of [state_dim+action_dim]. The hidden layer is two fully connected layers, each with 128 neurons, and the activation function is Leaky ReLU. The output is the environment state at the next moment, with a dimension of [state_dim].

[0146] (2) Sample multiple groups of (state, action, next_state) triplets from historical data; use mean square error (MSE) as the loss function: L = (1 / N)*Σ(y_true-y_pred)^2; use Adam optimizer and set the learning rate to 0.001.

[0147] Among them, state represents the current environmental state, including the temperature and humidity of each area, the outdoor humidity and temperature, as well as the cooling setting value and energy consumption value at the current moment. next_state represents the next environmental state, including the temperature and humidity of each area, the outdoor humidity and temperature, as well as the cooling setting value and energy consumption value at the next moment.

[0148] (3) Evaluate the performance of the model by the change in loss between the training set and the validation set. Use the Early Stopping method to prevent overfitting.

[0149] 4. Use the PPO algorithm to design and train the refrigeration system energy consumption optimization model, which includes the following steps:

[0150] (1) Design the Actor-Critic architecture. The input of the Actor network is the current state vector [state_dim], and the output is the mean and variance [mean, std_dev] of the set action of the refrigeration system. The network structure is a three-layer fully connected network with a hidden layer size of 256-128 and an activation function of Leaky ReLU. The input of the Critic network is the current state vector [state_dim], and the output is the value function (Value) of the current state. The network structure is similar to that of the Actor network.

[0151] (2) Design a PPO algorithm training strategy and use Advantage estimation (GAE-Lambda) to calculate the advantage function. Use the Clip method to constrain the step size of the policy update to ensure the stability of the policy update. The loss function includes two parts: policy loss and value function loss. Use the policy gradient method to update the Actor network and minimize the temporal difference error (TD Error) to update the Critic network.

[0152] (3) During the training process, the batch size of each training sample is set to 64, and 10 updates are performed in each training cycle (Epoch); the exploration rate is set to gradually decrease from 0.2 to 0.01; the total number of training rounds is set to 10,000 rounds.

[0153] It should be noted that the state transition prediction model can be regarded as a simulation of the real environment. In the real environment, we directly take actions to affect the state of the environment, and the state of the environment will change. The state transition prediction model simulates this process, so that our optimization model can be boldly tested on the state transition model, and then we can test and further optimize this preliminary optimization model in the real system, which is safer and more efficient to train.

[0154] 5. Deploy the trained Actor-Critic network to the actual cooling system of the data center and optimize the control in the following aspects:

[0155] (1) Real-time strategy application: Based on the current status data (such as temperature, humidity, etc.), the Actor network outputs the optimal cooling set point in real time and directly controls the power output of the refrigeration equipment.

[0156] (2) Online model update: During the actual operation of the data center, new environmental status data and feedback information are continuously collected, and the strategy is updated based on the new data. The update process uses an incremental learning strategy to prevent model overfitting.

[0157] It should be noted that the PPO algorithm training refers to the experience gained based on the exploration of our reinforcement learning agent on the state transition prediction model. However, the optimization model obtained by such training is based on virtual data after all and is not completely reliable, so it needs to be updated online in the real system.

[0158] Among them, the feedback information is mainly the real-time energy consumption value, because it involves the cumulative reward value (return) in the optimization model. If there is a significant decline, the optimization model needs to be updated; at the same time, user feedback can also guide the update. For example, if it is found that the temperature and humidity deviate greatly from the set value, we can guide the model update by increasing the coefficient of the temperature and humidity reward in the reward function of the training process.

[0159] It should be noted that the refrigeration system energy consumption optimization method provided in the embodiment of the present invention can respond to environmental changes in real time, and has the characteristics of self-learning ability and efficient intelligent control, aiming to further improve the energy utilization efficiency of the data center and reduce operating costs.

[0160] In yet another specific embodiment, the refrigeration system energy consumption optimization method includes:

[0161] S1. Define the states and actions of the data center cooling system energy consumption optimization scenario, and use the Markov decision process (MDP) to describe the cooling system energy consumption optimization process;

[0162] Among them, the state space of the energy consumption optimization scenario of the data center cooling system includes the temperature of each area, the humidity of each area, the outdoor humidity, the outdoor temperature, and the cooling set value at the previous moment, and the action space includes the cooling set value at the current moment.

[0163] Among them, the reward function of the Markov decision process consists of three parts: humidity reward, temperature reward, and energy consumption reward.

[0164] S2, real-time data collection and preprocessing of key data in the data center through sensors;

[0165] S3. Use a neural network to predict the state transition in the Markov decision process and build a state transition prediction model; the input parameters of the state transition prediction model are the environmental state and action at the current moment, and the output is the environmental state at the next moment.

[0166] Specifically, a data-driven approach is used to construct the state transfer prediction model based on a multi-layer perceptron (MLP) in a deep neural network.

[0167] S4. Based on the state transition prediction model, design and train the refrigeration system optimization model based on the PPO algorithm;

[0168] S5. Deploy the strategy output by the trained Actor-Critic network to the actual data center cooling system, and further update the optimization model during operation.

[0169] The energy consumption optimization method of the refrigeration system provided by the embodiment of the present invention first models the energy consumption optimization process of the refrigeration system as a Markov decision process, defines the state space and action space of the energy consumption optimization scenario of the refrigeration system; then, according to the measurement point data collected by the sensor, a neural network is used to predict the state transition in the Markov decision process, and a state transition prediction model is constructed, wherein the input parameters of the state transition prediction model are the environmental state and action at the current moment, and the output parameter is the environmental state at the next moment; then, based on the constructed state transition prediction model, a refrigeration system optimization model based on the proximal policy optimization PPO algorithm is designed and trained; finally, the strategy output by the refrigeration system optimization model trained based on the PPO algorithm is deployed to the actual refrigeration system for real-time control and optimization. The present invention, by modeling the energy consumption optimization process of the refrigeration system as a Markov decision process (MDP), and combining the neural network and the PPO algorithm, can not only achieve the optimal refrigeration effect and energy efficiency under different operating conditions, but also significantly enhance the model's adaptability to complex environmental states by using the accurate prediction of the state transition by the neural network and the efficient policy optimization capability of the PPO algorithm. In addition, the present invention makes full use of the large amount of real-time data collected by the sensor, and through continuous learning and optimization, it can realize intelligent control and optimization of the refrigeration system, effectively reduce energy consumption, and significantly improve the energy efficiency of the data center. It solves the problem that the existing refrigeration system energy consumption optimization method lacks the ability to adapt to complex environmental conditions, is difficult to achieve the optimal refrigeration effect and energy efficiency under different operating conditions, and cannot make full use of the large amount of real-time data collected by the sensor, resulting in difficulty in achieving the optimal energy consumption control effect.

[0170] Embodiment 2:

[0171] like Figure 4 As shown, this embodiment provides a refrigeration system energy consumption optimization device, which is used to execute the above refrigeration system energy consumption optimization method, including:

[0172] The optimization process modeling module 11 is used to model the energy consumption optimization process of the refrigeration system as a Markov decision process, and define the state space and action space of the refrigeration system energy consumption optimization scenario;

[0173] A state transition prediction module 12 is connected to the optimization process modeling module 11, and is used to predict the state transition in the Markov decision process using a neural network based on the measurement point data collected by the sensor, and to construct a state transition prediction model, wherein the input parameters of the state transition prediction model are the environmental state and action at the current moment, and the output parameter is the environmental state at the next moment;

[0174] An optimization model building module 13, connected to the state transfer prediction module 12, is used to design and train a refrigeration system optimization model based on the proximal strategy optimization PPO algorithm based on the constructed state transfer prediction model;

[0175] The real-time control optimization module 14 is connected to the optimization model building module 13 and is used to deploy the strategy output by the refrigeration system optimization model trained based on the PPO algorithm into the actual refrigeration system for real-time control and optimization.

[0176] Optionally, the state space includes the temperature of each area in the data center, the humidity of each area, the outdoor humidity, the outdoor temperature, and the cooling set value of the cooling system at the last moment;

[0177] The action space is defined as the refrigeration setting value of the refrigeration system at the current moment.

[0178] Optionally, the reward function of the Markov decision process consists of three parts: humidity reward, temperature reward, and energy consumption reward;

[0179] The reward function of the temperature reward is: R_temp=-|T_target-T_actual|, where T_target is the target temperature and T_actual is the actual temperature;

[0180] The reward function of the humidity reward is: R_hum=-|H_target-H_actual|, where H_target is the target humidity and H_actual is the actual humidity;

[0181] The reward function of the energy consumption reward is: R_energy=-E_consumed, where E_consumed is the energy consumption value of the refrigeration system;

[0182] The total reward function is: R_total = w1*R_temp+w2*R_hum+w3*R_energy, where w1, w2, and w3 are weighted coefficients used to balance the contributions of different parts.

[0183] Optionally, the state transition prediction module 12 includes:

[0184] A real-time acquisition unit, used to collect the data of the data center in real time through sensors, wherein the data of the data center includes the temperature of each area in the data center, the humidity of each area, the outdoor humidity, the outdoor temperature, and the refrigeration setting value and energy consumption data of the refrigeration system;

[0185] A preprocessing unit, used for preprocessing the collected measuring point data;

[0186] The state transfer prediction model construction unit is used to utilize the preprocessed measurement point data and use a data-driven method to construct the state transfer prediction model based on the multi-layer perceptron MLP in the deep neural network.

[0187] Optionally, the state transition prediction model building unit specifically includes:

[0188] A model architecture design unit, used to design a model architecture of the state transition prediction model, wherein the architecture includes an input layer, a hidden layer, and an output layer;

[0189] A triplet acquisition unit is used to obtain multiple sets of triplet including the current environment state, the current action, and the next environment state from the pre-processed measurement point data, wherein the current environment state and the next environment state both include the temperature of each area in the data center, the humidity of each area, the outdoor humidity, the outdoor temperature, and the refrigeration setting value and energy consumption data of the refrigeration system at the corresponding time; the action at the current time includes the refrigeration setting value of the refrigeration system;

[0190] The first construction unit is used to learn the law of state transfer according to the triples to obtain the constructed state transfer prediction model.

[0191] Optionally, the optimization model building module 13 includes:

[0192] A network design unit, used for designing the architecture of the Actor network and the Critic network of the PPO algorithm based on the constructed state transition prediction model, wherein the input of the Actor network is the state vector at the current moment, and the output is the mean and variance of the set action of the refrigeration system; the input of the Critic network is the state vector at the current moment, and the output is the value function of the current state;

[0193] A training strategy design unit is used to design a training strategy for the PPO algorithm, calculate the advantage function using the Advantage estimation, constrain the step size of the strategy update using the Clip method, update the Actor network using the policy gradient method, and update the Critic network using the minimized temporal difference error;

[0194] The model training unit is used to train the refrigeration system optimization model based on the PPO algorithm according to the training strategy.

[0195] Optionally, the device further comprises:

[0196] A data collection module, used to continuously collect new environmental status data and feedback information during the actual operation of the data center;

[0197] The optimization model updating module is used to update the refrigeration system optimization model through the new environmental state data and feedback information, wherein the updating process adopts an incremental learning strategy.

[0198] Embodiment 3:

[0199] refer to Figure 5 This embodiment provides a refrigeration system energy consumption optimization device, including a memory 21 and a processor 22, the memory 21 stores a computer program, and the processor 22 is configured to run the computer program to execute the refrigeration system energy consumption optimization method in Example 1.

[0200] The memory 21 is connected to the processor 22. The memory 21 may be a flash memory, a read-only memory or other memory. The processor 22 may be a central processing unit or a single-chip microcomputer.

[0201] Embodiment 4:

[0202] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for optimizing energy consumption of a refrigeration system in the above-mentioned embodiment 1 is implemented.

[0203] The computer-readable storage medium includes volatile or non-volatile, removable or non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, computer program modules or other data). Computer-readable storage media include, but are not limited to, RAM (Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable read only memory), flash memory or other memory technology, CD-ROM (Compact Disc Read-Only Memory), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer.

[0204] In summary, the energy consumption optimization method, device and readable storage medium of the refrigeration system provided by the embodiment of the present invention first model the energy consumption optimization process of the refrigeration system as a Markov decision process, and define the state space and action space of the energy consumption optimization scenario of the refrigeration system; then, according to the measurement point data collected by the sensor, a neural network is used to predict the state transition in the Markov decision process, and a state transition prediction model is constructed, wherein the input parameters of the state transition prediction model are the environmental state and action at the current moment, and the output parameter is the environmental state at the next moment; then, based on the constructed state transition prediction model, a refrigeration system optimization model based on the proximal policy optimization PPO algorithm is designed and trained; finally, the strategy output by the refrigeration system optimization model trained based on the PPO algorithm is deployed to the actual refrigeration system for real-time control and optimization. The present invention, by modeling the energy consumption optimization process of the refrigeration system as a Markov decision process (MDP), and combining the neural network and the PPO algorithm, can not only achieve the optimal refrigeration effect and energy efficiency under different operating conditions, but also significantly enhance the model's adaptability to complex environmental states by using the accurate prediction of the state transition by the neural network and the efficient policy optimization capability of the PPO algorithm. In addition, the present invention makes full use of the large amount of real-time data collected by the sensor, and through continuous learning and optimization, it can realize intelligent control and optimization of the refrigeration system, effectively reduce energy consumption, and significantly improve the energy efficiency of the data center. It solves the problem that the existing refrigeration system energy consumption optimization method lacks the ability to adapt to complex environmental conditions, is difficult to achieve the optimal refrigeration effect and energy efficiency under different operating conditions, and cannot make full use of the large amount of real-time data collected by the sensor, resulting in difficulty in achieving the optimal energy consumption control effect.

[0205] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present invention, but the present invention is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A method for optimizing energy consumption of a refrigeration system, characterized in that: The method comprises: The energy consumption optimization process of the refrigeration system is modeled as a Markov decision process, and the state space and action space of the refrigeration system energy consumption optimization scenario are defined; According to the measurement point data collected by the sensor, a neural network is used to predict the state transition in the Markov decision process, and a state transition prediction model is constructed, wherein the input parameters of the state transition prediction model are the environmental state and action at the current moment, and the output parameter is the environmental state at the next moment; Based on the constructed state transition prediction model, a refrigeration system optimization model based on the proximal strategy optimization PPO algorithm is designed and trained; The strategy output by the refrigeration system optimization model trained based on the PPO algorithm is deployed to the actual refrigeration system for real-time control and optimization.

2. The method according to claim 1, characterized in that: The state space includes the temperature of each area in the data center, the humidity of each area, the outdoor humidity, the outdoor temperature, and the cooling set value of the cooling system at the last moment; The action space is defined as the refrigeration setting value of the refrigeration system at the current moment.

3. The method according to claim 1, characterized in that The reward function of the Markov decision process consists of three parts: humidity reward, temperature reward, and energy consumption reward; The reward function of the temperature reward is: R_temp=-|T_target-T_actual|, where T_target is the target temperature and T_actual is the actual temperature; The reward function of the humidity reward is: R_hum=-|H_target-H_actual|, where H_target is the target humidity and H_actual is the actual humidity; The reward function of the energy consumption reward is: R_energy=-E_consumed, where E_consumed is the energy consumption value of the refrigeration system; The total reward function is: R_total = w1*R_temp+w2*R_hum+w3*R_energy, where w1, w2, and w3 are weighted coefficients used to balance the contributions of different parts.

4. The method according to claim 2, characterized in that: The method of using a neural network to predict the state transition in the Markov decision process based on the measurement point data collected by the sensor and constructing a state transition prediction model specifically includes: The sensors collect the data of the data center in real time, including the temperature of each area in the data center, the humidity of each area, the outdoor humidity, the outdoor temperature, and the refrigeration setting value and energy consumption data of the refrigeration system; Preprocessing the collected measuring point data; The state transfer prediction model is constructed based on the multi-layer perceptron MLP in the deep neural network by using the preprocessed measurement point data and a data-driven method.

5. The method according to claim 4, characterized in that The method of using the preprocessed measurement point data and using a data-driven method to construct the state transition prediction model based on a multi-layer perceptron MLP in a deep neural network specifically includes: Designing a model architecture of the state transition prediction model, wherein the architecture includes an input layer, a hidden layer, and an output layer; A plurality of triplets including the current environment state, the current action, and the next environment state are obtained from the pre-processed measurement point data, wherein the current environment state and the next environment state both include the temperature of each area in the data center, the humidity of each area, the outdoor humidity, the outdoor temperature, and the refrigeration setting value and energy consumption data of the refrigeration system at the corresponding time; the action at the current time includes the refrigeration setting value of the refrigeration system; The law of state transfer is learned according to the triples to obtain the constructed state transfer prediction model.

6. The method according to claim 2, characterized in that The state transition prediction model constructed based on the proposed method designs and trains a refrigeration system optimization model based on the proximal strategy optimization (PPO) algorithm, specifically including: Based on the constructed state transition prediction model, the architecture of the Actor network and the Critic network of the PPO algorithm is designed, wherein the input of the Actor network is the state vector at the current moment, and the output is the mean and variance of the set action of the refrigeration system; the input of the Critic network is the state vector at the current moment, and the output is the value function of the current state; Design the training strategy of the PPO algorithm, use Advantage estimation to calculate the advantage function, use Clip to constrain the step size of the policy update, use the policy gradient method to update the Actor network, and use the minimization of the temporal difference error to update the Critic network; The refrigeration system optimization model based on the PPO algorithm is trained according to the training strategy.

7. The method according to claim 2, characterized in that: After deploying the strategy output by the refrigeration system optimization model trained based on the PPO algorithm to the actual refrigeration system for real-time control and optimization, the method further includes: During the actual operation of the data center, continuously collect new environmental status data and feedback information; The refrigeration system optimization model is updated using the new environmental status data and feedback information, wherein the updating process adopts an incremental learning strategy.

8. A refrigeration system energy consumption optimization device, characterized in that: The device comprises: The optimization process modeling module is used to model the energy consumption optimization process of the refrigeration system as a Markov decision process and define the state space and action space of the refrigeration system energy consumption optimization scenario; A state transfer prediction module is connected to the optimization process modeling module and is used to predict the state transfer in the Markov decision process using a neural network based on the measurement point data collected by the sensor, and to construct a state transfer prediction model, wherein the input parameters of the state transfer prediction model are the environmental state and action at the current moment, and the output parameter is the environmental state at the next moment; An optimization model construction module, connected to the state transfer prediction module, is used to design and train a refrigeration system optimization model based on the proximal strategy optimization PPO algorithm based on the constructed state transfer prediction model; The real-time control optimization module is connected to the optimization model building module and is used to deploy the strategy output by the refrigeration system optimization model trained based on the PPO algorithm into the actual refrigeration system for real-time control and optimization.

9. A refrigeration system energy consumption optimization device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the method for optimizing energy consumption of a refrigeration system according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for optimizing energy consumption of a refrigeration system according to any one of claims 1 to 7 is implemented.

Images