Intelligent control system and control method of outlet water temperature of refrigeration unit based on reinforcement learning

Abstract

The present invention discloses a refrigeration unit outlet water temperature intelligent control system based on reinforcement learning and a control method thereof. The intelligent control system includes a refrigeration unit and an intelligent gateway; it also includes an air-conditioning system edge side controller, which includes a refrigeration unit energy efficiency model, an intelligent optimization module and a strategy execution module connected in sequence; wherein the intelligent gateway is used to transmit the actual operation data of the refrigeration unit to the air-conditioning system edge side controller; the air-conditioning system edge side controller is used to predict and optimize the actual operation data, and finally generate a chilled water outlet temperature optimization strategy and send it to the refrigeration unit. It solves the problem that the traditional chilled water outlet temperature control method is difficult to adjust the cooling capacity according to actual needs, which easily causes energy waste and increased operating costs.

Description

[Technical field]

[0001] The present invention belongs to the technical field of intelligent control of refrigeration units, and specifically relates to a refrigeration unit outlet water temperature intelligent control system based on reinforcement learning and a control method thereof. [Background technology]

[0002] Refrigeration units are core equipment in air-conditioning systems and industrial cooling systems. Their main function is to cool chilled water through a refrigeration cycle for space cooling and equipment cooling. During the operation of the refrigeration unit, the control of the chilled water outlet temperature is crucial to the system's energy efficiency, operating costs, and environmental comfort. Traditional chilled water outlet temperature control methods have certain limitations. With the development of intelligent control technology, intelligent control strategies based on the Internet of Things (IoT), big data analysis, and artificial intelligence (AI) have gradually become an important means to optimize the operation of refrigeration units.

[0003] In the existing solutions, many actual scenarios set the outlet water temperature to a fixed value or manually reset it by operation and maintenance personnel according to demand. This method is simple and easy to implement, but when the cooling load fluctuates greatly, it is difficult to adjust the cooling capacity according to actual demand, which easily leads to energy waste and increased operating costs. Others use single variable feedback control to dynamically adjust the chilled water outlet temperature set value based on the return water temperature or indoor temperature feedback value. Although the outlet water temperature can be adjusted according to the actual load to a certain extent, due to the limited system response speed, control lag, and failure to consider the actual performance characteristics of the refrigeration unit, there is still the problem of insufficient energy efficiency optimization. [Summary of the invention]

[0004] The purpose of the present invention is to provide a refrigeration unit outlet water temperature intelligent control system and control method based on reinforcement learning, so as to solve the problem that the traditional chilled water outlet temperature control method is difficult to automatically optimize the energy efficiency and adaptively optimize the control according to the actual operating performance curve of the refrigeration unit, which easily leads to low actual operating energy efficiency and increased operating costs.

[0005] The present invention adopts the following technical scheme: a method for intelligently controlling the outlet water temperature of a refrigeration unit based on reinforcement learning, based on an intelligent control system for the outlet water temperature of a refrigeration unit based on reinforcement learning, the intelligent control system comprising a refrigeration unit and an intelligent gateway; and also comprising an edge-side controller of an air-conditioning system, which comprises an energy efficiency model of a refrigeration unit, an intelligent optimization module and a strategy execution module which are sequentially data-connected;

[0006] The intelligent gateway is used to transmit the actual operation data of the refrigeration unit to the edge controller of the air-conditioning system; the edge controller of the air-conditioning system is used to predict and optimize the actual operation data, and finally generate a chilled water outlet temperature optimization strategy and send it to the refrigeration unit;

[0007] The control method includes the following contents:

[0008] Step 1: Collecting actual operation data of the refrigeration unit for a certain period of time through the intelligent gateway, wherein the actual operation data includes input characteristic variables and output quantities;

[0009] The input characteristic variables include the chilled water outlet temperature, cooling capacity Q ch , outdoor dry bulb temperature and cooling water supply temperature; the output includes the cooling coefficient COP of the refrigeration unit and the power consumption P of the refrigeration unit ch ;

[0010] Step 2: Use the actual operation data in step 1 to build an energy efficiency model for the refrigeration unit;

[0011] The specific construction process is as follows: based on the refrigeration unit energy efficiency mechanism formula, the actual operation data is standardized to obtain input variables and output variables, a feedforward neural network algorithm is used to build a refrigeration unit performance curve prediction model, and the historical data of input variables and output variables are trained. The input variable data used for testing is imported into the trained refrigeration unit performance curve prediction model. When the deviation between the output variable value of the test data and the actual value is ≤5%, the refrigeration unit energy efficiency model is obtained.

[0012] The beneficial effects of the present invention are:

[0013] 1. Realize the processing, analysis and effective use of the actual operation data of the refrigeration unit; by building an energy efficiency model for the refrigeration unit, use edge computing to standardize and analyze the data, obtain a prediction model that can characterize the performance curve of the refrigeration unit, and then call the prediction model of the refrigeration unit performance curve to achieve real-time prediction of the actual operation energy efficiency of the refrigeration unit. To achieve intelligent control of the chilled water outlet temperature set value, thereby achieving efficient operation of the refrigeration unit, reducing operation and maintenance costs, and extending equipment life.

[0014] 2. Use the DQN reinforcement learning algorithm to realize the generation of intelligent optimization strategy for the chilled water outlet temperature setting value of the refrigeration unit; through steps 3 and 4, build an intelligent agent suitable for energy efficiency optimization of the refrigeration unit, and obtain a deployable reinforcement learning model for intelligent optimization by continuously training historical data. The model has a unified data structure. In practical applications, it is only necessary to deploy the trained reinforcement learning model according to a certain type of refrigeration unit (such as industrial frequency screw, variable frequency screw, centrifuge). The intelligent agent will continue to learn according to the actual operating parameters to ensure that the optimization strategy can self-learn and adapt to the actual operating performance differences of the refrigeration unit in different scenarios. This makes the entire control method transferable and universal.

[0015] 3. Based on edge-side control technology, intelligent control of the chilled water outlet temperature set value of the refrigeration unit is realized; by building a deployable reinforcement learning model, the intelligent agent is trained using the actual operation data of the refrigeration unit, and the intelligent optimization module is called through the edge-side controller in the actual scenario to autonomously generate the optimization strategy for the chilled water outlet temperature set value, so as to realize self-learning and adaptive optimization of the chilled water outlet temperature, the controlled variable of the refrigeration unit, thereby improving the operating efficiency of the refrigeration unit and reducing the operating energy consumption of the refrigeration unit.

Brief Description of the Drawings

[0016] Figure 1 It is a structural diagram of an intelligent control method for outlet water temperature of a refrigeration unit based on reinforcement learning;

[0017] Figure 2 Graph of measured operation data of the refrigeration unit in Example 1 of the present invention;

[0018] Figure 3 The energy efficiency prediction effect of the refrigeration unit in Example 1 of the present invention;

[0019] Figure 4 This is a diagram showing the optimization effect of the chilled water outlet temperature based on the DQN algorithm in Example 1 of the present invention.

[0020] Among them, 1. Refrigeration unit, 2. Smart gateway, 3. Air-conditioning system edge controller, 4. Smart electricity meter, 5. Cooling meter, 31. Refrigeration unit energy efficiency model, 32. Smart optimization module, 33. Strategy execution module. [Specific implementation method]

[0021] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] The present invention provides a refrigeration unit outlet water temperature intelligent control system based on reinforcement learning and a control method thereof, based on a refrigeration unit outlet water temperature intelligent control system based on reinforcement learning, the control system includes a refrigeration unit 1 and an intelligent gateway 2; and also includes an air conditioning system edge side controller 3, which includes a refrigeration unit energy efficiency model 31, an intelligent optimization module 32 and a strategy execution module 33 which are sequentially data-connected;

[0023] The intelligent gateway 2 is used to transmit the actual operation data of the refrigeration unit 1 to the edge side controller 3 of the air conditioning system; the edge side controller 3 of the air conditioning system is used to predict and optimize the actual operation data, and finally generate a chilled water outlet temperature optimization strategy and send it to the refrigeration unit 1;

[0024] The control method includes the following contents:

[0025] Step 1: Collecting actual operation data of the refrigeration unit 1 for a certain period of time through the intelligent gateway 2, wherein the actual operation data includes input characteristic variables and output quantities;

[0026] The input characteristic variables include the chilled water outlet temperature, cooling capacity Q ch , outdoor dry bulb temperature and cooling water supply temperature; the output includes the cooling coefficient COP of the refrigeration unit and the power consumption P of the refrigeration unit ch ;

[0027] Step 2: constructing a refrigeration unit energy efficiency model 31 using the actual operation data in step 1;

[0028] The specific construction process is as follows: based on the energy efficiency mechanism formula of the refrigeration unit, the actual operation data is standardized to obtain input variables and output variables, a feedforward neural network algorithm is used to build a refrigeration unit performance curve prediction model, and the historical data of the input variables and output variables are trained, and the input variable data used for testing is imported into the trained refrigeration unit performance curve prediction model. When the deviation between the output variable value of the test data and the actual value is ≤5%, the refrigeration unit energy efficiency model 31 is obtained;

[0029] The input variables are the partial load rate PLR ​​of the refrigeration unit, the chilled water outlet temperature and the cooling water supply temperature, and the output variable is the ratio of the actual power of the refrigeration unit to the nameplate power η p The refrigeration efficiency ratio of the actual working condition of the refrigeration unit to the rated working condition is η cop The refrigeration unit performance curve prediction model is used to predict η p and η COP ; On the other hand, it serves as a reinforcement learning algorithm strategy network.

[0030] Step 3: deploying the refrigeration unit energy efficiency model 31 in the air conditioning system edge side controller 3;

[0031] Step 4: Optimization problem modeling: Combined with the outdoor temperature, use the input variables and output variables in step 2 to build the data structure required by the intelligent agent in the intelligent optimization module;

[0032] Step 5: Based on the intelligent agent constructed in step 4, a deployable reinforcement learning model file is generated using the DQN reinforcement learning algorithm;

[0033] Step 6: embed the deployable reinforcement learning model file generated in step 5 into the intelligent optimization module 32 to perform real-time interaction and dynamic optimization;

[0034] Combined with the outdoor temperature, the input variables and output variables are connected to the deployable reinforcement learning model through the intelligent optimization module 32, and the chilled water outlet temperature strategy is continuously generated based on the actual environment;

[0035] The strategy execution module 33 sends the chilled water outlet temperature strategy to the refrigeration unit 1 through the chilled water outlet temperature control point of the refrigeration unit 1, thereby realizing intelligent optimization control of the chilled water outlet temperature.

[0036] In some embodiments, the refrigeration unit energy efficiency model in step 2 is:

[0037]

[0038] Among them, P ch is the power consumption of the refrigeration unit, is the nameplate power of the refrigeration unit, Q ch is the cooling capacity, Q ch ' is the nameplate rated cooling capacity of the refrigeration unit, PLR is the partial load rate of the refrigeration unit, η p is the ratio of the actual power of the refrigeration unit to the nameplate power, PLR is the ratio of the actual cooling capacity of the refrigeration unit to the nameplate cooling capacity, η cop It is the ratio of the cooling efficiency of the refrigeration unit under actual working conditions to that under rated working conditions.

[0039] In some embodiments, the main parameters of the agent in step 4 are set as follows:

[0040] 1) The state S at the current time t t = db ,PLR,T chs , T cws , P ch , η cop},

[0041] Among them, T db is the outdoor dry bulb temperature, PLR is the partial load rate of the refrigeration unit, T chs is the chilled water outlet temperature, T cws The supply temperature of cooling water;

[0042] 2) Action A at the current time t t To adjust T chs Chilled water outlet temperature:

[0043] {A t ∈Z|0∈A t ∈2}, Z is an integer;

[0044]

[0045] Among them, (T chs min , T chs max ) are the upper and lower limits of the chilled water outlet temperature;

[0046] 4) Reward function: r = η cop +γ·|T chs -T set |;

[0047] Among them, r is the reward function, γ is the discount factor, T set It is the set value of chilled water outlet temperature.

[0048] In some embodiments, the control system further includes a smart meter 4 and a cooling meter 5, and the smart meter 4 and the cooling meter 5 are used to collect actual operation data of the refrigeration unit. For example, the power consumption of the refrigeration unit is collected through the smart meter.

[0049] Example

[0050] 1. Taking a certain refrigeration unit as an example, the actual operating parameters of the refrigeration unit are collected through the intelligent gateway, and then the data is processed. After the actual operating data of the refrigeration unit is processed, the actual operating data of the evaporation side is obtained. Figure 2 , where feature 1 is PLR, feature 2 is the chilled water outlet temperature on the evaporation side, and output 2 is the energy efficiency ratio η of the refrigeration unit cop .from Figure 2 It can be seen that the processed measured operation data of the refrigeration unit has an input and output structure consistent with the prediction algorithm, and its characteristic variables and output can better reflect the operating characteristics of the refrigeration unit.

[0051] 2. Use the input variables and output variables to train the feedforward neural network MPL model, and deploy the trained MPL algorithm refrigeration unit performance prediction model on the edge controller, select the test set data to predict η cop The effect is shown in Figure 3 , the root mean square error RMSE of the test set is 0.032187. Figure 3 It can be seen that the trained refrigeration unit performance prediction model can accurately predict the energy efficiency of the refrigeration unit, and its accuracy can meet the requirements of engineering applications.

[0052] 3. Build a DQN-based reinforcement learning model and conduct offline training tests. Deploy the tested reinforcement model in the edge controller intelligent optimization module 32 and connect it to the on-site environment for online testing. The test results are as follows: Figure 4 As shown, the chilled water outlet temperature setting value strategy is autonomously generated through a reinforcement learning model. Compared with the existing conventional control strategy, the control method of the present invention is more intelligent and effectively improves the operating energy efficiency of the refrigeration unit and reduces the operating energy consumption of the refrigeration unit.

[0053] Optimization effect of chilled water outlet temperature based on DQN Figure 4 In the DQN strategy, the outlet temperature of the chilled water changes hour by hour after optimization. Figure 4It can be seen that compared with the fixed value strategy, the DQN strategy can continuously optimize the chilled water outlet temperature setting value according to the changes in input parameters such as partial load rate and outdoor temperature change, so that the refrigeration unit η cop It always operates at the best energy efficiency condition when input parameters change, thus achieving energy-saving and efficient operation.

[0054] The relevant parameters of the entire reinforcement learning model testing process are set as follows:

[0055] 1) Parameters and environment settings:

[0056]

[0057] 2) Policy network definition:

[0058] Define a three-layer MLP fully connected network, with the input, hidden, and output layers all set to 128.

[0059] 3) DQN Agent:

[0060] Define the reward function as r = η cop +0.1·|T chs -T set |,

[0061] Set the initial chilled water outlet temperature to 8°C.

[0062] Repeat the following steps until the termination condition is reached:

[0063] Observe the current state s;

[0064] According to the ∈-greedy strategy, action a is selected, that is, adjusting the set value of the chilled water outlet temperature;

[0065] Execute action a to obtain the new state s', instant energy consumption and reward r;

[0066] Store (s,a,r,s') into the experience replay buffer;

[0067] Randomly sample small batches of data from the playback buffer and calculate the expected return under the adjusted chilled water outlet temperature setpoint.

[0068] Use gradient descent method to update the main network parameters θ;

[0069] Every several steps, the main network parameters in the DQN algorithm are synchronized to the target network.

[0070] The termination condition is that the maximum number of iterations is reached or the energy efficiency of the refrigeration unit reaches the expectation.

Claims

1. A method for intelligently controlling the outlet water temperature of a refrigeration unit based on reinforcement learning, characterized in that: A refrigeration unit outlet water temperature intelligent control system based on reinforcement learning, the intelligent control system comprising a refrigeration unit (1) and an intelligent gateway (2); and an air conditioning system edge side controller (3), which comprises a refrigeration unit energy efficiency model (31), an intelligent optimization module (32) and a strategy execution module (33) which are sequentially data-connected; The intelligent gateway (2) is used to transmit the actual operation data of the refrigeration unit (1) to the air-conditioning system edge side controller (3); the air-conditioning system edge side controller (3) is used to predict and optimize the actual operation data, and finally generate a chilled water outlet temperature optimization strategy and send it to the refrigeration unit (1); The control method includes the following contents: Step 1: collecting actual operation data of the refrigeration unit (1) for a certain period of time through the intelligent gateway (2), wherein the actual operation data includes input characteristic variables and output quantities; The input characteristic variables include the chilled water outlet temperature, cooling capacity Q ch , outdoor dry bulb temperature and cooling water supply temperature; the output includes the cooling coefficient COP of the refrigeration unit (1) and the power consumption P of the refrigeration unit ch ; Step 2: constructing a refrigeration unit energy efficiency model (31) using the actual operation data in step 1; The specific construction process is as follows: based on the energy efficiency mechanism formula of the refrigeration unit, the actual operation data is standardized to obtain input variables and output variables, a feedforward neural network algorithm is used to construct a refrigeration unit performance curve prediction model, and the historical data of the input variables and output variables are trained, and the input variable data used for testing is imported into the trained refrigeration unit performance curve prediction model. When the deviation between the output variable value of the test data and the actual value is ≤5%, the refrigeration unit energy efficiency model (31) is obtained; The input variables are the partial load rate PLR ​​of the refrigeration unit, the chilled water outlet temperature and the cooling water supply temperature, and the output variable is the ratio of the actual power of the refrigeration unit to the nameplate power η p The refrigeration efficiency ratio of the actual working condition of the refrigeration unit to the rated working condition is η cop ; Step 3: deploying the refrigeration unit energy efficiency model (31) in the air conditioning system edge side controller (3); Step 4: Optimization problem modeling: Combined with the outdoor temperature, use the input variables and output variables in step 2 to build the data structure required by the intelligent agent in the intelligent optimization module; Step 5: Based on the intelligent agent constructed in step (4), a deployable reinforcement learning model is generated using the DQN reinforcement learning algorithm; Step 6: embedding the deployable reinforcement learning model file generated in step 5 into the intelligent optimization module (32) to perform real-time interaction and dynamic optimization; Combined with the outdoor temperature, the input variables and output variables are connected to the deployable reinforcement learning model through the intelligent optimization module (32), and the chilled water outlet temperature strategy is continuously generated based on the actual environment; The strategy execution module (33) sends the chilled water outlet temperature strategy to the refrigeration unit (1) through the chilled water outlet temperature control point of the refrigeration unit (1), thereby realizing intelligent optimization control of the chilled water outlet temperature.

2. The intelligent control method for outlet water temperature of a refrigeration unit based on reinforcement learning according to claim 1, characterized in that: The refrigeration unit energy efficiency model (31) in step 2 is: Among them, P ch is the power consumption of the refrigeration unit, ch′ P is the nameplate power of the refrigeration unit, Q ch is the cooling capacity, Q ch ′ is the nameplate rated cooling capacity of the refrigeration unit, PLR is the partial load rate of the refrigeration unit, η p is the ratio of the actual power of the refrigeration unit to the nameplate power, PLR is the ratio of the actual cooling capacity or predicted cooling capacity of the refrigeration unit to the nameplate cooling capacity, η cop It is the ratio of the cooling efficiency of the refrigeration unit under actual working conditions to that under rated working conditions.

3. The intelligent control method for outlet water temperature of a refrigeration unit based on reinforcement learning according to claim 1 or 2, characterized in that: The main parameters of the intelligent body in step 4 are set as follows: 1) The state S at the current time t t = db ,PLR,T chs , T cws , P ch , η cop }, Among them, T db is the outdoor dry bulb temperature, PLR is the partial load rate of the refrigeration unit, T chs is the chilled water outlet temperature, T cws The supply temperature of cooling water; 2) Action A at the current time t t To adjust T chs Chilled water outlet temperature: {A t ∈Z|0≤A t ≤2}, Z is an integer; Among them, (T chsmin , T chsmax ) are the upper and lower limits of the chilled water outlet temperature; 3) Reward function: r = η cop +γ·|T chs -T set |; Among them, r is the reward function, γ is the discount factor, T set It is the set value of chilled water outlet temperature.

4. The intelligent control method for outlet water temperature of a refrigeration unit based on reinforcement learning according to claim 1 or 2, characterized in that: The control system further comprises an intelligent electric meter (4) and a cooling meter (5), wherein the intelligent electric meter (4) and the cooling meter (5) are used to collect actual operation data of the refrigeration unit.

Images