Cold storage energy-saving control method based on deep reinforcement learning

Through deep reinforcement learning technology, the operation of refrigeration equipment in cold storage is optimized in real time, which solves the shortcomings of traditional constant temperature control methods in adapting to cold storage load and environmental changes, and achieves efficient energy saving and precise temperature control, reducing operating costs and carbon emissions.

CN118912801AInactive Publication Date: 2024-11-08CHANGFENG INTELLIGENT CONTROL (SHANDONG) TECHNOLOGY DEVELOPMENT CO LTD
View PDF 0 Cites 7 Cited by

Patent Information

Application Number
CN202411154262.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Traditional constant temperature control methods are difficult to adapt to the dynamically changing loads and environmental conditions in cold storage, resulting in poor energy waste and control effects.

Method used

Using the energy-saving control method of cold storage based on deep reinforcement learning, through independent learning and intelligent decision-making, the temperature changes, load conditions and equipment operation status are tracked in real time, and the operating status of refrigeration equipment is dynamically optimized. Combined with tools such as OpenStudioSketchUp and EnergyPlus to build a simulation and real-time control environment. , use deep reinforcement learning model for optimization control.

Benefits of technology

It significantly improves the energy efficiency and temperature control accuracy of the cold storage system, reduces operating costs, adapts to load changes, achieves efficient energy utilization, extends equipment life, and reduces carbon emissions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118912801A_ABST
    Figure CN118912801A_ABST
Patent Text Reader

Abstract

The invention discloses a cold storage energy-saving control method based on deep reinforcement learning, and belongs to the technical field of cold storage energy-saving control. Comprising the following steps of construction of a refrigeration house simulation environment, construction of a refrigeration house real-time control environment, and training deployment and optimization of a deep reinforcement learning model. Through an intelligent and dynamic control strategy, the system adapts to load changes, achieves a remarkable energy-saving effect and high-precision temperature control, and achieves the efficient utilization of energy. And the service life of the equipment is prolonged, the maintenance cost is reduced, and good environmental friendliness and remarkable economic benefits are achieved. The method can be widely applied to various refrigeration storage systems due to the self-adaptive capability and the continuous optimization characteristic, long-term economic and social benefits are brought to enterprises, and popularization and application of the intelligent control technology are promoted.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of cold storage energy-saving control, and in particular to a cold storage energy-saving control method based on deep reinforcement learning. Background Art

[0002] In the fields of cold chain logistics and food preservation, energy-saving control of cold storage is a crucial topic. Cold storage needs to maintain a low temperature environment to ensure the quality of stored items, which usually consumes a lot of electricity. Therefore, how to minimize energy consumption while ensuring the stability of the cold storage temperature is an important research direction.

[0003] Traditional thermostatic control methods, although simple and widely used, have many deficiencies in terms of energy efficiency, accuracy and adaptability. They are usually difficult to adapt to dynamically changing loads and environmental conditions, resulting in energy waste and unsatisfactory control effects.

[0004] Constant temperature control is the simplest control method, which controls the start and stop of the refrigeration equipment through the temperature set point. When the temperature exceeds the set upper limit, the refrigeration equipment starts; when the temperature drops to the set lower limit, the refrigeration equipment stops. Constant temperature control cannot be adjusted intelligently according to actual load changes, and cannot cope with changes in the number of goods in the cold storage and the ambient temperature.

[0005] The present invention proposes a cold storage energy-saving control method based on deep reinforcement learning, which can track temperature changes, load conditions, equipment operation status and other data in real time. Through autonomous learning and intelligent decision-making, it can effectively overcome the shortcomings and deficiencies of traditional constant temperature control methods. Summary of the invention

[0006] The purpose of the present invention is to provide a cold storage energy-saving control method based on deep reinforcement learning. The control method based on deep reinforcement learning can significantly improve the energy efficiency and temperature control accuracy of the cold storage system through autonomous learning, intelligent decision-making and dynamic optimization, reduce operating costs, adapt to load changes, and achieve efficient use of energy.

[0007] To achieve the above object, the present invention provides a cold storage energy-saving control method based on deep reinforcement learning, comprising the following steps:

[0008] S1. Construction of cold storage simulation environment. Collect cold storage data through information and data acquisition. Use OpenStudioSketchUp plug-in or directly define the building geometry and the spatial structure of the cold storage in EnergyPlus to build a refrigeration system model. Verify and adjust the parameters of the model. Encapsulate the OpenAIGym environment interface.

[0009] S2. Build the real-time control environment for cold storage, connect to the cold storage PLC system, select the appropriate protocol Modbus or OPCUA, collect the real-time data of the cold storage, and provide control interfaces for equipment such as compressors and air coolers, and interact with the deep reinforcement learning model for integration and control;

[0010] S3. Deep reinforcement learning model training, deployment and optimization. Select appropriate deep reinforcement learning algorithms to pre-train the intelligent agent, deploy and connect the pre-trained intelligent agent to the cold storage real-time control environment system for real-time control and regular update and optimization, and evaluate and verify the model.

[0011] Preferably, in step S1, collecting cold storage data includes the following:

[0012] Building structure, floor plan and section of the cold storage, material and thickness of walls, roof and floor;

[0013] Equipment parameters, model, operating characteristics and power of the refrigeration equipment;

[0014] Environmental data, meteorological data of the location of the cold storage;

[0015] Historical data, including historical operating temperature data, power consumption data and equipment operating data of cold storage;

[0016] Electricity price data, specific time periods and prices of peak, valley and flat electricity prices.

[0017] Preferably, in step S1, constructing the simulation environment specifically includes the following steps:

[0018] Create a building geometry model, use the OpenStudioSketchUp plug-in or define the building geometry directly in EnergyPlus, and define the spatial structure of the cold storage, including exterior walls, interior walls, roof, floor, doors and windows;

[0019] Define materials and structures, set the material properties of walls, roofs, floors, etc. in EnergyPlus, including thermal conductivity, density, specific heat capacity, and define the thickness and order of each structural layer to ensure consistency with the actual cold storage structure;

[0020] Refrigeration system model building, adding refrigeration equipment in EnergyPlus, including compressors, condensers and evaporators, setting the equipment's operating parameters, cooling capacity, power consumption and efficiency, and defining the control strategies for refrigeration equipment start-stop control and load regulation;

[0021] Model verification and parameter adjustment: Based on the Python interface of the EMS module, the temperature and power consumption simulation data of the cold storage are output and compared with the collected historical operation data of the cold storage. According to the simulation results, the parameters in the building model, equipment model and control strategy are adjusted, and finally the OpenAIGym environment interface is encapsulated.

[0022] Preferably, in step S2, the construction of the cold storage real-time control environment specifically includes the following steps:

[0023] Collect electricity price data, connect to the PLC system, obtain the real-time temperature, power, and equipment operation status of the cold storage, connect to the meteorological data interface or install a small meteorological station to obtain real-time meteorological data, filter, normalize and process outliers to ensure data quality;

[0024] Build a real-time control environment, connect to the PLC system, realize the start and stop and power control of the compressor and air cooler, provide an action control interface for the deep reinforcement model, and use the GymAPI interface to encapsulate the environment.

[0025] Preferably, in step S3, the deep reinforcement learning model training deployment and optimization specifically includes the following steps:

[0026] S31, pre-training, select the appropriate deep reinforcement learning algorithm, initialize the intelligent agent, define the state space and action space of the cold storage control system, design the reward function, including energy consumption, temperature stability and equipment operation cost; run the DRL algorithm for training in the cold storage simulation environment, and through continuous trial and error and optimization, DRLAgent gradually learns the optimal control strategy;

[0027] S32, deployment, deploying the pre-trained agent and connecting it to the cold storage real-time control environment system. According to the real-time collected data, DRLAgent executes the pre-trained control strategy, adjusts the operating status of the refrigeration equipment, and records the data of the status, action, reward and next status to form experience playback storage;

[0028] S33. Regularly update and optimize, store the data collected during real-time operation in the experience replay pool for subsequent model updates, preprocess the collected data to remove noise and outliers, and ensure data quality; regularly extract data from the experience replay pool, use the online training mechanism to fine-tune the DRL model, train with small batches of data, and gradually update the parameters of the DRL model to adapt to the latest environmental changes; test the updated DRL model in a cold storage simulation environment, evaluate its control effect under different working conditions, and evaluate whether the performance of the new model is better than the old model by comparing indicators such as energy consumption and temperature fluctuations.

[0029] Preferably, in step S31, the specific design of the reward function is as follows:

[0030] Energy consumption reward

[0031] Objective: To encourage reduction of energy consumption in cold storage systems;

[0032] Design: Setting a benchmark - energy consumption value E baseline , calculate the current energy consumption E at each time step t , and take the difference between the two as part of the energy consumption reward;

[0033] Formula: R energy =-k energy ×(E t -E baseline )

[0034] Among them, k energy is the energy consumption weight coefficient;

[0035] Temperature Control Bonus

[0036] Objective: Keep the temperature inside the cold storage within the set range and reduce temperature fluctuations.

[0037] Design: Set target temperature T target And the allowable temperature deviation range ΔT. Calculate the current temperature T t Deviation from target temperature;

[0038] Formula: R temperature =-k temp ×max(0,|T t -T tar get |-ΔT)

[0039] Among them, k temp is the temperature weight coefficient;

[0040] Equipment Operation Rewards

[0041] Goal: Reduce frequent start and stop of equipment and extend equipment life;

[0042] Design: Set the penalty for each device start and stop, and calculate the number of device starts and stops N on / off ;

[0043] Formula: R operation =-k operation ×N on / off

[0044] Among them, k operation The weight factor for the device operation.

[0045] Electricity Price Incentive

[0046] Goal: Optimize equipment operating time and reduce operation during peak electricity price periods;

[0047] Design: According to the electricity price P in different periods t , impose penalties on operation during peak electricity price periods;

[0048] Formula: R price =-k price ×P t ×E t

[0049] Among them, k price is the electricity price weight coefficient;

[0050] Combination of reward functions

[0051] Combine the above reward functions to form the final comprehensive reward function. The comprehensive reward function should take into account energy consumption, temperature control, equipment operation and electricity price factors to ensure that the agent is optimized under multiple objectives.

[0052] Comprehensive reward function formula: R = R energy +R temperature +R operation +R price

[0053] By appropriately adjusting the weight coefficient k of each part price , k temp , k operation and k price , balance the impact between different objectives and ensure that the intelligent agent can achieve good results in both energy saving and temperature control.

[0054] Therefore, the present invention adopts a cold storage energy-saving control method based on deep reinforcement learning with the above structure, which has the following beneficial effects:

[0055] (1) Improve energy efficiency: Interact with the environment to learn and dynamically adjust, autonomously learn the optimal strategy, reduce unnecessary operations, improve energy efficiency, save energy, help the cold storage system reduce carbon emissions, and achieve environmental friendliness.

[0056] (2) Precise intelligent control: Real-time sensing of temperature changes and rapid adjustment of cooling strategies to ensure that the temperature is within the target range. The output power of the cooling equipment can be finely adjusted to make temperature control more precise and reduce temperature fluctuations.

[0057] (3) Cost savings: It can effectively adapt to the peak, valley and flat electricity price structure of the power grid and optimize the operation strategy of refrigeration equipment, thereby optimizing and saving electricity costs.

[0058] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic diagram of a cold storage simulation environment for a cold storage energy-saving control method based on deep reinforcement learning according to the present invention;

[0060] Figure 2 A schematic diagram of model verification and parameter adjustment in cold storage simulation environment construction of a cold storage energy-saving control method based on deep reinforcement learning of the present invention;

[0061] Figure 3 A schematic diagram of an OpenAI Gym environment interface encapsulated in a cold storage simulation environment construction of a cold storage energy-saving control method based on deep reinforcement learning of the present invention;

[0062] Figure 4 A schematic diagram of encapsulating the OpenAI Gym environment interface in a real-time control environment for building a cold storage energy-saving control method based on deep reinforcement learning in the present invention;

[0063] Figure 5 This is a schematic diagram of deep reinforcement learning model training deployment and optimization of a cold storage energy-saving control method based on deep reinforcement learning in the present invention. DETAILED DESCRIPTION

[0064] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0065] Unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0066] Example

[0067] like Figure 1-5 As shown, the present invention provides a cold storage energy-saving control method based on deep reinforcement learning, including the following methods:

[0068] S1. Construction of cold storage simulation environment

[0069] The cold storage simulation environment not only provides a necessary testing platform for model training, ensuring that the deep reinforcement learning model reaches the optimal state before being connected to the real-time control environment, but also provides a verification and evaluation mechanism for subsequent model update optimization.

[0070] Information and Data Collection:

[0071] Collect cold storage data, including the following:

[0072] Building structure, floor plan and section view of the cold storage, material and thickness of walls, roof and floor, etc.

[0073] Equipment parameters, model, operating characteristics, power, etc. of refrigeration equipment (compressor, condenser, evaporator, etc.).

[0074] Environmental data, meteorological data of the location of the cold storage, you can download the weather data file of the corresponding location from the EnergyPlus website.

[0075] Historical data, cold storage historical operation data (temperature, power consumption) and equipment operation data.

[0076] Electricity price data, specific time periods and prices of peak, valley and flat electricity prices.

[0077] Build a simulation environment:

[0078] Create the building geometry:

[0079] Define building geometry using OpenStudio, the SketchUp plugin or directly in EnergyPlus.

[0080] Define the spatial structure of the cold storage, including exterior walls, interior walls, roof, floor, doors and windows, etc.

[0081] Defining materials and construction

[0082] Set the material properties of walls, roofs, floors, etc. in EnergyPlus, including thermal conductivity, density, specific heat capacity, etc.

[0083] Define the thickness and order of each structural layer to ensure consistency with the actual cold storage structure.

[0084] Refrigeration system model construction:

[0085] Add refrigeration equipment in EnergyPlus, including compressor, condenser and evaporator.

[0086] Set the operating parameters of the equipment, such as cooling capacity, power consumption, efficiency, etc.

[0087] Define control strategies for refrigeration equipment, such as start-stop control, load regulation, etc.

[0088] Use the EMS module of EnergyPlus to write custom Python control interface logic to achieve intelligent control.

[0089] Model validation and parameter tuning

[0090] like Figure 2 As shown in the figure, based on the Python interface of the EMS module, the traditional constant temperature control method is implemented, and the temperature and power consumption simulation data of the cold storage are output and compared with the collected historical operation data of the cold storage. According to the simulation results, the parameters in the building model, equipment model and control strategy are adjusted to improve the simulation accuracy.

[0091] Rerun the simulation to verify the effect of the adjusted model until the model meets the accuracy requirements.

[0092] like Figure 3 As shown, it encapsulates the OpenAI,Gym environment interface. OpenAI,Gym, is a toolkit for developing and comparing reinforcement learning algorithms. It provides a rich collection of environments that allow users to test and evaluate various reinforcement learning algorithms. All environments have a unified interface, so that algorithms can be easily switched and tested in different environments. The main interfaces include,reset(),and,step(action),functions.,reset resets the environment to the initial state and returns the initial observation value.,step(action) performs an action in the environment and returns a tuple containing the next state, reward, whether to terminate, and other information.

[0093] S2. Construction of real-time control environment for cold storage

[0094] Connect to the cold storage PLC system, select the appropriate protocol Modbus or OPC, UA, collect real-time data of the cold storage, and provide equipment control interfaces such as compressors and air coolers, and interact with the deep reinforcement learning model for integration and control.

[0095] Here are the detailed build steps:

[0096] Data collection

[0097] Electricity price data: specific time periods and prices of peak, valley and flat electricity prices.

[0098] Connect to the PLC system to obtain real-time data on the cold storage such as temperature, power, and equipment operating status.

[0099] Connect to the meteorological data interface or install a small meteorological station to obtain real-time meteorological data (temperature, wind speed, etc.).

[0100] Filter, normalize and process outliers on the data to ensure data quality.

[0101] Building a real-time control environment

[0102] Equipment control, docking with PLC system, realizes the start and stop and power control of compressors and air coolers, and provides an action control interface for the deep enhancement model.

[0103] Encapsulate OpenAI and Gym environment interfaces

[0104] like Figure 4 As shown in the figure, by using Gym and API interface encapsulation environment, it is more convenient to connect the intelligent agents DRL and Agent trained in the simulation environment to the real-time control environment, and realize the rapid deployment and integration of deep reinforcement learning models.

[0105] like Figure 5 As shown in S3, deep reinforcement learning model training deployment and optimization, the cold storage energy-saving control method based on deep reinforcement learning (DRL) is divided into four main stages: pre-training, deployment, regular update optimization, and verification and evaluation. The following are the detailed implementation steps of each stage:

[0106] S31. Pre-training

[0107] Training the Agent

[0108] Algorithm selection: Select a suitable deep reinforcement learning algorithm (such as DDPG, DQN, etc.) and initialize the agent.

[0109] State and action space definition: Define the state space (ambient temperature, cold storage temperature, load, power consumption, electricity price, equipment operating status, etc.) and action space (such as compressor and air cooler start and stop, power adjustment, etc.) of the cold storage control system.

[0110] Reward function design: Design a reward function, which usually includes factors such as energy consumption, temperature stability, and equipment operating costs.

[0111] Specific design of reward function

[0112] Energy consumption reward

[0113] Objective: To encourage reduction of energy consumption in cold storage systems.

[0114] Design: Set a baseline energy consumption value, E baseline , calculate the current energy consumption, E, at each time step t , and the difference between the two is used as part of the energy consumption reward.

[0115] Formula: R energy =-k energy ×(E t -E baseline )

[0116] Among them, kenergy , is the energy consumption weight coefficient.

[0117] Temperature Control Bonus

[0118] Objective: Keep the temperature inside the cold storage within the set range and reduce temperature fluctuations.

[0119] Design: Set target temperature, T target , and the allowable temperature deviation range, ΔT. Calculate the current temperature, T t , and the deviation from the target temperature.

[0120] Formula: R temperature =-k temp ×max(0,|T t -T target |-ΔT)

[0121] Among them, k temp is the temperature weight coefficient.

[0122] Equipment Operation Rewards

[0123] Goal: Reduce frequent start and stop of equipment and extend equipment life.

[0124] Design: Set the penalty for each device start and stop. Calculate the number of device starts and stops N on / off .

[0125] Formula: R operation =-k operation ×N on / ofi

[0126] Among them, k operation , is the equipment operation weight coefficient.

[0127] Electricity Price Incentive

[0128] Goal: Optimize equipment operating time and reduce operation during peak electricity price periods.

[0129] Design: According to the electricity price P in different periods t , and penalties will be imposed for operating during peak electricity price periods.

[0130] Formula: R price =-k price ×P t ×E t

[0131] Among them, k price is the electricity price weight coefficient.

[0132] Combination of reward functions

[0133] The above-mentioned reward functions are combined to form the final comprehensive reward function. The comprehensive reward function should take into account energy consumption, temperature control, equipment operation and electricity price factors at the same time to ensure that the agent is optimized under multiple objectives.

[0134] Comprehensive reward function formula: R = R energy +R temperature +R operation +R price

[0135] By appropriately adjusting the weight coefficient k of each part price ,,k temp ,,k operation and k price , balance the impact between different objectives and ensure that the intelligent agent can achieve good results in both energy saving and temperature control.

[0136] Training process: In the cold storage simulation environment, the DRL algorithm is run for training. Through continuous trial and error and optimization, the DRL and Agent gradually learn the optimal control strategy.

[0137] S32. Deployment

[0138] Deploy Agent

[0139] Model deployment: deploy the pre-trained intelligent agent and connect it to the cold storage real-time control environment system.

[0140] Real-time control

[0141] Strategy Execution,Based on the real-time collected data, the DRL,Agent executes the pre-trained control strategy to adjust the,operating status of the refrigeration equipment.

[0142] Data recording, recording the data of state, action, reward and next state, forming experience replay storage.

[0143] Evaluation and Verification

[0144] Operation verification: Verify the control effect of the new model in actual cold storage operation to ensure that it can achieve the expected energy-saving goals in a real environment.

[0145] User feedback, collect feedback from cold storage operators and managers to further improve the control strategy.

[0146] S33. Regular update and optimization

[0147] Data storage and processing

[0148] The experience replay pool stores the data collected during real-time operation in the experience replay pool for subsequent model updates.

[0149] Data preprocessing: preprocess the collected data to remove noise and outliers to ensure data quality.

[0150] Regular updates

[0151] Model fine-tuning: Regularly extract data from the experience replay pool and use the online training mechanism to fine-tune the DRL model.

[0152] Small batch training uses small batches of data for training and gradually updates the parameters of the DRL model to adapt to the latest environmental changes.

[0153] Evaluation and Verification

[0154] Model testing,The updated DRL model is tested in a cold storage simulation environment to evaluate its control effect under different working conditions.

[0155] Performance evaluation, by comparing indicators such as energy consumption and temperature fluctuation, to evaluate whether the performance of the new model is better than the old model.

[0156] The present invention realizes intelligent control and continuous optimization of the cold storage system through four stages: pre-training, deployment, regular update optimization and verification evaluation. By building a simulation environment for pre-training, deploying to the actual environment for real-time control, and regularly performing online optimization, it ensures that the control system always maintains the best performance and ultimately achieves the goal of energy saving and consumption reduction.

[0157] The present invention has the following significant advantages:

[0158] 1. Significant energy saving effect

[0159] 1.1 Dynamic Optimization

[0160] Advantages: Through the energy-saving control method implemented by the present invention, the cold storage control system can dynamically optimize the operating status of the refrigeration equipment in real time.

[0161] Beneficial effect: Compared with traditional constant temperature control methods, energy consumption is significantly reduced, saving electricity costs.

[0162] 1.2. Utilization of peak and valley electricity prices

[0163] Advantages: Intelligent control systems can adjust operating strategies based on real-time electricity prices to avoid peak electricity price periods.

[0164] Beneficial effects: further reduce operating costs and improve economic benefits.

[0165] 2. High temperature control accuracy

[0166] 2.1. Precise control

[0167] Advantages: The energy-saving control method proposed in the present invention can learn and adapt to different environmental changes and keep the internal temperature of the cold storage within the set range.

[0168] Beneficial effects: Reduce temperature fluctuations, ensure the quality and safety of stored items.

[0169] 2.2 Intelligent Adjustment

[0170] Advantages: The system can intelligently adjust the operation of refrigeration equipment according to real-time temperature and load changes.

[0171] Beneficial effects: Improve temperature control accuracy and extend the shelf life of goods.

[0172] 3. Extended equipment life

[0173] 3.1. Reduce frequent starts and stops

[0174] Advantages: By optimizing equipment operation strategies, the frequent start and stop of key equipment such as compressors can be reduced.

[0175] Beneficial effects: Extend equipment life and reduce maintenance and replacement costs.

[0176] 3.2 Preventive maintenance

[0177] Advantages: Intelligent systems can predict potential failures and perform maintenance in advance based on equipment operation data.

[0178] Beneficial effects: Reduce unexpected downtime and ensure stable system operation.

[0179] 4. Strong adaptability

[0180] 4.1 Adaptability

[0181] Advantages: The energy-saving control method proposed in the present invention has adaptive capabilities and can be adjusted according to the actual conditions of different cold storages.

[0182] Beneficial effects: Suitable for cold storage systems of various sizes and types, with strong flexibility.

[0183] 4.2 Continuous Optimization

[0184] Advantages: The system continuously optimizes the control strategy through continuous online learning and adapts to changes in the environment and load.

[0185] Beneficial effect: Maintain high efficiency and energy-saving performance of the system in long-term operation.

[0186] 5. Environmentally friendly

[0187] 5.1. Reduce carbon emissions

[0188] Advantages: Energy-saving control strategies reduce the power consumption of cold storage systems and correspondingly reduce greenhouse gas emissions.

[0189] Beneficial effects: It complies with environmental protection policies and sustainable development requirements, and has a positive effect on environmental protection.

[0190] 6. Significant economic benefits

[0191] 6.1. Reduce operating costs

[0192] Advantages: Intelligent control systems significantly reduce cold storage operating costs by optimizing energy consumption and taking advantage of peak and valley electricity prices.

[0193] Beneficial effects: Improved the economic benefits and market competitiveness of the enterprise.

[0194] 6.2. Quick return on investment

[0195] Advantages: Due to the significant energy-saving effect, the cost of system investment can be recovered in a relatively short period of time through saved electricity bills.

[0196] Beneficial effects: Improved the project's return on investment and promoted the popularization and application of intelligent control technology.

[0197] Therefore, the present invention adopts the above-mentioned cold storage energy-saving control method based on deep reinforcement learning, and achieves significant energy-saving effects and high-precision temperature control through intelligent and dynamic control strategies, which not only prolongs the service life of the equipment and reduces maintenance costs, but also has good environmental friendliness and significant economic benefits. Its adaptive ability and continuous optimization characteristics enable this method to be widely used in various cold storage systems, bringing long-term economic and social benefits to enterprises.

[0198] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A cold storage energy-saving control method based on deep reinforcement learning, characterized in that: The following steps are involved: S1. Construction of cold storage simulation environment. Collect cold storage data through information and data acquisition. Use OpenStudioSketchUp plug-in or directly define the building geometry and the spatial structure of the cold storage in EnergyPlus to build a refrigeration system model. Verify and adjust the parameters of the model. Encapsulate the OpenAIGym environment interface. S2. Build the real-time control environment for cold storage, connect to the cold storage PLC system, select the appropriate protocol Modbus or OPCUA, collect the real-time data of the cold storage, and provide control interfaces for equipment such as compressors and air coolers, and interact with the deep reinforcement learning model for integration and control; S3. Deep reinforcement learning model training, deployment and optimization. Select appropriate deep reinforcement learning algorithms to pre-train the intelligent agent, deploy and connect the pre-trained intelligent agent to the cold storage real-time control environment system for real-time control and regular update and optimization, and evaluate and verify the model.

2. According to a cold storage energy-saving control method based on deep reinforcement learning according to claim 1, it is characterized in that: In step S1, collecting cold storage data includes the following: Building structure, floor plan and section of the cold storage, material and thickness of walls, roof and floor; Equipment parameters, model, operating characteristics and power of the refrigeration equipment; Environmental data, meteorological data of the location of the cold storage; Historical data, including historical operating temperature data, power consumption data and equipment operating data of cold storage; Electricity price data, specific time periods and prices of peak, valley and flat electricity prices.

3. According to a cold storage energy-saving control method based on deep reinforcement learning according to claim 1, it is characterized in that: In step S1, building a simulation environment specifically includes the following steps: Create a building geometry model, use the OpenStudioSketchUp plug-in or define the building geometry directly in EnergyPlus, and define the spatial structure of the cold storage, including exterior walls, interior walls, roof, floor, doors and windows; Define materials and structures, set the material properties of walls, roofs, floors, etc. in EnergyPlus, including thermal conductivity, density, specific heat capacity, and define the thickness and order of each structural layer to ensure consistency with the actual cold storage structure; Refrigeration system model building, adding refrigeration equipment in EnergyPlus, including compressors, condensers and evaporators, setting the equipment's operating parameters, cooling capacity, power consumption and efficiency, and defining the control strategies for refrigeration equipment start-stop control and load regulation; Model verification and parameter adjustment: Based on the Python interface of the EMS module, the temperature and power consumption simulation data of the cold storage are output and compared with the collected historical operation data of the cold storage. According to the simulation results, the parameters in the building model, equipment model and control strategy are adjusted, and finally the OpenAIGym environment interface is encapsulated.

4. According to a cold storage energy-saving control method based on deep reinforcement learning according to claim 1, it is characterized in that: In step S2, the construction of the cold storage real-time control environment specifically includes the following steps: Collect electricity price data, connect to the PLC system, obtain the real-time temperature, power, and equipment operation status of the cold storage, connect to the meteorological data interface or install a small meteorological station to obtain real-time meteorological data, filter, normalize and process outliers to ensure data quality; Build a real-time control environment, connect to the PLC system, realize the start and stop and power control of the compressor and air cooler, provide an action control interface for the deep reinforcement model, and use the GymAPI interface to encapsulate the environment.

5. According to a cold storage energy-saving control method based on deep reinforcement learning according to claim 1, it is characterized in that: In step S3, the deep reinforcement learning model training deployment and optimization specifically includes the following steps: S31, pre-training, select the appropriate deep reinforcement learning algorithm, initialize the intelligent agent, define the state space and action space of the cold storage control system, design the reward function, including energy consumption, temperature stability and equipment operation cost; run the DRL algorithm for training in the cold storage simulation environment, and through continuous trial and error and optimization, DRLAgent gradually learns the optimal control strategy; S32, deployment, deploying the pre-trained agent and connecting it to the cold storage real-time control environment system. According to the real-time collected data, DRLAgent executes the pre-trained control strategy, adjusts the operating status of the refrigeration equipment, and records the data of the status, action, reward and next status to form experience playback storage; S33. Regularly update and optimize, store the data collected during real-time operation in the experience replay pool for subsequent model updates, preprocess the collected data to remove noise and outliers, and ensure data quality; regularly extract data from the experience replay pool, use the online training mechanism to fine-tune the DRL model, train with small batches of data, and gradually update the parameters of the DRL model to adapt to the latest environmental changes; test the updated DRL model in a cold storage simulation environment, evaluate its control effect under different working conditions, and evaluate whether the performance of the new model is better than the old model by comparing indicators such as energy consumption and temperature fluctuations.

6. According to the cold storage energy-saving control method based on deep reinforcement learning according to claim 5, it is characterized in that: In step S31, the specific design of the reward function is as follows: Energy consumption reward Objective: To encourage reduction of energy consumption in cold storage systems; Design: Setting a benchmark - energy consumption value E baseline , calculate the current energy consumption E at each time step t , and take the difference between the two as part of the energy consumption reward; Formula: R energy = -k energy ×(E t - E baseline ) Among them, k energy is the energy consumption weight coefficient; Temperature Control Bonus Objective: Keep the temperature inside the cold storage within the set range and reduce temperature fluctuations. Design: Set target temperature T target And the allowable temperature deviation range ΔT. Calculate the current temperature T t Deviation from target temperature; Formula: R temperature = -ktemp×max(0, |T t -T target |-ΔT) Among them, k temp is the temperature weight coefficient; Equipment Operation Rewards Goal: Reduce frequent start and stop of equipment and extend equipment life; Design: Set the penalty for each device start and stop, and calculate the number of device starts and stops N on / off ; Formula: R operation =-k operation ×N on / off Among them, k operation The weight factor for the device operation. Electricity Price Incentive Goal: Optimize equipment operating time and reduce operation during peak electricity price periods; Design: According to the electricity price P in different periods t , impose penalties on operation during peak electricity price periods; Formula: R price =-k price ×P t ×E t Among them, k price is the electricity price weight coefficient; Combination of reward functions Combine the above reward functions to form the final comprehensive reward function. The comprehensive reward function should take into account energy consumption, temperature control, equipment operation and electricity price factors to ensure that the agent is optimized under multiple objectives. Comprehensive reward function formula: R = R energy +R temperature +R operation +R price By appropriately adjusting the weight coefficient k of each part price , k temp , k operation and k price , balance the impact between different objectives and ensure that the intelligent agent can achieve good results in both energy saving and temperature control.

Citation Information

Cited By

  • Control method of energy-saving composite refrigeration cabinet with intelligent wide temperature control

    CN119436725A

  • A control method for energy-saving composite refrigeration cabinet with intelligent wide-range temperature control

    CN119436725B

  • Data center energy consumption optimization method based on joint multi-objective optimization model

    CN119620842A

  • Grate cooler fan parameter optimization method, device and equipment and storage medium

    CN120180884A

  • Cold storage energy-saving control method based on deep reinforcement learning

    CN121210865A