A method, device, equipment and storage medium for interaction between a heating system and a power grid

Abstract

The present invention belongs to the field of heating supply and discloses a method, device, equipment, and storage medium for interaction between a heating system and a power grid. The method comprises: obtaining real-time electricity prices, dynamic threshold electricity prices, and operating data of a phase-change heat storage system, and inputting the obtained data into a pre-established Markov decision process model, which includes a heat storage process sub-model and a heating process sub-model; determining the current moment attribute as a heat storage moment or a heating moment based on the real-time electricity price, dynamic threshold electricity price, and heat load, and selecting a target sub-model from the two sub-models of the Markov decision process model based on the determined moment attribute; solving the target sub-model using a DQN algorithm, and controlling the operation of the phase-change heat storage system based on the solution. This method can accurately meet building energy needs, reduce user energy costs, promote real-time supply and demand balance in the power system, and alleviate power supply pressure during peak hours.

Description

Technical Field

[0001] The present invention belongs to the field of heating, and in particular relates to a method, device, equipment and storage medium for interaction between a heating system and a power grid. Background Art

[0002] Significantly increasing renewable energy generation and energy storage is a key path to decarbonizing the power sector. Energy storage refers to technologies that enable the cross-temporal and spatial transfer of energy between charging and discharging, reshaping the electricity supply and demand curve. It encompasses various types of storage, including cold storage, heat storage, and electricity storage. Phase change thermal storage offers advantages such as high energy storage density, constant temperature, manageable costs, and strong safety. When coupled with electric heating equipment for building heating, it can both meet building energy needs and enable interaction with the power grid.

[0003] In related technologies, the control signals for phase-change heat storage systems are often peak-valley and flat-rate electricity prices, and the control target is often the lowest operating cost. The control method is often rule-based control. Specifically, during the nighttime valley-rate electricity period, the heat source operates at maximum power to store thermal energy. During the daytime heating or peak electricity price period, the phase-change device operates at a fixed flow rate (or adjusts the flow rate based on the supply and return water temperature difference) to replace the heat source for heating. This control method has achieved peak-shaving and valley-filling effects to a certain extent, but the following problems still exist: 1) Peak-valley and flat-rate electricity prices are gradually failing to reflect the real-time status of power systems with a large proportion of renewable energy penetration. Although real-time electricity prices provide real-time guidance for demand-side regulation, phase-change heat storage systems cannot yet accurately respond to real-time electricity prices; 2) The heat storage and supply process is not precisely matched to the dynamically changing building heat load; 3) The time-differentiated characteristics of the phase-change heat storage device are ignored, namely, the storage and release power will change with the storage and release process.

[0004] Therefore, how to control the operation of the phase change storage heating system under the coupling of multiple dynamic characteristics of real-time demand of the power grid, time-varying load of buildings and differentiated power of phase change heat storage has become a technical problem that needs to be solved urgently. Summary of the Invention

[0005] In view of this, the present invention discloses a method, device, equipment and storage medium for interaction between a heating system and a power grid, which can solve the deficiencies in the related art.

[0006] To achieve the above purpose, the present invention discloses the following technical solutions:

[0007] According to a first aspect of the present invention, a method for interacting with a heating system and a power grid is proposed, which is applied to a phase-change heat storage system. The phase-change heat storage system includes an electric heating device and a phase-change heat storage device. The electric heating device is used to obtain electricity from the power grid to heat water, and the phase-change heat storage device is used to store heat and provide heat using the stored heat. The method includes:

[0008] Acquiring real-time electricity prices, dynamic threshold electricity prices, and operating data of the phase change heat storage and supply system, and inputting the acquired data into a pre-established Markov decision process model, wherein the Markov decision process model includes a heat storage process sub-model and a heating process sub-model, wherein the two sub-models correspond to different environmental states, action spaces, and reward functions;

[0009] Determining the moment attribute of the current moment as a heat storage moment or a heat supply moment according to the real-time electricity price, the dynamic threshold electricity price, and the heat load, and selecting a target sub-model from the two sub-models of the Markov decision process model based on the determined moment attribute;

[0010] The target sub-model is solved according to the DQN algorithm, and the operation of the phase change heat storage system is controlled according to the solution result, wherein the solution result is the control parameter for the electric heating equipment and the phase change heat storage device.

[0011] According to a second aspect of the present invention, a device for interacting with a heating system and a power grid is provided, which is applied to a phase-change heat storage system. The phase-change heat storage system further includes an electric heating device and a phase-change heat storage device. The electric heating device is used to obtain electricity from the power grid to heat water, and the phase-change heat storage device is used to store heat and provide heat through the stored hot water. The device includes:

[0012] An acquisition unit is configured to acquire real-time electricity prices, dynamic threshold electricity prices, and operating data of the phase change heat storage and supply system, and input the acquired data into a pre-established Markov decision process model, wherein the Markov decision process model includes a heat storage process sub-model and a heating process sub-model, and the two sub-models correspond to different environmental states, action spaces, and reward functions;

[0013] A selection unit: determining, according to the real-time electricity price, the dynamic threshold electricity price, and the heat load, whether the moment attribute of the current moment is a heat storage moment or a heat supply moment, and selecting a target sub-model from the two sub-models of the Markov decision process model based on the determined moment attribute;

[0014] Control unit: solves the target sub-model according to the DQN algorithm, and controls the operation of the phase change heat storage system according to the solution result, wherein the solution result is the control parameter for the electric heating equipment and the phase change heat storage device.

[0015] According to a third aspect of the present invention, an electronic device is provided, comprising:

[0016] processor;

[0017] a memory for storing processor-executable instructions;

[0018] The processor implements the steps of the method described in the first aspect by running the executable instructions.

[0019] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which computer instructions are stored. When the instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0020] As can be seen from the above technical solutions, the operating method of the phase-change heat storage system disclosed in the present invention receives the day-ahead real-time electricity price from the power grid and the predicted heat load of the building, inputs the electricity price and load sequence into the grid-interactive control model of the phase-change heat storage system, and couples the dynamic characteristics of the phase-change heat storage. The model outputs the hourly control parameters of the phase-change heat storage device and electric heating equipment for the day, including the heat storage and release time, the outlet water temperature of the electric heating equipment, and the temperature and flow rate of the heat exchange fluid entering the phase-change heat storage device. By implementing the present invention, the phase-change heat storage system can accurately meet the energy needs of the building, reduce user energy costs, promote real-time supply and demand balance of the power system, and alleviate power supply pressure during peak power periods. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is an architectural diagram of an interactive system between a heating system and a power grid provided by an exemplary embodiment;

[0022] Figure 2 is a flow chart of a method for interaction between a heating system and a power grid provided by an exemplary embodiment;

[0023] Figure 3 This is a schematic diagram of a DQN algorithm process provided by an exemplary embodiment;

[0024] Figure 4 is a schematic structural diagram of a device provided by an exemplary embodiment;

[0025] Figure 5 It is a block diagram of an interactive device between a heating system and a power grid provided by an exemplary embodiment. DETAILED DESCRIPTION

[0026] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, like numbers in different figures represent like or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible implementations consistent with one or more embodiments of the present invention. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of one or more embodiments of the present invention, as detailed in the appended claims.

[0027] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in the present invention. In some other embodiments, the method may include more or fewer steps than those described in the present invention. In addition, a single step described in the present invention may be broken down into multiple steps for description in other embodiments, and multiple steps described in the present invention may be combined into a single step for description in other embodiments.

[0028] To further illustrate the present invention, the following examples are provided:

[0029] Significantly increasing renewable energy generation and energy storage is a key path to decarbonizing the power sector. Energy storage refers to technologies that enable the cross-temporal and spatial transfer of energy between charging and discharging, reshaping the electricity supply and demand curve. It encompasses various types of storage, including cold storage, heat storage, and electricity storage. Phase change thermal storage offers advantages such as high energy storage density, constant temperature, manageable costs, and strong safety. When coupled with electric heating equipment for building heating, it can both meet building energy needs and enable interaction with the power grid.

[0030] In related technologies, the control signals for phase-change heat storage systems are often peak-valley and flat-rate electricity prices, and the control target is often the lowest operating cost. The control method is often rule-based control. Specifically, during the nighttime valley-rate electricity period, the heat source operates at maximum power to store thermal energy. During the daytime heating or peak electricity price period, the phase-change device operates at a fixed flow rate (or adjusts the flow rate based on the supply and return water temperature difference) to replace the heat source for heating. This control method has achieved peak-shaving and valley-filling effects to a certain extent, but the following problems still exist: 1) Peak-valley and flat-rate electricity prices are gradually failing to reflect the real-time status of power systems with a large proportion of renewable energy penetration. Although real-time electricity prices provide real-time guidance for demand-side regulation, phase-change heat storage systems cannot yet accurately respond to real-time electricity prices; 2) The heat storage and supply process is not precisely matched to the dynamically changing building heat load; 3) The time-differentiated characteristics of the phase-change heat storage device are ignored, namely, the storage and release power will change with the storage and release process.

[0031] In order to solve the deficiencies in the related art, the present invention proposes an interaction method between a heating system and a power grid.

[0032] Figure 1 This is an architectural diagram of a phase change heat storage system provided by an exemplary embodiment. Figure 1 As shown, the system includes an electric heating device 1, a first phase-change heat storage device 2, a second phase-change heat storage device 2', a plate heat exchanger 3, a building 4 using floor radiation heating, and five valves (V1, V2, V3, V4, V5).

[0033] In the first case, valves V1, V3, V4, and V5 are opened, V2 is closed, and the electric heating device 1 is turned on. At this time, the electric heating device 1 can heat the flowing water, and the heated flowing water enters the first phase change heat storage device 2 and the second phase change heat storage device 2' to store heat.

[0034] In the second case, valve V1 is opened, valves V2, V3, V4, and V5 are closed, and the electric heating device 1 is turned on. At this time, the electric heating device 1 directly heats the building water.

[0035] In the third case, valves V2, V3, and V4 are opened, V1 and V5 are closed, and the electric heating device 1 is turned off. At this time, the first phase-change heat storage device 2 and the second phase-change heat storage device 2' provide heat for the building water.

[0036] In actual situations, the electric heating equipment may be a heat pump, the maximum outlet water temperature may be 80°C, and the filling material in the phase change device may be paraffin with a phase change temperature of 60°C.

[0037] Figure 2 FIG. 1 is a flow chart of a method for interaction between a heating system and a power grid provided by an exemplary embodiment. Figure 2 As shown, the method is applied to a phase change heat storage and supply system, the phase change heat storage and supply system including an electric heating device and a phase change heat storage device, the electric heating device is used to obtain electricity from the power grid to heat water, and the phase change heat storage device is used to store heat and provide heat through the stored heat; the method may include the following steps:

[0038] Step 201: Acquire real-time electricity prices, dynamic threshold electricity prices, and operating data of the phase change heat storage and supply system, and input the acquired data into a pre-established Markov decision process model. The Markov decision process model includes a heat storage process sub-model and a heating process sub-model. The two sub-models correspond to different environmental states, action spaces, and reward functions.

[0039] The operating data may include the predicted heat load data of the building, as well as various parameters mentioned in subsequent expressions, which will not be described in detail here.

[0040] The electric heating equipment model is shown as follows:

[0041] ;

[0042] in, Indicates the electric power of the electric heating equipment (kW); It represents the thermal power (kW) of electric heating equipment; EER stands for energy efficiency ratio.

[0043] The phase change heat storage device model is shown as follows:

[0044] When the heat transfer fluid inlet temperature Greater than the melting temperature of the phase change material , the phase change heat storage device stores heat, and the heat storage power is:

[0045] ;

[0046] When the heat transfer fluid inlet temperature Lower than the solidification temperature of the phase change material , the phase change heat storage device releases heat, and the heat release power is:

[0047] ;

[0048] in, and is the mass flow rate of the heat transfer fluid for storage and release (kg / s); and is the constant-pressure specific heat of the heat storage and release fluid (kJ / (kg·K)); and is the thermal resistance of the phase change heat storage device (K / kW), which is a function related to the state of heat storage (SOC) and is obtained based on heat storage and release cycle experimental tests and polynomial fitting, as shown below:

[0049] ;

[0050] ;

[0051] in, , , , , , , and Coefficients identified for experimental tests.

[0052] The expression of dynamic threshold electricity price is:

[0053] ;

[0054] in, The valley price of local time-of-use electricity price (yuan / kWh); and are the mean and standard deviation of the day-ahead real-time electricity price published by the local power grid (yuan / kWh).

[0055] Step 202: determine the moment attribute of the current moment as a heat storage moment or a heat supply moment according to the real-time electricity price, the dynamic threshold electricity price and the heat load, and select a target sub-model from the two sub-models of the Markov decision process model based on the determined moment attribute.

[0056] The determining, based on the real-time electricity price, the dynamic threshold electricity price, and the heat load, that the moment attribute of the current moment is a heat storage moment or a heat supply moment includes: when the real-time electricity price is lower than the dynamic threshold electricity price, determining that the current moment is a heat storage moment, and the heat storage moment corresponds to a heat storage process sub-model; when the real-time electricity price is higher than the dynamic threshold electricity price and the heat load is greater than 0, determining that the current moment is a heat supply moment, and the heat supply moment corresponds to a heat supply process sub-model.

[0057] Heat storage process sub-model part:

[0058] The expression of the environmental state is:

[0059] ;

[0060] in, represent Moment The thermal storage state of each phase change thermal storage device (kWh); for The real-time electricity price at the moment (yuan / kWh).

[0061] The action space expression is:

[0062] ;

[0063] in, represent Enter the moment Heat exchange fluid flow rate of each phase change heat storage device (m3 / h); Represents the temperature of the heat exchange fluid entering the phase change heat storage device, that is, the outlet water temperature of the electric heating equipment (℃).

[0064] The reward function expression is designed as

[0065] ;

[0066] in, for The electric power consumed by the electric heating equipment at any moment (kW); for Electric power consumed by the water pump at any moment (kW); is the time interval, here we take 1 hour. When the real-time electricity price is lower than the dynamic threshold price, the reward is positive. The lower the electricity price, the greater the power consumption, and the greater the reward. The Markov decision process solution is more inclined to this situation.

[0067] Exothermic process sub-model part:

[0068] The expression of the environmental state is:

[0069] ;

[0070] in, represent The building heat load (kW) at the moment.

[0071] The action space expression is:

[0072] ;

[0073] in, represent Enter the moment The heat exchange fluid flow rate of each phase change heat storage device (m3 / h).

[0074] The reward function expression is:

[0075] ;

[0076] in, for Heating capacity at the time (kW); for Building heat load (kW) at the moment. When the reward function takes the maximum value, the supply-demand mismatch of the phase change storage heating system is minimized.

[0077] Step 203: Solve the target sub-model according to the DQN algorithm, and control the operation of the phase change heat storage system according to the solution result, wherein the solution result is the control parameter for the electric heating equipment and the phase change heat storage device.

[0078] DQN (Deep Q-Network) is a reinforcement learning algorithm that combines deep learning and Q-Learning to solve decision-making problems in high-dimensional state spaces. In this paper, DQN is used to train a phase-change heat storage and supply agent to optimize heat storage and release strategies.

[0079] Figure 3 This is a flowchart for training and testing a phase-change storage-heating DQN agent, which includes the following steps:

[0080] Step 301: Episode number = 1, initialize the heat storage agent, and divide 24 hours into 24 time steps;

[0081] Step 302: At time step t=0, the current evaluation network of the heat storage agent adopts Strategy selection action And execute, the environment returns the reward value and the new state ;

[0082] ;

[0083] Step 303: Convert the process data ( ) is stored in the experience pool and randomly sampled from the experience pool as training data for the current evaluation network;

[0084] Step 304: Calculate the Q value based on the current target network and update the weight of the evaluation network through gradient backpropagation and bias ;

[0085] Step 305: At time step t+=1, repeat steps 302-304 until t=24;

[0086] Step 306: The number of episodes is episode+=1. Repeat steps 301-305 to update the target network according to the set update frequency, thereby updating the evaluation network until the number of episodes reaches the set value.

[0087] Step 307: Save the evaluation network, and the training of the heat storage agent is completed.

[0088] In actual situations, you can set the initial is 1, The decay rate is 0.992, the learning rate is 0.0001, the discount factor is 0.995, the target network update frequency is 10, the experience pool size is 12*100, and the experience replay sampling size is 12.

[0089] The heating strategy of this invention can be summarized as follows: on days with low load and high electricity prices, direct heating using phase change thermal storage is preferred; on days with low load and low electricity prices, direct heating using heat pumps is appropriately adopted; on days with high load and low electricity prices, phase change thermal storage is preferred until electricity prices are relatively high, with direct heating using heat pumps being preferred when electricity prices are near zero; on days with high load and high electricity prices, direct heating using heat pumps should be adopted when electricity prices are relatively low, in order to enable phase change thermal storage to play a key role in grid interaction and meet building load demands when electricity prices are extremely high. Overall, heat storage is used when daily electricity prices are low to absorb excess electricity, while electricity substitution is performed when daily electricity prices are high, alleviating power shortages.

[0090] In this embodiment, the grid's day-ahead real-time electricity price and the building's predicted heat load are received. The price and load sequence are then input into a grid-interactive control model for the phase-change heat storage system. Coupled with the dynamic characteristics of phase-change heat storage, the model outputs the hourly control parameters for the phase-change heat storage device and electric heating equipment for the day, including heat storage and release times, the outlet water temperature of the electric heating equipment, and the temperature and flow rate of the heat exchange fluid entering the phase-change heat storage device. By implementing this invention, the phase-change heat storage system can accurately meet building energy needs, reduce user energy costs, promote real-time supply and demand balance in the power system, and alleviate power supply pressure during peak hours.

[0091] In one embodiment, the operation of the phase change heat storage system is subject to boundary conditions, and the boundary conditions include at least one of the following: system side boundary, grid side boundary condition, and user side boundary condition; wherein, the system side boundary is an electric heating coupled phase change heat storage system, the grid side boundary condition is a real-time electricity price published by the grid, and the user side boundary condition is a building heat load forecast.

[0092] The phase-change heat storage system can include a boundary condition preprocessing module, a phase-change heat storage agent module, and a phase-change heat storage system control module. The boundary condition preprocessing module is used to receive the real-time electricity price signal and the building's predicted heat load data released by the power grid on the day before and unify them to the same time scale (such as 1 hour or 15 minutes). It then calculates the dynamic threshold electricity price and outputs hourly time attributes. The phase-change heat storage agent module is used to establish a phase-change heat storage system model and a Markov decision process model of its interaction with the power grid. The DQN algorithm is used to train the phase-change heat storage agent. It inputs the real-time electricity price, time-varying load, and a dynamic model of the phase-change heat storage device, and outputs system control parameters, including heat storage and release time, the outlet water temperature of the electric heating equipment, and the flow rate of the heat exchange fluid in the phase-change heat storage device. The phase-change heat storage system control module is used to receive the system control parameters issued by the phase-change heat storage agent module and control the electric heating temperature control devices, system water pumps, and valves to ensure that the control parameters reach the target values.

[0093] Figure 4 This is a schematic structural diagram of a device provided by an exemplary embodiment. Figure 4At the hardware level, the device includes a processor 402, an internal bus 404, a network interface 406, a memory 408, and a non-volatile memory 410. Of course, it may also include hardware required for other functions. One or more embodiments of the present invention can be implemented based on software, such as the processor 402 reading the corresponding computer program from the non-volatile memory 410 into the memory 408 and then running it. Of course, in addition to software implementation, one or more embodiments of the present invention do not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0094] Please refer to Figure 5 , a device for interaction between a heating system and a power grid can be applied to Figure 5 The device shown in the figure is used to implement the technical solution of the present invention and is applied to a phase change heat storage system. The phase change heat storage system also includes an electric heating device and a phase change heat storage device. The electric heating device is used to obtain electricity from the power grid to heat water, and the phase change heat storage device is used to store heat and provide heat through the stored heat. The device includes:

[0095] An acquisition unit 501 is configured to acquire real-time electricity prices, dynamic threshold electricity prices, and operating data of the phase change heat storage and supply system, and input the acquired data into a pre-established Markov decision process model, wherein the Markov decision process model includes a heat storage process sub-model and a heating process sub-model, wherein the two sub-models correspond to different environmental states, action spaces, and reward functions;

[0096] a selection unit 502 for determining, based on the real-time electricity price, the dynamic threshold electricity price, and the heat load, whether the moment attribute of the current moment is a heat storage moment or a heat supply moment, and selecting a target sub-model from the two sub-models of the Markov decision process model based on the determined moment attribute;

[0097] The control unit 503 is used to solve the target sub-model according to the DQN algorithm, and control the operation of the phase change heat storage system according to the solution result, wherein the solution result is the control parameter for the electric heating equipment and the phase change heat storage device.

[0098] Optionally, the dynamic threshold electricity price is determined based on the valley electricity price in the local time-of-use electricity price and the day-ahead real-time electricity price data. The expression of the dynamic threshold electricity price is:

[0099] ;

[0100] in, It is the valley price electricity in the local time-of-use electricity price. and are the mean and standard deviation of the day-ahead real-time electricity price published by the local power grid.

[0101] Optional,

[0102] The expression of the environmental state of the heat storage process sub-model is:

[0103] ;

[0104] in, for Moment The heat storage state of a phase change heat storage device, for Real-time electricity prices at the moment;

[0105] The expression of the action space of the heat storage process sub-model is:

[0106] ;

[0107] in, for Enter the moment The heat exchange fluid flow rate of a phase change heat storage device, is the temperature of the heat exchange fluid entering the phase change heat storage device, which is the same as the outlet water temperature of the electric heating equipment;

[0108] The reward function of the heat storage process sub-model is expressed as:

[0109] ;

[0110] in, for The electric power consumed by the electric heating equipment at all times, for The electrical power consumed by the pump at any moment, is the time interval;

[0111] The expression of the environmental state S of the heating process sub-model is:

[0112] ;

[0113] in, represent Building heat load at any moment;

[0114] The expression of the action space of the heating process sub-model is:

[0115] ;

[0116] in, represent Enter the moment The heat exchange fluid flow rate of a phase change heat storage device, represent The flow rate of fluid heated by the electric heating equipment at any moment;

[0117] The reward function of the heating process sub-model is expressed as:

[0118] ;

[0119] in, for The heat supply at all times, for Building heat load at all times.

[0120] Optional,

[0121] The heating expression of the electric heating equipment is:

[0122] ;

[0123] in, Indicates the electric power of electric heating equipment. It represents the thermal power of electric heating equipment, and EER stands for energy efficiency ratio.

[0124] The heat storage and release expression of the phase change heat storage device is:

[0125] When the heat transfer fluid inlet temperature Greater than the melting temperature of the phase change material When , the heat storage power of the phase change heat storage device is:

[0126] ;

[0127] When the heat transfer fluid inlet temperature Lower than the solidification temperature of the phase change material When , the heat release power of the phase change heat storage device is:

[0128] ;

[0129] in, and are the mass flow rates of heat storage and heat transfer fluids, and are the constant pressure specific heat of the heat storage and release heat transfer fluids, and are the heat storage and release thermal resistances of the phase change heat storage device, respectively.

[0130] Optionally, the selection unit 502 is specifically configured to:

[0131] When the real-time electricity price is lower than the dynamic threshold electricity price, determining the current moment as a heat storage moment, the heat storage moment corresponding to a heat storage process sub-model;

[0132] When the real-time electricity price is higher than the dynamic threshold electricity price and the heat load is greater than 0, the current moment is determined to be a heating moment, and the heating moment corresponds to a heating process sub-model.

[0133] Optionally, the control unit 503 is specifically configured to:

[0134] use The policy selects and executes the action space to calculate the corresponding reward value and the new environment state;

[0135] The conversion process data is stored in the experience pool, and randomly sampled from the experience pool as the training data of the current evaluation network;

[0136] Calculate the Q value based on the current target network and update the weight of the evaluation network through gradient backpropagation and bias ;

[0137] Repeat the above steps until the number of update rounds reaches the set value.

[0138] Optionally, there are boundary conditions for the operation of the phase change heat storage system, and the boundary conditions include at least one of the following: system side boundary, grid side boundary condition, and user side boundary condition; wherein, the system side boundary is an electric heating coupled phase change heat storage system, the grid side boundary condition is the real-time electricity price published by the grid, and the user side boundary condition is the building heat load forecast.

[0139] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0140] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0141] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0142] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0143] For the computer-readable medium (or computer-readable storage medium) as described above or in any other form, computer instructions may be stored thereon, which, when executed by a processor, implement one or more of the above-mentioned embodiments, thereby realizing the technical solution of the present invention.

[0144] The present invention further provides a computer program that, when executed by a processor, implements one or more of the aforementioned embodiments, thereby realizing the technical solution of the present invention. The computer program may be recorded on the aforementioned or any other form of computer-readable medium, and the present invention is not limited thereto.

[0145] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0146] The foregoing description describes specific embodiments of the present invention. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0147] The terms used in one or more embodiments of the present invention are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of the present invention. The singular forms "a", "an", "the" and "the" used in one or more embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0148] It should be understood that although the terms first, second, third, etc. may be used to describe various information in one or more embodiments of the present invention, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of one or more embodiments of the present invention, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining."

[0149] The above description is merely a preferred embodiment of one or more embodiments of the present invention and is not intended to limit one or more embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present invention shall be included in the scope of protection of one or more embodiments of the present invention.

Claims

1. A method for interaction between a heating system and a power grid, characterized in that: Applied to a phase-change heat storage system, the phase-change heat storage system includes an electric heating device and a phase-change heat storage device, the electric heating device is used to obtain electricity from the power grid to heat water, and the phase-change heat storage device is used to store heat and provide heat through the stored heat; the method includes: The real-time electricity price, the dynamic threshold electricity price and the operating data of the phase change heat storage and supply system are obtained, and the obtained data are input into a pre-established Markov decision process model. The Markov decision process model includes a heat storage process sub-model and a heating process sub-model. The environmental state of the heat storage process sub-model is expressed as: ； in, for Moment The heat storage state of a phase change heat storage device, for Real-time electricity prices at the moment; The expression of the action space of the heat storage process sub-model is: ； in, for Enter the moment The heat exchange fluid flow rate of a phase change heat storage device, is the temperature of the heat exchange fluid entering the phase change heat storage device, which is the same as the outlet water temperature of the electric heating equipment; The reward function of the heat storage process sub-model is expressed as: ； in, for The electric power consumed by the electric heating equipment at all times, for The electrical power consumed by the pump at any moment, is the time interval; The expression of the environmental state S of the heating process sub-model is: ； in, represent Building heat load at any moment; The expression of the action space of the heating process sub-model is: ； in, represent Enter the moment The heat exchange fluid flow rate of each phase change heat storage device; The reward function of the heating process sub-model is expressed as: ； in, for The heat supply at all times, for Building heat load at all times; Determining the moment attribute of the current moment as a heat storage moment or a heat supply moment according to the real-time electricity price, the dynamic threshold electricity price, and the heat load, and selecting a target sub-model from the two sub-models of the Markov decision process model based on the determined moment attribute; Solving the target sub-model according to the DQN algorithm, and controlling the operation of the phase change heat storage system according to the solution result, wherein the solution result is the control parameter for the electric heating equipment and the phase change heat storage device; Solving the target sub-model according to the DQN algorithm includes: using The strategy selects and executes the action space to calculate the corresponding reward value and the new environment state; the conversion process data is stored in the experience pool and randomly sampled from the experience pool as the training data of the current evaluation network; the Q value is calculated according to the current target network, and the weight of the evaluation network is updated through gradient backpropagation and bias ; Repeat the above steps until the number of update rounds reaches the set value.

2. The method according to claim 1, characterized in that The dynamic threshold electricity price is determined based on the valley electricity price in the local time-of-use electricity price and the day-ahead real-time electricity price data. The expression of the dynamic threshold electricity price is: ； in, It is the valley price electricity in the local time-of-use electricity price. and are the mean and standard deviation of the day-ahead real-time electricity price published by the local power grid.

3. The method according to claim 1, characterized in that The heating expression of the electric heating equipment is: ； in, Indicates the electric power of electric heating equipment. It represents the thermal power of electric heating equipment, and EER stands for energy efficiency ratio; The heat storage and release expression of the phase change heat storage device is: When the heat transfer fluid inlet temperature Greater than the melting temperature of the phase change material When , the heat storage power of the phase change heat storage device is: ； When the heat transfer fluid inlet temperature Lower than the solidification temperature of the phase change material When , the heat release power of the phase change heat storage device is: ； in, and are the mass flow rates of heat storage and heat transfer fluids, and are the constant pressure specific heat of the heat storage and release heat transfer fluids, and are the heat storage and release thermal resistances of the phase change heat storage device, respectively.

4. The method according to claim 1, wherein The determining, based on the real-time electricity price, the dynamic threshold electricity price, and the heat load, that the time attribute of the current time is a heat storage time or a heat supply time includes: When the real-time electricity price is lower than the dynamic threshold electricity price, determining the current moment as a heat storage moment, the heat storage moment corresponding to a heat storage process sub-model; When the real-time electricity price is higher than the dynamic threshold electricity price and the heat load is greater than 0, the current moment is determined to be a heating moment, and the heating moment corresponds to a heating process sub-model.

5. The method according to claim 1, characterized in that The operation of the phase change heat storage system is subject to boundary conditions, and the boundary conditions include at least one of the following: system side boundary, grid side boundary condition, and user side boundary condition; wherein, the system side boundary is the electric heating coupled phase change heat storage system, the grid side boundary condition is the real-time electricity price published by the grid, and the user side boundary condition is the building heat load forecast.

6. An interactive device between a heating system and a power grid, characterized in that: Applied to a phase-change heat storage system, the phase-change heat storage system also includes an electric heating device and a phase-change heat storage device. The electric heating device is used to obtain electricity from the power grid to heat water, and the phase-change heat storage device is used to store heat and provide heat through the stored heat. The device includes: An acquisition unit acquires real-time electricity prices, dynamic threshold electricity prices, and operating data of the phase change heat storage and supply system, and inputs the acquired data into a pre-established Markov decision process model. The Markov decision process model includes a heat storage process sub-model and a heating process sub-model. The environmental state of the heat storage process sub-model is expressed as: ； in, for Moment The heat storage state of a phase change heat storage device, for Real-time electricity prices at the moment; The expression of the action space of the heat storage process sub-model is: ； in, for Enter the moment The heat exchange fluid flow rate of a phase change heat storage device, is the temperature of the heat exchange fluid entering the phase change heat storage device, which is the same as the outlet water temperature of the electric heating equipment; The reward function of the heat storage process sub-model is expressed as: ； in, for The electric power consumed by the electric heating equipment at all times, for The electrical power consumed by the pump at any moment, is the time interval; The expression of the environmental state S of the heating process sub-model is: ； in, represent Building heat load at any moment; The expression of the action space of the heating process sub-model is: ； in, represent Enter the moment The heat exchange fluid flow rate of each phase change heat storage device; The reward function of the heating process sub-model is expressed as: ； in, for The heat supply at all times, for Building heat load at all times; A selection unit: determining, according to the real-time electricity price, the dynamic threshold electricity price, and the heat load, whether the moment attribute of the current moment is a heat storage moment or a heat supply moment, and selecting a target sub-model from the two sub-models of the Markov decision process model based on the determined moment attribute; Control unit: solve the target sub-model according to the DQN algorithm, and control the operation of the phase change heat storage system according to the solution result, wherein the solution result is the control parameter for the electric heating equipment and the phase change heat storage device; the solution of the target sub-model according to the DQN algorithm includes: using The strategy selects and executes the action space to calculate the corresponding reward value and the new environment state; the conversion process data is stored in the experience pool and randomly sampled from the experience pool as the training data of the current evaluation network; the Q value is calculated according to the current target network, and the weight of the evaluation network is updated through gradient backpropagation and bias ; Repeat the above steps until the number of update rounds reaches the set value.

7. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor implements the steps of the method according to any one of claims 1 to 5 by running the executable instructions.

8. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images