Heat supply system and power grid interaction method, device and equipment and storage medium

Abstract

The invention belongs to the field of heat supply, and discloses a heat supply system and power grid interaction method, device and equipment and a storage medium, and the method comprises the steps: obtaining the real-time electricity price, the dynamic threshold electricity price and the operation data of a phase change heat storage system, inputting the obtained data into a pre-established Markov decision process model, the Markov decision process model comprises a heat storage process sub-model and a heat supply process sub-model; according to the real-time electricity price, the dynamic threshold electricity price and the thermal load, determining that the moment attribute of the current moment is a heat storage moment or a heat supply moment, and based on the determined moment attribute, selecting a target sub-model from two sub-models of a Markov decision process model; and solving the target sub-model according to a DQN algorithm, and controlling the operation of the phase change heat storage and supply system according to a solving result. Through the method, the building energy demand can be accurately met, the user energy cost is reduced, the real-time supply and demand balance of a power system is promoted, and the power supply pressure at the power peak is relieved.

Description

Technical Field

[0001] The present invention belongs to the field of heating, and particularly relates to an interaction method, device, equipment and storage medium between a heating system and a power grid. Background Art

[0002] Greatly increasing renewable energy generation and energy storage is an important path for the decarbonization of the power sector. Energy storage refers to the technology that realizes the cross-time and space transfer of energy and reshapes the power supply and demand curve between charging and discharging, including various types such as cold energy storage, heat energy storage, and electrical energy storage. Among them, phase change heat storage has the advantages of high energy storage density, constant temperature, controllable cost, and strong safety. Coupled with electric heating equipment for building heating, it can not only meet the building's energy demand but also realize interaction with the power grid.

[0003] In the related art, the regulation signals of the phase change heat storage and heating system are mostly peak-valley-flat electricity prices, the regulation objectives are mostly the lowest operating cost, and the regulation methods are mostly rule-based control. Specifically: during the night valley electricity period, the heat source operates at the maximum power to store heat energy; during the day for heating or peak electricity price period, the phase change device operates at a fixed flow rate (or adjusts the flow rate according to the supply and return water temperature difference) to replace the heat source for heating. This regulation method plays a role in peak shaving and valley filling to a certain extent, but there are still the following problems: 1) The peak-valley-flat electricity price gradually cannot reflect the real-time state of the power system with a large proportion of renewable energy penetration. Although the real-time electricity price makes real-time guidance for the regulation of the demand side, the phase change heat storage and heating system cannot accurately respond to the real-time electricity price; 2) The heat storage and heating process is not accurately matched with the dynamically changing building heat load; 3) The time-differentiated characteristics of the phase change heat storage device are ignored, that is, the heat storage / discharge power will change with the heat storage / discharge process.

[0004] Therefore, how to control the operation of the phase change heat storage and heating system under the coupling action of the multiple dynamic characteristics of the real-time demand of the power grid, the time-varying load of the building, and the differentiated power of the phase change heat storage has become an urgent technical problem to be solved. Summary of the Invention

[0005] In view of this, the present invention discloses an interaction method, device, equipment and storage medium between a heating system and a power grid, which can solve the deficiencies existing in the related art.

[0006] To achieve the above object, the technical solution disclosed by the present invention is as follows: According to the first aspect of the present invention, an interaction method between a heating system and a power grid is proposed, which is applied to a phase change heat storage and heating system. The phase change heat storage and heating system includes an electric heating device and a phase change heat storage device. The electric heating device is used to obtain electric energy from the power grid to heat water, and the phase change heat storage device is used for heat storage and heating by the stored heat; the method includes: Obtain the real-time electricity price, dynamic threshold electricity price, and the operation data of the phase change heat storage and heating system, and input the obtained data into a pre-established Markov decision process model. The Markov decision process model includes a heat storage process sub-model and a heating process sub-model, and the two sub-models correspond to different environmental states, action spaces, and reward functions; Determine whether the time attribute at the current moment is a heat storage moment or a heating moment according to the real-time electricity price, dynamic threshold electricity price, and heat load, and select a target sub-model from the two sub-models of the Markov decision process model based on the determined time attribute; Solve the target sub-model according to the DQN algorithm, and control the operation of the phase change heat storage and heating system according to the solution result. The solution result is the control parameters for the electric heating equipment and the phase change heat storage device.

[0007] According to the second aspect of the present invention, an interaction device between a heating system and a power grid is proposed, which is applied to a phase change heat storage and heating system. The phase change heat storage and heating system further includes electric heating equipment and a phase change heat storage device. The electric heating equipment is used to obtain electric energy from the power grid to heat water, and the phase change heat storage device is used for heat storage and heating with the stored hot water; the device includes: An acquisition unit: acquire the real-time electricity price, dynamic threshold electricity price, and the operation data of the phase change heat storage and heating system, and input the acquired data into a pre-established Markov decision process model. The Markov decision process model includes a heat storage process sub-model and a heating process sub-model, and the two sub-models correspond to different environmental states, action spaces, and reward functions; A selection unit: determine whether the time attribute at the current moment is a heat storage moment or a heating moment according to the real-time electricity price, dynamic threshold electricity price, and heat load, and select a target sub-model from the two sub-models of the Markov decision process model based on the determined time attribute; A control unit: solve the target sub-model according to the DQN algorithm, and control the operation of the phase change heat storage and heating system according to the solution result. The solution result is the control parameters for the electric heating equipment and the phase change heat storage device.

[0008] According to the third aspect of the present invention, an electronic device is proposed, including: A processor; A memory for storing processor-executable instructions; Wherein, the processor realizes the steps of the method as described in the first aspect by running the executable instructions.

[0009] According to the fourth aspect of the present invention, a computer-readable storage medium is proposed, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method as described in the first aspect are realized.

[0010] As can be seen from the above technical solutions, the operation method of the phase change heat storage and supply system disclosed in the present invention receives the day-ahead real-time electricity price of the power grid and the predicted heat load of the building, inputs the electricity price and load sequences into the regulation model of the interaction between the phase change heat storage and supply system and the power grid, and couples the dynamic characteristics of the phase change heat storage. The model outputs the hourly control parameters of the phase change heat storage device and the electric heating equipment on the current day, including the heat storage and release time, the outlet water temperature of the electric heating equipment, the temperature and flow rate of the heat exchange fluid entering the phase change heat storage device. By implementing the present invention, the phase change heat storage and supply system can accurately meet the building's energy demand, reduce the user's energy cost, promote the real-time supply-demand balance of the power system, and relieve the power supply pressure during the power peak. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is an architecture diagram of an interaction system between a heating system and a power grid provided by an exemplary embodiment; Figure 2 is a flowchart of an interaction method between a heating system and a power grid provided by an exemplary embodiment; Figure 3 is a schematic diagram of a DQN algorithm process provided by an exemplary embodiment; Figure 4 is a schematic structural diagram of a device provided by an exemplary embodiment; Figure 5 is a block diagram of an interaction device between a heating system and a power grid provided by an exemplary embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0012] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of the present invention. On the contrary, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of the present invention as detailed in the appended claims.

[0013] It should be noted that: in other embodiments, the steps of the corresponding methods are not necessarily executed in the order shown and described in the present invention. In some other embodiments, the steps included in the method may be more or less than those described in the present invention. In addition, a single step described in the present invention may be decomposed into multiple steps for description in other embodiments; and multiple steps described in the present invention may also be combined into a single step for description in other embodiments.

[0014] To further illustrate the present invention, the following embodiments are provided: Significantly increasing renewable energy generation and energy storage is an important path for the decarbonization of the power sector. Energy storage refers to the technology that realizes the cross-time and space transfer of energy during charging and discharging and reshapes the power supply and demand curve, including various types such as cold energy storage, heat energy storage, and electrical energy storage. Among them, phase change heat storage has the advantages of high energy storage density, constant temperature, controllable cost, and strong safety. Coupled with electric heating equipment for building heating, it can not only meet the building's energy needs but also achieve interaction with the power grid.

[0015] In related technologies, the regulation signals of the phase change heat storage and heating system are mostly peak-valley-flat electricity prices, the regulation targets are mostly the lowest operating cost, and the regulation methods are mostly rule-based control. Specifically: during the night valley electricity period, the heat source operates at the maximum power to store heat energy; during the day for heating or peak electricity price period, the phase change device operates at a fixed flow rate (or adjusts the flow rate according to the supply and return water temperature difference) to replace the heat source for heating. This regulation method plays a role in peak shaving and valley filling to a certain extent, but there are still the following problems: 1) The peak-valley-flat electricity price can gradually no longer reflect the real-time state of the power system with a large proportion of renewable energy penetration. Although the real-time electricity price makes real-time guidance for the regulation of the demand side, the phase change heat storage and heating system cannot accurately respond to the real-time electricity price; 2) The heat storage and heating process is not accurately matched with the dynamically changing building heat load; 3) The time-differentiated characteristics of the phase change heat storage device are ignored, that is, the charging / discharging power will change with the charging / discharging process.

[0016] To solve the deficiencies in related technologies, the present invention proposes an interaction method between a heating system and the power grid.

[0017] Figure 1 It is an architecture diagram of a phase change heat storage and heating system provided by an exemplary embodiment. As Figure 1 shown, the system includes an electric heating device 1, a first phase change heat storage device 2 and a second phase change heat storage device 2', a plate heat exchanger 3, a building 4 adopting floor radiant heating, and 5 valves (V1, V2, V3, V4, V5).

[0018] In the first case, valves V1, V3, V4, V5 are opened, V2 is closed, and the electric heating device 1 is turned on. At this time, the electric heating device 1 can heat the flowing water, and the heated flowing water enters the first phase change heat storage device 2 and the second phase change heat storage device 2' for heat storage.

[0019] In the second case, valve V1 is opened, V2, V3, V4, V5 are closed, and the electric heating device 1 is turned on. At this time, the electric heating device 1 directly heats the building water supply.

[0020] In the third case, valves V2, V3, V4 are opened, V1, V5 are closed, and the electric heating device 1 is turned off. At this time, the first phase change heat storage device 2 and the second phase change heat storage device 2' heat the building water supply.

[0021] In an actual situation, the electric heating device can be a heat pump, the maximum outlet water temperature can be 80 °C, and the filling material in the phase change device can be paraffin with a phase change temperature of 60 °C.

[0022] Figure 2 It is a flowchart of a method for the interaction between a heating system and a power grid provided by an exemplary embodiment. As Figure 2 shown, this method is applied to a phase change thermal storage heating system, which includes an electric heating device and a phase change thermal storage device. The electric heating device is used to obtain electric energy from the power grid to heat water, and the phase change thermal storage device is used for heat storage and heat supply through the stored heat. It may include the following steps: Step 201: Obtain the real-time electricity price, the dynamic threshold electricity price, and the operation data of the phase change thermal storage heating system, and input the obtained data into a pre-established Markov decision process model. The Markov decision process model includes a heat storage process sub-model and a heat supply process sub-model, and the two sub-models correspond to different environmental states, action spaces, and reward functions.

[0023] The operation data may include the predicted heat load data of the building and various parameters mentioned in the subsequent expressions, which will not be elaborated here.

[0024] The electric heating device model is shown as follows: ; Wherein, represents the electric power (kW) of the electric heating device; represents the heat power (kW) of the electric heating device; EER represents the energy efficiency ratio.

[0025] The phase change thermal storage device model is shown as follows: When the inlet temperature of the heat exchange fluid is greater than the melting temperature of the phase change material , the phase change thermal storage device stores heat, and the heat storage power is: ; When the inlet temperature of the heat exchange fluid is less than the solidification temperature of the phase change material , the phase change thermal storage device releases heat, and the heat release power is: ; Wherein, and are the mass flow rates (kg / s) of the heat storage and heat release heat exchange fluids; and are the constant pressure specific heats (kJ / (kg·K)) of the heat storage and heat release heat exchange fluids; and is the heat storage and release thermal resistance (K / kW) of the phase change heat storage device, which is a function related to the state of charge (SOC) and is obtained based on heat storage and release cycle experimental tests and polynomial fitting, as shown in the following formula: ; ; where, , , , , , , and are coefficients identified by experimental tests.

[0026] The expression for the dynamic threshold electricity price is: ; where, is the valley electricity price (yuan / kWh) in the local time-of-use electricity price; and are the mean and standard deviation (yuan / kWh) of the day-ahead real-time electricity price released by the local power grid, respectively.

[0027] Step 202: Determine the time attribute of the current moment as a heat storage moment or a heat supply moment according to the real-time electricity price, the dynamic threshold electricity price, and the heat load, and select a target sub-model from the two sub-models of the Markov decision process model based on the determined time attribute.

[0028] The determination of the time attribute of the current moment as a heat storage moment or a heat supply moment according to the real-time electricity price, the dynamic threshold electricity price, and the heat load includes: when the real-time electricity price is lower than the dynamic threshold electricity price, determining the current moment as a heat storage moment, and the heat storage moment corresponds to the heat storage process sub-model; when the real-time electricity price is higher than the dynamic threshold electricity price and the heat load is greater than 0, determining the current moment as a heat supply moment, and the heat supply moment corresponds to the heat supply process sub-model.

[0029] Part of the heat storage process sub-model: The expression for the environmental state is: ; where, represents the state of charge (kWh) of the th phase change heat storage device at the moment; is the real-time electricity price (yuan / kWh) at the

[0030] The expression for the action space is: ; Among them, represents the flow rate of the heat exchange fluid (m³ / h) entering the th phase change heat storage device at time represents the temperature of the heat exchange fluid entering the phase change heat storage device, that is, the outlet water temperature of the electric heating equipment (°C).

[0031] The reward function expression is designed as ; Among them, is the electric power consumed by the electric heating equipment (kW) at time is the electric power consumed by the water pump (kW) at time is the time interval, which is taken as 1 hour here. When the real-time electricity price is lower than the dynamic threshold electricity price, the reward is positive, and the lower the electricity price, the greater the power consumption and the greater the reward. The solution of the Markov decision process is more inclined to this situation.

[0032] Exothermic process sub-model part: The expression of the environmental state is: ; Among them, represents the building heat load (kW) at time

[0033] The action space expression is: ; Among them, represents the flow rate of the heat exchange fluid (m³ / h) entering the th phase change heat storage device at time

[0034] The reward function expression is: ; Among them, is the heat supply (kW) at time is the building heat load (kW) at time. When the reward function takes the maximum value, the supply-demand mismatch degree of the phase change heat storage and supply system is the smallest.

[0035] Step 203, solve the target sub-model according to the DQN algorithm, and control the operation of the phase change heat storage and supply system according to the solution result. The solution result is the control parameters for the electric heating equipment and the phase change heat storage device.

[0036] DQN (Deep Q-Network) is a reinforcement learning algorithm that combines deep learning and Q-Learning, used to solve decision-making problems in high-dimensional state spaces. In the present invention, DQN is used to train a phase change heat storage and supply agent to optimize heat storage and heat release strategies.

[0037] Figure 3 It is a flowchart for training and testing a phase change heat storage and supply DQN agent, including the following steps: Step 301: The number of episodes episode = 1, initialize the heat storage and supply agent, and divide 24 hours into 24 time steps; Step 302: At time step t = 0, the current evaluation network of the heat storage and supply agent uses the policy to select an action and execute it, and the environment returns a reward value and a new state ; ; Step 303: Store the transition process data ( ) in the experience pool, and randomly sample in the experience pool as the training data for the current evaluation network; Step 304: Calculate the Q value according to the current target network, and update the weights and biases of the evaluation network through gradient backpropagation; Step 305: Time step t += 1, repeat steps 302 - 304 until t = 24; Step 306: The number of episodes episode += 1, repeat steps 301 - 305, update the target network according to the set update frequency, so as to update the evaluation network until the number of episodes reaches the set value; Step 307: Save the evaluation network, and the heat storage and supply agent training is completed.

[0038] In the actual situation, the initial can be set to 1, the decay rate is 0.992, the learning rate is 0.0001, the discount factor is 0.995, the target network update frequency is 10, the size of the experience pool is 12 * 100, and the experience replay sampling size is 12.

[0039] The heating strategy of the present invention can be summarized as follows: on days with low load and high electricity prices, it is more inclined to adopt direct heating with phase change thermal energy storage; on days with low load and low electricity prices, reasonably adopt direct heating with heat pumps; on days with high load and low electricity prices, it is more inclined to reserve the phase change thermal energy storage for use when the electricity price is relatively high, and give priority to direct heating with heat pumps when the electricity price is close to zero; on days with high load and high electricity prices, in order to enable the phase change thermal energy storage device to play a key role in power grid interaction and meet the building load demand when the electricity price is extremely high, direct heating with heat pumps should be adopted when the electricity price is relatively low. Generally speaking, heat is stored at a low electricity price level every day to absorb excess power, and electric energy substitution is carried out at a high electricity price level every day to relieve the tension of power supply.

[0040] In this embodiment, by receiving the day-ahead real-time electricity price of the power grid and the predicted heat load of the building, the electricity price and load sequences are input into the regulation model of the power grid interaction of the phase change thermal energy storage heating system, and the dynamic characteristics of the phase change thermal energy storage are coupled. The model outputs the hourly control parameters of the phase change thermal energy storage device and the electric heating equipment on the current day, including the heat storage and release time, the outlet water temperature of the electric heating equipment, the temperature and flow rate of the heat exchange fluid entering the phase change thermal energy storage device. By implementing the present invention, the phase change thermal energy storage heating system can accurately meet the building energy demand, reduce the user's energy cost, promote the real-time supply-demand balance of the power system, and relieve the power supply pressure during peak power consumption.

[0041] In one implementation, the operation of the phase change thermal energy storage heating system has boundary conditions, and the boundary conditions include at least one of the following: system-side boundary, grid-side boundary condition, user-side boundary condition; wherein, the system-side boundary is an electric heating coupled with phase change thermal energy storage system, the grid-side boundary condition is the real-time electricity price released by the power grid, and the user-side boundary condition is the building heat load prediction.

[0042] The phase change thermal energy storage heating system can include a boundary condition preprocessing module, a phase change thermal energy storage heating agent module, and a phase change thermal energy storage heating system control module. The boundary condition preprocessing module is used to receive the real-time electricity price signal released by the power grid and the predicted heat load data of the building ahead of time, and unify them to the same time scale (such as 1h or 15min); secondly, calculate the dynamic threshold electricity price and output the time attribute per hour; the phase change thermal energy storage heating agent module is used to establish a phase change thermal energy storage heating system model and its Markov decision-making process model for power grid interaction, complete the training of the phase change thermal energy storage heating agent by using the DQN algorithm, input the real-time electricity price, time-varying load, and dynamic model of the phase change thermal energy storage device, and output system control parameters, including the heat storage and release time, the outlet water temperature of the electric heating equipment, and the flow rate of the heat exchange fluid of the phase change thermal energy storage device; the phase change thermal energy storage heating system control module is used to receive the system control parameters issued by the phase change thermal energy storage heating agent module, and control the temperature control device, system water pump and valve of the electric heating to make the control parameters reach the target value.

[0043] Figure 4It is a schematic structural diagram of a device provided by an exemplary embodiment. Please refer to Figure 4 , at the hardware level, the device includes a processor 402, an internal bus 404, a network interface 406, a memory 408, and a non-volatile memory 410. Of course, it may also include other hardware required for other functions. One or more embodiments of the present invention can be implemented in a software manner. For example, the processor 402 reads the corresponding computer program from the non-volatile memory 410 into the memory 408 and then runs it. Of course, in addition to the software implementation manner, one or more embodiments of the present invention do not exclude other implementation manners, such as logical devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logical unit, and can also be hardware or logical devices.

[0044] Please refer to Figure 5 , an interaction device between a heating system and a power grid can be applied to a device as shown in Figure 5 to implement the technical solution of the present invention. It is applied to a phase change heat storage heating system, and the phase change heat storage heating system further includes an electric heating device and a phase change heat storage device. The electric heating device is used to obtain electric energy from the power grid to heat water, and the phase change heat storage device is used to store heat and supply heat through the stored heat; the device includes: An acquisition unit 501, configured to acquire real-time electricity prices, dynamic threshold electricity prices, and operation data of the phase change heat storage heating system, and input the acquired data into a pre-established Markov decision process model. The Markov decision process model includes a heat storage process sub-model and a heat supply process sub-model, and the two sub-models correspond to different environmental states, action spaces, and reward functions; A selection unit 502, configured to determine the moment attribute at the current moment as a heat storage moment or a heat supply moment according to the real-time electricity price, the dynamic threshold electricity price, and the heat load, and select a target sub-model from the two sub-models of the Markov decision process model based on the determined moment attribute; A control unit 503, configured to solve the target sub-model according to the DQN algorithm, and control the operation of the phase change heat storage heating system according to the solution result. The solution result is control parameters for the electric heating device and the phase change heat storage device.

[0045] Optionally, the dynamic threshold electricity price is determined based on the valley electricity price in the local time-of-use electricity price and the day-ahead real-time electricity price data. The expression of the dynamic threshold electricity price is: ; Wherein, is the valley electricity price in the local time-of-use electricity price, and are respectively the mean and standard deviation of the day-ahead real-time electricity price released by the local power grid.

[0046] Optionally, the expression of the environmental state of the heat storage process sub-model is: ; where, is the heat storage state of the th phase change heat storage device at time is the real-time electricity price at time The expression of the action space of the heat storage process sub-model is: ; where, is the flow rate of the heat exchange fluid entering the th phase change heat storage device at time is the temperature of the heat exchange fluid entering the phase change heat storage device, and this temperature is the same as the outlet water temperature of the electric heating equipment; The expression of the reward function of the heat storage process sub-model is: ; where, is the electric power consumed by the electric heating equipment at time is the electric power consumed by the water pump at time is the time interval; The expression of the environmental state S of the heat supply process sub-model is: ; where, represents the building heat load at time The expression of the action space of the heat supply process sub-model is: ; where, represents the flow rate of the heat exchange fluid entering the th phase change heat storage device at time represents the flow rate of the fluid supplied by the electric heating equipment at time The expression of the reward function of the heat supply process sub-model is: ; where, is the heat supply at time is the building heat load at time

[0047] Optionally, the heating expression of the electric heating device is: ; wherein, represents the electric power of the electric heating device, represents the heat power of the electric heating device, and EER represents the energy efficiency ratio.

[0048] The heat storage and heat release expression of the phase change heat storage device is: When the inlet temperature of the heat exchange fluid is greater than the melting temperature of the phase change material , the heat storage power of the phase change heat storage device is: ; When the inlet temperature of the heat exchange fluid is less than the solidification temperature of the phase change material , the heat release power of the phase change heat storage device is: ; wherein, and are respectively the mass flow rates of the heat storage and heat release heat exchange fluids, and are respectively the specific heat capacities at constant pressure of the heat storage and heat release heat exchange fluids, and are respectively the heat storage and heat release thermal resistances of the phase change heat storage device.

[0049] Optionally, the selection unit 502 is specifically configured to: In the case that the real-time electricity price is lower than the dynamic threshold electricity price, determine the current moment as the heat storage moment, and the heat storage moment corresponds to the heat storage process sub-model; In the case that the real-time electricity price is higher than the dynamic threshold electricity price and the heat load is greater than 0, determine the current moment as the heat supply moment, and the heat supply moment corresponds to the heat supply process sub-model.

[0050] Optionally, the control unit 503 is specifically configured to: Adopt strategy to select and execute the action space to calculate the corresponding reward value and the new environmental state; Store the conversion process data in the experience pool, and randomly sample in the experience pool as the training data of the current evaluation network; Calculate the Q value according to the current target network, and update the weights and biases of the evaluation network through gradient backpropagation; Repeat the above operations until the number of updated rounds reaches the set value.

[0051] Optionally, the operation of the phase change heat storage and supply system has boundary conditions, which include at least one of the following: system side boundary, grid side boundary condition, and user side boundary condition; wherein, the system side boundary is an electric heating coupled phase change heat storage system, the grid side boundary condition is the real-time electricity price issued by the grid, and the user side boundary condition is the building heat load prediction.

[0052] The systems, devices, modules or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, and the specific form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver device, a game console, a tablet computer, a wearable device, or a combination of any several of these devices.

[0053] In a typical configuration, a computer includes one or more processors (CPUs), an input / output interface, a network interface, and a memory.

[0054] The memory may include non-permanent memory in the computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0055] The computer-readable medium includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of the computer's storage medium include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassette tapes, disk storage, quantum memory, graphene-based storage media, or other magnetic storage devices, or any other non-transmission media that can be used to store information accessible by a computing device. As defined herein, the computer-readable medium does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0056] Regarding the computer-readable medium (or computer-readable storage medium) described above or in any other form, computer instructions can be stored thereon, and when the instructions are executed by a processor, one or more of the above-described embodiments are implemented, thereby implementing the technical solution of the present invention.

[0057] The present invention also provides a computer program, which when executed by a processor implements one or more of the above-described embodiments, thereby implementing the technical solution of the present invention. Among them, the computer program can be specifically recorded on the computer-readable medium described above or in any other form, and the present invention does not limit this.

[0058] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such a process, method, commodity or device. Without further limitation, the element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, commodity or device including the said element.

[0059] The above describes specific embodiments of the present invention. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be executed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the specific order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0060] The terms used in one or more embodiments of the present invention are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of the present invention. The singular forms "a", "the" and "said" used in one or more embodiments of the present invention and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0061] It should be understood that although the terms first, second, third, etc. may be used in one or more embodiments of the present invention to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of one or more embodiments of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to a determination".

[0062] The above description is only a preferred embodiment of one or more embodiments of the present invention and is not intended to limit one or more embodiments of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of one or more embodiments of the present invention shall be included within the scope of protection of one or more embodiments of the present invention.

Claims

1. An interaction method between a heating system and a power grid, characterized in that, Applied to a phase change heat storage and supply system, the phase change heat storage and supply system includes an electric heating device and a phase change heat storage device. The electric heating device is used to obtain electric energy from the power grid to heat water, and the phase change heat storage device is used for heat storage and heat supply through the stored heat; the method includes: Obtain the real-time electricity price, dynamic threshold electricity price and the operation data of the phase change heat storage and supply system, and input the obtained data into a pre-established Markov decision process model. The Markov decision process model includes a heat storage process sub-model and a heat supply process sub-model. The two sub-models correspond to different environmental states, action spaces and reward functions; Determine the time attribute of the current moment as a heat storage moment or a heat supply moment according to the real-time electricity price, dynamic threshold electricity price and heat load, and select a target sub-model from the two sub-models of the Markov decision process model based on the determined time attribute; Solve the target sub-model according to the DQN algorithm, and control the operation of the phase change heat storage and supply system according to the solution result. The solution result is the control parameters for the electric heating device and the phase change heat storage device.

2. The method according to claim 1, wherein The dynamic threshold electricity price is determined based on the valley electricity price in the local time-of-use electricity price and the real-time electricity price data of the day before. The expression of the dynamic threshold electricity price is: ； Among them, is the valley price electricity in the local time-of-use electricity price, and are the mean and standard deviation of the day-ahead real-time electricity price released by the local power grid, respectively.

3. The method according to claim 1, wherein The expression of the environmental state of the heat storage process sub-model is: ； Among them, is the heat storage state of the th phase change heat storage device at time is the real-time electricity price at time The expression of the action space of the heat storage process sub-model is: ； Among them, is the flow rate of the heat exchange fluid entering the th phase change heat storage device at a certain moment, is the temperature of the heat exchange fluid entering the phase change heat storage device, and this temperature is the same as the outlet water temperature of the electric heating equipment; The expression of the reward function of the heat storage process sub-model is: ； Among them, is the electric power consumed by the electric heating equipment at time is the electric power consumed by the water pump at time is the time interval; The expression of the environmental state S of the heat supply process sub-model is: ； Among them, represents the building heat load at a certain moment; The expression of the action space of the heat supply process sub-model is: ； Among them, represents the heat exchange fluid flow rate entering the th phase change heat storage device at a certain moment; The expression of the reward function of the heat supply process sub-model is: ； Among them, is the heat supply at a certain moment, and is the building heat load at a certain moment.

4. The method according to claim 1, wherein The heating expression of the electric heating device is: ； Among them, represents the electric power of the electric heating device, represents the heat power of the electric heating device, and EER represents the energy efficiency ratio; The heat storage and release expression of the phase change heat storage device is: When the inlet temperature of the heat exchange fluid is greater than the melting temperature of the phase change material the heat storage power of the phase change heat storage device is: ； When the inlet temperature of the heat exchange fluid is lower than the solidification temperature of the phase change material , the heat release power of the phase change heat storage device is as follows: ； Wherein, and are respectively the mass flow rates of the heat storage and heat release heat exchange fluids, and are respectively the specific heat capacities at constant pressure of the heat storage and heat release heat exchange fluids, and are respectively the heat storage and heat release thermal resistances of the phase change heat storage device.

5. The method according to claim 1, characterized in that The determining the time attribute of the current moment as a heat storage moment or a heat supply moment according to the real-time electricity price, dynamic threshold electricity price and heat load includes: When the real-time electricity price is lower than the dynamic threshold electricity price, determine the current moment as the heat storage moment, and the heat storage moment corresponds to the heat storage process sub-model; When the real-time electricity price is higher than the dynamic threshold electricity price and the heat load is greater than 0, determine the current moment as the heat supply moment, and the heat supply moment corresponds to the heat supply process sub-model.

6. The method according to claim 1, characterized in that, The solving the target sub-model according to the DQN algorithm includes: Adopt Select and execute the action space using a strategy to calculate the corresponding reward value and the new environmental state; Store the conversion process data in the experience pool, and randomly sample in the experience pool as the training data of the current evaluation network; Calculate the Q value according to the current target network and update the weights of the evaluation network through gradient backpropagation and biases ; Repeat the above operations until the number of update rounds reaches the set value.

7. The method according to claim 1, wherein The operation of the phase change heat storage and supply system has boundary conditions, and the boundary conditions include at least one of the following: system-side boundary, grid-side boundary condition, user-side boundary condition; wherein, the system-side boundary is an electric heating coupled phase change heat storage system, the grid-side boundary condition is the real-time electricity price released by the grid, and the user-side boundary condition is the building heat load prediction.

8. An interactive device between a heating system and a power grid, characterized in that, Applied to a phase change heat storage and heating system, the phase change heat storage and heating system further includes an electric heating device and a phase change heat storage device. The electric heating device is used to obtain electric energy from the power grid to heat water, and the phase change heat storage device is used for heat storage and heating by the stored heat; the device includes: An acquisition unit: acquires real-time electricity price, dynamic threshold electricity price and operation data of the phase change heat storage and heating system, and inputs the acquired data into a pre-established Markov decision process model. The Markov decision process model includes a heat storage process sub-model and a heating process sub-model, and the two sub-models correspond to different environmental states, action spaces and reward functions; A selection unit: determines the time attribute of the current moment as a heat storage moment or a heating moment according to the real-time electricity price, dynamic threshold electricity price and heat load, and selects a target sub-model from the two sub-models of the Markov decision process model based on the determined time attribute; A control unit: solves the target sub-model according to the DQN algorithm, and controls the operation of the phase change heat storage and heating system according to the solution result. The solution result is control parameters for the electric heating device and the phase change heat storage device.

9. An electronic device, characterized in that, Includes: A processor; A memory for storing instructions executable by the processor; Wherein, the processor realizes the steps of the method according to any one of claims 1-7 by running the executable instructions.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that, When the instruction is executed by the processor, the steps of the method according to any one of claims 1-7 are realized.

Images