Frequency modulation-peak regulation collaborative optimization method and system for fused salt heat storage system

Abstract

The invention relates to the technical field of frequency modulation and peak regulation of a power grid, in particular to a frequency modulation-peak regulation collaborative optimization method and system for a fused salt heat storage system. The optimal slow strategy of the fused salt heat storage system based on the long-time scale peak regulation process is determined, and a reference output plan of a generator set is determined through the optimal slow strategy in the peak regulation process; determining the reference frequency of the power grid at each moment through the reference output plan; according to the reference frequency, the real-time state characteristic data and the actual operation action characteristic data of the power grid at each moment, determining an optimal fast strategy of the fused salt heat storage system based on a short time scale frequency modulation process through a reinforcement learning algorithm; and performing frequency modulation-peak regulation collaborative optimization of the fused salt heat storage system through the optimal slow strategy and the optimal fast strategy. According to the invention, the frequency modulation response speed is improved, the frequency deviation is reduced, the cost is saved, and the energy storage life is prolonged.

Description

Technical Field

[0001] The present invention relates to the technical field of power grid frequency regulation and peak regulation, and in particular to a frequency regulation and peak regulation collaborative optimization method and system for a molten salt heat storage system. Background Art

[0002] Molten salt thermal energy storage (TES) systems are widely used in renewable energy generation, particularly solar thermal and wind power. They primarily store thermal energy to balance grid loads, enabling frequency regulation, peak load regulation, and energy management. To more effectively utilize molten salt TES and enhance their role in power dispatch, a coordinated frequency and peak load optimization approach is needed.

[0003] To ensure grid stability and reliability and address issues such as load and power generation fluctuations, unstable renewable energy generation, and frequency fluctuations, frequency modulation and peak shaving are used to maintain system stability. Existing technologies typically implement frequency and peak shaving using reinforcement learning algorithms with a single timescale strategy. However, this single timescale strategy cannot handle both second-level frequency modulation and hour-level peak shaving. This increases the computational complexity and time consumption during training, making it difficult to converge to the optimal solution when the action space dimension in reinforcement learning is too large. This problem can be alleviated by simplifying the action space and adopting hierarchical learning methods. Summary of the Invention

[0004] The present invention provides a frequency regulation and peak regulation collaborative optimization method and system for a molten salt heat storage system, which are used to solve the problems of computational complexity and training time consumption faced in the existing power grid frequency regulation and peak regulation.

[0005] The purpose of the present invention can be achieved through the following technical solutions: A first aspect of the present invention is to provide a frequency modulation and peak regulation coordinated optimization method for a molten salt heat storage system, comprising: Acquire real-time state characteristic data in the molten salt heat storage system and relevant state characteristic data that affect the efficiency of the molten salt heat storage system; then acquire actual operation action characteristic data and control management action characteristic data in the molten salt heat storage system; Based on the relevant state characteristic data and control management action characteristic data, a reinforcement learning algorithm is used to determine the optimal slow strategy for the molten salt thermal storage system during the long-term peak-shaving process. The optimal slow strategy during the peak-shaving process is used to determine the benchmark output plan of the generator set; the benchmark output plan is used to determine the benchmark frequency of the power grid at each moment; Based on the grid's baseline frequency, real-time status data, and actual operational characteristics at each moment, a reinforcement learning algorithm is used to determine the optimal fast strategy for the molten salt thermal storage system's frequency regulation process over short timescales. The frequency regulation and peak regulation of the molten salt heat storage system are coordinated and optimized through the optimal slow strategy and the optimal fast strategy.

[0006] Furthermore, the optimal slow strategy of the molten salt thermal storage system based on the long-time peak regulation process is determined by a reinforcement learning algorithm based on the relevant state feature data and the control management action feature data, including: The first state space of long time scale is constructed through relevant state feature data, and the first action space of long time scale is constructed through control and management action feature data; through the first state space and first action space of long time scale, the optimal slow strategy of the molten salt heat storage system based on the long time scale peak regulation process is determined through a policy-based reinforcement learning algorithm.

[0007] Furthermore, the method of determining the optimal slow strategy of the molten salt thermal storage system in the long-time-scale peak regulation process by using a policy-based reinforcement learning algorithm through the long-time-scale first state space and the first action space includes: The objective function of the slow strategy is determined based on the first state space and the first action space of the long time scale. The objective function of the slow strategy is specifically expressed as:

[0008] Where, is the discount factor, Indicates the The discount factor in time steps, Indicates the The reward value of each time step, represents the total number of time steps in the interaction between the environment and the agent, represents the expected operation, that is, the expected value of making decisions according to the strategy π, represents the long-term cumulative reward of all time steps; The optimal slow strategy is obtained by the long-term cumulative reward of all time steps.

[0009] Furthermore, the optimal fast strategy for the molten salt thermal storage system based on the short-time-scale frequency modulation process is determined by a reinforcement learning algorithm based on the reference frequency, real-time state characteristic data, and actual operation action characteristic data of the power grid at each moment, including: At preset time intervals To continuously obtain the preset time before the current moment Real-time status characteristic data of the internal molten salt thermal storage system at all times; Obtain the maximum output power of the generator or energy storage device; A frequency stability constraint is constructed using the reference frequency of the power grid at each moment, and a penalty value is obtained when the frequency deviates from the target frequency. The penalty value when the frequency deviates from the target frequency is recorded as the penalty value of the first constraint. A generator and energy storage capacity constraint is constructed using the maximum output power of the generator or energy storage device, and a penalty value is obtained when the maximum output power limit of the generator or energy storage device is exceeded. The penalty value when the maximum output power limit of the generator or energy storage device is exceeded is recorded as the penalty value of the second constraint. Construct a second state space with a short time scale through real-time state feature data, and construct a second action space with a short time scale through actual operation action feature data; The objective function of the fast strategy is constructed through the penalty values of several constraints, the second state space and the second action space of the short time scale; the optimal fast strategy of the molten salt heat storage system based on the short time scale frequency modulation process is determined through the objective function of the fast strategy.

[0010] Furthermore, the frequency stability constraint is constructed by using the reference frequency of the power grid at each moment, and a penalty value is obtained when the frequency deviates from the target frequency, including:

[0011] Where, Indicates the frequency of the power grid at each moment, Indicates the reference frequency of the power grid at each moment, Indicates the preset frequency error, represents the frequency violation penalty coefficient, represents the maximum value function, Indicates the absolute value symbol, Indicates the penalty value when the frequency deviates from the target frequency; Among them, the frequency violation penalty coefficient is determined by hyperparameter tuning.

[0012] Furthermore, the generator and energy storage capacity constraints are constructed using the maximum output power of the generator or energy storage device, and a penalty value is obtained when the maximum output power limit of the generator or energy storage device is exceeded, including:

[0013] Where, Indicates the output power of the generator or energy storage device at each moment, Indicates the maximum output power of the generator or energy storage device; represents the maximum value function, represents the output power violation penalty coefficient, Indicates the penalty value when the maximum output power limit of the generator or energy storage device is exceeded; The penalty value for exceeding the maximum output power limit of the generator or energy storage device is determined by hyperparameter tuning.

[0014] Furthermore, the objective function of the fast strategy is constructed through the penalty values of several constraints, the second state space and the second action space of the short time scale; and the optimal fast strategy of the molten salt thermal storage system based on the short time scale frequency modulation process is determined through the objective function of the fast strategy, including:

[0015] Indicates the The discount factor in the update step, Indicates the The reward value of the update step, Represents the total number of update steps during the interaction between the environment and the agent; Indicates the The penalty value of the constraint, Indicates the The penalty intensity of a constraint, Indicates the expected operation, that is, according to the strategy The expected value of making a decision, represents the long-term cumulative reward of all update steps, represents the number of all constraints, Indicates the adjustment of the initial update time interval, that is, the duration of an update step is equal to the adjustment of the initial update time interval; the penalty intensity of each constraint is determined by hyperparameter tuning;

[0016] Where, Indicates the The real-time state feature is at the current moment The value of a reference time, Indicates the The mean value of all reference moments of the real-time state feature data at the current moment, Indicates the number of reference moments, Indicates the number of all real-time status features, Indicates the absolute value symbol, represents the linear normalization function, Indicates the preset initial update time interval; Among them, the preset value before the current moment A moment is used as the reference moment of the current moment; The optimal fast strategy is obtained through the long-term cumulative rewards of all update steps.

[0017] A second aspect of the present invention is to provide a frequency modulation and peak regulation coordinated optimization system for a molten salt heat storage system, comprising: Data acquisition module: used to obtain real-time status characteristic data in the molten salt heat storage system and related status characteristic data that affect the efficiency of the molten salt heat storage system; and then obtain actual operation action characteristic data and control management action characteristic data in the molten salt heat storage system; Long-time scale analysis module: used to determine the optimal slow strategy for the molten salt thermal storage system during long-time peak-shaving based on relevant state feature data and control management action feature data through a reinforcement learning algorithm. The module also determines the benchmark output plan of the generator set based on the optimal slow strategy during the peak-shaving process. The benchmark output plan is used to determine the benchmark frequency of the power grid at each moment. Short-time scale analysis module: This module is used to determine the optimal fast strategy for the molten salt thermal storage system in the short-time scale frequency regulation process based on the grid's reference frequency, real-time status characteristic data, and actual operation action characteristic data at each moment through a reinforcement learning algorithm. Collaborative optimization module: used to perform collaborative optimization of frequency regulation and peak regulation of molten salt thermal storage system through optimal slow strategy and optimal fast strategy.

[0018] The third aspect of the present invention is to provide an electronic device, comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the computer program, the frequency modulation and peak modulation coordinated optimization method of the molten salt heat storage system is implemented.

[0019] A fourth aspect of the present invention is to provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the frequency regulation and peak regulation collaborative optimization method of the molten salt heat storage system is implemented.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows: based on relevant state characteristic data and control management action characteristic data, the optimal slow strategy of the molten salt heat storage system in the long-term peak-shaving process is determined through a reinforcement learning algorithm, and the benchmark output plan of the generator set is determined through the optimal slow strategy in the peak-shaving process; the benchmark frequency of the power grid at each moment is determined through the benchmark output plan, thereby improving the accuracy of long-term stability strategy formulation; based on the benchmark frequency of the power grid at each moment, real-time state characteristic data and actual operation action characteristic data, the optimal fast strategy of the molten salt heat storage system in the short-term frequency regulation process is determined through a reinforcement learning algorithm, and the accuracy of transient strategy formulation of the molten salt heat storage system is improved through the constraint of the long-term stability strategy, and the amount of calculation is reduced based on the long-term stability strategy; the frequency regulation-peak regulation of the molten salt heat storage system is coordinated optimized through the optimal slow strategy and the optimal fast strategy, thereby improving the frequency regulation response speed, reducing the frequency deviation, saving costs and extending the energy storage life. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 The present invention provides a schematic flow chart of the steps of a frequency modulation and peak regulation collaborative optimization method for a molten salt heat storage system; Figure 2 The present invention provides a module flow diagram of a frequency modulation and peak regulation collaborative optimization system for a molten salt heat storage system. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] In response to the problems existing in the background technology, a frequency regulation and peak regulation collaborative optimization method and system for the molten salt heat storage system were studied and designed, which has important practical significance.

[0026] like Figure 1 As shown, the first aspect of the present invention is to provide a frequency modulation and peak regulation coordinated optimization method for a molten salt heat storage system, comprising the following steps: Step S001: Collect various state characteristic data and various action characteristic data in the molten salt heat storage system.

[0027] It should be noted that in order to solve the problems of computational complexity and training time consumption faced by conventional technologies where a single time scale strategy cannot simultaneously handle second-level frequency modulation and hour-level peak modulation, resulting in too many and too large dimensions, the present invention classifies the dimensions for hierarchical learning, thereby simplifying the state space and action space in the learning process.

[0028] Specifically, real-time state characteristic data and relevant state characteristic data affecting the efficiency of the molten salt heat storage system are obtained; and actual operation action characteristic data and control management action characteristic data in the molten salt heat storage system are obtained; Among them, real-time status characteristic data includes grid frequency, load demand, generator output (generator power generation and power), and the state of the energy storage system (SOC, State of Charge); related status characteristic data includes grid frequency, load demand, generator output, the state of the energy storage system, initial investment cost, operation and maintenance cost, energy conversion efficiency and cost, and weather conditions; actual operation action characteristic data includes generator output power and the charge and discharge power of the energy storage system; control and management action characteristic data includes the start and stop status of the unit, the unit's baseline charge and discharge power, and the SOC value.

[0029] Step S002: Based on the relevant state characteristic data and the control management action characteristic data, the optimal slow strategy of the molten salt thermal storage system in the long-term peak-shaving process is determined through a reinforcement learning algorithm. The benchmark output plan of the generator set is determined through the optimal slow strategy in the peak-shaving process; the benchmark frequency of the power grid at each moment is determined through the benchmark output plan.

[0030] It should be noted that in order to perform hierarchical learning for all dimensions, two policy-based reinforcement learning algorithms are used to analyze the fast strategy and slow strategy respectively. First, the start and stop status and benchmark output plan of all generator sets are determined through economic optimization for the slow strategy, and the fast strategy is constrained by the benchmark output plan to complete the frequency-peak regulation coordinated optimization of the molten salt heat storage system during transient changes.

[0031] Specifically, based on relevant state feature data and control management action feature data, a reinforcement learning algorithm is used to determine the optimal slow strategy for the molten salt thermal storage system during long-term peak regulation. Among them, the first state space of the long time scale is constructed by using the relevant state characteristic data, and the first action space of the long time scale is constructed by using the control management action characteristic data; based on the first state space and the first action space of the long time scale, the CPO algorithm (strategy reinforcement learning algorithm) is used to determine the optimal slow strategy for the molten salt heat storage system based on the long time scale peak regulation process; among them, the CPO (Conservative Policy Optimization) algorithm is a well-known technology and will not be described in detail here.

[0032] It should be noted that in order to make a small-scale adjustment to the grid frequency, load demand, generator output, energy storage system status, etc., a long-term and stable slow strategy can be formulated to determine some benchmark values, and analysis can be carried out based on the benchmark values.

[0033] Specifically, the objective function of the slow strategy is determined based on the first state space and the first action space of the long time scale. The objective function of the slow strategy is specifically expressed as:

[0034] Where, is the discount factor, Indicates the The discount factor in time steps, Indicates the The reward value of each time step, represents the total number of time steps in the interaction between the environment and the agent, represents the expected operation, that is, the expected value of making decisions according to the strategy π, represents the long-term cumulative reward of all time steps; The optimal slow strategy is obtained by the long-term cumulative reward of all time steps.

[0035] Among them, the strategy of each step in the interaction between the environment and the intelligent agent is also adjusted. The strategy adjustment process is specifically expressed as follows:

[0036] Where, Indicates the new strategy, represents the old policy, represents the KL divergence (Kullback-Leibler divergence) between the new and old strategies, Indicates a preset control threshold; wherein, in this embodiment, the preset control threshold , wherein in this embodiment, the preset control threshold There are no specific limitations on this, and the implementation can determine this based on specific circumstances. The process of obtaining the discount factor and the reward value for each time step is well-known technology and will not be described in detail here. In this embodiment, the time interval corresponding to one time step is one hour. There are no specific limitations on the time interval corresponding to a time step in this embodiment, and the implementation can determine this based on specific circumstances.

[0037] Among them, When the KL divergence between the new and old strategies is less than or equal to the preset control threshold When the KL divergence between the old and new strategies is greater than the preset control threshold The process of obtaining the KL divergence between the new and old strategies is a well-known technique and will not be described in detail here.

[0038] The optimal slow strategy is obtained through the long-term cumulative rewards of all time steps; the benchmark output plan of the generator set is determined through the optimal slow strategy; and the benchmark frequency of the power grid at each moment is determined through the benchmark output plan.

[0039] Step S003: According to the reference frequency, real-time state characteristic data and actual operation action characteristic data of the power grid at each moment, the optimal fast strategy of the molten salt thermal storage system based on the short-time scale frequency regulation process is determined through the reinforcement learning algorithm.

[0040] It should be noted that in the process of policy-based reinforcement learning (RL), constraints are required to ensure that the agent can make decisions within an appropriate range and achieve effective and stable learning results. The introduction of constraints helps solve many problems, especially in practical applications, ensuring that the model not only learns the optimal policy, but also ensures that the policy meets certain requirements such as safety, executability, and efficiency.

[0041] Specifically, based on the reference frequency, real-time status characteristic data and actual operation action characteristic data of the power grid at each moment, the optimal fast strategy of the molten salt heat storage system based on the short-time-scale frequency modulation process is determined through a reinforcement learning algorithm; wherein, in this embodiment, the determination process of the fast strategy is determined by the DPG algorithm; wherein, the DPG (Deep Deterministic Policy Gradient) algorithm is a well-known technology and will not be described in detail here.

[0042] Among them, at a preset time interval To continuously obtain the preset time before the current moment Real-time state characteristic data at all times in the molten salt heat storage system; wherein, in this embodiment, the preset time interval Seconds, preset duration Hours, wherein in this embodiment, the preset time interval and preset duration There are no specific restrictions, and implementers can decide based on specific circumstances.

[0043] Obtain the maximum output power of the generator or energy storage device; A frequency stability constraint is constructed based on the reference frequency of the power grid at each moment, and a penalty value is obtained when the frequency deviates from the target frequency. The frequency stability constraint is specifically expressed as:

[0044] Where, Indicates the frequency of the power grid at each moment, Indicates the reference frequency of the power grid at each moment, Indicates the preset frequency error, represents the frequency violation penalty coefficient, represents the maximum value function, Indicates the absolute value symbol, Indicates the penalty value when the frequency deviates from the target frequency; The frequency violation penalty coefficient is determined by hyperparameter tuning. The penalty value when the frequency deviates from the target frequency is recorded as the penalty value of the first constraint.

[0045] The generator and energy storage capacity constraints are constructed based on the maximum output power of the generator or energy storage device, and the penalty value when the maximum output power limit of the generator or energy storage device is exceeded is obtained; the generator and energy storage capacity constraints are specifically expressed as:

[0046] Where, Indicates the output power of the generator or energy storage device at each moment, Indicates the maximum output power of the generator or energy storage device; represents the maximum value function, represents the output power violation penalty coefficient, Indicates the penalty value when the maximum output power limit of the generator or energy storage device is exceeded; The penalty value when the maximum output power limit of the generator or energy storage device is exceeded is determined by hyperparameter tuning and recorded as the penalty value of the second constraint.

[0047] Construct a second state space with a short time scale through real-time state feature data, and construct a second action space with a short time scale through actual operation action feature data; The objective function of the fast strategy is constructed by using several constraint penalty values, the second state space and the second action space of short time scale. The objective function of the fast strategy is specifically expressed as:

[0048] Indicates the The discount factor in the update step, Indicates the The reward value of the update step, Represents the total number of update steps during the interaction between the environment and the agent; Indicates the The penalty value of the constraint, Indicates the The penalty intensity of a constraint, Indicates the expected operation, that is, according to the strategy The expected value of making a decision, represents the long-term cumulative reward of all update steps, represents the number of all constraints, Represents the adjustment of the initial update time interval, that is, the duration of an update step is equal to the adjustment of the initial update time interval; wherein, the penalty intensity of each constraint is determined by hyperparameter tuning.

[0049] It should be noted that, in the process of adjusting the output power of the generator and the charging and discharging power of the energy storage system, when the change fluctuation of all real-time state characteristic data of the second state space of the fast strategy is smaller, the interval does not need to be too short in the process of power adjustment and update by the fast strategy, otherwise it will only increase the amount of calculation; and when the change fluctuation of all real-time state characteristic data of the second state space of the fast strategy is greater, the interval does not need to be too long in the process of power adjustment and update by the fast strategy, otherwise the system will become more unstable due to untimely adjustment. Therefore, by adjusting the update time in the process of electric power adjustment of the fast strategy, it can more perfectly meet the adjusted update time interval.

[0050] Specifically, according to the fluctuation of all real-time status characteristic data, the preset initial update time interval is adjusted to obtain an adjusted initial update time interval; the adjusted initial update time interval is specifically expressed by the formula:

[0051] Where, Indicates the The real-time state feature is at the current moment The value of a reference time, Indicates the The mean value of all reference moments of the real-time state feature data at the current moment, Indicates the number of reference moments, Indicates the number of all real-time status features, Indicates the absolute value symbol, represents the linear normalization function, Indicates adjusting the initial update time interval. Indicates the preset initial update time interval; wherein, in this embodiment, the preset initial update time interval seconds, wherein in this embodiment, the preset initial update time interval There is no specific limitation and implementers can decide based on specific circumstances.

[0052] Among them, the preset value before the current moment A moment is used as the reference moment of the current moment; wherein, in this embodiment, the preset value , in this embodiment, the preset value There is no specific limitation and implementers can decide based on specific circumstances.

[0053] The optimal fast strategy is obtained through the long-term cumulative rewards of all update steps.

[0054] Step S004: Coordinated optimization of frequency regulation and peak regulation of the molten salt thermal storage system is performed through the optimal slow strategy and the optimal fast strategy.

[0055] The frequency regulation and peak regulation of the molten salt heat storage system are coordinated and optimized through the optimal slow strategy and the optimal fast strategy, thereby reducing the computational complexity and shortening the training time.

[0056] Through this collaborative optimization, the frequency regulation response speed of the molten salt thermal storage system was increased to 80ms, the frequency deviation was reduced by 32%, the peak-shaving economy was improved, the start-up and shutdown costs of coal-fired power were reduced by 18%, and the energy storage life was extended. In other words, the SOC (State of Charge) fluctuation range was reduced to 45%.

[0057] like Figure 2 As shown, the second aspect of the present invention is to provide a frequency modulation and peak regulation coordinated optimization system for a molten salt heat storage system, comprising the following steps: Data acquisition module 101: used to obtain real-time state characteristic data of the molten salt heat storage system and relevant state characteristic data affecting the efficiency of the molten salt heat storage system; and then obtain actual operation action characteristic data and control management action characteristic data in the molten salt heat storage system; Long-time scale analysis module 102: used to determine the optimal slow strategy for the molten salt thermal storage system during long-time peak-shaving based on relevant state feature data and control management action feature data through a reinforcement learning algorithm, determine a baseline output plan for the generator set based on the optimal slow strategy during the peak-shaving process, and determine a baseline frequency of the power grid at each moment based on the baseline output plan; Short-time-scale analysis module 103: used to determine the optimal fast strategy for the molten salt thermal storage system in the short-time-scale frequency modulation process based on the reference frequency, real-time state characteristic data and actual operation action characteristic data of the power grid at each moment through a reinforcement learning algorithm; Collaborative optimization module 104: used to perform collaborative optimization of frequency regulation and peak regulation of the molten salt thermal storage system through the optimal slow strategy and the optimal fast strategy.

[0058] The third aspect of the present invention is to provide an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, a frequency modulation and peak modulation coordinated optimization method of a molten salt heat storage system is implemented.

[0059] A fourth aspect of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements a frequency modulation and peak modulation coordinated optimization method for a molten salt heat storage system.

[0060] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, optical storage, etc.) containing computer-usable program code.

[0061] The present invention is described with reference to flowcharts and / or block diagrams of methods, systems, and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0062] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0063] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the present invention.

Claims

1. A frequency modulation and peak regulation collaborative optimization method for a molten salt heat storage system, characterized in that: include: Acquire real-time state characteristic data in the molten salt heat storage system and relevant state characteristic data that affect the efficiency of the molten salt heat storage system; Then obtain the actual operation action characteristic data and control management action characteristic data in the molten salt heat storage system; Based on the relevant state characteristic data and control management action characteristic data, a reinforcement learning algorithm is used to determine the optimal slow strategy for the molten salt thermal storage system during the long-term peak-shaving process. The optimal slow strategy during the peak-shaving process is used to determine the benchmark output plan of the generator set; the benchmark output plan is used to determine the benchmark frequency of the power grid at each moment; Based on the grid's baseline frequency, real-time status data, and actual operational characteristics at each moment, a reinforcement learning algorithm is used to determine the optimal fast strategy for the molten salt thermal storage system's frequency regulation process over short timescales. The frequency regulation and peak regulation of the molten salt heat storage system are coordinated and optimized through the optimal slow strategy and the optimal fast strategy.

2. The frequency modulation and peak regulation coordinated optimization method of a molten salt heat storage system according to claim 1, characterized in that: The method of determining the optimal slow strategy for the molten salt thermal storage system in a long-term peak regulation process based on the relevant state feature data and the control management action feature data through a reinforcement learning algorithm includes: The first state space of long time scale is constructed through relevant state feature data, and the first action space of long time scale is constructed through control and management action feature data; through the first state space and first action space of long time scale, the optimal slow strategy of the molten salt heat storage system based on the long time scale peak regulation process is determined through a policy-based reinforcement learning algorithm.

3. The frequency modulation and peak regulation coordinated optimization method of a molten salt heat storage system according to claim 2, characterized in that: The method of determining the optimal slow strategy of the molten salt thermal storage system based on the long-time-scale peak regulation process by using the first state space and the first action space of the long-time scale and the policy-based reinforcement learning algorithm includes: The objective function of the slow strategy is determined based on the first state space and the first action space of the long time scale. The objective function of the slow strategy is specifically expressed as: Where, is the discount factor, Indicates the The discount factor in time steps, Indicates the The reward value of each time step, represents the total number of time steps in the interaction between the environment and the agent, represents the expected operation, that is, the expected value of making decisions according to the strategy π, represents the long-term cumulative reward of all time steps; The optimal slow strategy is obtained by the long-term cumulative reward of all time steps.

4. The frequency modulation and peak regulation coordinated optimization method of a molten salt heat storage system according to claim 1, characterized in that: The optimal fast strategy for the molten salt thermal storage system based on the short-time-scale frequency regulation process is determined by a reinforcement learning algorithm based on the reference frequency, real-time status characteristic data, and actual operation action characteristic data of the power grid at each moment, including: At preset time intervals To continuously obtain the preset time before the current moment Real-time status characteristic data of the internal molten salt thermal storage system at all times; Obtain the maximum output power of the generator or energy storage device; A frequency stability constraint is constructed using the reference frequency of the power grid at each moment, and a penalty value is obtained when the frequency deviates from the target frequency. The penalty value when the frequency deviates from the target frequency is recorded as the penalty value of the first constraint. A generator and energy storage capacity constraint is constructed using the maximum output power of the generator or energy storage device, and a penalty value is obtained when the maximum output power limit of the generator or energy storage device is exceeded. The penalty value when the maximum output power limit of the generator or energy storage device is exceeded is recorded as the penalty value of the second constraint. Construct a second state space with a short time scale through real-time state feature data, and construct a second action space with a short time scale through actual operation action feature data; The objective function of the fast strategy is constructed through the penalty values of several constraints, the second state space and the second action space of the short time scale; the optimal fast strategy of the molten salt heat storage system based on the short time scale frequency modulation process is determined through the objective function of the fast strategy.

5. The frequency modulation and peak regulation coordinated optimization method of a molten salt heat storage system according to claim 4, characterized in that: The frequency stability constraint is constructed by using the reference frequency of the power grid at each moment, and a penalty value is obtained when the frequency deviates from the target frequency, including: Where, Indicates the frequency of the power grid at each moment, Indicates the reference frequency of the power grid at each moment, Indicates the preset frequency error, represents the frequency violation penalty coefficient, represents the maximum value function, Indicates the absolute value symbol, Indicates the penalty value when the frequency deviates from the target frequency; Among them, the frequency violation penalty coefficient is determined by hyperparameter tuning.

6. The frequency modulation and peak regulation coordinated optimization method of a molten salt heat storage system according to claim 4, characterized in that: The generator and energy storage capacity constraints are constructed using the maximum output power of the generator or energy storage device, and a penalty value is obtained when the maximum output power limit of the generator or energy storage device is exceeded, including: Where, Indicates the output power of the generator or energy storage device at each moment, Indicates the maximum output power of the generator or energy storage device; represents the maximum value function, represents the output power violation penalty coefficient, Indicates the penalty value when the maximum output power limit of the generator or energy storage device is exceeded; The penalty value for exceeding the maximum output power limit of the generator or energy storage device is determined by hyperparameter tuning.

7. The frequency modulation and peak regulation coordinated optimization method of a molten salt heat storage system according to claim 4, characterized in that: The objective function of the fast strategy is constructed by using a number of constraint penalty values, a second state space and a second action space of a short time scale; The optimal fast strategy for the molten salt thermal storage system based on the short-time-scale frequency modulation process is determined through the objective function of the fast strategy, including: Indicates the The discount factor in the update step, Indicates the The reward value of the update step, Represents the total number of update steps during the interaction between the environment and the agent; Indicates the The penalty value of the constraint, Indicates the The penalty intensity of a constraint, Indicates the expected operation, that is, according to the strategy The expected value of making a decision, represents the long-term cumulative reward of all update steps, represents the number of all constraints, Indicates the adjustment of the initial update time interval, that is, the duration of an update step is equal to the adjustment of the initial update time interval; the penalty intensity of each constraint is determined by hyperparameter tuning; Where, Indicates the The real-time state feature is at the current moment The value of a reference time, Indicates the The mean value of all reference moments of the real-time state feature data at the current moment, Indicates the number of reference moments, Indicates the number of all real-time status features, Indicates the absolute value symbol, represents the linear normalization function, Indicates the preset initial update time interval; Among them, the preset value before the current moment A moment is used as the reference moment of the current moment; The optimal fast strategy is obtained through the long-term cumulative rewards of all update steps.

8. A frequency modulation and peak regulation coordinated optimization system for a molten salt heat storage system, characterized in that: include: Data acquisition module: used to obtain real-time status characteristic data in the molten salt heat storage system and relevant status characteristic data that affect the efficiency of the molten salt heat storage system; Then obtain the actual operation action characteristic data and control management action characteristic data in the molten salt heat storage system; Long-time scale analysis module: used to determine the optimal slow strategy for the molten salt thermal storage system during long-time peak-shaving based on relevant state feature data and control management action feature data through a reinforcement learning algorithm. The module also determines the benchmark output plan of the generator set based on the optimal slow strategy during the peak-shaving process. The benchmark output plan is used to determine the benchmark frequency of the power grid at each moment. Short-time scale analysis module: This module is used to determine the optimal fast strategy for the molten salt thermal storage system in the short-time scale frequency regulation process based on the grid's reference frequency, real-time status characteristic data, and actual operation action characteristic data at each moment through a reinforcement learning algorithm. Collaborative optimization module: used to perform collaborative optimization of frequency regulation and peak regulation of molten salt thermal storage system through optimal slow strategy and optimal fast strategy.

9. An electronic device, characterized in that: It comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the frequency regulation and peak regulation collaborative optimization method of a molten salt heat storage system according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the frequency regulation and peak regulation coordinated optimization method of a molten salt heat storage system according to any one of claims 1 to 7 is implemented.

Images