Intra-day scheduling method and device for power distribution network containing compressed air energy storage system

Abstract

The invention discloses an intra-day scheduling method and device for a power distribution network containing a compressed air energy storage system, and relates to the technical field of power distribution networks, and the method comprises the steps: constructing a power distribution network model containing the compressed air energy storage system and a power distribution network simulation environment; constructing a reinforcement learning agent according to the characteristics of a thermal power generating unit and an energy storage power station in the power distribution network model; based on a near-end strategy optimization algorithm, using a reinforcement learning agent to interact with a power distribution network simulation environment to obtain a trained reinforcement learning agent; performing intra-day scheduling of the actual power distribution network by using the trained reinforcement learning agent to obtain a scheduling strategy of each scheduling time period; and updating the scheduling strategy of the trained reinforcement learning agent, and performing new intra-day scheduling of one day. According to the method, the optimal strategy is learned through interaction of the intelligent agent and the environment, the reliability and stability of the power distribution network are improved, the operation cost is reduced, and a new thought and method are provided for efficient operation of the power distribution network.

Description

Technical Field

[0001] The present invention relates to the technical field of distribution networks, and in particular to a method and device for daily dispatching of a distribution network including a compressed air energy storage system. Background Art

[0002] In the context of the global energy structure transformation, the proportion of renewable energy in the distribution network continues to increase. As important renewable energy sources, wind power and photovoltaic power have the advantages of being clean and sustainable, but their intermittent and uncertain nature poses a huge challenge to the stable operation of the distribution network. In order to effectively respond to these characteristics of renewable energy and improve the reliability and stability of the distribution network, energy storage technology has become one of the key solutions.

[0003] Compressed air energy storage and electrochemical energy storage power stations are gradually playing an important role in distribution networks due to their respective advantages. Compressed air energy storage has the characteristics of high energy density, long service life, and relatively low cost; electrochemical energy storage has the advantages of fast response speed and flexible adjustment. In a distribution network containing multiple energy sources, in order to achieve efficient intraday scheduling of the distribution network and give full play to the efficiency of various energy sources and energy storage equipment, the introduction of intelligent decision-making methods has become an inevitable trend. Reinforcement learning, as an intelligent decision-making method, can learn the optimal strategy through the interaction between the intelligent agent and the environment, and provides new ideas and methods for the intraday scheduling of distribution networks containing compressed air energy storage.

[0004] The intraday distribution network dispatching method based on model predictive control is a common dispatching method at present. This method establishes a mathematical model of the distribution network, makes ultra-short-term predictions of renewable energy output and load demand in the future, and solves the optimal dispatching scheme through optimization algorithm based on the prediction results and the operation constraints of the distribution network. Finally, the distribution network is controlled in real time according to the dispatching scheme obtained. The advantage of this method is that it can fully consider the future situation and make dispatching decisions in advance, thereby improving the stability and reliability of the distribution network. At the same time, the optimality of the dispatching scheme can be guaranteed to a certain extent by solving the optimization algorithm. However, this method also has some shortcomings: first, the accuracy of the prediction depends on the quality of historical data and the performance of the prediction algorithm, while the output and load demand of renewable energy have great uncertainty, which may lead to a large deviation between the prediction results and the actual situation. Second, the computational complexity is high. Especially for large-scale distribution network systems, solving the optimal dispatching scheme requires a lot of computing resources and time, which is difficult to meet the requirements of real-time dispatching.

[0005] Therefore, in order to meet actual needs, a distribution network intraday scheduling technology containing a compressed air energy storage system is provided. Summary of the invention

[0006] In view of the defects existing in the prior art, the purpose of the present invention is to provide a method and device for daily scheduling of a distribution network containing a compressed air energy storage system, aiming to overcome the shortcomings of the prior art, better cope with the intermittent and uncertain nature of renewable energy through interactive learning of optimal strategies between intelligent agents and the environment, improve the reliability and stability of the distribution network, and reduce operating costs at the same time, providing new ideas and methods for the efficient operation of the distribution network.

[0007] In order to achieve the above purpose, the technical solution adopted by the present invention is: In a first aspect, the present application provides a method for daily dispatching of a distribution network including a compressed air energy storage system, the method comprising the following steps: S1, build a distribution network model and distribution network simulation environment containing a compressed air energy storage system; S2, constructing a reinforcement learning agent based on a neural network according to the characteristics of the thermal power units and energy storage power stations in the distribution network model; S3, based on the proximal strategy optimization algorithm, using the reinforcement learning agent to interact with the distribution network simulation environment, so that the reinforcement learning agent learns to obtain the optimal scheduling strategy, and obtains a trained reinforcement learning agent; S4, using the trained reinforcement learning agent to perform intraday dispatch of the actual distribution network, determine the charging and discharging power of the energy storage power station in each dispatching period, optimize the output and standby of each thermal power unit, obtain the dispatching strategy for each dispatching period, and complete the intraday dispatch of one day; S5, after completing the intraday scheduling for one day, the scheduling strategy of the trained reinforcement learning agent is updated, and S4 is repeated to perform intraday scheduling for a new day.

[0008] On the basis of the above technical solution, the distribution network model includes multiple thermal power units, wind power stations, photovoltaic power stations, multiple electrochemical energy storage power stations, compressed air energy storage power stations and loads.

[0009] On the basis of the above technical solution, the compression power constraint of the compressor in the compressed air energy storage power station is: ; in, is the power of the compressor during period t, and as well as are the upper and lower limits of the compressor power, is a binary variable representing the operating status of the compressor during period t, 1 means the compressor is in working state. 0 means the compressor is in shutdown state; The expansion power constraint of the expander in the compressed air energy storage power station is: ; in, is the power of the expander during period t, and as well as are the upper and lower limits of the expander power, is a binary variable representing the operating status of the expander at time period t, 1 means the expander is in working state. 0 means the expander is in shutdown state; Limited by the internal structure of the compressed air energy storage system, the operating states of the compressor and expander are constrained as follows: ; The mathematical model and constraints of the air storage chamber in the compressed air energy storage power station are: ; ; ; ; in, is the air pressure value of the air storage chamber during period t, respectively subject to the lower limit of the air pressure value And upper pressure limit Constraints, as well as are the pressure increase rate and pressure decrease rate of the air storage chamber in period t, which are respectively related to the compression power in the same period , Expansion power are proportional to, and their proportional coefficients are and , is a scheduling period; The mathematical model and constraints of the heat storage tank in the compressed air energy storage power station are: ; ; ; ; in, is the heat storage capacity of the heat storage tank during period t, subject to the lower limit of heat storage capacity and heat storage limit Constraints, is the heat generation power of the heat exchanger during the air compression stage, and the compression power during the same period is proportional to, and the proportionality coefficient is , is the heat absorption power of the heat exchanger during the air expansion stage, and the expansion power during the same period is proportional to, and the proportionality coefficient is ; The backup constraints in the compressed air energy storage power station are: ; in, as well as are the positive reserve capacity and negative reserve capacity of the compressed air energy storage system in the compressed charging state, as well as They are respectively the positive reserve capacity and negative reserve capacity of the compressed air energy storage system in the expansion discharge state; The mathematical model and constraints of the electrochemical energy storage power station are: ; ; ; in, is the energy stored in the i-th electrochemical energy storage power station during period t, and and are their lower and upper limits respectively, and are the charging power and discharging power of the i-th electrochemical energy storage power station in period t, respectively, and and are the upper limits of the power when the i-th electrochemical energy storage power station is in the charging or discharging state, and The charging and discharging efficiencies of the i-th electrochemical energy storage power station are respectively calculated by using the binary variables To represent the working state of the i-th electrochemical energy storage power station in period t, 1 indicates that it is in charging state. 0 indicates that it is in the discharge state; The power constraint of the thermal power unit is: ; ; ; in, It represents the power of thermal power unit j in time period t, which is subject to the upper limit in the running state. and lower limit Constraints, is a binary variable representing the operating status of thermal power unit j in period t, 1 means that the thermal power unit j is in the running state. 0 means it is in shutdown state, which is determined by the day-ahead scheduling plan. and are the limits of the ramp-up rate and ramp-down rate of thermal power unit j respectively; The reserve constraint of the thermal power unit is: ; ; in, and They represent the positive reserve capacity and negative reserve capacity of thermal power unit j in period t respectively; The renewable energy output constraints are as follows: ; in, is the actual dispatch output of renewable energy in period t, and is the maximum output of renewable energy in period t; The power balance constraint of the distribution network system is: ; in, is the load in period t, is the number of electrochemical energy storage power stations, is the number of thermal power units; The backup constraint of the distribution network system is: ; The lower limit of the reserve capacity of the distribution network system is related to the load and renewable energy output In a linear relationship, their proportions are and .

[0010] On the basis of the above technical solution, in step S2: The constructed reinforcement learning agent is used in each period t according to the distribution network status of the current period And the reward calculated by the reward function , output action , to adjust the charging and discharging power of the compressed air energy storage power station and each electrochemical energy storage power station; The current state of the distribution network Including the renewable energy generation sequence and load sequence in the past 24 hours, the energy status of each electrochemical energy storage power station in the current period, the air pressure value of the air storage chamber of the compressed air energy storage power station in the current period, the heat stored in the heat storage tank of the compressed air energy storage power station in the current period, the start and stop plan of each thermal power unit scheduled a few days ago, and the actual output of the previous period; Reward function for reinforcement learning agents It is set as the inverse of the deviation between the output of the thermal power unit, the charge and discharge power of the compressed air energy storage, and the charge and discharge power of the electrochemical energy storage power station and the corresponding day-ahead dispatch plan in the current period t. The calculation formula is: ; in, , as well as is the corresponding coefficient of deviation, and , , , , is the corresponding day-ahead scheduling plan value; The actions taken by the reinforcement learning agent based on the current state of the distribution network Including compressed air energy storage power stations and the charging and discharging power of various electrochemical energy storage power stations.

[0011] On the basis of the above technical solution, the reinforcement learning agent includes a strategy neural network and a value network; The strategy neural network includes a gated recurrent unit and a fully connected layer. The strategy neural network processes the state at each moment as follows: The renewable energy generation sequence and load sequence of the past 24 hours are input into a gated recurrent unit, and the time series characteristics are output; The energy state of each electrochemical energy storage power station in the current period, the air pressure value of the compressed air energy storage power station and the heat stored in the heat storage tank, the start and stop plan of the thermal power unit scheduled a day ago, and the actual output of the previous period are input into a fully connected layer network, and the state characteristics are output; The time series features and state features are input into another fully connected layer network together to output the charging and discharging status of the compressed air energy storage power station and each electrochemical energy storage power station at the current moment, as well as the mean and standard deviation of the Gaussian distribution of the charging and discharging power.

[0012] Based on the above technical solution, the value network of the reinforcement learning agent includes a fully connected layer, the distribution network state of the input period t , output the value in this state .

[0013] On the basis of the above technical solution, S3 comprises the following steps: S31. Model parameter initialization: Initialize the parameters of the policy network and value network, and set the learning rate , Discount Factor and truncation parameters , and the smoothing discount coefficient for the generalized advantage estimate to calculate the advantage function ; S32, collect data: The reinforcement learning agent interacts with the distribution network simulation environment for multiple rounds to perform intraday scheduling; In each time period t, the distribution network state After inputting the strategy neural network, the charging and discharging status of the compressed air energy storage power station and the electrochemical energy storage power station in the current period and the mean and standard deviation of the Gaussian distribution of the charging and discharging power are output; The reinforcement learning agent determines the charging and discharging power of each energy storage power station in the current period by random sampling, and then cuts the charging and discharging power according to the constraints of the energy storage power station to obtain the action , the reinforcement learning agent takes action To adjust the charging and discharging power during this period; After determining the charging and discharging power of each energy storage power station, the linear programming method is used to minimize the deviation between the daily dispatch output of the thermal power unit and the day-ahead dispatch, and to determine the output of each thermal power unit in this period and the positive and negative spare capacity allocated to the thermal power unit and the compressed air energy storage power station; Calculate the reward for this period through the reward function calculation expression , and calculate the distribution network state for the next period through the state equation ; After completing multiple rounds of interactions to complete the day's intraday scheduling, store the status of each time period ,action ,award and the next state , forming a complete trajectory; S33, Strategy Update: The advantage function of each time period t is smoothly estimated by generalized advantage estimation. The advantage function formula is: ; in, ; The loss function of the calculation strategy neural network, the advantage function formula is: ; in, and Respectively represent the state under the new and old strategies Take Action probability; Calculate the loss function of the value network. The loss function formula is as follows: ; Perform back-propagation updates of the policy network and value network, and update network parameters according to the gradient descent method; S34, multiple rounds of iterations: Repeat S32 and S33 until a specific number of iterations is met.

[0014] On the basis of the above technical solution, S4 comprises the following steps: S41, in time period t, the distribution network status Input the strategy neural network, output the charge and discharge status of each energy storage power station and the mean and standard deviation of the Gaussian distribution of the charge and discharge power, obtain the preliminary charge and discharge strategy through random sampling, tailor the charge and discharge strategy according to the constraints of the energy storage power station in the distribution network model, and determine the charge and discharge power of each energy storage power station; S42, the start and stop status of the thermal power unit is determined by the day-ahead dispatch plan, and the deviation between the daily dispatch output of the thermal power unit and the output of the corresponding day-ahead dispatch plan in time period t is minimized as the optimization goal, and the linear programming method is used for optimization to determine the daily dispatch output of the thermal power unit and the positive and negative spare capacity allocated to the thermal power unit and the compressed air energy storage power station; S43: Execute the intraday dispatching strategy for period t and collect the distribution network status information for the next period , repeat S41 until the intraday scheduling is completed.

[0015] On the basis of the above technical solution, S5 comprises the following steps: After completing the day's intraday scheduling, the status of each time period in the day is ,action ,award and the next state Conduct learning, optimize the dispatching strategy of energy storage power stations, and participate in the intraday dispatching of the distribution network the next day.

[0016] In a second aspect, the present application further provides a distribution network intraday dispatching device including a compressed air energy storage system, the device comprising: A pre-built module for building a distribution network model and a distribution network simulation environment including a compressed air energy storage system; An intelligent agent building module, which is used to build a reinforcement learning intelligent agent based on a neural network according to the characteristics of the thermal power units and energy storage power stations in the distribution network model; An agent training module, which is used to use the reinforcement learning agent to interact with the distribution network simulation environment based on a proximal strategy optimization algorithm, so that the reinforcement learning agent learns to obtain an optimal scheduling strategy and obtains a trained reinforcement learning agent; The daily dispatching module is used to use the trained reinforcement learning agent to perform intraday dispatching of the actual distribution network, determine the charging and discharging power of the energy storage power station in each dispatching period, optimize the output and standby of each thermal power unit, obtain the dispatching strategy for each dispatching period, and complete the intraday dispatching of one day; The subsequent scheduling module is used to update the scheduling strategy of the trained reinforcement learning agent after completing the intraday scheduling of the day, and control the scheduling module of the day to perform the intraday scheduling of a new day.

[0017] Compared with the prior art, the advantages of the present invention are: The present invention aims to overcome the shortcomings of the existing technology, learn the optimal strategy through the interaction between the intelligent agent and the environment, better cope with the intermittency and uncertainty of renewable energy, improve the reliability and stability of the distribution network, reduce operating costs, and provide new ideas and methods for the efficient operation of the distribution network. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A flowchart of the steps of a method for daily dispatching of a distribution network including a compressed air energy storage system according to an embodiment of the present invention; Figure 2 This is a principle flow chart of a method for daily dispatching of a distribution network including a compressed air energy storage system according to an embodiment of the present invention; Figure 3 A strategy update process of a reinforcement learning agent in a method for daily dispatching of a distribution network containing a compressed air energy storage system according to an embodiment of the present invention; Figure 4 4 is a structural block diagram of a distribution network intraday dispatching device including a compressed air energy storage system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0021] The embodiments of the present invention are further described in detail below with reference to the accompanying drawings.

[0022] The embodiments of the present application provide a method and device for daily scheduling of a distribution network containing a compressed air energy storage system, aiming to overcome the shortcomings of the prior art, better cope with the intermittent and uncertain nature of renewable energy through interactive learning of optimal strategies between an intelligent agent and the environment, improve the reliability and stability of the distribution network, and reduce operating costs, thereby providing new ideas and methods for the efficient operation of the distribution network.

[0023] In order to achieve the above technical effects, the overall idea of ​​this application is as follows: A method for intraday dispatching of a distribution network including a compressed air energy storage system, the method comprising the steps of: S1, build a distribution network model and distribution network simulation environment containing a compressed air energy storage system; S2, based on the characteristics of thermal power units and energy storage power stations in the distribution network model, builds a reinforcement learning agent based on neural networks; S3, based on the proximal strategy optimization algorithm, uses the reinforcement learning agent to interact with the distribution network simulation environment, so that the reinforcement learning agent learns to obtain the optimal dispatching strategy and obtains the trained reinforcement learning agent; S4, using the trained reinforcement learning agent to perform intraday dispatch of the actual distribution network, determine the charging and discharging power of the energy storage power station in each dispatch period, optimize the output and reserve of each thermal power unit, obtain the dispatch strategy for each dispatch period, and complete the intraday dispatch of the day; S5, after completing the intraday scheduling for one day, the scheduling strategy of the trained reinforcement learning agent is updated, and S4 is repeated to perform intraday scheduling for a new day.

[0024] The embodiments of the present application are further described in detail below in conjunction with the accompanying drawings.

[0025] First, see Figures 1 to 3 As shown, an embodiment of the present application provides a method for daily scheduling of a distribution network including a compressed air energy storage system, the method comprising the following steps: S1, build a distribution network model and distribution network simulation environment containing a compressed air energy storage system; S2, based on the characteristics of thermal power units and energy storage power stations in the distribution network model, builds a reinforcement learning agent based on neural networks; S3, based on the proximal strategy optimization algorithm, uses the reinforcement learning agent to interact with the distribution network simulation environment, so that the reinforcement learning agent learns to obtain the optimal dispatching strategy and obtains the trained reinforcement learning agent; S4, using the trained reinforcement learning agent to perform intraday dispatch of the actual distribution network, determine the charging and discharging power of the energy storage power station in each dispatch period, optimize the output and reserve of each thermal power unit, obtain the dispatch strategy for each dispatch period, and complete the intraday dispatch of the day; S5, after completing the intraday scheduling for one day, the scheduling strategy of the trained reinforcement learning agent is updated, and S4 is repeated to perform intraday scheduling for a new day.

[0026] In the embodiments of the present application, it is intended to overcome the deficiencies of the prior art, learn the optimal strategy through the interaction between the intelligent agent and the environment, better cope with the intermittency and uncertainty of renewable energy, improve the reliability and stability of the distribution network, and reduce operating costs, thereby providing new ideas and methods for the efficient operation of the distribution network.

[0027] Furthermore, the distribution network model includes multiple thermal power units, wind power stations, photovoltaic power stations, multiple electrochemical energy storage power stations, compressed air energy storage power stations and loads.

[0028] Furthermore, the compression power constraint of the compressor in the compressed air energy storage power station is: ; in, is the power of the compressor during period t, and as well as are the upper and lower limits of the compressor power, is a binary variable representing the operating status of the compressor during period t, 1 means the compressor is in working state. 0 means the compressor is in shutdown state; The expansion power constraint of the expander in the compressed air energy storage power station is: ; in, is the power of the expander during period t, and as well as are the upper and lower limits of the expander power, is a binary variable representing the operating status of the expander at time period t, 1 means the expander is in working state. 0 means the expander is in shutdown state; Limited by the internal structure of the compressed air energy storage system, the operating states of the compressor and expander are constrained as follows: ; The mathematical model and constraints of the air storage chamber in the compressed air energy storage power station are: ; ; ; ; in, is the air pressure value of the air storage chamber during period t, respectively subject to the lower limit of the air pressure value And upper pressure limit Constraints, as well as are the pressure increase rate and pressure decrease rate of the air storage chamber in period t, which are respectively related to the compression power in the same period , Expansion power are proportional to, and their proportional coefficients are and , is a scheduling period; The mathematical model and constraints of the heat storage tank in the compressed air energy storage power station are: ; ; ; ; in, is the heat storage capacity of the heat storage tank during period t, subject to the lower limit of heat storage capacity and heat storage limit Constraints, is the heat generation power of the heat exchanger during the air compression stage, and the compression power during the same period is proportional to, and the proportionality coefficient is , is the heat absorption power of the heat exchanger during the air expansion stage, and the expansion power during the same period is proportional to, and the proportionality coefficient is ; The backup constraints in the compressed air energy storage power station are: ; in, as well as are the positive reserve capacity and negative reserve capacity of the compressed air energy storage system in the compressed charging state, as well as They are respectively the positive reserve capacity and negative reserve capacity of the compressed air energy storage system in the expansion discharge state; The mathematical model and constraints of the electrochemical energy storage power station are: ; ; ; in, is the energy stored in the i-th electrochemical energy storage power station during period t, and and are their lower and upper limits respectively, and are the charging power and discharging power of the i-th electrochemical energy storage power station in period t, respectively, and and are the upper limits of the power when the i-th electrochemical energy storage power station is in the charging or discharging state, and The charging and discharging efficiencies of the i-th electrochemical energy storage power station are respectively calculated by using the binary variables To represent the working state of the i-th electrochemical energy storage power station in period t, 1 indicates that it is in charging state. 0 indicates that it is in the discharge state; The power constraint of the thermal power unit is: ; ; ; in, It represents the power of thermal power unit j in time period t, which is subject to the upper limit in the running state. and lower limit Constraints, is a binary variable representing the operating status of thermal power unit j in period t, 1 means that the thermal power unit j is in the running state. 0 means it is in shutdown state, which is determined by the day-ahead scheduling plan. and are the limits of the ramp-up rate and ramp-down rate of thermal power unit j respectively; The reserve constraint of the thermal power unit is: ; ; in, and They represent the positive reserve capacity and negative reserve capacity of thermal power unit j in period t respectively; The renewable energy output constraints are as follows: ; in, is the actual dispatch output of renewable energy in period t, and is the maximum output of renewable energy in period t; The power balance constraint of the distribution network system is: ; in, is the load in period t, is the number of electrochemical energy storage power stations, is the number of thermal power units; The backup constraint of the distribution network system is: ; The lower limit of the reserve capacity of the distribution network system is related to the load and renewable energy output In a linear relationship, their proportions are and .

[0029] Further, in step S2: The constructed reinforcement learning agent is used in each period t according to the distribution network status of the current period And the reward calculated by the reward function , output action , to adjust the charging and discharging power of the compressed air energy storage power station and each electrochemical energy storage power station; The current state of the distribution network Including the renewable energy generation sequence and load sequence in the past 24 hours, the energy status of each electrochemical energy storage power station in the current period, the air pressure value of the air storage chamber of the compressed air energy storage power station in the current period, the heat stored in the heat storage tank of the compressed air energy storage power station in the current period, the start and stop plan of each thermal power unit scheduled a few days ago, and the actual output of the previous period; Reward function for reinforcement learning agents It is set as the inverse of the deviation between the output of the thermal power unit, the charge and discharge power of the compressed air energy storage, and the charge and discharge power of the electrochemical energy storage power station and the corresponding day-ahead dispatch plan in the current period t. The calculation formula is: ; in, , as well as is the corresponding coefficient of deviation, and , , , , is the corresponding day-ahead scheduling plan value; The actions taken by the reinforcement learning agent based on the current state of the distribution network Including compressed air energy storage power stations and the charging and discharging power of various electrochemical energy storage power stations.

[0030] Furthermore, the reinforcement learning agent includes a strategy neural network and a value network; The strategy neural network includes a gated recurrent unit and a fully connected layer. The strategy neural network processes the state at each moment as follows: The renewable energy generation sequence and load sequence of the past 24 hours are input into a gated recurrent unit, and the time series characteristics are output; The energy state of each electrochemical energy storage power station in the current period, the air pressure value of the compressed air energy storage power station and the heat stored in the heat storage tank, the start and stop plan of the thermal power unit scheduled a day ago, and the actual output of the previous period are input into a fully connected layer network, and the state characteristics are output; The time series features and state features are input into another fully connected layer network together to output the charging and discharging status of the compressed air energy storage power station and each electrochemical energy storage power station at the current moment, as well as the mean and standard deviation of the Gaussian distribution of the charging and discharging power.

[0031] Furthermore, the value network of the reinforcement learning agent includes a fully connected layer, and the state of the distribution network in the input period t , output the value in this state .

[0032] Further, the S3 comprises the following steps: S31. Model parameter initialization: Initialize the parameters of the policy network and value network, and set the learning rate , Discount Factor and truncation parameters , and the smoothing discount coefficient for the generalized advantage estimate to calculate the advantage function ; S32, collect data: The reinforcement learning agent interacts with the distribution network simulation environment for multiple rounds to perform intraday scheduling; In each time period t, the distribution network state After inputting the strategy neural network, the charging and discharging status of the compressed air energy storage power station and the electrochemical energy storage power station in the current period and the mean and standard deviation of the Gaussian distribution of the charging and discharging power are output; The reinforcement learning agent determines the charging and discharging power of each energy storage power station in the current period by random sampling, and then cuts the charging and discharging power according to the constraints of the energy storage power station to obtain the action , the reinforcement learning agent takes action To adjust the charging and discharging power during this period; After determining the charging and discharging power of each energy storage power station, the linear programming method is used to minimize the deviation between the daily dispatch output of the thermal power unit and the day-ahead dispatch, and to determine the output of each thermal power unit in this period and the positive and negative spare capacity allocated to the thermal power unit and the compressed air energy storage power station; Calculate the reward for this period through the reward function calculation expression , and calculate the distribution network state for the next period through the state equation ; After completing multiple rounds of interactions to complete the day's intraday scheduling, store the status of each time period ,action ,award and the next state , forming a complete trajectory; As shown in the attached figure of the specification Figure 3 As shown, S33, strategy update: The advantage function of each time period t is smoothly estimated by generalized advantage estimation. The advantage function formula is: ; in, ; The loss function of the calculation strategy neural network, the advantage function formula is: ; in, and Respectively represent the state under the new and old strategies Take Action probability; Calculate the loss function of the value network. The loss function formula is as follows: ; Perform back-propagation updates of the policy network and value network, and update network parameters according to the gradient descent method; S34, multiple rounds of iterations: Repeat S32 and S33 until a specific number of iterations is met.

[0033] Further, the S4 comprises the following steps: S41, in time period t, the distribution network status Input the strategy neural network, output the charge and discharge status of each energy storage power station and the mean and standard deviation of the Gaussian distribution of the charge and discharge power, obtain the preliminary charge and discharge strategy through random sampling, tailor the charge and discharge strategy according to the constraints of the energy storage power station in the distribution network model, and determine the charge and discharge power of each energy storage power station; S42, the start and stop status of the thermal power unit is determined by the day-ahead dispatch plan, and the deviation between the daily dispatch output of the thermal power unit and the output of the corresponding day-ahead dispatch plan in time period t is minimized as the optimization goal, and the linear programming method is used for optimization to determine the daily dispatch output of the thermal power unit and the positive and negative spare capacity allocated to the thermal power unit and the compressed air energy storage power station; S43: Execute the intraday dispatching strategy for period t and collect the distribution network status information for the next period , repeat S41 until the intraday scheduling is completed.

[0034] Further, the S5 comprises the following steps: After completing the day's intraday scheduling, the status of each time period in the day is ,action ,award and the next state Conduct learning, optimize the dispatching strategy of energy storage power stations, and participate in the intraday dispatching of the distribution network the next day.

[0035] In summary, the technical solution of the embodiment of the present application can be used for the dispatching of smart grids to achieve efficient integration of renewable energy sources such as wind energy and solar energy. Through the intelligent dispatching of compressed air energy storage systems, the stability and reliability of the power grid can be improved, and it is particularly suitable for power grid systems with a high proportion of renewable energy connected to the grid. It aims to solve the intraday scheduling problem of distribution networks containing compressed air energy storage, cope with the intermittency and uncertainty of renewable energy (wind power and photovoltaic power), improve the reliability and stability of distribution networks, and reduce the operating costs of distribution networks.

[0036] Specifically, the following issues need to be addressed: First, the traditional scheduling method based on simple rules has low scheduling quality and cannot give full play to the role of complex constraints and objective functions.

[0037] Second, the existing scheduling methods based on model predictive control have high computational complexity and rely on forecast data, and have poor adaptability to the uncertainty of renewable energy and load.

[0038] Furthermore, the beneficial effects of the technical solution of the embodiment of the present application are as follows: First, improve the reliability and stability of the distribution network: The method of the embodiment of the present application can effectively adjust the charging and discharging strategy of the compressed air energy storage system, making the distribution network more stable when facing fluctuations in renewable energy sources such as wind power and photovoltaic power, and significantly reducing the grid operation risks caused by the uncertainty of renewable energy.

[0039] Second, flexible adaptation to complex constraints: The proximal strategy optimization algorithm is adopted, so that this method can adapt to multiple complex constraints in power grid dispatching, such as energy storage system capacity limitation, charging and discharging rate limitation, and start and stop constraints of generator sets, thereby ensuring the feasibility and optimality of the dispatching scheme.

[0040] Third, enhance the intelligence level of the system: Through the introduction of a reinforcement learning model, the embodiment of the present application enables the compressed air energy storage system to obtain the optimal scheduling strategy in continuous environmental interaction, possess certain adaptive and learning capabilities, and be able to automatically adjust the charging and discharging strategy during operation, further improving the intelligence level of scheduling.

[0041] Second, see Figure 4 As shown, an embodiment of the present application provides a distribution network intraday scheduling device including a compressed air energy storage system, the device comprising: A pre-built module for building a distribution network model and a distribution network simulation environment including a compressed air energy storage system; An intelligent agent building module, which is used to build a reinforcement learning intelligent agent based on a neural network according to the characteristics of the thermal power units and energy storage power stations in the distribution network model; An agent training module, which is used to use the reinforcement learning agent to interact with the distribution network simulation environment based on a proximal strategy optimization algorithm, so that the reinforcement learning agent learns to obtain an optimal scheduling strategy and obtains a trained reinforcement learning agent; The daily dispatching module is used to use the trained reinforcement learning agent to perform intraday dispatching of the actual distribution network, determine the charging and discharging power of the energy storage power station in each dispatching period, optimize the output and standby of each thermal power unit, obtain the dispatching strategy for each dispatching period, and complete the intraday dispatching of one day; The subsequent scheduling module is used to update the scheduling strategy of the trained reinforcement learning agent after completing the intraday scheduling of the day, and control the scheduling module of the day to perform the intraday scheduling of a new day.

[0042] In the embodiments of the present application, it is intended to overcome the deficiencies of the prior art, learn the optimal strategy through the interaction between the intelligent agent and the environment, better cope with the intermittency and uncertainty of renewable energy, improve the reliability and stability of the distribution network, and reduce operating costs, thereby providing new ideas and methods for the efficient operation of the distribution network.

[0043] It should be noted that the specific working content of each module in the intraday scheduling device of the distribution network containing a compressed air energy storage system is as described in the corresponding contents of each step in the intraday scheduling method of the distribution network containing a compressed air energy storage system mentioned in the first aspect, and each module is used to execute the corresponding step and the sub-steps within the step.

[0044] It should be noted that the intraday dispatching device for the distribution network containing a compressed air energy storage system mentioned in the second aspect is similar to the technical principles of the intraday dispatching method for the distribution network containing a compressed air energy storage system mentioned in the first aspect in terms of technical issues, technical means and technical effects, and will not be elaborated here.

[0045] In the description of the present application, it should be noted that the terms "upper", "lower", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. Unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0046] It should be noted that, in this application, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0047] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A method for daily dispatching of a distribution network including a compressed air energy storage system, characterized in that: The method comprises the following steps: S1, build a distribution network model and distribution network simulation environment containing a compressed air energy storage system; S2, constructing a reinforcement learning agent based on a neural network according to the characteristics of the thermal power units and energy storage power stations in the distribution network model; S3, based on the proximal strategy optimization algorithm, using the reinforcement learning agent to interact with the distribution network simulation environment, so that the reinforcement learning agent learns to obtain the optimal scheduling strategy, and obtains a trained reinforcement learning agent; S4, using the trained reinforcement learning agent to perform intraday dispatch of the actual distribution network, determine the charging and discharging power of the energy storage power station in each dispatching period, optimize the output and standby of each thermal power unit, obtain the dispatching strategy for each dispatching period, and complete the intraday dispatch of one day; S5, after completing the intraday scheduling for one day, the scheduling strategy of the trained reinforcement learning agent is updated, and S4 is repeated to perform intraday scheduling for a new day.

2. The method for dispatching a distribution network containing a compressed air energy storage system during the day according to claim 1, characterized in that: The distribution network model includes multiple thermal power units, a wind power station, a photovoltaic power station, multiple electrochemical energy storage power stations, a compressed air energy storage power station and loads.

3. The method for dispatching a distribution network containing a compressed air energy storage system during the day according to claim 2, characterized in that: The compression power constraint of the compressor in the compressed air energy storage power station is: ； in, is the power of the compressor during period t, and as well as are the upper and lower limits of the compressor power, is a binary variable representing the operating status of the compressor during period t, 1 means the compressor is in working state. 0 means the compressor is in shutdown state; The expansion power constraint of the expander in the compressed air energy storage power station is: ； in, is the power of the expander during period t, and as well as are the upper and lower limits of the expander power, is a binary variable representing the operating status of the expander at time period t, 1 means the expander is in working state. 0 means the expander is in shutdown state; Limited by the internal structure of the compressed air energy storage system, the operating states of the compressor and expander are constrained as follows: ; The mathematical model and constraints of the air storage chamber in the compressed air energy storage power station are: ； ； ； ； in, is the air pressure value of the air storage chamber during period t, respectively subject to the lower limit of the air pressure value And upper pressure limit Constraints, as well as are the pressure increase rate and pressure decrease rate of the air storage chamber in period t, which are respectively related to the compression power in the same period , Expansion power are proportional to, and their proportional coefficients are and , is a scheduling period; The mathematical model and constraints of the heat storage tank in the compressed air energy storage power station are: ； ； ； ； in, is the heat storage capacity of the heat storage tank during period t, subject to the lower limit of heat storage capacity and heat storage limit Constraints, is the heat generation power of the heat exchanger during the air compression stage, and the compression power during the same period is proportional to, and the proportionality coefficient is , is the heat absorption power of the heat exchanger during the air expansion stage, and the expansion power during the same period is proportional to, and the proportionality coefficient is ; The backup constraints in the compressed air energy storage power station are: ； in, as well as are the positive reserve capacity and negative reserve capacity of the compressed air energy storage system in the compressed charging state, as well as They are respectively the positive reserve capacity and negative reserve capacity of the compressed air energy storage system in the expansion discharge state; The mathematical model and constraints of the electrochemical energy storage power station are: ； ； ； in, is the energy stored in the i-th electrochemical energy storage power station during period t, and and are their lower and upper limits respectively, and are the charging power and discharging power of the i-th electrochemical energy storage power station in period t, respectively, and and are the upper limits of the power when the i-th electrochemical energy storage power station is in the charging or discharging state, and The charging and discharging efficiencies of the i-th electrochemical energy storage power station are respectively calculated by using the binary variables To represent the working state of the i-th electrochemical energy storage power station in period t, 1 indicates that it is in charging state. 0 indicates that it is in the discharge state; The power constraint of the thermal power unit is: ； ； ； in, It represents the power of thermal power unit j in time period t, which is subject to the upper limit in the running state. and lower limit Constraints, is a binary variable representing the operating status of thermal power unit j in period t, 1 means that the thermal power unit j is in the running state. 0 means it is in shutdown state, which is determined by the day-ahead scheduling plan. and are the limits of the ramp-up rate and ramp-down rate of thermal power unit j respectively; The reserve constraint of the thermal power unit is: ； ； in, and They represent the positive reserve capacity and negative reserve capacity of thermal power unit j in period t respectively; The renewable energy output constraints are as follows: ； in, is the actual dispatch output of renewable energy in period t, and is the maximum output of renewable energy in period t; The power balance constraint of the distribution network system is: ； in, is the load in period t, is the number of electrochemical energy storage power stations, is the number of thermal power units; The backup constraint of the distribution network system is: ； The lower limit of the reserve capacity of the distribution network system is related to the load and renewable energy output In a linear relationship, their proportions are and .

4. The method for daily dispatching of a distribution network including a compressed air energy storage system according to claim 1, characterized in that: In step S2: The constructed reinforcement learning agent is used in each period t according to the distribution network status of the current period And the reward calculated by the reward function , output action , to adjust the charging and discharging power of the compressed air energy storage power station and each electrochemical energy storage power station; The current state of the distribution network Including the renewable energy generation sequence and load sequence in the past 24 hours, the energy status of each electrochemical energy storage power station in the current period, the air pressure value of the air storage chamber of the compressed air energy storage power station in the current period, the heat stored in the heat storage tank of the compressed air energy storage power station in the current period, the start and stop plan of each thermal power unit scheduled a few days ago, and the actual output of the previous period; Reward function for reinforcement learning agents It is set as the inverse of the deviation between the output of the thermal power unit, the charge and discharge power of the compressed air energy storage, and the charge and discharge power of the electrochemical energy storage power station and the corresponding day-ahead dispatch plan in the current period t. The calculation formula is: ； in, , as well as is the corresponding coefficient of deviation, and , , , , is the corresponding day-ahead scheduling plan value; The actions taken by the reinforcement learning agent based on the current state of the distribution network Including compressed air energy storage power stations and the charging and discharging power of various electrochemical energy storage power stations.

5. The method for dispatching a distribution network containing a compressed air energy storage system during the day according to claim 4, characterized in that: The reinforcement learning agent includes a strategy neural network and a value network; The strategy neural network includes a gated recurrent unit and a fully connected layer. The strategy neural network processes the state at each moment as follows: The renewable energy generation sequence and load sequence of the past 24 hours are input into a gated recurrent unit, and the time series characteristics are output; The energy state of each electrochemical energy storage power station in the current period, the air pressure value of the compressed air energy storage power station and the heat stored in the heat storage tank, the start and stop plan of the thermal power unit scheduled a day ago, and the actual output of the previous period are input into a fully connected layer network, and the state characteristics are output; The time series features and state features are input into another fully connected layer network together to output the charging and discharging status of the compressed air energy storage power station and each electrochemical energy storage power station at the current moment, as well as the mean and standard deviation of the Gaussian distribution of the charging and discharging power.

6. The method for daily dispatching of a distribution network including a compressed air energy storage system according to claim 4, characterized in that: The value network of the reinforcement learning agent includes a fully connected layer, inputting the distribution network status at time period t , output the value in this state .

7. The method for daily dispatching of a distribution network including a compressed air energy storage system according to claim 1, characterized in that: The S3 comprises the following steps: S31. Model parameter initialization: Initialize the parameters of the policy network and value network, and set the learning rate , Discount Factor and truncation parameters , and the smoothing discount coefficient for the generalized advantage estimate to calculate the advantage function ; S32, collect data: The reinforcement learning agent interacts with the distribution network simulation environment for multiple rounds to perform intraday scheduling; In each time period t, the distribution network state After inputting the strategy neural network, the charging and discharging status of the compressed air energy storage power station and the electrochemical energy storage power station in the current period and the mean and standard deviation of the Gaussian distribution of the charging and discharging power are output; The reinforcement learning agent determines the charging and discharging power of each energy storage power station in the current period by random sampling, and then cuts the charging and discharging power according to the constraints of the energy storage power station to obtain the action , the reinforcement learning agent takes action To adjust the charging and discharging power during this period; After determining the charging and discharging power of each energy storage power station, the linear programming method is used to minimize the deviation between the daily dispatch output of the thermal power unit and the day-ahead dispatch, and to determine the output of each thermal power unit in this period and the positive and negative spare capacity allocated to the thermal power unit and the compressed air energy storage power station; Calculate the reward for this period through the reward function calculation expression , and calculate the distribution network state for the next period through the state equation ; After completing multiple rounds of interactions to complete the day's intraday scheduling, store the status of each time period ,action ,award and the next state , forming a complete trajectory; S33, Strategy Update: The advantage function of each time period t is smoothly estimated by generalized advantage estimation. The advantage function formula is: ； in, ; The loss function of the calculation strategy neural network, the advantage function formula is: ； in, and Respectively represent the state under the new and old strategies Take Action probability; Calculate the loss function of the value network. The loss function formula is as follows: ； Perform back-propagation updates of the policy network and value network, and update network parameters according to the gradient descent method; S34, multiple rounds of iterations: Repeat S32 and S33 until a specific number of iterations is met.

8. The method for daily dispatching of a distribution network including a compressed air energy storage system according to claim 1, characterized in that: The S4 comprises the following steps: S41, in time period t, the distribution network status Input the strategy neural network, output the charge and discharge status of each energy storage power station and the mean and standard deviation of the Gaussian distribution of the charge and discharge power, obtain the preliminary charge and discharge strategy through random sampling, tailor the charge and discharge strategy according to the constraints of the energy storage power station in the distribution network model, and determine the charge and discharge power of each energy storage power station; S42, the start and stop status of the thermal power unit is determined by the day-ahead dispatch plan, and the deviation between the daily dispatch output of the thermal power unit and the output of the corresponding day-ahead dispatch plan in time period t is minimized as the optimization goal, and the linear programming method is used for optimization to determine the daily dispatch output of the thermal power unit and the positive and negative spare capacity allocated to the thermal power unit and the compressed air energy storage power station; S43: Execute the intraday dispatching strategy for period t and collect the distribution network status information for the next period , repeat S41 until the intraday scheduling is completed.

9. The method for daily dispatching of a distribution network including a compressed air energy storage system according to claim 1, characterized in that: The S5 comprises the following steps: After completing the day's intraday scheduling, the status of each time period in the day is ,action ,award and the next state Conduct learning, optimize the dispatching strategy of energy storage power stations, and participate in the intraday dispatching of the distribution network the next day.

10. A distribution network intraday dispatching device including a compressed air energy storage system, characterized in that: The device comprises: A pre-built module for building a distribution network model and a distribution network simulation environment including a compressed air energy storage system; An intelligent agent building module, which is used to build a reinforcement learning intelligent agent based on a neural network according to the characteristics of the thermal power units and energy storage power stations in the distribution network model; An agent training module, which is used to use the reinforcement learning agent to interact with the distribution network simulation environment based on a proximal strategy optimization algorithm, so that the reinforcement learning agent learns to obtain an optimal scheduling strategy and obtains a trained reinforcement learning agent; The daily dispatching module is used to use the trained reinforcement learning agent to perform intraday dispatching of the actual distribution network, determine the charging and discharging power of the energy storage power station in each dispatching period, optimize the output and standby of each thermal power unit, obtain the dispatching strategy for each dispatching period, and complete the intraday dispatching of one day; The subsequent scheduling module is used to update the scheduling strategy of the trained reinforcement learning agent after completing the intraday scheduling of the day, and control the scheduling module of the day to perform the intraday scheduling of a new day.

Images