Low-carbon operation optimization method for optical storage charging station containing hybrid energy storage
By introducing a hybrid system of flywheel energy storage and battery energy storage in the optical storage charging station and using improved near-end strategy optimization algorithms, the problems of supply and demand mismatch and instability of strategy updates during peak periods are solved. The system's scheduling capabilities and learning efficiency have been improved, and efficient low-carbon operation has been achieved.
Patent Information
- Application Number
- CN202410949364.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-16
- Publication Date
- 2024-11-08
AI Technical Summary
Due to the small energy storage power density of traditional optical storage charging stations, they cannot meet the needs of peak charging periods of electric vehicles, and the existing reinforcement learning algorithms have problems such as unstable policy updates and low training efficiency when optimizing scheduling.
Adopt a hybrid energy storage system with flywheel energy storage and battery energy storage, combined with the improved near-end strategy optimization algorithm of step-type punishment coefficients, build a reinforced learning interactive environment, optimize the low-carbon operation scheduling model of optical storage charging stations, and improve the system scheduling Ability and learning efficiency.
It realizes providing high output power during peak periods, meeting user needs, reducing grid pressure, improving equipment utilization, and reducing agent training time through improved algorithms to enhance system robustness and convergence.
Smart Images

Figure CN118920534A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of optimized scheduling of electric vehicle charging stations, and in particular to a low-carbon operation optimization method for a photovoltaic charging station containing hybrid energy storage. Background Art
[0002] Traditional photovoltaic charging stations mostly use battery energy storage, which has the characteristics of large storage capacity, long storage time but low power density. With the rapid development of electric vehicles, traditional photovoltaic charging stations are limited by the grid transmission power and battery energy storage power density, and cannot meet the needs of users during peak charging periods. To address this problem, the conventional solution is to increase the output power of the charging station by reconstructing grid-connected facilities and increasing the storage capacity, so as to meet the load demand during the short charging peak period at the cost of increasing investment and reducing equipment utilization. Flywheel energy storage, as a storage method with extremely high power density, is widely used in large power grid system frequency regulation. Applying it to electric vehicle charging stations can use its extremely high output power to increase the output power of the charging station in a short time, and achieve a balance between supply and demand during charging peak periods without increasing the capacity of the power grid and energy storage system.
[0003] In addition, the uncertainty of the source and load of the photovoltaic charging station makes it difficult to solve the system optimization. To address this problem, many existing studies have adopted reinforcement learning algorithms to solve it, and use its excellent training and learning mechanism to find the global optimal strategy. However, traditional reinforcement learning algorithms, such as the Advantage Actor Critic (A2C) algorithm, cannot stably update the strategy due to the variance of sample estimation and the sensitivity of hyperparameter selection. To solve this problem, the Proximal Policy Optimization (PPO) algorithm introduces importance sampling technology and step-size clipping mechanism to ensure the monotonic improvement of strategy performance during model training, while also reducing the variance in the policy gradient algorithm, thereby improving the stability of learning and avoiding the defects of the A2C algorithm. However, the classic PPO algorithm requires that there should not be too large a difference between the new and old strategies when updating the strategy. Although this constraint can ensure the stability of performance, it also limits the pace and efficiency of strategy updates, resulting in low training efficiency. At the same time, the guiding effect of its reward function during training is limited, and it takes a long time of training to obtain an excellent intelligent agent. The reason is that the traditional fixed penalty coefficient cannot give full play to the role of the penalty function, cannot amplify the differences between similar actions, and guide the intelligent agent to identify the optimal strategy. Summary of the invention
[0004] The present invention proposes a hybrid energy storage system containing flywheel energy storage and battery energy storage, which improves the dispatching capacity of the integrated photovoltaic storage and charging station to meet the charging needs of users during the peak charging period. At the same time, in order to solve the problem that the optimization dispatching algorithm of the photovoltaic storage charging station in the prior art may not be able to stably update the strategy, cannot distinguish similar strategies well, resulting in limited guidance effect and poor optimization effect of the intelligent agent strategy, a low-carbon operation optimization method of the photovoltaic storage charging station containing hybrid energy storage is provided.
[0005] The above technical problem of the present invention is mainly solved by the following technical solution: a low-carbon operation optimization method of a photovoltaic storage charging station with hybrid energy storage, comprising the following steps: S1. Establish a hybrid energy storage system model including a flywheel energy storage system and a battery energy storage system; S2. Establish the objective function of the low-carbon optimization scheduling model of photovoltaic and energy storage charging stations; S3. Setting constraints for the low-carbon optimization scheduling model of photovoltaic and energy storage charging stations; S4. Based on the objective function and constraints, a reinforcement learning interactive environment is constructed, and the improved proximal strategy optimization algorithm combined with a step-by-step penalty coefficient is used to solve the low-carbon optimization scheduling model of the photovoltaic storage charging station to obtain the optimal scheduling strategy.
[0006] The present invention establishes a hybrid energy storage system model containing a flywheel energy storage system to construct a low-carbon optimization scheduling model for a photovoltaic charging station, then establishes a reinforcement learning interactive environment, and finally uses an improved proximal strategy optimization algorithm combined with a step-by-step penalty coefficient to solve the low-carbon optimization scheduling model for a photovoltaic charging station to obtain a global optimal scheduling strategy. The present invention improves the scheduling capability of the system during peak charging periods by establishing a hybrid energy storage model containing flywheel energy storage, establishes a low-carbon optimization objective function containing abandoned light and carbon trading costs, improves the photovoltaic absorption rate of photovoltaic charging stations and reduces the total carbon emissions of the system, and uses an improved proximal strategy optimization algorithm to solve the optimization scheduling model for photovoltaic charging stations, enhances learning efficiency, reduces the training time of intelligent agents, prevents training instability caused by excessive exploration, thereby improving the convergence of the algorithm, and controls the possibility of taking high-risk behaviors, thereby improving the robustness of the system.
[0007] The hybrid energy storage system model includes a flywheel energy storage system and a battery energy storage system. The flywheel energy storage system is a mechanical energy storage method with the advantages of high power density, high charge and discharge frequency, and fast response speed. The disadvantage is that the power loss is large and it is only suitable for short-term energy storage. It can provide power for the power station for a short time during the peak load period to meet user needs. The battery energy storage system is a chemical energy storage method with the advantages of high energy density and large capacity, but it has the disadvantage of limited unit output power. It can ensure long-term storage of electric energy, shave peaks and fill valleys, improve wind and solar power consumption, and reduce operating costs. The hybrid energy storage system model established by the present invention combines the advantages and disadvantages of flywheel energy storage and battery energy storage. It is based on flywheel energy storage to bear the peak load of the charging station to ensure the balance of source and load supply and demand when the charging load increases sharply; it is based on large-capacity battery energy storage to bear the basic load of the charging station to ensure the full consumption of photovoltaics. The energy time shifting of the hybrid energy storage system is used to achieve the requirements of ensuring consumption in the valley and supply in the peak of the photovoltaic storage charging station.
[0008] The flywheel energy storage system of the hybrid energy storage photovoltaic charging station of the present invention can temporarily supply high output power and provide more charging services during peak demand periods, thereby reducing the pressure on the power grid, improving the utilization rate of equipment, and increasing the operating income of the system; at the same time, the improved proximal strategy optimization algorithm can enhance learning efficiency and reduce the training time of the intelligent body, prevent training instability caused by excessive exploration, thereby improving the convergence of the algorithm, and control the possibility of taking high-risk behaviors, thereby improving the robustness of the system.
[0009] As a preferred solution, step S2 specifically includes: Taking into account the environmental benefits of carbon emission costs, the objective function is established with the goal of maximizing the total revenue of the charging station within a scheduling cycle T, which is generally 24 hours. The total revenue includes electricity sales revenue, electricity purchase cost, carbon emission cost, capacity electricity fee and operation and maintenance cost.
[0010] The objective function of the low-carbon optimization scheduling model of photovoltaic charging stations is a scheduling cycle, which is generally the total revenue of 24 hours a day. The total revenue includes the total synthesis of electricity sales revenue, electricity purchase cost, carbon emission cost, capacity electricity fee and operation and maintenance cost.
[0011] The objective function not only takes into account economic benefits such as operation and maintenance costs, but also environmental benefits such as carbon emission costs. The optimal economic benefits, i.e., revenue rewards, are obtained by adjusting the scheduling strategies in different time periods. The specific objective function is: maxE=E EV +E G -C G -C C -C R -C PV -C BS -C FS Where, E is the daily revenue of the charging station, E EVE is the revenue from selling electricity to EVs. G is the revenue from selling electricity to the grid, C G is the cost of purchasing electricity from the grid for the charging station, C C is the CO2 emission cost of the charging station, C R is the capacity electricity fee, C PV , C BS , C FS The operation and maintenance costs of photovoltaic and energy storage are respectively. The time-of-use electricity price is used to calculate the revenue from selling electricity to EVs.
[0012] As a preferred solution, the constraints constructed in step S3 include: energy storage constraints, power balance constraints, and power exchange constraints between the charging station and the power grid.
[0013] As a preferred solution, the step S4 specifically includes: S4.1. Obtain source-load data through the CNN_BIGRU_ATTENTION prediction model to form a charging station operation scenario; to meet the requirement of reinforcement learning training for a large amount of source-load data.
[0014] S4.2. Design action space and state space based on optimal scheduling decision quantities and constraints; S4.3. Design a reward function of the reinforcement learning algorithm based on the objective function and the constraint conditions, wherein the reward function includes a benefit reward and a penalty reward, and the penalty reward is constructed according to a designed step-by-step penalty coefficient; By obtaining the four-tuple data consisting of the current period state, action, reward function and the next period state, the four-tuple data is used to build a reinforcement learning interactive environment, and the four-tuple data is used as the basic data for training and learning.
[0015] S4.4. An improved proximal strategy optimization algorithm combined with a step-by-step penalty coefficient is used to solve the low-carbon optimization scheduling model of photovoltaic charging stations and output the global optimal strategy.
[0016] As a preferred solution, the specific process of step S4.1 includes: S4.1.1. Use Copula function to select features, and select key features that determine photovoltaic output and load size by comparing correlations; the Copula function can describe the nonlinear relationship between features and prediction results. Compared with the traditional Pearson correlation coefficient feature selection method, the Copula function can better perform feature selection.
[0017] S4.1.2, use K-means algorithm to perform clustering and classify different meteorological modes; S4.1.3. Use the CNN-BIGRU-ATTENTION combined neural network model for prediction.
[0018] As a preferred solution, the CNN-BIGRU-ATTENTION combined neural network model includes a CNN module, a BIGRU module and an ATTENTION module. The CNN module includes a Conv2D neural network to extract features of input data, and is connected to the BIGRU module through a batch normalization layer, a leaky corrected linear activation function unit LeakyRelu and a deformation layer Reshape. The BIGRU module is connected to the ATTENTION module, and the ATTENTION module outputs prediction results. The BIGRU module can effectively process time series data, capture context information in both directions, and mine the time correlation of input features; the ATTENTION module can focus on the key features of the input to improve the performance of the model, and use the data obtained from the previous module to generate prediction results to achieve high-quality predictions.
[0019] As a preferred solution, the state space, action space and reward function are designed based on optimizing the scheduling decision quantity, constraints and objectives; the scheduling decision quantity is the scheduling means, including energy storage, electricity purchase, etc.
[0020] The state space is composed of the predicted photovoltaic power generation power in period t, the electric vehicle charging power, the maximum photovoltaic power generation output, the electricity price sold by the power station to EVs, the electricity price sold to the grid, the electricity purchase price of the charging station, the carbon emission factor, the 0-1 variable of the charging and discharging state of the battery energy storage and flywheel energy storage, the 0-1 variable of the power station's purchase and sale of electricity to the grid, the charge state of the battery energy storage and flywheel energy storage, and the charge state of the battery energy storage and flywheel energy storage in period t-1; The action space is composed of the actual photovoltaic consumption in period t, the power purchase and sale of electricity from the power grid, and the charging and discharging power of the power storage and flywheel energy storage.
[0021] The reward function is the sum of the income reward and the penalty reward. The penalty reward uses a step-by-step penalty coefficient to weight and calculate the photovoltaic consumption exceeding the limit and the constraint condition exceeding the limit. The constraint condition exceeding the limit includes the capacity of the power storage and flywheel energy storage exceeding the limit in the t period, the inconsistency between the initial and final states of the power storage and flywheel energy storage exceeding the limit, and the electric power balance exceeding the limit in the t period.
[0022] By limiting the constraints through penalty rewards and constructing penalty rewards by designing step-by-step penalty coefficients, the intelligent agent can be guided to learn and train better and reduce training time.
[0023] As a preferred solution, the step-by-step penalty coefficient in step S4.3 is specifically expressed as follows: Where n represents different constraints, n∈{BS', BS, FS', FS, e, PV}, BS' and BS are the initial and final state consistency constraints and capacity constraints of electric energy storage, FS' and FS are the initial and final state consistency constraints and capacity constraints of flywheel energy storage, e is the electric power balance constraint, PV is the photovoltaic output constraint, θ n is the penalty coefficient for the nth type of constraint, a n , b n 、c n d n is the boundary of the nth type of constraint violation interval, C n is the over-limit value of each constraint, β n1 , β n2 , β n3 , β n4 is the penalty coefficient for the nth type of constraint that exceeds the limit value within each interval.
[0024] As a preferred solution, based on the constructed reinforcement learning interactive environment, the proximal strategy optimization algorithm is used to train the intelligent agent, so that the intelligent agent can learn by trial and error during the interaction with the environment, obtain the trained intelligent agent, input real-time data online, and output the global optimal strategy.
[0025] In the optimization objective function of this scheme, an improved proximal optimization algorithm is used to solve it, and the model with the highest cumulative reward value is found for online application to obtain the best scheduling strategy.
[0026] As a preferred solution, the proximal strategy optimization algorithm is used to train the agent. The specific process includes: S4.4.1, initializing various hyperparameters of the system, and new , the old policy network η old And the value network is initialized, the number of periodic time periods T, the limit of the number of synchronous updates M and the limit of the number of synchronous rounds N are set, and the number of synchronous updates m = 1; S4.4.2, let n = 1, n is the number of synchronization rounds; S4.4.3, let t = 1, t is the time period, and initialize the state; S4.4.4. According to the state s of period t t , action a is determined by the old policy network t , and the action a t Act on the environment and obtain the reward function r t and the environmental state s in the next period t+1 , forming a quaternion <s t ,a t ,r t ,s t+1 >, and the quaternion <s t ,a t ,r t,s t+1 >Put it into the sample pool; S4.4.5. Determine whether a round of training is completed. If so, update the value network parameters according to the sample data, and then update the new strategy network η new , proceed to the next step; if not, then t=t+1, return to step S4.4.4; S4.4.6, determine whether the number of synchronization rounds reaches the synchronization round limit, if so, the new strategy network η new The parameters of the old policy network η are updated synchronously old , proceed to the next step; if not, then n=n+1, return to step S4.4.3; S4.4.7. Determine whether the number of synchronous updates reaches the synchronous update limit. If so, stop training; if not, m=m+1, and return to step S4.4.2.
[0027] The elements of reinforcement learning include the intelligent agent and the interactive environment. The environment provides initial observation information, and the intelligent agent takes action based on the initial observation information and transmits it to the interactive environment. The interactive environment generates the environmental state for the next period based on the action, and returns the reward generated by the previous action and the state of the next period to the intelligent agent. The intelligent agent takes the next action based on the reward and the state of the next period.
[0028] The advantages of the present invention are: By establishing a hybrid energy storage system model with a flywheel energy storage system, a low-carbon optimization scheduling model for photovoltaic charging stations is constructed, and then a reinforcement learning interactive environment is established. Finally, an improved proximal strategy optimization algorithm combined with a step-by-step penalty coefficient is used to solve the low-carbon optimization scheduling model for photovoltaic charging stations and obtain the global optimal scheduling strategy.
[0029] By establishing a hybrid energy storage model with flywheel energy storage, the dispatching capacity of the system during peak charging periods is improved, a low-carbon optimization objective function including abandoned light and carbon trading costs is established, the photovoltaic absorption rate of the photovoltaic storage charging station is improved and the total carbon emissions of the system are reduced, and an improved proximal strategy optimization algorithm is used to solve the optimal scheduling model of the photovoltaic storage charging station, enhance learning efficiency, reduce the training time of the intelligent agent, prevent training instability caused by excessive exploration, thereby improving the convergence of the algorithm, and control the possibility of taking high-risk behaviors, thereby improving the robustness of the system. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 It is a schematic diagram of a process of the present invention.
[0031] Figure 2 It is a flow chart of obtaining source load data prediction based on the CNN_BIGRU_ATTENTION prediction model in the present invention.
[0032] Figure 3 It is a schematic diagram of the principle of solving the low-carbon optimization scheduling model of the photovoltaic storage charging station based on the improved proximal strategy optimization algorithm in the present invention.
[0033] Figure 4 It is a flow chart of solving the low-carbon optimization scheduling model of the photovoltaic storage charging station based on the improved proximal strategy optimization algorithm in the present invention.
[0034] Figure 5 It is a schematic diagram of the principle of the CNN-BIGRU-ATTENTION prediction model in the present invention. DETAILED DESCRIPTION
[0035] The technical solution of the present invention is further specifically described below through embodiments and in conjunction with the accompanying drawings.
[0036] Example: This embodiment provides a low-carbon operation optimization method for a photovoltaic charging station with hybrid energy storage, such as Figure 1 As shown, the following steps are included: S1. Establish a hybrid energy storage system model including a flywheel energy storage system and a battery energy storage system; S2. Establish the objective function of the low-carbon optimization scheduling model of photovoltaic and energy storage charging stations; S3. Setting constraints for the low-carbon optimization scheduling model of photovoltaic and energy storage charging stations; S4. Based on the objective function and constraints, a reinforcement learning interactive environment is constructed, and the improved proximal strategy optimization algorithm combined with a step-by-step penalty coefficient is used to solve the low-carbon optimization scheduling model of the photovoltaic storage charging station to obtain the optimal scheduling strategy.
[0037] The present invention firstly establishes a hybrid energy storage system model including a flywheel energy storage system to construct a low-carbon optimization scheduling model for a photovoltaic charging station, then establishes a reinforcement learning interactive environment, and finally adopts an improved proximal strategy optimization algorithm combined with a step-by-step penalty coefficient to solve the low-carbon optimization scheduling model for a photovoltaic charging station, and obtains a global optimal scheduling strategy. The hybrid energy storage model including flywheel energy storage is established to improve the scheduling capacity of the system during the peak charging period, a low-carbon optimization objective function including abandoned light and carbon trading costs is established, the photovoltaic absorption rate of the photovoltaic charging station is improved, and the total carbon emissions of the system are reduced. The improved proximal strategy optimization algorithm is used to solve the optimization scheduling model for the photovoltaic charging station, enhance learning efficiency, reduce the training time of the intelligent body, prevent training instability caused by excessive exploration, thereby improving the convergence of the algorithm, and control the possibility of taking high-risk behaviors, thereby improving the robustness of the system.
[0038] As a preferred solution of this embodiment, a hybrid energy storage system model including a flywheel energy storage system and a battery energy storage system is established in step S1, wherein the flywheel energy storage system is a mechanical energy storage method, which has the advantages of high power density, high charging and discharging frequency and fast response speed, but has the disadvantage of large power loss and is only suitable for short-term energy storage. It can provide power for the power station for a short time during the peak load period to meet user needs. The battery energy storage system is a chemical energy storage method, which has the advantages of high energy density and large capacity, but has the disadvantage of limited unit output power. It can ensure long-term storage of electric energy, shaving peaks and filling valleys, improving wind and solar power consumption, and reducing operating costs. The hybrid energy storage system model combines the advantages and disadvantages of flywheel energy storage and battery energy storage. Based on the flywheel energy storage, it bears the peak load of the charging station to ensure the balance of source and load supply and demand when the charging load increases sharply; based on the large-capacity battery energy storage, it bears the basic load of the charging station to ensure the full absorption of photovoltaics. The energy time shifting of the hybrid energy storage system realizes the requirements of ensuring the absorption of photovoltaic charging stations in the valley and ensuring the supply in the peak.
[0039] The specific formula of the hybrid energy storage system model is as follows: s c (t)+s d (t)≤1 f c (t)+f d (t)≤1 SOC fin =SOC0 FOC fin =FOC0 SOC min ≤SOC(t)≤SOC max FOC min ≤FOC(t)≤FOC max In the formula, the superscripts c and d represent charging and discharging, respectively, and the subscripts BS and FS represent battery energy storage and flywheel energy storage, respectively. are the charging and discharging power of the storage in period t, They are respectively the charge and discharge power of the flywheel energy storage during period t, is the rated capacity of the storage, are the maximum charge and discharge power of battery energy storage, are the upper and lower limits of the flywheel energy storage charging and discharging power, s c (t), s d (t), f c (t), f d(t) are 0-1 variables of battery energy storage and flywheel energy storage in period t, indicating the charging and discharging state of energy storage, η BS c , η BS d , η FS c , η FS d are the charging and discharging efficiencies of battery energy storage and flywheel energy storage, SOC0 and SOC fin They are the charge states of the battery energy storage at the beginning and end of the day, FOC0, FOC fin They are the charge states of the flywheel energy storage at the beginning and end of the day, SOC(t) and FOC(t) are the charge states of the battery energy storage and flywheel energy storage at time t, respectively. max , SOC min 、FOC max 、FOC min They are the maximum and minimum charge states of battery energy storage and flywheel energy storage, respectively. The charge and discharge power of battery energy storage is constrained by the rated capacity of the bidirectional power converter, and the charge and discharge of flywheel energy storage is limited by its rated capacity. The SOC and FOC states must be consistent from beginning to end.
[0040] As a preferred solution of this embodiment, the specific process of step S2 includes: considering the environmental benefits of carbon emission costs, establishing an objective function with the goal of maximizing the total revenue of the charging station, and the total revenue includes electricity sales revenue, electricity purchase costs, carbon emission costs, capacity electricity charges and operation and maintenance costs.
[0041] The objective function of the above-mentioned low-carbon optimization scheduling model of photovoltaic charging stations is a scheduling cycle, which is generally the total revenue of 24 hours a day. The total revenue includes the total synthesis of electricity sales revenue, electricity purchase cost, carbon emission cost, capacity electricity fee and operation and maintenance cost. The objective function not only takes into account economic benefits such as operation and maintenance costs, but also environmental benefits such as carbon emission costs. The optimal economic benefits, i.e., revenue rewards, are obtained by adjusting the scheduling strategies in different time periods. The specific objective function is: maxE=E EV +E G -C G -C C -C R -C PV -C BS -C FS Where, E is the daily revenue of the charging station, E EV E is the revenue from selling electricity to EVs. G is the revenue from selling electricity to the grid, C G is the cost of purchasing electricity from the grid for the charging station, C C is the CO2 emission cost of the charging station, C R is the capacity electricity fee, CPV , C BS , C FS The operation and maintenance costs of photovoltaic and energy storage are respectively. The time-of-use electricity price is used to calculate the revenue from selling electricity to EV. Specifically, the revenue from selling electricity is: In the formula, θ EV (t) is the electricity price sold by the power station to EV during period t, θ G (t) is the electricity price sold to the power grid during period t, P EV (t) is the EV charging power during period t, P G- (t) is the power delivered by the charging station to the grid during period t.
[0042] Electricity purchase cost: Where θ(t) is the electricity purchase price of the charging station during period t, P G+ (t) is the power obtained from the grid during period t.
[0043] Carbon emission costs: In the formula, θ R is the unit capacity electricity fee, It is the maximum value of the total average load of the charging station every 30 minutes in a month.
[0044] Capacity electricity fee: In the formula, θ R is the unit capacity electricity fee, It is the maximum value of the total average load of the charging station every 30 minutes in a month.
[0045] Operation and maintenance costs: In the formula, θ PP ,θ E ,θ BB ,θ FF They are the photovoltaic unit power operation and maintenance cost, battery energy storage unit capacity operation and maintenance cost, flywheel energy storage unit capacity operation and maintenance cost, battery energy storage unit electricity dynamic cost, flywheel energy storage unit electricity dynamic cost, is the rated capacity of the photovoltaic unit, is the rated capacity of the flywheel energy storage, r p , r', r" are the discount rates of photovoltaic, battery energy storage and flywheel energy storage equipment, y, y', y" are the service life of photovoltaic, battery energy storage and flywheel energy storage equipment respectively.
[0046] As a preferred solution of this embodiment, in step S3, the constraints of the low-carbon optimization scheduling model of the photovoltaic charging station are set, and the constraints include energy storage constraints, power balance constraints, and power exchange constraints between the charging station and the power grid. Specifically, energy storage constraints: The energy storage constraints are the model constraints of the hybrid energy storage system.
[0047] Power balance constraints: Photovoltaic power generation constraints: In the formula, It is the maximum output of photovoltaic power generation in period t, that is, the predicted power of photovoltaic power generation.
[0048] Power exchange constraints between charging station and grid: u G+ (t)+u G- (t)≤1 In the formula, u G+ (t),u G- (t) is a 0-1 variable representing the purchase and sale of electricity from the power grid by the charging station during period t. is the maximum exchange power between the charging station and the grid.
[0049] As a preferred solution of this embodiment, step S4 specifically includes the following process: S4.1. Obtain source-load data through the CNN_BIGRU_ATTENTION prediction model to form a charging station operation scenario to meet the requirement of reinforcement learning training for a large amount of source-load data.
[0050] Further, such as Figure 2 As shown, step S4.1 specifically includes: S4.1.1. Use Copula function for feature selection, and select key features that determine photovoltaic output and load size by comparing correlations; Copula function can describe the nonlinear relationship between features and prediction results. Compared with the traditional Pearson correlation coefficient feature selection method, Copula function can better perform feature selection; S4.1.2, use K-means algorithm to perform clustering and classify different meteorological modes; S4.1.3. Use the CNN-BIGRU-ATTENTION combined neural network model for prediction.
[0051] The designed CNN-BIGRU-ATTENTION combined neural network prediction model includes three main modules: CNN, BIGRU and ATTENTION. Figure 5 As shown in the figure, the CNN module uses the Conv2D neural network to extract the features of the input data and is connected to the BIGRU module through the batch normalization layer, the leakage corrected linear activation function unit LeakyRelu, and the deformation layer Reshape. The BIGRU module is connected to the ATTENTION module; the BIGRU module can effectively process time series data, capture context information in both directions, and mine the time correlation of input features; the ATTENTION module can focus on the key features of the input to improve the performance of the model, and use the data obtained from the previous modules to generate prediction results to achieve high-quality predictions.
[0052] S4.2. Design the action space and state space based on the optimal dispatch decision quantity and constraints; the dispatch decision quantity is the dispatch means, including energy storage, electricity purchase, etc.
[0053] The state space is based on the photovoltaic power generation prediction power in period t Electric vehicle charging power P EV (t), maximum photovoltaic power generation output The price of electricity sold by the power station to EVs θ EV (t), the price of electricity sold to the grid θ G (t), the electricity purchase price of the charging station θ(t), the carbon emission factor λ(t), the 0-1 variable s of the charging and discharging state of the battery energy storage and flywheel energy storage c (t), s d (t), fc(t), fd(t), 0-1 variable u for power purchase and sale from the power grid G+ (t),u G- (t), the state of charge SOC(t) and FOC(t) of the battery storage and flywheel energy storage, and the state of charge SOC(t-1) and FOC(t-1) of the battery storage and flywheel energy storage in the period t-1. The specific representation of the state space is as follows: The action space is composed of the actual photovoltaic consumption in period t, the power purchase and sale of electricity from the power grid, and the charging and discharging power of the power storage and flywheel energy storage. The specific expression is as follows:
[0054] S4.3. Designing the Reward Function r of the Reinforcement Learning Algorithm Based on the Objective Function and Constraints t ,The reward function includes the revenue reward and the penalty reward, and the penalty reward is constructed by designing a step-by-step penalty coefficient.
[0055] In this embodiment, the reward function is the sum of the income reward and the penalty reward. The penalty reward is a step-by-step penalty coefficient for weighted sum calculation of the photovoltaic absorption capacity exceeding the limit and the constraint condition exceeding the limit. The constraint condition exceeding the limit includes the capacity of the electric storage and flywheel energy storage exceeding the limit in the t period, the inconsistent state of the electric storage and flywheel energy storage exceeding the limit, and the electric power balance exceeding the limit in the t period. Specifically, the photovoltaic absorption capacity exceeding the limit in the t period C PV (t): The capacity of the electric storage and flywheel energy storage exceeds the limit C during period t BS (t), C FS (t): C BS (t) = -f(SOC max ,SOC min ,SOC(t)) C FS (t) = -f(FOC max ,FOC min ,FOC(t)) In the formula, f(x, y, z) is the function for calculating the degree of constraint violation, which is specifically expressed as follows: f(x,y,z)=(|xz|+|yz|-|yx|) / 2 In the formula, z is the constrained quantity, and x and y are the upper and lower bounds of the constraint quantity. Obviously, when z∈[x,y], f(x,y,z)=0, and When , f(x,y,z) reflects the degree of violation of the constraint z: the more z exceeds the boundary range, the larger the f(x,y,z) value is, indicating that the constraint violation is more serious.
[0056] The initial and final states of the electric storage and flywheel energy storage are inconsistent and exceed the limit C BS' , C FS' : C BS' =-(SOC fin -SOC0) C FS' =-(FOC fin -FOC0) The electric power balance exceeds the limit during period t: The reward function formula is as follows: Where θ PV ,θ BS ,θ FS ,θ BS' ,θ FS' ,θ e is the penalty coefficient for different constraints; θPV C PV (t),θ BS C BS (t),θ FS C FS (t),θ BS' C BS' ,θ FS' C FS' ,θ e C e (t) is the penalty function of different constraints.
[0057] By creating an action a t , status t , reward function r t and the next period state s t+1 The quadruple data is used to construct the reinforcement learning interactive environment, and the quadruple data is used as the basic data for training and learning.
[0058] As a preferred solution of this embodiment, the specific process of step S4.4 includes: based on the constructed reinforcement learning interactive environment, the improved proximal strategy optimization algorithm combined with a step-by-step penalty coefficient is used to train the intelligent agent, so that the intelligent agent learns by trial and error during the interaction with the environment, obtains the trained intelligent agent, inputs real-time data online, and outputs the global optimal strategy.
[0059] The step-by-step penalty coefficient is specifically expressed as follows: Where n represents different constraints, n∈{BS', BS, FS', FS, e, PV}, BS' and BS are the initial and final state consistency constraints and capacity constraints of electric energy storage, FS' and FS are the initial and final state consistency constraints and capacity constraints of flywheel energy storage, e is the electric power balance constraint, PV is the photovoltaic output constraint, θ n is the penalty coefficient for the nth type of constraint, a n , b n 、c n d n is the boundary of the nth type of constraint violation interval, C n is the over-limit value of each constraint, β n1 , β n2 , β n3 , β n4 is the penalty coefficient for the nth type of constraint that exceeds the limit value within each interval.
[0060] The above-mentioned proximal strategy optimization algorithm is used to train the intelligent agent containing the low-carbon optimization scheduling model of the photovoltaic storage charging station, such as Figure 4 As shown, the specific steps include: S4.4.1. Initialize various hyperparameters of the system and initialize the new strategy network η new , the old policy network η old And the value network is initialized, the number of periodic time periods T, the limit of the number of synchronous updates M and the limit of the number of synchronous rounds N are set, and the number of synchronous updates m = 1; S4.4.2, let n = 1, n is the number of synchronization rounds; S4.4.3, let t = 1, t is the time period, and initialize the state; S4.4.4. According to the state s of period t t , action a is determined by the old policy network t , and the action a t Act on the environment and obtain the reward function r t and the environmental state s in the next period t+1 , forming a quaternion <s t ,a t ,r t ,s t+1 >, and convert the quaternion t ,a t ,r t ,s t+1 >Put it into the sample pool to facilitate subsequent training of the agent parameters; S4.4.5. Determine whether a round of training t≥T is completed. If so, update the value network parameters according to the sample data, and then update the new strategy network η new , proceed to the next step; if not, then t=t+1, return to step S4.4.4; S4.4.6. Determine whether the number of synchronization rounds reaches the synchronization round limit n ≥ N. If so, set the new strategy network η new The parameters of the old policy network η are updated synchronously old , proceed to the next step; if not, then n=n+1, return to step S4.4.3; S4.4.7. Determine whether the number of synchronous updates reaches the synchronous update limit m≥M. If so, stop training; if not, m=m+1, and return to step S4.4.2.
[0061] The reinforcement learning elements of this scheme include two parts: intelligent agent and interactive environment, such as Figure 3 As shown in Figure 1, after the environment gives the initial observation information, the agent makes corresponding actions. After the action is input into the environment, the environment gives the agent rewards and the state of the environment at the next moment. After obtaining the training sample, the value network updates the parameters, and the new strategy network η new Update the parameters. After a certain batch of updates, the new strategy network η new Synchronize parameters to the old policy network η old The updated old policy network ηold Continue to interact with the environment to obtain training samples.
[0062] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.
[0063] Although this article uses more terms such as hybrid energy storage system model, low-carbon optimization scheduling model of photovoltaic storage charging station, step-by-step penalty coefficient, improved proximal strategy optimization algorithm, etc., it does not exclude the possibility of using other terms. These terms are used only to more conveniently describe and explain the essence of the present invention; interpreting them as any additional restrictions is contrary to the spirit of the present invention.
Claims
1. A low-carbon operation optimization method for a photovoltaic charging station with hybrid energy storage, characterized in that: The following steps are involved: S1. Establish a hybrid energy storage system model including a flywheel energy storage system and a battery energy storage system; S2. Establish the objective function of the low-carbon optimization scheduling model of photovoltaic and energy storage charging stations; S3. Setting constraints for the low-carbon optimization scheduling model of photovoltaic and energy storage charging stations; S4. Based on the objective function and constraints, a reinforcement learning interactive environment is constructed, and the improved proximal strategy optimization algorithm combined with a step-by-step penalty coefficient is used to solve the low-carbon optimization scheduling model of the photovoltaic storage charging station to obtain the optimal scheduling strategy.
2. According to claim 1, a method for optimizing low-carbon operation of a photovoltaic charging station with hybrid energy storage is characterized in that The step S2 specifically includes: Considering the environmental benefits of carbon emission costs, an objective function is established with the goal of maximizing the total revenue of the charging station. The total revenue includes electricity sales revenue, electricity purchase cost, carbon emission cost, capacity electricity fee and operation and maintenance cost.
3. The low-carbon operation optimization method of a photovoltaic charging station with hybrid energy storage according to claim 1 is characterized in that The constraints constructed in step S3 include: energy storage constraints, power balance constraints, and power exchange constraints between the charging station and the power grid.
4. A low-carbon operation optimization method for a photovoltaic charging station with hybrid energy storage according to claim 1, 2 or 3, characterized in that The specific process of step S4 includes: S4.
1. Obtain source-load data through the CNN_BIGRU_ATTENTION prediction model to form a charging station operation scenario; S4.
2. Design action space and state space based on optimal scheduling decision quantities and constraints; S4.
3. Design a reward function of the reinforcement learning algorithm based on the objective function and the constraint conditions, wherein the reward function includes a benefit reward and a penalty reward, and the penalty reward is constructed according to a designed step-by-step penalty coefficient; S4.
4. An improved proximal strategy optimization algorithm combined with a step-by-step penalty coefficient is used to solve the low-carbon optimization scheduling model of photovoltaic charging stations and output the global optimal strategy.
5. The low-carbon operation optimization method of a photovoltaic charging station with hybrid energy storage according to claim 4 is characterized in that The specific process of step S4.1 includes: S4.1.
1. Use Copula function to select features and select key features that determine photovoltaic output and load size by comparing relevance; S4.1.2, use K-means algorithm to perform clustering and classify different meteorological modes; S4.1.
3. Use the CNN-BIGRU-ATTENTION combined neural network model for prediction.
6. The low-carbon operation optimization method of a photovoltaic charging station with hybrid energy storage according to claim 4 is characterized in that: Design state space, action space and reward function based on optimal scheduling decisions, constraints and goals; The state space is composed of the predicted photovoltaic power generation power in period t, the electric vehicle charging power, the maximum photovoltaic power generation output, the electricity price sold by the power station to EVs, the electricity price sold to the grid, the electricity purchase price of the charging station, the carbon emission factor, the 0-1 variable of the charging and discharging state of the battery energy storage and flywheel energy storage, the 0-1 variable of the power station's purchase and sale of electricity to the grid, the charge state of the battery energy storage and flywheel energy storage, and the charge state of the battery energy storage and flywheel energy storage in period t-1; The action space is composed of the actual photovoltaic consumption in period t, the power purchase and sale of electricity from the power grid, and the charging and discharging power of the power storage and flywheel energy storage; The reward function includes profit reward and penalty reward. The penalty reward uses a step-by-step penalty coefficient to weight and calculate the excessive photovoltaic consumption and the excessive constraints. The excessive constraints include the excessive capacity of the power storage and flywheel energy storage in the t period, the inconsistent initial and final states of the power storage and flywheel energy storage, and the excessive power balance in the t period.
7. The low-carbon operation optimization method of a photovoltaic charging station with hybrid energy storage according to claim 4 is characterized in that The step-by-step penalty coefficient in step S4.3 is specifically expressed as follows: Where n represents different constraints, n∈{BS', BS, FS', FS, e, PV}, BS' and BS are the initial and final state consistency constraints and capacity constraints of electric energy storage, FS' and FS are the initial and final state consistency constraints and capacity constraints of flywheel energy storage, e is the electric power balance constraint, PV is the photovoltaic output constraint, θ n is the penalty coefficient for the nth type of constraint, a n , b n , c n , d n is the boundary of the nth type of constraint violation interval, C n is the over-limit value of each constraint, β n1 , β n2 , β n3 , β n4 is the penalty coefficient for the nth type of constraint that exceeds the limit value within each interval.
8. The low-carbon operation optimization method of a photovoltaic charging station with hybrid energy storage according to claim 4 is characterized in that Step S4.4 specifically includes: Based on the constructed reinforcement learning interactive environment, the proximal strategy optimization algorithm is used to train the intelligent agent, obtain the trained intelligent agent, input real-time data online, and output the global optimal strategy.
9. The low-carbon operation optimization method of a photovoltaic charging station with hybrid energy storage according to claim 8 is characterized in that The aforementioned use of the proximal strategy optimization algorithm to train the agent includes the following specific processes: S4.4.
1. Initialize various hyperparameters of the system and initialize the new strategy network η new , the old policy network η old And the value network is initialized, the number of periodic time periods T, the limit of the number of synchronous updates M and the limit of the number of synchronous rounds N are set, and the number of synchronous updates m = 1; S4.4.2, let n = 1, n is the number of synchronization rounds; S4.4.3, let t = 1, t is the time period, and initialize the state; S4.4.
4. According to the state s of period t t , action a is determined by the old policy network t , and the action a t Act on the environment and obtain the reward function r t and the environmental state s in the next period t+1 , forming a quaternion <s t ,a t ,r t ,s t+1 >, and the quaternion <s t ,a t ,r t ,s t+1 >Put it into the sample pool; S4.4.
5. Determine whether a round of training is completed. If so, update the value network parameters according to the sample data, and then update the new strategy network η new , proceed to the next step; if not, then t=t+1, return to step S4.4.4; S4.4.6, determine whether the number of synchronization rounds reaches the synchronization round limit, if so, the new strategy network η new The parameters of the old policy network η are updated synchronously old , proceed to the next step; if not, then n=n+1, return to step S4.4.3; S4.4.
7. Determine whether the number of synchronous updates reaches the synchronous update limit. If so, stop training; if not, m=m+1, and return to step S4.4.
2.
10. The low-carbon operation optimization method of a photovoltaic charging station with hybrid energy storage according to claim 5 is characterized in that The CNN-BIGRU-ATTENTION combined neural network model includes a CNN module, a BIGRU module and an ATTENTION module. The CNN module includes a Conv2D neural network to extract features of input data and is connected to the BIGRU module through a batch normalization layer BatchNomalization, a leakage corrected linear activation function unit LeakyRelu and a deformation layer Reshape. The BIGRU module is connected to the ATTENTION module, and the ATTENTION module outputs a prediction result.
Citation Information
Cited By
Intra-day scheduling method and device for power distribution network containing compressed air energy storage system
CN119726971A
