A hybrid energy storage configuration and dispatching method based on digital twinning

Abstract

This invention belongs to the field of power system automation and energy storage technology. It provides a hybrid energy storage configuration and scheduling method based on digital twins. This invention constructs a digital twin and establishes a two-layer collaborative optimization framework. The upper layer optimizes the energy storage capacity configuration, and the lower layer trains the near-end strategy optimization scheduling PPO intelligent agent. At the same time, it introduces an energy storage aging cost model. Finally, it deploys online and uses digital twin advanced simulation to correct scheduling instructions, thereby reducing the annual comprehensive cost, extending the energy storage life, and realizing safe and efficient hybrid energy storage collaborative operation.

Description

Technical Field

[0001] This invention belongs to the field of power system automation and energy storage technology, specifically relating to a hybrid energy storage configuration and scheduling method based on digital twins. Background Technology

[0002] With the continuous increase in the penetration rate of renewable energy, the output of wind power, photovoltaic power, and other sources in microgrids exhibits strong randomness and intermittency, posing a severe challenge to system power balance and power quality. Hybrid energy storage systems combine the complementary characteristics of energy storage and power storage, effectively mitigating power fluctuations at different time scales, and have become a key technical means to improve the stable operation of microgrids.

[0003] Currently, research on hybrid energy storage systems mainly focuses on two levels: capacity configuration and operation scheduling. In terms of capacity configuration, common methods include rule-based empirical design, linear programming, and particle swarm optimization. However, these methods often separate configuration from scheduling, ignoring the strong coupling between the two, resulting in poor economic efficiency of the configuration scheme in actual scheduling. In terms of operation scheduling, traditional methods often employ filtering allocation, fuzzy logic, or model predictive control, relying on fixed rules or simplified system models, making it difficult to adapt to the complex dynamic changes of renewable energy and load. In recent years, deep reinforcement learning has been gradually introduced into the field of energy storage scheduling, capable of learning optimal strategies from interactions. However, its training process relies on a large amount of real-world environmental interaction, and existing methods are mostly designed for single energy storage or fixed configurations, lacking the ability to model the multi-media collaboration, aging evolution, and real-time bidirectional interactions of hybrid energy storage.

[0004] Furthermore, with a high proportion of renewable energy integrated into the grid, microgrids exhibit characteristics such as multi-timescale coupling and strong random disturbances in their operation. Traditional optimization methods based on offline data struggle to accurately reflect the aging process of energy storage systems during actual operation. Meanwhile, deep reinforcement learning training relies heavily on environmental interactions; training directly within the physical system poses safety risks, while simulation environments based on simplified models are insufficient to characterize complex physical mechanisms such as energy storage aging, power constraints, and multi-media collaboration. Therefore, there is an urgent need to develop a hybrid energy storage collaborative optimization method that integrates capacity configuration and operational scheduling, considers the energy storage aging process, and enables real-time interaction between the physical system and the digital model. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a hybrid energy storage configuration and scheduling method based on digital twins, thereby resolving the issues in the prior art. The technical solution adopted by this invention is as follows:

[0006] A hybrid energy storage configuration and scheduling method based on digital twins includes:

[0007] Step S1, construct a digital twin of the hybrid energy storage system: the digital twin is an interactive digital mirror of the physical entity of the hybrid energy storage system in virtual space, and the digital twin includes an energy storage equivalent circuit model, an energy storage aging model, and an energy storage power decomposition model based on variational mode decomposition.

[0008] Step S2, the upper layer performs energy storage capacity configuration optimization: optimize the parameters of the energy storage power decomposition model based on variational mode decomposition, allocate the net load power to different energy storage media according to frequency characteristics, with the goal of minimizing the annual comprehensive economic cost, iteratively optimize the rated power and rated capacity of each energy storage medium based on digital twins, and obtain the optimal configuration scheme.

[0009] Step S3, lower layer performs hybrid energy storage system scheduling optimization: PPO agent is built inside the digital twin, and PPO agent is trained by the near-end policy optimization algorithm. PPO agent is used to learn the optimal scheduling strategy of the dynamic characteristics and aging evolution law of hybrid energy storage system.

[0010] Step S4, two-layer collaborative solution process: the upper layer iteratively updates the configuration scheme of energy storage capacity, the lower layer trains PPO agent for each configuration scheme and calculates the operating cost, and feeds the operating cost back to the upper layer for evaluation, until convergence to obtain the optimal configuration scheme and corresponding scheduling strategy.

[0011] Step S5: Deploy the optimal configuration scheme and corresponding scheduling strategy to the physical entity of the hybrid energy storage system, realize real-time data synchronization between the physical entity and the virtual model through the digital twin, and issue the scheduling instructions for execution after correcting them based on the digital twin.

[0012] Furthermore, in step S1, the energy storage equivalent circuit model is used to describe the terminal voltage, state of charge, and charge / discharge efficiency of the vanadium redox flow battery, lithium battery, and supercapacitor.

[0013] The energy storage aging model is used to calculate the cumulative energy storage life loss. For lithium batteries or vanadium redox flow batteries, the Arrhenius formula is used to calculate the capacity decay rate; for supercapacitors, the rainflow counting method is used to calculate the cycle life.

[0014] Furthermore, in step S1, the energy storage power decomposition model based on variational mode decomposition is used to decompose the net load reference power into K intrinsic mode function components and their corresponding center frequencies. The optimization problem includes minimizing the sum of the estimated bandwidths of all intrinsic mode function components and satisfying the constraint that the sum of all intrinsic mode function components is equal to the original signal.

[0015] A penalty factor and Lagrange multipliers are introduced to construct an augmented Lagrange function to solve the optimization problem; and an alternating direction multiplier method is used for iterative updates, with the iterative process continuing until the preset convergence tolerance is met.

[0016] Furthermore, in step S2, the parameters of the energy storage power decomposition model based on variational mode decomposition are optimized based on the whale optimization algorithm.

[0017] Furthermore, in step S2, when allocating the net load power to different energy storage media according to frequency characteristics, the following steps are included:

[0018] The energy storage power decomposition model based on variational mode decomposition decomposes the net load power into K intrinsic mode function components. According to the dynamic response characteristics of different types of energy storage media, the decomposed intrinsic mode function components are grouped from low to high frequency: the intrinsic mode function components representing low-frequency fluctuations are assigned to vanadium redox flow batteries; the intrinsic mode function components representing mid-frequency fluctuations are assigned to lithium batteries; and the intrinsic mode function components representing high-frequency fluctuations are assigned to supercapacitors.

[0019] The formula for calculating net load power is as follows:

[0020]

[0021]

[0022] In the formula: and These are the photovoltaic and wind power outputs at time t, respectively. It is the power of the hybrid energy storage system at time t; It is the power of the electrical load at time t; It represents the power purchased from the main grid at time t; , and These represent the charging and discharging power of the vanadium redox flow battery, lithium battery, and supercapacitor at time t, respectively.

[0023] Furthermore, in step S2, when minimizing the annual comprehensive economic cost and iteratively optimizing the rated power and rated capacity of each energy storage medium based on the digital twin to obtain the optimal configuration scheme, the following steps are included:

[0024] The objective function is expressed as:

[0025]

[0026] In the formula: For investment costs, For the operating costs reported from the lower level, To maintain costs, For disposal costs;

[0027] The constraints include power constraints, state of charge constraints, and charge / discharge constraints for the hybrid energy storage system.

[0028] in:

[0029] The power constraint of a hybrid energy storage system is expressed as:

[0030]

[0031] In the formula: These are the rated power of the vanadium redox flow battery, lithium battery, and supercapacitor, respectively.

[0032] The state of charge constraint is expressed as:

[0033]

[0034] In the formula: , These are the upper and lower limits of the state of charge of a vanadium redox flow battery. , These are the upper and lower limits of the state of charge of a lithium battery. , These are the upper and lower limits of the state of charge of a supercapacitor; These represent the state of charge values ​​of the vanadium redox flow battery, lithium battery, and supercapacitor at time t, respectively.

[0035] The charge and discharge constraints of a hybrid energy storage system are expressed as follows:

[0036]

[0037]

[0038] In the formula: Let t be the total electrical charge of the hybrid energy storage system at time t; The charging efficiency of hybrid energy storage systems. This represents the discharge power of the hybrid energy storage system.

[0039] Furthermore, the investment cost is expressed as follows:

[0040]

[0041] In the formula: This is the rated power of the vanadium redox flow battery; This is the rated capacity of the vanadium redox flow battery; This refers to the rated power of the lithium battery. This refers to the rated capacity of the lithium battery. This refers to the rated power of the supercapacitor; This refers to the rated capacitance of the supercapacitor. The unit power cost of a vanadium redox flow battery; The unit price per capacity of a vanadium redox flow battery; This refers to the unit price per unit of power in lithium batteries; This refers to the unit price per unit capacity of lithium batteries; This refers to the power unit price of a supercapacitor. This refers to the unit price of the supercapacitor's capacity. The annual investment costs are for vanadium redox flow batteries, lithium batteries, and supercapacitors, respectively.

[0042] The operating cost of lower-level feedback is expressed as:

[0043]

[0044] In the formula: For the operating costs reported from the lower level, To incur penalties, For the cost of wind and solar power generation, To reduce the cost of energy storage charging and discharging, These are the unit generation costs of wind power and solar power, and the cost of purchasing electricity from the main grid, respectively. In order to purchase power from the main grid at time t, Let t be the discharge power of the hybrid energy storage system.

[0045] Maintenance costs include the sum of annual maintenance costs for vanadium redox flow batteries, lithium batteries, and supercapacitors;

[0046] The disposal cost includes the sum of the annual disposal costs of vanadium redox flow batteries, lithium batteries, and supercapacitors.

[0047] Furthermore, step S3 includes:

[0048] Step S301: Define the state space and action space;

[0049] state space Defined as:

[0050]

[0051] In the formula: t is the current scheduling time, Let be the load power, wind power, and photovoltaic power at time t, respectively. The values ​​of state of charge (SOC) at time t represent the values ​​of the vanadium redox flow battery, lithium battery, and supercapacitor, respectively. This is the reference power signal allocated to the current energy storage unit after variational mode decomposition. This represents the trend of net load changes within a future time window.

[0052] Action space Defined as:

[0053]

[0054] Action values ​​output by the PPO agent Per-unit value, actual charge / discharge power The calculation formula is:

[0055]

[0056] In the formula: Let i be the rated power of the corresponding energy storage unit, and i be the index variable of the energy storage unit. This represents the per-unit operating value of a vanadium redox flow battery. This refers to the per-unit operating value of the lithium battery. This refers to the per-unit value of the supercapacitor's operation.

[0057] Step S302: Construct a reward function to guide the PPO agent in learning the optimal scheduling strategy; the reward function is expressed as:

[0058]

[0059] In the formula: For operating costs, To incur penalties, Costs related to aging;

[0060] Step S303: The PPO agent is trained offline within the digital twin. The training uses historical data and generated scenarios to iterate the PPO agent, enabling the PPO agent to learn the optimal scheduling strategy to adapt to the dynamic characteristics of the hybrid energy storage system.

[0061] Furthermore, step S4 includes:

[0062] Step 401: Initialization; Set the population size and maximum number of iterations for the whale optimization algorithm. Given a convergence threshold ε, an initial configuration scheme population is randomly generated within the feasible region. Simultaneously, the network structure and hyperparameters of the near-end policy optimization algorithm are defined;

[0063] Step 402: Upper-level iteration begins; iteration number k=0, current global optimal cost is infinity;

[0064] Step 403: Lower-level scheduling evaluation; For each individual in the current population, construct a simulation environment in the digital twin, train the PPO agent to the maximum number of training steps, and obtain the optimal scheduling strategy; Run the simulation based on the optimal scheduling strategy and calculate the average annual operating cost; Combine the investment cost to calculate the annual comprehensive economic cost;

[0065] Step 404: Upper-level configuration update; Using daily comprehensive cost as the fitness, run the whale optimization algorithm's encirclement, bubble net attack, and random search mechanisms to update the population position and generate a new generation of configuration schemes;

[0066] Step 405: Convergence Judgment; If the optimal total cost of the new generation configuration scheme satisfies If the condition is met, terminate the iteration and output the optimal configuration and corresponding scheduling strategy; otherwise, set the iteration count k = k + 1. Return to step 403;

[0067] In the formula: The current optimal annual comprehensive economic cost calculated for the next-generation configuration scheme. This represents the historically best annual comprehensive economic cost recorded in the previous iteration.

[0068] Furthermore, step S5 includes:

[0069] Step S501: Deploy the optimal configuration scheme obtained in step 2 and the PPO agent trained in step S303 to the local controller of the hybrid energy storage system.

[0070] Step S502: Real-time data acquisition and mapping;

[0071] Execution in each scheduling cycle:

[0072] Sensor data from the hybrid energy storage system is uploaded to the digital twin, updating the current state of the virtual image in real time; the collected real-time data is then organized into a state space.

[0073] Step S503: Scheduling instruction generation; Input the state space into the PPO agent, output the corresponding action, and obtain the actual charging and discharging power instructions of each energy storage after inverse normalization;

[0074] Step S504: Instruction issuance: In the digital twin, based on the current state and the action instructions to be executed, the system trajectory for the next 15 minutes is simulated; if it is predicted that the state of charge value of any energy storage will exceed the safety limit, or if it is predicted that the cumulative lifetime loss will exceed the daily allowable threshold, the digital twin sends a correction signal to the controller.

[0075] This invention has the following beneficial effects: The upper layer of this invention uses the whale optimization algorithm to optimize energy storage configuration parameters, while the lower layer trains a PPO agent in a digital twin to generate the optimal scheduling strategy, achieving strong coupling and collaborative optimization of configuration and scheduling; at the same time, it achieves reasonable power allocation of multiple energy storage media through variational mode decomposition, giving full play to the complementary advantages of vanadium redox flow batteries, lithium batteries and supercapacitors, effectively reducing the total life cycle cost, improving the renewable energy absorption rate and system operation stability, and solving the problems of configuration and scheduling disconnect, poor economic efficiency and difficulty in adapting to high proportion of renewable energy fluctuations in traditional methods, thus possessing significant engineering application value. Attached Figure Description

[0076] Figure 1 This is a diagram of the overall architecture of the present invention;

[0077] Figure 2 This is a flowchart of the two-layer collaborative optimization solution based on WOA and PPO of the present invention. Detailed Implementation

[0078] The following will be described in conjunction with embodiments of the present invention. Figures 1-2 The technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Unless otherwise specified, the technical means used in the embodiments are conventional means well known to those skilled in the art.

[0079] like Figures 1-2 This invention provides a hybrid energy storage configuration and scheduling method based on digital twins. By constructing a digital twin and establishing a two-layer collaborative optimization framework, the upper layer optimizes energy storage capacity configuration, while the lower layer trains a near-end strategy optimization scheduling PPO agent. Simultaneously, an energy storage aging cost model is introduced. Finally, online deployment and advanced simulation using the digital twin are employed to correct scheduling commands, thereby reducing the annual overall cost, extending energy storage lifespan, and achieving safe and efficient collaborative operation of hybrid energy storage.

[0080] Step S1: Construct a digital twin of the hybrid energy storage system (HESS): The digital twin is an interactive digital mirror of the physical entity of the hybrid energy storage system in virtual space. The digital twin includes an energy storage equivalent circuit model, an energy storage aging model, and an energy storage power decomposition model based on variational mode decomposition.

[0081] The digital twin is an interactive digital mirror of the physical hybrid energy storage system in virtual space. It not only possesses the ability to map the operating state of the physical system in real time, but also embeds an intelligent decision-making learning module based on deep reinforcement learning, enabling the digital twin to have self-learning and self-evolution capabilities. The digital twin includes the following functional modules:

[0082] 1. Energy Storage Physical Behavior Mapping Model Module: Used to map the electrical characteristics and state of charge of physical energy storage systems;

[0083] 2. Energy Storage Aging Evolution Model Module: Used to calculate the lifespan decay process of each energy storage unit;

[0084] 3. Power decomposition module based on variational mode decomposition: used to allocate net load power to different energy storage media according to frequency characteristics;

[0085] 4. Real-time data acquisition and bidirectional mapping module: used to realize real-time data synchronization between the physical system and the digital model;

[0086] 5. Intelligent decision-making learning module based on deep reinforcement learning: As an embedded component of the digital twin, it utilizes the environmental feedback provided by the physical behavior mapping model and the aging evolution model, and performs iterative policy learning within the digital twin through the proximal policy optimization (PPO) algorithm to generate a hybrid energy storage scheduling strategy.

[0087] The intelligent decision-making learning module is embedded within the digital twin, forming a closed-loop interaction with the physical behavior mapping model module and the aging evolution model module, thereby realizing the fusion of model-driven and data-driven learning.

[0088] Step S101: Based on the design parameters, historical operating data, and equipment nameplate parameters of the physical hybrid energy storage system, establish a sub-model in the digital twin platform that includes the following:

[0089] 1. Energy storage equivalent circuit model:

[0090] This model is used to describe the terminal voltage, state of charge (SOC), and charge / discharge efficiency of vanadium redox flow batteries (VRB), lithium batteries (LIB), and supercapacitors (SC). Among them:

[0091] (1) Terminal voltage equation:

[0092] The second-order RC equivalent circuit consists of an ohmic internal resistance R0, two polarization capacitors C1 and C2, and polarization resistors R1 and R2. The terminal voltage U(t) is:

[0093]

[0094] In the formula, It represents the current (positive for charging, negative for discharging). Let be the open-circuit voltage, a function of SOC and temperature T, typically fitted as a polynomial:

[0095]

[0096] In the formula, a0, a1, a2, a3, a4 are polynomial fitting coefficients.

[0097] U1(t) and U2(t) are the polarization voltages of two RC circuits, satisfying:

[0098]

[0099]

[0100] (2) Calculation of State of Charge (SOC):

[0101] SOC uses the ampere-hour integration method and takes into account charge and discharge efficiency:

[0102]

[0103] In the formula, C rated For rated capacity, η coulomb For Coulomb efficiency.

[0104] (3) Charge and discharge efficiency:

[0105] Energy efficiency η energy Determined by both coulombic efficiency and voltage efficiency:

[0106]

[0107] In the formula, U avg,dis and U avg,ch Average terminal voltage during the discharge and charge processes, respectively.

[0108] 2. Energy storage aging model: used to calculate the cumulative energy storage life loss.

[0109] (1) Aging model of lithium battery / vanadium redox flow battery

[0110] Capacity decay rate based on the Arrhenius formula:

[0111]

[0112] In the formula, Q loss The percentage is the capacity loss, A is the pre-exponential factor, and E is the capacity loss percentage. a Let be the activation energy, R be the gas constant, T be the absolute temperature, Ah = ʃ|I(t)|dt be the cumulative ampere-hour throughput, and z be the power law exponent.

[0113] Incremental losses caused by each charge / discharge cycle:

[0114]

[0115] In the formula, Δt is the time step, and |I| is the absolute value of the current.

[0116] (2) Rainflow counting method cycle life model (applicable to supercapacitors):

[0117] Based on Miner's linear cumulative rule:

[0118]

[0119] In the formula, D represents cumulative fatigue damage, DoD represents the depth of discharge, and N... f The maximum number of cycles under a specific depth of discharge or state of charge variation is calculated using the following formula:

[0120]

[0121] In the formula, N0,k is a constant, and the damage increment corresponding to each half-cycle of charge and discharge is: .

[0122] 3. Energy storage power decomposition model based on variational mode decomposition (VMD):

[0123] Used to decompose the net load reference power into K intrinsic mode function (IMF) components.

[0124] (1) Constrained variational problems

[0125] The mathematical model of VMD aims to find K modes and their corresponding center frequencies. The goal is to minimize the sum of the estimated bandwidths of all modes while satisfying the constraint that the sum of all modes equals the original signal. This constrained variational problem is formulated as follows:

[0126]

[0127] In the formula: For the k-th modal component, Let be the center frequency of the k-th modal component. For the Dirac function, for Norm.

[0128] (2) Solve using the augmented Lagrangian function:

[0129] To solve the constrained optimization problem shown in the above equation, a quadratic penalty factor is introduced. and Lagrange multipliers Constructing the augmented Lagrange function :

[0130]

[0131] (3) Alternating Direction Multiplier Method (ADMM) Iterative Update:

[0132] ADMM is used to iteratively solve the problem in the frequency domain, updating the modal components, center frequency, and Lagrange multipliers sequentially.

[0133] Modal component update:

[0134]

[0135] Center frequency update:

[0136]

[0137] Lagrange multipliers update:

[0138]

[0139] In the formula: This represents the Fourier transform, where n is the number of iterations. To update the step size, it is usually taken as follows: To ensure convergence.

[0140] (4) Convergence condition:

[0141] The iterative process continues until the preset convergence tolerance is met. The convergence condition is:

[0142]

[0143] Step S102: Deployment and data acquisition of the digital twin:

[0144] The initial digital twin is deployed on a local edge server and synchronized with the Physical Hybrid Energy Storage System (HESS) via Modbus TCP / IP protocol. To achieve bidirectional synchronization between the physical system and the digital twin model, temperature and power sensors are deployed on each energy storage unit (vanadium redox flow battery VRB, lithium-ion battery LIB, and supercapacitor SC) of the physical HESS. The following operational data are collected in real time at 1-minute sampling intervals: power P(t) of each energy storage unit; state of charge (SOC)(t) of each energy storage unit; temperature T(t) of each energy storage unit; number of charge / discharge cycles N(t) of each energy storage unit; and wind power output P. w (t), Photovoltaic output power P pv (t); Load power P L (t) and the power purchased from the main grid P grid (t). The collected real-time data stream is continuously uploaded to the digital twin engine, and a digital twin model corresponding one-to-one with the physical HESS is built in the digital twin platform, thereby realizing real-time bidirectional synchronization between the physical system and the digital model.

[0145] Step S2: Upper layer performs energy storage capacity configuration optimization: Optimize the parameters of the energy storage power decomposition model based on variational mode decomposition, allocate the net load power to different energy storage media according to frequency characteristics, with the goal of minimizing the annual comprehensive economic cost, iteratively optimize the rated power and rated capacity of each energy storage media based on digital twins, and obtain the optimal configuration scheme.

[0146] Step S201: Parameter optimization of the energy storage power decomposition model based on variational mode decomposition:

[0147] Regarding the penalty factor in variational mode decomposition To address the challenge of choosing the modality number K, the Whale Optimization Algorithm (WOA) is used to optimize key VMD parameters. Its core principle involves iterative optimization by simulating three behaviors: whale encirclement, bubble net attack, and random search.

[0148] In WOA, the position of each individual whale represents a candidate solution, i.e., a set of VMD parameters. Let the search space dimension be D and the population size be N, then the position of the i-th whale at iteration t can be represented as... ,in correspond , Corresponding to K.

[0149] The whale optimization algorithm mainly includes the following three behavioral patterns:

[0150] (1) Surround the prey:

[0151] Whales identify and swim around the current optimal location (prey), and their position update formula is:

[0152]

[0153] In the formula: This represents the position of the globally optimal individual in the current iteration. For the coefficient vector, A random vector in the range [0,1] is the convergence factor.

[0154] (2) Bubble web attack:

[0155] Whales approach their prey in a spiral motion, mimicking bubble-net hunting behavior. The position update formula is:

[0156]

[0157] In the formula: b is a constant for the spiral shape, and l is a random number in the range [-1, 1].

[0158] (3) Random search:

[0159] When the coefficient At this time, individual whales do not follow the current optimal solution, but randomly select a reference individual to search, in order to enhance their global exploration capabilities. The position update formula is:

[0160]

[0161] In the formula: This represents the position of an individual randomly selected from the current population.

[0162] To evaluate VMD parameters The quality of the decomposition is determined by the envelope entropy of the decomposed signal sequence; a smaller envelope entropy indicates a more regular signal and a better decomposition effect. Its mathematical expression is:

[0163]

[0164] In the formula: For the reason The envelope signal obtained by performing a Hilbert transform. Represents the k-th intrinsic mode function (IMF) component. envelope signal The normalized magnitude probability E at the j-th data point k Let M be the envelope entropy and M be the number of data points.

[0165] Step S202: Construct a power allocation strategy:

[0166] The net load power is calculated based on the real-time collected data. The expression for calculating the net load power is as follows:

[0167]

[0168]

[0169] In the formula: and These are the photovoltaic and wind power outputs at time t, respectively. It is the power of the hybrid energy storage system at time t; It is the power of the electrical load at time t; It represents the power purchased from the main grid at time t; , and These represent the charging and discharging power of the vanadium redox flow battery, lithium battery, and supercapacitor at time t, respectively.

[0170] The net load power is decomposed into K IMF components. Based on the dynamic response characteristics of different types of energy storage media, the decomposed IMF components are grouped according to frequency from low to high: the IMF components representing low-frequency fluctuations (the first m low-frequency components, i.e., to ) are assigned to the all-vanadium redox flow cell; the IMF components characterizing the mid-frequency fluctuations (the m-th to n-th mid-frequency components, i.e. to Allocate to lithium-ion batteries; the remaining IMF components characterizing high-frequency fluctuations (n+1 to the Kth high-frequency components, i.e. to The supercapacitors are allocated to this allocation strategy. The mathematical expression of this strategy is as follows:

[0171]

[0172] In the formula: , and These are the charging and discharging power commands assigned to the vanadium redox flow battery, lithium-ion battery, and supercapacitor at time t, respectively.

[0173] Step 203: Upper-level model: Hybrid energy storage system capacity configuration:

[0174] With the goal of minimizing the annual comprehensive economic cost, the rated power P of each energy storage unit is iteratively optimized in the virtual simulation module of the digital twin. rated With rated capacity E rated .

[0175] 1. Objective function:

[0176] The upper level aims to minimize the annual comprehensive economic cost, which can be mathematically expressed as follows:

[0177]

[0178] In the formula: For investment costs, For the operating costs reported from the lower level, To maintain costs, For disposal costs.

[0179] Investment costs:

[0180]

[0181] In the formula: This is the rated power of the vanadium redox flow battery; This is the rated capacity of the vanadium redox flow battery; This refers to the rated power of the lithium battery. This refers to the rated capacity of the lithium battery. This refers to the rated power of the supercapacitor; This refers to the rated capacitance of the supercapacitor. The unit power cost of a vanadium redox flow battery; The unit price per capacity of a vanadium redox flow battery; This refers to the unit price per unit of power in lithium batteries; This refers to the unit price per unit capacity of lithium batteries; This refers to the power unit price of a supercapacitor. This refers to the unit price of the supercapacitor's capacity. These are the annual investment costs for vanadium redox flow batteries, lithium batteries, and supercapacitors, respectively.

[0182] Operating cost: The operating cost is calculated by the lower-level scheduling PPO agent.

[0183]

[0184] In the formula: For operating costs, To incur penalties, For the cost of wind and solar power generation, To reduce the cost of energy storage charging and discharging, These are the unit generation costs of wind power and solar power, and the cost of purchasing electricity from the main grid, respectively. In order to purchase power from the main grid at time t, Let t be the HESS discharge power at time t.

[0185] Maintenance costs:

[0186]

[0187] In the formula: The unit power maintenance cost for the entire vanadium liquid flow; The unit capacity maintenance cost for the entire vanadium liquid flow; Maintenance cost per unit power of lithium batteries; Maintenance cost per unit capacity of lithium batteries; The unit power maintenance cost of a supercapacitor; Maintenance cost per unit capacity of a supercapacitor; The annual maintenance costs are for vanadium redox flow batteries, lithium batteries, and supercapacitors, respectively.

[0188] Disposal costs:

[0189]

[0190] In the formula: The unit power disposal cost of the entire vanadium liquid flow; The unit capacity disposal cost of the entire vanadium liquid flow; The unit power disposal cost of lithium batteries; The unit capacity disposal cost of lithium batteries; Cost per unit power of supercapacitors; The unit capacity disposal cost of supercapacitors; The annual disposal costs are for vanadium redox flow batteries, lithium batteries, and supercapacitors, respectively.

[0191] 2. Constraints:

[0192] (1) The power constraint of the hybrid energy storage system is expressed as:

[0193]

[0194] In the formula: These are the rated power of vanadium redox flow batteries, lithium batteries, and supercapacitors, respectively.

[0195] (2) The state of charge constraint is expressed as:

[0196]

[0197] In the formula: , These are the upper and lower limits of the state of charge (SOC) of the vanadium liquid flow. , These are the upper and lower limits of the state of charge (SOC) of lithium batteries. , These are the upper and lower limits of the supercapacitor's state of charge (SOC). The SOC values ​​at time t represent the total vanadium redox flow, lithium battery, and supercapacitor, respectively.

[0198] (3) The charging and discharging constraints of the hybrid energy storage system are expressed as:

[0199]

[0200]

[0201] In the formula: The total charge at time t is HESS. To improve the charging efficiency of energy storage systems. This represents the discharge power of the energy storage system.

[0202] Step S3: Lower layer performs hybrid energy storage system scheduling optimization: PPO agent is built inside the digital twin. The PPO agent is trained through the near-end policy optimization algorithm. The PPO agent is used to learn the optimal scheduling strategy of the dynamic characteristics and aging evolution law of the hybrid energy storage system.

[0203] Step S301: Define the state space and action space:

[0204] The state space needs to be constructed to comprehensively reflect the current operating status and short-term trends of the energy storage system, providing sufficient information for the decision-making of the PPO agent. The designed state space (vector) is as follows:

[0205]

[0206] In the formula: t is the current scheduling time, Let be the load power, wind power, and photovoltaic power at time t, respectively. The SOC values ​​at time t are those of the flow battery, lithium battery, and supercapacitor, respectively. This is the reference power signal allocated to the current energy storage unit after VMD decomposition. This represents the trend of net load changes within a future time window.

[0207] Action space (vector) The decisions that the PPO agent can execute at each scheduling moment are defined. Considering the dynamic characteristics of different energy storage systems, the charging and discharging power is normalized and limited to the rated power range. The actions are defined as follows:

[0208]

[0209] Action values ​​output by the PPO agent Per-unit value, actual charge / discharge power Determined by the following formula:

[0210]

[0211] In the formula: Here, i represents the rated power of the corresponding energy storage unit, and i is the index variable of the energy storage unit, representing the three types of energy storage. This represents the per-unit operating value of a vanadium redox flow battery. This refers to the per-unit operating value of the lithium battery. This is the per-unit value for the operation of the supercapacitor.

[0212] Step S302: Construct the reward function:

[0213] 1. Reward function:

[0214] reward function The design of the system used to guide the PPO agent in learning the optimal scheduling strategy must balance economic operation and safety constraints, including the cost of wind and solar power generation. and energy storage charging and discharging costs Instant rewards The total cost is negative, and a penalty for constraint violation is introduced:

[0215]

[0216] In the formula: For operating costs, To incur penalties, Costs related to aging.

[0217] Operating costs:

[0218]

[0219] In the formula: These are the unit generation costs of wind power and solar power, and the cost of purchasing electricity from the main grid, respectively. In order to purchase power from the main grid at time t, Let t be the HESS discharge power at time t.

[0220] Cost of punishment:

[0221] To prevent the PPO agent from making decisions that violate physical constraints, a penalty is set:

[0222]

[0223]

[0224] In the formula: The penalty coefficient is... These are the energy storage SOC and the power threshold, respectively.

[0225] Aging costs:

[0226] Based on the aging model constructed in step S101, the formula for calculating the energy storage aging cost is as follows:

[0227]

[0228] In the formula, E vrb,rated E lib,rated E sc,rated These are the rated capacities of vanadium redox flow batteries, lithium batteries, and supercapacitors, respectively; c vrb,rep c lib,rep c sc,rep The replacement costs per unit capacity for vanadium redox flow batteries, lithium batteries, and supercapacitors are respectively, Q loss,end,vrb Q loss,end,lib Q loss,end,sc These are the capacity loss thresholds at the end of the lifespan of vanadium redox flow batteries, lithium batteries, and supercapacitors, respectively.

[0229] 2. Constraints:

[0230] Power constraints of hybrid energy storage systems:

[0231]

[0232] In the formula: These are the rated power of the vanadium redox flow battery, lithium battery, and supercapacitor, respectively.

[0233] State of charge constraints:

[0234]

[0235] Charge and discharge constraints of hybrid energy storage systems:

[0236]

[0237]

[0238] Step S303: PPO agent training:

[0239] The PPO agent, serving as the intelligent decision-making and learning unit within the digital twin, undergoes offline training within the digital twin environment. This training utilizes historical data and generated scenarios to iterate extensively on the PPO agent, enabling it to learn the optimal scheduling strategy adapted to the dynamic characteristics of the hybrid energy storage system.

[0240] 1. Training environment setup:

[0241] The digital twin is used as the training environment for the PPO agent. This environment encapsulates the energy storage equivalent circuit model and aging model established in step S101, and can calculate the state (SOC, etc.) of the next moment based on the actions output by the PPO agent (i.e., the charging and discharging power commands of each energy storage unit) and return an instant reward.

[0242] 2. PPO agent network structure:

[0243] The PPO agent contains two neural networks:

[0244] Actor Network (Policy Network): Output is in the state Take action below probability distribution .

[0245] Critic Network (Value Network): Output in state Value estimation This refers to the cumulative discount reward expected to be obtained starting from this state.

[0246] 3. Dominance function estimation:

[0247] To reduce the variance of the policy gradient and maintain unbiasedness, generalized advantage estimation (GAE) is used to calculate the advantage function. t :

[0248]

[0249] In the formula, γ∈[0,1] is the discount factor, λ∈[0,1] is the GAE parameter, and δ t The timing difference error is calculated using the following formula:

[0250]

[0251] 4. Policy network parameter update:

[0252] PPO limits the policy update magnitude through a pruning mechanism to avoid destructive updates. The objective function is:

[0253]

[0254] In the formula, The probability ratio of the new and old strategies is ε, where ε is the clipping hyperparameter; the clip() function constrains the input to a specified interval.

[0255] Policy network parameter update:

[0256]

[0257] In the formula, θ is the parameter vector of the policy network, and η is the learning rate of the policy network. This is the gradient operator.

[0258]

[0259] In the formula, θ t Here are the policy network parameters at the t-th iteration, and Adam() is the Adam optimizer function.

[0260] 5. Value network parameter update:

[0261] The Critic network updates by minimizing the mean squared error loss:

[0262]

[0263] In the formula, L VF Let ϕ be the loss function of the value network, and let ϕ be the parameter vector of the value network. For the expectation of time step t, V(s) t For the value network in state s t The output state value estimate, The target value of the state value is calculated using the following formula:

[0264]

[0265] Value network parameter update:

[0266]

[0267] In the formula, β is the learning rate of the value network.

[0268] 6. Data Collection and Experience Review:

[0269] In each iteration, the PPO agent interacts with the environment to generate a trajectory:

[0270]

[0271] In the formula: τ is a complete trajectory generated by the interaction between the PPO agent and the environment. Let T be the state at time T. Let T be the action performed by the agent. Execute the action at time T Afterwards, immediate rewards for environmental feedback, To perform the action After that, the state transitions to the next moment, where T is the total time step of the trajectory;

[0272] The data is stored in an experience buffer. Then, batches are sampled from the buffer, the aforementioned loss is calculated, and the network parameters are updated, repeating this process for multiple rounds.

[0273] 7. Model saving:

[0274] After training converges, the weight parameters of the policy network and value network are saved as the final scheduling policy model.

[0275] Step S4: Two-layer collaborative solution process: The upper layer iteratively updates the energy storage capacity configuration scheme, and the lower layer trains the PPO agent for each configuration scheme and calculates the operating cost. The operating cost is fed back to the upper layer for evaluation until convergence is obtained to obtain the optimal capacity configuration and the corresponding scheduling strategy.

[0276] Step 401: Initialization. Set the WOA population size and maximum number of iterations. Given a convergence threshold ε, an initial configuration scheme population is randomly generated within the feasible region. At the same time, the network structure and hyperparameters of the PPO algorithm are set.

[0277] Step 402: Upper-level iteration begins. When k=0, the current global optimal cost is infinity.

[0278] Step 403: Lower-level scheduling evaluation. For each individual in the current population... :

[0279] a) Construct a simulation environment within a digital twin platform;

[0280] b) Train the PPO agent to the maximum number of training steps to obtain the optimal scheduling strategy;

[0281] c) Run simulations based on the optimal scheduling strategy and calculate the average annual operating cost; d) Combine investment costs to calculate the annual comprehensive economic cost.

[0282] Step 404: Upper-level configuration update. Using daily comprehensive cost as the fitness metric, run WOA's encirclement, bubble net attack, and random search mechanisms to update the population position and generate a new generation configuration scheme.

[0283] Step 405: Convergence Judgment. If the optimal total cost of the next generation satisfies... If the condition is met, the iteration terminates, and the optimal configuration and corresponding scheduling strategy are output; otherwise, let k = k + 1. Return to step 403.

[0284] In the formula: The current optimal annual comprehensive economic cost calculated for the next-generation configuration scheme. This represents the historically best annual comprehensive economic cost recorded in the previous iteration.

[0285] Step S5: Deploy the optimal configuration scheme and corresponding scheduling strategy to the physical entity of the hybrid energy storage system, realize real-time data synchronization between the physical entity and the virtual model through the digital twin, and issue the scheduling instructions for execution after correcting them based on the digital twin;

[0286] Step S501: Deploy the optimal configuration scheme determined in step S203 and the PPO scheduling policy network trained in step S303 to the local controller of the physical HESS.

[0287] Step S502: Real-time data acquisition and mapping;

[0288] After the system is put into actual operation, the following sub-steps are executed in each scheduling cycle:

[0289] 1. Sensor data from the physical system is uploaded to the digital twin, updating the current state of the virtual image in real time.

[0290] 2. Organize the collected real-time data into the state vector s defined in step S301. t .

[0291] Step S503: Generation of scheduling instructions;

[0292] s t Input PPO's Actor network, output action A t After inverse normalization, the actual charging and discharging power commands for each energy storage unit are obtained:

[0293]

[0294] Step S504: Instruction issued;

[0295] In a digital twin, based on the current state s t and the instruction A to be executed t The system rapidly simulates the system trajectory for the next 15 minutes (including changes in the State of Charge (SOC) of each energy storage unit and accumulated lifetime loss). If it is predicted that the SOC of any energy storage unit will exceed the safety limit, or that the accumulated lifetime loss will exceed the daily allowable threshold, the digital twin platform sends a correction signal to the controller. The controller makes fine adjustments based on the instructions and sends the final instructions to the PCS controller of each energy storage unit via Modbus TCP. The PCS then executes the corresponding charging and discharging actions.

[0296] The above embodiments are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Any modifications, alterations, alterations, or substitutions made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A hybrid energy storage configuration and scheduling method based on digital twins, characterized in that, include: Step S1, construct a digital twin of the hybrid energy storage system: the digital twin is an interactive digital mirror of the physical entity of the hybrid energy storage system in virtual space, and the digital twin includes an energy storage equivalent circuit model, an energy storage aging model, and an energy storage power decomposition model based on variational mode decomposition. Step S2, the upper layer performs energy storage capacity configuration optimization: optimize the parameters of the energy storage power decomposition model based on variational mode decomposition, allocate the net load power to different energy storage media according to frequency characteristics, with the goal of minimizing the annual comprehensive economic cost, iteratively optimize the rated power and rated capacity of each energy storage medium based on digital twins, and obtain the optimal configuration scheme. Step S3, lower layer performs hybrid energy storage system scheduling optimization: PPO agent is built inside the digital twin, and PPO agent is trained by the near-end policy optimization algorithm. PPO agent is used to learn the optimal scheduling strategy of the dynamic characteristics and aging evolution law of hybrid energy storage system. Step S4, two-layer collaborative solution process: the upper layer iteratively updates the configuration scheme of energy storage capacity, the lower layer trains PPO agent for each configuration scheme and calculates the operating cost, and feeds the operating cost back to the upper layer for evaluation, until convergence to obtain the optimal configuration scheme and corresponding scheduling strategy. Step S5: Deploy the optimal configuration scheme and corresponding scheduling strategy to the physical entity of the hybrid energy storage system, realize real-time data synchronization between the physical entity and the virtual model through the digital twin, and issue the scheduling instructions for execution after correcting them based on the digital twin.

2. The hybrid energy storage configuration and scheduling method based on digital twins according to claim 1, characterized in that, In step S1, the energy storage equivalent circuit model is used to describe the terminal voltage, state of charge, and charge / discharge efficiency of the vanadium redox flow battery, lithium battery, and supercapacitor. The energy storage aging model is used to calculate the cumulative energy storage life loss. For lithium batteries or vanadium redox flow batteries, the Arrhenius formula is used to calculate the capacity decay rate; for supercapacitors, the rainflow counting method is used to calculate the cycle life.

3. The hybrid energy storage configuration and scheduling method based on digital twins according to claim 1, characterized in that, In step S1, the energy storage power decomposition model based on variational mode decomposition is used to decompose the net load reference power into K intrinsic mode function components and their corresponding center frequencies. The optimization problem includes minimizing the sum of the estimated bandwidths of all intrinsic mode function components and satisfying the constraint that the sum of all intrinsic mode function components is equal to the original signal. A penalty factor and Lagrange multipliers are introduced to construct an augmented Lagrange function to solve the optimization problem; and an alternating direction multiplier method is used for iterative updates, with the iterative process continuing until the preset convergence tolerance is met.

4. The hybrid energy storage configuration and scheduling method based on digital twins according to claim 3, characterized in that, In step S2, the parameters of the energy storage power decomposition model based on variational mode decomposition are optimized based on the whale optimization algorithm.

5. The hybrid energy storage configuration and scheduling method based on digital twins according to claim 3, characterized in that, In step S2, when allocating the net load power to different energy storage media according to frequency characteristics, the following steps are included: The energy storage power decomposition model based on variational mode decomposition decomposes the net load power into K intrinsic mode function components. According to the dynamic response characteristics of different types of energy storage media, the decomposed intrinsic mode function components are grouped from low to high frequency: the intrinsic mode function components representing low-frequency fluctuations are assigned to vanadium redox flow batteries; the intrinsic mode function components representing mid-frequency fluctuations are assigned to lithium batteries; and the intrinsic mode function components representing high-frequency fluctuations are assigned to supercapacitors. The formula for calculating net load power is as follows: In the formula: and These are the photovoltaic and wind power outputs at time t, respectively. It is the power of the hybrid energy storage system at time t; It is the power of the electrical load at time t; It is the power purchased from the main grid at time t; , and These represent the charging and discharging power of the vanadium redox flow battery, lithium battery, and supercapacitor at time t, respectively.

6. The hybrid energy storage configuration and scheduling method based on digital twins according to claim 5, characterized in that, In step S2, with the goal of minimizing the annual comprehensive economic cost, the optimal configuration scheme is obtained by iteratively optimizing the rated power and rated capacity of each energy storage medium based on the digital twin, including: The objective function is expressed as: In the formula: For investment costs, For the operating costs reported from the lower level, To maintain costs, For disposal costs; The constraints include power constraints, state of charge constraints, and charge / discharge constraints for the hybrid energy storage system. in: The power constraint of a hybrid energy storage system is expressed as: In the formula: These are the rated power of the vanadium redox flow battery, lithium battery, and supercapacitor, respectively. The state-of-charge constraint is expressed as: In the formula: , These are the upper and lower limits of the state of charge of a vanadium redox flow battery. , These are the upper and lower limits of the state of charge of a lithium battery. , These are the upper and lower limits of the state of charge of a supercapacitor; These are the state of charge values ​​of the vanadium redox flow battery, lithium battery, and supercapacitor at time t, respectively. The charge and discharge constraints of a hybrid energy storage system are expressed as follows: In the formula: Let t be the total electrical charge of the hybrid energy storage system at time t; The charging efficiency of hybrid energy storage systems. This represents the discharge power of the hybrid energy storage system.

7. A hybrid energy storage configuration and scheduling method based on digital twins according to claim 6, characterized in that, Investment costs are expressed as follows: In the formula: This is the rated power of the vanadium redox flow battery; This is the rated capacity of the vanadium redox flow battery; This refers to the rated power of the lithium battery. This refers to the rated capacity of the lithium battery. This refers to the rated power of the supercapacitor; This refers to the rated capacitance of the supercapacitor. The unit power cost of a vanadium redox flow battery; The unit price per capacity of the vanadium redox flow battery; This refers to the unit price per unit of power in lithium batteries; This refers to the unit price per capacity of lithium batteries; This refers to the power unit price of a supercapacitor. This refers to the unit price of the supercapacitor's capacity. The annual investment costs are for vanadium redox flow batteries, lithium batteries, and supercapacitors, respectively. The operating cost of lower-level feedback is expressed as: In the formula: For the operating costs reported from the lower level, To incur penalties, For the cost of wind and solar power generation, To reduce the cost of energy storage charging and discharging, These are the unit generation costs of wind power and solar power, and the cost of purchasing electricity from the main grid, respectively. In order to purchase power from the main grid at time t, Let t be the discharge power of the hybrid energy storage system. Maintenance costs include the sum of annual maintenance costs for vanadium redox flow batteries, lithium batteries, and supercapacitors; The disposal cost includes the sum of the annual disposal costs of vanadium redox flow batteries, lithium batteries, and supercapacitors.

8. The hybrid energy storage configuration and scheduling method based on digital twins according to claim 6, characterized in that, Step S3 includes: Step S301: Define the state space and action space; state space Defined as: In the formula: t is the current scheduling time, Let be the load power, wind power, and photovoltaic power at time t, respectively. The values ​​of state of charge (SOC) at time t represent the values ​​of the vanadium redox flow battery, lithium battery, and supercapacitor, respectively. This is the reference power signal allocated to the current energy storage unit after variational mode decomposition. This represents the trend of net load changes within a future time window. Action space Defined as: Action values ​​output by the PPO agent Per-unit value, actual charge / discharge power The calculation formula is: In the formula: Let i be the rated power of the corresponding energy storage unit, and i be the index variable of the energy storage unit. This represents the per-unit operating value of a vanadium redox flow battery. This refers to the per-unit operating value of the lithium battery. This refers to the per-unit value of the supercapacitor's operation. Step S302: Construct a reward function to guide the PPO agent in learning the optimal scheduling strategy; the reward function is expressed as: In the formula: For operating costs, To incur penalties, Costs related to aging; Step S303: The PPO agent is trained offline within the digital twin. The training uses historical data and generated scenarios to iterate the PPO agent, enabling the PPO agent to learn the optimal scheduling strategy to adapt to the dynamic characteristics of the hybrid energy storage system.

9. A hybrid energy storage configuration and scheduling method based on digital twins according to claim 8, characterized in that, Step S4 includes: Step 401: Initialization; Set the population size and maximum number of iterations for the whale optimization algorithm. Given a convergence threshold ε, an initial configuration scheme population is randomly generated within the feasible region. Simultaneously, the network structure and hyperparameters of the near-end policy optimization algorithm are defined; Step 402: Upper-level iteration begins; iteration number k=0, current global optimal cost is infinity; Step 403: Lower-level scheduling evaluation; For each individual in the current population, construct a simulation environment in the digital twin, train the PPO agent to the maximum number of training steps, and obtain the optimal scheduling strategy; Run the simulation based on the optimal scheduling strategy and calculate the average annual operating cost; Combine the investment cost to calculate the annual comprehensive economic cost; Step 404: Upper-level configuration update; Using daily comprehensive cost as the fitness, run the whale optimization algorithm's encirclement, bubble net attack, and random search mechanisms to update the population position and generate a new generation of configuration schemes; Step 405: Convergence Judgment; If the optimal total cost of the new generation configuration scheme satisfies If the condition is met, terminate the iteration and output the optimal configuration and corresponding scheduling strategy; otherwise, set the iteration count k = k + 1. Return to step 403; In the formula: The current optimal annual comprehensive economic cost calculated for the next-generation configuration scheme. This represents the historical best annual comprehensive economic cost recorded in the previous generation iteration.

10. A hybrid energy storage configuration and scheduling method based on digital twins according to claim 8, characterized in that, Step S5 includes: Step S501: Deploy the optimal configuration scheme obtained in step 2 and the PPO agent trained in step S303 to the local controller of the hybrid energy storage system. Step S502: Real-time data acquisition and mapping; Execution in each scheduling cycle: Sensor data from the hybrid energy storage system is uploaded to the digital twin, updating the current state of the virtual image in real time; the collected real-time data is then organized into a state space. Step S503: Scheduling instruction generation; Input the state space into the PPO agent, output the corresponding action, and obtain the actual charging and discharging power instructions of each energy storage after inverse normalization; Step S504: Instruction issuance: In the digital twin, based on the current state and the action instructions to be executed, the system trajectory for the next 15 minutes is simulated; if it is predicted that the state of charge value of any energy storage will exceed the safety limit, or if it is predicted that the cumulative lifetime loss will exceed the daily allowable threshold, the digital twin sends a correction signal to the controller.

Images