Dynamic network reconstruction strategy for digital electric power energy storage system

Abstract

The invention relates to a dynamic network reconstruction strategy of a digital electric power energy storage system in the technical field of electric power energy storage. The strategy comprises the following steps: S1, building an energy storage system; the system consists of an intelligent layer, a control layer and a data layer, s2, modeling an energy storage network; learning an optimal control strategy of the energy storage network by adopting a graph capsule neural network; s3, dynamic network reconstruction; generating a fault emergency response model based on a Markov decision process; by systematically optimizing a topological structure and an optimal control strategy of the energy storage network, rapid isolation of a fault area and dynamic adjustment and reconfiguration of working modes of energy storage equipment and load equipment are realized, the energy storage network is ensured to still operate stably in a fault situation, and the energy storage efficiency is improved. The flexibility and elasticity (fault tolerance) of the energy storage system in a dynamic environment are improved; by reducing the energy loss in the transmission process, balancing the network load and preventing local faults, the technical problems existing in the prior art are solved, and the sustainable development of a novel renewable energy system is promoted.

Description

Technical Field

[0001] The present invention belongs to the technical field of electric energy storage, and specifically provides a dynamic network reconstruction strategy for a digital electric energy storage system. Background Art

[0002] As an important component of the power system, the energy storage unit can convert and store energy in the form of electricity for emergency use. It plays a key role in the flexibility and stability of the power grid. It is also an important means to promote the widespread application of renewable energy and improve energy efficiency. At present, new renewable energy power generation technology is developing rapidly as an important component of the new power system. Based on the characteristics of energy storage technology, it is mainly classified into energy-type and power-type energy storage technology. Common energy-type energy storage technologies include compressed air energy storage, pumped water energy storage and battery energy storage, and common power-type energy storage technologies include flywheel energy storage, supercapacitor energy storage and superconducting energy storage. Because traditional energy storage units are always facing economic constraints and safety concerns, digital energy storage can discretize energy flow and information flow, which can improve the economy and safety of energy storage systems.

[0003] Energy storage systems have a certain degree of flexibility in maintaining their functions and performance in the face of external disturbances or changes (such as fluctuations in electricity demand, unstable renewable energy generation, equipment failures, natural disasters, etc.). However, as the coverage of energy storage system networks becomes wider and wider, the associations between nodes become more and more complex, and the interactions and interdependencies between subnetworks or nodes are gradually increasing, which also brings challenges such as network interference migration and the possibility of cascading failures, and also weakens the elasticity of energy storage systems to a certain extent. Dynamically reconstructing the network topology of energy storage systems and reasonably deploying network connection methods can effectively isolate fault areas, thereby improving the utilization efficiency of energy storage resources, reducing energy waste and loss, and promoting the access of more renewable energy and reducing dependence on traditional fossil energy. Therefore, combining energy storage with digital technology to achieve dynamic reconstruction of the energy storage network topology, fault isolation, and optimization of transmission paths and energy storage status can provide important technical support for promoting energy transformation and promoting the development of renewable energy.

[0004] When the energy storage network faces topological changes such as faults, the existing solutions mainly focus on two methods: one is the heuristic routing optimization algorithm to optimize the link weight setting or split ratio under different traffic demands or network topologies. However, when faced with changes in network topology, the previous routing configuration cannot adapt to the new environment, and the routing strategy needs to be recalculated, which takes a lot of time and has high computational overhead. The other is to start from deep reinforcement learning based on neural networks, which is suitable for dealing with complex dynamic environments. However, when the topological structure of the network changes, the trained neural network model cannot be generalized well to other network topologies, and retraining the model still consumes a lot of time and computing resources. Existing research has obvious limitations in the optimization and fault management of energy storage systems in dynamic environments. Most methods still stay in the optimization under static conditions, ignoring the dynamic changes of complex power grid environments and the dependencies between multiple nodes; especially in complex energy storage networks with multiple nodes and multiple energy types, how to achieve real-time adaptive reconstruction of network topology to cope with uncertainty and sudden failures is still a key problem that has not been effectively solved. Therefore, how to design a universal fault management method that can adaptively face various network fault situations is a technical problem that needs to be solved urgently. Summary of the invention

[0005] In view of the above situation, the present invention provides a dynamic network reconstruction strategy for a digital electric energy storage system. Based on the rapid development of graph neural networks and reinforcement learning, it captures the complex dependencies between system nodes and adjusts the system configuration in real time to achieve adaptive optimization and fault isolation, overcomes the limitations of existing methods (such as the inability to dynamically reconfigure the network and the lack of robustness in emergency response), thereby improving the flexibility and resilience (fault tolerance) of the energy storage system in a dynamic environment; by reducing energy losses during transmission, balancing network loads and preventing local failures, it solves the technical problems existing in the existing technology and promotes the sustainable development of new renewable energy systems.

[0006] In order to achieve the above purpose, the present invention adopts the following technical solutions:

[0007] A dynamic network reconstruction strategy for a digital power storage system, the steps are as follows:

[0008] S1. Energy storage system construction; the energy storage system consists of an intelligent layer, a control layer and a data layer; the intelligent layer obtains the current state of the energy storage system, and extracts the node feature information of the energy storage network based on the graph neural network, makes decisions through the reinforcement learning agent, and the action information of the decision is transmitted to the control layer; the control layer receives the action information from the intelligent layer, generates control instructions according to the rules and transmits the control instructions to the data layer; the data layer includes the energy storage network, which includes power generation equipment, energy storage equipment, load equipment and a control center. The energy storage network receives the control instructions from the control layer to make adjustments, and updates the energy storage network, sends the update information to the control center, and feeds back the update information to the control layer, which is then transmitted to the intelligent layer by the control layer;

[0009] S2. Energy storage network modeling; Graph capsule neural network is used to learn the optimal control strategy in the energy storage network. Compared with other graph capsule neural networks, it can more effectively capture the structural information of the energy storage network, and the enhanced state representation helps to calculate better actions than traditional feature abstraction networks (such as MLP); through graph capsule neural network construction, feature extraction, capsule encoding, dynamic routing and graph capsule neural network update, the global information of the energy storage system is gradually extracted and optimized; first, the graph capsule neural network is used to extract node embeddings to form primary capsules; then, the attention module is applied to scale the node embeddings, and then the graph capsules are generated through dynamic routing; finally, dynamic routing is applied again to perform graph classification;

[0010] The graph capsule neural network is constructed with the devices and connection relationships in the energy storage network as nodes and edges respectively, which can be represented as a graph structure G = (N, E), where nodes represent energy storage devices, substations, loads, etc., and edges represent physical or logical connections between devices, such as transmission lines or communication links.

[0011] Based on the graph capsule neural network, corresponding features are extracted for each node and edge, such as the capacity, charging and discharging status, and location of the energy storage device, the transmission capacity and distance of the edge, etc.

[0012] The graph capsule neural network encodes the features of each node and edge, and each node and edge is represented as a capsule, which is composed of a set of feature vectors. These feature vectors not only describe the properties of nodes and edges, but also capture the spatial and functional relationships between them.

[0013] Through a dynamic routing algorithm, information is transmitted between capsules, and features are aggregated and combined layer by layer to capture high-order structural information of the energy storage network.

[0014] Finally, combined with the message passing mechanism of the graph capsule neural network, the feature vector of the capsule is used to iteratively update the graph structure, gradually extract the global information of the energy storage system, and achieve in-depth understanding and optimization of the entire network.

[0015] S3. Dynamic network reconstruction: In order to quickly implement effective emergency response measures when a fault occurs, a combined control method based on the concept of energy discretization and digitization is proposed, covering two modules: reconstruction of the energy storage network topology and reconfiguration of energy storage devices and load devices.

[0016] This method can achieve rapid isolation of fault areas, dynamically adjust the working mode of energy storage equipment, and reconfigure the operating status of energy storage equipment and load equipment by systematically optimizing the topological structure and operating strategy of the energy storage network, thereby ensuring that the energy storage network can maintain efficient and stable operation under various fault scenarios.

[0017] Specifically, the core of this method is to discretize and digitize the energy flow, and to more finely manage and control the energy distribution and transmission of each node and edge.

[0018] This method follows Kirchhoff's law to change the line impedance and rearranges the network structure through switching operations, thereby alleviating faults. This allows the system to respond quickly when a fault occurs, flexibly adjust the network's topology and operating strategies, minimize the impact of the fault on the overall system, and improve fault recovery efficiency.

[0019] After isolating the fault area and reconstructing the network topology, the system can further optimize the network's operating efficiency and stability by actively adjusting the charging and discharging status of the energy storage equipment.

[0020] When the network topology changes, the system will not only reallocate the energy transmission path, but also dynamically adjust the working mode of the energy storage equipment according to the current load demand, fault conditions and the status of each energy storage node.

[0021] Energy storage devices with higher energy storage capacity can be regarded as power sources. When the nodes in the faulty area cannot be connected to the power generation nodes, the energy storage devices discharge to ensure that the isolated areas that are not connected to the power generation nodes can still work for a period of time.

[0022] The fault emergency response in the energy storage network is expressed as a Markov decision process (MDP) in the graph domain, denoted as M = (S, A, P tr ,R), this tuple represents the state, action, transition probability and reward respectively, and is defined as follows:

[0023] (1) Status (S):

[0024] The state consists of relevant observations in the energy storage network, representing the current operating status of the network. It includes node variables, edge variables, network topology, and other system variables, expressed as

[0025]

[0026] in, Represents estimated or predicted values ​​of active and reactive power demand of a node; Represents the active and reactive power generation of the node; V N Represents the three-phase voltage measured at the bus, that is, the node in the diagram; V viol Indicates the voltage violation constraint value in the network; the edge variable includes the power flow through the network branch, represented by l E ; The operating topology of the network is T, and the total energy supply in the network is represented by E supp ;The state variable O includes the fault scenario, i.e., multi-line faults in the network, including switch faults;

[0027] The inoperability of the faulty switch is handled through a shielding mechanism, which is represented by the state variable μ; that is, if the switch operates normally, μ=1; if the switch cannot be operated due to a line fault, μ=0, and the system will shield the switch and no longer perform operations on this switch.

[0028] (2) Action (A):

[0029] The control actions for emergency response include switch operation, energy storage device state of charge (SOC) operation, and load working state operation. Therefore, the action space is expressed as

[0030]

[0031] Among them, N switch It represents the number of switchable lines, including section switches and tie switches. The number of operable energy storage devices in the network is represented by N storage .

[0032] The line switch is controlled by the binary variable δ sw , 0 means the switch is open, and 1 means the switch is closed. The SOC of the energy storage device is represented by the variable δ st Indicates that 1 indicates that the energy storage device is charging, -1 indicates that the energy storage device is discharging, and 0 indicates that the energy storage device is idle.

[0033] The working state of the load is determined by δ ld Indicates that 1 means the load is in working state, and 0 means the load is in idle state.

[0034] (3) Transition probability P tr :

[0035] The transition probability captures the dynamic characteristics of the network under emergency response and is expressed as

[0036] P tr (s' t+1 |s t ,a t).

[0037] This means that after taking action a at time step t, the network state transitions from state s at time step t to state s' at time step t + 1. The transition probabilities are learned through the interaction of the agent with the environment.

[0038] (4) Reward (R):

[0039] The reinforcement learning algorithm on the reward-guided graph takes the optimal control action to mitigate the impact of failures in the energy storage network. The formula is as follows:

[0040]

[0041] The reward reflects the energy provided by maximizing E supp and minimize the voltage constraint violation V viol To improve the resilience of energy storage networks.

[0042] In order to consider the network imbalance under specific fault conditions and switch operations, the operation constraint C of the energy storage network is introduced in the reward. viol , proactively avoiding misalignments and constraint violations caused by faults or switching operations.

[0043] Due to topology changes caused by faults and switching operations, the energy storage network may contain multiple independent network parts, each with different active components (such as transformers, regulators, generators, loads, energy storage devices, etc.) and their corresponding state variables.

[0044] In some scenarios, the isolation of these components from a strong backup power source (substation) causes the network state to become dysregulated, making it difficult to achieve node power balance within a preset error range.

[0045] This misalignment causes the power flow to fail to converge, which is identified by a flag in the solver. At this point, the reward value is zero because the solver cannot accurately reflect the effect of the switch on the network state.

[0046] On the other hand, network operation with large voltage violations is not feasible as this would lead to immediate network collapse.

[0047] To prevent the agent from taking actions that lead to invalid states, a penalty term V is added to the reward viol The goal is to keep voltage levels within acceptable ranges and ensure sustainable network operations.

[0048] For each bus i∈N, the voltage violation beyond its upper and lower limits V is evaluated as follows after power flow estimation:

[0049]

[0050]

[0051] in, Represents a set of phase connections of a bus. Voltage measurements and supplied energy are estimated in per unit values ​​(pu) and are relative to the reference voltage kV of the corresponding network. base and reference power MVA base Perform calculations.

[0052] Per-unit calculations in power systems eliminate the unit problem and are equivalent to normalizing using their base values:

[0053]

[0054] Where |N| is the cardinality of the network bus set, ΔV max and ΔV min Indicates violations of maximum and minimum voltage limits, respectively.

[0055] Therefore, when a fault occurs, the agent can not only quickly identify the fault area and reconfigure the switch status, but also reconfigure the charge and discharge status of the energy storage device and the configuration of the load according to the actual network topology, thereby effectively isolating the fault and ensuring the normal operation of other parts of the network. In a dynamic environment, the agent can find an optimal path through repeated exploration and experience accumulation to minimize the impact of the fault on the network operation.

[0056] Furthermore, the dynamic network reconstruction method includes the construction of a learning architecture, which combines the graph capsule neural network and the policy gradient method to construct a policy network for learning energy storage network control and output corresponding actions to ensure the efficiency and safety of system operation.

[0057] Each node i in the energy storage network diagram has an active power demand Reactive power demand These node attributes are first projected into a high-dimensional space through linear transformation to generate initial node features

[0058]

[0059] in, is the learnable weight matrix, b init is a learnable bias vector, and h0 represents the dimension after projection. These features are then processed by a multi-layer graph capsule neural network, where each layer of the graph capsule neural network extracts node embeddings layer by layer through a polynomial graph convolution filter:

[0060]

[0061] Among them, L is the graph Laplacian matrix, p is the order of the statistical moment, k is the order of the convolution filter, and F (l-1) (X,L) is the output of the l-1th layer, is a learnable weight matrix. Node embedding not only contains the information of the node itself, but also integrates the information aggregated from multiple neighboring nodes.

[0062] As the number of graph convolution layers increases and the order of convolution filters increases, the model can capture more complex graph structure information.

[0063] The output of each layer is formed by concatenating the convolution results of all orders to form the final node embedding:

[0064]

[0065] After linear transformation, the final node embedding matrix F is generated Nodes =F (Le) (X,L)×W F The node embeddings are then processed through a series of linear transformations to generate a global graph embedding:

[0066] F graph =Mean(W g2 ×(W g1 ×F Nodes )) (7)

[0067] The embedding is obtained by taking the average of all node embeddings, thus providing a global state representation for the entire distribution network.

[0068] In addition to the graph structure information, some state variables in the energy storage network cannot be directly represented by the nodes in the graph, such as the supplied energy, voltage violations, and power flow on the edges. This information is crucial to the operational safety and operability of the network, so it is necessary to generate context embedding as additional state information through a feedforward neural network:

[0069] F context =F feedforward (Concat([E supp, V viol, l E ])) (8)

[0070] These contextual information and graph embeddings together constitute the state representation of the entire system, which is further processed by a multi-layer perceptron to generate the final state embedding:

[0071] F final =MLP(F graph +F context ) (9)

[0072] Furthermore, the dynamic network reconstruction method also includes a model training process, which is as follows:

[0073] (1) Data preparation and scenario generation:

[0074] Failures in energy storage systems often originate from localized faults, which can lead to cascading effects.

[0075] Specifically, faults in the network are mainly caused by line faults, which are simulated by a graph capsule neural network based approach.

[0076] To simulate this behavior, a subgraph approach with random edge removal is adopted.

[0077] The method involves randomly selecting nodes N from the energy storage system graph capsule neural network s ∈N, and create nodes with different radii R around these nodes s ≤R max (maximum radius) of the subgraph.

[0078] Setting R max =G dia / 2, where G dia is the diameter of the graph.

[0079] In each selected subgraph, randomly remove a portion of edges F s ∈E to simulate the impact of local accidents.

[0080] The proportion of edge failures gradually increases from 0 to 50%.

[0081] By changing the node N s , subgraph radius R s and remove edge F s , generating scenarios with multiple line faults for model training.

[0082] Furthermore, in each scenario, the load multipliers and generation points are varied by randomly selecting them from annual configuration files in the OpenDSS package with hourly resolution.

[0083] (2) Training process:

[0084] The training process generates samples simulating various fault scenarios on the energy storage network by introducing line faults, adjusting the capacity of loads and energy storage devices, and considering different fault conditions.

[0085] The faults in the network are mainly caused by distribution line faults, which are simulated using a graph capsule neural network-based method. The two key points of energy storage network operation, load demand and energy storage capacity, are randomly extracted from the annual load curve provided by OpenDSS.

[0086] Considering that the state and action space will expand rapidly with the increase of network scale, fault scenarios and number of devices, methods such as Q learning cannot provide a lot of storage and computing power in practice. Therefore, the Proximal Policy Optimization (PPO) algorithm is used to train the policy network.

[0087] The training process involves collecting experience in the form of tuples containing state, action, reward, and next state. PPO runs based on rollout operations, each of which consists of a fixed number of steps, Nsteps. Weight updates are performed in batches of Nbatch (≤Nsteps) after the rollout is completed, minimizing the cost function consisting of the policy gradient loss and the state value loss through backpropagation. The policy network is trained for a total of Ntotal steps.

[0088] The present invention also includes other components that enable it to be used normally, all of which are conventional means in the art. In addition, devices or components not limited in the present invention all adopt the existing technology in the art.

[0089] The beneficial effects of the present invention are as follows:

[0090] 1. In view of the phenomenon that traditional numerical data or image data cannot effectively represent the interconnection relationship between nodes and edges in large and complex networks, and thus cannot clearly define the dynamic characteristics and operating status of the energy storage network, the idea of ​​graph neural network is used to represent the energy storage network as a graph, and used as a state representation for reinforcement learning, thereby improving the real-time and accuracy of energy storage network management.

[0091] 2. The switch control method based on the idea of ​​energy discretization and digitization can reconstruct the network and isolate the fault area through switch operation when a fault occurs, which can significantly enhance the robustness and fault tolerance of the energy storage system, thereby ensuring the safety of the system and overcoming the problem that the traditional energy storage network cannot automatically reconstruct the network when a local fault occurs, thus leading to the spread of faults and waste of resources, thereby improving the economy of the system.

[0092] 3. An intelligent control mechanism for energy storage devices based on dynamic topology reconstruction is proposed. This mechanism can flexibly respond to load demand and fault conditions when the network topology changes by actively adjusting the charging and discharging status of the energy storage devices. It can also act as a power source for discharge when necessary to ensure continuous power supply in isolated island areas, thereby optimizing the operation efficiency and stability of the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 This is a schematic diagram of the structure of the energy storage system in Example 1.

[0094] Figure 2 This is a schematic diagram of capsule neural network modeling in Example 1.

[0095] Figure 3 Schematic diagram of the reinforcement learning process in Example 2.

[0096] Figure 4 This is the convergence diagram of the reward function for training the strategy model in Example 3; wherein, Figure 4 a is a comparison chart of DRO and MLP for the IEEE 34 bus distribution test network. Figure 4 b is a comparison diagram of DRO and MLP for the IEEE 123 bus distribution test network.

[0097] Figure 5 is the state of the decision variables obtained by different models in three scenarios in Example 4; wherein, Figure 5 a represents the switch decision result diagram of the IEEE 34 bus distribution test network, dark color indicates the switch is closed, and light color indicates the switch is open; Figure 5 b represents the decision result diagram of energy storage equipment in the IEEE 34 bus distribution test network, where yellow indicates that the energy storage equipment is idle, green indicates that the energy storage equipment is discharged, and red indicates that the energy storage equipment is charged; Figure 5 c represents the load state decision result of the IEEE 34 bus distribution test network. The dark color indicates that the load is working, and the light color indicates that the load is idle.

[0098] Figure 6 This is a schematic diagram of the DRO optimization results based on fault scenario 1 on the IEEE 34-bus distribution test network in Example 4.

[0099] Figure 7 is the state of the decision variables obtained by different models in three scenarios in Example 4; wherein, Figure 7 a represents the switch decision result diagram of the IEEE 123 bus distribution test network, where dark colors represent closed switches and light colors represent open switches; Figure 7 b represents the decision result diagram of energy storage equipment in the IEEE 123 bus distribution test network, where yellow indicates that the energy storage equipment is idle, green indicates that the energy storage equipment is discharged, and red indicates that the energy storage equipment is charged; Figure 7 c represents the load status decision result of the IEEE 123 bus distribution test network. The dark color indicates that the load is working, and the light color indicates that the load is idle.

[0100] Figure 8 This is a schematic diagram of DRO optimization results based on fault scenario 1 on the IEEE 123 bus distribution test network in Example 4.

[0101] Fig. 9 This is an energy comparison diagram provided by the IEEE 34 bus distribution test network and the IEEE 123 bus distribution test network under different model decisions in Example 4; wherein, Fig. 9a Comparison of the energy provided by the IEEE 34 bus distribution test network under three different fault conditions; Fig. 9 b Comparison of the energy provided by the IEEE 123 bus distribution test network under three different fault conditions.

[0102] Fig.10 The comparison of the voltage constraint violations of the IEEE 34 bus distribution test network and the IEEE 123 bus distribution test network under different model decisions in Example 4 is shown; wherein, Fig.10 a Comparison of the IEEE 34-bus distribution test network violating voltage constraints under three different fault conditions; Fig.10 b Comparison of voltage constraint violations of the IEEE 123 bus distribution test network under three different fault conditions. DETAILED DESCRIPTION

[0103] The technical solution of the present invention will be clearly and completely described below in conjunction with specific embodiments. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments.

[0104] Example 1

[0105] A dynamic network reconstruction strategy for a digital power storage system, the specific steps are as follows:

[0106] S1. Energy storage system construction; Figure 1 As shown, the energy storage system consists of an intelligent layer, a control layer and a data layer; the intelligent layer obtains the current state of the energy storage system, and extracts the node feature information of the energy storage network based on the graph neural network, makes decisions through the reinforcement learning agent, and the action information of the decision is transmitted to the control layer; the control layer receives the action information from the intelligent layer, generates control instructions according to the rules and transmits the control instructions to the data layer; the data layer includes the energy storage network, which includes power generation equipment, energy storage equipment, load equipment and a control center. The energy storage network receives the control instructions from the control layer to make adjustments, and updates the energy storage network, sends the update information to the control center, and feeds back the update information to the control layer, which is then transmitted to the intelligent layer by the control layer;

[0107] S2. Energy storage network modeling; Graph capsule neural network is used to learn the optimal control strategy in the energy storage network. Compared with other graph capsule neural networks, it can more effectively capture the structural information of the energy storage network, and the enhanced state representation helps to calculate better actions than traditional feature abstraction networks (such as MLP); through graph capsule neural network construction, feature extraction, capsule encoding, dynamic routing and graph capsule neural network update, the global information of the energy storage system is gradually extracted and optimized;

[0108] like Figure 2As shown in the figure, first, a graph capsule neural network is used to extract node embeddings to form primary capsules; then, an attention module is applied to scale the node embeddings, and then graph capsules are generated through dynamic routing; finally, dynamic routing is applied again to perform graph classification;

[0109] The graph capsule neural network is constructed with the devices and connection relationships in the energy storage network as nodes and edges respectively, which can be represented as a graph structure G = (N, E), where nodes represent energy storage devices, substations, loads, etc., and edges represent physical or logical connections between devices, such as transmission lines or communication links.

[0110] Based on the graph capsule neural network, corresponding features are extracted for each node and edge, such as the capacity, charging and discharging status, and location of the energy storage device, the transmission capacity and distance of the edge, etc.

[0111] The graph capsule neural network encodes the features of each node and edge, and each node and edge is represented as a capsule, which is composed of a set of feature vectors. These feature vectors not only describe the properties of nodes and edges, but also capture the spatial and functional relationships between them.

[0112] Through a dynamic routing algorithm, information is transmitted between capsules, and features are aggregated and combined layer by layer to capture high-order structural information of the energy storage network.

[0113] Finally, combined with the message passing mechanism of the graph capsule neural network, the feature vector of the capsule is used to iteratively update the graph structure, gradually extract the global information of the energy storage system, and achieve in-depth understanding and optimization of the entire network.

[0114] S3. Dynamic network reconstruction: In order to quickly implement effective emergency response measures when a fault occurs, a combined control method based on the concept of energy discretization and digitization is proposed, covering two modules: reconstruction of the energy storage network topology and reconfiguration of energy storage devices and load devices.

[0115] This method can achieve rapid isolation of fault areas, dynamically adjust the working mode of energy storage equipment, and reconfigure the operating status of energy storage equipment and load equipment by systematically optimizing the topological structure and operating strategy of the energy storage network, thereby ensuring that the energy storage network can maintain efficient and stable operation under various fault scenarios.

[0116] Specifically, the core of this method is to discretize and digitize the energy flow, and to more finely manage and control the energy distribution and transmission of each node and edge.

[0117] This method follows Kirchhoff's law to change the line impedance and rearranges the network structure through switching operations, thereby alleviating faults. This allows the system to respond quickly when a fault occurs, flexibly adjust the network's topology and operating strategies, minimize the impact of the fault on the overall system, and improve fault recovery efficiency.

[0118] After isolating the fault area and reconstructing the network topology, the system can further optimize the network's operating efficiency and stability by actively adjusting the charging and discharging status of the energy storage equipment.

[0119] When the network topology changes, the system will not only reallocate the energy transmission path, but also dynamically adjust the working mode of the energy storage equipment according to the current load demand, fault conditions and the status of each energy storage node.

[0120] Energy storage devices with higher energy storage capacity can be regarded as power sources. When the nodes in the faulty area cannot be connected to the power generation nodes, the energy storage devices discharge to ensure that the isolated areas that are not connected to the power generation nodes can still work for a period of time.

[0121] Example 2

[0122] Based on the dynamic network reconstruction strategy of the digital power energy storage system described in Example 1, the emergency response to faults in the energy storage network is expressed as a Markov decision process (MDP) in the graph domain, denoted as M=(S, A, P tr ,R);

[0123] like Figure 3 As shown, this tuple represents the state, action, transition probability and reward respectively, and is defined as follows:

[0124] (1) Status (S):

[0125] The state consists of relevant observations in the energy storage network, representing the current operating status of the network. It includes node variables, edge variables, network topology, and other system variables, expressed as

[0126]

[0127] in, Represents estimated or predicted values ​​of active and reactive power demand of a node; Represents the active and reactive power generation of the node; V N Represents the three-phase voltage measured at the bus, that is, the node in the diagram; V viol Indicates the voltage violation constraint value in the network; the edge variable includes the power flow through the network branch, represented by l E ; The operating topology of the network is T, and the total energy supply in the network is represented by E supp;The state variable O includes the fault scenario, i.e., multi-line faults in the network, including switch faults;

[0128] The inoperability of the faulty switch is handled through a shielding mechanism, which is represented by the state variable μ; that is, if the switch operates normally, μ=1; if the switch cannot be operated due to a line fault, μ=0, and the system will shield the switch and no longer perform operations on this switch.

[0129] (2) Action (A):

[0130] The control actions for emergency response include switch operation, energy storage device state of charge (SOC) operation, and load working state operation. Therefore, the action space is expressed as

[0131]

[0132] Among them, N switch It represents the number of switchable lines, including section switches and tie switches. The number of operable energy storage devices in the network is represented by N storage .

[0133] The line switch is controlled by the binary variable δ sw , 0 means the switch is open, and 1 means the switch is closed. The SOC of the energy storage device is represented by the variable δ st Indicates that 1 indicates that the energy storage device is charging, -1 indicates that the energy storage device is discharging, and 0 indicates that the energy storage device is idle.

[0134] The working state of the load is determined by δ ld Indicates that 1 means the load is in working state, and 0 means the load is in idle state.

[0135] (3) Transition probability P tr :

[0136] The transition probability captures the dynamic characteristics of the network under emergency response and is expressed as

[0137] P tr (s' t+1 |s t ,a t ).

[0138] This means that after taking action a at time step t, the network state transitions from state s at time step t to state s' at time step t + 1. The transition probabilities are learned through the interaction of the agent with the environment.

[0139] (4) Reward (R):

[0140] The reinforcement learning algorithm on the reward-guided graph takes the optimal control action to mitigate the impact of failures in the energy storage network. The formula is as follows:

[0141]

[0142] The reward reflects the energy provided by maximizing E supp and minimize the voltage constraint violation V viol To improve the resilience of energy storage networks.

[0143] In order to consider the network imbalance under specific fault conditions and switch operations, the operation constraint C of the energy storage network is introduced in the reward. viol , proactively avoiding misalignments and constraint violations caused by faults or switching operations.

[0144] Due to topology changes caused by faults and switching operations, the energy storage network may contain multiple independent network parts, each with different active components (such as transformers, regulators, generators, loads, energy storage devices, etc.) and their corresponding state variables.

[0145] In some scenarios, the isolation of these components from a strong backup power source (substation) causes the network state to become dysregulated, making it difficult to achieve node power balance within a preset error range.

[0146] This misalignment causes the power flow to fail to converge, which is identified by a flag in the solver. At this point, the reward value is zero because the solver cannot accurately reflect the effect of the switch on the network state.

[0147] On the other hand, network operation with large voltage violations is not feasible as this would lead to immediate network collapse.

[0148] To prevent the agent from taking actions that lead to invalid states, a penalty term V is added to the reward viol The goal is to keep voltage levels within acceptable ranges and ensure sustainable network operations.

[0149] For each bus i∈N, the voltage violation beyond its upper and lower limits V is evaluated as follows after power flow estimation:

[0150]

[0151] in, Represents a set of phase connections of a bus. Voltage measurements and supplied energy are estimated in per unit values ​​(pu) and are relative to the reference voltage kV of the corresponding network. base and reference power MVA base Perform calculations.

[0152] Per-unit calculations in power systems eliminate the unit problem and are equivalent to normalizing using their base values:

[0153]

[0154] Where |N| is the cardinality of the network bus set, ΔV max and ΔV min Indicates violations of maximum and minimum voltage limits, respectively.

[0155] Therefore, when a fault occurs, the agent can not only quickly identify the fault area and reconfigure the switch status, but also reconfigure the charge and discharge status of the energy storage device and the configuration of the load according to the actual network topology, thereby effectively isolating the fault and ensuring the normal operation of other parts of the network. In a dynamic environment, the agent can find an optimal path through repeated exploration and experience accumulation to minimize the impact of the fault on the network operation.

[0156] Example 3

[0157] Based on the dynamic network reconstruction strategy of the digital power energy storage system described in Example 2, the dynamic network reconstruction method also includes the construction of a learning architecture, combining the graph capsule neural network and the policy gradient method to construct a policy network for learning energy storage network control and output corresponding actions to ensure the efficiency and safety of system operation.

[0158] Each node i in the energy storage network diagram has an active power demand Reactive power demand These node attributes are first projected into a high-dimensional space through linear transformation to generate initial node features

[0159]

[0160] in, is the learnable weight matrix, b init is a learnable bias vector, and h0 represents the dimension after projection. These features are then processed by a multi-layer graph capsule neural network, where each layer of the graph capsule neural network extracts node embeddings layer by layer through a polynomial graph convolution filter:

[0161]

[0162] Among them, L is the graph Laplacian matrix, p is the order of the statistical moment, K is the order of the convolution filter, and F (l-1) (X,L) is the output of the l-1th layer, is a learnable weight matrix. Node embedding not only contains the information of the node itself, but also integrates the information aggregated from multiple neighboring nodes.

[0163] As the number of graph convolution layers increases and the order of convolution filters increases, the model can capture more complex graph structure information.

[0164] The output of each layer is formed by concatenating the convolution results of all orders to form the final node embedding:

[0165]

[0166] After linear transformation, the final node embedding matrix F is generated Nodes =F (Le) (X,L)×W F The node embeddings are then processed through a series of linear transformations to generate a global graph embedding:

[0167] F graph =Mean(W g2 ×(W g1 ×F Nodes )) (7)

[0168] The embedding is obtained by taking the average of all node embeddings, thus providing a global state representation for the entire distribution network.

[0169] In addition to the graph structure information, some state variables in the energy storage network cannot be directly represented by the nodes in the graph, such as the supplied energy, voltage violations, and power flow on the edges. This information is crucial to the operational safety and operability of the network, so it is necessary to generate context embedding as additional state information through a feedforward neural network:

[0170] F context =F feedforward (Concat([E supp, V viol, l E ])) (8)

[0171] These contextual information and graph embeddings together constitute the state representation of the entire system, which is further processed by a multi-layer perceptron to generate the final state embedding:

[0172] F final =MLP(F graph +F context ) (9)

[0173] Example 4

[0174] Based on the dynamic network reconstruction strategy of the digital power storage system described in Example 3, the dynamic network reconstruction method also includes a model training process, which is as follows:

[0175] (1) Data preparation and scenario generation:

[0176] Failures in energy storage systems often originate from localized faults, which can lead to cascading effects.

[0177] Specifically, faults in the network are mainly caused by line faults, which are simulated by a graph capsule neural network based approach.

[0178] To simulate this behavior, a subgraph approach with random edge removal is adopted.

[0179] The method involves randomly selecting nodes N from the energy storage system graph capsule neural network s ∈N, and create nodes with different radii R around these nodes s ≤R max (maximum radius) of the subgraph.

[0180] Setting R max =G dia / 2, where G dia is the diameter of the graph.

[0181] In each selected subgraph, randomly remove a portion of edges F s ∈E to simulate the impact of local accidents.

[0182] The proportion of edge failures gradually increases from 0 to 50%.

[0183] By changing N s , R s and F s , generating scenarios with multiple line faults for model training.

[0184] Furthermore, in each scenario, the load multipliers and generation points are varied by randomly selecting them from annual configuration files in the OpenDSS package with hourly resolution.

[0185] (2) Training process:

[0186] The training process generates samples simulating various fault scenarios on the energy storage network by introducing line faults, adjusting the capacity of loads and energy storage devices, and considering different fault conditions.

[0187] The faults in the network are mainly caused by distribution line faults, which are simulated using a graph capsule neural network-based method. The two key points of energy storage network operation, load demand and energy storage capacity, are randomly extracted from the annual load curve provided by OpenDSS.

[0188] Considering that the state and action space will expand rapidly with the increase of network scale, failure scenarios and number of devices, methods such as Q learning cannot provide a lot of storage and computing power in practice. Therefore, the Proximal Policy Optimization (PPO) algorithm is used to train the policy network, which is implemented in the stable-baselines3 Python library.

[0189] The training process involves collecting experience in the form of tuples containing state, action, reward, and next state. PPO runs based on rollout operations, each of which consists of a fixed number of steps Nsteps. Weight updates are performed in batches of Nbatch (≤Nsteps) after the rollout is completed, minimizing the cost function consisting of policy gradient loss and state value loss through backpropagation. The policy network is trained for a total of Ntotal steps.

[0190] In order to evaluate the performance of the proposed model and to assess the impact of local and global structural information in the encoding process, a comparative experiment was conducted with another learning-based framework, MLP. This framework also uses the PPO algorithm, and the policy network is based on the MLP architecture. To ensure a fair comparison, MLP and DRO were trained under the same settings.

[0191] It is worth noting that for any given or existing network, this process only needs to be performed offline one or several times, the trained network model does not need to be retrained, and current computing resources are fully sufficient to support this operation. Therefore, for the training of real-time fault management decision support systems, this is a reasonable offline investment, because once the system is trained, it can be used for real-time decision support for a long time.

[0192] (3) Training details:

[0193] Take the IEEE 34 bus distribution test network and the IEEE 123 bus distribution test network in the IEEE typical model as examples:

[0194] In the IEEE 34-bus distribution test network, the MLP can be trained for up to 1.5 million steps. In the IEEE 123-bus distribution test network, the MLP can be trained for up to 500,000 steps.

[0195] To ensure fairness, the same settings were applied to DRO.

[0196] Specifically, DRO is trained for 1.5 million steps in the IEEE 34-bus distribution test network and 0.5 million steps in the IEEE 123-bus distribution test network.

[0197] Furthermore, the Adam optimization algorithm is used to enhance the training process.

[0198] like Figure 4 As shown in the figure, the real-time reward convergence value of the trained DRO model is always better than the current baseline solution, and the DRO model converges faster. This observation proves the excellent performance of DRO in effectively managing network faults and optimizing the operating status of energy storage networks.

[0199] The present invention also uses the open source distribution system simulator OpenDSS to implement and simulate the energy storage network. The energy storage network is modeled using the general generator, solar photovoltaic elements, and energy storage elements in OpenDSS.

[0200] Switches are defined on the lines with associated switch controls, while the disabling or enabling properties of the loads are used to unload or pick up the loads, and the energy storage devices adjust the charging, discharging or idle state according to the current capacity and the system status. The implementation of the switching operation may lead to the formation of multiple sections within the network, which can achieve the isolation of the faulty part of the network and the normal operation of other areas. The energy storage device in the discharged state can be regarded as a power source, providing energy to the node isolated from the power generation equipment due to the fault.

[0201] The proposed framework is evaluated using an improved version of the IEEE 34 bus distribution test network as a case study energy storage network. Nine switches are considered for the line switches in the IEEE 34 bus distribution test network. All buses except the substation bus are considered to be load buses or energy storage device buses. The total grid-connected load of the network is 2.04MW. A grid-forming distributed power source with a capacity of 146kW is connected to the 890 bus, a grid feeder distributed power source with a capacity of 96kW is connected to the 820 bus, and five energy storage devices with energy storage capacities of 20%, 80%, 90%, 50%, and 10% are connected at 860, 844, 816, 836, and 822, respectively. Loads are connected at 808, 818, 816, 824, 828, 854, and 832.

[0202] In the experiment, three different fault scenarios are set up, each consisting of three different line faults. Figure 5 Shown are the states of decision variables obtained in different models in the three scenarios. Figure 5 a represents the switch decision result diagram of the IEEE 34 bus distribution test network, where dark colors indicate that the switch is closed and light colors indicate that the switch is open.

[0203] Figure 5 b represents the decision result diagram of energy storage equipment in the IEEE 34 bus distribution test network. Yellow indicates that the energy storage equipment is idle, green indicates that the energy storage equipment is discharged, and red indicates that the energy storage equipment is charged. Figure 5 c represents the load state decision result of the IEEE 34 bus distribution test network. The dark color indicates that the load is working, and the light color indicates that the load is idle.

[0204] On the IEEE 34 bus distribution test network, the network reconstruction and transmission optimization results based on fault scenario 1 are shown in Figure 2. Figure 6As shown in the figure, fault scenario 1 sets four connection faults: 814-828, 828-830, 834-858, and 888-890. The switches at 814-828 and 828-830 cannot be turned off.

[0205] 834-858, as a highly central connection line, also failed. After the failure, the grid-forming distributed power source at the 890 bus was disconnected from the overall network, and the energy storage devices at 816 and 844, with high capacity, acted as power sources for discharge. By performing switch operations, the network structure was reconstructed, the direction of energy flow was changed, and the energy storage network was able to continue to operate.

[0206] In this embodiment, an experiment is also conducted on the IEEE 123 bus distribution test network. Similar to the experiment on the IEEE 34 bus distribution test network, an improved version of the IEEE 123 bus distribution test network is used as the energy storage network for the research case, and 22 switches are considered to be configured.

[0207] A grid-forming distributed power source with a capacity of 259kW is connected to busbars 39, 46, 71, 75, and 79 respectively, a grid feeder distributed power source with a capacity of 80kW is connected to busbars 11, 82, 91, and 104 respectively, 7 energy storage devices with storage capacities of 90%, 80%, 70%, 50%, 10%, 80%, and 90% are connected at 33, 56, 82, 84, 92, 96, and 108 respectively, and loads are connected at 16 buses: 11, 16, 20, 22, 30, 48, 49, 53, 64, 65, 66, 69, 88, 107, 109, and 113.

[0208] Three different fault scenarios were set up in the experiment, each consisting of three different line faults. Figure 7 As shown, the states of decision variables obtained in different models for the three scenarios include Figure 7 a shows the status of the circuit breaker, Figure 7 b shows the status of the energy storage device. Figure 7 c shows the status of the load.

[0209] On the IEEE 123 bus distribution test network, the network reconstruction and transmission optimization results based on fault scenario 1 are shown in the figure below: Figure 8 As shown in the figure. Fault scenario 1 sets three connection faults at 13-18, 51-151, and 65-66. After the fault occurs, the energy storage devices at the four buses 33, 56, 96, and 100 have higher capacity and act as power sources for discharge. By performing switch operations, the network structure is reconstructed, the direction of energy flow is changed, and the energy storage network can continue to operate.

[0210] In the experiment, the present invention modified the IEEE 34 bus distribution test network and the IEEE 123 bus distribution test network, and constructed the corresponding IEEE 34 bus distribution test network and IEEE 123 bus distribution test network energy storage network. The traditional model, a reinforcement learning model using a multilayer perceptron MLP as a policy network and a baseline model were used to compare the performance of the DRO model and the MLP model proposed in the present invention in real-time decision-making and network resilience improvement.

[0211] The energy supplied in the event of an interruption can be used as a measure of resilience, e.g. Fig. 9 The results show that the DRO model proposed in the present invention provides higher energy in different network and fault scenarios, ensuring the economy of the energy storage system.

[0212] In addition, compared with the MLP baseline model, the DRO model better captures the topology of the distribution network in the decision-making process, has less voltage violation constraints, shows better performance and better complies with the constraints, ensuring the safety of the energy storage system, such as Fig.10 shown.

[0213] Three different fault conditions were proposed for the two test networks, and all of them can make correct decisions to maintain the continuous and stable operation of the energy storage network without retraining the model. This proves the excellent generalization ability of DRO. In networks of the same size, only one training is needed to deal with all fault conditions, saving training time and computing resources, and providing guarantees for subsequent practical applications.

[0214] The real-time energy storage network fault management model DRO based on graph reinforcement learning proposed in this invention successfully integrates the topological information of the energy storage network into the reinforcement learning decision-making framework, effectively improving the model's adaptability and fault management efficiency in a dynamic environment.

[0215] Through the comparison of experimental results, the DRO model not only outperforms traditional methods in terms of energy supply, but also demonstrates good decision-making accuracy and network reconstruction capabilities in complex scenarios facing different faults and network changes. In addition, the model fully mines the local and global structural information of the energy storage network by integrating the graph capsule neural network, providing strong support for coping with complex energy storage equipment management and charging and discharging scheduling.

[0216] The technical solution of the present invention is not limited to the above-mentioned specific embodiments. Without departing from the scope and spirit of the described embodiments, many modifications and changes are obvious to ordinary technicians in this technical field. Any technical deformation made within the spirit and principles of the present invention falls within the protection scope of the present invention.

Claims

1. A dynamic network reconstruction strategy for a digital power storage system, characterized in that: Here are the steps: S1. Energy storage system construction; the energy storage system consists of an intelligent layer, a control layer and a data layer; the intelligent layer obtains the current state of the energy storage system, and extracts the node feature information of the energy storage network based on the graph neural network, and transmits the action information decided by the reinforcement learning agent to the control layer; the control layer receives the action information, and generates control instructions according to the rules and transmits them to the data layer; the data layer includes the energy storage network, the energy storage network receives the control instructions and makes adjustments, and updates the energy storage network to the control center; S2. Energy storage network modeling; Graph capsule neural network is used to learn the optimal control strategy of energy storage network. The graph capsule neural network is constructed with the devices and connection relationships in the energy storage network as nodes and edges respectively. On the basis of the graph capsule neural network, the corresponding features are extracted for each node and edge, and the features of each node and edge are encoded. Each node and edge is represented as a capsule, which consists of a set of feature vectors. Through the dynamic routing algorithm, information is transmitted between capsules, and features are aggregated and combined layer by layer to capture the high-order structural information of the energy storage network. Then, combined with the message passing mechanism of the graph capsule neural network, the feature vector of the capsule is used for iterative update to gradually extract and optimize the global information of the energy storage system. S3. Dynamic network reconstruction: A combined control method based on the concepts of energy discretization and digitization, covering the reconstruction of the energy storage network topology and the reconfiguration of energy storage devices and load devices. By systematically optimizing the topology of the energy storage network and the optimal control strategy, it can achieve rapid isolation of the fault area and dynamic adjustment and reconfiguration of the working modes of energy storage devices and load devices. A fault emergency response model is generated based on the Markov decision process to ensure that the energy storage network can still operate stably under fault scenarios.

2. A digital power storage system dynamic network reconstruction strategy according to claim 1, characterized in that: Step S3 includes the construction of a learning architecture; by combining the graph capsule neural network with the policy gradient method, a policy network for learning energy storage network control is constructed and the corresponding actions are output to ensure the efficiency and safety of system operation.

3. A digital power storage system dynamic network reconstruction strategy according to claim 2, characterized in that: Step S3 also includes a model training process; by introducing line faults, adjusting the capacity of load equipment and energy storage equipment, and considering different fault scenarios, samples simulating various fault scenarios are generated.

4. A digital power storage system dynamic network reconstruction strategy according to claim 1, characterized in that: The Markov decision process for emergency response to faults in the energy storage network in step S3 is denoted as M=(S,A,P tr ,R) Among them, S, A, P tr , R represent state, action, transition probability and reward respectively.

5. A digital power storage system dynamic network reconstruction strategy according to claim 4, characterized in that: The state S consists of relevant observations in the energy storage network and represents the current operating status of the energy storage network; Including node variables, edge variables, network topology and other system variables, expressed as in, Represents estimated or predicted values ​​of active and reactive power demand of a node; Represents the active and reactive power generation of the node; V N Represents the three-phase voltage measured by the bus node; V viol Indicates that the voltage in the energy storage network violates the constraint value; l E represents the power flow through the energy storage network branch; the operating topology of the network is represented by T; the total energy supply in the energy storage network is represented by E supp ; The state variable O represents the fault scenario; μ represents the inoperability of the faulty switch handled by the shielding mechanism; if the switch operates normally, μ=1; if the switch fails and cannot be operated, μ=0, so the switch is shielded.

6. A digital power storage system dynamic network reconstruction strategy according to claim 5, characterized in that: Action A includes switch operation, energy storage device charging state operation and load working state operation; the action space is expressed as Among them, N switch represents the number of switchable lines, including section switches and tie switches; the number of operable energy storage devices in the energy storage network is represented by N storage ; The line switch is controlled by the binary variable δ sw Indicates that 0 means the switch is open and 1 means the switch is closed; binary variable δ st Indicates the energy storage charging state, 1 indicates that the energy storage device is charging, -1 indicates that the energy storage device is discharging, and 0 indicates that the energy storage device is idle; binary variable δ ld Indicates the working status of the load. 1 indicates that the load is in working state, and 0 indicates that the load is in idle state.

7. A digital power storage system dynamic network reconstruction strategy according to claim 6, characterized in that: Transition probability P tr It is used to capture the dynamic characteristics of the energy storage network under fault emergency response, expressed as P tr (s' t+1 |s t ,a t ) It means that after action a is implemented at time step t, the state of the energy storage network is transferred from state s at time step t to state s' at time step t+1, and the transition probability is learned through interactive reinforcement learning between the agent and the environment.

8. A digital power storage system dynamic network reconfiguration strategy according to claim 7, characterized in that: The reward R is used to take the optimal control action through the reinforcement learning algorithm to alleviate the impact of failures in the energy storage network. The formula is: By maximizing the energy supply E supp and minimize the voltage constraint violation V viol , to improve the resilience of the energy storage network, and introduce the operation constraint C of the energy storage network viol , to proactively avoid misalignments and constraint violations caused by faults or switching operations.

9. A digital power storage system dynamic network reconfiguration strategy according to claim 2, characterized in that: In the process of building the learning architecture, the attributes of each node in the energy storage network are projected into a high-dimensional space through linear transformation to generate initial node features. The initial node features are processed by a multi-layer graph capsule neural network, and the node embedding is extracted layer by layer through the graph convolution filter. The output of each layer is formed by concatenating the convolution results of all orders to form the final node embedding. The final node embedding matrix is ​​generated through linear transformation, and then a series of linear transformations are processed to generate the global graph embedding, thereby providing a global state representation for the energy storage network.

10. A digital power storage system dynamic network reconfiguration strategy according to claim 3, characterized in that: In the early stage of the model training process, data preparation and scenario generation are required. Based on the graph capsule neural network, the subgraph method with random edge removal is used to simulate the fault scenarios in the energy storage network. By randomly selecting nodes from the energy storage system graph capsule neural network, subgraphs with different radii are created around the selected nodes; within each selected subgraph, a portion of the edges are randomly removed to simulate the impact of local faults, and by changing the selected nodes, subgraph radius and removed edges, scenarios with multiple line faults are generated for model training.

Images