A method for energy storage EMS fault ride-through adaptive control based on reinforcement learning

Abstract

The application discloses a kind of energy storage EMS fault crossing adaptive control methods based on reinforcement learning, comprising the following steps: S1, based on energy storage EMS system, constructs topological symbol dictionary and topological constraint graph, involve power energy storage technical field, the present application is by constructing topological symbol dictionary, topological constraint graph and extracting power grid and local measurement signal characteristics, generate fault agent state vector, based on fault agent state vector by barrier function check, implementability constraint screening, time coordination and multi-objective decision, while the action meets the safety requirement of equipment, filter out the safe executable action that can maximize grid support margin, optimize power regulation, so that effective support to power grid can be realized in fault crossing process, and the occurrence of device overload, circulating current risk safety problem can be reduced, realizes the dynamic balance of fault support efficiency and equipment safety, effectively alleviate the contradiction between the effectiveness of fault support and equipment safety.

Description

Technical Field

[0001] This invention relates to the field of power energy storage technology, and in particular to an adaptive control method for fault ride-through of energy storage EMS based on reinforcement learning. Background Technology

[0002] Energy storage EMS refers to the energy management system designed for energy storage power stations or integrated energy storage systems. Fault ride-through refers to the ability of connected power generation equipment or energy storage equipment to maintain grid-connected operation and not immediately disconnect from the grid when a grid fault causes abnormal fluctuations in voltage or frequency at the grid connection point, and even to provide necessary support to the grid as required.

[0003] Existing fault ride-through control methods for energy storage EMS systems have the following problems in practical applications: During grid connection failures, current control methods mainly rely on rule-based control logic or traditional optimization algorithms, lacking a reliable control action screening mechanism. Blindly executing cross-through support actions can lead to some actions meeting short-term support requirements but potentially causing secondary failures such as device overvoltage or circulating current surges due to exceeding safety boundaries, thus reducing the reliability of the fault cross-through process. The generated control commands often have the problem of triggering circulating currents, affecting the reliability of the EMS system. Summary of the Invention

[0004] To address the technical problems existing in the background art, this invention proposes an adaptive control method for fault ride-through of energy storage EMS based on reinforcement learning.

[0005] This invention proposes a reinforcement learning-based adaptive control method for fault ride-through in energy storage systems (EMS), comprising the following steps: S1. Based on the energy storage EMS system, construct a topology symbol dictionary and a topology constraint graph; acquire grid measurement signals and local measurement signals of the energy storage EMS system during the grid connection fault period and extract features; combine the topology symbol dictionary and the topology constraint graph to construct a topology embedding vector to generate a fault proxy state vector. As an explanation, an energy storage EMS system, also known as an energy storage management system, is an existing control system used to intelligently monitor, schedule, and optimize the energy flow, operating status, and safety performance of an energy storage system. S2. Obtain current continuity constraints, energy conservation constraints, and no-circulation current criteria, construct a safety invariant set, and generate a fault ride-through evaluation value by combining the fault proxy state vector. S3. Based on the energy storage EMS system, construct a topology reconfiguration action set and a multi-source energy allocation action set according to the topology symbol dictionary and topology constraint diagram, and generate a candidate action set by combining the fault agent state vector. S4. Analyze the candidate action set to obtain safe and executable actions.

[0006] Preferably, in S1, based on the energy storage EMS system, a topology symbol dictionary and a topology constraint diagram are constructed as follows: Obtain the hardware switchable power path and the preset switching order of the hardware switchable power path in the energy storage EMS system, and generate a topology symbol dictionary by symbolically encoding the hardware switchable power path and the preset switching order of the hardware switchable power path. The path connectivity and mutual exclusion relationships between hardware switchable power paths are generated based on the topology symbol dictionary, and a topology constraint graph is generated based on the path connectivity and mutual exclusion relationships between hardware switchable power paths.

[0007] Preferably, in S1, the grid measurement signals and local measurement signals of the energy storage EMS system during the grid connection fault period are acquired and their features are extracted. A topology embedding vector is then constructed by combining the topology symbol dictionary and the topology constraint graph to generate a fault proxy state vector, as follows: Based on the grid measurement signals and local measurement signals of the energy storage EMS system during the grid connection fault period, voltage deviation characteristics, deviation trend characteristics, phase-locked reference phase characteristics, energy storage state of charge characteristics, thermal stress characteristics, and DC ripple characteristics are extracted. Based on the grid measurement signals and local measurement signals of the energy storage EMS system during the grid connection fault period, the instantaneous power flow between each hardware switchable power path and the grid, and the equivalent impedance between the hardware switchable power path and the grid are obtained. The instantaneous power flow and equivalent impedance are graph-embedded using a topology symbol dictionary and a topology constraint graph to obtain a topology embedding vector; The fault proxy state vector is generated by concatenating voltage deviation characteristics, deviation trend characteristics, phase-locked reference phase characteristics, energy storage state of charge characteristics, thermal stress characteristics, DC ripple characteristics, and topology embedding vector.

[0008] Preferably, in S2, current continuity constraints, energy conservation constraints, and no-circulating current criteria are obtained to construct a safety-invariant set. This set is then combined with the fault proxy state vector to generate a fault ride-through evaluation value, as follows: By obtaining the current continuity constraint, energy conservation constraint, and no-circulation current criterion, a safe and invariant set jointly defined by the current continuity constraint, energy conservation constraint, and no-circulation current criterion is obtained; Based on the fault agent state vector and combined with the security invariant set, a barrier function term is generated according to the preset barrier function. By statistically analyzing the grid measurement signals and local measurement signals of the energy storage EMS system during the grid connection fault period, we can obtain the grid connection support margin benefits, device lifetime and thermal cost penalties, circulating current suppression penalties, and power quality constraint benefits. The reward function shaping algorithm combines the barrier function term with the grid-connected support margin benefit, device lifetime and thermal cost penalty, circulating current suppression penalty, and power quality constraint benefit to generate a fault ride-through evaluation value.

[0009] Preferably, in S3, based on the energy storage EMS system, a topology reconfiguration action set and a multi-source energy allocation action set are constructed according to the topology symbol dictionary and topology constraint diagram. Combined with the fault agent state vector, a candidate action set is generated as follows: Based on the topology symbol dictionary and topology constraint graph, obtain directed switching sequences that satisfy the legal switching order and no reentrancy, form a set of directed switching sequences, and use the set of directed switching sequences as the topology reconstruction action set; Obtain the energy storage path of the energy storage EMS system, obtain the allocation coefficients that satisfy the constraints of energy conservation and power balance based on the energy storage path, form a set of allocation coefficients, and use the set of allocation coefficients as a set of multi-source energy allocation actions; The joint action space is formed by the Cartesian product of the topological reconstruction action set and the multi-source energy allocation action set; Based on the fault agent state vector, a policy function is used to map the fault agent state vector to the joint action space. The mapping result is then subjected to a feasibility screening using preset rules to generate a set of candidate actions. Among them, the mapping results are subjected to a feasibility screening using preset rules, which include preset security rules, hardware limitations, action compatibility requirements, and preset thresholds for fault traversal evaluation values.

[0010] Preferably, in S4, the set of candidate actions is analyzed to obtain safe and executable actions, as follows: Obtain the library of feasible constraints, which includes switch interlocking, prohibition of reentry, minimum converter interval, phase alignment, power balance, and available energy constraints for energy storage. Each candidate action in the candidate action set is validated using a preset barrier function and a library of implementable constraints. Candidate actions that do not meet the requirements of the preset barrier function or the library of implementable constraints are deleted. The remaining candidate actions are used to generate a set of barrier-enabled actions. As an explanation, the set of barrier actionable actions is a set of candidate actions that satisfy both the barrier function requirements and the requirements of the implementability constraint library. The projection operator technique is used to project the candidate action set into the barrier action set to generate the first projected action. Based on the non-circulating flow criterion, a non-circulating flow determination criterion is generated by combining the action after the first projection. The actions after the first projection are filtered according to the non-circulating flow determination criterion to generate a set of non-circulating flow actions. The non-circulating current actions in the non-circulating current action set are converted into command signals, generating a topology control signal set, a power regulation signal set, and a topology trigger sequence candidate set. The topology control signal set and power regulation signal set are synchronized and interlocked according to the phase-locked reference phase, sampling period and dead time to generate a time-coordinated candidate instruction set. Based on the time-coordinated candidate instruction set, a multi-objective decision algorithm is used to comprehensively score the topology control signal and its corresponding power adjustment signal in the time-coordinated candidate instruction set. The topology control signal and its corresponding power adjustment signal are sorted from high to low according to the comprehensive score, and the top-ranked topology control signal and its corresponding power adjustment signal are selected to generate a safe and executable action.

[0011] Preferably, the first projected actions are filtered according to the non-circulating determination criterion to generate a non-circulating action set, as follows: Based on the topological constraint graph, extract all possible branch connection relationships that can form closed loops from the candidate action set to generate a loop description set; Based on the loop description set, obtain the potential difference between adjacent branches caused by the action after the first projection; After obtaining the first projection, the current direction and preset power flow direction of each branch in the closed loop are obtained, and the deviation between the current direction and the preset flow direction is obtained by vector operation. The probability of circulating current risk for the first projected action is generated based on the potential difference between adjacent branches and the deviation of the current direction from the preset flow direction. The probability of circulating current risk is generated based on the potential difference between adjacent branches and the deviation of the current direction from the preset flow direction, which can be achieved through a weighted fusion algorithm; The first post-projection action is verified using a preset barrier function. The first post-projection action that does not meet the preset barrier function requirements is deleted, and the first post-projection action that meets the preset barrier function requirements is used as the second post-projection action. The second projected action with a circulation risk probability less than a preset threshold is considered a non-circulating action, and a set of non-circulating actions is generated.

[0012] Preferably, the non-circulating current actions in the non-circulating current action set are converted into command signals to generate a topology control signal set, a power regulation signal set, and a topology trigger sequence candidate set, as follows: The acyclic actions in the acyclic action set include topology remodeling-related actions and energy allocation-related actions; When converting non-circulating current actions in the non-circulating current action set into command signals, topology reconfiguration-related actions generate topology control signals, forming a topology control signal set, and energy distribution-related actions generate power regulation signals, forming a power regulation signal set. The topology trigger sequence corresponding to the topology control signal in the topology control signal set forms a candidate set of topology trigger sequences.

[0013] Preferably, the topology control signal set and power regulation signal set are synchronized and interlocked according to the phase-locked reference phase, sampling period and dead time to generate a timing-coordinated candidate instruction set, as follows: Phase-locked loop (PLL) reference phase synchronization means aligning the switching time of the topology trigger sequence corresponding to the topology control signal and the effective time of the power regulation signal with the fundamental phase of the power grid to avoid reactive power impact caused by phase abrupt changes. The sampling period synchronization is as follows: the update frequency of the topology control signal set and the power regulation signal set is matched with the measurement sampling period of the energy storage EMS system to ensure that the command regulation rhythm is consistent with the state monitoring rhythm. The dead-time interlock check is as follows: In the topology trigger sequence corresponding to the topology control signal set, verify whether the opening and closing interval of the switching device meets the preset minimum dead time to avoid the risk of circulating current caused by the topology switching not being completed when the power command takes effect; Topology trigger sequences and their corresponding topology control signals that do not meet the preset minimum dead time are removed, and power adjustment signals corresponding to topology control signals that do not meet the preset minimum dead time are removed, forming a set of candidate instructions after timing coordination. As an explanation, the power regulation signal corresponding to the topology control signal refers to the fact that both the topology control signal and the power regulation signal are derived from the non-circulating action in the non-circulating action set. Therefore, the power regulation signal corresponding to the topology control signal refers to the fact that both belong to the same non-circulating action.

[0014] Preferably, the topology control signal ranked first and its corresponding power adjustment signal are selected to generate a safe and executable action, as follows; For the topology control signal ranked first and its corresponding power adjustment signal, the topology trigger sequence in the candidate set of the topology trigger sequence corresponding to the topology control signal is selected as the topology trigger sequence, and the power adjustment signal corresponding to the topology control signal is converted into a power support command. A safe and executable action is generated from the selected topology trigger sequence and power support instructions; As an explanation: the power support command and topology trigger sequence in the safe executable action are executed as fault traversal.

[0015] The proposed reinforcement learning-based adaptive control method for fault ride-through of energy storage EMS has the following beneficial technical effects: 1. This application generates a fault proxy state vector by constructing a topology symbol dictionary, a topology constraint graph, and extracting features of the power grid and local measurement signals. Based on the fault proxy state vector, it uses barrier function verification, feasibility constraint screening, timing coordination, and multi-objective decision-making to screen out safe and executable actions that maximize grid connection support margin and optimize power regulation while ensuring that the actions meet equipment safety requirements. This enables effective support to the power grid during fault ride-through while reducing the occurrence of device overload and circulating current risks, achieving a dynamic balance between fault support effectiveness and equipment safety. This effectively alleviates the current contradiction between fault support effectiveness and equipment safety, and improves the reliability of the EMS system through an effective control action screening mechanism.

[0016] 2. By using a feasibility constraint library and barrier functions to perform dual verification on candidate actions, actions that do not meet safety and hardware limitations are eliminated, generating a set of barrier-free actions. This avoids the risk of equipment damage or escalation of failures due to action violations from the source, ensuring the hardware safety of the energy storage EMS system during fault ride-through. Actions are filtered by a non-circulating current criterion, and the probability of circulating current risk is generated by combining the potential difference between adjacent branches and the current direction deviation. This selects a set of non-circulating current actions, which greatly reduces the probability of circulating current problems during topology switching or power distribution, and reduces device thermal stress and energy loss.

[0017] 3. By filtering actions through the non-circulating current criterion and combining the potential difference between adjacent branches and the deviation of current direction to generate the probability of circulating current risk, a set of non-circulating current actions is selected, which greatly reduces the probability of circulating current problems during topology switching or power distribution, and reduces device thermal stress and energy loss. Attached Figure Description

[0018] Figure 1 This is a flowchart of an adaptive control method for fault ride-through of energy storage EMS based on reinforcement learning, according to the present invention. Detailed Implementation

[0019] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar symbols denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0020] like Figure 1 The illustrated method for adaptive fault ride-through control of energy storage EMS based on reinforcement learning includes the following steps: S1. Based on the energy storage EMS system, construct a topology symbol dictionary and a topology constraint graph; acquire grid measurement signals and local measurement signals of the energy storage EMS system during the grid connection fault period and extract features; combine the topology symbol dictionary and the topology constraint graph to construct a topology embedding vector to generate a fault proxy state vector. As an explanation, an energy storage EMS system, also known as an energy storage management system, is an existing control system used to intelligently monitor, schedule, and optimize the energy flow, operating status, and safety performance of an energy storage system. S2. Obtain current continuity constraints, energy conservation constraints, and no-circulation current criteria, construct a safety invariant set, and generate a fault ride-through evaluation value by combining the fault proxy state vector. S3. Based on the energy storage EMS system, construct a topology reconfiguration action set and a multi-source energy allocation action set according to the topology symbol dictionary and topology constraint diagram, and generate a candidate action set by combining the fault agent state vector. S4. Analyze the candidate action set to obtain safe and executable actions.

[0021] This application generates a fault proxy state vector by constructing a topology symbol dictionary, a topology constraint graph, and extracting features from the power grid and local measurement signals. Based on the fault proxy state vector, it performs barrier function verification, feasibility constraint screening, timing coordination, and multi-objective decision-making. While ensuring that the actions meet the equipment safety requirements, the multi-objective decision-making algorithm selects safe and executable actions that maximize grid connection support margin and optimize power regulation. This enables effective support to the power grid during fault ride-through while reducing the occurrence of device overload and circulating current risks, achieving a dynamic balance between fault support effectiveness and equipment safety, and effectively alleviating the current contradiction between fault support effectiveness and equipment safety.

[0022] In an optional embodiment, in S1, based on the energy storage EMS system, a topology symbol dictionary and a topology constraint diagram are constructed as follows: Obtain the hardware switchable power path and the preset switching order of the hardware switchable power path in the energy storage EMS system, and generate a topology symbol dictionary by symbolically encoding the hardware switchable power path and the preset switching order of the hardware switchable power path. The path connectivity and mutual exclusion relationships between hardware switchable power paths are generated based on the topology symbol dictionary, and a topology constraint graph is generated based on the path connectivity and mutual exclusion relationships between hardware switchable power paths.

[0023] In an optional embodiment, in S1, the grid measurement signals and local measurement signals of the energy storage EMS system during the grid connection fault period are acquired and their features are extracted. A topology embedding vector is then constructed by combining the topology symbol dictionary and the topology constraint graph to generate a fault proxy state vector, as follows: Based on the grid measurement signals and local measurement signals of the energy storage EMS system during the grid connection fault period, voltage deviation characteristics, deviation trend characteristics, phase-locked reference phase characteristics, energy storage state of charge characteristics, thermal stress characteristics, and DC ripple characteristics are extracted. Based on the grid measurement signals and local measurement signals of the energy storage EMS system during the grid connection fault period, the instantaneous power flow between each hardware switchable power path and the grid, and the equivalent impedance between the hardware switchable power path and the grid are obtained. The instantaneous power flow and equivalent impedance are graph-embedded using a topology symbol dictionary and a topology constraint graph to obtain a topology embedding vector; The fault proxy state vector is generated by concatenating voltage deviation characteristics, deviation trend characteristics, phase-locked reference phase characteristics, energy storage state of charge characteristics, thermal stress characteristics, DC ripple characteristics, and topology embedding vector. In an optional embodiment, in S2, current continuity constraints, energy conservation constraints, and no-circulation current criteria are obtained to construct a safety invariant set. This set is then combined with the fault proxy state vector to generate a fault ride-through evaluation value, as follows: By obtaining the current continuity constraint, energy conservation constraint, and no-circulation current criterion, a safe and invariant set jointly defined by the current continuity constraint, energy conservation constraint, and no-circulation current criterion is obtained; Based on the fault agent state vector and combined with the security invariant set, a barrier function term is generated according to the preset barrier function. By statistically analyzing the grid measurement signals and local measurement signals of the energy storage EMS system during the grid connection fault period, we can obtain the grid connection support margin benefits, device lifetime and thermal cost penalties, circulating current suppression penalties, and power quality constraint benefits. The reward function shaping algorithm is used to combine the barrier function term with the grid-connected support margin benefit, device lifetime and thermal cost penalty, circulating current suppression penalty, and power quality constraint benefit to generate a fault ride-through evaluation value. In an optional embodiment, in S3, based on the energy storage EMS system, a topology reconfiguration action set and a multi-source energy allocation action set are constructed according to the topology symbol dictionary and topology constraint diagram. Combined with the fault agent state vector, a candidate action set is generated as follows: Based on the topology symbol dictionary and topology constraint graph, obtain directed switching sequences that satisfy the legal switching order and no reentrancy, form a set of directed switching sequences, and use the set of directed switching sequences as the topology reconstruction action set; Obtain the energy storage path of the energy storage EMS system, obtain the allocation coefficients that satisfy the constraints of energy conservation and power balance based on the energy storage path, form a set of allocation coefficients, and use the set of allocation coefficients as a set of multi-source energy allocation actions; The joint action space is formed by the Cartesian product of the topological reconstruction action set and the multi-source energy allocation action set; The purpose of the Cartesian product is to cover all possible combinations of actions; Based on the fault agent state vector, a policy function is used to map the fault agent state vector to the joint action space. The mapping result is then subjected to a feasibility screening using preset rules to generate a set of candidate actions. Among them, the mapping results are subjected to a feasibility screening using preset rules, which include preset security rules, hardware limitations, action compatibility requirements, and preset thresholds for fault traversal evaluation values.

[0024] In an optional embodiment, in S4, the set of candidate actions is analyzed to obtain safe and executable actions, as follows: Obtain the library of feasible constraints, which includes switch interlocking, prohibition of reentry, minimum converter interval, phase alignment, power balance, and available energy constraints for energy storage. Each candidate action in the candidate action set is validated using a preset barrier function and a library of implementable constraints. Candidate actions that do not meet the requirements of the preset barrier function or the library of implementable constraints are deleted. The remaining candidate actions are used to generate a set of barrier-enabled actions. As an explanation, the set of barrier actionable actions is a set of candidate actions that satisfy both the barrier function requirements and the requirements of the implementability constraint library. The projection operator technique is used to project the candidate action set into the barrier action set to generate the first projected action. Based on the non-circulating flow criterion, a non-circulating flow determination criterion is generated by combining the action after the first projection. The actions after the first projection are filtered according to the non-circulating flow determination criterion to generate a set of non-circulating flow actions. The actions after the first projection are filtered according to the non-circulating flow determination criterion to generate a set of non-circulating flow actions, as follows: Based on the topological constraint graph, extract all possible branch connection relationships that can form closed loops from the candidate action set to generate a loop description set; Based on the loop description set, obtain the potential difference between adjacent branches caused by the action after the first projection; After obtaining the first projection, the current direction and preset power flow direction of each branch in the closed loop are obtained, and the deviation between the current direction and the preset flow direction is obtained by vector operation. The probability of circulating current risk for the first projected action is generated based on the potential difference between adjacent branches and the deviation of the current direction from the preset flow direction. The probability of circulating current risk is generated based on the potential difference between adjacent branches and the deviation of the current direction from the preset flow direction, which can be achieved through a weighted fusion algorithm; The first post-projection action is verified using a preset barrier function. The first post-projection action that does not meet the preset barrier function requirements is deleted, and the first post-projection action that meets the preset barrier function requirements is used as the second post-projection action. The second projected action with a circulation risk probability less than a preset threshold is defined as a non-circulating action, and a set of non-circulating actions is generated. The non-circulating current actions in the non-circulating current action set are converted into command signals, generating a topology control signal set, a power regulation signal set, and a topology trigger sequence candidate set. The non-circulating current actions in the non-circulating current action set are converted into command signals, generating a topology control signal set, a power regulation signal set, and a topology trigger sequence candidate set, as follows: The acyclic actions in the acyclic action set include topology remodeling-related actions and energy allocation-related actions; When converting non-circulating current actions in the non-circulating current action set into command signals, topology reconfiguration-related actions generate topology control signals, forming a topology control signal set, and energy distribution-related actions generate power regulation signals, forming a power regulation signal set. The topology trigger sequence corresponding to the topology control signal in the topology control signal set forms a candidate set of topology trigger sequences. The topology control signal set and power regulation signal set are synchronized and interlocked according to the phase-locked reference phase, sampling period and dead time to generate a time-coordinated candidate instruction set. The topology control signal set and power regulation signal set are synchronized and interlocked according to the phase-locked reference phase, sampling period, and dead time, generating a time-coordinated candidate instruction set as follows: Phase-locked loop (PLL) reference phase synchronization means aligning the switching time of the topology trigger sequence corresponding to the topology control signal and the effective time of the power regulation signal with the fundamental phase of the power grid to avoid reactive power impact caused by phase abrupt changes. The sampling period synchronization is as follows: the update frequency of the topology control signal set and the power regulation signal set is matched with the measurement sampling period of the energy storage EMS system to ensure that the command regulation rhythm is consistent with the state monitoring rhythm. The dead-time interlock check is as follows: In the topology trigger sequence corresponding to the topology control signal set, verify whether the opening and closing interval of the switching device meets the preset minimum dead time to avoid the risk of circulating current caused by the topology switching not being completed when the power command takes effect; Topology trigger sequences and their corresponding topology control signals that do not meet the preset minimum dead time are removed, and power adjustment signals corresponding to topology control signals that do not meet the preset minimum dead time are removed, forming a set of candidate instructions after timing coordination. As an explanation, the power regulation signal corresponding to the topology control signal refers to the fact that both the topology control signal and the power regulation signal are derived from the non-circulating action in the non-circulating action set. Therefore, the power regulation signal corresponding to the topology control signal refers to the fact that both belong to the same non-circulating action. Based on the time-coordinated candidate instruction set, a multi-objective decision algorithm is used to comprehensively score the topology control signal and its corresponding power adjustment signal in the time-coordinated candidate instruction set. The topology control signal and its corresponding power adjustment signal are sorted from high to low according to the comprehensive score. The topology control signal and its corresponding power adjustment signal ranked first are selected to generate a safe and executable action. When using a multi-objective decision-making algorithm to comprehensively score the topology control signals and their corresponding power regulation signals in the candidate instruction set after timing coordination, the objectives of the multi-objective decision-making algorithm can be set as grid-connected support margin and optimized power regulation. The topology control signal with the highest priority and its corresponding power regulation signal are selected to generate a safe and executable action, as follows; For the topology control signal ranked first and its corresponding power adjustment signal, the topology trigger sequence in the candidate set of the topology trigger sequence corresponding to the topology control signal is selected as the topology trigger sequence, and the power adjustment signal corresponding to the topology control signal is converted into a power support command. A safe and executable action is generated from the selected topology trigger sequence and power support instructions; As an explanation: the power support command and topology trigger sequence in the safe executable action are executed as fault traversal.

[0025] In S4, candidate actions are double-verified through an implementability constraint library and barrier functions to eliminate actions that do not meet safety and hardware limitations, generating a set of barrier-allowed actions. This avoids the risk of equipment damage or fault expansion caused by action violations from the source, ensuring the hardware safety of the energy storage EMS system during fault ride. Actions are filtered through a non-circulating current criterion, and the probability of circulating current risk is generated by combining the potential difference between adjacent branches and the current direction deviation. This selects a set of non-circulating current actions, which greatly reduces the probability of circulating current problems during topology switching or power distribution, and reduces device thermal stress and energy loss.

[0026] By filtering actions using a non-circulating current criterion and generating a circulating current risk probability by combining the potential difference between adjacent branches and the current direction deviation, a set of actions without circulating current is selected. This significantly reduces the probability of circulating current problems during topology switching or power allocation, and reduces device thermal stress and energy loss. Through timing coordination mechanisms such as phase-locked reference phase alignment, sampling period synchronization, and dead-time interlock checks, the execution rhythm of topology control signals and power regulation signals is ensured to match the grid status and equipment response characteristics. This avoids reactive power impacts caused by phase abrupt changes or secondary faults caused by command asynchrony, and improves the stability of the fault ride-through process.

[0027] Furthermore, any content not described in detail in this specification is existing technology known to those skilled in the art.

[0028] In the embodiments provided by this invention, it should be understood that the disclosed system or method can be implemented in other ways. For example, the embodiments of the invention described above are merely illustrative; for instance, the division of modules is only a logical functional division, and there may be other division methods in actual implementation.

[0029] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0030] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or in the form of hardware plus software functional modules.

[0031] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the basic characteristics of the present invention.

[0032] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A fault ride-through adaptive control method for energy storage EMS based on reinforcement learning, characterized in that, Includes the following steps: S1. Based on the energy storage EMS system, construct a topology symbol dictionary and topology constraint graph; Acquire grid measurement signals and local measurement signals of the energy storage EMS system during the grid connection fault period; Based on the grid measurement signals and local measurement signals of the energy storage EMS system during the grid connection fault period, voltage deviation characteristics, deviation trend characteristics, phase-locked reference phase characteristics, energy storage state of charge characteristics, thermal stress characteristics, and DC ripple characteristics are extracted. Based on the grid measurement signals and local measurement signals of the energy storage EMS system during the grid connection fault period, the instantaneous power flow between each hardware switchable power path and the grid, and the equivalent impedance between the hardware switchable power path and the grid are obtained. The instantaneous power flow and equivalent impedance are graph-embedded using a topology symbol dictionary and a topology constraint graph to obtain a topology embedding vector; The fault proxy state vector is generated by concatenating voltage deviation characteristics, deviation trend characteristics, phase-locked reference phase characteristics, energy storage state of charge characteristics, thermal stress characteristics, DC ripple characteristics, and topology embedding vector. S2. Obtain current continuity constraints, energy conservation constraints, and no-circulation current criteria, construct a safety invariant set, and generate a fault ride-through evaluation value by combining the fault proxy state vector. S3. Based on the energy storage EMS system, construct a topology reconfiguration action set and a multi-source energy allocation action set according to the topology symbol dictionary and topology constraint diagram, and generate a candidate action set by combining the fault agent state vector. S4. Analyze the candidate action set to obtain safe and executable actions, as follows: Obtain the library of feasible constraints, which includes switch interlocking, prohibition of reentry, minimum converter interval, phase alignment, power balance, and available energy constraints for energy storage. Each candidate action in the candidate action set is validated using a preset barrier function and a library of implementable constraints. Candidate actions that do not meet the requirements of the preset barrier function or the library of implementable constraints are deleted. The remaining candidate actions are used to generate a set of barrier-enabled actions. The projection operator technique is used to project the candidate action set into the barrier action set to generate the first projected action. Based on the non-circulating flow criterion, a non-circulating flow determination criterion is generated by combining the action after the first projection. The actions after the first projection are filtered according to the non-circulating flow determination criterion to generate a set of non-circulating flow actions. The non-circulating current actions in the non-circulating current action set are converted into command signals, generating a topology control signal set, a power regulation signal set, and a topology trigger sequence candidate set. The topology control signal set and power regulation signal set are synchronized and interlocked according to the phase-locked reference phase, sampling period and dead time to generate a time-coordinated candidate instruction set. Based on the time-coordinated candidate instruction set, a multi-objective decision algorithm is used to comprehensively score the topology control signal and its corresponding power adjustment signal in the time-coordinated candidate instruction set. The topology control signal and its corresponding power adjustment signal are sorted from high to low according to the comprehensive score, and the top-ranked topology control signal and its corresponding power adjustment signal are selected to generate a safe and executable action.

2. The reinforcement learning-based adaptive control method for fault ride-through of energy storage EMS according to claim 1, characterized in that, In S1, based on the energy storage EMS system, a topology symbol dictionary and topology constraint diagram are constructed as follows: Obtain the hardware switchable power path and the preset switching order of the hardware switchable power path in the energy storage EMS system, and generate a topology symbol dictionary by symbolically encoding the hardware switchable power path and the preset switching order of the hardware switchable power path. The path connectivity and mutual exclusion relationships between hardware switchable power paths are generated based on the topology symbol dictionary, and a topology constraint graph is generated based on the path connectivity and mutual exclusion relationships between hardware switchable power paths.

3. The reinforcement learning-based adaptive control method for fault ride-through of energy storage EMS according to claim 1, characterized in that, In S2, current continuity constraints, energy conservation constraints, and no-circulation current criteria are obtained to construct a safety invariant set. This set is then combined with the fault proxy state vector to generate a fault ride-through evaluation value, as follows: By obtaining the current continuity constraint, energy conservation constraint, and no-circulation current criterion, a safe and invariant set jointly defined by the current continuity constraint, energy conservation constraint, and no-circulation current criterion is obtained; Based on the fault agent state vector and combined with the security invariant set, a barrier function term is generated according to the preset barrier function. By statistically analyzing the grid measurement signals and local measurement signals of the energy storage EMS system during the grid connection fault period, we can obtain the grid connection support margin benefits, device lifetime and thermal cost penalties, circulating current suppression penalties, and power quality constraint benefits. The reward function shaping algorithm combines the barrier function term with the grid-connected support margin benefit, device lifetime and thermal cost penalty, circulating current suppression penalty, and power quality constraint benefit to generate a fault ride-through evaluation value.

4. The reinforcement learning-based adaptive control method for fault ride-through of energy storage EMS according to claim 2, characterized in that, In S3, based on the energy storage EMS system, a topology reconfiguration action set and a multi-source energy allocation action set are constructed according to the topology symbol dictionary and topology constraint diagram. Combined with the fault agent state vector, a candidate action set is generated as follows: Based on the topology symbol dictionary and topology constraint graph, obtain directed switching sequences that satisfy the legal switching order and no reentrancy, form a set of directed switching sequences, and use the set of directed switching sequences as the topology reconstruction action set; Obtain the energy storage path of the energy storage EMS system, and based on the energy storage path, obtain the allocation coefficients that satisfy the constraints of energy conservation and power balance, forming a set of allocation coefficients, and use the set of allocation coefficients as a set of multi-source energy allocation actions; The joint action space is formed by the Cartesian product of the topological reconstruction action set and the multi-source energy allocation action set; Based on the fault agent state vector, a policy function is used to map the fault agent state vector to the joint action space. The mapping results are then subjected to a feasibility screening using preset rules to generate a set of candidate actions. Among them, the mapping results are subjected to a feasibility screening using preset rules, which include preset security rules, hardware limitations, action compatibility requirements, and preset thresholds for fault traversal evaluation values.

5. The reinforcement learning-based adaptive control method for fault ride-through of energy storage EMS according to claim 1, characterized in that, The actions after the first projection are filtered according to the non-circulating flow determination criterion to generate a set of non-circulating flow actions, as follows: Based on the topological constraint graph, extract all possible branch connection relationships that can form closed loops from the candidate action set to generate a loop description set; Based on the loop description set, obtain the potential difference between adjacent branches caused by the action after the first projection; After obtaining the first projection, the current direction and preset power flow direction of each branch in the closed loop are obtained, and the deviation between the current direction and the preset flow direction is obtained by vector operation. The probability of circulating current risk for the first projected action is generated based on the potential difference between adjacent branches and the deviation of the current direction from the preset flow direction. The first post-projection action is verified using a preset barrier function. The first post-projection action that does not meet the preset barrier function requirements is deleted, and the first post-projection action that meets the preset barrier function requirements is used as the second post-projection action. The second projected action with a circulation risk probability less than a preset threshold is considered a non-circulating action, and a set of non-circulating actions is generated.

6. The reinforcement learning-based adaptive control method for fault ride-through of energy storage EMS according to claim 1, characterized in that, The non-circulating current actions in the non-circulating current action set are converted into command signals, generating a topology control signal set, a power regulation signal set, and a topology trigger sequence candidate set, as follows: The acyclic actions in the acyclic action set include topology remodeling-related actions and energy allocation-related actions; When converting non-circulating current actions in the non-circulating current action set into command signals, topology reconfiguration-related actions generate topology control signals, forming a topology control signal set, and energy distribution-related actions generate power regulation signals, forming a power regulation signal set. The topology trigger sequence corresponding to the topology control signal in the topology control signal set forms a candidate set of topology trigger sequences.

7. The reinforcement learning-based adaptive control method for fault ride-through of energy storage EMS according to claim 1, characterized in that, The topology control signal set and power regulation signal set are synchronized and interlocked according to the phase-locked reference phase, sampling period, and dead time, generating a time-coordinated candidate instruction set as follows: Phase-locked loop (PLL) reference phase synchronization means aligning the switching time of the topology trigger sequence corresponding to the topology control signal and the effective time of the power regulation signal with the fundamental phase of the power grid. The sampling period synchronization is achieved by matching the update frequency of the topology control signal set and the power regulation signal set with the measurement sampling period of the energy storage EMS system. The dead-time interlock check is as follows: In the topology trigger sequence corresponding to the topology control signal set, verify whether the opening and closing interval of the switching device meets the preset minimum dead time. Topology trigger sequences and their corresponding topology control signals that do not meet the preset minimum dead time are removed, and power adjustment signals corresponding to topology control signals that do not meet the preset minimum dead time are also removed, forming a set of candidate instructions after timing coordination.

8. The reinforcement learning-based adaptive control method for fault ride-through of energy storage EMS according to claim 1, characterized in that, The topology control signal with the highest priority and its corresponding power regulation signal are selected to generate a safe and executable action, as follows: For the topology control signal ranked first and its corresponding power adjustment signal, the topology trigger sequence in the candidate set of the topology trigger sequence corresponding to the topology control signal is selected as the topology trigger sequence, and the power adjustment signal corresponding to the topology control signal is converted into a power support command. A safe and executable action is generated from the selected topology trigger sequence and power support instructions.

Images