Layered intelligent reasoning method for collaborative operation of end-to-end power grid switch

Abstract

The invention discloses a hierarchical intelligent reasoning method for collaborative operation of end-to-end power grid switches. The method is used for solving the problem that in the prior art, a wheeling and maintenance complete switching operation scheme cannot be directly generated. A graph structure model is designed by utilizing Neo4j, a power flow model and a main wiring model are embedded into the graph model, and a knowledge base environment capable of reflecting a topological relation and power flow characteristics of a power grid-transformer substation-switch group is constructed. Considering the correlation between the two types of topologies, and taking the switch group as an intermediate node to establish a hierarchical optimization model; the upper layer takes the switch group as the bottommost layer node, and optimizes the operation sequence of the switch group on the basis of considering the power flow constraint, the network topology constraint, the operation safety constraint and the loop closing impact constraint; the lower layer models the main wiring of the transformer substation, and optimizes the operation sequence of the switches in the group on the premise of meeting the operation safety; the method can directly infer the model of a wheeling and switching complete scheme in an end-to-end manner, and is of great significance for improving the operation and maintenance efficiency of a power grid.

Description

A hierarchical intelligent reasoning method for end-to-end grid switch cooperative operation Technical Field

[0001] This invention belongs to the field of substation switching operation reasoning technology, and in particular relates to a hierarchical intelligent reasoning method for end-to-end grid switch collaborative operation. Background Technology

[0002] In order to ensure the continuity of power supply to users, improve operational efficiency, and reduce operational risks during power grid operation, the load of the equipment to be repaired must first be transferred to other normal equipment through a transfer operation, and then the equipment under repair must be de-energized for maintenance through a maintenance operation. Intelligent power grid operation can automatically generate operation plans for transfer and maintenance, providing technical support for intelligent power grid operation. Transfer finds the optimal power supply channel at the power grid level, and maintenance requires optimizing the sequence of switch operations within the substation. In actual operation, a large number of independent operation tasks are generated, and operation steps are generated according to the tasks. The entire process is complex and inefficient, contains a large number of redundant steps, and requires a lot of manual correction. Therefore, it is necessary to design an end-to-end power grid switch collaborative operation hierarchical intelligent reasoning method to solve the above problems. Summary of the Invention

[0003] The technical problem this invention aims to solve is to provide a hierarchical intelligent reasoning method for end-to-end grid switch collaborative operation. It utilizes Neo4j to design a graph structure model, embedding both the power flow model and the main wiring model into the graph model to construct a knowledge base environment that reflects the topological relationships and power flow characteristics of the grid-substation-switch group. A hierarchical optimization model is established using the switch group as an intermediate node. The upper layer uses the switch group as the bottom-level node, optimizing the operation sequence of the switch group based on power flow constraints, network topology constraints, operational safety constraints, and loop-closing impact constraints. The lower layer models the main wiring of the substation, optimizing the operation sequence of switches within the group while ensuring operational safety. This method can directly deduce complete transfer and switching schemes end-to-end, which is of great significance for improving grid operation and maintenance efficiency.

[0004] To achieve the above technical effects, the technical solution adopted by this invention is: a hierarchical intelligent reasoning method for end-to-end power grid switch collaborative operation, comprising the following steps: S1, constructing a knowledge graph model based on Neo4j, the model including electrical equipment, the bay to which the equipment belongs, and the substation topology, realizing multi-level association representation at the equipment level, bay level, and station level; S2, embedding a power flow calculation model into the graph model to realize power flow analysis of the power grid under different topology states; S3, updating the working status of each bay in the graph model; S4, calling the power flow calculation module to determine the outage area caused by maintenance, and searching the maintenance bay set and the transfer bay set in the graph model. S5, Construct a reinforcement learning environment for the power transfer optimization model, and configure the D3QN agent, state space, action space, and reward / penalty function; S6, Use the power transfer optimization model to infer the optimal combination of power transfer bays; S7, Construct a reinforcement learning environment for the switching optimization model, and configure the MT-D3QN agent, action space, state space, and reward / penalty function so that it can infer the optimal operation sequence of switches within multiple bays in parallel; S8, Input the maintenance bay set and the power transfer bay set output by the power transfer optimization model into the switching optimization model respectively, and infer the switch operation sequence within each bay, thereby realizing the generation of a complete end-to-end power transfer and maintenance switching scheme.

[0005] Preferably, in step S1, the knowledge graph model stores the basic information of the substation through entities, relationships, and attributes. Specifically, this includes: entities comprising equipment and bays; equipment attributes covering equipment name, equipment type, bay to which it belongs, and voltage level information; relationship attributes including "contains," "connected," and "disconnected"; wherein, the "contains" relationship indicates the hierarchical relationship between a bay and its internal equipment; the "connected" and "disconnected" relationships are used to describe the connection status between different equipment and are determined by the open or closed state of the switch; entities Represents the interval entity and the device entities contained within the interval; relationship Represents the relationships between entities. This indicates the hierarchical relationship between a spacer entity and its internal device entities; Used to describe the connection and disconnection between different entities, and determined by the on / off state of the switch connected to the entity; Attribute A set of real-time status information used to describe entities, including interval status, switch status, bus voltage deviation, transformer load rate, line load rate, and loop current.

[0006] Preferably, in step S2, the power flow calculation model includes a data storage module, a topology analysis module, and a power flow calculation module: the data storage module is used to store electrical data such as node voltage, line load rate, and transformer load rate; the topology analysis module adjusts the topology of the power flow calculation model synchronously according to the updated action interval information after an interval action occurs in the graph model; the power flow calculation module performs power flow calculations according to the Newton-Raphson method and stores the calculation results in the data storage module.

[0007] Preferably, step 3 includes six working states: Operating status: The circuit breakers and disconnectors in the bay are all in the closed state, and the grounding switch is in the open state. In hot standby mode, all disconnect switches in the bay are closed, while the circuit breakers and grounding switches are open. In cold standby mode, all circuit breakers, disconnectors, and grounding switches within the bay are in the open position. Maintenance status: The circuit breakers and disconnectors in the bay are all in the open position, and the grounding switch is in the closed position. Transition state: The state between the four adjacent state transitions mentioned above. Unfavorable condition: When the grounding switch is in the closed state, the circuit breaker or disconnector is in the closed state.

[0008] Preferably, in step S4, the search process for the maintenance interval set and the transfer interval set is as follows: Detailed information about the maintenance equipment is retrieved from the graph model, and this information is input into the power flow calculation model; Power flow analysis is performed by simulating a maintenance equipment disconnection scenario, calculating current, voltage, and power system parameters, identifying power outage areas caused by equipment shutdown, and feeding back the power outage area information to the graph model; A path search is performed in the graph model centered on the power outage area, and the switching intervals along the path are recorded as the transfer interval set; Starting from the maintenance equipment, a power supply path for non-switching equipment connected to the maintenance equipment is searched in the graph model, and the switching intervals along the path are recorded as the maintenance interval set; wherein, the set is used... Record the intervals between the supply space and the maintenance space: In the above formula, hour, Represents the set of maintenance intervals; hour, Represents the set of transfer intervals; This indicates the number of the switch interval in the path.

[0009] Preferably, in step S5, the state space, action space, and reward / penalty function of the power transfer optimization model are defined as follows: The power transfer optimization model assumes the power outage state of the transformer substation. Network topology Closed-loop inrush current Closed-loop steady-state current Line load rate Voltage offset of nodes Transformer load rate Switching interval operating status The state space information for power transfer optimization is used as the state space information; after removing the intervals that overlap with the maintenance interval set from the power transfer interval set, it becomes the action space of the power transfer optimization model; the reward function of the power transfer optimization model comprehensively considers grid safety and operational efficiency, and penalizes node voltage deviation, line and transformer load rate, loop current and repeated operations. The main objective is to ensure no lost radio stations and meet safety constraints, and the secondary objective is to reduce the number of interval operations to improve system efficiency.

[0010] Preferably, in step S6, the reasoning process of the transfer optimization model is as follows: The D3QN agent selects an interval in the action space to perform an action, and the interval switches from a non-running state to a running state. At the same time, the graph model performs topology updates and the power flow calculation model performs power flow analysis. The D3QN agent analyzes the reward based on the power flow calculation results, and through continuous iterative training, it infers the transfer interval operation sequence with the maximum reward.

[0011] Preferably, in step S7, the state space, action space, and reward / penalty function of the switching optimization model are defined as follows: The state space information of the switching optimization model includes the working state of the task interval and the opening and closing state of the switching equipment in each interval; the transfer interval set and maintenance interval set obtained based on the upper-level D3QN inference are iterated through respectively; the switching equipment included in each interval is determined by graph model search, and the action space of transfer switching and maintenance switching is constructed; the reward function is designed according to the closeness between the target state and the current equipment state to ensure that the operation is close to the target state; at the same time, repeated operations are penalized to prevent the model from frequently executing invalid or repetitive actions.

[0012] Preferably, the safety constraints and operational efficiency of the integrated balancing system are combined, and a reward / penalty function is designed based on state-space information; a voltage deviation penalty is set. Maximum node voltage deviation for When the node voltage deviation If the deviation exceeds this range, penalties will be imposed based on the degree of deviation, as follows: ;in, It is a proportional factor that determines the sensitivity of rewards to deviations from the target limit; Represents the initial constant; sets the line load rate penalty. Maximum allowable load rate of the line It is 100%, when the actual load rate of the line is... When this limit is exceeded, penalties will be imposed according to the degree of overload, as follows: Set loop current penalty When the interval switches to the operating state, the closed-loop inrush current The setting value of the instantaneous overcurrent protection must not be exceeded. Closed-loop steady-state current The overcurrent protection setting value must not be exceeded. The details are as follows: Set a penalty for transformer exceeding limits. Maximum allowable load rate of transformer The load factor is 100% when the transformer load rate is... Penalties will be imposed based on the degree of overload after exceeding the limit, as detailed below: Set penalties for repetitive actions When the interval transitions from a non-running state to a running state, if the interval switches back to a non-running state due to repeated actions, an invalid action is generated; the details are as follows: ;in, Indicates the number of times the action is repeated at intervals. Indicates the total number of intervals in the action space; when there are no lost radio zones and all safety constraints are met, a reward is set. To reduce the number of interval operations and improve system efficiency; specifically as follows: ;in, Indicates the number of intervals between operations. This represents the total number of intervals in the action space; in the state-action evaluation of the transfer optimization model, the agent selects an action from the action space, executes it, updates the state, and calculates the reward; the agent optimizes the policy by evaluating the Q-value of each state-action pair to maximize the cumulative reward; the expression for the Q-value is as follows: ;in, Indicates the current state and actions Q value, Indicates the discount factor. The system represents immediate rewards; it designs a dynamic adjustment mechanism for reward weights; the Q-value is composed of a weighted sum of multiple reward items, therefore the impact of each reward item on the Q-value can be represented by the gradient of the Q-value relative to the weight of the reward item; the loss function can be calculated using the chain rule. Reward weight The gradient is as follows: ;in, Represents the loss function Follow The rate of change of the value reflects The sensitivity of the value to loss; express The value depends on the weight of that reward item. The rate of change reflects the weight. right The influence of the value; based on the calculated gradient, update the reward weights using gradient descent: ;in, Indicates the first Round weights Indicates the updated number Round weights The learning rate is represented here. To avoid overestimation of Q-values ​​in the optimization model, the Q-value is updated using the dual network structure of the D3QN algorithm. The specific Q-value update is represented as follows: ;in, This indicates the current state of online network computation. and actions Q value, Indicates the current instant reward. Indicates the discount factor. Indicates the learning rate. This indicates that the next state is calculated through the target network. Executable actions The maximum target Q value; Indicates that the online network will choose the next state. The action with the largest Q value.

[0013] Preferably, step S8 specifically includes the following steps: receiving the list of switching intervals and maintenance intervals output by the upper-layer D3QN, splitting them into two parallel subtasks: switching and maintenance, and assigning a unique ID to each task; performing feature encoding on each task, fusing the number of interval switches, initial state, and target state information to generate a low-dimensional embedding vector, enabling the model to accurately distinguish the operational logic of different tasks; the MT-D3QN agent selects interval actions from the action space, updates the state of the graph model, calculates the Q-value of each task, and updates the reward weight of the task; extracting the common logic of the five-prevention rules and the general knowledge of the timing constraints of switch operations by sharing the parameters of the fully connected FC layer, realizing cross-task knowledge transfer; when an unfavorable state occurs in a task interval, the agent immediately stops the current round and starts a new round for retraining; through continuous iteration, ensuring that the reward gradually converges to the optimal value; and outputting the switching sequence of switching and maintenance with the optimal reward.

[0014] The beneficial effects of this invention are as follows: This invention considers the structural similarity between the power grid topology and main wiring diagrams and the network map. Using Neo4j, a graph structure model is designed, embedding both the power flow model and the main wiring model into the graph model to construct a knowledge base environment that reflects the topological relationships and power flow characteristics of the power grid, substations, and switchgear groups. Considering the association between the two types of topologies, a hierarchical optimization model is established using switchgear groups as intermediate nodes. The upper layer uses switchgear groups as the bottom-level nodes, optimizing the operation sequence of switchgear groups based on power flow constraints, network topology constraints, operational safety constraints, and loop-closing impact constraints. The lower layer models the main wiring diagrams of substations, optimizing the operation sequence of switches within the group while ensuring operational safety. Researching a model capable of directly inferring complete transfer and switching schemes end-to-end is of great significance for improving the efficiency of power grid operation and maintenance. Attached Figure Description

[0015] Figure 1 is a schematic diagram of the method flow according to an embodiment of the present invention. Detailed Implementation

[0016] Example 1: As shown in Figure 1, a hierarchical intelligent reasoning method for end-to-end power grid switch collaborative operation includes the following steps: S1, constructing a knowledge graph model based on Neo4j, the model includes electrical equipment, the bay to which the equipment belongs, and the substation topology, realizing multi-level association representation at the equipment level, bay level, and station level; S2, embedding a power flow calculation model into the graph model to realize power flow analysis of the power grid under different topology states; S3, updating the working status of each bay in the graph model; S4, calling the power flow calculation module to determine the outage area caused by maintenance, and searching for the maintenance bay set and the transfer bay set in the graph model; S5, constructing... The reinforcement learning environment for the power transfer optimization model is configured with a D3QN agent, state space, action space, and reward / penalty function; S6, the optimal power transfer bay combination is inferred using the power transfer optimization model; S7, the reinforcement learning environment for the switching optimization model is constructed, and the MT-D3QN agent, action space, state space, and reward / penalty function are configured to enable it to infer the optimal operation sequence of switches within multiple bays in parallel; S8, the maintenance bay set and the power transfer bay set output by the power transfer optimization model are respectively input into the switching optimization model to infer the switch operation sequence within each bay, thereby realizing the generation of a complete end-to-end power transfer and maintenance switching scheme.

[0017] Preferably, in step S1, the knowledge graph model stores the basic information of the substation through entities, relationships, and attributes. Specifically, this includes: entities comprising equipment and bays; equipment attributes covering equipment name, equipment type, bay to which it belongs, and voltage level information; relationship attributes including "contains," "connected," and "disconnected"; wherein, the "contains" relationship indicates the hierarchical relationship between a bay and its internal equipment; the "connected" and "disconnected" relationships are used to describe the connection status between different equipment and are determined by the open or closed state of the switch; entities Represents the interval entity and the device entities contained within the interval; relationship Represents the relationships between entities. This indicates the hierarchical relationship between a spacer entity and its internal device entities; Used to describe the connection and disconnection between different entities, and determined by the on / off state of the switch connected to the entity; Attribute A set of real-time status information used to describe entities, including interval status, switch status, bus voltage deviation, transformer load rate, line load rate, and loop current.

[0018] Preferably, in step S2, the power flow calculation model includes a data storage module, a topology analysis module, and a power flow calculation module: the data storage module is used to store electrical data such as node voltage, line load rate, and transformer load rate; the topology analysis module adjusts the topology of the power flow calculation model synchronously according to the updated action interval information after an interval action occurs in the graph model; the power flow calculation module performs power flow calculations according to the Newton-Raphson method and stores the calculation results in the data storage module.

[0019] Preferably, step 3 includes six working states: Operating status: The circuit breakers and disconnectors in the bay are all in the closed state, and the grounding switch is in the open state. In hot standby mode, all disconnect switches in the bay are closed, while the circuit breakers and grounding switches are open. In cold standby mode, all circuit breakers, disconnectors, and grounding switches within the bay are in the open position. Maintenance status: The circuit breakers and disconnectors in the bay are all in the open position, and the grounding switch is in the closed position. Transition state: The state between the four adjacent state transitions mentioned above. Unfavorable condition: When the grounding switch is in the closed state, the circuit breaker or disconnector is in the closed state.

[0020] Preferably, in step S4, the search process for the maintenance interval set and the transfer interval set is as follows: Detailed information about the maintenance equipment is retrieved from the graph model, and this information is input into the power flow calculation model; Power flow analysis is performed by simulating a maintenance equipment disconnection scenario, calculating current, voltage, and power system parameters, identifying power outage areas caused by equipment shutdown, and feeding back the power outage area information to the graph model; A path search is performed in the graph model centered on the power outage area, and the switching intervals along the path are recorded as the transfer interval set; Starting from the maintenance equipment, a power supply path for non-switching equipment connected to the maintenance equipment is searched in the graph model, and the switching intervals along the path are recorded as the maintenance interval set; wherein, the set is used... Record the intervals between the supply space and the maintenance space: In the above formula, hour, Represents the set of maintenance intervals; hour, Represents the set of transfer intervals; This indicates the number of the switch interval in the path.

[0021] Preferably, in step S5, the state space, action space, and reward / penalty function of the power transfer optimization model are defined as follows: The power transfer optimization model assumes the power outage state of the transformer substation. Network topology Closed-loop inrush current Closed-loop steady-state current Line load rate Voltage offset of nodes Transformer load rate Switching interval operating status The state space information for power transfer optimization is used as the state space information; after removing the intervals that overlap with the maintenance interval set from the power transfer interval set, it becomes the action space of the power transfer optimization model; the reward function of the power transfer optimization model comprehensively considers grid safety and operational efficiency, and penalizes node voltage deviation, line and transformer load rate, loop current and repeated operations. The main objective is to ensure no lost radio stations and meet safety constraints, and the secondary objective is to reduce the number of interval operations to improve system efficiency.

[0022] Preferably, in step S6, the reasoning process of the transfer optimization model is as follows: The D3QN agent selects an interval in the action space to perform an action, and the interval switches from a non-running state to a running state. At the same time, the graph model performs topology updates and the power flow calculation model performs power flow analysis. The D3QN agent analyzes the reward based on the power flow calculation results, and through continuous iterative training, it infers the transfer interval operation sequence with the maximum reward.

[0023] Preferably, in step S7, the state space, action space, and reward / penalty function of the switching optimization model are defined as follows: The state space information of the switching optimization model includes the working state of the task interval and the opening and closing state of the switching equipment in each interval; the transfer interval set and maintenance interval set obtained based on the upper-level D3QN inference are iterated through respectively; the switching equipment included in each interval is determined by graph model search, and the action space of transfer switching and maintenance switching is constructed; the reward function is designed according to the closeness between the target state and the current equipment state to ensure that the operation is close to the target state; at the same time, repeated operations are penalized to prevent the model from frequently executing invalid or repetitive actions.

[0024] Preferably, the safety constraints and operational efficiency of the integrated balancing system are combined, and a reward / penalty function is designed based on state-space information; a voltage deviation penalty is set. Maximum node voltage deviation for When the node voltage deviation If the deviation exceeds this range, penalties will be imposed based on the degree of deviation, as follows: ;in, It is a proportional factor that determines the sensitivity of rewards to deviations from the target limit; Represents the initial constant; sets the line load rate penalty. Maximum allowable load rate of the line It is 100%, when the actual load rate of the line is... When this limit is exceeded, penalties will be imposed according to the degree of overload, as follows: Set loop current penalty When the interval switches to the operating state, the closed-loop inrush current The setting value of the instantaneous overcurrent protection must not be exceeded. Closed-loop steady-state current The overcurrent protection setting value must not be exceeded. The details are as follows: Set a penalty for transformer exceeding limits. Maximum allowable load rate of transformer The load factor is 100% when the transformer load rate is... Penalties will be imposed based on the degree of overload after exceeding the limit, as detailed below: Set penalties for repetitive actions When the interval transitions from a non-running state to a running state, if the interval switches back to a non-running state due to repeated actions, an invalid action is generated; the details are as follows: ;in, Indicates the number of times the action is repeated at intervals. Indicates the total number of intervals in the action space; when there are no lost radio zones and all safety constraints are met, a reward is set. To reduce the number of interval operations and improve system efficiency; specifically as follows: ;in, Indicates the number of intervals between operations. This represents the total number of intervals in the action space; in the state-action evaluation of the transfer optimization model, the agent selects an action from the action space, executes it, updates the state, and calculates the reward; the agent optimizes the policy by evaluating the Q-value of each state-action pair to maximize the cumulative reward; the expression for the Q-value is as follows: ;in, Indicates the current state and actions Q value, Indicates the discount factor. The system represents immediate rewards; it designs a dynamic adjustment mechanism for reward weights; the Q-value is composed of a weighted sum of multiple reward items, therefore the impact of each reward item on the Q-value can be represented by the gradient of the Q-value relative to the weight of the reward item; the loss function can be calculated using the chain rule. Reward weight The gradient is as follows: ;in, Represents the loss function Follow The rate of change of the value reflects The sensitivity of the value to loss; express The value depends on the weight of that reward item. The rate of change reflects the weight. right The influence of the value; based on the calculated gradient, update the reward weights using gradient descent: ;in, Indicates the first Round weights Indicates the updated number Round weights The learning rate is represented here. To avoid overestimation of Q-values ​​in the optimization model, the Q-value is updated using the dual network structure of the D3QN algorithm. The specific Q-value update is represented as follows: ;in, This indicates the current state of online network computation. and actions Q value, Indicates the current instant reward. Indicates the discount factor. Indicates the learning rate. This indicates that the next state is calculated through the target network. Executable actions The maximum target Q value; Indicates that the online network will choose the next state. The action with the largest Q value.

[0025] Preferably, step S8 specifically includes the following steps: receiving the list of switching intervals and maintenance intervals output by the upper-layer D3QN, splitting them into two parallel subtasks: switching and maintenance, and assigning a unique ID to each task; performing feature encoding on each task, fusing the number of interval switches, initial state, and target state information to generate a low-dimensional embedding vector, enabling the model to accurately distinguish the operational logic of different tasks; the MT-D3QN agent selects interval actions from the action space, updates the state of the graph model, calculates the Q-value of each task, and updates the reward weight of the task; extracting the common logic of the five-prevention rules and the general knowledge of the timing constraints of switch operations by sharing the parameters of the fully connected FC layer, realizing cross-task knowledge transfer; when an unfavorable state occurs in a task interval, the agent immediately stops the current round and starts a new round for retraining; through continuous iteration, ensuring that the reward gradually converges to the optimal value; and outputting the switching sequence of switching and maintenance with the optimal reward.

[0026] Example 2: Further, in step S1, the entity This represents the bay entity and the equipment entities contained within it. Bays are divided into switch bays and component bays. Switch bays are combinations of circuit breakers, disconnectors, and grounding switches; component bays include transformers, busbars, and lines. When searching by bay nodes, the number of devices searched for a path can be significantly reduced, improving search efficiency.

[0027] relation Represents the relationships between entities. This indicates the hierarchical relationship between a spacer entity and its internal device entities; Used to describe the connection and disconnection between different entities, and determined by the on / off state of the switch connected to the entity.

[0028] property A set of real-time status information used to describe entities, including interval status, switch status, bus voltage deviation, transformer load rate, line load rate, and loop current.

[0029] Furthermore, in step S6, the optimal transfer path is inferred using the transfer optimization model. The specific reasoning process is as follows: The D3QN agent starts from the action space... Choose an appropriate action interval.

[0030] Interval search is performed in the graph model, and power flow analysis is conducted on the selected interval using the power flow calculation model to update the power flow information in the graph model.

[0031] Based on the current trend, the D3QN agent calculates the Q-value to evaluate the merits of the path and adjusts the reward weights accordingly.

[0032] When the current constraint is violated, the AI ​​will immediately stop the current round and start a new round for retraining, continuously iterating to ensure that the reward converges to the optimal value.

[0033] Input the optimal power transfer interval and maintenance interval set into the lower-level switching optimization model.

[0034] Further, in step S7, a reinforcement learning environment for the switching optimization model is constructed, configuring the MT-D3QN agent, action space, state space, and reward / penalty function to enable parallel reasoning of the optimal operation sequence of switches within multiple bays; specifically as follows: In the switching optimization stage, the model takes the power transfer and maintenance operation tasks generated by the upper-level power transfer optimization as input, and searches for bays after task transfer in the graph model. Circuit breakers, disconnectors, and grounding switches within the bay are recorded, and these switching devices are used as the switching action space. The switching action space is recorded using sets. Record the switching action space: In the above formula, This indicates the number of the switchgear.

[0035] The state space definition for switching optimization is as follows: ;in, The task interval represents the working status, including six working states: running state, hot standby state, cold standby state, maintenance state, transition state, and poor state; x represents the status of the switching equipment, including the type of switching equipment and its open / closed state; n represents the switch number currently in operation.

[0036] Setting up reward and penalty functions guides the model to reason about policies in the correct state transition direction, preventing misoperations from leading to inferior states.

[0037] To prevent repeated switching actions from causing invalid operation, a penalty is set for repeated switching actions. Specifically as follows: ;in, Indicates the number of times an action is repeated. This indicates the total number of active switches.

[0038] To optimize switching operations, a reward function is designed based on the proximity between the target state and the current equipment state. State priorities are set in the order of operating state, hot standby state, cold standby state, maintenance state, and suboptimal state, with priorities increasing from 1 to 5. No priority is assigned to transition states. Specific reward function. as follows: ;in, Indicates the priority of the target state; Indicates the priority of the current switch state.

[0039] The MT-D3QN algorithm employs a task feature encoding mechanism, transforming task information into low-dimensional embedding vectors, which are then concatenated with state features as input, thereby enhancing the model's ability to distinguish tasks. To construct the task embedding vectors, a maximum number of margins is given. and embedding dimension Construct an embedding matrix This represents the mapping between task IDs and embedding vectors, specifically in the form of: ;in, This is the embedding vector corresponding to task ID b. It is the identifier of the task, representing the index of the embedding vector corresponding to each task.

[0040] MT-D3QN introduces an adaptive weighting mechanism based on task-cumulative rewards, dynamically allocating training resources according to the real-time performance of each task to ensure efficient convergence of all tasks and that critical operations are not ignored. The formula for calculating task weights is: ;in, This represents the weight of task ID b; This represents the cumulative reward for a task, used to measure task performance. The higher the cumulative reward, the greater the importance of the task. This represents the temperature parameter, which controls the sensitivity of the reward calculation when determining its weight.

[0041] To optimize model training, MT-D3QN introduces a multi-task total loss function. This function combines the losses from all tasks with a regularization term; the specific loss function is as follows: ;in, This represents the D3QN loss function for task ID b; Indicates the total number of tasks; This represents the regularization coefficient, used to control the complexity of the model. This represents the L2 norm of the model parameters, reducing excessively large parameter values ​​and thus preventing model overfitting.

Claims

1. A hierarchical intelligent reasoning method for end-to-end grid switch cooperative operation, characterized in that, Includes the following steps: S1. Construct a knowledge graph model based on Neo4j. The model includes electrical equipment, the bays to which the equipment belongs, and the substation topology, achieving multi-level association representation at the equipment, bay, and station levels. S2. Embed a power flow calculation model into the graph model to perform power flow analysis on the power grid under different topology states. S3. Update the operating status of each bay in the graph model. S4. Call the power flow calculation module to determine the outage areas caused by maintenance and search the maintenance bay set and the transfer bay set in the graph model. S5. Construct a reinforcement learning environment for the transfer optimization model and configure D3QN. S6, using the power transfer optimization model to infer the optimal power transfer bay combination; S7, constructing a reinforcement learning environment for the switching optimization model, configuring the MT-D3QN agent, action space, state space, and reward / penalty function to enable it to infer the optimal operation sequence of switches within multiple bays in parallel; S8, inputting the maintenance bay set and the power transfer bay set output by the power transfer optimization model into the switching optimization model respectively, and inferring the switch operation sequence within each bay, thereby realizing the generation of a complete end-to-end power transfer and maintenance switching scheme.

2. The hierarchical intelligent reasoning method for end-to-end grid switch collaborative operation according to claim 1, characterized in that, In step S1, the knowledge graph model stores the basic information of the substation through entities, relationships, and attributes. Specifically, entities include equipment and bays; equipment attributes cover equipment name, equipment type, bay to which it belongs, and voltage level information; relationship attributes include "contains," "connected," and "disconnected"; where "contains" indicates the membership relationship between a bay and its internal equipment; "connected" and "disconnected" relationships describe the connection status between different devices and are determined by the open or closed state of the switch; entities Represents the interval entity and the device entities contained within the interval; relationship Represents the relationships between entities. This indicates the hierarchical relationship between a spacer entity and its internal device entities; Used to describe the connection and disconnection between different entities, and determined by the on / off state of the switch connected to the entity; Attribute A set of real-time status information used to describe entities, including interval status, switch status, bus voltage deviation, transformer load rate, line load rate, and loop current.

3. The hierarchical intelligent reasoning method for end-to-end grid switch cooperative operation according to claim 1, characterized in that, In step S2, the power flow calculation model includes a data storage module, a topology analysis module, and a power flow calculation module: Data storage module: used to store electrical data such as node voltage, line load rate, and transformer load rate; Topology analysis module: after an interval action occurs in the graph model, it synchronously adjusts the topology of the power flow calculation model according to the updated action interval information; Power flow calculation module: performs power flow calculations according to the Newton-Raphson method and stores the calculation results in the data storage module.

4. The hierarchical intelligent reasoning method for end-to-end grid switch cooperative operation according to claim 1, characterized in that, Step 3 includes six working states: Operating status: The circuit breakers and disconnectors in the bay are all in the closed state, and the grounding switch is in the open state. In hot standby mode, all disconnect switches in the bay are closed, while the circuit breakers and grounding switches are open. In cold standby mode, all circuit breakers, disconnectors, and grounding switches within the bay are in the open position. Maintenance status: The circuit breakers and disconnectors in the bay are all in the open position, and the grounding switch is in the closed position. Transition state: The state between the four adjacent state transitions mentioned above. Unfavorable condition: When the grounding switch is in the closed state, the circuit breaker or disconnector is in the closed state.

5. The hierarchical intelligent reasoning method for end-to-end grid switch cooperative operation according to claim 1, characterized in that, In step S4, the search process for the maintenance interval set and the transfer interval set is as follows: Detailed information about the maintenance equipment is retrieved from the graph model and input into the power flow calculation model; power flow analysis is performed by simulating the maintenance equipment disconnection scenario, calculating current, voltage, and power system parameters, identifying the power outage areas caused by equipment shutdown, and feeding back the power outage area information to the graph model; a path search is performed in the graph model centered on the power outage area to other power outage areas, and the switching intervals in the path are recorded as the transfer interval set; starting from the maintenance equipment, the power supply path for non-switching equipment connected to the maintenance equipment is searched in the graph model, and the switching intervals in the path are recorded as the maintenance interval set; where the set is used... Record the intervals between the supply space and the maintenance space: In the above formula, hour, Represents the set of maintenance intervals; hour, Represents the set of transfer intervals; This indicates the number of the switch interval in the path.

6. The hierarchical intelligent reasoning method for end-to-end grid switch cooperative operation according to claim 1, characterized in that, In step S5, the state space, action space, and reward / penalty function of the power transfer optimization model are defined as follows: The power transfer optimization model assumes the transformer area is in a power outage state. Network topology Close-loop inrush current Closed-loop steady-state current Line load rate Voltage offset of nodes Transformer load rate Switching interval operating status The state space information for power transfer optimization is used as the state space information; after removing the intervals that overlap with the maintenance interval set from the power transfer interval set, it becomes the action space of the power transfer optimization model; the reward function of the power transfer optimization model comprehensively considers grid safety and operational efficiency, and penalizes node voltage deviation, line and transformer load rate, loop current and repeated operations. The main objective is to ensure no lost radio stations and meet safety constraints, and the secondary objective is to reduce the number of interval operations to improve system efficiency.

7. The hierarchical intelligent reasoning method for end-to-end grid switch cooperative operation according to claim 1, characterized in that, In step S6, the reasoning process of the transfer optimization model is as follows: The D3QN agent selects an interval in the action space to perform actions. The interval switches from a non-running state to a running state. At the same time, the graph model performs topology updates and the power flow calculation model performs power flow analysis. The D3QN agent analyzes the reward based on the power flow calculation results. Through continuous iterative training, it infers the order of operations for the transfer interval with the maximum reward.

8. The hierarchical intelligent reasoning method for end-to-end grid switch cooperative operation according to claim 1, characterized in that, In step S7, the state space, action space, and reward / penalty function of the switching optimization model are defined as follows: The state space information of the switching optimization model includes the working state of the task interval and the opening and closing state of the switching equipment in each interval; the transfer interval set and maintenance interval set obtained based on the upper-level D3QN inference are iterated through respectively; the switching equipment contained in each interval is determined by graph model search, and the action space of transfer switching and maintenance switching is constructed; the reward function is designed according to the closeness between the target state and the current equipment state to ensure that the operation is close to the target state; at the same time, repeated operations are penalized to prevent the model from frequently executing invalid or repetitive actions.

9. The hierarchical intelligent reasoning method for end-to-end power grid switch cooperative operation according to claim 6, characterized in that, The safety constraints and operational efficiency of the integrated balancing system are considered, and a reward / penalty function is designed based on state-space information; a voltage deviation penalty is also set. Maximum node voltage deviation for When the node voltage deviation If the deviation exceeds this range, penalties will be imposed based on the degree of deviation, as follows: ;in, It is a proportional factor that determines the sensitivity of rewards to deviations from the target limit; Represents the initial constant; sets the line load rate penalty. Maximum allowable load rate of the line It is 100%, when the actual load rate of the line is... When this limit is exceeded, penalties will be imposed according to the degree of overload, as follows: Set loop current penalty When the interval switches to the operating state, the closed-loop inrush current The setting value of the instantaneous overcurrent protection must not be exceeded. Closed-loop steady-state current The overcurrent protection setting value must not be exceeded. The details are as follows: Set a penalty for transformer exceeding limits. Maximum allowable load rate of transformer The load factor is 100% when the transformer load rate is... Penalties will be imposed based on the degree of overload after exceeding the limit, as detailed below: Set penalties for repetitive actions When the interval transitions from a non-running state to a running state, if the interval switches back to a non-running state due to repeated actions, an invalid action is generated; the details are as follows: ;in, Indicates the number of times the action is repeated at intervals. Indicates the total number of intervals in the action space; when there are no lost radio zones and all safety constraints are met, a reward is set. To reduce the number of interval operations and improve system efficiency; specifically as follows: ;in, Indicates the number of intervals between operations. This represents the total number of intervals in the action space; in the state-action evaluation of the transfer optimization model, the agent selects an action from the action space, executes it, updates the state, and calculates the reward; the agent optimizes the policy by evaluating the Q-value of each state-action pair to maximize the cumulative reward; the expression for the Q-value is as follows: ;in, Indicates the current state and actions Q value, Indicates the discount factor. The system represents immediate rewards; it designs a dynamic adjustment mechanism for reward weights; the Q-value is composed of a weighted sum of multiple reward items, therefore the impact of each reward item on the Q-value can be represented by the gradient of the Q-value relative to the weight of the reward item; the loss function can be calculated using the chain rule. Reward weight The gradient is as follows: ;in, Represents the loss function Follow The rate of change of the value reflects The sensitivity of the value to loss; express The value depends on the weight of that reward item. The rate of change reflects the weight. right The influence of the value; based on the calculated gradient, update the reward weights using gradient descent: ;in, Indicates the first Round weights Indicates the updated number Round weights The learning rate is represented here. To avoid overestimation of Q-values ​​in the optimization model, the Q-value is updated using the dual network structure of the D3QN algorithm. The specific Q-value update is represented as follows: ;in, This indicates the current state of online network computation. and actions Q value, Indicates the current instant reward. Indicates the discount factor. Indicates the learning rate. This indicates that the next state is calculated through the target network. Executable actions The maximum target Q value; Indicates that the online network will choose the next state. The action with the largest Q value.

10. The hierarchical intelligent reasoning method for end-to-end grid switch cooperative operation according to claim 1, characterized in that, The specific steps of step S8 are as follows: Receive the list of transfer intervals and maintenance intervals output by the upper layer D3QN, split them into two parallel subtasks: transfer and maintenance, and assign a unique ID to each task. Each task is feature-encoded, and information on the number of interval switches, initial state, and target state is fused to generate a low-dimensional embedding vector, enabling the model to accurately distinguish the operational logic of different tasks. The MT-D3QN agent selects interval actions from the action space, updates the state of the graph model, calculates the Q-value of each task, and updates the reward weight of the task. By sharing the parameters of the fully connected FC layer, the common logic of the five-prevention rules and the general knowledge of the timing constraints of switch operations are extracted, realizing cross-task knowledge transfer. When an unfavorable situation occurs during a task interval, the agent immediately stops the current round and starts a new round for retraining; Through continuous iteration, the reward gradually converges to the optimal value; the optimal switching sequence for power supply and maintenance is output.

Images