Knowledge graph-fused reinforcement learning switching operation anti-error verification method

Abstract

The invention relates to the technical field of automation and intelligent operation and maintenance of a power system, in particular to a knowledge graph-fused reinforcement learning switching operation anti-error verification method, which systematically extracts a multi-dimensional anti-error rule covering an operation sequence, an equipment state and an electrical safety distance by constructing an operation ticket knowledge graph, and improves the accuracy of the operation ticket knowledge graph. The defect that a traditional single-station anti-error system is incomplete in rule coverage is overcome, meanwhile, in combination with deep mining of a reinforcement learning model on historical operation data, implicit anti-error rules can be automatically extracted, illegal scenes which are not covered by a traditional rule base are supplemented, overall-process and multi-level accurate verification of switching operation is achieved, and the verification efficiency is improved. And the risks of misoperation and missing detection are greatly reduced.

Description

Technical Field

[0001] This invention relates to the field of power system automation and intelligent operation and maintenance technology, and more specifically, to a reinforcement learning-based method for preventing errors in switching operations that integrates knowledge graphs. Background Technology

[0002] With rapid economic development and increasing social demand, the electricity demand of the power grid continues to rise. In particular, with the large-scale integration of renewable energy sources such as hydropower, wind power, and photovoltaics, and the rapid increase in the number of power grid substations, the complexity and operational pressure of the power grid are increasing. Switching operations are a key link in substation operation, and their accuracy and timeliness directly affect the operational efficiency and safety of the power grid. Traditional switching operations often lack real-time dynamic guidance, and operators are prone to misoperation or omission when facing complex field conditions, which may lead to huge economic losses and safety consequences. With the acceleration of the digitalization and intelligentization of the power system, the safety and efficiency of substation switching operations face higher requirements.

[0003] In existing technologies, most anti-misoperation systems adopt a single-station operation mode. Under the centralized control station operation and maintenance mode, the anti-misoperation function is not perfect and lacks a regional power grid-level collaborative anti-misoperation mechanism. It is difficult to achieve global consistency verification of inter-station interconnection lines and multi-substation linkage operations, which poses risks of erroneous remote control and operation. Therefore, how to combine knowledge graphs and reinforcement learning to achieve real-time dynamic anti-misoperation verification in order to adjust the anti-misoperation strategy and ensure the accuracy and safety of each operation is the problem to be solved by this invention. To this end, a reinforcement learning anti-misoperation verification method for switching operations that integrates knowledge graphs is proposed. Summary of the Invention

[0004] The purpose of this invention is to provide a reinforcement learning-based method for preventing errors in switching operations that integrates knowledge graphs, in order to solve the aforementioned problems in the prior art.

[0005] This invention is achieved through the following technical solution: A reinforcement learning-based method for preventing errors in switching operations, incorporating knowledge graphs, includes: Collect operation ticket data of the regional power grid, construct an operation ticket knowledge graph, and form operation ticket templates and error prevention rule base; Based on the knowledge graph of operation tickets, combined with information on multi-source heterogeneous equipment and error prevention rules, the information on multi-source heterogeneous equipment includes information on primary and secondary equipment ledgers, real-time status and wiring methods; Based on reinforcement learning algorithms, a reinforcement learning model based on error prevention strategies is constructed by defining the state space, action space, and reward function, with the power grid state and switching operation history as inputs. Train a reinforcement learning model, and generate and adjust error prevention strategies through interactive learning; Real-time acquisition of power grid topology and equipment status data, combined with operation ticket knowledge graph and reinforcement learning model to verify switching operations, and output the verification results; Based on the results of the error prevention and verification operations, the knowledge graph and reinforcement learning model of the operation ticket are continuously updated.

[0006] Preferably, the construction of the operation ticket knowledge graph includes: The operation ticket data is preprocessed. Based on natural language processing technology, key elements are extracted from the preprocessed operation ticket data, including operation tasks, equipment objects, action instructions and equipment status information. A graph database is used to construct a structured operation ticket knowledge graph in the form of nodes and edges.

[0007] Preferably, the process of forming the operation ticket template and the error prevention rule base includes: The operation dependencies and state transition paths between nodes are extracted using a graph traversal algorithm. Constraints in three dimensions—operation sequence constraints, equipment state transition condition constraints, and electrical safety distance constraints—are obtained from the topology. A logical expression model is constructed based on these three dimensions of constraints, and the model is stored in a structured manner to form a rule base for preventing errors. Identify and cluster the target paths and node sequences that repeatedly appear in the operation ticket knowledge graph and are used to complete the same type of switching operation task; An operation ticket template is generated for each type of switching operation task. The operation ticket template is a standardized sequence that includes operation steps, operation objects and expected state related data. The generated anti-misoperation rule base and the operation ticket template are integrated into a unified knowledge system.

[0008] Preferably, the construction of the logical expression model includes: The operation sequence constraints include:

[0009] In the formula, To judge the data, The sequence number of the current step. The sequence number of the currently executed steps, when If so, proceed with the current step normally. If the value is not equal to 0, then output a signal to stop executing the current step; The device state transition condition constraints include: Determine whether the current device status is on or off. If it is on, determine whether the sum of the current grid output power and the device power is less than the upper limit of the grid output power. If it is, output a signal that the device can be turned on; otherwise, output a signal that the device cannot be turned on. The electrical safety distance constraints include:

[0010] In the formula, For current electrical safety distances, For minimum electrical safety distance, Rated power of the equipment The power of the equipment used to calibrate the minimum electrical safety distance. Preferably, the step of combining operation ticket knowledge graph with multi-source heterogeneous device information and error prevention rules includes: Based on the semantic framework of the operation ticket knowledge graph, a unified data model is constructed for primary and secondary equipment ledgers, real-time status and wiring information. Through data adapters and interface services, information from heterogeneous devices from different data sources is collected and integrated in real time, and semantic alignment and entity association are performed based on a unified data model. The integrated information on heterogeneous devices from multiple sources is coupled with the rule base for preventing errors.

[0011] Preferably, the construction of the reinforcement learning model based on the error prevention strategy includes: The input includes the power grid status and switching operation history. The power grid status includes the real-time power grid topology and equipment status. The switching operation history includes the operation type, operation timestamp, and operation result. The state space is defined as the set of operating states of all equipment in the power grid, and the action space is the set of executable error prevention and verification decisions. The reward function is used to quantify the consequences of actions and to construct an automatic learning and optimization framework for error prevention strategies.

[0012] Preferably, the training of the reinforcement learning model, through interactive learning to generate and adjust the error prevention strategy, includes: The power grid status and anti-misoperation rules are encapsulated into a reinforcement learning simulation environment. State transition logic and switching operation constraints are defined. Operation sequence samples are generated through interaction between the agent and the environment. The status, actions and reward feedback are recorded. At the same time, an experience playback pool is built. A fault prevention strategy network is constructed based on the PPO algorithm. The input is the current power grid state, and the output is the probability distribution of fault prevention verification actions during switching operations. The parameters of the fault prevention strategy network are iteratively optimized using data from the experience playback pool. The expected reward is maximized through gradient ascent, and the network gradually converges to a stable fault prevention strategy. The performance indicators of the training-based error prevention strategy, namely the interception rate of illegal operations and the improvement of operational efficiency, are verified in a simulation environment. After the error prevention performance indicators are evaluated and the system is deployed to the actual power grid dispatching system, new operation data is collected in real time to update the experience playback pool, and the adjustment cycle is set to adjust the model parameters to adapt to changes in the power grid topology.

[0013] Preferably, the construction of the error prevention strategy network based on the PPO algorithm includes: A fault prevention strategy network is constructed based on the PPO algorithm. The current power grid state is encoded as a state vector and input into the fault prevention strategy network to generate the probability distribution of fault prevention verification actions for switching operations. The actions are executed according to the probability sampling. The environment simulates state transitions based on physical rules and safety constraints to generate the next state, immediate reward and termination flag. During training, batches of experience data are randomly sampled from the experience replay pool, and the parameters of the error prevention strategy network are iteratively updated using the gradient ascent algorithm. The optimization objective is to maximize the long-term cumulative expected reward. Through continuous iterative optimization, the parameters of the error prevention strategy network are converged to a stable state, forming an interpretable error prevention strategy.

[0014] Preferably, the step of combining the operation ticket knowledge graph with a reinforcement learning model to verify the switching operation and outputting the verification result includes: The real-time power grid status is structurally mapped to the operation tickets to be verified. The physical device status is encoded into node feature vectors through graph neural networks, and the operation ticket steps are converted into edge attributes with temporal weights to construct a state-operation joint embedding space. The state representation in the state-operation joint embedding space is input into a deep Q-network. The state-action value function calculates an expected reward value for each switching operation step to be executed in the operation ticket. Based on the expected reward value, the initial decision sequence is generated using the maximum value principle. Parallel verification is initiated using the knowledge graph of the operation ticket. Based on the historical state change patterns of the equipment, electrical coupling relationships, and predefined temporal dependency rules, logical reasoning is performed on the same operation ticket to detect potential conflicts in the initial decision and output a set of constraints. The set of constraints detected by the operation ticket knowledge graph is back-injected into the reinforcement learning model through the constraint propagation mechanism. The weight parameters of the state-action value function are dynamically adjusted according to the constraints to suppress the probability of selecting illegal actions, forcibly correct decision biases, and generate a verification report containing the conclusion of interception or passage. If it is intercepted, the violation type and correction suggestions are marked, and correction suggestions are provided through the reasoning path of the operation ticket knowledge graph.

[0015] Preferred options also include: The system continuously collects the anti-misoperation verification operation result data for each switching operation, covering the operation time and equipment status. The collected anti-misoperation verification operation result data is then cleaned, standardized in format, and stored. Based on the new knowledge from the results of the error prevention and verification operation, a knowledge graph construction algorithm is used to integrate the new knowledge into the existing operation ticket knowledge graph and optimize the graph structure. The technical solution of the present invention has at least the following advantages and beneficial effects: This invention provides a reinforcement learning-based method for preventing and verifying switching operations using knowledge graphs. By constructing an operation ticket knowledge graph, it systematically extracts multi-dimensional anti-misoperation rules covering operation sequence, equipment status, and electrical safety distance, overcoming the shortcomings of incomplete rule coverage in traditional single-station anti-misoperation systems. At the same time, by combining reinforcement learning models with in-depth mining of historical operation data, it can automatically extract implicit anti-misoperation rules, supplementing violation scenarios not covered by traditional rule bases, and achieving accurate verification of the entire switching operation process at multiple levels, greatly reducing the risk of misoperation and missed detection.

[0016] Secondly, a dynamic power grid state model was constructed to collect power grid topology and equipment status data in real time. Combined with the anti-misoperation rules in the operation ticket knowledge graph, real-time dynamic verification of the entire switching operation process was realized. Compared with the traditional static anti-misoperation system, it can dynamically adjust the verification logic according to the real-time operating status of the power grid, ensuring the consistency between the operation command and the current power grid status, significantly reducing the risk of misoperation caused by state mismatch, and improving the real-time performance and dynamic adaptability of the verification. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the execution process of the present invention; Figure 2 This is a flowchart illustrating one embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0020] The independently described modules or sub-modules may or may not be physically separated; they may be implemented in software or hardware, and some modules or sub-modules may be implemented in software, with the processor calling the software to implement the function of these modules or sub-modules, while other modules or sub-modules may be implemented in hardware, such as through hardware circuits. Furthermore, some or all of the modules can be selected to achieve the purpose of this application's solution according to actual needs.

[0021] Please refer to Figures 1-2 This invention provides a reinforcement learning-based method for preventing errors in switching operations by integrating knowledge graphs.

[0022] S1: Collect operation ticket data of the regional power grid, construct an operation ticket knowledge graph, and form operation ticket templates and error prevention rule base.

[0023] Specifically, the process involves collecting operation ticket data from the regional power grid, extracting operation tasks, equipment objects, action instructions, and equipment status information, constructing an operation ticket knowledge graph, forming standardized operation ticket templates and an error prevention rule base, collecting historical and real-time operation ticket data from the regional power grid covering various typical switching operation scenarios, and preprocessing the operation ticket data, including cleaning, deduplication, and standardization processes to ensure data quality and format consistency. Based on natural language processing technology, key elements, including operation tasks, equipment objects, action instructions, and equipment status information, are extracted from the preprocessed operation ticket data. Graph database technology is used to construct a structured operation ticket knowledge graph in the form of nodes and edges. The operation ticket knowledge graph is analyzed to summarize the general logic and constraints of switching operations, forming a standardized error prevention rule base, abstracting common operation patterns, and generating standardized operation ticket templates to achieve the standardization and intelligentization of switching operations. The error prevention rule base includes three dimensions: operation sequence constraints, equipment state transition conditions, and electrical safety distance requirements. In addition, the process of creating standardized operation ticket templates and error prevention rule bases is as follows: A multi-dimensional analysis is performed on the preprocessed operation ticket knowledge graph. A graph traversal algorithm (depth-first search) is used to extract the operation dependencies and state transition paths between nodes. The general logic of switching operations implied by nodes and edges is analyzed, and constraints in three dimensions—operation sequence constraints, equipment state transition conditions, and electrical safety distance requirements—are summarized from the topology. These general logics and constraints are then transformed into formal logical expressions, structured, and stored to form a standardized anti-error rule base. Specifically, the operation sequence constraint is derived by analyzing operation dependencies to determine the order of operation steps; the equipment state transition condition is extracted from the state transition path to determine the prerequisite and target states of the equipment operation; and the electrical safety distance requirement is determined by combining... The power grid topology is analyzed, and the safe distance between equipment operations is determined. Typical paths and node sequences that repeatedly appear in the operation ticket knowledge graph for completing the same type of switching operation task are identified and clustered. Common operation steps, equipment objects, and state transition sequences in typical paths are abstracted and patterned into common operation patterns. A standardized operation ticket template is generated for each type of typical switching operation task, clearly defining the standardized sequence of operation steps, operation objects, and expected states to ensure the uniformity of the operation process. The generated error prevention rule base and operation ticket template are then integrated into a unified knowledge system. The error prevention rule base provides real-time verification basis for each step of the operation ticket template, and the operation ticket template itself becomes the concrete embodiment of the standards maintained by the error prevention rule base. Identify and cluster the target paths and node sequences that repeatedly appear in the operation ticket knowledge graph and are used to complete the same type of switching operation task; An operation ticket template is generated for each type of switching operation task. The operation ticket template is a standardized sequence that includes operation steps, operation objects and expected state related data. The generated anti-misoperation rule base and the operation ticket template are integrated into a unified knowledge system.

[0024] The construction of the logical expression model includes: The operation sequence constraints include:

[0025] In the formula, To judge the data, The sequence number of the current step. The sequence number of the currently executed steps, when If so, proceed with the current step normally. If the value is not equal to 0, then output a signal to stop executing the current step; The device state transition condition constraints include: Determine whether the current device status is on or off. If it is on, determine whether the sum of the current grid output power and the device power is less than the upper limit of the grid output power. If it is, output a signal that the device can be turned on; otherwise, output a signal that the device cannot be turned on. The electrical safety distance constraints include:

[0026] In the formula, For current electrical safety distances, For minimum electrical safety distance, Rated power of the equipment The power of the equipment used to calibrate the minimum electrical safety distance.

[0027] The specific work involves: comprehensively collecting historical and real-time operation ticket data of the regional power grid, covering various typical switching operation scenarios, including but not limited to bus switching operations, main transformer power outage and restoration operations, line power outage and restoration operations, capacitor switching, and grounding switch / ground wire operations. The collected operation ticket data is then preprocessed by data cleaning, deduplication, and standardization. Specifically, invalid, erroneous, or redundant records are removed through cleaning; duplicate operation tickets are deduplicated using hash algorithms or rule matching; and standardized processing unifies equipment naming, action descriptions, and status representation formats to address the heterogeneity of multi-source data and ensure the data quality for subsequent analysis.

[0028] Based on natural language processing (NLP) technology, preprocessed operation ticket data is segmented, part-of-speech tagging (POS), and entity recognition is performed to extract key elements including operation tasks, equipment objects, action instructions, and equipment status information. Through entity recognition and relation extraction techniques, unstructured text data is transformed into structured semantic representations. An operation ticket knowledge graph is then constructed using a graph database, with operation tasks, equipment objects, and action instructions as nodes, and operation dependencies and state transition paths as edges, forming a structured operation ticket knowledge graph that intuitively displays the logical chain of switching operations. The operation ticket knowledge graph is analyzed to summarize the general logic and constraints of switching operations, forming a rule base for error prevention covering three dimensions: operation sequence constraints, equipment state transition conditions, and electrical safety distance requirements. Simultaneously, common operation patterns are abstracted from the knowledge graph to generate standardized operation ticket templates for each typical switching operation task. These templates specify the standardized sequence of steps for the switching operation task, the expected equipment state transitions at each step, and the logical order that must be followed, ensuring the standardization and uniformity of operation steps, equipment state transitions, and logical order. S2: Based on the operation ticket knowledge graph combined with multi-source heterogeneous equipment information and error prevention rules, the multi-source heterogeneous equipment information includes primary and secondary equipment ledgers, real-time status and wiring information.

[0029] Specifically, based on the operation ticket knowledge graph, information from multiple heterogeneous equipment sources and anti-misoperation rules are integrated. The information from multiple heterogeneous equipment sources includes primary and secondary equipment ledgers, real-time status, and wiring information to ensure real-time data matching and consistency. Based on the semantic framework of the operation ticket knowledge graph, a unified data model covering primary and secondary equipment ledgers, real-time status, and wiring information is constructed to provide a core paradigm for standardized data access and mapping. Through data adapters and interface services, information from multiple heterogeneous equipment sources such as equipment management systems, SCADA systems, and relay protection devices is collected and integrated in real time. Semantic alignment and entity association are performed according to the unified data model to ensure real-time data matching and logical consistency. The integrated information from multiple heterogeneous equipment sources is coupled with the anti-misoperation rule base to provide factual data support for each anti-misoperation rule, thereby driving accurate and dynamic anti-misoperation verification throughout the entire switching operation process. The specific work involves: constructing a unified data model based on the semantic framework established by the operation ticket knowledge graph, covering the ledger information, real-time status, and wiring information of primary and secondary equipment, defining the core attributes, status characteristics, and topological relationships of equipment entities. The core attributes include ledger information such as equipment ID, type, model, and commissioning date; the status characteristics include real-time status such as running, hot standby, and cold standby; and the topological relationships include wiring information such as electrical connection methods between equipment and the bay to which they belong. Then, through ontology modeling technology, the core concepts of equipment, status, and wiring and their semantic relationships are clarified, forming a structured data paradigm.

[0030] Through data adapters and interface services, multi-source heterogeneous equipment information from data sources such as equipment management systems, Supervisory Control and Data Acquisition (SCADA) systems, and relay protection devices is collected and integrated in real time. The data adapters require customized development for the protocol formats of different data sources, completing data parsing and initial cleaning. The interface services provide a unified data access layer, supporting cross-system data calls and synchronization. During integration, semantic alignment is performed based on a unified data model, associating equipment entities from different data sources through unique identifiers and verifying data logical consistency to form a globally consistent data view, ensuring real-time data matching and logical consistency. The integrated and aligned multi-source heterogeneous equipment information is deeply coupled with a knowledge graph-based anti-error rule base. By establishing dynamic associations between anti-error rules and multi-source heterogeneous equipment information, accurate factual data support is provided for each formalized anti-error rule, enabling abstract verification logic to perform calculations and reasoning based on the actual real-time state of the power grid. This drives precise anti-error verification throughout the entire switching operation process, significantly improving the accuracy and reliability of the verification. S3: Based on reinforcement learning algorithms, using grid status and switching operation history as inputs, define the state space, action space and reward function, and construct a reinforcement learning model based on error prevention strategy.

[0031] By combining reinforcement learning algorithms, an automatic learning and optimization framework for error prevention strategies is constructed. This framework uses grid status and switching operation history as inputs to define a state space, action space, and reward function. The core elements of the reinforcement learning model are clearly defined, with grid status encompassing real-time grid topology and equipment status, and switching operation history including operation type, timestamp, and result. The state space is defined as the set of operating states of all key grid equipment, and the action space is the set of executable error prevention verification decisions. The reward function quantifies the consequences of actions, using safety (penalty for violations), efficiency (reward for operation duration), and compliance with procedures (penalty for voltage fluctuations) as core indicators, thereby constructing the automatic learning and optimization framework for error prevention strategies. The specific work involves: combining reinforcement learning algorithms, defining the power grid status and switching operation history as the input to the reinforcement learning model. The power grid status comprehensively covers real-time power grid topology and equipment status information, and is dynamically updated through the fusion of SCADA system and equipment ledger data. The switching operation history includes operation type (closing, opening, phase verification, etc.), operation timestamp, and operation result (success / failure / violation), which needs to be extracted from event records and stored in a standardized manner. Based on the determined input to the reinforcement learning model, the state space, action space, and reward function are defined. The state space is defined as the set of operating states of all key equipment in the power grid.

[0032] The action space is defined as the executable anti-misoperation verification decisions; the state space is a high-dimensional, structured representation that encompasses the real-time opening and closing positions and energized states of primary equipment such as circuit breakers, disconnectors, main transformers, and lines, as well as key signals of secondary equipment such as relay protection device pressure plate states and setting zones. Furthermore, real-time power grid topology relationships, such as electrical island division and bus operation modes, are also integrated into the state space through graph embedding, forming a digital mirror reflecting the current operating conditions of the power grid. The action space corresponds to the set of executable anti-misoperation verification decisions, representing the proactive intervention measures the system can take when facing specific states. These include: following the current operation steps, alarming and blocking the current operation, suggesting alternative safe operation sequences, or automatically executing auxiliary safety verification instructions. The reward function is designed to quantitatively evaluate the anti-misoperation verification actions taken by the agent. The consequences revolve around three core indicators: safety, efficiency, and compliance with procedures. Safety is achieved through penalties for violations: if the model allows operations that should be prohibited (such as pulling a disconnect switch under load), a large negative reward (-100) is given; if a legal operation is prohibited, a small negative reward (-10) is given to balance conservatism. Efficiency is incentivized through operation time rewards: the shorter the operation completion time, the higher the reward (e.g., +1 for every 1 second shortened), with an upper limit (+50) to prevent the model from excessively pursuing speed. Compliance with procedures is constrained by voltage fluctuation penalties: if an operation causes bus voltage fluctuations to exceed the procedure threshold (±5% of rated voltage), a linear penalty is applied based on the fluctuation amplitude (-5 for every 1% exceeding the threshold). Based on the input of the reinforcement learning model and the defined state space, action space, and reward function, an automatic learning and optimization framework for error prevention strategies is constructed. S4: Train the reinforcement learning model, and generate and adjust the error prevention strategy through interactive learning.

[0033] Specifically, based on the automatic learning and optimization framework for error prevention strategies, a reinforcement learning model is trained. Error prevention strategies are generated and adjusted through interactive learning. The power grid state and error prevention rules are encapsulated into a reinforcement learning simulation environment. State transition logic and switching operation constraints are defined. Operation sequence samples are generated through interaction between the agent and the environment, and the state, actions, and reward feedback are recorded. At the same time, an experience replay pool is constructed for training the reinforcement learning model. An error prevention strategy network is constructed based on the PPO algorithm. The input is the current power grid state, and the output is the probability distribution of the error prevention verification actions of switching operations. The parameters of the error prevention strategy network are iteratively optimized using the data from the experience replay pool. The expected reward is maximized through gradient ascent, and the exploration and utilization are balanced to avoid local optima. The network gradually converges to a stable error prevention strategy. The error prevention performance indicators of the trained error prevention strategy are verified in the simulation environment, namely the illegal operation interception rate and the improvement in operation efficiency. After the error prevention performance indicators are evaluated and the system is deployed to the actual power grid dispatching system, new operation data is collected in real time to update the experience replay pool, and the model parameters are fine-tuned periodically to adapt to changes in the power grid topology, forming a closed-loop learning mechanism for continuous optimization. The formula for calculating the violation interception rate is as follows: ; In the formula: The violation interception rate is used to quantify the ability to identify and prevent dangerous operations. This represents the number of violations successfully intercepted during the test, i.e., the number of scenarios in which the "alarm and block" action is correctly output. This represents the total number of violation scenarios used in the test. The ideal outcome means that all illegal injection attempts have been blocked without any false alarms, resulting in the highest level of security. A value approaching 0 indicates that the system has almost failed to intercept any illegal operations, resulting in extremely poor security and complete failure. When measuring safety, the closer to 100%, the better. The expression for calculating the improvement in operational efficiency is as follows: ; In the formula: The degree of improvement in operational efficiency is used to quantify the optimization effect on the operational process when performing normal and safe operations, that is, the proportion of time saved, which is expressed as a relative percentage of efficiency improvement; The baseline time for performing a normal operational task refers to the average time it takes to complete the task using traditional manual methods or existing static systems. The average time taken to complete the same normal operation task under the guidance of error prevention strategies; This indicates that the error prevention strategy is faster and more efficient than the baseline method. A tendency toward 0 indicates that the error prevention strategy is as efficient as the benchmark method. This indicates that error prevention strategies are actually slower and less efficient than benchmark methods. The efficiency gain should be greater than 0 and as large as possible. Furthermore, the process of constructing a fault-prevention strategy network based on the PPO algorithm is as follows: A fault prevention strategy network is constructed based on the PPO (Proximal Policy Optimization) algorithm. The current power grid state is encoded as a state vector and input into the fault prevention strategy network to generate the probability distribution of fault prevention verification actions for switching operations. Actions are executed based on probability sampling. The environment simulates state transitions according to physical rules and safety constraints, generating the next state, immediate reward, and termination flag. At the same time, illegal actions are intercepted through hard constraints to ensure safety. During training, batches of experience data are randomly sampled from the experience replay pool, and the parameters of the fault prevention strategy network are iteratively updated through the gradient ascent algorithm. Its optimization objective is to maximize the long-term cumulative expected reward. The importance sampling and policy constraint mechanism introduced by the PPO algorithm effectively balances the exploration of new strategies and the utilization of known efficient strategies, avoiding the training process from getting trapped in local optima too early. Through continuous iterative optimization, the decision-making ability of the fault prevention strategy network is gradually enhanced, and the parameters of the fault prevention strategy network are converged to a stable state to form an interpretable fault prevention strategy, thereby improving the interception rate of illegal operations and the efficiency of normal operation of the system. The calculation expression for the probability distribution of the anti-misoperation verification action during switching operations is as follows: ; In the formula: To be in a given power grid state Under these conditions, the error prevention strategy network selects to execute a specific error prevention verification action. The probability is the output of the error prevention strategy network, a probability distribution over all available actions; The parameters (weights and biases) of the error prevention strategy network are optimized through training. For action space A specific error prevention verification action represents a specific error prevention verification decision. Given a power grid state, the current power grid state vector is represented, which encodes all the key information of the current power grid, including the state of primary equipment, the state of secondary equipment, and the topological relationships represented by graph embedding. To prevent errors, the policy network is in a given power grid state. Below are the steps for error prevention and verification. The scoring function; For the action space All possible error prevention verification actions The scores are summed, including possible error-prevention verification actions. For actions with a high probability of error correction, specifically, if the probability of error correction for the current action exceeds a preset judgment threshold, it is considered a possible action for error correction. After training, the error correction strategy will tend to stabilize in a given state. Under these conditions, the probability of the optimal action will approach 1, while the probability of other suboptimal or dangerous actions will approach 0, indicating that the anti-misoperation strategy network has learned which safest and most effective anti-misoperation verification action should be taken under what power grid conditions. The specific work involves: encapsulating the power grid state and anti-misoperation rules into an interactive reinforcement learning simulation environment, and then defining the state transition logic: simulating power grid topology changes and equipment state updates based on the current state and agent actions; simultaneously setting switching operation constraints (such as prohibiting disconnecting switches under load and intercepting bus voltage exceeding limits) to ensure the legality of actions; when the agent interacts with the environment, generating operation sequence samples that record state, actions, and reward feedback, i.e., state-action-reward pairs, where reward feedback comprehensively considers safety, efficiency, and procedural compliance; all samples are recorded in an experience replay pool to provide a reusable data foundation for offline training of the reinforcement learning model; constructing an anti-misoperation strategy network based on the PPO algorithm, using the current power grid state as input and the probability distribution of switching operation anti-misoperation verification actions as output; during the training phase, using the interaction data stored in the experience replay pool, continuously iteratively updating the parameters of the anti-misoperation strategy network through a gradient ascent algorithm, with the optimization objective being to maximize the long-term accumulated expected reward. By introducing importance sampling and policy constraint mechanisms, the relationship between exploring new strategies and utilizing known high-return strategies is effectively balanced, avoiding the training process from getting trapped in local optima too early. Through continuous optimization, the decision-making ability of the anti-error strategy network gradually improves, converging to a stable and interpretable anti-error strategy. The anti-error performance indicators of the trained anti-error strategy are verified in a simulation environment, including the interception rate of illegal operations and the improvement in operation efficiency. The interception rate is tested through historical illegal operation scenarios (target ≥ 99%), and the efficiency improvement is tested through normal operation sequences (operation time reduced by more than 20%). After the targets are met, the anti-error strategy is deployed to the actual power grid dispatch system, and new operation data (including equipment status changes and operation result feedback) are collected in real time. The experience replay pool is dynamically updated, and the model parameters are fine-tuned periodically with new data, so that the model can dynamically adapt to the evolution of the power grid topology, the adjustment of the operation mode, and the commissioning of new equipment, thus forming a continuously optimized closed-loop learning mechanism to ensure the long-term effectiveness of the anti-error verification capability. S5: Real-time acquisition of power grid topology and equipment status data, combined with operation ticket knowledge graph and reinforcement learning model to verify switching operations, and output the verification results; By deploying IoT sensors and SCADA systems to collect real-time data on power grid topology and equipment status, and combining this with error prevention rules in the operation ticket knowledge graph, a dynamic power grid status model is constructed. Based on the reinforcement learning model and semantic reasoning of the operation ticket knowledge graph, a multi-dimensional set of verification rules covering electrical five-prevention, logic interlocking, and timing constraints is generated. This achieves an upgrade from static rules to dynamic verification logic that adapts to changes in power grid topology. The real-time power grid status (power grid topology and equipment status data) and the operation ticket to be verified are input into the reinforcement learning model. The reinforcement learning model evaluates the compliance of the operation through the state-action value function, corrects decision biases by combining the operation ticket knowledge graph, and outputs verification results including interception / pass conclusions. Furthermore, the process of outputting the verification results, including the interception / pass conclusion, is as follows: The real-time power grid status (including power grid topology and equipment status data) is structurally mapped to the operation tickets to be verified (including operation step sequences, equipment action order, and timing constraints). A graph neural network encodes the physical equipment status into node feature vectors, and the operation ticket steps are converted into edge attributes with temporal weights. Then, all node feature vectors and edge attributes representing switching operations are integrated to construct a state-operation joint embedding space, which serves as the input representation layer of the reinforcement learning model. This ensures a unified representation of spatiotemporal state and operational semantics. Physical equipment (circuit breakers, disconnectors, busbars, etc.) are used as nodes, and a graph neural network (GNN) is employed. The message passing mechanism aggregates the state information (such as voltage and opening / closing position) of each node and its neighbors, and encodes each physical device into a node feature vector. This node feature vector contains the physical device's own state and its context information in the global topology. The reinforcement learning model evaluates the compliance of operations based on the state-action value function (Q-value). The state representation of the state-action joint embedding space is input into the deep Q-network (DQN). The DQN's state-action value function calculates an expected reward value for each switching operation step to be executed in the operation ticket. Based on the expected reward value, an initial decision sequence is generated using the maximum value principle. Simultaneously, the operation ticket knowledge graph initiates parallel verification. Based on the historical state change patterns of equipment, electrical coupling relationships, and predefined timing dependency rules, it performs logical reasoning on the same operation ticket to detect potential conflicts in the initial decision (covering violations of the five electrical safety regulations, failure of logical interlocking, and timing misalignment; violations of the five electrical safety regulations mean violating the most basic safety procedures; failure of logical interlocking means ignoring the implicit logical dependencies or mutual exclusion relationships between equipment; timing misalignment means that the operation sequence or time interval does not meet the technical requirements). It outputs a set of constraints, clearly indicating under what conditions the switching operation is prohibited or needs to be modified. The set of constraints detected by the operation ticket knowledge graph is back-injected into the reinforcement learning model through the constraint propagation mechanism. The weight parameters of the state-action value function are dynamically adjusted according to the constraints to suppress the probability of selecting illegal actions, forcibly correct decision biases, and finally generate a verification report containing interception / pass conclusions. If it is intercepted, the violation type (electrical / logical / timing) and correction suggestions are marked, and specific correction suggestions and interpretable decision basis are provided through the interpretable reasoning path of the operation ticket knowledge graph to ensure the traceability of verification results and engineering credibility. The specific work involves: using deployed IoT sensors and Supervisory Control and Data Acquisition (SCADA) systems to collect real-time data on the power grid topology and equipment status (covering the operating status of various primary and secondary equipment, electrical measurement data, and the opening and closing position signals of switches and disconnectors); combining this data with the error prevention rules defined in the operation ticket knowledge graph to construct a dynamic power grid state model. Using a graph structure as the core, physical devices are abstracted as nodes, and weighted edges represent the electrical connections and logical constraints between devices. Node attributes and edge weights are updated based on real-time data. A time-series alignment mechanism synchronizes the time stamps of multi-source data, eliminating acquisition delay differences between heterogeneous systems and ensuring the spatiotemporal consistency of state representation. Based on semantic reasoning using a reinforcement learning model and an operation ticket knowledge graph, a three-layer verification rule system covering electrical safety, logical interlocking, and timing constraints is constructed. Explicit rules for electrical safety are extracted from the operation ticket knowledge graph and transformed into constraints understandable to the reinforcement learning model using natural language processing. Implicit logical interlocking rules are derived using the coupling relationships within the operation ticket knowledge graph, and the rule coverage is expanded by combining the state transition prediction capability of reinforcement learning. Timing constraints are introduced, and timing consistency verification rules are generated based on the step sequence of the operation ticket and the equipment action delay, thus generating a multi-dimensional verification rule set covering traditional electrical safety, logical interlocking, and timing constraints. Real-time power grid status and the operation ticket to be verified are input into the reinforcement learning model. This model quantifies and evaluates operational compliance using a state-action value function (Q-value): taking real-time grid status as input, it calculates the expected reward after executing each step in the operation ticket (positive reward for compliant operations, negative penalty for non-compliant operations), and generates an initial decision based on the principle of maximizing value. Simultaneously, it introduces an operation ticket knowledge graph to perform secondary verification of the reinforcement learning decision: by tracing historical changes in equipment status, it determines whether switching operations conform to physical laws; if a decision deviation is found, the action value function is corrected through the constraint propagation mechanism of the operation ticket knowledge graph, forcibly suppressing the probability of selecting non-compliant actions, and finally outputting a verification report containing interception / pass conclusions, and marking the type of violation (electrical five-prevention, logical conflict, or timing error) and correction suggestions; S6: Based on the results of error prevention and verification operations, continuously update the operation ticket knowledge graph and reinforcement learning model.

[0034] Based on the results of multi-dimensional error prevention verification operations, the knowledge graph of operation tickets and reinforcement learning models are continuously updated to improve the accuracy of error prevention verification for switching operations.

[0035] The system continuously collects anti-misoperation verification operation result data for each switching operation, covering operation time and equipment status. The collected anti-misoperation verification operation result data is cleaned to remove noise and outliers, and stored in a unified format. Based on new knowledge in the anti-misoperation verification operation results (such as the operation mode after the new equipment is put into operation and the non-covered violation scenarios), a knowledge graph construction algorithm is used to integrate the new knowledge into the existing operation ticket knowledge graph, improve the nodes and edges, optimize the graph structure, correct the logical defects of the original anti-misoperation rules, enhance the ability to express and reason about anti-misoperation knowledge of switching operations, and form an intelligent anti-misoperation verification system with continuous learning and self-improvement capabilities. The specific work content is as follows: Continuously collect the anti-misoperation verification operation result data for each switching operation, covering the timestamp of the operation, operation type, involved equipment objects and their status changes. Through real-time aggregation of the anti-misoperation verification operation result data, a raw data record covering the entire operation process is formed. The collected anti-misoperation verification operation result data is then cleaned, including removing outliers caused by communication interruptions or sensor malfunctions, correcting time-series errors, standardizing data formats and units of measurement, and then standardizing and encoding the cleaned anti-misoperation verification operation result data, persistently storing it in a structured form to a time-series database. Based on the collected and processed anti-error verification operation results data, we analyze the new knowledge contained therein. This new knowledge includes new operating modes formed after the commissioning of new equipment, as well as previously uncovered violation scenarios. Using knowledge graph construction algorithms, we identify new equipment entities and their relationships, and integrate them into the existing operation ticket knowledge graph in the form of nodes and edges. At the same time, by performing pattern mining on violation scenarios not covered in historical operations, we extract new constraint rules and logical relationships, correct logical defects or coverage blind spots in the original anti-error rules, and form an intelligent anti-error verification system with continuous learning and self-improvement capabilities.

[0036] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0037] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. This computer software product, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0038] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A reinforcement learning-based method for preventing errors in switching operations, incorporating knowledge graphs, characterized in that... include: Collect operation ticket data of the regional power grid, construct an operation ticket knowledge graph, and form operation ticket templates and error prevention rule base; Based on the knowledge graph of operation tickets, combined with information on multi-source heterogeneous equipment and error prevention rules, the information on multi-source heterogeneous equipment includes information on primary and secondary equipment ledgers, real-time status and wiring methods; Based on reinforcement learning algorithms, a reinforcement learning model based on error prevention strategies is constructed by defining the state space, action space, and reward function, with the power grid state and switching operation history as inputs. Train a reinforcement learning model, and generate and adjust error prevention strategies through interactive learning; Real-time acquisition of power grid topology and equipment status data, combined with operation ticket knowledge graph and reinforcement learning model to verify switching operations, and output the verification results; Based on the results of the error prevention and verification operations, the knowledge graph and reinforcement learning model of the operation ticket are continuously updated.

2. The method for preventing errors in switching operations using reinforcement learning and integrating knowledge graphs as described in claim 1, characterized in that, The construction of the operation ticket knowledge graph includes: The operation ticket data is preprocessed. Based on natural language processing technology, key elements are extracted from the preprocessed operation ticket data, including operation tasks, equipment objects, action instructions and equipment status information. A graph database is used to construct a structured operation ticket knowledge graph in the form of nodes and edges.

3. The method for preventing errors in switching operations using reinforcement learning and integrating knowledge graphs, as described in claim 2, is characterized in that... The creation of the operation ticket template and the error prevention rule base includes: The operation dependencies and state transition paths between nodes are extracted using a graph traversal algorithm. Constraints in three dimensions—operation sequence constraints, equipment state transition condition constraints, and electrical safety distance constraints—are obtained from the topology. A logical expression model is constructed based on these three dimensions of constraints, and the model is stored in a structured manner to form a rule base for preventing errors. Identify and cluster the target paths and node sequences that repeatedly appear in the operation ticket knowledge graph and are used to complete the same type of switching operation task; An operation ticket template is generated for each type of switching operation task. The operation ticket template is a standardized sequence that includes operation steps, operation objects and expected state related data. The generated anti-misoperation rule base and the operation ticket template are integrated into a unified knowledge system.

4. The method for preventing errors in switching operations using reinforcement learning and integrating knowledge graphs, as described in claim 3, is characterized in that... The constructed logical expression model includes: The operation sequence constraints include: In the formula, To judge the data, The sequence number of the current step. The sequence number of the currently executed steps, when If so, proceed with the current step normally. If the value is not equal to 0, then output a signal to stop executing the current step; The device state transition condition constraints include: Determine whether the current device status is on or off. If it is on, determine whether the sum of the current grid output power and the device power is less than the upper limit of the grid output power. If it is, output a signal that the device can be turned on; otherwise, output a signal that the device cannot be turned on. The electrical safety distance constraints include: In the formula, For current electrical safety distances, For minimum electrical safety distance, Rated power of the equipment The power of the equipment used to calibrate the minimum electrical safety distance.

5. The method for preventing errors in switching operations using reinforcement learning and integrating knowledge graphs, as described in claim 3, is characterized in that... The combination of operation ticket knowledge graph and multi-source heterogeneous device information with error prevention rules includes: Based on the semantic framework of the operation ticket knowledge graph, a unified data model is constructed for primary and secondary equipment ledgers, real-time status and wiring information. Through data adapters and interface services, information from heterogeneous devices from different data sources is collected and integrated in real time, and semantic alignment and entity association are performed based on a unified data model. The integrated information on heterogeneous devices from multiple sources is coupled with the rule base for preventing errors.

6. The method for preventing misoperation during switching operations using reinforcement learning and incorporating knowledge graphs, as described in claim 5, is characterized in that... The construction of the reinforcement learning model based on the error prevention strategy includes: The input includes the power grid status and switching operation history. The power grid status includes the real-time power grid topology and equipment status. The switching operation history includes the operation type, operation timestamp, and operation result. The state space is defined as the set of operating states of all equipment in the power grid, and the action space is the set of executable error prevention and verification decisions. The reward function is used to quantify the consequences of actions and to construct an automatic learning and optimization framework for error prevention strategies.

7. The method for preventing misoperation during switching operations using reinforcement learning and incorporating knowledge graphs as described in claim 6, characterized in that, The training reinforcement learning model, through interactive learning, generates and adjusts error prevention strategies, including: The power grid status and anti-misoperation rules are encapsulated into a reinforcement learning simulation environment. State transition logic and switching operation constraints are defined. Operation sequence samples are generated through interaction between the agent and the environment. The status, actions and reward feedback are recorded. At the same time, an experience playback pool is built. A fault prevention strategy network is constructed based on the PPO algorithm. The input is the current power grid state, and the output is the probability distribution of fault prevention verification actions during switching operations. The parameters of the fault prevention strategy network are iteratively optimized using data from the experience playback pool. The expected reward is maximized through gradient ascent, and the network gradually converges to a stable fault prevention strategy. The performance indicators of the training-based error prevention strategy, namely the interception rate of illegal operations and the improvement of operational efficiency, are verified in a simulation environment. After the error prevention performance indicators are evaluated and the system is deployed to the actual power grid dispatching system, new operation data is collected in real time to update the experience playback pool, and the adjustment cycle is set to adjust the model parameters to adapt to changes in the power grid topology.

8. The method for preventing misoperation during switching operations using reinforcement learning and incorporating knowledge graphs, as described in claim 6, is characterized in that... The error prevention strategy network constructed based on the PPO algorithm includes: A fault prevention strategy network is constructed based on the PPO algorithm. The current power grid state is encoded as a state vector and input into the fault prevention strategy network to generate the probability distribution of fault prevention verification actions for switching operations. The actions are executed according to the probability sampling. The environment simulates state transitions based on physical rules and safety constraints to generate the next state, immediate reward and termination flag. During training, batches of experience data are randomly sampled from the experience replay pool, and the parameters of the error prevention strategy network are iteratively updated using the gradient ascent algorithm. The optimization objective is to maximize the long-term cumulative expected reward. Through continuous iterative optimization, the parameters of the error prevention strategy network are converged to a stable state, forming an interpretable error prevention strategy.

9. The method for preventing misoperation during switching operations using reinforcement learning and incorporating knowledge graphs, as described in claim 8, is characterized in that... The process of verifying switching operations by combining the operation ticket knowledge graph with a reinforcement learning model, and outputting the verification results, includes: The real-time power grid status is structurally mapped to the operation tickets to be verified. The physical device status is encoded into node feature vectors through graph neural networks, and the operation ticket steps are converted into edge attributes with temporal weights to construct a state-operation joint embedding space. The state representation in the state-operation joint embedding space is input into a deep Q-network. The state-action value function calculates an expected reward value for each switching operation step to be executed in the operation ticket. Based on the expected reward value, the initial decision sequence is generated using the maximum value principle. Parallel verification is initiated using the knowledge graph of the operation ticket. Based on the historical state change patterns of the equipment, electrical coupling relationships, and predefined temporal dependency rules, logical reasoning is performed on the same operation ticket to detect potential conflicts in the initial decision and output a set of constraints. The set of constraints detected by the operation ticket knowledge graph is back-injected into the reinforcement learning model through the constraint propagation mechanism. The weight parameters of the state-action value function are dynamically adjusted according to the constraints to suppress the probability of selecting illegal actions, forcibly correct decision biases, and generate a verification report containing the conclusion of interception or passage. If it is intercepted, the violation type and correction suggestions are marked, and correction suggestions are provided through the reasoning path of the operation ticket knowledge graph.

10. The method for preventing errors in switching operations using reinforcement learning and integrating knowledge graphs, as described in claim 9, is characterized in that... Also includes: The system continuously collects the anti-misoperation verification operation result data for each switching operation, covering the operation time and equipment status. The collected anti-misoperation verification operation result data is then cleaned, standardized, and stored. Based on the new knowledge in the results of the error prevention and verification operation, a knowledge graph construction algorithm is used to integrate the new knowledge into the existing operation ticket knowledge graph and optimize the graph structure.

Images