A distribution network regulation dual constraint control method, system, device and medium

Abstract

This application relates to a dual-constraint control method, system, device, and medium for distribution network regulation. The method includes: constructing a distribution network regulation knowledge graph by integrating the process rules of the dispatching and control platform with the physical rules of the distribution automation system; constructing a regulation simulation environment integrating a physical simulation engine and a rule query interface based on the knowledge graph; training a deep reinforcement learning agent to obtain the trained deep reinforcement learning agent; and based on the trained deep reinforcement learning agent, parsing the target operation ticket into an initial action sequence, and simulating and deducing the initial action sequence in conjunction with the current real-time state of the power grid, outputting safety assessment results and optimization suggestions. This method, by constructing a knowledge graph and simulation environment, and training a deep reinforcement learning agent, achieves full-sequence simulation and intelligent evaluation of operation tickets, thereby improving the safety, compliance, and operational efficiency of distribution network remote control operations and effectively avoiding the risk of illegal operations.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, and in particular relates to a dual-constraint control method, system, device and medium for power distribution network regulation. Background Technology

[0002] As a crucial link connecting the power generation and consumption sides of the power system, the safety and accuracy of the distribution network's regulation and operation directly determine the reliability of power supply and the level of production safety. With the continuous advancement of smart grid construction, intelligent dispatching and control platforms and distribution automation systems have been gradually adopted in the field of distribution network regulation and control. These systems enable the flow and control of all distribution network regulation and control operations, as well as the collection and transmission of real-time distribution automation data, effectively improving the automation and informatization level of distribution network regulation and control, and becoming an important technical support for distribution network regulation and control work. In actual distribution network regulation and control work, remote control operation is the core execution link, encompassing multiple major operational tasks such as daily power outage and restoration operations, complex maintenance operations during spring and autumn inspections, and equipment startup and power supply. These operational tasks are often characterized by a fast execution pace, numerous parallel operations, and complex equipment relationships, placing extremely high demands on the accuracy and standardization of dispatching operations.

[0003] However, the current remote control operation of distribution network dispatching still lacks comprehensive and effective error prevention and verification technologies. The entire process of drafting, reviewing, issuing, and distributing dispatching operation tickets has not yet achieved systematic error prevention and foolproof control, posing significant safety hazards to power grid operation. Furthermore, during periods of heavy maintenance workload, dispatchers need to perform multiple power outage and restoration operations simultaneously. When dealing with complex maintenance, power restoration, and other critical operations, human error can easily lead to violations such as skipping steps or issuing further operation instructions without verifying the actual status of equipment. Such errors can not only cause power grid production accidents and power outage losses but also threaten personal safety and reduce the efficiency of distribution network dispatching.

[0004] As an illustration, while some distribution network control systems currently have simple operation verification functions, these often only verify the static rules of a single system. They fail to achieve integrated modeling and data sharing between the dispatching and control platform and the distribution automation system, making it impossible to establish a dual-constraint error prevention system that considers both process rules and power grid physical rules. Furthermore, the parsing of operation tickets often relies on manual or simple text processing methods, resulting in insufficient entity recognition accuracy. This makes it difficult to accurately interpret core information such as the operating equipment and operation type in operation instructions, easily leading to parsing errors due to semantic ambiguity and non-standard formats, creating potential risks for subsequent operations. In addition, existing safety assessments for remote control operations are mostly static, single-point verifications, lacking the ability to dynamically simulate and extrapolate based on the real-time operating status of the power grid. This makes it impossible to make full-process safety predictions for the action sequences of operation tickets, and also makes it difficult to provide accurate optimization suggestions for risky operation sequences. Summary of the Invention

[0005] Based on this, it is necessary to provide a dual-constraint control method, system, equipment, and medium for distribution network regulation to address the aforementioned technical problems. This aims to improve the intelligence and dynamic predictability of remote control operation for distribution network regulation in preventing errors, and to enhance the accuracy of operation ticket parsing and the safety management capabilities for complex operation tasks.

[0006] Firstly, this application provides a dual-constraint control method for distribution network regulation, including:

[0007] Based on the process rule data of the dispatch and control platform and the power grid physical rule data of the distribution automation system, a structured knowledge base is generated, including physical entity nodes, process entity nodes and cross-domain relationships between physical entity nodes and process entity nodes. Based on the structured knowledge base, a distribution network control knowledge graph is constructed.

[0008] Based on the distribution network control knowledge graph, a control simulation environment is constructed. The control simulation environment is used to simulate the state evolution of the distribution network by integrating the power grid physical simulation engine and the rule query interface of the distribution network control knowledge graph, calculate the dynamic safety margin, and calculate the comprehensive reward signal according to the static rules and dynamic safety margin in the distribution network control knowledge graph.

[0009] In a controlled simulation environment, a deep reinforcement learning agent is trained to obtain the trained deep reinforcement learning agent. The deep reinforcement learning agent takes the state representation output by the controlled simulation environment as input, analyzes it through a preset neural network, outputs the probability distribution of remote control operation actions, selects the target remote control operation action according to the probability distribution, inputs the target remote control operation action into the controlled simulation environment, and updates the network parameters of the preset neural network with the goal of maximizing the cumulative reward. The cumulative reward is obtained by accumulating the comprehensive reward signal.

[0010] The target operation ticket is obtained. Based on the trained deep reinforcement learning agent, the target operation ticket is parsed into an initial action sequence. The initial action sequence is then simulated and deduced in combination with the current real-time state of the power grid, and the safety assessment results and optimization suggestions are output.

[0011] In one embodiment, the process rule data includes operation ticket business data. Based on the process rule data of the scheduling and control platform and the power grid physical rule data of the distribution automation system, a structured knowledge base is generated, including physical entity nodes, process entity nodes, and cross-domain relationships between physical entity nodes and process entity nodes, including:

[0012] The power grid physical rule data is obtained from the power distribution automation system, and the power grid equipment ledger and power grid topology connection data are obtained from the power distribution production management system. Based on the power grid physical rule data, power grid equipment ledger and power grid topology connection data, physical entity nodes are generated and corresponding static attributes are configured to establish the electrical topology relationship between each physical entity node.

[0013] Operation ticket business data is extracted from the scheduling and control platform, the operation ticket business data is structured and parsed to generate operation ticket entity nodes and operation step entity nodes, the operation ticket entity nodes and operation step entity nodes are identified as process entity nodes, the hierarchical relationship between operation ticket entity nodes and operation step entity nodes is established, and the sequential dependency relationship between each operation step entity node is established.

[0014] By using a pre-defined natural language processing model for the power sector, entity matching processing is performed on the operation instructions corresponding to the entity nodes of the operation steps to establish the operation association between the entity nodes of the operation steps and the physical entity nodes.

[0015] Combining the preset power dispatch expert rules and power grid physical constraints, establish physical precondition relationships and physical poststate relationships between operation step entity nodes and physical entity nodes, and regard the operation association relationship, physical precondition relationship, and physical poststate relationship as the cross-domain association relationship between physical entity nodes and process entity nodes.

[0016] Integrate physical entity nodes, process entity nodes, electrical topology relationships, membership relationships, sequential dependencies, and cross-domain associations to generate a structured knowledge base.

[0017] In one embodiment, a control simulation environment is constructed based on a distribution network control knowledge graph, including:

[0018] The power grid topology and equipment electrical parameters are extracted from the distribution network control knowledge graph, and a power grid node branch model is constructed based on the power grid topology and equipment electrical parameters;

[0019] The power grid node branch model is used as the simulation carrier of the power grid physical simulation engine. The power flow calculation engine and the dynamic transient simulator are integrated to build a power grid physical simulation engine for simulating the changes in the electrical state of the distribution network. Among them, the power flow calculation engine is used to calculate the steady-state operation state of the distribution network, and the dynamic transient simulator is used to simulate the transient process of the distribution network after the execution of remote control operation actions.

[0020] A two-way data synchronization mechanism is established between the power grid physical simulation engine and the distribution network control knowledge graph. The two-way data synchronization mechanism is used to write the dynamic operation data output by the power grid physical simulation engine back to the distribution network control knowledge graph, update the dynamic operation attributes of physical entity nodes, and synchronize the static attributes and static rules of physical entity nodes in the distribution network control knowledge graph to the power grid physical simulation engine as constraints for the power grid physical simulation engine to perform simulation calculations.

[0021] A state aggregation module is constructed to acquire electrical operation data from the simulation results output by the power grid physical simulation engine after simulating state evolution based on remote control operation actions. The electrical operation data is numerically characterized to obtain numerical features. Based on the operation object corresponding to the remote control operation action, a local subgraph including the operation object and associated nodes is extracted from the distribution network control knowledge graph. The local subgraph is then coded with graph structure features to obtain graph structure features. The numerical features and graph structure features are weighted and fused to obtain the state representation.

[0022] A reward calculation module is constructed to obtain a preset optimal number of operation steps. Based on the static rules in the distribution network control knowledge graph, the current remote control operation is checked for rule compliance, and the results are converted into rule compliance indicators. The remote control operation is input into the power grid physical simulation engine, which performs state evolution simulation to obtain simulation results. Based on the simulation results, the dynamic safety margin is calculated and converted into a dynamic safety margin indicator. The operation efficiency indicator is determined based on the ratio of the currently executed operation steps to the preset optimal number of operation steps. Finally, the rule compliance indicator, dynamic safety margin indicator, and operation efficiency indicator are weighted and summed to calculate the comprehensive reward signal.

[0023] The power grid physical simulation engine, two-way data synchronization mechanism, state aggregation module, and reward calculation module are integrated and configured. The rule query interface of the distribution network control knowledge graph is called as the interaction channel, and data interaction is carried out through a preset communication protocol to build a control simulation environment.

[0024] In one embodiment, a deep reinforcement learning agent is trained in a controlled simulation environment to obtain a trained deep reinforcement learning agent, including:

[0025] S1. Construct a deep reinforcement learning agent, which includes a pre-defined neural network, a target policy network, and a target value network. The pre-defined neural network includes a numerical feature encoder, a graph feature encoder, a graph attention fusion layer, a policy network, and a value network. The numerical feature encoder encodes the numerical features in the state representation, obtaining encoded numerical features. The graph feature encoder encodes the graph structure features in the state representation, obtaining encoded graph structure features. The graph attention fusion layer fuses the encoded numerical features and the encoded graph structure features to obtain a fused feature vector. The policy network takes the fused feature vector as input and outputs the probability distribution of remote control actions. The value network takes the fused feature vector as input and outputs a state value estimate. The target policy network and target value network are obtained by replicating the network structures and parameters of the policy network and value network, respectively.

[0026] S2. Initialize the experience replay buffer, set the preset batch size, fixed update interval and training convergence threshold;

[0027] S3. Based on the deep reinforcement learning agent, conduct agent interaction training in a controlled simulation environment to obtain the trained deep reinforcement learning agent. The agent interaction training corresponds to the following steps:

[0028] S31. Call the preset neural network to process the current state representation of the control simulation environment output, and output the probability distribution of remote control operation actions.

[0029] S32. Add exploration noise to the probability distribution of remote control operation actions to obtain the added probability distribution. Select the target remote control operation action according to the added probability distribution, input the target remote control operation action into the control simulation environment, and obtain the comprehensive reward signal and the next state representation fed back by the control simulation environment. Construct an experience tuple including the state representation, the target remote control operation action, the comprehensive reward signal and the next state representation, and store the experience tuple in the experience playback buffer.

[0030] S33. When the number of experience tuples stored in the experience replay buffer reaches the preset batch size, the training experience tuples are sampled from the experience replay buffer, the target value is calculated through the target policy network and the target value network, and a value network loss function is constructed. With the goal of maximizing the cumulative reward, the value network loss function is minimized to obtain the updated first network parameters. The updated first network parameters are assigned to the value network to obtain the optimized value network. Here, the cumulative reward is the accumulated value of the comprehensive reward signal along the training time sequence.

[0031] S34. After each fixed update interval is reached after executing step S32, a policy network loss function is constructed. With the goal of maximizing the cumulative reward, the policy network loss function is minimized to obtain the updated second network parameters. The updated second network parameters are then assigned to the policy network to obtain the optimized policy network. Based on a preset soft update coefficient, the parameters of the optimized policy network are synchronized to the target policy network using a soft update method, and the parameters of the optimized value network are synchronized to the target value network to obtain the optimized target policy network and the optimized target value network. The optimized target policy network and the optimized target value network are used to calculate the target value for the next round.

[0032] S35. Determine whether the target remote control operation action output by the deep reinforcement learning agent meets the preset operation target. Count the number of operations that meet the preset operation target. Calculate the task success rate during training based on the ratio of the number of operations to the total number of interactions of the deep reinforcement learning agent. Calculate the value loss value corresponding to the value network loss function. Repeat steps S32 to S34. When the task success rate reaches the training convergence threshold and the value network loss value is less than or equal to the preset loss threshold, stop training and obtain the trained deep reinforcement learning agent.

[0033] In one embodiment, based on a trained deep reinforcement learning agent, the target operation ticket is parsed into an initial action sequence, and a full-sequence simulation is performed on the initial action sequence in conjunction with the current real-time state of the power grid. The simulation outputs a safety assessment result and optimization suggestions, including:

[0034] The target operation ticket is parsed in a structured manner to extract the operation object and operation type data, and an initial action sequence is generated.

[0035] Read the current real-time status data of the power grid from the distribution network control knowledge graph, load the current real-time status data of the power grid into the control simulation environment, and obtain the real-time control simulation environment;

[0036] Input the initial action sequence into the trained deep reinforcement learning agent and output the probability distribution of the remote control operation action corresponding to each action in the initial action sequence.

[0037] The probability distribution of the remote control operation actions corresponding to each action is input into the real-time control simulation environment. The initial action sequence is subjected to full sequence simulation and deduction. The changes in power grid operation indicators and safety margins during the simulation process are recorded to obtain real-time simulation results. Among them, the power grid operation indicators include node voltage amplitude, branch load rate, active power, reactive power and system frequency.

[0038] The safety assessment results are generated based on the real-time simulation results. The corresponding optimization operation sequence is generated based on the safety assessment results. The optimization suggestions are obtained by combining the optimization operation sequence, and the safety assessment results and optimization suggestions are output.

[0039] In one embodiment, the formula for calculating the dynamic safety margin is:

[0040]

[0041] in, Indicates the first The dynamic safety margin of the distribution network, with a value range of 0-1; Indicates the number of dimensions of the core electrical parameters; Indicates the first The first core electrical parameter Step-by-step actual running value; Indicates the first Lower limit of safety threshold for core electrical parameters; Indicates the first Upper limit of safety threshold for core electrical parameters; Indicates the first The center bias coefficient of the core electrical parameters; Indicates the first Weighting index of core electrical parameters.

[0042] In one embodiment, the mathematical expression for the policy network loss function is:

[0043]

[0044] in, This represents the network loss value of the policy; Represents the set of learnable parameters of the policy network; Indicates the experience replay buffer The expected value of the sampled training experience tuples is calculated. The training experience tuples include the first... Step-by-step power distribution network status representation Target remote control operation actions Comprehensive reward signal and execution The next Step-by-step power distribution network status representation ; The parameter is The strategy network in the distribution network state Output remote control operation actions The probability of; The advantage function is used to characterize the value advantage of the current remote control action compared to the average remote control action. This represents the entropy regularization coefficient; Entropy represents the probability distribution of actions output by the policy network.

[0045] Secondly, this application also provides a dual-constraint control system for distribution network regulation, comprising:

[0046] The knowledge graph construction module is used to generate a structured knowledge base based on the process rule data of the scheduling and control platform and the power grid physical rule data of the distribution automation system. This base includes physical entity nodes, process entity nodes, and cross-domain relationships between physical entity nodes and process entity nodes. Based on the structured knowledge base, a distribution network control knowledge graph is constructed.

[0047] The simulation environment construction module is used to build a control simulation environment based on the distribution network control knowledge graph. The control simulation environment is used to simulate the state evolution of the distribution network by integrating the power grid physical simulation engine and the rule query interface of the distribution network control knowledge graph, calculate the dynamic safety margin, and calculate the comprehensive reward signal according to the static rules and dynamic safety margin in the distribution network control knowledge graph.

[0048] The reinforcement learning training module is used to train a deep reinforcement learning agent in a controlled simulation environment, resulting in a trained deep reinforcement learning agent. The deep reinforcement learning agent takes the state representation output by the controlled simulation environment as input, analyzes it through a pre-defined neural network, outputs a probability distribution of remote control operation actions, selects a target remote control operation action based on the probability distribution, inputs the target remote control operation action into the controlled simulation environment, and updates the network parameters of the pre-defined neural network with the goal of maximizing cumulative reward. The cumulative reward is obtained by accumulating the comprehensive reward signal.

[0049] The operation safety verification module is used to obtain the target operation ticket. Based on the trained deep reinforcement learning agent, the target operation ticket is parsed into an initial action sequence. Combined with the current real-time state of the power grid, the initial action sequence is simulated and deduced in full sequence, and the safety assessment results and optimization suggestions are output.

[0050] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the first aspect.

[0051] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the first aspect.

[0052] The aforementioned dual-constraint control method, system, equipment, and medium for distribution network regulation first integrates multi-source data from the dispatching and control platform and the distribution automation system to generate a structured knowledge base and construct a distribution network regulation knowledge graph, laying the foundation for subsequent dual-constraint control of distribution network regulation. Second, a regulation simulation environment is constructed based on the knowledge graph, integrating a power grid physical simulation engine and a rule query interface. This enables the simulation of distribution network state evolution and the calculation of comprehensive reward signals, overcoming the shortcomings of traditional single static rule verification. Furthermore, a deep reinforcement learning agent is trained in the simulation environment. By pre-setting a neural network to analyze the state and update network parameters, the intelligence and safety of distribution network remote control operation decisions are improved. Finally, the target operation ticket is analyzed and combined with the real-time state of the power grid for full-sequence simulation deduction, outputting safety assessment results and optimization suggestions. This achieves dynamic error prevention and intelligent evaluation of the entire process of distribution network regulation operation tickets, ensuring the inherent safety of distribution network regulation operation. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 A flowchart of a dual-constraint control method for distribution network regulation is provided as an exemplary embodiment of the present invention;

[0055] Figure 2 A flowchart illustrating a method for generating security assessment results and optimization suggestions, provided as an exemplary embodiment of the present invention;

[0056] Figure 3 This is a schematic diagram of a dual-constraint control system for power distribution network regulation, provided as an exemplary embodiment of the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0058] In one embodiment, such as Figure 1 As shown, a dual-constraint control method for distribution network regulation is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0059] S101: Based on the process rule data of the dispatch and control platform and the power grid physical rule data of the distribution automation system, a structured knowledge base is generated, including physical entity nodes, process entity nodes and cross-domain relationships between physical entity nodes and process entity nodes. Based on the structured knowledge base, a distribution network control knowledge graph is constructed.

[0060] Specifically, the process rule data of the dispatch and control platform can include process-related rules such as the circulation specifications of distribution network control operation tickets and the requirements for the sequence of operation steps, while the power grid physical rule data of the distribution automation system can include physical rules such as power grid equipment parameters, topology connection relationships, and electrical operation constraints. Based on the core features extracted from the above two types of data, the attribute information of physical entity nodes and process entity nodes can be defined, and cross-domain association relationships between physical entity nodes and process entity nodes can be mined and established to generate a structured knowledge base. This knowledge base can realize the structured storage and management of dual-constraint related data. Based on this structured knowledge base, a distribution network control knowledge graph can be constructed. The graph structure characteristics of the knowledge graph can be further utilized to intuitively and efficiently represent the attributes of various entity nodes and the association relationships between nodes, realizing the visual modeling and rapid query and retrieval of dual-constraint knowledge of distribution network control. This provides a unified and complete knowledge carrier for the subsequent construction of control simulation environment and the training of deep reinforcement learning agents.

[0061] S102: Based on the distribution network control knowledge graph, a control simulation environment is constructed. The control simulation environment is used to simulate the state evolution of the distribution network by integrating the power grid physical simulation engine and the rule query interface of the distribution network control knowledge graph, calculate the dynamic safety margin, and calculate the comprehensive reward signal according to the static rules and dynamic safety margin in the distribution network control knowledge graph.

[0062] Specifically, in constructing this control simulation environment, the power grid physical simulation engine possesses the ability to simulate changes in the electrical state of the distribution network, while the rule query interface enables real-time retrieval and verification of static rules in the knowledge graph. By integrating these two, the simulation environment can simultaneously possess the ability to simulate physical state evolution and verify rule compliance. Illustratively, upon receiving a remote control operation, the control simulation environment can first simulate the evolution of the distribution network's operating state using the power grid physical simulation engine, and then calculate a dynamic safety margin based on the physical constraints of the power grid operation. This dynamic safety margin accurately reflects the physical safety level of the distribution network after the operation is executed. Furthermore, it can retrieve static rules from the knowledge graph through the rule query interface to determine the compliance of the remote control operation. Subsequently, combining the verification results of the static rules with the dynamic safety margin, a comprehensive reward signal can be obtained through quantitative fusion calculation. This signal serves as the reward and punishment basis for subsequent deep reinforcement learning agent training, realizing the quantification of rewards and punishments across dual constraint dimensions, and providing guidance for the agent's subsequent policy learning.

[0063] S103: In a controlled simulation environment, a deep reinforcement learning agent is trained to obtain a trained deep reinforcement learning agent. The deep reinforcement learning agent takes the state representation output by the controlled simulation environment as input, analyzes it through a preset neural network, outputs the probability distribution of remote control operation actions, selects the target remote control operation action according to the probability distribution, inputs the target remote control operation action into the controlled simulation environment, and updates the network parameters of the preset neural network with the goal of maximizing the cumulative reward. The cumulative reward is obtained by accumulating the comprehensive reward signal.

[0064] Specifically, training a deep reinforcement learning agent in a control simulation environment leverages the trial-and-error learning characteristics of reinforcement learning. This allows the agent to learn safe operational decision-making strategies that comply with the dual constraints of distribution network control in a simulation environment free from actual operational risks, avoiding the operational risks associated with direct training in a real distribution network control system. The deep reinforcement learning agent takes the state representation output from the control simulation environment as input. This state representation integrates the physical operating state and rule-constrained state of the distribution network. The agent can use a pre-set neural network to analyze this state representation, extracting core state feature information, and outputting a probability distribution of remote control actions based on the feature analysis results. This probability distribution quantitatively characterizes the safety and feasibility of each remote control action under the current grid state. The agent can also select target remote control operation actions based on the probability distribution and input them into the control simulation environment. After the simulation environment executes the action, it can provide a corresponding comprehensive reward signal. The agent accumulates the comprehensive reward signals obtained from each step of the operation to obtain the cumulative reward. With the goal of maximizing the cumulative reward, the agent can iteratively update the network parameters of the preset neural network through parameter update algorithms such as gradient descent, so that the agent can continuously optimize the operation action selection strategy. The above training process is repeated until the agent's operation decision-making ability reaches a stable state, and a deep reinforcement learning agent is obtained after training. This agent has the ability to determine the safety of operation actions based on the real-time status of the distribution network, and can provide decision support for the intelligent evaluation of subsequent operation tickets.

[0065] S104: Obtain the target operation ticket. Based on the trained deep reinforcement learning agent, parse the target operation ticket into an initial action sequence. Combine the current real-time state of the power grid to perform a full-sequence simulation of the initial action sequence and output the safety assessment results and optimization suggestions.

[0066] Specifically, by combining the trained deep reinforcement learning agent with the actual distribution network control operation ticket management, dual-constraint error prevention and verification of operation tickets can be achieved. Illustratively, the target operation ticket drafted by the dispatcher can be obtained from the dispatch control platform first. Relying on the parsing capabilities of the trained deep reinforcement learning agent, the operation ticket in natural language form can be converted into a machine-recognizable initial action sequence, achieving structured parsing of the operation ticket. Secondly, real-time operating status data of the current power grid can be retrieved from the distribution network control knowledge graph or the distribution automation system. This real-time status is used as the initial state for simulation. Combined with the initial action sequence, a full-sequence simulation is conducted in the control simulation environment. During the simulation, the deep reinforcement learning agent can perform a safety feasibility assessment of each step in the initial action sequence and simulate the changes in the distribution network state after each step is executed through the control simulation environment. Based on the results of the full-sequence simulation, the overall safety level, risky operation steps, and risk causes of the initial action sequence can be further quantitatively analyzed, generating safety assessment results including safety level and risk point location. Furthermore, for the identified risk points, corresponding operation sequence optimization suggestions can be generated and output based on the dual constraint requirements. Through the above process, not only can dynamic error prevention and verification of the drafting and review of operation tickets be achieved, but also problems such as illegal step-by-step operation and operation without verification of equipment status can be accurately identified, thereby reducing human error from the source and ensuring the inherent safety of remote control operation of distribution network.

[0067] The aforementioned method effectively addresses the problems of fragmented process and physical constraints and the lack of a dual anti-error system in traditional methods by constructing a cross-domain knowledge graph through the integration of process rules from the dispatch and control platform and physical rules from the distribution automation system. Secondly, by building a control simulation environment based on the knowledge graph and calculating dynamic safety margins, it overcomes the limitation of traditional static verification in predicting the cumulative risks of operation sequences. Furthermore, training a deep reinforcement learning agent in the simulation environment effectively solves the problems of reliance on manual operation ticket parsing and low entity recognition accuracy in traditional methods. It also enables decision-making based on the agent's output action probability distribution, overcoming the limitation of traditional anti-error verification lacking intelligent decision-making capabilities. Finally, by parsing the target operation ticket and performing full-sequence simulation, the method outputs safety assessment results and optimization suggestions, achieving dynamic closed-loop management of remote control operation risks and improving the safety, accuracy, and intelligence level of distribution network control.

[0068] In one embodiment, based on the process rule data of the scheduling and control platform and the power grid physical rule data of the distribution automation system, a structured knowledge base is generated, including physical entity nodes, process entity nodes, and cross-domain relationships between physical entity nodes and process entity nodes, including:

[0069] The power grid physical rule data is obtained from the power distribution automation system, and the power grid equipment ledger and power grid topology connection data are obtained from the power distribution production management system. Based on the power grid physical rule data, power grid equipment ledger and power grid topology connection data, physical entity nodes are generated and corresponding static attributes are configured to establish the electrical topology relationship between each physical entity node.

[0070] Operation ticket business data is extracted from the scheduling and control platform, the operation ticket business data is structured and parsed to generate operation ticket entity nodes and operation step entity nodes, the operation ticket entity nodes and operation step entity nodes are identified as process entity nodes, the hierarchical relationship between operation ticket entity nodes and operation step entity nodes is established, and the sequential dependency relationship between each operation step entity node is established.

[0071] By using a pre-defined natural language processing model for the power sector, entity matching processing is performed on the operation instructions corresponding to the entity nodes of the operation steps to establish the operation association between the entity nodes of the operation steps and the physical entity nodes.

[0072] Combining the preset power dispatch expert rules and power grid physical constraints, establish physical precondition relationships and physical poststate relationships between operation step entity nodes and physical entity nodes, and regard the operation association relationship, physical precondition relationship, and physical poststate relationship as the cross-domain association relationship between physical entity nodes and process entity nodes.

[0073] Integrate physical entity nodes, process entity nodes, electrical topology relationships, membership relationships, sequential dependencies, and cross-domain associations to generate a structured knowledge base.

[0074] Specifically, in this embodiment, the process rule data may include operation ticket business data, which covers all business information related to the circulation of distribution network control operation tickets, such as operation task name, operation time, operation step description, operation execution subject, and review subject. The power grid physical rule data collected from the distribution automation system may include equipment electrical constraint parameters, power grid operating physical laws, fault isolation logic, etc. Simultaneously, power grid equipment ledgers and power grid topology connection data can be retrieved from the distribution production management system. The power grid equipment ledger contains basic equipment information such as equipment model, rated voltage, rated current, rated power, installation location, and commissioning time. The power grid topology connection data can be presented in the form of a node-branch association table, recording the connection relationships between each equipment node and branch attributes. Based on the above data, an ontology-based approach can be used to define a category system for physical entity nodes, including core categories such as transformer nodes, switch nodes, line nodes, and bus nodes. Each category corresponds to a preset set of static attributes. A data mapping algorithm is used to fill the specific information in the power grid equipment ledger into the static attribute fields of the corresponding physical entity nodes, thereby enabling the attribute configuration of physical entity nodes. When establishing the electrical topology relationship between each physical entity node, a node adjacency matrix can be constructed based on the power grid topology connection data. In the matrix, an element with a value of 1 indicates that there is a direct electrical connection between the two corresponding nodes, and a value of 0 indicates that there is no direct electrical connection. By traversing the branch connection information in the node-branch association table, the value of each element in the adjacency matrix can be determined, thereby representing the electrical connection logic between physical entity nodes and ensuring that the topology relationship is consistent with the actual power grid structure.

[0075] Specifically, operation ticket business data can be extracted from the historical and real-time flow databases of the dispatch and control platform, and then structuredly parsed using the grammatical rules and semantic features of the power operation language. For example, a word segmentation algorithm based on a custom dictionary is first used to segment the operation ticket business data. The custom dictionary includes power industry-specific terminology, standardized descriptions of equipment names, and keywords for operation actions, which can avoid ambiguity caused by term splitting during the segmentation process. Subsequently, a named entity recognition model is used to extract key information such as operation ticket number, operation task type, operation object, and operation action. The extracted overall information of the operation ticket is encapsulated into operation ticket entity nodes. Each operation ticket entity node can contain attributes such as ticket number, task name, creation time, and status. Furthermore, each independent operation instruction in the operation ticket can be used as an operation step entity node. Each operation step entity node contains attributes such as step number, operation description, and operation type, generating process entity nodes. The operation ticket entity nodes and operation step entity nodes together constitute a set of process entity nodes. When establishing the hierarchical relationship between operation ticket entity nodes and operation step entity nodes, a key-value mapping mechanism can be used. The unique identifier of the operation ticket entity node is used as the key, and a list of unique identifiers of all corresponding operation step entity nodes is used as the value. This mapping relationship establishes the association between the operation ticket and its corresponding operation step. Furthermore, a time-series index value can be assigned to each operation step entity node. The index value increments sequentially according to the execution order of the operation steps in the operation ticket. The order of execution of the operation steps is represented by the size of the index values. Simultaneously, directed edges are used to record the dependencies between adjacent operation step entity nodes, ensuring that the execution order of the operation steps conforms to the distribution network control specifications, thereby establishing the sequential dependencies between each operation step entity node.

[0076] Furthermore, a pre-defined natural language processing (NLP) model for the power sector can be used to perform entity matching processing on the operation instructions corresponding to the entity nodes of the operation steps. This NLP model can be based on the RoBERTa model architecture. During the pre-training phase, it is trained using a power operation corpus, which can contain text data such as historical operation tickets, dispatching procedures, and equipment operation manuals. Pre-training allows the model to master the semantic features and expression habits of power sector language. In the fine-tuning phase, it can be specifically trained for entity recognition tasks within operation instructions. The output layer is labeled with classification tags such as equipment entities, operation action entities, and status entities to ensure the model can accurately identify key entities in the operation instructions. During entity matching, the operation instructions are first pre-processed to remove redundant spaces, punctuation marks, and other irrelevant characters, standardizing the instructions to a uniform format. Then, the trained model identifies the operation object (i.e., the target equipment entity) in the operation instructions, extracts the key attribute features of the equipment entity, such as equipment number, voltage level, and affiliated substation, and compares these attribute features with the static attributes of all physical entity nodes using a similarity calculation algorithm such as cosine similarity. When the similarity value is greater than a preset threshold, the two are determined to be the same entity. An operation association relationship is established between the operation step entity node and the physical entity node. The association relationship is stored in the form of a triple (operation step entity node ID, operation association, physical entity node ID).

[0077] Specifically, the pre-defined power dispatch expert rules can be formulated based on long-term practical experience in distribution network control and industry standards, including operational ticket execution procedures, equipment operation sequence requirements, and fault handling operation guidelines, such as "before closing the switch, the corresponding line must be free of grounding faults." Meanwhile, the physical constraints of the power grid can be determined based on the physical characteristics of the equipment and the operating rules of the power grid, including the rated voltage, rated current, and rated power limits of the equipment, the branch transmission power threshold, and the allowable fluctuation range of node voltage. Furthermore, the physical precondition relationship characterizes the state requirements that the corresponding physical entity node must meet before executing a certain operation step. For example, for each operation step entity node, according to its operation type and operation object, the corresponding preconditions are matched from the power dispatch expert rules, and the quantitative indicators are clarified in combination with the power grid physical constraints. For example, the physical preconditions of a certain closing operation step are "the current operating state of the operation object equipment is cold standby" and "the grounding resistance of the corresponding line is greater than the preset threshold". By associating the preconditions with the static attributes and operating status data of the physical entity node, a physical precondition relationship can be formed, and it can be stored in the form of a triple (operation step entity node ID, physical precondition, physical entity node state requirement). The physical post-state relationship characterizes the target state that the corresponding physical entity node should reach after executing a certain operation step. For example, based on the state transition logic in the power grid physical rule data and combined with the physical impact of the operation action, the target values ​​such as the operating state and electrical parameters of the physical entity node after the operation step is executed can be determined. For instance, after the tripping operation step is executed, the state of the corresponding switch node changes from "operating" to "tripping", and the transmission power of the corresponding branch becomes 0. Associating this target state with the operation step entity node and the physical entity node can form a physical post-state relationship, which can also be stored in the form of triples. The above-mentioned operation association relationship, physical precondition relationship, and physical post-state relationship together constitute the cross-domain association relationship between physical entity nodes and process entity nodes, which can realize the deep association between the two types of entity nodes.

[0078] Finally, based on physical entity nodes, process entity nodes, electrical topology relationships, membership relationships, sequential dependencies, and cross-domain associations, a graph data model can be adopted as the underlying storage architecture. Physical entity nodes and process entity nodes are the vertices of the graph, and various relationships are the edges. Redundant, conflicting, and missing data are removed using data cleaning algorithms to construct a structured knowledge base. Redundant data includes duplicate entity nodes and relationship records; conflicting data includes data with inconsistent attribute values ​​and contradictory relationship logic; and missing data can be reasonably supplemented based on existing data using an association completion algorithm. This structured knowledge base has a standardized data format and clear association logic, supporting rapid querying, modification, and expansion of entity attributes and relationships. It achieves structured storage and unified management of knowledge related to physical and process constraints in the distribution network control process.

[0079] In one embodiment, a control simulation environment is constructed based on a distribution network control knowledge graph, including:

[0080] The power grid topology and equipment electrical parameters are extracted from the distribution network control knowledge graph, and a power grid node branch model is constructed based on the power grid topology and equipment electrical parameters;

[0081] The power grid node branch model is used as the simulation carrier of the power grid physical simulation engine. The power flow calculation engine and the dynamic transient simulator are integrated to build a power grid physical simulation engine for simulating the changes in the electrical state of the distribution network. Among them, the power flow calculation engine is used to calculate the steady-state operation state of the distribution network, and the dynamic transient simulator is used to simulate the transient process of the distribution network after the execution of remote control operation actions.

[0082] A two-way data synchronization mechanism is established between the power grid physical simulation engine and the distribution network control knowledge graph. The two-way data synchronization mechanism is used to write the dynamic operation data output by the power grid physical simulation engine back to the distribution network control knowledge graph, update the dynamic operation attributes of physical entity nodes, and synchronize the static attributes and static rules of physical entity nodes in the distribution network control knowledge graph to the power grid physical simulation engine as constraints for the power grid physical simulation engine to perform simulation calculations.

[0083] A state aggregation module is constructed. This module is used to acquire electrical operation data from the simulation results output by the power grid physical simulation engine after simulating state evolution based on remote control operation actions. The electrical operation data is numerically feature-encoded to obtain numerical features. Based on the operation object corresponding to the remote control operation action, a local subgraph including the operation object and associated nodes is extracted from the distribution network control knowledge graph. The local subgraph is then graph structure feature-encoded to obtain graph structure features. The numerical features and graph structure features are weighted and fused to obtain the state representation.

[0084] A reward calculation module is constructed. This module is used to obtain the preset optimal number of operation steps, perform rule compliance verification on the current remote control operation based on the static rules in the distribution network control knowledge graph, obtain the verification result, and convert the verification result into a rule compliance index. The remote control operation is input into the power grid physical simulation engine, and the simulation results are obtained by performing state evolution simulation through the power grid physical simulation engine. The dynamic safety margin is calculated based on the simulation results and converted into a dynamic safety margin index. The operation efficiency index is determined based on the ratio of the currently executed operation steps to the preset optimal number of operation steps. The rule compliance index, dynamic safety margin index, and operation efficiency index are weighted and summed to calculate the comprehensive reward signal.

[0085] The power grid physical simulation engine, two-way data synchronization mechanism, state aggregation module, and reward calculation module are integrated and configured. The rule query interface of the distribution network control knowledge graph is called as the interaction channel, and data interaction is carried out through a preset communication protocol to build a control simulation environment.

[0086] Specifically, the power grid topology stored in the distribution network control knowledge graph can be represented as graph nodes and edges. Based on the graph query language of the knowledge graph, physical entity nodes such as transformers, switches, lines, and buses, as well as the electrical connection edges between nodes, can be filtered out. Furthermore, the electrical parameters of the equipment associated with each physical entity node can be extracted, including parameters such as equipment impedance, admittance, rated capacity, turns ratio, and voltage regulation method. Subsequently, when constructing the power grid node branch model, standard node branch modeling methods from the field of power system analysis can be adopted. The distribution network is abstracted into a set of nodes and a set of branches, with the node admittance matrix serving as the core mathematical representation of the model. The diagonal elements of the node admittance matrix represent the self-admittance of the corresponding node, while the off-diagonal elements represent the mutual admittance between nodes. Both self-admittance and mutual admittance can be calculated based on the extracted equipment electrical parameters. This matrix can then accurately describe the electrical coupling relationships between the nodes of the distribution network, providing a foundation for the numerical calculations of the subsequent power grid physical simulation engine.

[0087] Specifically, the power flow calculation engine can use the Newton-Raphson method as its core calculation algorithm. This algorithm can achieve numerical calculation of the steady-state operation of the distribution network by iteratively solving the node power balance equations. The calculation results include steady-state electrical quantities such as voltage amplitude, voltage phase angle, active power, reactive power, and current of each node. Its core principle is to construct a Jacobian matrix based on the node admittance matrix and iterate step by step through the correction equations until the power balance accuracy requirements are met. The dynamic transient simulator can use the implicit trapezoidal integral method as the differential equation solving algorithm to simulate the transient process of the distribution network after the execution of remote control operation. Its core is to establish a set of electromechanical or electromagnetic transient differential equations of the distribution network. The set of equations can be constructed based on the grid node branch model and the transient mathematical model of the equipment, such as generator rotor motion equations, excitation system regulation equations, speed regulation system equations, etc. By solving the set of equations through numerical integration, the dynamic change curve of electrical quantities after the execution of the operation can be obtained, reproducing the entire process of the distribution network transitioning from transient to new steady state. By integrating the power flow calculation engine with the dynamic transient simulator, the constructed power grid physical simulation engine can realize the full-process electrical state evolution simulation of "steady-state initial value calculation - transient process simulation - new steady-state result output", covering all electrical state change scenarios after remote control operation of the distribution network.

[0088] Specifically, the bidirectional data synchronization mechanism between the power grid physical simulation engine and the distribution network control knowledge graph is the core technical means to achieve collaboration between the simulation environment and the knowledge graph. This bidirectional data synchronization mechanism can employ a combination of event-triggered and timed-triggered synchronization modes. The event-triggered condition is that the power grid physical simulation engine executes a remote control operation and completes a state evolution simulation. The timed-triggered frequency can be set according to the dynamic changes in the distribution network's operating state. During the synchronization process from the distribution network control knowledge graph to the power grid physical simulation engine, the synchronized content includes the static attributes and static rules of physical entity nodes. Static attributes can include equipment rated parameters, topological connection relationships, etc., while static rules can include physical constraints on equipment operation, safety thresholds for power grid operation, etc. The synchronization process is achieved through a data mapping algorithm, which transforms the static attributes and rules stored in the knowledge graph as triples into numerical and logical constraints recognizable by the power grid physical simulation engine. These constraints are then directly embedded into the power balance equations of the power flow calculation engine and the differential equations of the dynamic transient simulator, serving as boundary conditions and constraints for the simulation calculations. During the synchronization process from the power grid physical simulation engine to the distribution network control knowledge graph, the content synchronized is the dynamic operating data output by the simulation, such as the real-time voltage of each node, the real-time load rate of the branch, and the real-time frequency of the system. The synchronization process is achieved through an entity matching algorithm, which can accurately associate the dynamic electrical quantities obtained from the simulation calculation with the physical entity nodes in the knowledge graph, update the dynamic operating attribute fields of the physical entity nodes, and enable the knowledge graph to reflect the operating status of the distribution network in real time during the simulation process, providing real-time knowledge support for subsequent state aggregation and reward calculation.

[0089] Furthermore, the core function of the state aggregation module is to achieve the fusion representation of the physical operating state and topological rule state of the distribution network, providing standardized input states for the deep reinforcement learning agent. In the state aggregation module, the simulation results output by the power grid physical simulation engine after completing the state evolution simulation are first obtained. From these results, core electrical operating data such as node voltage amplitude, branch load rate, system frequency, active power, and reactive power are extracted. The minimum-maximum normalization method is used to map various electrical quantities to a unified numerical range for these electrical operating data. Then, high-dimensional feature extraction is performed through a fully connected layer, ultimately obtaining fixed-dimensional numerical features. Subsequently, the state aggregation module can extract a local subgraph from the distribution network control knowledge graph based on the operation object corresponding to the remote control operation action. The extraction rule can be to select all physical entity nodes within their 1-hop to 2-hop neighborhood and the relationships between these nodes, centered on the physical entity node of the operation object. This local subgraph can accurately represent the local power grid topology environment where the operation object is located. Furthermore, when encoding graph structure features for local subgraphs, a graph attention network can be used as the core encoding model. By calculating the attention weights of neighboring nodes to the central node, differentiated extraction of topological structure features can be achieved, ultimately yielding fixed-dimensional graph structure features. By performing a linear transformation on the two types of features using a preset weight matrix, and then performing nonlinear fusion through an activation function, a state representation that can be directly analyzed by the deep reinforcement learning agent can be obtained.

[0090] In a demonstrative sense, the reward calculation module can accurately calculate the dynamic safety margin by combining the physical operating characteristics of the distribution network, and integrate rule compliance and operational efficiency to form a comprehensive reward signal. In this module, a preset optimal number of operation steps can be obtained first. This optimal number of operation steps is determined based on historical best operation cases and expert experience in distribution network control, serving as a benchmark for operational efficiency evaluation. Secondly, the current remote control operation can be validated for rule compliance based on static rules in the distribution network control knowledge graph. For example, by retrieving power dispatching expert rules and grid physical constraint rules stored in the knowledge graph through a rule query interface, a rule matching algorithm is used to compare the operation type, operation object, and execution sequence of the current operation with the static rules one by one, outputting a "compliant" or "non-compliant" validation result. Then, through quantization mapping, the validation result is converted into a rule compliance index. This index ranges from 0 to 1, with a larger value indicating higher rule compliance of the operation.

[0091] Specifically, the reward calculation module can input remote control operation actions into the power grid physical simulation engine. The simulation engine performs state evolution simulation to obtain simulation results including node voltage amplitude, branch load rate, and system frequency. Based on these simulation results, a multi-dimensional coupling algorithm is used to calculate the dynamic safety margin. For example, its mathematical expression is:

[0092]

[0093] in, is the dynamic safety margin of the distribution network at step t, and its value ranges from 0 to 1. The larger the value, the higher the operational safety of the distribution network after the current operation is performed. The number of dimensions for core electrical parameters involved in safety margin calculations, in distribution network control scenarios, The value is 3, which corresponds to the three core electrical parameter dimensions: node voltage amplitude, branch load rate, and system frequency, respectively. The value of the i-th core electrical parameter is the actual operating value after the t-th simulation step. This value is directly taken from the output of the power grid physical simulation engine. This represents the lower limit of the safety threshold for the i-th core electrical parameter. The upper limit of the safety threshold for the i-th core electrical parameter is obtained from the static rules of the distribution network control knowledge graph, and it represents the legal safety boundary for distribution network operation. The center bias coefficient is the i-th core electrical parameter. Its function is to characterize the sensitivity of the parameter to deviating from the center value of the safety threshold. The value of the center bias coefficient is determined based on the impact characteristics of the parameter on the safe operation of the distribution network. This is the weighting index for the i-th core electrical parameter, used to enhance the influence of key electrical parameters on the dynamic safety margin, making the safety margin calculation results more consistent with the actual operational risk characteristics of the distribution network. After calculating the dynamic safety margin, it is directly used as the dynamic safety margin index without additional quantification conversion. This index maintains a consistent value range with the rule compliance index.

[0094] Specifically, the reward calculation module can calculate an operational efficiency index, which is calculated by subtracting the ratio of the current number of executed operation steps to the preset optimal number of operation steps from 1. This index ranges from 0 to 1, with a higher value indicating higher execution efficiency of the current operation sequence. By weighted summing the rule compliance index, dynamic safety margin index, and operational efficiency index, a comprehensive reward signal can be obtained. This signal can simultaneously reflect the rule compliance, physical security, and execution efficiency of the operation.

[0095] Based on the power grid physical simulation engine, bidirectional data synchronization mechanism, state aggregation module, and reward calculation module, a microservice architecture can be adopted. Each module is encapsulated as an independent service unit, and data interaction between service units is achieved through a pre-defined communication protocol. For example, this communication protocol can use the gRPC protocol based on TCP / IP, which has high transmission efficiency and strong cross-platform compatibility, meeting the transmission requirements of large amounts of real-time data during simulation. The rule query interface of the distribution network control knowledge graph can serve as a unified channel for interaction between each module and the knowledge graph. The interface adopts a RESTful architecture design, supporting fast querying and calling of various rules and entity attributes. The final constructed control simulation environment can simultaneously achieve high-precision evolution simulation of the distribution network physical state, real-time verification of rule constraints, and quantification of reward and punishment signals under dual constraints.

[0096] In one embodiment, a deep reinforcement learning agent is trained in a controlled simulation environment to obtain a trained deep reinforcement learning agent, including:

[0097] S1011. Construct a deep reinforcement learning agent, which includes a pre-defined neural network, a target policy network, and a target value network. The pre-defined neural network includes a numerical feature encoder, a graph feature encoder, a graph attention fusion layer, a policy network, and a value network. The numerical feature encoder encodes the numerical features in the state representation, obtaining encoded numerical features. The graph feature encoder encodes the graph structure features in the state representation, obtaining encoded graph structure features. The graph attention fusion layer fuses the encoded numerical features and the encoded graph structure features to obtain a fused feature vector. The policy network takes the fused feature vector as input and outputs the probability distribution of remote control actions. The value network takes the fused feature vector as input and outputs a state value estimate. The target policy network and target value network are obtained by replicating the network structures and parameters of the policy network and value network, respectively.

[0098] S1012. Initialize the experience replay buffer, set the preset batch size, fixed update interval and training convergence threshold;

[0099] S1013. Based on the deep reinforcement learning agent, conduct agent interaction training in a controlled simulation environment to obtain the trained deep reinforcement learning agent. The agent interaction training corresponds to the following steps:

[0100] S10131. Call the preset neural network to process the current state representation of the control simulation environment output, and output the probability distribution of remote control operation actions;

[0101] S10132. Add exploration noise to the probability distribution of remote control operation actions to obtain the added probability distribution. Select the target remote control operation action according to the added probability distribution, input the target remote control operation action into the control simulation environment, and obtain the comprehensive reward signal and the next state representation fed back by the control simulation environment. Construct an experience tuple including the state representation, the target remote control operation action, the comprehensive reward signal and the next state representation, and store the experience tuple in the experience playback buffer.

[0102] S10133. When the number of experience tuples stored in the experience replay buffer reaches the preset batch size, training experience tuples are sampled from the experience replay buffer. The target value is calculated through the target policy network and the target value network. A value network loss function is constructed. With the goal of maximizing the cumulative reward, the value network loss function is minimized to obtain the updated first network parameters. The updated first network parameters are assigned to the value network to obtain the optimized value network. Here, the cumulative reward is the accumulated value of the comprehensive reward signal along the training time sequence.

[0103] S10134. After each execution of step S10132, a fixed update interval is reached, a policy network loss function is constructed. With the goal of maximizing the accumulated reward, the policy network loss function is minimized to obtain the updated second network parameters. These updated second network parameters are then assigned to the policy network to obtain the optimized policy network. Based on a preset soft update coefficient, the parameters of the optimized policy network are synchronized to the target policy network using a soft update method, and the parameters of the optimized value network are synchronized to the target value network, resulting in the optimized target policy network and the optimized target value network. The optimized target policy network and the optimized target value network are used to calculate the target value for the next round.

[0104] S10135. Determine whether the target remote control operation action output by the deep reinforcement learning agent meets the preset operation target. Count the number of operations that meet the preset operation target. Calculate the task success rate during training based on the ratio of the number of operations to the total number of interactions of the deep reinforcement learning agent. Calculate the value loss value corresponding to the value network loss function. Repeat steps S10132 to S10134. When the task success rate reaches the training convergence threshold and the value network loss value is less than or equal to the preset loss threshold, stop training to obtain the trained deep reinforcement learning agent.

[0105] Specifically, a pre-defined neural network serves as the core feature extraction and decision-making module for the intelligent agent. The numerical feature encoder employs a multi-layer fully connected neural network architecture. The input layer receives numerical feature data from the state representation, the hidden layer uses the ReLU activation function to achieve non-linear transformation of the numerical features, and the output layer maps the feature dimensions to a pre-defined fixed dimension, ultimately obtaining the encoded numerical features. This encoding process achieves high-dimensional feature extraction of electrical operating parameters through matrix multiplication, effectively characterizing the intrinsic correlation of quantitative indicators such as distribution network node voltage and branch load rate. The graph feature encoder can be based on a graph attention network architecture. The input is the node and edge features of a local subgraph of the distribution network. Differential extraction of topological structure features is achieved by calculating the attention weights of neighboring nodes to the central node. Furthermore, the encoder output layer can use mean pooling to aggregate node-level features into global graph structure features, resulting in encoded graph structure features that accurately reflect the power grid topology environment in which the operation object is located. In the graph attention fusion layer, an attention weighted fusion mechanism can be adopted. First, the encoded numerical features and graph structure features are dimensionally aligned. Then, the fusion coefficients of the two types of features are calculated through the attention weight matrix. Finally, the fused feature vector is obtained through weighted summation and nonlinear activation function processing. This vector contains both the physical operation quantitative features of the distribution network and the topological rule association features.

[0106] Schematic, the policy network can employ a fully connected network of the same dimension as the numerical feature encoder as its main body. Taking the fused feature vector as input, it outputs the probability distribution of remote control operation actions through multiple fully connected layers and a Softmax activation function. This probability distribution quantitatively characterizes the safety feasibility and execution value of each operation action under the current distribution network state. The value network can also adopt a network structure symmetrical to the policy network. Taking the fused feature vector as input, it outputs a state value estimate through fully connected layers and a linear output layer. This estimate characterizes the long-term benefit of executing the optimal operation strategy under the current distribution network state. Furthermore, the target policy network and target value network are obtained by replicating the network structures and parameters of the policy network and value network, respectively. Initially, their parameters are completely consistent with the corresponding main network. During subsequent training, a soft update method can be used to gradually synchronize the main network parameters. This design effectively reduces the target value estimation bias during training and improves the stability of agent training.

[0107] Furthermore, the experience replay buffer can adopt a first-in, first-out (FIFO) queue storage structure. The buffer capacity is determined based on the state space and action space dimensions of the distribution network operation scenario, and is used to store experience tuples generated during training. The preset batch size is the sampling unit for model training in the experience replay buffer, and its value matches the network parameter dimension and distribution network state feature dimension of the deep reinforcement learning agent. The fixed update interval is the parameter update period of the value network and policy network, and this interval can be determined based on the temporal characteristics of distribution network remote control operations and the learning rate of the agent. The training convergence threshold is the target threshold for the task success rate, and the preset loss threshold is the upper limit threshold of the value network loss function, constituting the termination condition for agent training.

[0108] Specifically, in the interactive training of a deep reinforcement learning agent, a pre-defined neural network can be invoked to process the current power distribution network state representation output by the simulation environment, following the aforementioned feature encoding-fusion-decision chain. Finally, the probability distribution of all selectable remote control actions can be output. This probability distribution ranges from 0 to 1, with the sum of the probabilities of all actions being 1, representing the probability of each action being selected in the current state. Subsequently, exploration noise can be added to the probability distribution of the remote control actions. This noise can be generated using a Gaussian noise model, with a mean of 0 and a variance dynamically adjusted according to the complexity of the power distribution network operation space. This increases the randomness of the agent's action exploration, preventing the agent from getting trapped in local optima. When selecting a target remote control action based on the probability distribution with added exploration noise, a roulette wheel sampling method can be used. The probability distribution is used as the sampling weight, and one action is randomly selected from all selectable actions as the target remote control action. This selection method ensures a high selection rate for high-probability actions while retaining exploration opportunities for low-probability actions. The current distribution network state, the target remote control operation action, the comprehensive reward signal, and the next distribution network state after executing the target remote control operation action are used as the four core elements of a tuple. These elements are stored in key-value pair format, and each element is associated with a unique identifier, thus constructing an experience tuple. Furthermore, when storing experience tuples in the experience replay buffer, if the current storage capacity of the buffer reaches its limit, the oldest stored experience tuple can be removed to ensure that the latest training experience data is always stored.

[0109] Indicatively, when the number of stored training experience tuples reaches a preset batch size, a batch of training experience tuples can be sampled from the experience replay buffer. The target value is then calculated through the target policy network and the target value network, constructing a value network loss function. The goal is to maximize the cumulative reward while minimizing the value network loss function. The cumulative reward is the accumulated value of the comprehensive reward signal along the training timeline, representing the agent's long-term gain from the initial training moment to the current moment, and is the core optimization objective of model training. Based on the principle of temporal difference learning, the target policy network outputs the action probability distribution for the next state, the target value network outputs the value estimate for the next state, and the target value network outputs the value estimate for the next state. The target value is then calculated by combining this with the current comprehensive reward signal. The value network loss function can be constructed using the mean squared error between the target value and the state value estimate output by the value network. By maximizing the cumulative reward, the stochastic gradient descent algorithm can be used to minimize the value network loss function. The gradients of each parameter of the value network are calculated through backpropagation, and the parameters are updated according to the learning rate to obtain the updated first network parameters. These updated parameters are then assigned to the value network to obtain the optimized value network.

[0110] Specifically, after each execution of step S10132 and reaching a fixed update interval, a policy network loss function can be constructed. The goal is to maximize the cumulative reward while minimizing the policy network loss function. This policy network loss function can combine the advantage function and the entropy regularization term, and its expression is as follows: , Let the policy network loss value be . This is the set of learnable parameters for the policy network; Indicates the experience replay buffer Calculate the mathematical expectation of the sampled training experience tuples; The parameter is The strategy network in the distribution network state Output remote control operation actions The probability of; The advantage function represents the value advantage of the current remote control action compared to the average remote control action. Its calculation formula is as follows: , The action value estimate is calculated from the state value estimate output by the value network and the advantage function. For state value estimation; It represents the entropy regularization coefficient, used to balance the exploration and utilization capabilities of the agent and avoid local optima caused by the large operation space of the distribution network; The entropy represents the probability distribution of actions output by the policy network, and its calculation formula is: A higher entropy value indicates a stronger randomness in action selection. The parameter update process of the policy network also employs the stochastic gradient descent algorithm, aiming to maximize cumulative reward while minimizing the policy network loss function. The gradients of each parameter of the policy network are calculated through backpropagation, and the parameters are updated according to the learning rate to obtain the updated second network parameters. These updated parameters are then assigned to the policy network to complete the optimization, resulting in the optimized policy network. The soft update method uses the exponential moving average method, whose mathematical expression is: , These are the parameters of the target policy network or the target value network. To optimize the parameters of the policy network or the value network, A preset soft update coefficient, ranging from 0 to 1, is used to smoothly update the target network parameters, avoiding training oscillations caused by sudden changes in the main network parameters. After the soft update, an optimized target policy network and an optimized target value network are obtained. These are used to calculate the target value in the next round of training to ensure the stability and accuracy of the target value.

[0111] Specifically, the preset operation targets are determined based on the actual needs of distribution network remote control operations, such as restoring the distribution network state to the target operating condition, ensuring no violations in remote control operations, and achieving a preset threshold for the dynamic safety margin of the distribution network. By matching the target remote control operation actions output by the agent with the preset operation targets, a logical comparison algorithm can be used to output the judgment result. The task success rate is calculated as the ratio of the number of operations that satisfy the preset operation targets to the total number of interactions. This indicator characterizes the agent's ability to complete distribution network remote control operation tasks, with a value ranging from 0 to 1; a larger value indicates a stronger task completion ability. The value loss value is the average of the value network loss function, reflecting the accuracy of the value network's estimation of state value; a smaller value indicates higher estimation accuracy. Repeat steps S10132 to S10134 until the training termination condition is met, that is, when the task success rate reaches the training convergence threshold and the value network loss value is less than or equal to the preset loss threshold, the decision strategy of the agent can be regarded as reaching a stable state. After stopping training, a deep reinforcement learning agent can be obtained after training. This agent can accurately output safe and compliant remote control operation actions based on the real-time status of the distribution network, providing decision support for the intelligent evaluation of subsequent operation tickets.

[0112] In one embodiment, such as Figure 2 As shown, based on the trained deep reinforcement learning agent, the target operation ticket is parsed into an initial action sequence, and a full-sequence simulation is performed on the initial action sequence in conjunction with the current real-time state of the power grid. The simulation outputs safety assessment results and optimization suggestions, including:

[0113] S201: Perform structured parsing on the target operation ticket, extract operation object and operation type data, and generate an initial action sequence; read the current real-time status data of the power grid from the distribution network control knowledge graph, load the current real-time status data of the power grid into the control simulation environment, and obtain the real-time control simulation environment;

[0114] S202: Input the initial action sequence into the trained deep reinforcement learning agent and output the probability distribution of the remote control operation action corresponding to each action in the initial action sequence;

[0115] S203: Input the probability distribution of the remote control operation actions corresponding to each action into the real-time control simulation environment, perform full sequence simulation on the initial action sequence, record the changes in power grid operation indicators and safety margin during the simulation process, and obtain real-time simulation results; among them, power grid operation indicators include node voltage amplitude, branch load rate, active power, reactive power and system frequency;

[0116] S204: Generate a safety assessment result based on the real-time simulation result, generate a corresponding optimization operation sequence based on the safety assessment result, combine the optimization operation sequence to obtain optimization suggestions, and output the safety assessment result and optimization suggestions.

[0117] Specifically, the target operation ticket is usually presented in the form of natural language text or semi-structured form. Based on the preset power field natural language processing model built in the above embodiments, it can be parsed to generate an initial action sequence. Each action element in the sequence can include the operation object identifier, operation type and execution time sequence index to ensure the orderliness and executability of the action sequence.

[0118] Subsequently, real-time power grid status data can be read from the distribution network control knowledge graph, including the dynamic operating attributes of each physical entity node, such as node voltage amplitude, branch load rate, equipment operating status, and system frequency. This data can be synchronized to the distribution network control knowledge graph through the real-time acquisition interface of the distribution automation system and stored in the form of dynamic attribute fields of physical entity nodes. Furthermore, a state mapping algorithm can be used to associate the real-time status data in the knowledge graph with the power grid node and branch models in the control simulation environment. Through entity matching, dynamic operating attributes are mapped to the simulation parameters of the corresponding nodes and branches, updating the initial power injection and voltage constraints of the power flow calculation engine, and simultaneously updating the initial state variables of the dynamic transient simulator. This ensures that the initial state of the simulation environment is completely consistent with the actual operating state of the current power grid, guaranteeing the realism and reliability of the entire simulation series.

[0119] Furthermore, the initial action sequence is input into the trained deep reinforcement learning agent. The deep reinforcement learning agent can process the initial action sequence using an action-by-action evaluation mode. That is, for each action element in the sequence, the local subgraph corresponding to the operation object is first extracted from the distribution network control knowledge graph. The local subgraph is centered on the physical entity node of the operation object and includes all physical entity nodes and their relationships within its 1 to 2 hop neighborhood. Then, the current real-time state data of the power grid and the features of the local subgraph are input into a preset neural network. The electrical operation data and topology features are encoded by numerical feature encoder and graph feature encoder, respectively. Then, a fused feature vector is generated by a graph attention fusion layer. Finally, the fused feature vector is input into the policy network. The policy network outputs the probability distribution of the remote control operation action corresponding to the action through a fully connected layer and a softmax activation function. This probability distribution quantitatively represents the safety feasibility and execution value of performing the action under the current power grid state, providing a decision-making basis for subsequent simulation and deduction.

[0120] Specifically, by inputting the probability distribution of the remote control operation actions corresponding to each action into the real-time control simulation environment, a full-sequence simulation can be performed on the initial action sequence. This employs a time-series progressive execution mode, executing each action sequentially according to the time index of the initial action sequence. The execution flow for each action is as follows: first, the probability distribution corresponding to the action is input into the real-time control simulation environment; the simulation environment performs state evolution simulation through the power grid physical simulation engine; the power flow calculation engine calculates the steady-state electrical quantities after the action execution; and the dynamic transient simulator simulates the transient process, outputting power grid operation indicators such as node voltage amplitude, branch load rate, active power, reactive power, and system frequency. Subsequently, based on the power grid operation indicators, the dynamic safety margin can be calculated using the formulas in the above embodiments. By recording the power grid operation indicators and dynamic safety margin step by step during the simulation process, a full-sequence real-time simulation result can be formed. This result fully reflects the evolution process of the power grid state and the change in safety level after the execution of the initial action sequence.

[0121] Specifically, based on the dynamic safety margin changes in the full-sequence simulation, the safety margin after each action can be determined by setting a preset safety threshold. When the safety margin is lower than the threshold, the action is marked as a risky operation. Simultaneously, the location of the risk point, the corresponding grid operation indicators, and the safety margin value are recorded to form a risk list. Furthermore, based on a trained deep reinforcement learning agent, for risky operations, the alternative action with the highest probability value and satisfying safety constraints can be selected from the probability distribution output by the policy network to replace the risky action in the initial action sequence. At the same time, the temporal index of subsequent actions is adjusted to generate a corresponding optimized operation sequence, ensuring the compliance and safety of the optimized operation sequence. Based on the optimized operation sequence and the risk list, further optimization suggestions can be generated, clarifying the causes of the risk points, the optimized operation steps, and precautions, such as adjusting the operation order, replacing the operation object, and supplementing pre-checks. Finally, the safety assessment results and optimization suggestions can be output in a standardized report format to provide decision support for distribution network control personnel.

[0122] Based on the same inventive concept, this application also provides a distribution network regulation dual-constraint control system for implementing the aforementioned distribution network regulation dual-constraint control method. The solution provided by this system is similar to the implementation scheme described in the above method; therefore, the specific limitations of one or more embodiments of the distribution network regulation dual-constraint control system provided below can be found in the limitations of the distribution network regulation dual-constraint control method described above, and will not be repeated here.

[0123] In one exemplary embodiment, such as Figure 3 As shown, a dual-constraint control system 300 for distribution network regulation is provided, comprising:

[0124] The knowledge graph construction module 301 is used to generate a structured knowledge base based on the process rule data of the scheduling and control platform and the power grid physical rule data of the distribution automation system. This base includes physical entity nodes, process entity nodes, and cross-domain relationships between physical entity nodes and process entity nodes. Based on the structured knowledge base, a distribution network control knowledge graph is constructed.

[0125] The simulation environment construction module 302 is used to construct a control simulation environment based on the distribution network control knowledge graph. The control simulation environment is used to simulate the state evolution of the distribution network by integrating the power grid physical simulation engine and the rule query interface of the distribution network control knowledge graph, calculate the dynamic safety margin, and calculate the comprehensive reward signal according to the static rules and dynamic safety margin in the distribution network control knowledge graph.

[0126] The reinforcement learning training module 303 is used to train a deep reinforcement learning agent in a controlled simulation environment to obtain a trained deep reinforcement learning agent. The deep reinforcement learning agent takes the state representation output by the controlled simulation environment as input, analyzes it through a preset neural network, outputs a probability distribution of remote control operation actions, selects a target remote control operation action based on the probability distribution, inputs the target remote control operation action into the controlled simulation environment, and updates the network parameters of the preset neural network with the goal of maximizing the cumulative reward. The cumulative reward is obtained by accumulating the comprehensive reward signal.

[0127] The operation safety verification module 304 is used to obtain the target operation ticket, and based on the trained deep reinforcement learning agent, it parses the target operation ticket into an initial action sequence, and performs a full-sequence simulation and deduction of the initial action sequence in combination with the current real-time state of the power grid, and outputs the safety assessment results and optimization suggestions.

[0128] In one exemplary embodiment, the present invention also provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the dual-constraint control method for power distribution network regulation according to this application. A multi-core processor is preferred to improve the system's parallel processing capability. The memory provides sufficient temporary storage space to support program execution and data processing. The memory capacity should be large enough to accommodate large amounts of data and computational tasks.

[0129] In one exemplary embodiment, the present invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of a dual-constraint control method for power distribution network regulation according to the present application. The computer-readable storage medium may include: a read-only memory, a random access memory, a solid-state drive, or an optical disk, etc.

[0130] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A dual-constraint control method for distribution network regulation, characterized in that, The method includes: Based on the process rule data of the scheduling and control platform and the power grid physical rule data of the distribution automation system, a structured knowledge base is generated, including physical entity nodes, process entity nodes and cross-domain association relationships between the physical entity nodes and the process entity nodes. Based on the structured knowledge base, a distribution network control knowledge graph is constructed. Based on the distribution network control knowledge graph, a control simulation environment is constructed. The control simulation environment is used to simulate the state evolution of the distribution network through the rule query interface of the integrated power grid physical simulation engine and the distribution network control knowledge graph, calculate the dynamic safety margin, and calculate the comprehensive reward signal according to the static rules in the distribution network control knowledge graph and the dynamic safety margin. In the controlled simulation environment, a deep reinforcement learning agent is trained to obtain a trained deep reinforcement learning agent. The deep reinforcement learning agent takes the state representation output by the controlled simulation environment as input, parses it through a preset neural network, outputs the probability distribution of the remote control operation action, selects a target remote control operation action based on the probability distribution, inputs the target remote control operation action into the controlled simulation environment, and updates the network parameters of the preset neural network with the goal of maximizing the cumulative reward. The cumulative reward is obtained by accumulating the comprehensive reward signal. The target operation ticket is obtained. Based on the trained deep reinforcement learning agent, the target operation ticket is parsed into an initial action sequence. The initial action sequence is then simulated and deduced in combination with the current real-time state of the power grid. The safety assessment results and optimization suggestions are then output.

2. The method according to claim 1, characterized in that, The process rule data includes operation ticket business data; The process rule data based on the scheduling and control platform and the power grid physical rule data of the distribution automation system generate a structured knowledge base including physical entity nodes, process entity nodes, and cross-domain relationships between the physical entity nodes and the process entity nodes, including: The power grid physical rule data is obtained from the power distribution automation system, and the power grid equipment ledger and power grid topology connection data are obtained from the power distribution production management system. Based on the power grid physical rule data, the power grid equipment ledger and the power grid topology connection data, the physical entity nodes are generated and their corresponding static attributes are configured, and the electrical topology relationship between each physical entity node is established. Operation ticket business data is extracted from the scheduling and control platform, the operation ticket business data is structured and parsed to generate operation ticket entity nodes and operation step entity nodes, the operation ticket entity nodes and the operation step entity nodes are determined as the process entity nodes, the membership relationship between the operation ticket entity nodes and the operation step entity nodes is established, and the sequential dependency relationship between each operation step entity node is established. Using a pre-defined natural language processing model for the power sector, entity matching processing is performed on the operation instructions corresponding to the operation step entity nodes to establish the operation association between the operation step entity nodes and the physical entity nodes. Combining preset power dispatch expert rules and power grid physical constraints, establish physical precondition relationships and physical poststate relationships between the operation step entity nodes and the physical entity nodes, and use the operation association relationship, the physical precondition relationship, and the physical poststate relationship as cross-domain association relationships between the physical entity nodes and the process entity nodes. The structured knowledge base is generated by integrating the physical entity nodes, process entity nodes, electrical topology relationships, membership relationships, sequential dependencies, and cross-domain association relationships.

3. The method according to claim 1, characterized in that, The construction of a control simulation environment based on the distribution network control knowledge graph includes: The power grid topology and equipment electrical parameters are extracted from the power distribution network control knowledge graph, and a power grid node branch model is constructed based on the power grid topology and equipment electrical parameters; The power grid node branch model is used as the simulation carrier of the power grid physical simulation engine. The power flow calculation engine and the dynamic transient simulator are integrated to construct the power grid physical simulation engine for simulating changes in the electrical state of the distribution network. The power flow calculation engine is used to calculate the steady-state operation state of the distribution network, and the dynamic transient simulator is used to simulate the transient process of the distribution network after the execution of the remote control operation. A two-way data synchronization mechanism is established between the power grid physical simulation engine and the distribution network control knowledge graph. The two-way data synchronization mechanism is used to write the dynamic operation data output by the power grid physical simulation engine back to the distribution network control knowledge graph, update the dynamic operation attributes of the physical entity nodes, and synchronize the static attributes and static rules of the physical entity nodes in the distribution network control knowledge graph to the power grid physical simulation engine as constraints for the power grid physical simulation engine to perform simulation calculations. A state aggregation module is constructed, which is used to acquire electrical operation data from the simulation results output by the power grid physical simulation engine after simulating state evolution based on the remote control operation action; the electrical operation data is numerically feature-encoded to obtain numerical features; according to the operation object corresponding to the remote control operation action, a local subgraph including the operation object and associated nodes is extracted from the distribution network control knowledge graph; the local subgraph is graph structure feature-encoded to obtain graph structure features; the numerical features and the graph structure features are weighted and fused to obtain a state representation; A reward calculation module is constructed, which is used to obtain a preset optimal number of operation steps, perform rule compliance verification on the current remote control operation based on the static rules in the distribution network control knowledge graph, obtain the verification result, and convert the verification result into a rule compliance index; input the remote control operation into the power grid physical simulation engine, perform state evolution simulation through the power grid physical simulation engine to obtain simulation results, calculate the dynamic safety margin based on the simulation results, and convert the dynamic safety margin into a dynamic safety margin index; determine the operation efficiency index based on the ratio of the currently executed operation steps to the preset optimal number of operation steps; and calculate the comprehensive reward signal by weighted summation of the rule compliance index, the dynamic safety margin index, and the operation efficiency index. The power grid physical simulation engine, the two-way data synchronization mechanism, the state aggregation module, and the reward calculation module are integrated and configured. The rule query interface of the distribution network control knowledge graph is called as the interaction channel, and data interaction is carried out through a preset communication protocol to construct the control simulation environment.

4. The method according to claim 3, characterized in that, The process of training a deep reinforcement learning agent in the controlled simulation environment to obtain a trained deep reinforcement learning agent includes: S1. Construct a deep reinforcement learning agent, which includes the preset neural network, a target policy network, and a target value network. The preset neural network includes a numerical feature encoder, a graph feature encoder, a graph attention fusion layer, a policy network, and a value network. The numerical feature encoder encodes the numerical features in the state representation to obtain encoded numerical features. The graph feature encoder encodes the graph structure features in the state representation to obtain encoded graph structure features. The graph attention fusion layer fuses the encoded numerical features and the encoded graph structure features to obtain a fused feature vector. The policy network takes the fused feature vector as input and outputs the probability distribution of the remote control operation. The value network takes the fused feature vector as input and outputs a state value estimate. The target policy network and the target value network are obtained by replicating the network structures and parameters of the policy network and the value network, respectively. S2. Initialize the experience replay buffer, set the preset batch size, fixed update interval and training convergence threshold; S3. Based on the deep reinforcement learning agent, perform agent interaction training in the controlled simulation environment to obtain the trained deep reinforcement learning agent, wherein the agent interaction training corresponds to the following steps: S31. The preset neural network is invoked to process the current state representation output by the control simulation environment, and the probability distribution of the remote control operation action is output. S32. Add exploration noise to the probability distribution of the remote control operation action to obtain the added probability distribution. Select the target remote control operation action according to the added probability distribution. Input the target remote control operation action into the control simulation environment. Obtain the comprehensive reward signal and the next state representation fed back by the control simulation environment. Construct an experience tuple including the state representation, the target remote control operation action, the comprehensive reward signal, and the next state representation. Store the experience tuple in the experience playback buffer. S33. When the number of experience tuples stored in the experience replay buffer reaches the preset batch size, training experience tuples are sampled from the experience replay buffer, and the target value is calculated through the target policy network and the target value network. A value network loss function is constructed, with the goal of maximizing the cumulative reward, and the value network loss function is minimized to obtain the updated first network parameters. The updated first network parameters are assigned to the value network to obtain the optimized value network; wherein, the cumulative reward is the accumulated value of the comprehensive reward signal along the training time sequence; S34. After each execution of step S32, the fixed update interval number is reached, a strategy network loss function is constructed. With the goal of maximizing the accumulated reward, the strategy network loss function is minimized to obtain updated second network parameters. These updated second network parameters are then assigned to the strategy network to obtain an optimized strategy network. Based on a preset soft update coefficient, the parameters of the optimized strategy network are synchronized to the target strategy network using a soft update method, and the parameters of the optimized value network are synchronized to the target value network to obtain an optimized target strategy network and an optimized target value network. The optimized target strategy network and the optimized target value network are used to calculate the target value for the next round. S35. Determine whether the target remote control operation action output by the deep reinforcement learning agent meets the preset operation target, count the number of operations that meet the preset operation target, calculate the task success rate during training based on the ratio of the number of operations to the total number of interactions of the deep reinforcement learning agent, and calculate the value loss value corresponding to the value network loss function; repeat steps S32 to S34, and stop training when the task success rate reaches the training convergence threshold and the value network loss value is less than or equal to the preset loss threshold, to obtain the trained deep reinforcement learning agent.

5. The method according to claim 1, characterized in that, The deep reinforcement learning agent, based on the trained code, parses the target operation ticket into an initial action sequence, and performs a full-sequence simulation of the initial action sequence in conjunction with the current real-time state of the power grid, outputting a safety assessment result and optimization suggestions, including: The target operation ticket is structured and parsed to extract the operation object and operation type data, and the initial action sequence is generated. Read the current real-time status data of the power grid from the distribution network control knowledge graph, load the current real-time status data of the power grid into the control simulation environment, and obtain the real-time control simulation environment; The initial action sequence is input into the trained deep reinforcement learning agent, and the probability distribution of the remote control operation action corresponding to each action in the initial action sequence is output. The probability distribution of the remote control operation actions corresponding to each action is input into the real-time control simulation environment. The initial action sequence is subjected to full-sequence simulation and deduction. The changes in power grid operation indicators and safety margin during the simulation process are recorded to obtain the real-time simulation results. The power grid operation indicators include node voltage amplitude, branch load rate, active power, reactive power and system frequency. A security assessment result is generated based on the real-time simulation result. A corresponding optimization operation sequence is generated based on the security assessment result. The optimization suggestion is obtained by combining the optimization operation sequence. The security assessment result and the optimization suggestion are then output.

6. The method according to claim 3, characterized in that, The formula for calculating the dynamic safety margin is as follows: in, Indicates the first The dynamic safety margin of the distribution network is defined in step [step], and its value ranges from 0 to 1. Indicates the number of dimensions of the core electrical parameters; Indicates the first The first core electrical parameter Step actual running value; Indicates the first Lower limit of safety threshold for core electrical parameters; Indicates the first Upper limit of safety threshold for core electrical parameters; Indicates the first The center bias coefficient of the core electrical parameters; Indicates the first Weighting index of core electrical parameters.

7. The method according to claim 4, characterized in that, The mathematical expression for the policy network loss function is: in, This represents the network loss value of the policy; This represents the set of learnable parameters of the policy network; This indicates the experience playback buffer. The expected value of the sampled training experience tuples is calculated, wherein the training experience tuples include the first... Step-by-step power distribution network status representation Target remote control operation actions Comprehensive reward signal and execution The next Step-by-step power distribution network status representation ; The parameter is The strategy network in the distribution network state Output remote control operation actions The probability of; The advantage function is used to characterize the value advantage of the current remote control action compared to the average remote control action. This represents the entropy regularization coefficient; The entropy represents the probability distribution of actions output by the policy network.

8. A dual-constraint control system for distribution network regulation, characterized in that, The system includes: The knowledge graph construction module is used to generate a structured knowledge base, including physical entity nodes, process entity nodes, and cross-domain relationships between the physical entity nodes and the process entity nodes, based on the process rule data of the scheduling and control platform and the power grid physical rule data of the distribution automation system. Based on the structured knowledge base, a distribution network control knowledge graph is constructed. The simulation environment construction module is used to construct a control simulation environment based on the distribution network control knowledge graph. The control simulation environment is used to simulate the state evolution of the received remote control operation actions by integrating the power grid physical simulation engine and the rule query interface of the distribution network control knowledge graph, calculate the dynamic safety margin, and calculate the comprehensive reward signal according to the static rules in the distribution network control knowledge graph and the dynamic safety margin. A reinforcement learning training module is used to train a deep reinforcement learning agent in the controlled simulation environment to obtain a trained deep reinforcement learning agent. The deep reinforcement learning agent takes the state representation output by the controlled simulation environment as input, parses it through a preset neural network, outputs the probability distribution of the remote control operation action, selects a target remote control operation action according to the probability distribution, inputs the target remote control operation action into the controlled simulation environment, and updates the network parameters of the preset neural network with the goal of maximizing the cumulative reward. The cumulative reward is obtained by accumulating the comprehensive reward signal. The operation safety verification module is used to obtain the target operation ticket, parse the target operation ticket into an initial action sequence based on the trained deep reinforcement learning agent, and perform a full-sequence simulation of the initial action sequence in combination with the current real-time state of the power grid, and output safety assessment results and optimization suggestions.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images