An intelligent control method for distribution network based on knowledge graph and deep reinforcement learning

Abstract

The application provides a power distribution network intelligent control method based on a knowledge graph and deep reinforcement learning, relates to the technical field of power distribution network intelligent control, obtains a structured data set and an operation state sample by preprocessing multi-source data of a power distribution network; constructs a power distribution network knowledge graph, performs knowledge reasoning, and outputs knowledge features and a safety action set under a current operation state; under the constraint of the safety action set, performs offline constraint training on the operation state sample and the knowledge features, and outputs a control strategy model of the power distribution network; further generates an original control action, matches and corrects the original control action, and executes after safety verification; updates the power distribution network knowledge graph according to an online execution result, and updates the control strategy model in at least one of a periodic retraining mode and an event-triggered incremental optimization mode. The application can realize safe adaptive generation of a power distribution network control strategy under a complex operation scenario.

Description

Technical Field

[0001] This application relates to the field of intelligent control technology for power distribution networks, and more specifically, to an intelligent control method for power distribution networks based on knowledge graphs and deep reinforcement learning. Background Technology

[0002] With the continuous integration of distributed power sources, energy storage devices, electric vehicles, and flexible loads, the distribution network is gradually transforming from a traditional unidirectional power supply network into a complex system with multiple sources, multiple states, and multiple control objects operating collaboratively. The intelligent control technology of the distribution network can coordinate and control voltage regulating equipment, reactive power compensation devices, energy storage devices, and switching equipment according to the network operating status. This is used to suppress voltage fluctuations, alleviate branch overloads, and optimize power flow distribution, providing technical support for improving power quality, equipment utilization efficiency, and operational regulation capabilities under complex operating conditions.

[0003] In existing technologies, control methods based on power flow calculations or optimization models typically rely on accurate network topology, line parameters, equipment models, and prediction data. When the output of distributed power sources, load levels, or network structure change, problems such as large model maintenance workload, time-consuming solution processes, and insufficient adaptation of control results easily arise. Control methods based on expert rules are limited by preset scenarios and human experience, making it difficult to cover multi-device coupling, continuous state changes, and sudden operating conditions. The generated control actions are often relatively conservative. Although deep reinforcement learning can learn control strategies through interaction with historical samples and the environment, if there is a lack of explicit utilization of topology connections, equipment adjustment boundaries, and operating rules, the training or online decision-making process may still output control actions that cause voltage overruns, branch overloads, out-of-bounds energy storage charge states, or switch malfunctions. Furthermore, it is difficult to explain the basis for action selection. Therefore, how to achieve safe and adaptive generation of distribution network control strategies under complex operating scenarios has become a challenge for the industry. Summary of the Invention

[0004] This application provides a distribution network intelligent control method based on knowledge graphs and deep reinforcement learning, which can realize the safe adaptive generation of distribution network control strategies under complex operating scenarios.

[0005] Firstly, this application provides a smart control method for power distribution networks based on knowledge graphs and deep reinforcement learning, comprising the following steps: Acquire multi-source data of the power distribution network, including operational data, topology parameters, equipment parameters, expert rules, and historical control cases; The multi-source data is preprocessed to obtain a structured data set and a running status sample; A distribution network knowledge graph is constructed based on the structured data set, and knowledge reasoning is performed to output the knowledge features and safety action set under the current operating state. Under the constraints of the set of safety actions, offline constraint training is performed using the operating state samples and the knowledge features to output a control strategy model for the power distribution network. The original control actions are generated through the control strategy model, and the original control actions are matched and corrected. After security verification, they are sent out for execution. The distribution network knowledge graph is updated based on the online execution results, and the control strategy model is updated using at least one of periodic retraining and event-triggered incremental optimization.

[0006] In this embodiment, preprocessing the multi-source data to obtain a structured data set and a running status sample specifically includes: The operational data, topology parameters, equipment parameters, expert rules, and historical control cases in the multi-source data are preprocessed to form basic data with a unified format. The basic data is subjected to entity extraction, relation extraction, and attribute mapping to convert unstructured and semi-structured content into a structured data set. The basic and derived features of the operating status are extracted from the basic data, and the operating status samples are determined based on the basic and derived features.

[0007] In this embodiment, constructing a distribution network knowledge graph based on the structured data set, performing knowledge reasoning, and outputting the knowledge features and safety action set under the current operating state specifically includes: The structured data set is deduplicated, and the topological connection relationships, device access relationships, rule triggering relationships and case association relationships between each entity are extracted to generate a power distribution network knowledge graph. The operation constraints of the distribution network are converted into safety constraint functions, and the expert rules are converted into rule judgment functions. Then, the safety constraint functions and the rule judgment functions are attached to the distribution network knowledge graph. An initial action candidate set is generated based on the current operating status. The candidate actions in the initial action candidate set are filtered by calling the distribution network knowledge graph. After verification by operating safety constraints, a set of safe actions under the current operating status is generated. Based on the current operating status, topological relationships, equipment constraints, expert rules, and historical similar case information related to the current operating status are extracted from the distribution network knowledge graph to form knowledge features of the current operating status.

[0008] In this embodiment, an initial action candidate set is generated based on the current operating state. The candidate actions in the initial action candidate set are filtered by calling the distribution network knowledge graph. After verification by operating safety constraints, a set of safe actions for the current operating state is generated, specifically including: Based on the current operating status, enumerate the action options of various controllable devices to form an initial action candidate set; The initial action candidate set is filtered by calling the expert rules in the power distribution network knowledge graph, and the candidate actions that do not meet the requirements of applicable scenario, control object, action type and expert rule priority are eliminated; Based on the equipment operating boundary, the candidate actions obtained from the filtering are trimmed, and the actions are restricted to the constraints of equipment capacity, adjustment range, action step size and number of actions. The candidate actions after trimming are verified by voltage constraints, branch current constraints, energy storage state of charge constraints and topology safety constraints. The candidate actions that pass all the verifications are taken as the set of safe actions under the current operating state.

[0009] In this embodiment, under the constraints of the set of safety actions, offline constraint training is performed using the operating state samples and the knowledge features to output the control strategy model of the distribution network, specifically including: The running state samples are fused with the knowledge features to construct a reinforcement learning state; Using the reinforcement learning state as input to the policy network, a hybrid action space containing discrete action output branches and continuous action output branches is constructed, and the exploration range of the policy network is limited by the set of safe actions. Construct the policy loss function; The policy network is iteratively trained using the reinforcement learning state and the set of safety actions, combined with the policy loss function, until the convergence condition is met, and the distribution network control policy model is output.

[0010] In this embodiment, the reinforcement learning state is used as the input to the policy network to construct a hybrid action space containing discrete action output branches and continuous action output branches, and the exploration scope of the policy network is limited by the set of safe actions. Specifically, this includes: The reinforcement learning state is used as the input to the policy network, and a joint representation is extracted through a shared feature extraction layer; Using the joint representation as input, discrete action probabilities are output through the discrete action output branch, and continuous control parameters are output through the continuous action output branch, thus forming a hybrid action space; The discrete action output branches are masked and filtered using the set of security actions, and the continuous action output branches are pruned according to the device operation boundary, thus limiting the exploration space of the policy network to the range of the set of security actions.

[0011] In this embodiment, generating original control actions through the control strategy model, matching and correcting the original control actions, and issuing them for execution after security verification specifically includes: The control strategy model is invoked based on the real-time operating status to generate the original control actions; The original control action is matched with the actions in the set of safety actions. If the match is successful, the matched action is used as a candidate control action. If a match fails, the original control action is modified according to the set of safety actions to obtain candidate control actions; The candidate control actions are subjected to operational safety constraint verification and expert rule constraint verification. Once the verification is passed, the actions are issued for execution.

[0012] In this embodiment, the process of generating the original control action by invoking the control strategy model based on the real-time operating status specifically includes: The real-time operating status is generated based on the real-time collected power distribution network operation data. Based on the real-time operating status, the power distribution network knowledge graph is invoked to perform online knowledge reasoning, thereby obtaining online knowledge features; The real-time operating status is fused with the online knowledge features and then input into the control strategy model to generate the original control action.

[0013] In this embodiment, updating the distribution network knowledge graph based on the online execution results specifically includes: Select online execution results that meet both security and optimization requirements as new running cases; The newly added operation cases are converted into case entities, written into the distribution network knowledge graph, and associated with the corresponding feeders, nodes, equipment, and rules. Based on the newly added operational data corresponding to the newly added operational cases, update the attributes of equipment entities and rule entities in the power distribution network knowledge graph.

[0014] In this embodiment, updating the control strategy model using at least one of periodic retraining and event-triggered incremental optimization specifically includes: Determine whether the preset time period has been reached, and monitor in real time whether the operating performance indicators are lower than the preset threshold; If the preset time period is reached, a training sample set containing new samples, historical samples, and similar cases from the knowledge graph is obtained from the sample library to retrain the control strategy model. If the running performance is lower than the preset threshold, new samples and risk samples are obtained from the sample library to incrementally optimize the control strategy model. The updated strategy model was validated offline. If the verification passes, the current online policy model is replaced and the model version number is recorded. If the verification fails, the original model is retained and the update result is recorded. If the security constraint violation rate increases when the new model is running online, it will automatically revert to the previous stable version.

[0015] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: The system acquires multi-source data from the distribution network, including operational data, topology parameters, equipment parameters, expert rules, and historical control cases. It preprocesses the multi-source data to obtain a structured data set and operational state samples. Based on the structured data set, it constructs a distribution network knowledge graph and performs knowledge reasoning to output knowledge features and a set of safety actions under the current operational state. Under the constraints of the safety action set, it performs offline constraint training using the operational state samples and the knowledge features to output a control strategy model for the distribution network. It generates original control actions using the control strategy model, performs matching and correction on these actions, and issues them for execution after safety verification. Based on the online execution results, it updates the distribution network knowledge graph and uses at least one of periodic retraining and event-triggered incremental optimization to update the control strategy model.

[0016] Therefore, in this application, at least one of periodic retraining and event-triggered incremental optimization can be used to update the control strategy model. Firstly, preprocessing multi-source data transforms operating states, network connections, equipment regulation boundaries, and historical control experience scattered across different systems into a unified structured data set and operating state samples, providing a complete data foundation for the accurate representation of complex operating conditions and the generation of control strategies. Secondly, constructing a distribution network knowledge graph based on the structured data set and performing knowledge reasoning transforms distribution network topology relationships, equipment attributes, operating constraints, and expert rules into associative and inferable knowledge features. Based on the current operating state, a set of safe actions is defined, eliminating unexecutable actions that may cause voltage overruns, branch overloads, energy storage charge state out-of-bounds, or switch malfunctions from the source of control action generation. Furthermore, under the constraints of the set of safe actions, offline constraint training is performed using operating state samples and knowledge features, enabling the control strategy model to learn different operating states and... While mapping relationships between control actions, the system is continuously constrained by equipment boundaries and safety rules, reducing the likelihood of unsafe actions generated during unconstrained exploration in deep reinforcement learning and improving the adaptability of control strategies to fluctuations in renewable energy output, load changes, and multi-equipment coupling conditions. Simultaneously, by generating original control actions through the control strategy model and performing matching corrections and safety checks on these actions, the system can correct control results where the action type, controlled object, and action amplitude do not conform to the current safe action range. This ensures that the final issued control actions retain the adjustment intent of the strategy model while meeting the actual operating boundaries of the distribution network. Finally, the distribution network knowledge graph is updated based on online execution results, and the control strategy model is updated through periodic retraining or event-triggered incremental optimization. This continuously supplements the knowledge representation and strategy learning process with new operating scenarios, equipment state changes, and control execution effects, enabling the control strategy to dynamically adjust with changes in the distribution network operating environment. This improves the safe generation and continuous adaptability of distribution network control actions in complex operating scenarios.

[0017] In summary, the technical solution adopted in this application can realize the safe adaptive generation of distribution network control strategies under complex operating scenarios. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this embodiment of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1This is an exemplary flowchart of a smart control method for power distribution networks based on knowledge graphs and deep reinforcement learning, provided in this application. Figure 2 This is based on the intelligent control architecture diagram of the power distribution network provided in this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] This application provides a method for intelligent control of distribution networks based on knowledge graphs and deep reinforcement learning. The core of this method is to acquire multi-source data from the distribution network, including operational data, topology parameters, equipment parameters, expert rules, and historical control cases. The multi-source data is preprocessed to obtain a structured data set and operational state samples. A distribution network knowledge graph is constructed based on the structured data set, and knowledge reasoning is performed to output knowledge features and a set of safety actions under the current operational state. Under the constraints of the safety action set, offline constraint training is performed using the operational state samples and the knowledge features to output a control strategy model for the distribution network. Original control actions are generated using the control strategy model, and these actions are matched and corrected before being executed after safety verification. The distribution network knowledge graph is updated based on the online execution results, and the control strategy model is updated using at least one of periodic retraining and event-triggered incremental optimization.

[0022] Example 1 To better understand the above technical solutions, a detailed description will be provided below with reference to the accompanying drawings and specific implementation methods. Figure 1 As shown in the figure, this is an exemplary flowchart of a smart control method for power distribution networks based on knowledge graphs and deep reinforcement learning according to this embodiment of the application, including the following steps: In step S1, multi-source data of the distribution network is acquired, including operating data, topology parameters, equipment parameters, expert rules, and historical control cases.

[0023] It should be noted that the multi-source data mentioned in this application refers to a dataset jointly provided by different business systems, monitoring terminals, and operation management data of the distribution network, and used for structured data conversion, operation status sample construction, and control strategy training; the operation data refers to data reflecting the changes in electrical quantities and equipment operation status at each acquisition time of the distribution network; the topology parameters refer to network structure parameters describing the connection relationships between feeders, bus nodes, branches, switches, transformers, tie lines, and various electrical devices; the equipment parameters refer to parameters reflecting the physical attributes, adjustable range, and safe operation boundaries of distribution network equipment; the expert rules refer to operation control knowledge used to limit the applicable scenarios, triggering conditions, control objects, recommended actions, action boundaries, and execution priorities of control actions; and the historical control cases refer to control experience data that records historical operation scenarios, control actions, action execution results, and safety constraint satisfaction.

[0024] In practical implementation, firstly, through the real-time communication interfaces, database interfaces, or file import interfaces of the distribution automation system, dispatch automation system, feeder terminal, distribution transformer terminal, smart meter, energy storage battery management system, photovoltaic inverter monitoring system, and charging pile monitoring terminal, node voltage, branch current, active power, reactive power, load power, distributed power output, energy storage charge status, switch status, and voltage regulation equipment tap position are read. Simultaneously, the acquisition time and data source of each data item are recorded. The read electrical quantity data and equipment status records are used as operational data. Among these, node voltage, branch current, active power, reactive power, load power, distributed power output, energy storage charge status, switch status, and voltage regulation equipment tap position are recorded. Branch current and power data can be acquired at sampling periods of 5 minutes, 15 minutes, or 30 minutes. Switch status, protection action status, and equipment alarm information are acquired using event-triggered methods. Secondly, feeder numbers, bus node numbers, branch numbers, switch numbers, transformer numbers, tie line numbers, and corresponding connection records are read from the distribution network geographic information system, production management system, distribution automation master station system, and feeder model files. Records describing the electrical connection relationships of each node and device are used as topology parameters. Simultaneously, data is also retrieved from equipment ledgers, equipment nameplates, control terminal parameter configuration files, and... The operation and maintenance records are used to read equipment capacity, rated voltage, rated current, adjustment range, action step size, action count limit, upper and lower limits of charging and discharging power, and upper and lower limits of energy storage state of charge. Records used to define the controllable range and operating boundaries of the equipment are used as equipment parameters. Next, applicable scenarios, triggering conditions, controlled objects, recommended actions, action boundaries, priorities, and exception conditions are extracted from dispatching operation procedures, distribution network operation control guidelines, field operation experience, historical dispatching records, protection setting sheets, and expert interview records. Rule records with completed content splitting and source identification are used as expert rules. Finally, case number, occurrence time, feeder number, operation scenario label, over-limit type, control action sequence, action execution result, voltage improvement, network loss change, safety constraint satisfaction, and operating cost change are read from historical dispatching logs, automatic control execution records, alarm handling records, equipment action records, and operation analysis reports. Records containing complete operation scenarios, control processes, and execution results are used as historical control cases. The data composed of the above operation data, topology parameters, equipment parameters, expert rules, and historical control cases are used as multi-source data of the distribution network.

[0025] In step S2, the multi-source data is preprocessed to obtain a structured data set and a running status sample.

[0026] In this embodiment, the preprocessing of the multi-source data to obtain a structured data set and a running status sample can be achieved through the following steps: The operational data, topology parameters, equipment parameters, expert rules, and historical control cases in the multi-source data are preprocessed to form basic data with a unified format. The basic data is subjected to entity extraction, relation extraction, and attribute mapping to convert unstructured and semi-structured content into a structured data set. The basic and derived features of the operating status are extracted from the basic data, and the operating status samples are determined based on the basic and derived features.

[0027] It should be noted that, in this application, the basic data refers to data that can be directly read by subsequent data processing after the time base, data format, identification coding, and measurement units have been unified; the structured data set refers to a data set organized according to the data structure of entities, relationships, attributes, and rules, and used to construct a distribution network knowledge graph; the basic features refer to features directly extracted from the operational data that reflect the current electrical quantities and equipment status of the distribution network; the derived features refer to features obtained based on the numerical relationships or temporal changes between basic features, and reflect the operational deviations, equipment adjustment margins, and degree of operational changes in the distribution network; and the operational status samples refer to distribution network operational status data organized according to a unified time section and fixed feature order, used for offline constraint training.

[0028] In practical implementation, firstly, the distribution network control cycle is used as the time alignment benchmark. Operational data is organized according to a joint index composed of equipment number and acquisition time. Periodically acquired data is assigned to the corresponding control cycle, and switch status, protection action status, and equipment alarm information are assigned to the control cycle in which the event occurred. For continuous measurement data such as node voltage, branch current, active power, and reactive power, data exceeding the physical value range is eliminated using the allowable operating boundaries in the equipment parameters. When a single control cycle is missing, linear interpolation is performed using two valid sample values ​​before and after the missing position. If the missing data is continuous for more than one control cycle, no corresponding time segment is selected to construct the operational status sample, and the data is mapped according to a preset unit. The radiation relationship converts similar electrical quantities to the same unit of measurement; the identification fields of topology parameters and equipment parameters are unified according to feeder number, bus node number, branch number, and equipment number, and the start node, end node, and access device in each topology connection record are checked to see if they can be associated with each other through the same number; expert rules are organized according to the field order of "applicable scenario - triggering condition - controlled object - recommended action - action boundary - priority - exception condition", and historical control cases are organized according to the field order of "case number - occurrence time - feeder number - operation scenario label - limit violation type - control action sequence - action execution result - safety constraint satisfaction status", and the completion time, format, identifier, and text are recorded. The unified data section serves as the foundational data. Secondly, using feeder number, bus node number, branch number, switch number, transformer number, equipment number, rule number, and case number as entity identification fields, data records corresponding to feeders, bus nodes, branches, switches, transformers, tie lines, distributed power sources, energy storage devices, reactive power compensation devices, on-load tap-changing transformers, expert rules, and historical control cases are extracted from the foundational data. The unique number and object type in each data record are used as entity data. Topology connection relationships are extracted according to the starting and ending objects in the topology connection records, equipment access relationships are extracted according to the equipment number and access node, and the applicable scenarios and control cases in the expert rules are extracted. The system extracts the trigger relationships and action recommendation relationships of the rules for objects and recommended actions. It extracts the case association relationships according to the feeder number, equipment number, operation scenario label and control action sequence in the historical control cases, and uses the extracted association records as relational data. Then, based on the entity number, it writes the equipment capacity, rated voltage, rated current, adjustment range, action step size, action number limit, upper and lower limits of charging and discharging power and energy storage state of charge upper and lower limits into the attribute fields of the corresponding entity. It also writes the trigger conditions, action boundaries, priorities and exception conditions in the expert rules into the attribute fields of the corresponding rule entity. The result of collecting entity data, relational data and attribute data according to a unified field structure is used as a structured data set.Finally, following the same control cycle, node voltage, branch current, active power, reactive power, energy storage state of charge, switch status, voltage regulator settings, load data, and distributed generation output are read from the basic data, and these data are used as basic characteristics. The difference between the node real-time voltage and the rated voltage or the median of the safe operating range is used as the voltage deviation; the ratio of the branch real-time current to the branch's allowable current carrying capacity is used as the branch load rate; the remaining range of the current energy storage state of charge relative to the upper and lower limits of the energy storage state of charge is used as the remaining energy storage capacity; the load change amplitude of adjacent control cycles is used as the load change rate; and the change amplitude of distributed generation output within continuous control cycles is used as the renewable energy volatility. The current control cycle is then used as the data for the next control cycle. The differences in voltage regulation equipment settings, reactive power compensation switching status, energy storage charging / discharging power, and switching status between the previous control cycle and the previous cycle are used as equipment action changes. Voltage deviation, branch load rate, remaining energy storage capacity, load change rate, renewable energy volatility, and equipment action changes are used as derived features. Basic and derived features with fixed upper and lower limits are transformed to a unified numerical range using the minimum-maximum normalization method. The load change rate and renewable energy volatility are eliminated by the mean-variance normalization method. Switching status and reactive power compensation switching status are encoded using binary encoding. The normalized and encoded basic and derived features are arranged in a preset field order, and the resulting feature sequence is used as the operating status sample.

[0029] In step S3, a power distribution network knowledge graph is constructed based on the structured data set, and knowledge reasoning is performed to output the knowledge features and safety action set under the current operating state.

[0030] In this embodiment, constructing a distribution network knowledge graph based on the structured data set, performing knowledge reasoning, and outputting the knowledge features and safety action set under the current operating state can be achieved through the following steps: The structured data set is deduplicated, and the topological connection relationships, device access relationships, rule triggering relationships and case association relationships between each entity are extracted to generate a power distribution network knowledge graph. The operation constraints of the distribution network are converted into safety constraint functions, and the expert rules are converted into rule judgment functions. Then, the safety constraint functions and the rule judgment functions are attached to the distribution network knowledge graph. An initial action candidate set is generated based on the current operating status. The candidate actions in the initial action candidate set are filtered by calling the distribution network knowledge graph. After verification by operating safety constraints, a set of safe actions under the current operating status is generated. Based on the current operating status, topological relationships, equipment constraints, expert rules, and historical similar case information related to the current operating status are extracted from the distribution network knowledge graph to form knowledge features of the current operating status.

[0031] It should be noted that the distribution network knowledge graph described in this application represents a knowledge structure that organizes the distribution network structure, equipment status, operating boundaries, expert rules, and historical control cases in terms of entities, entity attributes, and relationships between entities; the topology connection relationship represents the electrical connection relationship between feeders, bus nodes, branches, switches, transformers, and tie lines; the equipment access relationship represents the attribution relationship between distribution network equipment and access nodes and their respective feeders; the rule triggering relationship represents the association relationship between expert rules and applicable scenarios, triggering conditions, and control objects; the case association relationship represents the association relationship between historical control cases and corresponding feeders, nodes, equipment, expert rules, and operating scenarios; the security constraint function represents an executable judgment function that verifies whether control actions conform to the distribution network operating boundaries; the rule The judgment function represents an executable judgment function that verifies whether a control action conforms to the applicable scenario, triggering condition, controlled object, action type, action boundary, priority, and exception conditions in the expert rules; the topology relationship represents the connection relationship between feeders, nodes, branches, and equipment related to the current operating state in the distribution network knowledge graph; the equipment constraints represent the equipment capacity, adjustment range, action step size, action number limit, and safe operating boundary related to the current operating state; the historical similar case information represents historical control case information that matches the current operating state in terms of operating scenario, over-limit type, affected area, and controlled object; and the knowledge feature represents the feature obtained by organizing the topology relationship, equipment constraints, expert rules, and historical similar case information related to the current operating state in a unified field order.

[0032] In practice, firstly, the entity number, entity type, entity name, feeder, node, and equipment number are read from the structured data set. The feeder number, node number, equipment number, rule number, and case number are used as unique identifiers for the corresponding entities. Entity records with the same unique identifier from different data sources are merged into the same entity. When entity records from different data sources do not contain the same unique identifier, the equipment number, access node, equipment type, and entity name are compared sequentially. The corresponding entities are merged only when all compared fields point to the same distribution network object, and the attribute fields and update times from each data source are written into the merged entity. Finally, the starting entity, ending entity, and connection are read according to the node branch connection matrix or topology connection table. In the following direction, the read electrical connection records are used as topology connection relationships. Based on the equipment number, equipment access node, and associated feeder, the equipment ownership records are read and used as equipment access relationships. Based on the applicable scenarios and triggering conditions in the expert rules, the association records between the expert rules and the controlled objects are read and used as rule triggering relationships. Based on the operating scenarios, limit exceedance types, affected areas, and controlled objects in historical control cases, the association records between historical control cases and feeders, nodes, equipment, expert rules, and operating scenarios are read and used as case association relationships. Following the triplet format of "starting entity—relationship type—ending entity," the deduplicated entities are used as the graph. The graph nodes are defined as follows: topology connections, device access relationships, rule triggering relationships, and case association relationships are used as relationship edges between graph nodes. The attribute fields of each entity are written into the corresponding graph node. The knowledge structure containing the graph nodes, relationship edges, and entity attributes is then used as the distribution network knowledge graph. Next, the node voltage upper and lower limits, branch allowable current carrying capacity, energy storage charge state upper and lower limits, equipment capacity, adjustment range, action step size, and action count limits are read from the device parameters. The node connection relationships, branch connection relationships, and switch connection status are also read from the topology parameters. The current operating status and the control action to be verified are set as function inputs. The parameters for node voltage being within the allowable upper and lower limits, branch current not exceeding the allowable current carrying capacity, and energy storage charge state being within the allowable upper and lower limits are also considered. Within the safety upper and lower limits, the equipment actions do not exceed the equipment capacity, adjustment range, action step size, and action number limits, and the switching actions do not cause unplanned islanding or ring network operation. These are set as the verification pass conditions. The executable condition expression configured according to the above function input, verification conditions, and output results is used as the safety constraint function. The applicable scenarios and triggering conditions in the expert rules are set as the rule entry point. The control object, action type, action boundary, and priority are set as comparison fields. The exception conditions are set as the prohibition pass conditions. When the control action to be verified meets all comparison fields and does not trigger the exception conditions, the rule pass result is output. The executable condition expression configured according to the above rule entry point, comparison fields, and output results is used as the rule judgment function.Based on node number, branch number, equipment number, and rule number, safety constraint functions are associated with corresponding nodes, branches, equipment, and topology relationships. Rule judgment functions are associated with corresponding expert rules, control objects, and action types, completing the mounting of safety constraint functions and rule judgment functions in the distribution network knowledge graph. Next, an initial action candidate set is generated based on the current operating state. The distribution network knowledge graph is used to filter candidate actions in the initial action candidate set, and after verification by operating safety constraints, a set of safe actions for the current operating state is generated. Finally, based on the feeder number, node number, branch number, and equipment number in the current operating state, corresponding entities are retrieved in the distribution network knowledge graph. Connection records between relevant feeders, nodes, branches, and equipment are read along the topology connection relationships. These connection records are used as topology relationships. The capacity, adjustment range, action step size, action count limit, and safe operating boundary of relevant equipment are read along the equipment access relationships. The process involves several steps: First, the read attribute records are used as device constraints. Then, the running scenarios and status values ​​in the current running state are compared item by item with the applicable scenarios and triggering conditions in the rule triggering relationship. The control objects, action types, action boundaries, priorities, and exception conditions that meet the triggering conditions are read, and the read rule records are used as expert rules. Next, the running scenarios, over-limit types, affected areas, and control objects in the current running state are compared item by item with the corresponding fields in the case association relationship. Historical control cases are arranged from highest to lowest based on the number of identical fields. When the number of identical fields is the same, they are arranged from most recent to oldest based on the case occurrence time. The control actions, action execution results, and safety constraint satisfaction status of the top-ranked historical control cases are used as historical similar case information. Finally, the connection records in the topology relationship, attribute records in the device constraints, rule records in the expert rules, and historical similar case information are arranged in a fixed field order, and the resulting arrangement is used as the knowledge features of the current running state.

[0033] In this embodiment, generating an initial action candidate set based on the current operating state, filtering candidate actions in the initial action candidate set by calling the distribution network knowledge graph, and then generating a set of safe actions for the current operating state after verification by operating safety constraints can be achieved through the following steps: Based on the current operating status, enumerate the action options of various controllable devices to form an initial action candidate set; The initial action candidate set is filtered by calling the expert rules in the power distribution network knowledge graph, and the candidate actions that do not meet the requirements of applicable scenario, control object, action type and expert rule priority are eliminated; Based on the equipment operating boundary, the candidate actions obtained from the filtering are trimmed, and the actions are restricted to the constraints of equipment capacity, adjustment range, action step size and number of actions. The candidate actions after trimming are verified by voltage constraints, branch current constraints, energy storage state of charge constraints and topology safety constraints. The candidate actions that pass all the verifications are taken as the set of safe actions under the current operating state.

[0034] It should be noted that, in this application, the action options refer to the control action types and values ​​that various controllable devices can select under the current operating state; the initial action candidate set refers to the set of candidate actions obtained after enumerating the action options of various controllable devices; the filtered candidate actions refer to candidate actions that meet the requirements of applicable scenarios, controlled objects, action types, and expert rule priorities in the expert rules; the device operating boundary refers to the executable range of actions jointly limited by device capacity, adjustment range, action step size, and action number constraints; the trimmed candidate actions refer to the candidate actions retained after correcting the action amplitude or eliminating invalid actions according to the device operating boundary; the voltage constraint refers to the operating restriction that the node voltage is within the allowable voltage range; the branch current constraint refers to the operating restriction that the branch current does not exceed the allowable current carrying capacity; the energy storage state of charge constraint refers to the operating restriction that the energy storage state of charge is within the safe upper and lower limits; the topology safety constraint refers to the network connection restriction that the distribution network does not experience unplanned islanding or ring network operation after the switching action; and the safety action set refers to the set of candidate actions that have passed all verifications of voltage constraints, branch current constraints, energy storage state of charge constraints, and topology safety constraints.

[0035] In practice, firstly, the device number, switch status, on / off status, voltage regulating device level, distributed power output, energy storage charge status, and controllable load status of various controllable devices are read from the current operating status, and the corresponding device parameters are associated according to the device number. For switching equipment, reactive power compensation devices, and on-load tap-changing transformers, the switchable status or level is listed item by item according to the current switch status, on / off status, and voltage regulating device level. For distributed power sources, energy storage devices, and controllable loads, starting from the current power value, the adjustment values ​​are listed step by step in both increasing and decreasing directions according to the action step size. Each adjustment value records the action type, controlled object, and controllable load status. The action direction and amplitude are used to select the recorded control actions as action options, and the action options of various controllable devices are aggregated to form an initial action candidate set. Secondly, based on the node voltage, branch current, energy storage charge state, and switch state in the current operating state, expert rules matching the current operating state are retrieved from the distribution network knowledge graph through rule-triggered relationships. The action type, controlled object, and action direction in the initial action candidate set are compared with the applicable scenario, controlled object, and action type in the expert rules, respectively, and candidate actions that do not match the applicable scenario, control object, or action type are eliminated. When the same controlled object matches... When multiple expert rules are available, the corresponding candidate actions are arranged from highest to lowest priority. For candidate actions with opposite directions or those that cannot be executed simultaneously within the same control cycle, the candidate action with the higher expert rule priority is retained. The candidate actions retained after the above field comparison and expert rule priority filtering are used as the filtered candidate actions. Next, based on the equipment number in the filtered candidate actions, the corresponding equipment capacity, adjustment range, action step size, and action count constraints are read. For candidate actions whose action amplitude exceeds the equipment capacity or adjustment range, the action amplitude is corrected to the boundary value within the corresponding equipment operating boundary closest to the original action amplitude. Candidate actions whose amplitude does not conform to the action step size are corrected to the nearest integer multiple of the action step size in a direction not exceeding the original action amplitude. For candidate actions where the gear change of the voltage regulating device exceeds the single allowable adjustment range, the gear change is corrected to the single allowable adjustment range. Then, the number of actions executed by the corresponding device within the statistical period limited by the action count constraint is read, and the number of actions executed is added to the number of actions to be executed corresponding to the candidate action. When the sum exceeds the action count constraint, the corresponding candidate action is removed. The candidate actions retained after completing the action amplitude correction, action step size correction, gear change correction, and action count verification are used as the trimmed candidate actions.Finally, for each trimmed candidate action, the current operating status, reactive power compensation switching status, voltage regulation equipment level, distributed generation output, energy storage charging / discharging power, or controllable load power are updated according to the action type. The updated node load, distributed generation output, energy storage charging / discharging power, reactive power compensation switching status, voltage regulation equipment level, and topology connection relationship are then input into the distribution network power flow calculation program. When using the forward-backward substitution power flow calculation method, the power node voltage is used as the initial value for calculation. The branch current corresponding to the injected power at each node is accumulated along the branch from the end node to the power node. Then, the node voltage is updated step by step from the power node to the end node based on the branch impedance and branch current. The above process is repeated until the maximum node voltage difference between two adjacent calculations is less than the voltage convergence accuracy preset by the distribution network power flow calculation program. The node voltage output by the power flow calculation is compared with the corresponding voltage constraints item by item, and the branch current output by the power flow calculation is compared with the corresponding branch current constraints item by item. The comparison process is as follows: For energy storage charging and discharging power adjustment actions, the change in electricity within the control cycle is calculated based on the current energy storage state of charge, energy storage charging and discharging power, control cycle length, and charging and discharging efficiency. During charging, the change in electricity is included in the current energy storage state of charge; during discharging, the change in electricity is deducted from the current energy storage state of charge. The energy storage state of charge after the action is then compared with the energy storage state of charge constraints. For switch opening and closing actions, the connection relationships between nodes and branches are updated according to the switch state after the action. A depth-first traversal is performed on the updated connection relationships, starting from the power node. If there are untraversed load nodes, it is considered an unplanned islanding; if a visited node other than the previous node is encountered during the traversal, it is considered a ring network operation. Candidate actions that fail any verification are eliminated. Candidate actions that pass all verifications—voltage constraints, branch current constraints, energy storage state of charge constraints, and topology safety constraints—are taken as the set of safe actions for the current operating state.

[0036] In step S4, under the constraints of the set of safety actions, offline constraint training is performed using the operating state samples and the knowledge features to output the control strategy model of the power distribution network.

[0037] In this embodiment, under the constraints of the set of safety actions, offline constraint training is performed using the operating state samples and the knowledge features to output the control strategy model of the distribution network. This can be achieved through the following steps: The running state samples are fused with the knowledge features to construct a reinforcement learning state; Using the reinforcement learning state as input to the policy network, a hybrid action space containing discrete action output branches and continuous action output branches is constructed, and the exploration range of the policy network is limited by the set of safe actions. Construct the policy loss function; The policy network is iteratively trained using the reinforcement learning state and the set of safety actions, combined with the policy loss function, until the convergence condition is met, and the distribution network control policy model is output.

[0038] It should be noted that, in this application, the reinforcement learning state refers to the state input obtained by fusing the operating state samples and knowledge features at the same control moment; the reward function refers to the function that evaluates the comprehensive benefit of control actions based on network loss cost, voltage deviation cost, equipment action cost, and operating cost; the constraint cost function refers to the function that evaluates the degree of violation of safety constraints of control actions based on voltage over-limit cost, branch current over-limit cost, energy storage charge state over-limit cost, and expert rule violation cost; the joint optimization objective refers to the training objective of improving the output of the reward function and reducing the output of the constraint cost function; the policy loss function refers to the function composed of policy benefit term and constraint cost term, which evaluates the policy network parameter update direction; the convergence condition refers to the stopping training condition corresponding to the offline constraint training reaching the requirements of average cumulative reward change, constraint cost, safety constraint violation rate, control action safety pass rate, or network loss change; and the distribution network control strategy model refers to the strategy model that outputs distribution network control actions based on the reinforcement learning state after completing offline constraint training.

[0039] In practice, firstly, the operating state samples and knowledge features are time-aligned according to the feeder number and control cycle. Operating state samples and knowledge features corresponding to the same feeder at the same control time are selected, maintaining the order of basic and derived features in the operating state samples, and maintaining the order of topological relationships, equipment constraints, expert rules, and historical similar case information in the knowledge features. The two parts of data are then concatenated along the feature dimension to construct the reinforcement learning state according to the following formula: in, Indicates control time The corresponding reinforcement learning state, Indicates control time The corresponding running status sample, Indicates control time The corresponding knowledge features; the spliced ​​feature sequence is used as the reinforcement learning state, so that the power distribution network operation information in the operation state sample and the knowledge reasoning results in the knowledge features remain corresponding in the same state input.

[0040] Secondly, using the reinforcement learning state as the input to the policy network, a hybrid action space containing discrete action output branches and continuous action output branches is constructed, and the exploration range of the policy network is limited by the set of safe actions. Next, the network loss, node voltage, equipment action changes, and operating costs before and after the execution of the corresponding control action for each training sample are read. Each cost is then normalized to its minimum and maximum values ​​within the training sample set. The ratio of the current network loss to the baseline network loss is used as the network loss cost; the deviation of each node voltage from the reference voltage is used as the voltage deviation cost; the change in each control action within adjacent control cycles is used as the equipment action cost; and the sum of the electricity purchase cost, curtailment cost, energy storage charging and discharging cost, and controllable load adjustment cost is used as the operating cost. The reward function is then output according to the following formula: in, Indicates control time The corresponding reward function output, Indicates network loss cost, Indicates the cost of voltage deviation. Indicates the cost of equipment operation. This represents the operating cost; the voltage deviation cost is calculated using the following formula: in, The weighting factor represents the cost of voltage deviation. Represents the set of nodes in the distribution network. Represents a node During control time node voltage, Represents a node The reference voltage; calculate the equipment operating cost according to the following formula: in, The weighting coefficient represents the cost of equipment operation. Represents the controlled object During control time Controlling actions, Represents the controlled object The control actions at the previous control moment; set according to the distribution network operation control objectives. and When the control objective focuses on reducing voltage deviation, improve When the control objective focuses on reducing frequent equipment operations, improving And keep the corresponding weight coefficients unchanged during the same offline constraint training process.

[0041] Read the node voltage, branch current, energy storage state of charge, and expert rule verification results after the control action is executed, and calculate the constraint cost function according to the following formula: in, Indicates control time The corresponding constraint cost function output, Indicates the cost of exceeding voltage limits This indicates the cost of exceeding the branch current limit. This indicates the cost of exceeding the state of charge limit for energy storage. This indicates the cost of violating expert rules; the cost of exceeding voltage limits is calculated using the following formula: in, Represents a node The upper limit of voltage, Represents a node The lower voltage limit; when the node voltage is between the upper and lower voltage limits, the voltage over-limit cost for the corresponding node is zero. When the node voltage exceeds the upper voltage limit or falls below the lower voltage limit, the excess portion is included in the voltage over-limit cost. The branch current over-limit cost is calculated according to the following formula: in, Indicates a branch During control time The branch current, Indicates a branch The allowable current carrying capacity; when the branch current does not exceed the allowable current carrying capacity, the branch current over-limit cost for the corresponding branch is zero; when the branch current exceeds the allowable current carrying capacity, the excess portion is included in the branch current over-limit cost. The energy storage state of charge over-limit cost is calculated according to the following formula: in, Indicates energy storage device During control time The state of charge of energy storage, Indicates energy storage device The upper limit of the state of charge of energy storage, Indicates energy storage device The upper and lower limits of the energy storage state of charge; when the energy storage state of charge is within the safe upper and lower limits, the cost of exceeding the limit for the corresponding energy storage device is zero; when the energy storage state of charge exceeds the safe upper and lower limits, the excess portion is included in the cost of exceeding the limit. The cost of violating the expert rule is calculated according to the following formula: in, Indicates the first Expert rules for reinforcement learning states and control actions The result of the rule judgment, The indicator function is represented by a variable; when the rule judgment result is not satisfied, the corresponding indicator function is set to one, and when the rule judgment result is satisfied, the corresponding indicator function is set to zero. The sum of all the times that the expert rule is not satisfied is taken as the cost of violating the expert rule.

[0042] The power distribution network control process can be expressed as a constrained Markov decision process according to the following formula: in, This indicates a constrained Markov decision process. Representing the state space, Represents the action space, Represents the state transition function. Represents the reward function, Represents the constraint cost function. This represents the discount factor; the optimization objective of the reward function is to maximize the cumulative reward within the control period. in, Representation Strategy This represents the cumulative reward corresponding to the strategy. Indicates according to strategy The generated state-action trajectory This represents the number of control moments contained in the trajectory; the optimization objective of the constraint cost function is to ensure that the cumulative constraint cost does not exceed the constraint threshold. in, This represents the cumulative constraint cost corresponding to the strategy. This represents the permissible constraint threshold. The constraint threshold is set based on the allowable upper and lower limits of node voltage, allowable current carrying capacity of branches, safe upper and lower limits of energy storage state of charge, and expert rule requirements. When it is required that no safety constraint violations occur during the training process, the constraint threshold is set to a non-negative number close to zero. The policy benefit term and constraint cost term are combined into the policy loss function according to the following formula: in, Represents the policy loss function. Indicates the policy network parameters, This represents the strategy payoff term based on the advantage function. Represents the constraint cost term. This represents the constraint penalty coefficient. When the cumulative constraint cost exceeds the constraint threshold, the constraint penalty coefficient is increased, causing the policy network to reduce the output probability of the corresponding control action. When the cumulative constraint cost does not exceed the constraint threshold, the constraint penalty coefficient is maintained or decreased. The resulting function is used as the policy loss function.

[0043] Finally, a constrained actor-critic algorithm framework is adopted, and a policy update method based on proximal policy optimization (PPO) is used to organize historical operating data, historical control cases, and simulation interaction samples into offline training samples according to the following formula: in, Indicates control time The corresponding offline training samples, Represents a set of safety actions. This represents the original action output by the policy network. Indicates the actions after being restricted by the set of safe actions. This represents the output of the reward function. This represents the output of the constraint cost function. This represents the next reinforcement learning state after action execution or simulation verification. The reinforcement learning state is input into the policy network. The action selection range is limited by the set of safe actions. The value network is called to estimate the cumulative reward corresponding to the reinforcement learning state, and the difference between the actual cumulative reward and the value network estimation result is used as the advantage function input. Simultaneously, the constraint cost network is called to accumulate the constraint cost at each control time step. The policy network parameters are updated according to the following formula: And ensure that the updated policy network parameters satisfy: in, Indicates the first Rounds of iterative training of the corresponding policy network parameters Indicates the first The parameters of the policy network are trained in each iteration. Indicates network parameters based on the current policy The obtained advantage function Indicates policy network parameters The corresponding cumulative constraint cost; the gradient of the policy loss function relative to the policy network parameters is calculated through backpropagation, and the policy network parameters are updated in the direction of increasing the policy reward term and reducing the constraint cost term. After each round of iteration training, the average cumulative reward, constraint cost, safety constraint violation rate, control action safety pass rate, policy network loss function, and value network loss function are statistically analyzed. When at least one of the following conditions occurs within several consecutive rounds of iteration training: the average cumulative reward change is less than a preset threshold, the constraint cost is stably lower than the constraint threshold, the safety constraint violation rate is lower than a preset proportion, the control action safety pass rate reaches a preset requirement, and the changes in the policy network loss function and value network loss function tend to stabilize, the corresponding training result is taken as the result that satisfies the convergence condition, offline constraint training is stopped, and the completed policy network structure and policy network parameters are taken as the distribution network control policy model according to the following formula: in, Indicates the policy network parameters Next, based on the reinforcement learning state Output control action Distribution network control strategy model.

[0044] In this embodiment, using the reinforcement learning state as input to the policy network, constructing a hybrid action space containing discrete action output branches and continuous action output branches, and limiting the exploration range of the policy network with the set of safe actions can be achieved through the following steps: The reinforcement learning state is used as the input to the policy network, and a joint representation is extracted through a shared feature extraction layer; Using the joint representation as input, discrete action probabilities are output through the discrete action output branch, and continuous control parameters are output through the continuous action output branch, thus forming a hybrid action space; The discrete action output branches are masked and filtered using the set of security actions, and the continuous action output branches are pruned according to the device operation boundary, thus limiting the exploration space of the policy network to the range of the set of security actions.

[0045] It should be noted that, in this application, the shared feature extraction layer refers to a network layer that performs common feature mapping on the running state samples and knowledge features in the reinforcement learning state; the joint representation is the feature vector output of the shared feature extraction layer and jointly input to the discrete action output branch and the continuous action output branch; the discrete action output branch refers to the network branch that outputs the probability of actions corresponding to switching actions, reactive power compensation device switching states, and voltage regulation equipment gear positions; the continuous action output branch refers to the network branch that outputs the active power regulation amount of distributed power sources, the reactive power regulation amount of distributed power sources, and the charging and discharging power of energy storage; the discrete action probability refers to the probability that each discrete action is selected by the policy network in the current reinforcement learning state; the continuous control parameter refers to the control value corresponding to each continuous action in the current reinforcement learning state; the hybrid action space refers to the range of action values ​​composed of discrete actions and continuous actions; the mask filtering refers to the filtering method of filtering out discrete actions that do not belong to the safe action set and retaining discrete actions that belong to the safe action set; the exploration space refers to the range of actions that the policy network can sample during iterative training.

[0046] In specific implementation, firstly, the running state samples and knowledge features are read according to the fixed arrangement order of the fields in the reinforcement learning state. The read values ​​are then written into the policy network input vector sequentially, ensuring that the length of the policy network input vector is consistent with the total number of fields contained in the reinforcement learning state. The shared feature extraction layer performs weighted combination of each field in the policy network input vector through fully connected operations. This establishes feature associations between information such as node voltage, branch current, active power, reactive power, energy storage state of charge, switch state, and voltage regulation equipment level in the running state samples and topological relationships, equipment constraints, expert rules, and historical similar case information in the knowledge features within the same network layer. Then, these associations are processed sequentially through a linear rectification function. Elements retain output values ​​greater than zero and set output values ​​less than or equal to zero to zero; the output dimension of the shared feature extraction layer is set to the input dimension jointly received by the discrete action output branch and the continuous action output branch, and the feature vectors that have completed the fully connected operation and the linear rectification operation are used as the joint representation; next, the joint representation is input into the discrete action output branch and the continuous action output branch respectively, wherein the discrete action output branch is set with the same number of output units as the number of selectable actions of the switching action, the switching state of the reactive power compensation device, and the voltage regulating equipment position, and each output unit corresponds to one discrete action; an exponential operation is performed on the output value of each output unit, and then each exponential operation result is divided by the sum of all exponential operation results. The sum of all outputs is equal to one. The normalized output is used as the discrete action probability, and a mapping relationship is established between the discrete action probability and the corresponding discrete action according to the discrete action number. The continuous action output branch sets up output units according to the number of control variables of distributed power source active power regulation, distributed power source reactive power regulation, and energy storage charging and discharging power. One set of output units gives the mean of each continuous action, and another set of output units gives the standard deviation of each continuous action. The standard deviation output value is restricted to a value greater than zero after non-negative mapping. During iterative training, random sampling is performed according to the mean and standard deviation of each continuous action, and the sampled values ​​are used as continuous control parameters. The discrete action... The discrete action values ​​corresponding to the probabilities and the continuous action values ​​corresponding to the continuous control parameters are used together as the hybrid action space. Finally, according to the arrangement order of each discrete action in the discrete action output branch, the set of safe actions is converted into a mask sequence of the same length. Discrete actions belonging to the set of safe actions are assigned a value of one, and discrete actions not belonging to the set of safe actions are assigned a value of zero. The discrete action probabilities are multiplied by the corresponding mask values ​​one by one, and discrete actions with a product of zero are eliminated. The remaining discrete action probabilities are then normalized again based on the sum of the remaining discrete action probabilities. The result of the renormalization is used as the discrete action probabilities after mask filtering, so that the policy network only samples from discrete actions with a mask value of one.Based on the device number corresponding to the continuous actions, the device operating boundaries corresponding to the active power regulation, reactive power regulation, and energy storage charging / discharging power of the distributed power source are read from the set of safe actions and device parameters. When the continuous control parameter is lower than the lower limit of the device operating boundary, the lower limit value is used as the pruned continuous control parameter; when the continuous control parameter is higher than the upper limit of the device operating boundary, the upper limit value is used as the pruned continuous control parameter; when the continuous control parameter is within the device operating boundary, its original value remains unchanged. The discrete action probabilities after masking and the pruned continuous control parameters are used as the basis for the policy network to sample actions in the current reinforcement learning state, ensuring that both discrete and continuous actions are within the set of safe actions, and restricting the exploration space of the policy network to the set of safe actions.

[0047] In step S5, the original control action is generated through the control strategy model, and the original control action is matched and corrected. After security verification, it is sent out for execution.

[0048] In this embodiment, the process of generating original control actions through the control strategy model, matching and correcting the original control actions, and then issuing them for execution after security verification can be achieved through the following steps: The control strategy model is invoked based on the real-time operating status to generate the original control actions; The original control action is matched with the actions in the set of safety actions. If the match is successful, the matched action is used as a candidate control action. If a match fails, the original control action is modified according to the set of safety actions to obtain candidate control actions; The candidate control actions are subjected to operational safety constraint verification and expert rule constraint verification. Once the verification is passed, the actions are issued for execution.

[0049] It should be noted that, in this application, the candidate control action refers to the control action to be verified obtained after the original control action is matched or modified by the set of safety actions; the operation safety constraint verification refers to the process of verifying the safety boundary of the node voltage, branch current, energy storage state of charge, equipment action boundary and topology state after the candidate control action is executed; the expert rule constraint verification refers to the process of verifying whether the candidate control action conforms to the operation sequence, action priority and prohibition conditions under the current operation scenario.

[0050] In specific implementation, firstly, the control strategy model is invoked based on the real-time operating status to generate original control actions; secondly, the action type, control device, action direction, and action amplitude in the original control actions are read. The action type includes voltage regulation level adjustment, reactive power compensation switching, energy storage charging and discharging power adjustment, distributed power reactive power adjustment, and switch opening and closing operations. The control device is represented by the corresponding device number, and the action amplitude is represented by the level change, number of switching groups, charging and discharging power value, reactive power adjustment amount, or switch target state. Following the order of action type, control device, and action amplitude, the original control actions are compared item by item with the actions in the safety action set. When an action type exists in the safety action set... When actions of the same type, control device, and amplitude are within the corresponding safety boundary, the matched action is used as a candidate control action. Next, if no action in the safety action set simultaneously meets the requirements for action type, control device, and amplitude, actions with the same action type and control device as the original control action are first selected from the safety action set. The absolute difference between each action amplitude and the original control action amplitude is calculated, and the action with the smallest absolute difference is used as a candidate control action. If no action with the same action type and control device is found in the safety action set, control devices that can act on the current over-limit area are retrieved along the control action relationships in the distribution network knowledge graph, and then the action is selected according to expert rules. Actions are prioritized from highest to lowest. When expert rule priorities are the same, the action with the same direction as the original control action and the smallest difference in action amplitude is selected as the candidate control action. For voltage regulation gear adjustment, when the original control action exceeds the allowable gear range or the maximum single adjustment step size, the allowable gear closest to the original gear is selected. When the original gear increase or decrease direction would cause the node voltage to deviate further from the safe operating range, the allowable gear in the opposite direction from the safe action set is selected or the current gear is maintained. For reactive power compensation switching, when the original number of switching groups exceeds the allowable switching range, or does not meet the switching interval and action number limits, the switching group within the allowable range that is closest to the original number of switching groups is selected. For the number of switching operations, when the corresponding switching action would cause voltage overcompensation, reduce the number of switching groups or select the safety action of canceling the switching; for the regulation of energy storage charging and discharging power, when the original charging and discharging power exceeds the rated power of energy storage or the boundary of energy storage state of charge, adjust the charging and discharging power to the allowable value closest to the original charging and discharging power in the set of safety actions; when the energy storage state of charge is close to the lower limit, select the action of limiting discharge or zero power; when the energy storage state of charge is close to the upper limit, select the action of limiting charging or zero power; for the reactive power regulation of distributed power sources, when the original reactive power regulation exceeds the regulation range corresponding to the inverter capacity and power factor, adjust the reactive power regulation to the allowable value closest to the original reactive power regulation in the set of safety actions.For switch opening and closing operations, if the original control action would cause unplanned islanding, ring network operation, or branch overload, the corresponding switch opening and closing operation is cancelled, and a switch action that meets the topology safety requirements is selected as a candidate control action from the set of safety actions. Finally, the real-time operating status, topology parameters, equipment parameters, and candidate control actions are input into the distribution network power flow calculation program. The reactive power compensation switching status, voltage regulation equipment tap, distributed power output, energy storage charging and discharging power, and switch status are updated according to the candidate control actions. The node voltage and branch current after the action are obtained through power flow calculation. The node voltage is compared with the upper and lower allowable voltage limits item by item, and the branch current is compared with the corresponding branch allowable current carrying capacity item by item. The energy storage state of charge is updated according to the current energy storage state of charge, candidate charging and discharging power, control cycle length, and charging and discharging efficiency. The updated energy storage state of charge is compared with the upper and lower limits of the energy storage state of charge. The candidate control actions are calculated according to the equipment capacity, adjustment range, and action steps. The length and number of actions limit the verification of equipment action boundaries; for switch opening and closing operations, the node and branch connection relationships are updated according to the switch status after the action. Starting from the power node, the updated topology connection relationships are traversed. When all load nodes can be reached from the power node and no closed loop appears during the traversal, the topology safety constraint verification result is recorded as passed; then, expert rules matching the real-time operating status in the distribution network knowledge graph are called, and the control object, action type, action direction, and action amplitude of the candidate control action are compared item by item with the operation sequence, action priority, and prohibition conditions in the expert rules; when the node voltage, branch current, energy storage state of charge, equipment action boundary, and topology status all meet the operating safety constraints, and the candidate control action conforms to the expert rules, the candidate control action is executed, and the execution action is issued to the corresponding control equipment according to the equipment number; if any verification fails, the candidate control action is not issued, and the matching correction process is returned to reselect the candidate control action.

[0051] In this embodiment, generating the original control action by invoking the control strategy model based on the real-time operating status can be achieved through the following steps: The real-time operating status is generated based on the real-time collected power distribution network operation data. Based on the real-time operating status, the power distribution network knowledge graph is invoked to perform online knowledge reasoning, thereby obtaining online knowledge features; The real-time operating status is fused with the online knowledge features and then input into the control strategy model to generate the original control action.

[0052] It should be noted that, in this application, the real-time operating status refers to the status data reflecting the node voltage, branch current, active power, reactive power, energy storage state of charge, and equipment status at the current control moment; the online knowledge reasoning refers to the knowledge processing process of retrieving related entities, relationships, and attributes in the distribution network knowledge graph based on the real-time operating status, and calling matching expert rules and historical control cases; the online knowledge features refer to the knowledge features obtained by organizing the topological relationships, equipment constraints, expert rules, and historical similar case information that match the real-time operating status in a fixed field order; and the original control action refers to the control action directly output by the control strategy model based on the real-time operating status and online knowledge features, and has not yet been corrected by matching the safety action set.

[0053] In practice, firstly, real-time power distribution network operation data is read according to the current control cycle. Duplicate records are deleted using a composite index of equipment number, data type, and timestamp. The dimensions of node voltage, branch current, active power, reactive power, and energy storage charging / discharging power are standardized according to the rated units recorded in the equipment parameters. Continuous measurements are compared item by item with the valid value range in the equipment parameters. Continuous measurements exceeding the valid value range are treated as missing data. When a continuous measurement field is missing, the measurement value from the most recent valid control cycle for the same equipment is read. When switch status, switching status, or voltage regulating equipment tap position is missing, the status value from the most recent valid control cycle for the same equipment is used. The process is repeated according to the control cycle. The system performs time alignment on records from each device, and reads node voltage, branch current, active power, reactive power, energy storage state of charge, switch status, voltage regulator settings, load, and distributed generation output as basic characteristics. Voltage deviation is calculated based on the difference between node voltage and reference voltage; branch load rate is calculated based on the ratio of branch current to allowable current carrying capacity; remaining energy storage capacity is obtained based on the distance between the energy storage state of charge and its upper and lower limits; load change rate is calculated based on the load difference between the current and previous control cycles; and renewable energy output fluctuation is calculated based on the distributed generation output difference between adjacent control cycles. Finally, the system considers the voltage regulator settings and reactive power compensation in adjacent control cycles. The changes in switching status, energy storage charging and discharging power, and switch status are used to obtain changes in equipment operation. Normalization parameters saved during the offline constraint training phase of the control strategy model are used to normalize the basic and derived features respectively. The normalization results are then arranged according to the field order of the operating state samples, and the resulting state data is used as the real-time operating state at the current moment. Next, the feeder number, node number, branch number, and equipment number associated with the real-time operating state are read. The corresponding entities are located in the distribution network knowledge graph. Connection records between relevant feeders, nodes, branches, and equipment are read along the topological connection relationships. The equipment capacity, adjustment range, action step size, and action count of the corresponding equipment are read along the equipment access relationships. Number of limits and safe operating boundaries; Substitute the node voltage, branch current, energy storage charge state and equipment status in the real-time operating status into the trigger conditions of the expert rules, retain the expert rules that meet the trigger conditions, and read the corresponding control object, action type, action boundary, priority and exception conditions; Then compare the corresponding fields in the case association relationship between the operating scenario, over-limit type, affected area and control object corresponding to the real-time operating status, arrange the historical control cases from high to low according to the number of the same fields, and arrange them from the most recent to the oldest according to the time of occurrence when the number of the same fields is the same, and read the control actions, action execution results and safety constraint satisfaction status in the historical control cases that are ranked first;Following the field order of knowledge features, topological relationships are encoded using binary encoding. Numerical fields in device constraints are normalized using normalized parameters saved during the offline constraint training phase of the control strategy model. Triggering results and action types in expert rules are encoded using binary encoding. Control actions and action execution results in historical similar case information are normalized. The encoded results obtained in this order are used as online knowledge features. Finally, following the field order used when constructing the reinforcement learning state, the real-time running state and online knowledge features are concatenated along the feature dimension to ensure that the number and position of the concatenated input fields are consistent with the reinforcement learning state during the offline constraint training phase of the control strategy model. The splicing result is input into the control strategy model, and a joint representation is obtained through the shared feature extraction layer. Then, the discrete action output branch outputs the discrete action probabilities corresponding to switching actions, reactive power compensation device switching states, and voltage regulation device positions. The continuous action output branch outputs the continuous control parameters corresponding to the active power regulation, reactive power regulation, and energy storage charging / discharging power of the distributed power source. The action with the highest discrete action probability is selected from the discrete action output branch, and the continuous control parameters given by the continuous action output branch are read. The corresponding action type, control device, action direction, and action amplitude are combined according to the order of the control action fields, and the combined control action is used as the original control action.

[0054] In step S6, the distribution network knowledge graph is updated based on the online execution results, and the control strategy model is updated using at least one of periodic retraining and event-triggered incremental optimization.

[0055] In this embodiment, updating the distribution network knowledge graph based on the online execution results can be achieved using the following steps: Select online execution results that meet both security and optimization requirements as new running cases; The newly added operation cases are converted into case entities, written into the distribution network knowledge graph, and associated with the corresponding feeders, nodes, equipment, and rules. Based on the newly added operational data corresponding to the newly added operational cases, update the attributes of equipment entities and rule entities in the power distribution network knowledge graph.

[0056] It should be noted that the online execution results mentioned in this application refer to the device response, changes in operating status, safety verification results, and changes in optimization indicators recorded after the control action is issued; the safety requirement means that the node voltage, branch current, energy storage state of charge, device action, and topology status all meet the corresponding operating boundaries, and the control action meets the screening conditions of expert rules; the optimization requirement means that at least one of voltage deviation, branch load rate, network loss, operating cost, peak-valley load difference, or control command safety pass rate is better than the baseline strategy or the screening conditions of the previous control cycle; the newly added operating case means that both the safety requirement and the optimization requirement are met, and the operation can be recorded. The online execution results of scenarios, control actions, and execution effects; the case entity represents a graph entity with a unique case number, carrying the occurrence time, operating scenario, affected area, control action, execution result, security verification result, and changes in optimization indicators; the newly added operating data represents the operating data, equipment response data, and rule triggering data before and after the execution of the control action corresponding to the newly added operating case; the equipment entity represents a graph entity carrying the operating status, available capacity, number of actions, fault status, and adjustment boundary of distribution network equipment; the rule entity represents a graph entity carrying the applicable scenarios, trigger thresholds, recommended action priorities, and historical effective counts of expert rules.

[0057] In specific implementation, firstly, the online execution results within the same control cycle are read according to the issuance time of the control action. The node voltage after the action is executed is compared item by item with the allowable upper and lower voltage limits, the branch current is compared item by item with the allowable current carrying capacity, and the energy storage state of charge is compared item by item with the upper and lower limits of the energy storage state of charge. The actual response status of the equipment, the number of actions executed, the action amplitude, and the topology connection status after the switch action are also read. When the node voltage does not exceed the limit, the branch current does not exceed the allowable current carrying capacity, the energy storage state of charge is within the safe upper and lower limits, the actual response of the equipment is successful, the equipment action does not exceed the number of actions and action boundary limits, the switch action does not cause unplanned islanding or ring network operation, and the expert rule constraint verification passes, the following will be implemented: The online execution result should be recorded as meeting safety requirements. Then, using the operating data before the control action execution as the data for the previous control cycle, calculate the voltage deviation, branch load rate, network loss, operating cost, and peak-valley load difference before and after the control action execution. Also, read the control command safety pass rate. Compare each calculation result with the corresponding result from the previous control cycle. When at least one optimization indicator improves and the remaining optimization indicators do not exceed the corresponding operating limits, the corresponding online execution result is recorded as meeting optimization requirements. Online execution results that simultaneously meet both safety and optimization requirements are considered new operating cases. Next, generate a unique case number according to the feeder number, occurrence time, and incrementing sequence number. Record the case number, occurrence time, operating scenario, and affected... The affected area, control actions, execution results, safety verification results, and optimization index changes are written into the case attribute fields, and the data records with completed attribute field writing are used as case entities. The corresponding feeder is retrieved based on the feeder number in the newly added running case, the corresponding node is retrieved based on the node number in the affected area, the corresponding device is retrieved based on the device number in the control action, and the corresponding rule is retrieved based on the rule number in the rule trigger data. The association relationships between the case entity and the corresponding feeder, node, device, and rule are established in the form of "case entity—association type—corresponding entity," and the case entities and association relationships are written into the distribution network knowledge graph. Finally, the device entity is retrieved based on the device number in the newly added running data, and the data after the action is executed is recorded. The device status is written to the running status attribute, the current adjustable capacity of the device is written to the available capacity attribute, the number of actual actions of the device this time is accumulated to the action count attribute, the fault status attribute is updated according to the device response results and alarm records, and when the newly added running data contains a valid device running boundary, the adjustment boundary attribute is updated with the latest valid device running boundary; the rule entity is retrieved according to the rule number in the newly added running data, and when the applicable scenario of the rule entity does not record the running scenario corresponding to the newly added running case, the running scenario is added to the applicable scenario attribute, the actual trigger value corresponding to the same rule when meeting the safety requirements and optimization requirements is written to the valid trigger value record, and the minimum and maximum values ​​in the valid trigger value record are taken to update the trigger threshold range;Increment the historical validity count of each rule entity by one. Then, rank the recommended actions according to the historical validity count of each rule entity in the same operating scenario from highest to lowest. Write the ranking order into the priority attribute of the recommended actions. Write the device entities and rule entities that have completed attribute updates back into the distribution network knowledge graph.

[0058] In this embodiment, updating the control strategy model using at least one of periodic retraining and event-triggered incremental optimization can be achieved through the following steps: Determine whether the preset time period has been reached, and monitor in real time whether the operating performance indicators are lower than the preset threshold; If the preset time period is reached, a training sample set containing new samples, historical samples, and similar cases from the knowledge graph is obtained from the sample library to retrain the control strategy model. If the running performance is lower than the preset threshold, new samples and risk samples are obtained from the sample library to incrementally optimize the control strategy model. The updated strategy model was validated offline. If the verification passes, the current online policy model is replaced and the model version number is recorded. If the verification fails, the original model is retained and the update result is recorded. If the security constraint violation rate increases when the new model is running online, it will automatically revert to the previous stable version.

[0059] It should be noted that, in this application, the preset time period refers to the strategy model update interval pre-set according to the requirements of distribution network operation and management; the operation performance index refers to the performance index obtained after converting the number of voltage overruns, branch overloads, control command failures, equipment non-response, safety verification failures, and operating costs into the same evaluation direction; the preset threshold refers to the lower limit of operation performance that triggers incremental optimization of the control strategy model; the new sample refers to the operation sample written into the sample library by the online execution results within the current preset time period; the historical sample refers to the operation sample corresponding to typical operation scenarios and effective control experience retained in the sample library; the knowledge graph similar case refers to the historical control case in the distribution network knowledge graph that matches the current operation scenario in terms of overrun type, affected area, and control object; the training The training sample set represents a set of samples organized according to a unified sample field, consisting of new samples, historical samples, and similar cases from the knowledge graph. The risk sample represents a running sample whose control actions pass safety verification but have high constraint costs or whose operating state is close to the safety boundary. The offline verification represents the process of verifying the control effect and safety constraints of the updated strategy model in a simulation environment or historical playback environment. The current online strategy model represents the strategy model currently responsible for generating online control actions. The model version number represents a version identifier that distinguishes the results of each strategy model update and the deployment order. The current update result represents a model update record that records the training sample range, training time, offline verification indicators, and verification conclusions. The previous stable version represents the strategy model version that passed offline verification and maintained safe online operation before the new model was deployed.

[0060] In specific implementation, firstly, the system clock, the most recent update time of the control strategy model, and a preset time period are read. The most recent update time is then extended by one preset time period to obtain the next update time. When the system clock reaches or exceeds the next update time, the periodic retraining trigger result is recorded. Simultaneously, based on the cumulative number of voltage overruns, branch overloads, control command failures, equipment non-response times, safety verification failures, and operating costs within a fixed monitoring window, each data point is inversely normalized according to its corresponding allowable upper limit. This ensures that when the number of overruns, failures, or operating costs increases, the corresponding evaluation value decreases. The evaluation value after inverse normalization is used as the operating performance indicator, and each item is compared with a preset threshold. When any operating performance... When an indicator falls below a preset threshold, the event triggers incremental optimization results. Secondly, when recording periodic retraining trigger results, new samples are read from the sample library based on a time range consisting of the most recent update time and the current time. Historical samples are read hierarchically from the sample library according to operating scenarios such as normal operation, heavy load operation, renewable energy fluctuations, and fault recovery. Then, based on the operating scenario, over-limit type, affected area, and controlled object in the new samples, the distribution network knowledge graph is searched. Case entities with a large number of matching fields are converted into similar cases in the knowledge graph. The operating status, control actions, safety action sets, reward values, constraint costs, and next operating status in the similar cases in the knowledge graph are converted into sample fields consistent with the new and historical samples. Duplicate records are removed according to feeder number, control time, equipment number, and control action. The field order, normalized parameters, and historical sample proportions from the initial offline constraint training of the control strategy model are retained, and the cleaned sample set is used as the training sample set. Using the current network structure and model parameters of the control strategy model as the starting point for retraining, the training sample set is input according to the original offline constraint training process. Backpropagation and model parameter updates are performed through the policy loss function until the average cumulative reward, constraint cost, safety constraint violation rate, control action safety pass rate, and network loss satisfy the original convergence conditions. The retrained policy model is then used as the updated policy model. Finally, when an event triggers incremental optimization results, the running data is read from the sample library. New samples are written after the performance index falls below a preset threshold. Risk samples with high constraint costs or whose operating state is close to the safety boundary are read. Duplicate records in the new samples and risk samples are deleted according to the control time and device number. Training batches are constructed hierarchically according to sample type so that each training batch containing risk samples retains the corresponding new samples. The model parameters of the current control strategy model are used as the starting point for incremental optimization. The new samples and risk samples are input into the strategy network. The strategy network parameters are updated according to the output of the reward function and the output of the constraint cost function. When the average constraint cost in several consecutive rounds of incremental optimization no longer decreases or has fallen below the constraint threshold, the model parameter update is stopped. The strategy model that has completed incremental optimization is used as the updated strategy model.When both periodic retraining trigger results and event-triggered incremental optimization results exist, retraining is first performed using the training sample set. Then, incremental optimization is performed on the retrained strategy model based on the newly added samples and risk samples, and the final strategy model is used as the updated strategy model. Subsequently, the same verification sample set is input into the updated strategy model and the original model in a simulation environment or historical playback environment, and the average reward value, safety constraint violation rate, voltage overload count, branch overload count, network loss, operating cost, and control action safety pass rate are recorded respectively. When the safety constraint violation rate of the updated strategy model is not higher than that of the original model, the control action safety pass rate is not lower than that of the original model, and at least one of the following is true: average reward value increased, voltage overload count decreased, branch overload count decreased, network loss decreased, or operating cost decreased, the offline verification result is recorded as verification passed; otherwise, the offline verification result is recorded as verification failed. Subsequently, when the offline verification result is successful, the version number is incremented based on the current online policy model's version number. The model version number, training sample range, training time, offline verification metrics, and activation time are written into the model update record. The updated policy model is then written into the online deployment directory, and the current online policy model is set to point to the updated policy model. When the offline verification result is unsuccessful, the current online policy model remains unchanged, and the unsuccessful verification conclusion is written into this update result. After the new model is put into online operation, the security constraint violation rate is calculated according to the same monitoring window as before the update. When the security constraint violation rate of the new model is higher than the security constraint violation rate of the previous stable version within the corresponding monitoring window, the new model is stopped from being called, the network structure and model parameters of the previous stable version are read, the current online policy model is switched to the previous stable version, and the rollback time and rollback reason are recorded in this update result.

[0061] It should also be noted that the reference Figure 2As shown in the figure, this is a diagram of the intelligent control architecture for distribution networks provided in this application. The architecture includes a data input layer, a data preprocessing layer, a knowledge graph construction and reasoning layer, an offline training module, an online operation module, and a feedback update module. The data input layer receives operational data, topology parameters, equipment parameters, expert rules, and historical control cases. The data preprocessing layer performs data cleaning, alignment and fusion, and feature extraction on various types of data, and then transmits the processing results to the knowledge graph construction and reasoning layer. The knowledge graph construction and reasoning layer completes entity relationship extraction, rule modeling, and knowledge reasoning, and provides knowledge features and a set of safety actions to the offline training module. The offline training module sequentially completes state construction, action space definition, reward and constraint design, deep reinforcement learning training, and control strategy model output. The online operation module calls the knowledge reasoning and control strategy model based on the real-time state perception results, completing strategy model invocation, control decision generation, safety verification, and execution. The feedback update module receives the distribution network operation results, updates the knowledge graph, sample library, and control strategy model, and returns the update results to the aforementioned processing stage, thus constituting a closed-loop update process for intelligent control of the distribution network.

[0062] Therefore, in this application, at least one of periodic retraining and event-triggered incremental optimization can be used to update the control strategy model. Firstly, preprocessing multi-source data transforms operating states, network connections, equipment regulation boundaries, and historical control experience scattered across different systems into a unified structured data set and operating state samples, providing a complete data foundation for the accurate representation of complex operating conditions and the generation of control strategies. Secondly, constructing a distribution network knowledge graph based on the structured data set and performing knowledge reasoning transforms distribution network topology relationships, equipment attributes, operating constraints, and expert rules into associative and inferable knowledge features. Based on the current operating state, a set of safe actions is defined, eliminating unexecutable actions that may cause voltage overruns, branch overloads, energy storage charge state out-of-bounds, or switch malfunctions from the source of control action generation. Furthermore, under the constraints of the set of safe actions, offline constraint training is performed using operating state samples and knowledge features, enabling the control strategy model to learn different operating states and... While mapping relationships between control actions, the system is continuously constrained by equipment boundaries and safety rules, reducing the likelihood of unsafe actions generated during unconstrained exploration in deep reinforcement learning and improving the adaptability of control strategies to fluctuations in renewable energy output, load changes, and multi-equipment coupling conditions. Simultaneously, by generating original control actions through the control strategy model and performing matching corrections and safety checks on these actions, the system can correct control results where the action type, controlled object, and action amplitude do not conform to the current safe action range. This ensures that the final issued control actions retain the adjustment intent of the strategy model while meeting the actual operating boundaries of the distribution network. Finally, the distribution network knowledge graph is updated based on online execution results, and the control strategy model is updated through periodic retraining or event-triggered incremental optimization. This continuously supplements the knowledge representation and strategy learning process with new operating scenarios, equipment state changes, and control execution effects, enabling the control strategy to dynamically adjust with changes in the distribution network operating environment. This improves the safe generation and continuous adaptability of distribution network control actions in complex operating scenarios.

[0063] In summary, the technical solution adopted in this application can realize the safe adaptive generation of distribution network control strategies under complex operating scenarios.

[0064] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0065] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compactdisc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0066] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

Claims

1. A method for intelligent control of power distribution networks based on knowledge graphs and deep reinforcement learning, characterized in that, Includes the following steps: Acquire multi-source data of the distribution network, including operational data, topology parameters, equipment parameters, expert rules, and historical control cases; The multi-source data is preprocessed to obtain a structured data set and a running status sample; A distribution network knowledge graph is constructed based on the structured data set, and knowledge reasoning is performed to output the knowledge features and safety action set under the current operating state. Under the constraints of the set of safety actions, offline constraint training is performed using the operating state samples and the knowledge features to output a control strategy model for the power distribution network. The original control actions are generated through the control strategy model, and the original control actions are matched and corrected. After security verification, they are sent out for execution. The distribution network knowledge graph is updated based on the online execution results, and the control strategy model is updated using at least one of periodic retraining and event-triggered incremental optimization.

2. The intelligent control method for power distribution networks based on knowledge graphs and deep reinforcement learning as described in claim 1, characterized in that, Preprocessing the multi-source data to obtain a structured data set and operational status samples specifically includes: The runtime data, topology parameters, equipment parameters, expert rules, and historical control cases in the multi-source data are preprocessed to form basic data with a unified format. The basic data is subjected to entity extraction, relation extraction, and attribute mapping to convert unstructured and semi-structured content into a structured data set. The basic and derived features of the operating status are extracted from the basic data, and the operating status samples are determined based on the basic and derived features.

3. The intelligent control method for power distribution networks based on knowledge graphs and deep reinforcement learning as described in claim 1, characterized in that, Based on the structured data set, a distribution network knowledge graph is constructed, and knowledge reasoning is performed to output the knowledge features and safety action set of the current operating state, specifically including: The structured data set is deduplicated, and the topological connection relationships, device access relationships, rule triggering relationships and case association relationships between each entity are extracted to generate a power distribution network knowledge graph. The operation constraints of the distribution network are converted into safety constraint functions, and the expert rules are converted into rule judgment functions. Then, the safety constraint functions and the rule judgment functions are attached to the distribution network knowledge graph. An initial action candidate set is generated based on the current operating status. The candidate actions in the initial action candidate set are filtered by calling the distribution network knowledge graph. After verification by operating safety constraints, a set of safe actions under the current operating status is generated. Based on the current operating status, topological relationships, equipment constraints, expert rules, and historical similar case information related to the current operating status are extracted from the distribution network knowledge graph to form knowledge features of the current operating status.

4. The intelligent control method for power distribution networks based on knowledge graphs and deep reinforcement learning as described in claim 3, characterized in that, An initial action candidate set is generated based on the current operating status. The candidate actions in the initial action candidate set are filtered by calling the distribution network knowledge graph. After verification by operating safety constraints, a set of safe actions for the current operating status is generated, which specifically includes: Based on the current operating status, enumerate the action options of various controllable devices to form an initial action candidate set; The initial action candidate set is filtered by calling the expert rules in the power distribution network knowledge graph, and the candidate actions that do not meet the requirements of applicable scenario, control object, action type and expert rule priority are eliminated; Based on the equipment operating boundary, the candidate actions obtained from the filtering are trimmed, and the actions are restricted to the constraints of equipment capacity, adjustment range, action step size and number of actions. The candidate actions after trimming are verified by voltage constraints, branch current constraints, energy storage state of charge constraints and topology safety constraints. The candidate actions that pass all the verifications are taken as the set of safe actions under the current operating state.

5. The intelligent control method for power distribution networks based on knowledge graphs and deep reinforcement learning as described in claim 1, characterized in that, Under the constraints of the set of safety actions, offline constraint training is performed using the operating state samples and the knowledge features to output a control strategy model for the power distribution network, specifically including: The running state samples are fused with the knowledge features to construct a reinforcement learning state; Using the reinforcement learning state as input to the policy network, a hybrid action space containing discrete action output branches and continuous action output branches is constructed, and the exploration range of the policy network is limited by the set of safe actions. Construct the policy loss function; The policy network is iteratively trained using the reinforcement learning state and the set of safety actions, combined with the policy loss function, until the convergence condition is met, and the distribution network control policy model is output.

6. The intelligent control method for power distribution networks based on knowledge graphs and deep reinforcement learning as described in claim 5, characterized in that, Using the reinforcement learning state as input to the policy network, a hybrid action space containing discrete action output branches and continuous action output branches is constructed, and the exploration scope of the policy network is limited by the set of safe actions. Specifically, this includes: The reinforcement learning state is used as the input to the policy network, and a joint representation is extracted through a shared feature extraction layer; Using the joint representation as input, discrete action probabilities are output through the discrete action output branch, and continuous control parameters are output through the continuous action output branch, thus forming a hybrid action space; The discrete action output branches are masked and filtered using the set of security actions, and the continuous action output branches are pruned according to the device operation boundary, thus limiting the exploration space of the policy network to the range of the set of security actions.

7. The intelligent control method for power distribution networks based on knowledge graphs and deep reinforcement learning as described in claim 1, characterized in that, The process involves generating initial control actions through the control strategy model, matching and correcting these initial control actions, and then issuing them for execution after security verification. Specifically, this includes: The control strategy model is invoked based on the real-time operating status to generate the original control actions; The original control action is matched with the actions in the set of safety actions. If the match is successful, the matched action is used as a candidate control action. If a match fails, the original control action is modified according to the set of safety actions to obtain candidate control actions; The candidate control actions are subjected to operational safety constraint verification and expert rule constraint verification. Once the verification is passed, the actions are issued for execution.

8. The intelligent control method for power distribution networks based on knowledge graphs and deep reinforcement learning as described in claim 7, characterized in that, Based on the real-time operating status, the control strategy model is invoked to generate the original control actions, specifically including: The real-time operating status is generated based on the real-time collected power distribution network operation data. Based on the real-time operating status, the power distribution network knowledge graph is invoked to perform online knowledge reasoning, thereby obtaining online knowledge features; The real-time operating status is fused with the online knowledge features and then input into the control strategy model to generate the original control action.

9. The intelligent control method for power distribution networks based on knowledge graphs and deep reinforcement learning as described in claim 1, characterized in that, Updating the distribution network knowledge graph based on the online execution results specifically includes: Select online execution results that meet both security and optimization requirements as new running cases; The newly added operation cases are converted into case entities, written into the distribution network knowledge graph, and associated with the corresponding feeders, nodes, equipment, and rules. Based on the newly added operational data corresponding to the newly added operational cases, update the attributes of equipment entities and rule entities in the power distribution network knowledge graph.

10. The intelligent control method for power distribution networks based on knowledge graphs and deep reinforcement learning as described in claim 1, characterized in that, Updating the control strategy model using at least one of periodic retraining and event-triggered incremental optimization specifically includes: Determine whether the preset time period has been reached, and monitor in real time whether the operating performance indicators are lower than the preset threshold; If the preset time period is reached, a training sample set containing new samples, historical samples, and similar cases from the knowledge graph is obtained from the sample library to retrain the control strategy model. If the running performance is lower than the preset threshold, new samples and risk samples are obtained from the sample library to incrementally optimize the control strategy model. The updated strategy model was validated offline. If the verification passes, the current online strategy model is replaced and the model version number is recorded. If the verification fails, the original model is retained and the update result is recorded. If the security constraint violation rate increases when the new model is running online, it will automatically revert to the previous stable version.

Images