A method and system for optimizing control of switch operation of a ring main unit based on reinforcement learning

Abstract

The application discloses a kind of ring network cabinet switch operation optimization control method and system based on reinforcement learning.The method obtains current operation task and ring network cabinet operation data, determines current executable action set;According to target operation result, decompose target operation item and determine candidate switch operation;The successor operation data of candidate action in current executable action set is predicted, and the executability of the candidate switch operation corresponding to the successor target operation item is checked;The successor reachable action space is formed by the candidate action checked, and the current switch operation instruction is output in the successor reachable action space by reinforcement learning model.Thereby, the candidate action that can cause the successor target operation item difficult to continuously execute can be constrained before action decision, and the executability and control reliability of ring network cabinet continuous switch operation are improved.

Description

Technical Field

[0001] This invention relates to the field of power system operation control technology, and in particular to a method and system for optimizing the operation control of ring main unit switches based on reinforcement learning. Background Technology

[0002] Ring main units (RMMs) are widely used in distribution network segmentation, interconnection, fault isolation, and load transfer scenarios. During fault isolation, maintenance switching, backup power connection, or operation mode adjustment, RMMs typically need to perform multiple switching operations consecutively. Each switching operation changes the subsequent power supply path, the operability conditions of candidate switches, and the health margin of related equipment. Therefore, RMM switching control is not a single-step action selection problem, but rather an operational control problem with a continuous sequence of actions.

[0003] Existing switch operation optimization methods based on rules, search algorithms, or reinforcement learning typically first determine the currently executable actions based on topological connectivity, switch open / closed positions, anti-misoperation interlocking states, and electrical safety constraints. Then, an optimization model or reinforcement learning model selects an operation from these executable actions. These methods can prevent actions that clearly do not meet the current execution conditions from entering the control process, but they usually focus more on whether the current action itself is executable and the immediate operational results after the current action is executed.

[0004] In continuous switching operation scenarios, a candidate action may currently satisfy topology conditions, open / closed position conditions, anti-misoperation interlocking conditions, and operation permission conditions. However, executing this candidate action may leave subsequent target operations without a viable candidate switching action. For example, a tie switch may currently be able to close to restore part of the load, but this action may change the power supply path and increase the load on associated switches, causing critical switches used for fault isolation or load restoration to no longer meet health margins or electrical safety constraints, thus making it difficult to continue the subsequent operation chain.

[0005] Therefore, how to avoid making selections based solely on the feasibility of the current action during the continuous switching operation decision-making process of ring main units, while ignoring the impact of the current action on subsequent operation paths, the availability status of candidate switches, and continuous operation capabilities, is a technical problem that needs to be solved in the optimized control of ring main unit switching operations. Summary of the Invention

[0006] The present invention aims to provide a method and system for optimizing the operation control of ring main unit switches based on reinforcement learning, so as to solve the problem that the existing methods mainly make decisions based on the feasibility of the current action, and it is difficult to judge the impact of the action on the feasibility of subsequent target operation items before the action is output.

[0007] To achieve the above objectives, this invention provides a reinforcement learning-based optimized control method for ring main unit switch operation, comprising: Obtain the current operation task and the ring main unit operation data, and determine the current set of executable actions based on the current operation task and the ring main unit operation data; Based on the target operation result corresponding to the current operation task, the current operation task is decomposed into multiple target operation items, and candidate switch operations corresponding to each target operation item are determined. For candidate actions in the current set of executable actions, predict the subsequent operating data of the ring main unit after executing the candidate action, and determine the subsequent target operation item based on the subsequent operating data of the ring main unit; Based on the subsequent operation data of the ring main unit, the feasibility of the candidate switch operations corresponding to the subsequent target operation items is checked, and the candidate actions that pass the feasibility check are included in the subsequent reachable action space. The current operation task, the ring main unit operation data, and the subsequent reachable action space are input into the reinforcement learning model, and the reinforcement learning model outputs the current switching operation command.

[0008] Furthermore, the ring main unit operation data includes ring main unit topology data, candidate switch operation constraint data, and candidate switch health assessment data; The step of determining the set of currently executable actions based on the current operation task and the ring main unit's operating data includes: A set of basic actions is generated based on the current operation task. The basic actions in the set of basic actions consist of candidate switches and corresponding switch operation types. The matching relationship between the basic action and the topology action condition corresponding to the current operation task is determined based on the ring network cabinet topology data; Based on the candidate switch operation constraint data, determine the matching relationship between the basic action and the opening / closing position conditions, anti-misoperation interlocking conditions, control mode conditions, and operation permission conditions; The basic actions that simultaneously match the topological action conditions, the separation and combination position conditions, the anti-misoperation interlocking conditions, the control mode conditions, and the operation permission conditions are determined as the currently executable actions to form the set of currently executable actions.

[0009] Furthermore, the step of decomposing the current operation task into multiple target operation items based on the target operation result corresponding to the current operation task includes: Determine the target topology of the ring main unit based on the target operation results; The target topology relationship is compared with the ring main unit topology data in the ring main unit operation data to determine the target section that needs to be adjusted; Based on the topological role of the target section in the power supply path, determine the corresponding operation function type; The target operation item is generated according to the operation function type, and the topological relationship corresponding to the target segment in the target topological relationship is determined as the completion condition of the target operation item.

[0010] Further, determining the candidate switch operation corresponding to each of the target operation items includes: Determine the target segment and completion conditions corresponding to the target operation item; Determine the control range of the candidate switch in the current ring main unit topology, and determine the topology change result after the candidate switch performs a tripping or closing operation; When the control range covers the target section and the topology change result meets the completion condition, the candidate switch and its corresponding switch operation type are determined as the candidate switch operation corresponding to the target operation item.

[0011] Furthermore, the prediction of subsequent operating data of the ring main unit after executing the candidate action includes: Based on the candidate switches and switch operation types corresponding to the candidate actions, predict the subsequent topology data and subsequent operation constraint data after executing the candidate actions; Determine the health margin of candidate switches based on candidate switch health assessment data; Based on the impact of the candidate actions on the health margin of the candidate switches, predict the health margin migration data after executing the candidate actions; The subsequent topology data, the subsequent operation constraint data, and the health margin migration data are used as the subsequent operation data of the ring main unit.

[0012] Further, determining the health margin of the candidate switch based on the candidate switch health assessment data includes: Extract health characterization quantities from the candidate switch health assessment data to characterize the continuous operation capability of the candidate switches; The health evaluation value of the candidate switch is determined based on the health characterization parameters. The health margin of the candidate switch is determined based on the difference between the health evaluation value and the corresponding minimum automatic operation health threshold.

[0013] Further, the predicted health margin migration data after executing the candidate action includes: Identify the target candidate switch directly affected by the candidate action, and determine the direct health consumption of the target candidate switch based on the switch operation type of the candidate action; Based on the power supply path changes or load transfer relationships formed after the execution of the candidate actions, determine the associated candidate switches affected by the candidate actions, and determine the indirect health consumption of the associated candidate switches. Based on the direct health consumption, the indirect health consumption, and the current health margin of the candidate switch, health margin migration data after executing the candidate action is obtained.

[0014] Furthermore, the step of determining the subsequent target operation item based on the subsequent operation data of the ring main unit includes: The completion conditions of each target operation item are compared with the subsequent topology data in the subsequent operation data of the ring network cabinet. Mark the target operation item whose completion condition is met in the subsequent topology data as the target operation item that has been achieved; The target operation items other than the target operation items that have been achieved are identified as subsequent target operation items.

[0015] Further, the step of performing executability checks on candidate switch operations corresponding to the subsequent target operation item based on the subsequent operation data of the ring main unit, and incorporating the candidate actions that pass the executability check into the subsequent reachable action space, includes: Determine the candidate switch operation corresponding to the subsequent target operation item; Based on the subsequent operation data of the ring main unit, the matching relationship between the candidate switch operation and the subsequent execution conditions is determined. The subsequent execution conditions include topology correspondence conditions, health margin conditions, anti-misoperation interlocking conditions, electrical safety conditions, and action sequence conditions. When each of the subsequent target operation items has a candidate switch operation that matches the subsequent execution condition, the corresponding candidate action is included in the subsequent reachable action space. The step of outputting the current switch operation command by the reinforcement learning model includes: evaluating candidate actions in the subsequent reachable action space by the reinforcement learning model, and outputting the current switch operation command based on the evaluation result.

[0016] Secondly, a ring main unit switch operation optimization control system based on reinforcement learning includes: The executable action determination module is configured to acquire the current operation task and the ring main unit operation data, and determine the current set of executable actions based on the current operation task and the ring main unit operation data; The target operation item determination module is configured to decompose the current operation task into multiple target operation items based on the target operation result corresponding to the current operation task, and determine the candidate switch operation corresponding to each target operation item; The subsequent operation prediction module is configured to predict the subsequent operation data of the ring main unit after executing the candidate action in the current set of executable actions, and determine the subsequent target operation item based on the subsequent operation data of the ring main unit. The executability verification module is configured to perform executability verification on the candidate switch operations corresponding to the subsequent target operation item based on the subsequent operation data of the ring network cabinet, and to include the candidate actions that pass the executability verification into the subsequent reachable action space; The decision output module is configured to input the current operation task, the ring main unit operation data, and the subsequent reachable action space into the reinforcement learning model, and the reinforcement learning model outputs the current switching operation command.

[0017] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. By employing a technical approach that determines the set of currently executable actions based on the current operational task and ring main unit operating data, candidate actions participating in subsequent decisions are first constrained by the current topology, switch operation constraints, and task requirements. This reduces the number of switch actions involved in subsequent decisions due to topology mismatches, incompatible switch opening / closing positions, restrictions imposed by anti-misoperation interlocking, or lack of operational permission, thereby improving the basic executability of the current switch operation.

[0018] 2. By employing a technique that decomposes the current operation task into multiple target operation items and predicts the subsequent operating data of the ring main unit after execution based on candidate actions in the current set of executable actions, the system can determine the impact of a candidate action on subsequent topology relationships, operational constraint states, and candidate switch health margins before the actual execution of the candidate action. This avoids selecting actions solely based on the current feasibility of a candidate action while ignoring its impact on subsequent operation paths and continuous operation capabilities.

[0019] 3. By employing a technique that verifies the executability of candidate switch operations corresponding to subsequent target operation items based on subsequent operating data of the ring main unit, and incorporating the verified candidate actions into the subsequent reachable action space, the reinforcement learning model outputs the current switch operation command within the action space constrained by subsequent reachability. This reduces the risk that the execution of the current action may lead to difficulties in the continuous execution of subsequent fault isolation, load transfer, power restoration, or operation mode adjustments, thereby improving the reliability of the optimized control of continuous switch operations in the ring main unit. Attached Figure Description

[0020] Figure 1 This is a flowchart of the optimized control method for the switch operation of a ring main unit according to Embodiment 1 of this application; Figure 2A flowchart illustrating the process of determining the set of currently executable actions provided in this application embodiment; Figure 3 This is a schematic diagram illustrating the correspondence between target operation items and candidate switch operations provided in the embodiments of this application.

[0021] Figure 4 A schematic diagram of the flow architecture of the ring main unit switch operation optimization control method provided in the embodiments of this application; Figure 5 This is a schematic diagram of the module structure of the system in Embodiment 2 of this application. Detailed Implementation

[0022] Example 1 Reference Figure 1 This embodiment proposes a reinforcement learning-based optimization control method for ring main unit (RNB) switch operation. It should be understood that the topology analysis, anti-misoperation interlocking judgment, electrical safety verification, reinforcement learning model training, and model invocation involved in the following steps can be implemented using existing distribution automation master stations, feeder automation systems, RNB measurement and control terminals, edge computing terminals, or dispatch control systems. The improvement focus of this application is not on a single sensor, communication protocol, or neural network structure itself, but rather on: before the reinforcement learning model outputs the current switch operation command, a set of currently executable actions is first formed based on the current operation task and RNB operating data; then, the subsequent operating state after the candidate actions are executed is predicted; and a subsequent reachable action space is formed based on the executability of the subsequent target operation item, enabling the reinforcement learning model to select the current action within the action space constrained by subsequent reachability.

[0023] In this embodiment, the current operation task can be one or more of the following: fault section isolation, power restoration to non-fault sections, load transfer, planned maintenance power outage, restoration of operation mode after maintenance, or adjustment of operation mode. The ring main unit (RNB) operation data includes RNB topology data, candidate switch operation constraint data, and candidate switch health assessment data. Specifically, the RNB topology data includes the connection relationships between incoming lines, outgoing lines, busbars, tie branches, load sections, and candidate switches; the candidate switch operation constraint data includes the candidate switch's open / close position, anti-misoperation interlocking status, control mode, operation permission status, and protection interlocking status; and the candidate switch health assessment data includes at least one of the following: historical number of actions, most recent action time, mechanism action time, energy storage status, coil current characteristics, contact temperature rise, abnormal alarm records, and maintenance status.

[0024] The aforementioned data can originate from distribution automation master stations, ring main unit monitoring and control terminals, protection monitoring and control devices, intelligent switch controllers, temperature sensors, current acquisition units, equipment ledgers, and historical maintenance records. When thresholds, weights, and evaluation parameters are involved, they can be determined based on equipment manufacturer technical manuals, national standards, industry standards, enterprise operating procedures, historical operating samples, dispatching expert experience, or trial operation calibration results. Different voltage levels, different models of ring main units, and different switching mechanisms can be configured with corresponding parameters. The formulas given in this embodiment are used to illustrate one feasible data processing method and are not limited to the only calculation method.

[0025] S1. Obtain the current operation task and the ring main unit operation data, and determine the current set of executable actions based on the current operation task and the ring main unit operation data.

[0026] Specifically, the ring main unit operation data includes ring main unit topology data, candidate switch operation constraint data, and candidate switch health assessment data; the step of determining the current set of executable actions based on the current operation task and the ring main unit operation data includes the following steps.

[0027] S111. Generate a set of basic actions based on the current operation task. The basic actions in the set of basic actions consist of candidate switches and corresponding switch operation types.

[0028] In this embodiment, let the current time be... The candidate switch set is The set of switch operation types is ,in Indicates the tripping operation. This indicates a closing operation. (From the candidate switch) and operation type Constituting basic movements Then the set of basic actions can be represented as: ,in, This is the set of basic actions corresponding to the current task. Indicates the first Each candidate switch performs an operation type Basic movements; This is the set of candidate switches that can be included in the task analysis at the current moment; This is a set of switch operation types. The above set only represents the enumeration results of basic actions and does not mean that all basic actions can be executed in the current state.

[0029] S112. Determine the matching relationship between the basic action and the topology action condition corresponding to the current operation task based on the ring network cabinet topology data.

[0030] Specifically, the ring main unit topology data can be represented as follows: ,in This represents a set of nodes, which may include bus nodes, incoming nodes, outgoing nodes, load nodes, and tie nodes. This represents a set of edges, where the edges are used to characterize switches, lines, or connecting branches; This represents the operational attributes corresponding to a node or edge. These attributes may include energized status, load connection status, switch on / off status, and fault identifier. Topology action conditions can be determined by scheduling rules, a fault prevention rule base, or task templates. For example, a fault isolation task requires candidate actions to change the connectivity between the faulty section and the power supply side; a load transfer task requires candidate actions to establish connectivity between the target load section and the backup power supply path.

[0031] For any basic action You can set a topology matching flag. When basic movements Acts on the current topology When the topological action conditions corresponding to the current operation task are met, ;otherwise .in, It is a dimensionless binary variable.

[0032] S113. Determine the matching relationship between the basic action and the opening / closing position conditions, anti-misoperation interlocking conditions, control mode conditions, and operation permission conditions based on the candidate switch operation constraint data.

[0033] Specifically, for any basic action Set matching identifiers for the separation and combination positions respectively. Anti-misoperation interlocking matching identifier Control method matching identifier Matching identifier with operation license The above identifiers are all dimensionless binary variables, taking values ​​of... or For example, when the candidate switch is currently in the open position and the basic operation requires closing, the open / close position conditions can be considered matched; when the candidate switch is under local control and the current operation task requires remote automatic control, the control mode conditions can be considered mismatched; when the candidate switch is subject to anti-misoperation interlocking or protection interlocking, the anti-misoperation interlocking conditions or operation permission conditions can be considered mismatched.

[0034] S114. The basic actions that simultaneously match the topological action condition, the separation and combination position condition, the anti-misoperation interlocking condition, the control mode condition, and the operation permission condition are determined as the currently executable actions to form the set of currently executable actions.

[0035] Reference Figure 2 When determining the set of currently executable actions, a set of basic actions can first be generated based on the current operation task. Then, the basic actions are sequentially matched and judged according to topological action conditions, separation and combination position conditions, anti-misoperation interlocking conditions, control mode conditions, and operation permission conditions. Basic actions that match all of the above conditions are determined as currently executable actions; basic actions that do not match any of the conditions are not included in the current set of executable actions. In this way, all candidate actions entering subsequent operation prediction and reinforcement learning decision-making can meet the basic operation requirements of the current operation task and the ring network cabinet's operating state.

[0036] The current set of executable actions can be represented as: in, This is the set of currently executable actions; A set of basic actions; , , , and These represent the basic movements. The matching results are for topological action conditions, separation and combination position conditions, anti-misoperation interlocking conditions, control mode conditions, and operation permission conditions. Since each matching identifier is a dimensionless binary variable, the above product does not involve the addition of different physical dimensions.

[0037] When critical status data of a candidate switch is missing, communication is interrupted, or the status is unreliable, the relevant matching identifier for the corresponding action of that candidate switch can be set to [value]. Alternatively, the candidate switch can be transferred to a manual review process to avoid incomplete data directly entering the automated operation decision-making process.

[0038] S2. Based on the target operation result corresponding to the current operation task, decompose the current operation task into multiple target operation items, and determine the candidate switch operation corresponding to each target operation item.

[0039] Specifically, the step of decomposing the current operation task into multiple target operation items based on the target operation result corresponding to the current operation task includes the following steps.

[0040] S211. Determine the target topology of the ring main unit based on the target operation results.

[0041] In this embodiment, the target operation result corresponding to the current operation task can be represented as a target topological relationship. Target topological relationships It can be determined by dispatch instructions, fault location results, load transfer strategies, maintenance plans, or operation mode adjustment schemes. For example, in fault isolation and load restoration scenarios, the target topology relationship may include the faulty section being disconnected from the power supply side, the non-faulty power-loss section being connected to the backup power supply path, and the tie switch being in the target open / closed state.

[0042] S212. Compare the target topology relationship with the ring main unit topology data in the ring main unit operation data to determine the target section that needs to be adjusted.

[0043] The target segment can be represented as ,in The index for the target operation item. Target segment. The target section can be a fault section that needs isolation, a load section that needs power restoration, a branch section that needs power path switching, or a bus section that needs to maintain a safety boundary. The target section can be determined by comparing it with the current topology. Topological relationship with the target The differences in node connectivity, edge states, and charged states are implemented.

[0044] S213. Determine the corresponding operation function type based on the topological role of the target section in the power supply path.

[0045] Operational function types can include fault isolation, load restoration, backup power connection, operating mode adjustment, maintenance outage, and safety boundary maintenance. Operational function types characterize the technical role of a target section in the current task and are not equivalent to specific switching actions. For example, the same target section may correspond to fault isolation or maintenance outage under different tasks; the same operational function type may also correspond to multiple candidate switching operations.

[0046] S214. Generate the target operation item according to the operation function type, and determine the topological relationship corresponding to the target segment in the target topological relationship as the completion condition of the target operation item.

[0047] Let the set of target operation items be Each target operation item Corresponding to a target segment One type of operation function And a completion condition .in, It can be represented by the connectivity in the target topology, the on / off state of the target switch, the energized state of the target section, or safety boundary conditions. Completion conditions The judgment result is a dimensionless binary variable, taking the value of or .

[0048] Specifically, determining the candidate switch operation corresponding to each of the target operation items includes the following steps.

[0049] S221. Determine the target segment and completion conditions corresponding to the target operation item.

[0050] For the target operation item Read its corresponding target segment and completion conditions Target section Derived from the topological difference analysis in step S212; completion conditions It originates from the correspondence between the target topology and the target segment in step S214.

[0051] S222. Determine the control range of the candidate switch in the current ring main unit topology, and determine the topology change result after the candidate switch performs a tripping or closing operation.

[0052] Candidate switch operation can be represented as ,in Indicates the target operation item Relevant candidate switch operation indexes. The control range of a candidate switch can be determined based on its location in the power supply path, adjacent nodes, upstream and downstream sections, and switch open / closed status. The topology change result after a candidate switch performs an open or close operation can be represented by a topology update function: ,in, Indicates candidate switch operation Acts on the current topology The resulting topological result; This represents the topology update function, which can be implemented according to the distribution network node-branch model, the adjacency matrix update rule, or the topology analysis rule in the distribution automation system. Indicates the target operation item The corresponding number Candidate switch operation.

[0053] S223. When the control range covers the target section and the topology change result satisfies the completion condition, the candidate switch and its corresponding switch operation type are determined as the candidate switch operation corresponding to the target operation item.

[0054] Reference Figure 3In an exemplary ring main unit topology, the main power supply, ring main unit A, ring main unit B, ring main unit C, ring main unit D, and backup power supply are connected sequentially, and candidate switches SW1 to SW5 are respectively set between adjacent power supply nodes. When the current operation task is fault section isolation and load transfer, the current operation task can be decomposed into target operation items such as fault section isolation, backup power supply access, and power failure load restoration based on the target operation result. For the fault section isolation target operation item, the SW2 and SW3 tripping operations, which can disconnect the fault section from the adjacent power supply section, can be identified as the corresponding candidate switch operations; for the backup power supply access target operation item, the SW5 closing operation, which can access the backup power supply, can be identified as the corresponding candidate switch operation; for the power failure load restoration target operation item, the SW4 closing operation, which can form a load transfer path, can be identified as the corresponding candidate switch operation. It should be understood that... Figure 3 This is only used to illustrate the method of determining the target operation item and the candidate switch operation, and does not limit the number of ring main units, the number of candidate switches, or the specific topology.

[0055] Target operation item The corresponding set of candidate switch operations can be represented as: ,in, For target operation item The corresponding set of candidate switch operations; Indicates candidate switch operation In the current topology The scope of control within; This indicates that the control range covers the target section. ; Indicates candidate switch operation The resulting topology achieves the target operation. The conditions for completion.

[0056] S3. For candidate actions in the current set of executable actions, predict the subsequent operating data of the ring main unit after executing the candidate action, and determine the subsequent target operation item based on the subsequent operating data of the ring main unit.

[0057] Specifically, the prediction of the subsequent operation data of the ring main unit after the candidate action is executed includes the following steps.

[0058] S311. Based on the candidate switch and switch operation type corresponding to the candidate action, predict the subsequent topology data and subsequent operation constraint data after executing the candidate action.

[0059] For the current set of executable actions any candidate action in The subsequent topology data after executing this candidate action can be represented as: ,in, Indicates the current topology Next, execute candidate actions The subsequent topological data obtained from post-prediction; This represents the topology update function. Subsequent operation constraint data can be represented as: ,in, This indicates the constraint data for the current candidate switch operation; Indicates the execution of candidate actions The subsequent operation constraint data obtained from the post-prediction; This represents a constraint state update function, which can update the constraint state based on changes in switch open / closed states, anti-misoperation interlocking rules, protection interlocking rules, remote / local control states, and scheduling operation ticket sequence rules.

[0060] S312. Determine the health margin of the candidate switches based on the candidate switch health assessment data.

[0061] Specifically, determining the health margin of a candidate switch based on the candidate switch health assessment data includes the following steps.

[0062] S3121. Extract health characterization quantities from the candidate switch health assessment data to characterize the continuous operation capability of the candidate switch.

[0063] Health indicators can include operating time, coil current characteristics, energy storage time, contact temperature rise, cumulative number of operations, historical anomaly count, and recent maintenance status. Since the raw data have different dimensions, they need to be normalized to dimensionless data before being used in the weighted evaluation. Let the first... The candidate switch of the first The original health characteristics are Its normalization result is .

[0064] For cost-type representations where larger values ​​indicate worse conditions, the following normalization method can be used: For yield-related metrics where larger values ​​indicate better performance, the following normalization method can be used: in, A minimal positive number with the same dimensions as the l-th health indicator, used to avoid and When they are equal, the denominator is zero; this occurs when the l-th health characteristic has been pre-converted to dimensionless data. It can also be set to a dimensionless, minimal positive number.

[0065] in, This is a dimensionless normalized result, with a value range of [value range missing]. The higher the value, the greater the degree of health deterioration. For the first The candidate switch of the first One original health characteristic; and The first The lower and upper limits of a health indicator; Indicates will Limited to Within the range. and These data can come from the equipment manufacturer's technical manual, the company's operating procedures, historical operating sample quantiles, or trial operation calibration results.

[0066] It should be noted that the dimension of the action time can be seconds, the dimension of the coil current can be amperes, the dimension of the contact temperature rise can be degrees Celsius, and the dimension of the cumulative number of actions can be times. All of the above original physical quantities have been normalized and converted into dimensionless variables before being included in the health evaluation formula; therefore, the subsequent weighted summation does not involve the direct addition of different dimensions.

[0067] S3122. Determine the health evaluation value of the candidate switch based on the health characterization quantity.

[0068] The health evaluation value of the candidate switch can be expressed as: ,in, Indicates the first The health evaluation values ​​of the candidate switches are dimensionless variables, and their range is [value range missing]. A higher value indicates a better health status for the candidate switch; This indicates the quantity of health indicators; Indicates the first The weights of each health characteristic measure satisfy and ; Indicates the first The candidate switch of the first A dimensionless normalized degradation quantity. Weight The calibration can be determined based on the equipment manufacturer's recommendations, historical fault sample training results, expert experience, or trial operation data.

[0069] S3123. Determine the health margin of the candidate switch based on the difference between the health evaluation value and the corresponding minimum automatic operation health threshold.

[0070] The health margin of a candidate switch can be expressed as: , in, It is a dimensionless, minimal positive number, used to avoid When the denominator is zero; This represents the relative health margin of the candidate switch relative to the lowest automatic operation health threshold, and its value ranges from [0,1]. Indicates the first Health evaluation values ​​of each candidate switch; Indicates the first The minimum automatic operation health threshold that each candidate switch is allowed to participate in automatic switching operation, with a value range of [value range missing]. . The threshold can be determined based on the equipment model, switch mechanism type, operating procedures, maintenance strategy, or historical fault samples. For example, a higher minimum automatic operation health threshold can be set for candidate switches that are frequently operated and perform important load transfer functions; for candidate switches used only in non-critical branches, a relatively lower threshold can be set based on the company's operating strategy.

[0071] S313. Based on the impact of the candidate action on the health margin of the candidate switch, predict the health margin migration data after executing the candidate action.

[0072] Specifically, the prediction of health margin migration data after executing the candidate action includes the following steps.

[0073] S3131. Determine the target candidate switch directly affected by the candidate action, and determine the direct health consumption of the target candidate switch based on the switch operation type of the candidate action.

[0074] Direct health consumption can be expressed as: ,in, Indicates the execution of candidate actions For the The direct health consumption generated by each candidate switch is a dimensionless variable. For direct action identification, when candidate action Directly acting on the first When there are candidate switches ,otherwise ; Indicates the first Each candidate switch performs an operation type The corresponding unit health consumption is a dimensionless variable. It can be determined based on the switch mechanism life curve, the manufacturer's rated mechanical life, historical operation statistics, or test calibration results.

[0075] S3132. Based on the power supply path changes or load transfer relationships formed after the execution of the candidate action, determine the associated candidate switches affected by the candidate action, and determine the indirect health consumption of the associated candidate switches.

[0076] When candidate actions When a change in power supply path or load transfer occurs, other candidate switches on the relevant branches may experience changes in current, temperature rise, or subsequent operating frequency. The appropriate action can be determined based on the candidate switch's behavior. The power flow estimation results or load distribution results before and after determine the first Current change of each candidate switch .in, The dimension is ampere; before being included in the indirect health consumption calculation, it is divided by the candidate switch rated current. This results in a dimensionless ratio.

[0077] Indirect health consumption can be expressed as: Due to the rated current of the candidate switch in the ring main unit Since this is a parameter from the equipment nameplate and is greater than 0, it can be directly used as the normalized denominator. When the system requires abnormal protection, a very small positive number with the same dimensions as the current can be set. and with As the denominator.

[0078] in, This represents the indirect health consumption caused by executing candidate action a on the i-th candidate switch; As an indicator of the associated impact, when the i-th candidate switch is affected by a change in the power supply path or load transfer caused by candidate action a. =1, otherwise =0; This is the load transfer impact coefficient, and its value range can be... ; Indicates the execution of candidate actions Before and after The change in current in the line or branch where each candidate switch is located, in amperes; Indicates the first The rated current of each candidate switch is in amperes. Because and They have the same dimensions, and the ratio of the two is dimensionless data.

[0079] S3133. Based on the direct health consumption, the indirect health consumption, and the current health margin of the candidate switch, obtain the health margin migration data after executing the candidate action.

[0080] Execute candidate actions After that, the The health margin transition results of the candidate switches can be expressed as: in, Indicates the execution of candidate actions The later Predicted health margin of each candidate switch; Indicates the current time. Health margin of each candidate switch; Indicates direct health consumption; This represents indirect health expenditure. All the above variables are dimensionless, and their values ​​are limited to a range of values ​​within a certain range. Inside. Consisting of each candidate switch. This constitutes health margin migration data.

[0081] S314. The subsequent topology data, the subsequent operation constraint data, and the health margin migration data are used as the subsequent operation data of the ring main unit.

[0082] Therefore, candidate actions The corresponding subsequent operation data of the ring main unit can be represented as: in, Indicates candidate actions The corresponding subsequent operation data of the ring main unit; Indicates subsequent topology data; This indicates constraint data for subsequent operations; Indicates the first Predicted health margin of each candidate switch.

[0083] Specifically, determining the subsequent target operation item based on the subsequent operation data of the ring main unit includes the following steps.

[0084] S321. The completion conditions of each target operation item are compared with the subsequent topology data in the subsequent operation data of the ring network cabinet.

[0085] For each target operation item To complete the conditions With subsequent topology data Perform a comparison. The completion flag for the target operation item can be represented as: in, Indicates the execution of candidate actions Post-target operation item The completion indicator; when subsequent topology data reaches the target operation item. When the completion conditions are met, ;otherwise . It is a dimensionless binary variable.

[0086] S322. Mark the target operation item whose completion condition is met in the subsequent topology data as the target operation item that has been achieved.

[0087] when At that time, the target operation item Mark the action as achieved. Actions that have achieved their objectives do not need to participate in subsequent feasibility checks.

[0088] S323. Determine the target operation items other than the target operation items that have been achieved as subsequent target operation items.

[0089] Candidate Action The corresponding set of successor target operations can be represented as: in, Indicates the execution of candidate actions The set of subsequent target operation items that need to be processed; This represents the set of target operation items obtained in step S2; Indicates the target operation item The completion identifier in subsequent topology data.

[0090] S4. Based on the subsequent operation data of the ring main unit, perform executability verification on the candidate switch operations corresponding to the subsequent target operation items, and include the candidate actions that pass the executability verification into the subsequent reachable action space.

[0091] Specifically, the step of performing an executable check on the candidate switch operation corresponding to the subsequent target operation item based on the subsequent operation data of the ring main unit, and incorporating the candidate action that passes the executable check into the subsequent reachable action space, includes the following steps.

[0092] S411. Determine the candidate switch operation corresponding to the subsequent target operation item.

[0093] For the set of subsequent target operations Each subsequent target operation item in Read the corresponding candidate switch operation set If in the candidate action After execution, if a candidate switch operation needs to be re-evaluated due to changes in topology state or switch open / closed state, it can be based on subsequent topology data. and subsequent operation constraint data Regenerate the set of candidate switch operations corresponding to the subsequent target operation item.

[0094] After executing candidate action a, the set of candidate switch operations corresponding to the subsequent target operation r can be represented as follows: .in, Based on subsequent topology data Subsequent operation constraint data And the completion conditions of the subsequent target operation item r are redefined.

[0095] S412. Based on the subsequent operation data of the ring main unit, determine the matching relationship between the candidate switch operation and the subsequent execution conditions. The subsequent execution conditions include topology correspondence conditions, health margin conditions, anti-misoperation interlocking conditions, electrical safety conditions, and action sequence conditions.

[0096] For subsequent target operation items Candidate switch operation Set the matching flag for subsequent execution. : in, Indicates the execution of candidate actions Afterwards, candidate switch operation The comprehensive matching results of subsequent execution conditions; Indicates the topology-corresponding condition matching identifier; Indicates a health margin condition matching identifier; Indicates the matching identifier for anti-misoperation interlocking conditions; Indicates electrical safety condition matching; This indicates an action sequence condition matching identifier. All of the above identifiers are dimensionless binary variables.

[0097] Health margin conditions can be determined as follows: when the candidate switch is operated The corresponding candidate switch executes the candidate action. If the health margin is not lower than the reserved health margin threshold, the health margin condition is determined to be matched; otherwise, the health margin condition is determined to be mismatched. The reserved health margin threshold can be expressed as... Its value range is It can be determined based on the switchgear model, critical load level, continuous operation frequency requirements, historical fault statistics, or enterprise operating procedures.

[0098] In this embodiment, the topology correspondence condition is used to determine whether a candidate switch operation can act on the target section corresponding to the subsequent target operation item; the anti-misoperation blocking condition is used to determine whether the candidate switch operation meets the blocking rules in the subsequent state; the electrical safety condition is used to determine whether the candidate switch operation meets the current limit, voltage constraint, grounding state, circulating current constraint, and power supply safety boundary; and the action sequence condition is used to determine whether the candidate switch operation conforms to the scheduling operation ticket sequence or the preset operation sequence constraint. The specific judgment rules of the above conditions can be derived from the distribution automation system rule base, the anti-misoperation blocking rule base, the scheduling operation procedure, and the expert configuration template.

[0099] S413. When each of the subsequent target operation items has a candidate action corresponding to a candidate switch operation that matches the subsequent execution conditions, it is included in the subsequent reachable action space.

[0100] For candidate actions You can set a subsequent reachable identifier. : Where P(a,t) represents the successor reachability identifier corresponding to candidate action a; This represents the set of subsequent target operations after executing candidate action a; This represents the set of candidate switch operations corresponding to the subsequent target operation item r after executing candidate action a; Indicates candidate switch operation The comprehensive matching result of subsequent execution conditions. For any subsequent target operation item. If it has at least one candidate switch operation that satisfies the subsequent execution conditions, then If each subsequent target operation item has candidate switch operations that satisfy the subsequent execution conditions, then .when When it is an empty set, it represents a candidate action. After execution, all target operation items have met the completion conditions, and can be... Take as .

[0101] The reachable action space can be represented as: ,in, Indicates the space of actions that can be reached by successor; This represents the set of currently executable actions; Indicates candidate actions The successor reachable identifier. Therefore, the candidate actions in the successor reachable action space not only satisfy the operational constraints of the current state, but also the successor target operation item after its execution can still be processed by the candidate switch operation that satisfies the successor execution conditions.

[0102] S414. The subsequent reachable action space is used as the action constraint input of the reinforcement learning model, so that the reinforcement learning model can evaluate the current switching operation action.

[0103] S5. Input the current operation task, the ring main unit operation data, and the subsequent reachable action space into the reinforcement learning model, and have the reinforcement learning model output the current switching operation command.

[0104] Reference Figure 4 The reinforcement learning-based ring main unit (RNB) switch operation optimization control method provided in this embodiment can be implemented by a combination of an executable action determination module, a target operation item determination module, a subsequent operation prediction module, an executability verification module, and a reinforcement learning decision output module. Specifically, after the current operation task and RNB operation data are input, the executable action determination module determines the current set of executable actions, the target operation item determination module determines the target operation item and its corresponding candidate switch operation, the subsequent operation prediction module predicts the subsequent operation data of the RNB after the candidate action is executed and determines the subsequent target operation item, the executability verification module forms the subsequent reachable action space, and the reinforcement learning decision output module outputs the current switch operation command within the subsequent reachable action space. After the current switch operation command is executed, the execution feedback is used to update the RNB operation data and enter the next round of switch operation optimization control.

[0105] In this embodiment, the state input of the reinforcement learning model may include the current operation task code, the current ring network cabinet topology code, the candidate switch operation constraint code, the candidate switch health margin vector, the target operation item completion state vector, and the subsequent reachable action space mask. The state vector can be represented as: ,in, This indicates that the reinforcement learning model at time 10:00 Status input; Indicates the current operation task code; Indicates the current ring main unit topology code; Indicates the constraint code for candidate switch operation; Represents the candidate switch health margin vector; This represents the completion state vector of the target operation item; This represents the mask for the reachable action space. Before entering the model, all the above encoded quantities are converted into dimensionless values, binary variables, or embedding vectors. The original physical quantities such as current, voltage, temperature rise, and action time are processed by normalizing to rated values, upper and lower limits, or historical samples.

[0106] The space mask for subsequent reachable actions can be represented as: in, Indicates candidate actions Action space mask; when candidate actions Belongs to the subsequent reachable action space hour, ;otherwise This mask is used to limit the range of actions a reinforcement learning model can choose, ensuring that the model evaluates and outputs only within the space of subsequent reachable actions.

[0107] Reinforcement learning models can employ deep Q-networks, dominant action evaluation networks, proximal policy optimization networks, or other reinforcement learning models capable of outputting action evaluation results. In one implementation, a deep Q-network is used to calculate the state-action value, and the current switch operation action can be represented as: ,in, This represents the current switching action output by the reinforcement learning model; The parameter is The reinforcement learning model for the state Next candidate action State-action value evaluation results; Indicates model parameters; This represents the space of subsequent reachable actions. The state-action value evaluation result is a dimensionless numerical value used to characterize the expected comprehensive reward obtained after performing a candidate action.

[0108] During the model training phase, training samples can be constructed based on historical operation records of ring main units, simulation operation data, dispatch operation ticket samples, and power flow simulation results of the distribution network. The reward function can be expressed as: in, Indicates time The overall reward value is a dimensionless variable; This indicates the reward for achieving the target action item; This indicates a reward for meeting electrical safety constraints and anti-misoperation interlocking constraints; This represents the reward for maintaining a healthy margin for candidate switches; Indicates the number of operation steps or operation cost penalty; , , and These are reward weights, all of which are dimensionless and non-negative parameters. All the above reward components are normalized to a preset interval before entering the reward function, for example... or This avoids directly adding data with different dimensions. Reward weights can be determined through training with historical operation samples, calibration by scheduling experts, simulation verification, or parameter tuning during trial runs.

[0109] When training with a deep Q-network, the target value can be expressed as: The loss function can be expressed as: in, Represents the target state-action value; This represents the discount factor, with a value range of 100%. ; This represents the space of actions that can be reached in the next moment; Indicate the target network; Indicates the target network parameters; This represents the model training loss. If there is no target operation to continue executing in the next time step, or if the task has already achieved the target operation result, then the subsequent value item can be set to zero. Alternatively, it can be processed as a terminated state.

[0110] During the online application phase of the model, the reinforcement learning model receives state input. and subsequent reachable action space The system calculates the state-action value evaluation result for each candidate action in the subsequent reachable action space and outputs the candidate action with the highest evaluation result as the current switch operation instruction. The current switch operation instruction may include the target candidate switch number, operation type, operation time, verification result, and execution confirmation identifier. Before the output instruction is sent to the ring main unit actuator, it may also undergo a secondary safety verification by the distribution automation system or dispatch control system; if the secondary safety verification fails, the automatic issuance can be terminated and the process can be transferred to manual confirmation.

[0111] To mitigate the risk of insufficient disclosure, this embodiment further explains that: the network structure of the reinforcement learning model can employ existing multilayer perceptrons, graph neural networks, recurrent neural networks, or combinations thereof; model training samples can be derived from historical scheduling operation tickets, simulation samples, and samples verified by human experts; model parameters can be obtained through existing training methods such as backpropagation, experience replay, and target network updates. This application does not necessarily limit the specific number of network layers, activation functions, or optimizers; the core lies in forming a subsequent reachable action space through steps S1 to S4, and using this action space as the action constraint input for the reinforcement learning model to improve the adaptability between continuous switching operation decisions and the subsequent operable states of the ring main unit.

[0112] This embodiment involves , , , , , , , , , , and These parameters can be determined based on the equipment manufacturer's technical manual, national standards, industry standards, enterprise operating procedures, historical operating samples, simulation samples, expert experience, or trial operation calibration results. For different models of ring main units, switchgear with different rated current levels, and loads of different importance levels, separate parameter tables can be created and accessed by the system.

[0113] If missing values ​​exist in the candidate switch health assessment data, they can be supplemented using the most recent valid sample value, the statistical average of devices of the same model, or manually configured default values, and a confidence level indicator should be set for the supplemented data. If the lack of critical data prevents the completion of the safety verification, the corresponding candidate action will not be included in the current set of executable actions or the subsequent reachable action space. If the upper and lower limits are equal in the normalization calculation, a very small positive number can be used. Adjust the denominator, or temporarily remove the health indicator from the health assessment value calculation and renormalize the remaining weights.

[0114] Therefore, this embodiment can simultaneously consider the feasibility of the current action, the progress status of the target operation item, the reachability of subsequent operations, and the changes in the health margin of the candidate switch before outputting the candidate action, so that the reinforcement learning model outputs the current switch operation command within the action space that meets the subsequent reachability conditions.

[0115] Example 2 Reference Figure 5 This embodiment proposes a reinforcement learning-based optimized control system for ring main unit switch operation, including: The executable action determination module is configured to acquire the current operation task and the ring main unit operation data, and determine the current set of executable actions based on the current operation task and the ring main unit operation data; The target operation item determination module is configured to decompose the current operation task into multiple target operation items based on the target operation result corresponding to the current operation task, and determine the candidate switch operation corresponding to each target operation item; The subsequent operation prediction module is configured to predict the subsequent operation data of the ring main unit after executing the candidate action in the current set of executable actions, and determine the subsequent target operation item based on the subsequent operation data of the ring main unit. The executability verification module is configured to perform executability verification on the candidate switch operations corresponding to the subsequent target operation item based on the subsequent operation data of the ring network cabinet, and to include the candidate actions that pass the executability verification into the subsequent reachable action space; The decision output module is configured to input the current operation task, the ring main unit operation data, and the subsequent reachable action space into the reinforcement learning model, and the reinforcement learning model outputs the current switching operation command.

[0116] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. For those skilled in the art, various modifications, substitutions, and combinations can be made to the above embodiments without departing from the concept of the present invention; all such modifications, substitutions, and combinations should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A method for optimizing the operation control of ring main unit switches based on reinforcement learning, characterized in that, include: The system acquires the current operation task and ring main unit (RNB) operation data, and determines the set of currently executable actions based on the current operation task and the RNB operation data. The current operation task includes fault section isolation, power restoration to non-fault sections, load transfer, planned maintenance power outage, post-maintenance restoration of operation mode, and operation mode adjustment. The RNB operation data includes RNB topology data, candidate switch operation constraint data, and candidate switch health assessment data. Based on the target operation result corresponding to the current operation task, the current operation task is decomposed into multiple target operation items, and candidate switch operations corresponding to each target operation item are determined. For candidate actions in the current set of executable actions, predict the subsequent operating data of the ring main unit after executing the candidate action, and determine the subsequent target operation item based on the subsequent operating data of the ring main unit; The prediction of subsequent operating data of the ring main unit after executing the candidate action includes: predicting subsequent topology data and subsequent operation constraint data after executing the candidate action based on the candidate switch and switch operation type corresponding to the candidate action; determining the health margin of the candidate switch based on the candidate switch health assessment data; predicting health margin migration data after executing the candidate action based on the impact of the candidate action on the health margin of the candidate switch; and using the subsequent topology data, the subsequent operation constraint data, and the health margin migration data as the subsequent operating data of the ring main unit. For the i-th candidate switch, its predicted health margin after executing candidate action a is determined according to the following relationship: In the formula Indicates the execution of candidate actions The next Predicted health margin of each candidate switch, Indicates the current time. Health margin of each candidate switch Indicates direct health consumption, Indicates indirect health consumption, generated by each candidate switch. This constitutes the health margin migration data; Based on the subsequent operation data of the ring main unit, the executability of candidate switch operations corresponding to the subsequent target operation items is checked, and the candidate actions that pass the executability check are included in the subsequent reachable action space. This includes: determining the candidate switch operations corresponding to the subsequent target operation items; judging the matching relationship between the candidate switch operations and subsequent execution conditions based on the subsequent operation data of the ring main unit, wherein the subsequent execution conditions include topology correspondence conditions, health margin conditions, anti-misoperation interlocking conditions, electrical safety conditions, and action sequence conditions; when each of the subsequent target operation items has candidate switch operations that match the subsequent execution conditions, the corresponding candidate actions are included in the subsequent reachable action space; the step of outputting the current switch operation command by the reinforcement learning model includes: evaluating the candidate actions in the subsequent reachable action space by the reinforcement learning model, and outputting the current switch operation command based on the evaluation results. The current operation task, the ring main unit operation data, and the subsequent reachable action space are input into the reinforcement learning model, and the reinforcement learning model outputs the current switching operation command.

2. The method for optimizing the operation control of ring main unit switches based on reinforcement learning according to claim 1, characterized in that, The step of determining the set of currently executable actions based on the current operation task and the ring main unit's operating data includes: A set of basic actions is generated based on the current operation task. The basic actions in the set of basic actions consist of candidate switches and corresponding switch operation types. The matching relationship between the basic action and the topology action condition corresponding to the current operation task is determined based on the ring network cabinet topology data; Based on the candidate switch operation constraint data, determine the matching relationship between the basic action and the opening / closing position conditions, anti-misoperation interlocking conditions, control mode conditions, and operation permission conditions; The basic actions that simultaneously match the topological action conditions, the separation and combination position conditions, the anti-misoperation interlocking conditions, the control mode conditions, and the operation permission conditions are determined as the currently executable actions to form the set of currently executable actions.

3. The reinforcement learning-based optimized control method for ring main unit switch operation as described in claim 1, characterized in that, The step of decomposing the current operation task into multiple target operation items based on the target operation result corresponding to the current operation task includes: The target topology of the ring main unit is determined based on the target operation results. The target topology includes the fault section being disconnected from the power supply side, the non-fault power failure section being connected to the backup power path, and the tie switch being in the target open / closed state. The target topology relationship is compared with the ring network cabinet topology data in the ring network cabinet operation data to determine the target section that needs to be adjusted. The target section includes the fault section that needs to be isolated, the load section that needs to be restored to power supply, the branch section that needs to switch the power supply path, or the bus section that needs to maintain the safety boundary. Based on the topological role of the target section in the power supply path, the corresponding operation function type is determined. The operation function type includes fault isolation, load restoration, backup power supply access, operation mode adjustment, maintenance power outage, and safety boundary maintenance. The target operation item is generated according to the operation function type, and the topological relationship corresponding to the target segment in the target topological relationship is determined as the completion condition of the target operation item. Each target operation item corresponds to a target segment, an operation function type and a completion condition. The completion condition is represented by the connectivity relationship in the target topological relationship, the switch target open / closed state, the target segment energized state or safety boundary conditions.

4. The reinforcement learning-based ring main unit switch operation optimization control method according to claim 3, characterized in that, The determination of the candidate switch operation corresponding to each of the target operation items includes: Determine the target segment and completion conditions corresponding to the target operation item; Determine the control range of the candidate switch in the current ring main unit topology, and determine the topology change result after the candidate switch performs a tripping or closing operation; When the control range covers the target section and the topology change result meets the completion condition, the candidate switch and its corresponding switch operation type are determined as the candidate switch operation corresponding to the target operation item.

5. The reinforcement learning-based optimized control method for ring main unit switch operation according to claim 1, characterized in that, The step of determining the health margin of candidate switches based on candidate switch health assessment data includes: Health characteristics are extracted from the candidate switch health assessment data to characterize the continuous operation capability of the candidate switch. These health characteristics include operating time, coil current characteristics, energy storage time, contact temperature rise, cumulative number of operating actions, number of historical anomalies, and recent maintenance status. The health evaluation value of the candidate switch is determined based on the health characterization parameters. The health margin of the candidate switch is determined based on the difference between the health evaluation value and the corresponding minimum automatic operation health threshold.

6. The method for optimizing the operation control of ring main unit switches based on reinforcement learning according to claim 1, characterized in that, The predicted health margin migration data after executing the candidate action includes: The target candidate switch directly affected by the candidate action is determined, and the direct health consumption of the target candidate switch is determined according to the switch operation type of the candidate action, wherein the direct health consumption is expressed as: In the formula Indicates the execution of candidate actions For the first The direct health consumption generated by each candidate switch. For direct action identification, Indicates the first Each candidate switch performs an operation type The corresponding unit health consumption at that time; Based on the power supply path changes or load transfer relationships formed after the execution of the candidate actions, determine the associated candidate switches affected by the candidate actions, and determine the indirect health consumption of the associated candidate switches. Based on the direct health consumption, the indirect health consumption, and the current health margin of the candidate switch, health margin transition data after executing the candidate action is obtained, wherein the indirect health consumption is represented as: In the formula This represents the indirect health cost incurred by executing candidate action a on the i-th candidate switch. For the association of impact identifiers, This is the load transfer impact coefficient. Indicates the execution of candidate actions Before and after The change in current in the line or branch where each candidate switch is located. Indicates the first The rated current of each candidate switch.

7. The method for optimizing the operation control of ring main unit switches based on reinforcement learning according to claim 1, characterized in that, The step of determining the subsequent target operation item based on the subsequent operation data of the ring main unit includes: The completion conditions of each target operation item are compared with the subsequent topology data in the subsequent operation data of the ring network cabinet. Mark the target operation item whose completion condition is met in the subsequent topology data as the target operation item that has been achieved; The target operation items other than the target operation items that have been achieved are identified as subsequent target operation items.

8. A ring main unit switch operation optimization control system based on reinforcement learning, characterized in that, The system is used to execute the reinforcement learning-based ring main unit switch operation optimization control method according to any one of claims 1 to 7, the system comprising: The executable action determination module is configured to acquire the current operation task and the ring main unit operation data, and determine the current set of executable actions based on the current operation task and the ring main unit operation data; The target operation item determination module is configured to decompose the current operation task into multiple target operation items based on the target operation result corresponding to the current operation task, and determine the candidate switch operation corresponding to each target operation item; The subsequent operation prediction module is configured to predict the subsequent operation data of the ring main unit after executing the candidate action in the current set of executable actions, and determine the subsequent target operation item based on the subsequent operation data of the ring main unit. The executability verification module is configured to perform executability verification on the candidate switch operations corresponding to the subsequent target operation item based on the subsequent operation data of the ring network cabinet, and to include the candidate actions that pass the executability verification into the subsequent reachable action space; The decision output module is configured to input the current operation task, the ring main unit operation data, and the subsequent reachable action space into the reinforcement learning model, and the reinforcement learning model outputs the current switching operation command.

Images