Multi-time scale differentiated operation method and training method for active distribution network

Abstract

This application provides a multi-timescale differentiated operation method and training method for active distribution networks, which can be applied to the field of active distribution network technology. The method includes: acquiring a first operating state of the active distribution network; at the first timescale, generating discrete control actions based on the first operating state through a first decision network, the discrete control actions including switching commands for capacitors, tap adjustment commands for on-load tap-changing transformers, and operation mode selection commands; at a second timescale, generating continuous control actions based on the first operating state and the discrete control actions through a second decision network, the duration of the second timescale being shorter than the duration of the first timescale, the continuous control actions including control commands for flexible interconnection devices and control commands for photovoltaic inverters; and controlling the active distribution network by combining the discrete control actions and the continuous control actions as a hybrid action.

Description

Technical Field

[0001] This application relates to the field of active distribution network technology, and more specifically, to a multi-timescale differentiated operation method and training method for active distribution networks. Background Technology

[0002] With the increasing integration of distributed renewable energy sources into distribution networks and the growing complexity of load characteristics, active distribution networks face higher demands in terms of voltage control, power flow regulation, peak shaving and valley filling, and operational economy. However, the operation and control methods for distribution networks in related technologies have low flexibility and are difficult to adapt to active distribution networks. Summary of the Invention

[0003] In view of this, embodiments of this application provide a multi-timescale differentiated operation method and training method for active distribution networks.

[0004] One aspect of this application provides a multi-timescale differentiated operation method for active distribution networks, comprising: acquiring a first operating state of the active distribution network, wherein the first operating state includes load power data, distributed generation output data, node voltage amplitude data, energy storage state of charge data, capacitor switching position data, on-load tap-changing transformer tap position data, photovoltaic inverter operating state data, and port status data of flexible interconnection devices, wherein the flexible interconnection devices include energy storage enhanced reconfigurable soft switches; at the first timescale, generating discrete control actions based on the first operating state through a first decision network, wherein the discrete control actions include switching commands for capacitors, tap position adjustment commands for on-load tap-changing transformers, and operation mode selection commands; at the second timescale, generating continuous control actions based on the first operating state and discrete control actions through a second decision network, wherein the duration of the second timescale is shorter than the duration of the first timescale, and the continuous control actions include control commands for flexible interconnection devices and control commands for photovoltaic inverters, wherein the first decision network and the second decision network are trained based on a deep reinforcement learning framework; and controlling the active distribution network by treating the discrete control actions and continuous control actions as hybrid actions.

[0005] According to embodiments of this application, the operating modes include a voltage optimization mode and an economic optimization mode. In the voltage optimization mode, the operating objective is to maintain the node voltage of the active distribution network within a safe range. In the economic optimization mode, the operating objective is to minimize the operating cost of the active distribution network. Specifically, when the voltage safety margin of the active distribution network is less than or equal to a preset threshold, the voltage optimization mode is the target mode. When the voltage safety margin of the active distribution network is greater than the preset threshold, the economic optimization mode is the target mode.

[0006] According to an embodiment of this application, the flexible interconnect device is configured with a first energy storage unit and a second energy storage unit on the DC side; the continuous control action includes charging and discharging power commands to the first energy storage unit and the second energy storage unit to realize energy storage and release across time periods.

[0007] According to an embodiment of this application, the control period of the first time scale is on the hour level, and the control period of the second time scale is on the minute level; the active distribution network is controlled by a combination of discrete control actions and continuous control actions, including: controlling the active distribution network once based on discrete control actions within each period of the first time scale, and controlling the active distribution network multiple times based on continuous control actions with the second time scale as the period.

[0008] According to embodiments of this application, the control commands for the flexible interconnect device are generated under the constraints of port capacity allocation constraints, port power constraints, and port reconfiguration constraints. The port capacity allocation constraint indicates that the sum of the capacities of each port of the flexible interconnect device does not exceed the total port capacity, the port power constraint indicates that the apparent power of each port of the flexible interconnect device does not exceed the upper limit of the capacity of the corresponding port, and the port reconfiguration constraint indicates the on / off state between each port of the flexible interconnect device and the corresponding feeder.

[0009] According to the embodiments of this application, the operation of an active distribution network is limited by power balance constraints, node voltage constraints, and radial topology constraints. The power balance constraint indicates that the active power injected and reactive power injected into each node of the active distribution network are equal to the active power outflow and reactive power outflow of the corresponding node, respectively. The node voltage constraint indicates that the voltage amplitude of each node of the active distribution network is within a preset safe operating range. The radial topology constraint indicates that the feeder closure of the active distribution network does not form a loop.

[0010] According to an embodiment of this application, the method further includes: obtaining a second operating state of the active distribution network after hybrid action control; calculating a voltage deviation value and an operating cost value based on the second operating state, wherein the voltage deviation value is used to evaluate the regulation effect of the hybrid action on the node voltage, and the operating cost value includes network loss cost, energy storage attenuation cost and capacitor switching cost; and generating a reward signal based on the voltage deviation value and the operating cost value to evaluate the execution effect of the hybrid action.

[0011] Another aspect of this application provides a training method for multi-timescale differentiated operation of active distribution networks. The method includes: acquiring sample first operating states of the active distribution network, wherein the sample first operating states include sample load power data, sample distributed generation output data, sample node voltage amplitude data, sample energy storage state of charge data, sample capacitor switching position data, sample on-load tap-changing transformer position data, sample photovoltaic inverter operating state data, and sample flexible interconnection device port status data; at a first timescale, generating sample discrete control actions based on the sample first operating states through a first decision network, wherein the sample discrete control actions include switching commands for sample capacitors and on-load tap-changing transformers. The system generates gear adjustment commands and selection commands for sample operating modes. At a second time scale, a second decision network generates continuous control actions based on the sample's first operating state and discrete control actions. The duration of the second time scale is shorter than that of the first time scale. The continuous control actions include control commands for the sample flexible interconnect device and control commands for the sample photovoltaic inverter. The sample discrete control actions and continuous control actions are treated as a sample hybrid action to control the active distribution network, resulting in the sample's second operating state. Based on the voltage deviation and operating cost of the active distribution network under the sample's second operating state, a reward for the active distribution network is obtained. Based on the reward, the parameters of the first and second decision networks are updated.

[0012] According to an embodiment of this application, the reward includes a voltage deviation reward and an operating cost reward. Based on the reward, the parameters of the first decision network and the second decision network are updated, including: in each iteration of training, storing the first running state of the sample, the mixed action of the sample, the reward, and the second running state of the sample as empirical data in the empirical replay pool; randomly sampling at least one piece of empirical data from the empirical replay pool; and updating the parameters of the first decision network and the second decision network based on at least one piece of empirical data.

[0013] Another aspect of this application provides an electronic device, including: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0014] Another aspect of this application provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0015] Another aspect of this application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the steps of the above-described method.

[0016] According to embodiments of this application, discrete control actions for capacitors, on-load tap-changing transformers, and operating modes are generated at a first time scale, and continuous control actions for flexible interconnection devices and photovoltaic inverters are generated at a second time scale. The discrete and continuous control actions are used as hybrid actions to control the active distribution network. The duration of the first time scale is longer than that of the second time scale, thereby enabling cross-time-period regulation of the active distribution network. This expands the time-dimensional adjustment capability of the active distribution network, and can more effectively cope with fluctuations in new energy output and load uncertainty, improving the flexibility and robustness of the active distribution network operation. Attached Figure Description

[0017] The above and other objects, features and advantages of this application will become clearer from the following description of embodiments of this application with reference to the accompanying drawings, which will be explained in conjunction with the drawings.

[0018] Figure 1 An exemplary system architecture is shown, according to embodiments of this application, which can be applied to multi-timescale differentiated operation methods and training methods for active distribution networks.

[0019] Figure 2 A flowchart of a multi-timescale differentiated operation method for active distribution networks according to an embodiment of this application is shown.

[0020] Figure 3 A schematic diagram of the architecture of ER-SOP according to an embodiment of this application is shown.

[0021] Figure 4 A schematic diagram of the optimized architecture of a multi-timescale differentiated operation method for active distribution networks according to an embodiment of this application is shown.

[0022] Figure 5 A flowchart is shown for a training method for multi-timescale differentiated operation of an active distribution network according to an embodiment of this application.

[0023] Figure 6 A schematic diagram of experimental node relationships for a multi-timescale differentiated operation method for active distribution networks according to an embodiment of this application is shown.

[0024] Figure 7 A performance diagram of a multi-time-scale differentiated operation method and comparison algorithm for active distribution networks according to an embodiment of this application is shown.

[0025] Figure 8A A schematic diagram is shown of the node voltage that remains unchanged over the time scale in the multi-time-scale differentiated operation method for active distribution networks according to an embodiment of this application.

[0026] Figure 8BA schematic diagram is shown of the node voltage at a single time scale in the multi-time-scale differentiated operation method for active distribution networks according to an embodiment of this application.

[0027] Figure 8C A schematic diagram of the multi-time-scale differentiated operation method for active distribution networks according to an embodiment of this application is shown, which is a node voltage with dual time scales in terms of time scale.

[0028] Figure 8D An embodiment according to this application is shown. Figure 8A , Figure 8B , Figure 8C A diagram illustrating cost changes over three time scales.

[0029] Figure 9A A schematic diagram of the voltage amplitude distribution in the multi-time-scale differentiated operation method for active distribution networks according to an embodiment of this application is shown, including the fast and slow time scheduling of SOPs.

[0030] Figure 9B A schematic diagram of the voltage amplitude distribution of the multi-time-scale differentiated operation method for active distribution networks according to an embodiment of this application is shown in the case of R-SOP fast and slow time scheduling.

[0031] Figure 9C A schematic diagram of voltage amplitude distribution is shown in the multi-time-scale differentiated operation method for active distribution networks according to an embodiment of this application, including ER-SOP fast and slow time scheduling.

[0032] Figure 9D An embodiment according to this application is shown. Figure 9A , Figure 9B , Figure 9C A diagram illustrating cost changes over three time scales.

[0033] Figure 10 A schematic diagram of ER-SOP fast-slow time adaptive scheduling for a multi-time-scale differentiated operation method for active distribution networks according to an embodiment of this application is shown.

[0034] Figure 10 In the diagram, (a) represents the changes in energy storage and SOP power. Figure 10 (b) in the figure represents the change in the power of the photovoltaic inverter. Figure 10 (c) in the figure represents the change in active power at SOP. Figure 10 In the figure, (d) represents the change in reactive power at SOP. Figure 10 In the figure, (e) represents the change in the capacitor range distribution. Figure 10 (f) in the figure represents the change in the on-load tap changer position distribution.

[0035] Figure 11 A block diagram of an electronic device suitable for implementing the methods described above, according to an embodiment of this application, is illustrated schematically. Detailed Implementation

[0036] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0037] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0038] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0039] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0040] In the embodiments of this application, the collection, updating, analysis, processing, use, transmission, provision, disclosure, and storage of data (e.g., including but not limited to user personal information) comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures have been taken to prevent unauthorized access to user personal information data and to safeguard user personal information security and network security.

[0041] In the embodiments of this application, the user's authorization or consent was obtained before obtaining or collecting the user's personal information.

[0042] The distribution network control methods in related technologies usually rely on fixed topologies or single flexible resources, which are difficult to adapt to the dynamic operation requirements in multiple time periods and multiple scenarios.

[0043] Compared to traditional switches, soft open points (SOPs) can quickly and continuously adjust the active power between connected feeders and provide reactive power support to both sides as needed, achieving the advantages of closed-loop operation without its disadvantages. Furthermore, SOPs have extremely fast response times, quickly addressing fluctuations in distributed energy resources and loads. Traditional SOP structures have expanded from two-port to multi-port forms, typically combined with model-driven methods such as optimal power flow for control. However, under complex time-varying conditions, fixed-objective, fixed-mode control frameworks still struggle to meet the multi-objective, adaptive operation requirements of active distribution networks.

[0044] To enhance the control flexibility of distribution networks, reconfigurable soft-open points (R-SOPs) can be selected. By selecting switches and asymmetric voltage source converters (VSCs), flexible interconnection between multiple feeders can be achieved, expanding the control space while reducing overall converter capacity requirements. To improve the feasibility and cost-effectiveness of R-SOP deployment, multiple feeders need to be selected in geographically adjacent areas. However, R-SOPs primarily enhance power redistribution capabilities in the spatial dimension, and still fall short in addressing cross-time-period regulation needs caused by fluctuations in renewable energy output and time-varying load uncertainties. In contrast, energy storage-integrated soft-open points (E-SOPs) further introduce energy storage units, providing both spatial and temporal flexibility. However, most related technologies still employ fixed optimization frameworks under preset objectives, making it difficult to achieve adaptive switching between different objectives based on operating conditions.

[0045] On the other hand, the high proportion of renewable energy integration increases the uncertainty of distribution network operation and also continuously increases the dimensionality of control variables and the complexity of decision-making. Traditional model-driven methods face significant challenges in terms of solution efficiency and online application. Deep Reinforcement Learning (DRL) has the ability to learn online and update policies, and can be well applied in the control of complex power systems.

[0046] Traditional SOP or R-SOP devices primarily rely on spatial power redistribution, thus limiting their ability to cope with temporal variations caused by fluctuations in renewable energy output and load uncertainties. Furthermore, control strategies with fixed objectives often struggle to dynamically balance voltage regulation and operational economy under changing operating conditions. Simultaneously, the hybrid decision-making characteristics resulting from discrete mode selection and continuous control actions also pose significant challenges to traditional optimization methods.

[0047] In view of this, embodiments of this application provide a multi-time-scale differentiated operation method for active distribution networks. By generating discrete control actions for capacitors, on-load tap-changing transformers, and operating modes at a first time scale, and generating continuous control actions for flexible interconnection devices and photovoltaic inverters at a second time scale, the discrete and continuous control actions are used as a hybrid action to control the active distribution network. The duration of the first time scale is longer than that of the second time scale, thereby enabling cross-time-period regulation of the active distribution network, expanding the time-dimensional adjustment capability of the active distribution network, more effectively coping with fluctuations in renewable energy output and load uncertainty, and improving the flexibility and robustness of the active distribution network operation.

[0048] Figure 1 An exemplary system architecture is shown, according to embodiments of this application, which can be applied to multi-timescale differentiated operation methods and training methods for active distribution networks. It should be noted that... Figure 1 The examples shown are merely examples of system architectures that can be applied to the embodiments of this application, in order to help those skilled in the art understand the technical content of this application, but do not mean that the embodiments of this application cannot be used in other devices, systems, environments or scenarios.

[0049] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0050] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social media platform software, etc. (for example only).

[0051] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0052] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0053] It should be noted that the multi-timescale differentiated operation method and training method for active distribution networks provided in this application embodiment can generally be executed by server 105. The multi-timescale differentiated operation method and training method for active distribution networks provided in this application embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Alternatively, the multi-timescale differentiated operation method and training method for active distribution networks provided in this application embodiment can also be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103, or by other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103.

[0054] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0055] Figure 2 A flowchart of a multi-timescale differentiated operation method for active distribution networks according to an embodiment of this application is shown.

[0056] like Figure 2 As shown, the method includes operations S210 to S240.

[0057] In operation S210, the first operating state of the active distribution network is obtained.

[0058] The first operating status includes load power data, distributed power output data, node voltage amplitude data, energy storage charge status data, capacitor switching status data, on-load tap changer switch status data, photovoltaic inverter operating status data, and port status data of flexible interconnection devices, including energy storage enhanced reconfigurable soft switches.

[0059] During operation S220, at the first time scale, discrete control actions are generated by the first decision network based on the first operating state.

[0060] Discrete control actions include switching commands for capacitors, tap position adjustment commands for on-load tap-changing transformers, and operation mode selection commands.

[0061] In operation S230, at the second time scale, continuous control actions are generated by the second decision network based on the first operating state and discrete control actions.

[0062] The duration of the second time scale is shorter than that of the first time scale. Continuous control actions include control commands for flexible interconnected devices and control commands for photovoltaic inverters. The first decision network and the second decision network are trained based on a deep reinforcement learning framework.

[0063] When operating S240, discrete control actions and continuous control actions are combined into a single action to control the active power distribution network.

[0064] An active distribution network can be a traditional distribution network with a large number of distributed power sources (distributed photovoltaic, decentralized wind power, energy storage systems, gas turbines, etc.) connected to it. These distributed power sources will actively inject power into the grid.

[0065] Load power data can be the active and reactive power data of feeders, nodes, etc. in an active distribution network. Distributed generation output data can be the active and reactive power values ​​currently injected into the grid by each distributed generation connected to the active distribution network. Node voltage amplitude data can be the current voltage amplitude of each node in the active distribution network. Energy storage state of charge data can be the current percentage of remaining energy in the energy storage unit. Capacitor switching position data can be the current operating position of the capacitor. On-load tap changing transformer position data can be the current tap changer position of the on-load tap changing transformer. Photovoltaic inverter operating status data can be the current output power, grid connection status, and operating mode of each photovoltaic inverter. Flexible interconnection device port status data can be the current voltage, power, and on / off status of each port of the flexible interconnection device. Flexible interconnection device can be an Energy-storage-enhanced Reconfigurable Soft Open Point (ER-SOP).

[0066] Discrete control actions can be actions used to control active distribution networks at a first time scale. Discrete control actions can be, for example, control actions targeting the current operating position of capacitors, the switching position of on-load tap-changing transformers, and the selection of operating modes.

[0067] Continuous control actions can be actions used to control active distribution networks on a second time scale. Continuous control actions can be, for example, control actions targeting the on-time of flexible interconnect devices or the on-time of photovoltaic inverters.

[0068] The first decision network can be pre-trained based on a deep reinforcement learning framework and can generate discrete control actions based on the first operating state. The second decision network can also be pre-trained based on a deep reinforcement learning framework and can generate continuous control actions based on the first operating state and the discrete control actions. The training process of the first and second decision networks will be described later.

[0069] Discrete and continuous control actions can be used as a hybrid approach to control active power distribution networks. The duration of the first time scale is longer than that of the second time scale; for example, the duration of the first time scale can include two, three, four, or five durations of the second time scale. Continuous control actions operate within the second time scale. That is, after a discrete control action is executed, a continuous control action will be executed multiple times within the first time scale, and each execution can be an action updated through the second decision network.

[0070] According to embodiments of this application, discrete control actions for capacitors, on-load tap-changing transformers, and operating modes are generated at a first time scale, and continuous control actions for flexible interconnection devices and photovoltaic inverters are generated at a second time scale. The discrete and continuous control actions are used as hybrid actions to control the active distribution network. The duration of the first time scale is longer than that of the second time scale, thereby enabling cross-time-period regulation of the active distribution network. This expands the time-dimensional adjustment capability of the active distribution network, and can more effectively cope with fluctuations in new energy output and load uncertainty, improving the flexibility and robustness of the active distribution network operation.

[0071] To implement the multi-timescale differentiated operation method for active distribution networks as described in this application, ER-SOP modeling can be performed first. Compared to the traditional R-SOP, E-R-SOP, by introducing energy storage units, further achieves cross-time period energy management while maintaining feeder reconfiguration and flexible power regulation capabilities. This structure effectively alleviates the port capacity constraints of R-SOP, improves the continuity and robustness of voltage regulation, reduces dependence on demand response, and significantly enhances the distribution network's adaptability to high-proportion renewable energy integration.

[0072] According to embodiments of this application, the control commands for the flexible interconnect device are generated under the constraints of port capacity allocation constraints, port power constraints, and port reconfiguration constraints. The port capacity allocation constraint indicates that the sum of the capacities of each port of the flexible interconnect device does not exceed the total port capacity, the port power constraint indicates that the apparent power of each port of the flexible interconnect device does not exceed the upper limit of the capacity of the corresponding port, and the port reconfiguration constraint indicates the on / off state between each port of the flexible interconnect device and the corresponding feeder.

[0073] The port capacity allocation constraint can be expressed by the following formula (1).

[0074] (1);

[0075] in, This represents the set of ports for a flexible interconnect device. This represents the capacity of the nth port. This represents the capacity ratio of the nth port. Indicates the total capacity of the port. This indicates the total number of ports.

[0076] Port power constraints can be expressed by the following formulas (2) to (5).

[0077] (2);

[0078] (3);

[0079] (4);

[0080] (5);

[0081] in, Represents a set of optional nodes. , , Let represent the DC-side active power, AC-side active power, and reactive power of node i at time t, respectively. Let represent the active power loss of node i at time t. Represents the loss function. Let represent the apparent power of node i at time t.

[0082] ER-SOP can perform topology reconfiguration based on the power transmission requirements of the feeders between VSCs. The port reconfiguration constraints can be expressed by the following formulas (6) and (7).

[0083] (6);

[0084] (7);

[0085] in, This means that only one feeder is connected to each of the VSC's n ports at a time. This indicates the feeder selection status and is a (0, 1) variable, where n is the port number. A value of 0 indicates that node i is not connected to the nth port. A value of 1 indicates that node i is connected to the nth port.

[0086] Figure 3 A schematic diagram of the architecture of ER-SOP according to an embodiment of this application is shown.

[0087] like Figure 3 As shown, the flexible interconnect device can be installed on the AC side, and the feeder is selected through a feeder selection model. The flexible interconnect device has two energy storage units on the DC side, namely a first energy storage unit and a second energy storage unit. Continuous control actions include charging and discharging power commands to the first and second energy storage units to achieve energy storage and release across time periods.

[0088] Even if one energy storage unit fails, the other energy storage unit can still maintain the overall safe operation. While retaining the original feeder reconfiguration and flexible power regulation capabilities, ER-SOP further possesses cross-time period energy management capabilities. As shown in formula (8), a time-coupled SOC dynamic constraint is introduced into the spatial power balance constraint, thus providing a unified physical basis for subsequent differentiated adaptive operation and intelligent control.

[0089] (8);

[0090] in, , , These represent the minimum charging / discharging power, power, and maximum power of the energy storage unit, respectively. , These represent the charging and discharging states of the energy storage unit at time t, respectively. This represents the storage capacity of the energy storage unit at time t. , These represent the charging efficiency and discharging efficiency of the energy storage unit, respectively. Represents the energy loss coefficient, where, Represents the energy storage charging state variable (0 or 1). The variable represents the energy storage discharge state (0 or 1), and T represents time.

[0091] Figure 4 A schematic diagram of the optimized architecture of a multi-timescale differentiated operation method for active distribution networks according to an embodiment of this application is shown.

[0092] like Figure 4As shown, for better illustration, the multi-time-scale two-stage optimization architecture of this application divides each day into 24 hours, and each hour into 4 time intervals. The first time scale is 1 hour long, and the second time scale is 15 minutes long. The first time scale is the slow scale, which generates discrete control actions with a control cycle on the order of hours, including switching commands for capacitors, tap adjustment commands for on-load tap-changing transformers, and operation mode selection commands. The second time scale is the fast scale, which generates continuous control actions with a control cycle on the order of minutes, including control commands for flexible interconnect devices and control commands for photovoltaic inverters.

[0093] like Figure 4 As shown, within each first time scale period, the active distribution network is controlled once based on discrete control actions, and then controlled multiple times based on continuous control actions with a second time scale period. The objective function can be a function used to calculate whether the discrete and continuous control actions are reasonable.

[0094] According to embodiments of this application, the operating modes include a voltage optimization mode and an economic optimization mode. In the voltage optimization mode, the operating objective is to maintain the node voltage of the active distribution network within a safe range. In the economic optimization mode, the operating objective is to minimize the operating cost of the active distribution network. Specifically, when the voltage safety margin of the active distribution network is less than or equal to a preset threshold, the voltage optimization mode is the target mode; when the voltage safety margin of the active distribution network is greater than the preset threshold, the economic optimization mode is the target mode.

[0095] The preset threshold can be 0. The voltage safety margin can be determined based on whether the node voltage is within the voltage deviation range. For example, in a 10kV active distribution network, the voltage deviation range can be (0.93pu, 1.07pu). If the node voltage is not within the voltage deviation range, the voltage safety margin can be less than or equal to 0, and the voltage optimization mode will be used as the target mode. If the node voltage is within the voltage deviation range, the voltage safety margin can be greater than 0, indicating that the voltage is normal, and the economic optimization mode will be used as the target mode.

[0096] Based on the voltage safety margin of the active distribution network, the control operation mode switches between voltage optimization mode and economic optimization mode. When the voltage safety margin of the active distribution network is less than or equal to a preset threshold, i.e., when the voltage exceeds the limit, priority is given to maintaining voltage stability to reduce safety risks. When the voltage safety margin of the active distribution network is greater than the preset threshold, i.e., when the voltage is stable, the focus shifts to the economic efficiency of operation.

[0097] (9);

[0098] in, This represents the voltage safety margin of node i at time t. , Represents a conditional function. This represents the node voltage of node i at time t. In the case of ≥1, =0, =1, in When it is less than 1, =1, =0; This indicates the lower limit of the node voltage. This indicates the upper limit of the node voltage. Taking a voltage deviation range of (0.93pu, 1.07pu) as an example, the lower limit of the node voltage is 0.93pu, and the upper limit of the node voltage is 1.07pu.

[0099] With node voltage Taking 1.10 PU as an example, at this time, =0, =1, based on the voltage safety margin obtained from formula (9) = (1.10 - 0.93) * 0 + (1.07 - 1.10) * 1 = -0.03 pu. When the node voltage is within the voltage deviation range (0.93 pu, 1.07 pu), When the node voltage is outside the voltage deviation range (0.93 pu, 1.07 pu), .

[0100] exist In this case, the voltage optimization mode is used as the target mode, focusing on voltage quality.

[0101] (10);

[0102] (11);

[0103] in, This indicates minimizing the voltage deviation. This represents the voltage deviation at node i at time t; This indicates a safety constraint penalty. If the voltage does not exceed the limit, then... If the voltage exceeds the limit, then This is the punishment that is imposed.

[0104] In this context, the energy storage unit serves as an auxiliary regulation resource in the voltage optimization mode, used to enhance voltage recovery capability rather than to perform economic optimization.

[0105] exist In this context, the economic optimization model, as the target model, focuses on operating costs.

[0106] (12)

[0107] (13);

[0108] in, Indicates operating costs, , , These represent network loss cost, energy storage degradation cost, and capacitor bank (CB) operating cost, respectively. This represents the active power of branch ij at time t. This represents the reactive power of branch ij at time t. This represents the impedance of branch ij at time t. , , These represent the network loss cost coefficient, energy storage dispatch cost coefficient, and CB dispatch cost coefficient, respectively. and The charging and discharging power of the energy storage at node i in time period t are respectively represented. This represents the number of actions of CB at node i within time period t. This represents the set of nodes in the optional node set that are equipped with energy storage. This represents the set of nodes in the optional node set that have capacitors configured. Indicates the charging power coefficient; This represents the discharge power coefficient.

[0109] The fundamental difference between voltage optimization mode and economic optimization mode lies in the different nature of their control objectives. By decoupling economic optimization from voltage regulation, the proposed dual-operation mode framework avoids objective conflicts and can achieve adaptive operation under time-varying conditions.

[0110] According to the embodiments of this application, the operation of an active distribution network is limited by power balance constraints, node voltage constraints, and radial topology constraints. The power balance constraint indicates that the active power injected and reactive power injected into each node of the active distribution network are equal to the active power outflow and reactive power outflow of the corresponding node, respectively. The node voltage constraint indicates that the voltage amplitude of each node of the active distribution network is within a preset safe operating range. The radial topology constraint indicates that the feeder closure of the active distribution network does not form a loop.

[0111] The power balance constraint can be expressed by the following formulas (14) and (15).

[0112] (14);

[0113] (15);

[0114] in, This represents the active power of node i at time t. This represents the reactive power of node i at time t. This represents the voltage at node i at time t. This represents the line conductance at time t. This represents the line susceptance at time t. This represents the voltage at node j at time t. Indicates angle.

[0115] The node voltage constraint can be expressed by the following formula (16).

[0116] (16);

[0117] in, , These represent the lower and upper limits of the node voltage, respectively. As an example, the voltage amplitude is [0.93, 1.07] pu.

[0118] The tie switch is a discrete device, through The value represents the switch state, 1 when closed and 0 when open. Assume each branch is equipped with a tie switch. The radial topology constraint can be represented by the following formula (17).

[0119] (17);

[0120] in, Indicates the total number of nodes. This indicates the total number of substations. Indicates the switching state of branch ij. This represents the set of branches.

[0121] According to embodiments of this application, the method may further include: obtaining a second operating state of the active distribution network after hybrid action control; calculating a voltage deviation value and an operating cost value based on the second operating state, wherein the voltage deviation value is used to evaluate the regulation effect of the hybrid action on the node voltage, and the operating cost value includes network loss cost, energy storage attenuation cost, and capacitor switching cost; and generating a reward signal based on the voltage deviation value and the operating cost value to evaluate the execution effect of the hybrid action.

[0122] When hybrid actions are applied to an active distribution network, various operating parameters of the network will change (such as voltage increases or decreases, power redistribution, etc.). The system needs to re-collect these changed data, i.e., the second operating state, including updated node voltages, energy storage SOC, line power, etc.

[0123] The updated actual voltage of each node is compared with its rated voltage, for example, 1.0 pu, and the difference is calculated node by node. The absolute value or the sum of squares is then taken to obtain the voltage deviation value. The smaller the deviation value, the closer the node voltage is to the rated value, and the better the voltage quality; the larger the deviation value, the worse the voltage quality. This value directly reflects the voltage correction effect of the hybrid operation.

[0124] Network loss cost can be the active power loss generated when current flows through the line; the greater the power flow and the longer the path, the greater the loss. The lifespan of an energy storage unit is shortened with each charge and discharge cycle; energy storage degradation cost can be the economic cost calculated by offsetting the lifespan loss of the energy storage unit. Capacitor switching cost can be the maintenance or depreciation cost incurred by mechanical wear and electrical shocks for each capacitor operation (connection or disconnection).

[0125] A reward signal can be generated by weighting the voltage deviation value and the operating cost value according to a certain weight. The sign and magnitude of this signal can directly reflect the quality of the mixed operation.

[0126] By acquiring the second operating state after the mixed action and calculating the voltage deviation value and operating cost value, the execution effect of each mixed action can be quantified and evaluated, forming a complete closed loop of decision-making, execution, evaluation and optimization.

[0127] Figure 5 A flowchart is shown for a training method for multi-timescale differentiated operation of an active distribution network according to an embodiment of this application.

[0128] like Figure 5 As shown, the method includes operations S510 to S560.

[0129] When operating S510, obtain the first operating state of the active distribution network sample.

[0130] The first operating state of the sample includes sample load power data, sample distributed generation output data, sample node voltage amplitude data, sample energy storage state of charge data, sample capacitor switching data, sample on-load tap-changing transformer tap data, sample photovoltaic inverter operating status data, and sample flexible interconnection device port status data. The data in the first operating state of the sample can be data generated during the actual operation of the active distribution network, or data obtained through simulation models.

[0131] When operating S520, at the first time scale, the first decision network generates discrete control actions for the sample based on the first running state of the sample.

[0132] The discrete control actions for the sample include switching commands for the sample capacitor, tap adjustment commands for the sample on-load tap-changing transformer, and selection commands for the sample operating mode.

[0133] During operation S530, at the second time scale, the second decision network generates continuous control actions for the sample based on the first running state of the sample and the discrete control actions of the sample.

[0134] The duration of the second time scale is shorter than that of the first time scale. The continuous control actions of the sample include control commands for the sample flexible interconnect device and control commands for the sample photovoltaic inverter.

[0135] In operation S540, the sample discrete control action and the sample continuous control action are treated as a sample mixed action to control the active distribution network, thereby obtaining the sample second operating state of the active distribution network.

[0136] When operating S550, the reward for the active distribution network is obtained based on the voltage deviation and operating cost of the active distribution network in the second operating state of the sample.

[0137] When operating S560, the parameters of the first decision network and the second decision network are updated based on the reward.

[0138] Before updating the parameters of the first and second decision networks, a state model, action model, reward function, state transition, and state-action value function can be established to formalize the energy storage and dynamic reconfiguration collaborative optimization problem into a Markov Decision Process (MDP). The state model, action model, reward function, state transition, and state-action value function are respectively shown in formulas (18) to (21).

[0139] (18);

[0140] in, This represents the running state at time t, which is also the first running state. , , , , , , These represent the injected active power, reactive power, photovoltaic active power, wind power active power, energy storage state of charge, capacitor switching vector, and on-load tap changer tap at time t, respectively.

[0141] (19);

[0142] in, This represents the mixed actions at time t, subject to runtime constraints. This represents the ER-SOP control strategy at time t. This represents the photovoltaic control strategy at time t. This represents the capacitor control strategy at time t. This represents the on-load tap-changing transformer control strategy at time t. This indicates the operating mode at time t.

[0143] After the first decision network generates the sample discrete control action, the second decision network can apply the sample discrete control action generated by the first decision network to the sample first operating state of the sample active distribution network, and generate sample continuous control action based on the sample active distribution network after the sample discrete control action is applied.

[0144] (20);

[0145] in, , Let these represent the voltage optimization function and the runtime optimization function at time t, respectively. , Let represent the normalized voltage optimization function and the runtime optimization function at time t, respectively. This represents the reference value of the voltage deviation at time t. This represents the baseline operating cost at time t. The value is 1 if the operating mode is voltage optimization mode, and 0 otherwise. The value is 1 if the operating mode is economic optimization mode, and 0 otherwise. Represents the objective function reward. This represents the penalty function.

[0146] (twenty one);

[0147] in, This indicates the state at the next moment, that is, the second running state. Represents the state transition function. This indicates uncertainties.

[0148] The state-value function can be used to evaluate the quality of mixed actions, as shown in the following formula (22).

[0149] (twenty two);

[0150] in, Represents the state-value function. Let represent the policy expectation, and k represent the number of time steps to move forward from the current time t. It is the discount factor at step k. This represents the reward at step k.

[0151] Define the optimal policy given the reward as follows: This can maximize the reward, as shown in the following formula (23).

[0152] (twenty three);

[0153] The first and second decision networks can be a unified deep reinforcement learning framework for hybrid action GA-SAC, achieving fully differentiable optimization of discrete and continuous control. Differentiated operating modes are embedded in the discrete policy space, eliminating rule-based mode switching mechanisms and thus achieving adaptive mode selection.

[0154] The first decision network can be a discrete control network based on the Gumbel-Softmax function, which uses the Gumbel-Softmax function to perform probabilistic selection of discrete control actions. It is suitable for deep reinforcement learning models with backpropagation. The action mechanism is shown in the following formula (24).

[0155] (twenty four);

[0156] in, The magnitude of the policy vector for the Capacitor Bank (CB) is given. For the on-load tap-changing transformer strategy vector magnitude, Select a modulus value for the operating mode. It is the discrete action probability vector rate. The inputs to a discrete policy network are state and preferences. Let H be H random vectors uniformly distributed on the interval from 0 to 1, and let G be noise used to introduce randomness. These are hyperparameters, where h is an H-dimensional neural network vector. Let h be the m-th h vector.

[0157] The Soft Actor-Critic (SAC) algorithm is an agent-based algorithm that can include an action network and an evaluation network. The action network generates energy-storing actions based on the current state, directly outputting continuous action values. The goal of policy optimization is to maximize the reward and policy. The evaluation network assesses the value of the generated control actions to ensure maximum reward. SAC employs a dual-Q network architecture to reduce excessive bias in the estimates. Each Q-network takes the agent's state observations and actions as input and outputs the corresponding Q-value estimate.

[0158] The action network is represented by the following formula (25).

[0159] (25);

[0160] in, Represents an action network. This represents the mean of the action distribution. Represents the standard deviation network. This is the extracted feature of a multilayer perceptron (MLP) with M connected layers.

[0161] (26);

[0162] in, This represents minimizing the policy loss function. Indicates the action network parameters. Indicates policy-based The action obtained, Indicating in strategy The state-value function under the following conditions Represents policy entropy. This represents the temperature parameter. A higher policy entropy indicates a wider exploration space and more flexible decision-making by the agent when interacting with the environment.

[0163] The action network is updated according to the following formula (27).

[0164] (27);

[0165] in, Let α represent the policy gradient of the action network, and α represent the learning rate.

[0166] The reinforcement learning framework based on maximum entropy enables agents to obtain higher rewards while maintaining high randomness in decision-making, thus balancing the relationship between action exploration and action adoption.

[0167] The evaluation network evaluates the actions output by the action network. The inputs are the agent's state and actions, and the output is the corresponding Q-function value.

[0168] (28);

[0169] Among them, F M Denotes the mapping of the Mth fully connected layer, F m This represents the mapping of the m-th fully connected layer.

[0170] During training, the parameters of the network are evaluated by minimizing the temporal error, and the loss functions are as follows: (29) and (30).

[0171] (29);

[0172] (30);

[0173] in, This represents minimizing the evaluation loss function. This represents the o-th evaluation network. This represents the evaluation network for the o-th objective. This represents the Q-function value of the evaluation network at time t, obtained based on the state prediction of the next time step. This represents the reward at time t obtained based on the reward function. This represents the total discount factor.

[0174] The target evaluation network and the evaluation network have the same structure, but the parameters are updated at a fixed frequency. The gradient parameter update process of the evaluation network can be expressed by the following formulas (31) and (32).

[0175] (31);

[0176] (32);

[0177] in, This represents the policy gradient of the evaluation network. This indicates the evaluation parameters of the network policy. Indicates the network policy parameters for evaluating the objective. This represents the soft update coefficient.

[0178] According to an embodiment of this application, the reward includes a voltage deviation reward and an operating cost reward. Based on the reward, the parameters of the first decision network and the second decision network are updated, including: in each iteration of training, storing the first running state of the sample, the mixed action of the sample, the reward, and the second running state of the sample as empirical data in the empirical replay pool; randomly sampling at least one piece of empirical data from the empirical replay pool; and updating the parameters of the first decision network and the second decision network based on at least one piece of empirical data.

[0179] After obtaining the second running state of a sample, the first running state, the mixed action, the reward, and the second running state of the sample can be stored as experience data in the experience replay pool. By sampling data from the experience replay pool, the parameters in the first and second decision networks can be updated.

[0180] The ER-SOP spatiotemporal coordinated control framework for active distribution networks proposed in this application introduces energy storage units based on the R-SOP's "feeder reconfigurability and asymmetric VSC port" structure. This enables the device to simultaneously possess cross-feeder power allocation capabilities and cross-time period energy buffering capabilities, thereby forming a "space-time" coordinated control mechanism. Unlike traditional R-SOPs, which mainly rely on spatial redistribution, the ER-SOP in this application further expands the temporal dimension adjustment capabilities of active distribution networks, which can more effectively cope with fluctuations in renewable energy output and load uncertainties, improving the flexibility and robustness of active distribution network operation.

[0181] This application embodiment achieves adaptive switching of control objectives by constructing a differentiated operation mechanism with dual operating modes: Addressing the differentiated requirements for voltage safety and economy under different operating scenarios in active distribution networks, this application embodiment constructs two operating modes: minimizing voltage deviation and minimizing operating costs. The selection of operating modes and equipment reconfiguration are uniformly incorporated into the discrete action space. This mechanism overcomes the shortcomings of traditional fixed-target setting methods that are difficult to adapt to dynamic operating condition changes, enabling the control strategy to switch optimization guidance online according to the system operating status, thereby improving the adaptability and scenario adaptability of operating decisions.

[0182] The reinforcement learning solution method proposed in this application, applicable to hybrid action spaces, achieves joint optimization of discrete and continuous control. Furthermore, addressing the strong coupling between discrete reconfiguration decision-making, operating mode switching, and continuous power regulation in ER-SOP control, this application employs SAC to learn the continuous control strategy and introduces the Gumbel-Softmax function to achieve differentiable sampling and end-to-end training of discrete control actions, thereby constructing a unified solution framework suitable for hybrid action spaces. This method improves the stability and convergence performance of strategy learning, providing an effective tool for online control of active distribution networks with complex and flexible resource participation.

[0183] Figure 6 A schematic diagram of experimental node relationships for a multi-timescale differentiated operation method for active distribution networks according to an embodiment of this application is shown.

[0184] like Figure 6As shown, an improved 123-node distribution system is selected to verify the multi-time-scale differentiated operation method for active distribution networks in this application embodiment. The equipment capacity configuration parameters for photovoltaic and capacitor banks are shown in Table 1 below. Nodes 32, 56, 62, 101, and 117 are equipped with five-port ER-SOPs with a capacity of 1MVA, a loss factor of 0.02, and an energy storage capacity of 2*2MW. Nodes 30, 48, 71, 88, and 123 are connected to capacitor banks with a capacity of 0.5MWh. An on-load tap-changing transformer is connected to the root node, and nodes 10, 32, 34, 81, 93, 96, 102, 107, 115, and 119 are connected to 0.5MW of photovoltaic power.

[0185] Table 1

[0186]

[0187] Figure 7 A performance diagram of a multi-time-scale differentiated operation method and comparison algorithm for active distribution networks according to an embodiment of this application is shown.

[0188] like Figure 7 As shown, the multi-time-scale differentiated operation method for active distribution networks in this application embodiment is compared with three other comparative algorithms. The multi-time-scale differentiated operation method for active distribution networks in this application embodiment is GS-SAC, and the comparative algorithms are DDPG-SAC, DDPG-TD3, and DDQN-SAC. All four algorithms adopt a multi-agent distributed decision-making architecture and maintain a dual-scale action network for fast and slow time. Each round contains 24 time periods and 96 time intervals. All algorithms exhibit significant fluctuations in the early stages of training, stemming from agent exploration and experience accumulation. GS-SAC reaches convergence first after 2000 rounds, demonstrating the highest learning efficiency. It is evident that GS-SAC in this application embodiment outperforms the comparative algorithms in terms of cumulative reward and policy stability.

[0189] Figure 8A A schematic diagram is shown of the node voltage that remains unchanged over the time scale in the multi-time-scale differentiated operation method for active distribution networks according to an embodiment of this application. Figure 8B A schematic diagram is shown of the node voltage at a single time scale in the multi-time-scale differentiated operation method for active distribution networks according to an embodiment of this application. Figure 8C A schematic diagram of the multi-time-scale differentiated operation method for active distribution networks according to an embodiment of this application is shown, which is a node voltage with dual time scales in terms of time scale. Figure 8D An embodiment according to this application is shown. Figure 8A , Figure 8B , Figure 8C A diagram illustrating cost changes over three time scales.

[0190] like Figures 8A-8D Taking a first time scale of 1 hour and a second time scale of 15 minutes as an example, the times in the figure are displayed in 15-minute increments. Without any changes, voltage exceeds limits on a large scale between 8:00 and 15:00. With a single time scale, voltage exceedances are reduced, but still occur between 12:00 and 15:00. With a dual time scale, voltage exceedances no longer occur, and power quality is guaranteed through optimization strategies. The cost changes for the three time scales are 3268, 1685, and 1158, respectively. The single and dual time scales save 48.44% and 64.57% of the cost compared to no optimization, respectively. The dual time scale saves 31.28% of the cost compared to the single time scale. Table 2 shows the voltage and cost analysis under the three time scales.

[0191] Table 2

[0192]

[0193] Figure 9A A schematic diagram of the voltage amplitude distribution in the multi-time-scale differentiated operation method for active distribution networks according to an embodiment of this application is shown, including SOP fast and slow time scheduling. Figure 9B A schematic diagram of the voltage amplitude distribution in the multi-time-scale differentiated operation method for active distribution networks according to an embodiment of this application is shown, including the R-SOP fast and slow time scheduling. Figure 9C A schematic diagram of the voltage amplitude distribution of the multi-time-scale differentiated operation method for active distribution networks according to an embodiment of this application is shown in the case of ER-SOP fast and slow time scheduling. Figure 9D An embodiment according to this application is shown. Figure 9A , Figure 9B , Figure 9C A diagram illustrating cost changes over three time scales.

[0194] like Figures 9A-9DUnder SOP (Standard Operating Procedure), the voltage distribution exhibits more pronounced hot and cold spots and stripe structures, indicating that under the influence of intraday load and photovoltaic output fluctuations, the voltage of some buses deviates significantly from the rated value during specific periods, resulting in insufficient voltage margin. With R-SOP, the overall voltage distribution tends to be concentrated, and the extreme voltage region is reduced, indicating that the reconfigurable port capacity and more flexible power flow regulation can effectively improve the voltage deviation of some nodes; however, local color band aggregation is still visible during periods of high fluctuation and at the end of the feeder bus section, reflecting local insufficient regulation under the constraints of capacity and control strategies. In contrast, ER-SOP exhibits the most uniform voltage thermal distribution, with further convergence of high / low voltage regions, enabling more stable maintenance of the entire grid voltage near the safe range and significantly reducing voltage fluctuations. These results demonstrate that ER-SOP, through enhanced cross-feeder power transfer capability and active and reactive power support capability, achieves a better match for differentiated voltage regulation needs at different times, thus demonstrating optimal voltage regulation performance.

[0195] The cumulative operating costs of the three schemes, SOP, R-SOP, and ER-SOP, over 96 time periods were RMB 2740, RMB 2104, and RMB 1600, respectively. Compared to the traditional SOP, the cumulative costs of R-SOP and ER-SOP were reduced by approximately 23.21% and 41.61%, respectively; compared to R-SOP, the cumulative cost of ER-SOP was further reduced by 23.95%. The results indicate that ER-SOP performs best in reducing the overall system operating cost. Table 3 shows the voltage and cost analysis of the three different SOPs.

[0196] Table 3

[0197]

[0198] Figure 10 A schematic diagram of ER-SOP fast-slow time adaptive scheduling for a multi-time-scale differentiated operation method for active distribution networks according to an embodiment of this application is shown.

[0199] like Figure 10 As shown, Figure 10 In the diagram, (a) represents the changes in energy storage and SOP power. Figure 10 (b) in the figure represents the change in the power of the photovoltaic inverter. Figure 10 (c) in the figure represents the change in active power at SOP. Figure 10 In the figure, (d) represents the change in reactive power at SOP. Figure 10 In the figure, (e) represents the change in the capacitor range distribution. Figure 10 In the figure (f), the change in the distribution of the on-load tap changer is shown. The ER-SOP achieves multi-objective regulation through adaptive switching of operating modes. Its operating characteristics are matched with the system status in real time, as shown below.

[0200] During periods of low fluctuation (periods 1–26 and 71–96): The operating mode is the economic optimization mode. During this phase, photovoltaic output is stable and there is no risk of voltage exceeding limits. ER-SOP aims for minimum adjustment costs: Energy storage strategy: Prioritize energy storage to avoid redundant discharge losses. SOP adjustment: Ensure system stability solely through basic power flow distribution, reducing operating energy consumption. Discrete equipment: Circuit breaker status is stable with no frequent switching; on-load tap changers maintain the 1.03 high-end position, reducing switching operation costs.

[0201] During peak PV fluctuation periods (periods 31–61): The operating mode is voltage optimization mode. When PV output surges and fluctuates dramatically, the system triggers a safety priority strategy, using multi-resource collaboration to suppress voltage overshoot and power flow congestion. Energy storage buffer: The energy storage system switches from charging to deep discharging to quickly respond to PV fluctuations and smooth out sudden changes in net load. SOP multi-port collaboration: Active power transfer; ports 56 / 101 output positive active power, directing surplus PV power to ports 32 / 60 / 117 for absorption, alleviating local feeder pressure; Reactive power support: All ports reach peak reactive power output, with ports 60 / 117 prioritizing voltage enhancement at the sensitive node at the 117 end, keeping fluctuations within ±0.03 pu. Discrete equipment linkage: The on-load tap changer tap is lowered from 1.03 to 0.985 to reduce the voltage reference; the CB triggers a small number of switches during periods 41–51, coordinating with SOP to optimize network topology.

[0202] Transition periods (26–31, 61–71): Collaborative optimization mode: During the transition phase of rapidly changing photovoltaic output, ER-SOP achieves seamless mode transition through smooth adjustment; Energy storage power: Avoids step power surges through charging and discharging; SOP pre-adjustment: Gradual increase and decrease of active / reactive power reserves adjustment margin for peak periods, balancing safety and economy. The ER-SOP mode selection mechanism achieves the optimization goal of "saving costs during low-fluctuation periods, ensuring safety during peak periods, and smooth transition during transition periods" through state perception, strategy switching, and multi-resource collaboration. Compared with the single control mode of traditional SOP, its adjustment resources are allocated on demand, improving voltage control accuracy while reducing operating costs, verifying the optimality of comprehensive performance.

[0203] Figure 11 A block diagram of an electronic device suitable for implementing the methods described above, according to an embodiment of this application, is illustrated schematically. Figure 11 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0204] like Figure 11As shown, an electronic device 1100 according to an embodiment of this application includes a processor 1101, which can perform various appropriate actions and processes according to a program stored in a ROM 1102 or a program loaded from a storage portion 1108 into a random access memory RAM 1103. The processor 1101 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1101 may also include onboard memory for caching purposes. The processor 1101 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0205] RAM 1103 stores various programs and data required for the operation of electronic device 1100. Processor 1101, ROM 1102, and RAM 1103 are interconnected via bus 1104. Processor 1101 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 1102 and / or RAM 1103. It should be noted that the programs may also be stored in one or more memories other than ROM 1102 and RAM 1103. Processor 1101 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0206] According to embodiments of this application, the electronic device 1100 may further include an input / output (I / O) interface 1105, which is also connected to a bus 1104. The electronic device 1100 may also include one or more of the following components connected to the input / output (I / O) interface 1105: an input section 1106 including a keyboard, mouse, etc.; an output section 1107 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1108 including a hard disk, etc.; and a communication section 1109 including a network interface card such as a LAN card, modem, etc. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the input / output (I / O) interface 1105 as needed. A removable medium 1111, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1110 as needed so that computer programs read from it can be installed into the storage section 1108 as needed.

[0207] According to embodiments of this application, the method flow according to embodiments of this application can be implemented as a computer software program. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1109, and / or installed from removable medium 1111. When the computer program is executed by processor 1101, it performs the functions defined in the system of embodiments of this application. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0208] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0209] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0210] For example, according to embodiments of this application, a computer-readable storage medium may include the ROM 1102 and / or RAM 1103 described above and / or one or more memories other than ROM 1102 and RAM 1103.

[0211] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods provided in the embodiments of this application. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the methods provided in the embodiments of this application.

[0212] When the computer program is executed by the processor 1101, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0213] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 1109, and / or installed from the removable medium 1111. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0214] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0215] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations are not explicitly described in this application. In particular, without departing from the spirit and teachings of this application, the features described in the various embodiments of this application can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of this application.

[0216] The embodiments of this application have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of this application. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Without departing from the scope of this application, those skilled in the art can make various substitutions and modifications, all of which should fall within the scope of this application.

Claims

1. A multi-time-scale differentiated operation method for active distribution networks, characterized in that, The method includes: The first operating state of the active distribution network is obtained, wherein the first operating state includes load power data, distributed power output data, node voltage amplitude data, energy storage charge status data, capacitor switching position data, on-load tap changer position data, photovoltaic inverter operating status data, and port status data of flexible interconnection device, wherein the flexible interconnection device includes energy storage enhanced reconfigurable soft switch. At the first time scale, discrete control actions are generated by the first decision network based on the first operating state. The discrete control actions include switching commands for capacitors, tap adjustment commands for on-load tap-changing transformers, and selection commands for operating modes. At the second time scale, a second decision network generates continuous control actions based on the first operating state and the discrete control actions. The duration of the second time scale is shorter than that of the first time scale. The continuous control actions include control commands for the flexible interconnect device and control commands for the photovoltaic inverter. The first decision network and the second decision network are trained based on a deep reinforcement learning framework. The discrete control action and the continuous control action are combined into a single action to control the active power distribution network.

2. The method according to claim 1, characterized in that, The operating modes include voltage optimization mode and economic optimization mode. In voltage optimization mode, the operating objective is to maintain the node voltage of the active distribution network within a safe range. In economic optimization mode, the operating objective is to minimize the operating cost of the active distribution network. Specifically, when the voltage safety margin of the active distribution network is less than or equal to a preset threshold, the voltage optimization mode is used as the target mode; when the voltage safety margin of the active distribution network is greater than the preset threshold, the economic optimization mode is used as the target mode.

3. The method according to claim 1, characterized in that, The flexible interconnect device is equipped with a first energy storage unit and a second energy storage unit on the DC side; The continuous control action includes charging and discharging power commands to the first energy storage unit and the second energy storage unit to achieve energy storage and release across time periods.

4. The method according to claim 1, characterized in that, The control period for the first time scale is on the hour level, and the control period for the second time scale is on the minute level. The method of controlling the active power distribution network by combining the discrete control action and the continuous control action as a hybrid action includes: Within each period of the first time scale, the active distribution network is controlled once based on the discrete control action, and the active distribution network is controlled multiple times based on the continuous control action with the second time scale as the period.

5. The method according to any one of claims 1 to 4, characterized in that, The control commands for the flexible interconnect device are generated under constraints of port capacity allocation, port power, and port reconfiguration. The port capacity allocation constraint indicates that the sum of the capacities of all ports of the flexible interconnect device does not exceed the total port capacity; the port power constraint indicates that the apparent power of each port of the flexible interconnect device does not exceed the upper limit of the capacity of the corresponding port; and the port reconfiguration constraint indicates the on / off state between each port of the flexible interconnect device and the corresponding feeder.

6. The method according to any one of claims 1 to 4, characterized in that, The operation of the active distribution network is limited by power balance constraints, node voltage constraints, and radial topology constraints. The power balance constraint indicates that the active power injected and reactive power injected into each node of the active distribution network are equal to the active power outflow and reactive power outflow of the corresponding node, respectively. The node voltage constraint indicates that the voltage amplitude of each node of the active distribution network is within a preset safe operating range. The radial topology constraint indicates that the feeder closure of the active distribution network does not form a loop.

7. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Obtain the second operating state of the active power distribution network after the hybrid action control; The voltage deviation value and the operating cost value are calculated based on the second operating state. The voltage deviation value is used to evaluate the regulation effect of the hybrid action on the node voltage. The operating cost value includes network loss cost, energy storage attenuation cost and capacitor switching cost. A reward signal is generated based on the voltage deviation value and the operating cost value to evaluate the performance of the hybrid action.

8. A training method for multi-timescale differentiated operation of active distribution networks, characterized in that, The method includes: The sample first operating state of the active distribution network is obtained, wherein the sample first operating state includes sample load power data, sample distributed power output data, sample node voltage amplitude data, sample energy storage state of charge data, sample capacitor switching position data, sample on-load tap-changing transformer position data, sample photovoltaic inverter operating state data, and sample flexible interconnection device port status data. At the first time scale, a first decision network generates discrete control actions for the sample based on the first operating state of the sample. The discrete control actions include switching instructions for the sample capacitor, tap adjustment instructions for the sample on-load tap-changing transformer, and selection instructions for the sample operating mode. At the second time scale, a second decision network generates a continuous control action for the sample based on the first operating state of the sample and the discrete control action of the sample. The duration of the second time scale is shorter than the duration of the first time scale. The continuous control action for the sample includes control commands for the sample flexible interconnect device and control commands for the sample photovoltaic inverter. The sample discrete control action and the sample continuous control action are used as a sample mixed action to control the active power distribution network, thereby obtaining the sample second operating state of the active power distribution network; The reward for the active distribution network is obtained based on the voltage deviation and operating cost of the active distribution network under the second operating state of the sample. Based on the reward, update the parameters of the first decision network and the parameters of the second decision network.

9. The method according to claim 8, characterized in that, The rewards include a voltage deviation reward and an operating cost reward. Updating the parameters of the first decision network and the second decision network based on these rewards includes: In each iteration of training, the first running state of the sample, the mixed action of the sample, the reward, and the second running state of the sample are stored as experience data in the experience replay pool; At least one piece of empirical data is randomly sampled from the empirical replay pool; The parameters of the first decision network and the second decision network are updated based on at least one piece of empirical data.

10. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 9.

Images