Power distribution network dispatching method and device

Abstract

The application relates to a power distribution network scheduling method and device, comprising: acquiring energy prediction power and load prediction power of a power distribution network at a current time; determining an uncertainty interval according to the energy prediction power and the load prediction power; inputting the energy prediction power, the load prediction power and the uncertainty interval into a strategy network to obtain an exploration scheduling action and a risk tolerance threshold; determining node voltage and line transmission power of the power distribution network according to the exploration scheduling action, and determining a comprehensive violation degree; updating the strategy network according to the comprehensive violation degree and the risk tolerance threshold to obtain an updated strategy network; re-inputting the energy prediction power, the load prediction power and the uncertainty interval into the updated strategy network, and scheduling resources of the power distribution network based on a deterministic scheduling action output by the updated strategy network. The application has active risk avoidance capability in extreme disturbance, and takes into account economy in normal operation and physical safety in extreme disturbance.

Description

Technical Field

[0001] This application relates to the field of smart grid technology, and in particular to a distribution network dispatching method and apparatus. Background Technology

[0002] With the high proportion of new energy sources such as wind power and photovoltaics being integrated into the distribution network, the distribution network has transformed from a traditional one-way passive power supply network into an active distribution network with bidirectional interaction between source and load. New energy output exhibits randomness, volatility, and intermittency, with discrepancies between predicted and actual output values, and the load also displays strong time-varying characteristics. Existing distribution network dispatching technologies typically first utilize time-series forecasting models such as Long Short-Term Memory (LSTM) networks to perform interval forecasts of new energy output and load, providing a prediction error range and fluctuation range at a certain confidence level. Then, dispatching instructions for flexible resources such as energy storage and transferable loads are generated based on optimization algorithms or deep reinforcement learning algorithms.

[0003] However, interval-based scheduling methods typically only provide fixed confidence intervals, covering normal fluctuation scenarios, but cannot cover out-of-distribution scenarios such as extreme weather and sudden load surges beyond the confidence interval. Deep reinforcement learning-based scheduling methods aim to maximize expected returns, which is a risk-neutral decision-making approach. In extreme scenarios, they may sacrifice physical safety for economic gains, and the nonlinearity of power flow in distribution networks makes it difficult to obtain the probability distribution of physical exceedance levels. Furthermore, tail risk metrics based on conditional value of risk are difficult to directly embed into the reinforcement learning training process. Therefore, related technologies struggle to simultaneously balance scheduling economics in normal scenarios and physical safety in extreme scenarios during reinforcement learning training. Summary of the Invention

[0004] Therefore, it is necessary to provide a distribution network dispatching method and device that can maintain economic efficiency during normal operation and have the ability to actively avoid risks during extreme disturbances, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a power distribution network dispatching method, including:

[0006] Obtain the predicted energy power and predicted load power of the distribution network at the current moment;

[0007] An uncertainty interval is determined based on the predicted energy power and the predicted load power; the uncertainty interval characterizes the normal fluctuation range of the energy and load of the distribution network.

[0008] The predicted energy power, the predicted load power, and the uncertainty interval are input into a trained policy network to obtain the exploratory scheduling action and risk tolerance threshold output by the policy network; the risk tolerance threshold represents the tolerance boundary of the comprehensive violation degree.

[0009] The node voltage and line transmission power of the distribution network are determined based on the exploration and scheduling actions, and the comprehensive violation degree is determined based on the node voltage and the line transmission power; the comprehensive violation degree characterizes the degree to which the node voltage and / or line transmission power of the distribution network exceeds the limit.

[0010] The policy network is updated based on the overall violation rate and the risk tolerance threshold to obtain the updated policy network.

[0011] The predicted energy power, the predicted load power, and the uncertainty interval are re-inputted into the updated strategy network, and the resources of the distribution network are scheduled based on the deterministic scheduling actions output by the updated strategy network.

[0012] Secondly, this application also provides a power distribution network dispatching device, comprising:

[0013] The acquisition module is used to acquire the predicted energy power and predicted load power of the distribution network at the current moment.

[0014] The first determining module is used to determine an uncertainty interval based on the predicted energy power and the predicted load power; the uncertainty interval characterizes the normal fluctuation range of the energy and load of the distribution network.

[0015] The input module is used to input the predicted energy power, the predicted load power, and the uncertainty interval into the trained policy network to obtain the exploratory scheduling action and risk tolerance threshold output by the policy network; the risk tolerance threshold represents the tolerance boundary of the comprehensive violation degree.

[0016] The second determining module is used to determine the node voltage and line transmission power of the distribution network based on the exploration and scheduling action, and to determine the comprehensive violation degree based on the node voltage and the line transmission power; the comprehensive violation degree characterizes the degree to which the node voltage and / or line transmission power of the distribution network exceeds the limit;

[0017] The update module is used to update the policy network based on the comprehensive violation rate and the risk tolerance threshold to obtain the updated policy network.

[0018] The scheduling module is used to re-input the predicted energy power, the predicted load power, and the uncertainty interval into the updated strategy network, and to schedule the resources of the distribution network based on the deterministic scheduling actions output by the updated strategy network.

[0019] The aforementioned distribution network dispatching method and device, by acquiring the predicted energy power and load power and combining them with the uncertainty interval characterizing the normal fluctuation range, enable the exploratory dispatching actions output by the strategy network to optimize the normal operating costs of resources, thereby maintaining economic efficiency under normal scenarios. At the same time, the strategy network synchronously outputs a risk tolerance threshold and determines the comprehensive violation degree based on node voltage and line transmission power after the dispatching action is executed. By comparing the comprehensive violation degree with the risk tolerance threshold, the risk of exceeding limits under extreme scenarios is transformed into an update of the strategy network. During the update process, the strategy network actively reduces the risk of exceeding limits, thereby possessing proactive risk avoidance capabilities under extreme disturbances, taking into account both economic efficiency during normal operation and physical safety under extreme disturbances. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is an application environment diagram of the power distribution network dispatching method in one embodiment;

[0022] Figure 2 This is a flowchart illustrating a power distribution network dispatching method in one embodiment;

[0023] Figure 3 This is a schematic diagram of the architecture of the power distribution network scheduling method in one embodiment;

[0024] Figure 4 This is a flowchart illustrating the steps for determining the uncertainty interval in one embodiment;

[0025] Figure 5 This is a diagram showing the topology and flexibility resource allocation of a distribution network system in one embodiment;

[0026] Figure 6 This is a schematic diagram of the power allocation results of each resource during a typical scheduling period in one embodiment.

[0027] Figure 7 This is a timeline diagram illustrating the balance between flexibility requirements and supply in one embodiment;

[0028] Figure 8 This is a structural block diagram of a power distribution network dispatching device in one embodiment;

[0029] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0031] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0032] The power distribution network dispatching method provided in this application can be applied to, for example, Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on the cloud or other network servers. Server 104 obtains the predicted energy power and load power of the distribution network at the current moment; determines the uncertainty interval based on the predicted energy power and load power; inputs the predicted energy power, load power, and uncertainty interval into a trained policy network to obtain the exploratory scheduling action and risk tolerance threshold output by the policy network; determines the node voltage and line transmission power of the distribution network based on the exploratory scheduling action, and determines the comprehensive violation degree based on the node voltage and line transmission power; updates the policy network based on the comprehensive violation degree and risk tolerance threshold to obtain the updated policy network; re-inputs the predicted energy power, load power, and uncertainty interval into the updated policy network, and schedules the resources of the distribution network based on the deterministic scheduling action output by the updated policy network, thereby realizing intelligent and adaptive scheduling of the distribution network. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, and projection equipment. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0033] In one exemplary embodiment, such as Figure 2 As shown, a distribution network dispatching method is provided, which can be applied to... Figure 1 Taking the server in the example, the explanation includes the following steps 201 to 206. Wherein:

[0034] Step 201: Obtain the predicted energy power and predicted load power of the distribution network at the current moment.

[0035] Energy forecast power refers to the server's prediction of the output power of renewable energy power generation equipment connected to the distribution network during future dispatch periods. Renewable energy power generation equipment mainly includes wind turbines and photovoltaic arrays; correspondingly, energy forecast power includes wind power forecast power and photovoltaic power forecast power. Energy forecast power reflects the amount of electricity that renewable energy units may generate under specific meteorological conditions and time scales. Load forecast power refers to the server's prediction of the total electricity consumption of all power users within the distribution network coverage area during future dispatch periods, reflecting the magnitude of electricity demand on the demand side.

[0036] The server acquires the predicted power output and load forecast data of new energy power generation equipment (including wind turbines and photovoltaic arrays) connected to the distribution network for future scheduling periods. In one embodiment, the server utilizes a Long Short-Term Memory (LSTM) network model to perform time-series predictions based on historical continuous time-series samples of wind and solar power output, historical continuous time-series samples of load, and meteorological variable data (including wind speed, vertical wind shear index, wind direction difference, temperature, humidity, etc.) to generate the predicted energy power and predicted load power for the current scheduling time. Specifically, the server forms a sliding window according to a preset scheduling step size, inputs historical continuous time-series samples of wind and solar power output, historical continuous time-series samples of load, and meteorological variable data as window samples into the LSM network, and outputs the predicted power value for each future period after passing through a shared fully connected layer, thereby obtaining the predicted energy power and predicted load power of the distribution network at the current time.

[0037] In another embodiment, the server can also directly receive pre-processed energy forecast power and load forecast power data from external weather forecasting and load forecasting systems. After acquiring the pre-processed energy forecast power and load forecast power data, the server cleans and normalizes the data to eliminate the impact of dimensional differences on the subsequent training process.

[0038] Step 202: Determine the uncertainty interval based on the predicted energy power and predicted load power. The uncertainty interval characterizes the normal fluctuation range of energy and load in the distribution network.

[0039] "Power output" refers to the actual electrical power generated by power generation equipment, and is an indicator of power generation capacity. Specifically, in this embodiment, renewable energy output refers to the active power injected into the distribution network by wind turbine generators and photovoltaic power generation arrays. The magnitude of renewable energy output determines the system's renewable energy power supply level. Corresponding to output is predicted power. Ideally, predicted power should be as close as possible to actual output. However, due to the randomness of meteorological conditions and the inherent errors of prediction models, a difference between the two is inevitable. The difference between output and predicted power is the prediction error. Specifically, when the predicted value is higher than the actual output, the prediction error is positive, meaning that the actual renewable energy power obtained by the system is lower than expected; when the predicted value is lower than the actual output, the prediction error is negative, meaning that the actual renewable energy power is higher than expected.

[0040] As defined above, the existence of forecast errors means there will always be a certain deviation between the predicted power output and the actual output. Simultaneously, a similar deviation exists between the predicted load power and the actual load demand on the load side, primarily due to the randomness and uncertainty of user electricity consumption behavior. Under normal weather conditions and regular load variations, this deviation is not entirely random, but rather exhibits a relatively stable distribution characteristic in terms of statistical regularity. That is, most deviations fall within a predictable numerical range, which is the normal fluctuation range. In other words, the normal fluctuation range is a quantitative description, in a statistical sense, of the random fluctuations in wind and solar power output around the predicted values ​​and the load forecast deviations.

[0041] In one embodiment, the server can construct an uncertainty interval to uniformly quantify the aforementioned conventional fluctuation range. The uncertainty interval serves to integrate the prediction deviations and fluctuation ranges of the renewable energy and load sides, forming a numerical range that comprehensively reflects the overall uncertainty of the system, providing a basis for the subsequent decision-making of the strategy network regarding system uncertainty. This uncertainty interval is presented in the form of a flexibility demand interval. Flexibility demand refers to the adjustment capacity the system needs from flexible resources such as energy storage devices and transferable loads to balance the random fluctuations and prediction deviations of renewable energy output and load. Flexibility demand is further divided into upward flexibility demand and downward flexibility demand. Upward flexibility demand refers to the upward power adjustment capability provided by flexible resources (including energy storage discharge, transferable load reduction, etc.) when the actual renewable energy output is less than the predicted output, or the actual load demand is greater than the predicted demand. Downward flexibility demand refers to the downward power adjustment capability provided by flexible resources (including energy storage charging, transferable load increase, etc.) when the actual renewable energy output is greater than the predicted output, or the actual load demand is less than the predicted demand. Upward flexibility requirements and downward flexibility requirements correspond to the system's adjustment needs in two unbalanced directions: "supply shortage" and "supply exceeding demand," respectively.

[0042] First, the server obtains the predicted energy power and corresponding actual output power of the distribution network at multiple points in time over a historical period. Based on the difference between the predicted energy power and the actual output power at each point, it calculates energy prediction error samples for multiple points, thereby obtaining the statistical distribution of the energy prediction error. Then, based on the statistical distribution of each energy prediction error sample, the server determines the energy prediction error range within a preset confidence level.

[0043] Then, the server obtains the predicted load power and the corresponding actual load power at multiple times within the historical period, calculates the load prediction error sample, and then determines the load fluctuation range based on the statistical distribution of the error sample at a preset confidence level.

[0044] Finally, the server combines the energy forecasting error range and the load fluctuation range to determine the overall uncertainty range of the system. Specifically, the server calculates the upward and downward adjustment demands of the system under the worst-case scenario based on the energy forecasting error range and the load fluctuation range, respectively, and then obtains the upper and lower bounds of the upward and downward flexibility demand ranges, which are used together as the final expression of the uncertainty range.

[0045] Step 203: Input the predicted energy power, predicted load power, and uncertainty interval into the trained policy network to obtain the exploratory scheduling action and risk tolerance threshold output by the policy network; the risk tolerance threshold represents the tolerance boundary of the comprehensive violation degree.

[0046] The policy network, in deep reinforcement learning, is a policy function approximation network used to directly map the action an agent should take based on the current environmental state. In this embodiment, the policy network adopts the Actor network structure from the proximal policy optimization algorithm. The Actor network structure uses a deep neural network as a function approximator, with the current environmental state as input and the probability distribution or deterministic action value of the action space as output. The input data of the policy network includes predicted energy power, predicted load power, and uncertainty intervals. It may also include the state of charge of the energy storage system and the cumulative offset of transferable loads.

[0047] The risk tolerance threshold is used to characterize the tolerance boundary of the overall violation degree, which is a quantitative indicator obtained by comprehensively evaluating the degree of voltage exceedance at distribution network nodes and the degree of power exceedance in line transmission. Numerically, the risk tolerance threshold is equivalent to the risk value quantile of the overall system violation degree at a given confidence level, representing the upper limit of physical violations that the agent can accept from a risk management perspective under the current operating conditions.

[0048] It should be noted that, within the framework of deep reinforcement learning, an intelligent agent refers to an autonomous decision-making entity capable of perceiving the state of the environment, making decisions, and interacting with the environment. In this embodiment, the intelligent agent is abstracted as the control unit of the distribution network dispatch decision system, used to issue dispatch instructions to various flexible resources (including energy storage systems, transferable loads, etc.) within the distribution network. The intelligent agent consists of a policy network and a value network. The policy network is the decision-making module of the intelligent agent, taking the environmental state (including predicted power, predicted load power, uncertainty interval, energy storage state of charge, etc.) as input and outputting a probability distribution or deterministic action in the action space. The value network is the evaluation module of the intelligent agent, responsible for assessing the quality of the current state or action, and used to assist in the training and updating of the policy network. The two work together. At each dispatch time, the intelligent agent generates dispatch actions based on the current state through the policy network and issues them to the distribution network environment. After the environment executes the action, it returns a reward signal and the state at the next time step. The intelligent agent uses the reward signal to simultaneously update the parameters of the value network and the policy network, so that the actions output by the policy network are continuously optimized in the sense of long-term cumulative reward. In other words, the policy network is the specific mathematical implementation of the agent's decision-making function, while the agent is a complete decision-making entity that includes the policy network, value network, and training and update mechanism. Inputting data into the policy network and obtaining output is essentially the process by which the agent makes decisions through its policy network module; and updating the policy network, as discussed later, is essentially the process by which the agent optimizes the parameters of its core decision-making module using reward signals from environmental feedback.

[0049] To enable the agent to simultaneously pursue economic efficiency and possess the ability to perceive and avoid physical security risks, this embodiment extends the action space of the standard Markov decision process, constructing an augmented action space. The augmented action space comprises two subspaces: a physical action subspace, where actions correspond to scheduling commands that can be directly issued to physical devices in the distribution network; and a virtual action subspace, where actions do not correspond to any physical device but rather to risk tolerance thresholds. The policy network makes decisions within this augmented action space, simultaneously outputting physical scheduling commands and risk tolerance thresholds.

[0050] The server inputs an input vector, consisting of energy forecast power, load forecast power, uncertainty interval, and other necessary state variables, into the policy network. The policy network then maps the input vector into an output value within the augmented action space, which is the decision result of the policy network in the current state.

[0051] It should be noted that during the training phase, the policy network outputs the probability distribution parameters of the action space. The agent randomly samples from this to obtain exploratory scheduling action samples (i.e., exploratory scheduling actions). The risk tolerance threshold is synchronously sampled and output by the policy network. Its value reflects the agent's risk preference in the current operating state. When the risk tolerance threshold is high, the agent is relatively lenient towards physical violations, resulting in smaller penalties in the short term but potentially hiding higher tail risks. When the risk tolerance threshold is low, the agent is conservative, improving safety but potentially increasing operating costs. Guided by reward signals, the policy network autonomously learns the appropriate risk tolerance threshold for each operating state, achieving dynamic adaptive adjustment of risk preference.

[0052] Step 204: Determine the node voltage and line transmission power of the distribution network based on the exploration and scheduling actions, and determine the comprehensive violation degree based on the node voltage and line transmission power; the comprehensive violation degree characterizes the degree of exceeding the limit of the node voltage and / or line transmission power of the distribution network.

[0053] Among them, the comprehensive violation degree is a continuous index that maps multi-dimensional physical security constraints into a single scalar value. The larger the value, the closer the physical operating state of the system is to or beyond the security boundary, and the higher the security risk.

[0054] First, the server calculates the injected power of each node in the distribution network based on the energy storage charging and discharging power instructions and the transferable load dispatching quantity instructions contained in the exploration scheduling action, combined with the energy forecast power and the load forecast power.

[0055] Then, the server performs AC power flow calculations based on the injected power of each node to determine the voltage of each node and the transmission power of each line. If the power flow calculation fails to converge, it indicates that the scheduling action has caused the system to enter an infeasible operating region. At this point, the safety circuit breaker mechanism is triggered, terminating the current scheduling cycle and returning a maximum negative reward.

[0056] When the power flow calculation converges, the server determines the voltage violation value based on the deviation of each node's voltage from a preset voltage threshold, and the power violation value based on the deviation of each line's transmitted power from a preset power threshold. Finally, the server takes the larger of the voltage and power violation values ​​as the overall violation degree to reflect the most severe physical violation risk faced by the system.

[0057] Step 205: Update the policy network based on the overall violation rate and risk tolerance threshold to obtain the updated policy network.

[0058] The server constructs a physical security penalty based on conditional risk value by utilizing the comprehensive violation degree and the risk tolerance threshold output by the policy network, and generates the final reward signal by combining it with the economic reward component. After the reward signal is fed back to the agent, the agent uses a proximal policy optimization algorithm to update the parameters of both the policy network and the value network simultaneously. Specifically, the goal of updating the policy network is to adjust the network parameters to increase the probability of taking actions that yield higher rewards in a given state; the goal of updating the value network is to adjust the network parameters to make its estimation of state value more accurate. Through multiple iterative updates, the parameters of the policy network are gradually optimized, and its output scheduling actions continuously approach the optimal policy while balancing economy and security.

[0059] Step 206: Re-input the predicted energy power, predicted load power, and uncertainty interval into the updated strategy network, and schedule the resources of the distribution network based on the deterministic scheduling actions output by the updated strategy network.

[0060] During the training phase, the policy network outputs noisy exploration scheduling actions under the agent's exploration mechanism to fully explore the state-action space.

[0061] During the deployment phase, the agent disables the exploration mechanism, and its policy network outputs deterministic scheduling actions based on the current state. These deterministic scheduling actions are the final dispatch instructions. The server converts the deterministic scheduling actions into specific control signals for each flexible resource and sends them to field devices via the communication network, thus completing the actual dispatch control of the distribution network resources.

[0062] This embodiment can be run in, for example... Figure 3 Above the illustrated architecture, this architecture consists of three layers: a data prediction layer, a risk perception and decision-making layer, and a physical safety verification layer. The data prediction layer, based on multi-source time-series data (wind and solar power output, load, meteorological data, etc.), outputs wind / solar power predictions, along with their 95% confidence level prediction error range and load fluctuation range, using a long short-term memory network interval prediction model after preprocessing. The risk perception and decision-making layer combines the above prediction information with state variables such as energy storage and flexible loads to construct the state space of an augmented Markov decision process. An augmented proximal strategy is used to optimize the agent during training, enabling simultaneous decision-making in both the physical and virtual action spaces. Physical violation risks are embedded into reward signals through conditional risk value penalty feedback. The physical safety verification layer performs AC power flow calculations on scheduling actions, calculates the overall physical violation degree of the system, and feeds back penalty signals to update the agent.

[0063] Figure 3 For a schematic diagram of the resource power allocation results during a typical scheduling period in one embodiment, please refer to [link / reference]. Figure 3The diagram shows the resource power allocation corresponding to the deterministic scheduling actions output by the strategy network trained in this embodiment. It includes time-series curves for baseline load, grid power purchase, load reduction / increase, photovoltaic output, energy storage charging / discharging, grid backfeeding, and wind power output. It can be seen that during peak photovoltaic and wind power output periods, the energy storage system charges to absorb excess renewable energy, while simultaneously increasing the load transfer. During periods of insufficient renewable energy output, the energy storage system discharges and reduces some load, effectively smoothing net load fluctuations, and the overall grid power purchase remains at a low level.

[0064] In this embodiment, by acquiring the predicted energy power and load power and combining them with the uncertainty interval characterizing the normal fluctuation range, the exploratory scheduling actions output by the strategy network can optimize the normal operating costs of resources, thereby maintaining economic efficiency under normal scenarios. At the same time, the strategy network synchronously outputs the risk tolerance threshold and determines the comprehensive violation degree based on the node voltage and line transmission power after the scheduling action is executed. By comparing the comprehensive violation degree with the risk tolerance threshold, the risk of exceeding the limit under extreme scenarios is transformed into an update of the strategy network. During the update process, the strategy network actively reduces the risk of exceeding the limit, thereby having the ability to actively avoid risks under extreme disturbances, taking into account both the economy under normal operation and the physical safety under extreme disturbances.

[0065] Figure 4 For a flowchart illustrating the steps involved in determining the uncertainty interval in one embodiment, please refer to [link / reference]. Figure 4 In one exemplary implementation, determining the uncertainty interval based on the predicted energy power and the predicted load power includes:

[0066] Step 401: Obtain the predicted energy power and corresponding actual output power of the distribution network at multiple times within a historical period.

[0067] Among them, the predicted power output refers to the estimated output value of new energy power generation equipment at a specific moment obtained through the prediction model, while the actual output power refers to the power value actually injected into the grid by the new energy power generation equipment at the same moment, which is collected by the measuring device deployed at the grid connection point.

[0068] First, the server retrieves recorded data from multiple moments within a historical period from the database or the data acquisition and monitoring control system.

[0069] Then, the server extracts the predicted renewable energy output data and the corresponding actual output data for each moment from these records to obtain the predicted energy power and the corresponding actual output power for each moment. The server organizes the historical data at a preset sampling interval to form pairs of time series samples, each sample containing the predicted energy power and the actual output power for the same moment.

[0070] Step 402: Determine energy prediction error samples for multiple time periods based on the predicted energy power and the actual output power at each time point.

[0071] For each pair of time-series samples, the server calculates the energy prediction error. For each time point, the server subtracts the actual output power from the predicted energy power at that time to calculate the energy prediction error sample for that time. The server performs the above calculation for each time point within the historical period, obtaining a set of energy prediction error samples containing multiple prediction error values.

[0072] It should be noted that the energy prediction error sample may be positive or negative. When the predicted energy power is greater than the actual value, the energy prediction error sample is positive, indicating that the prediction result is optimistic and that the actual renewable energy power obtained by the system is lower than expected; when the predicted energy power is less than the actual value, the energy prediction error is negative, indicating that the prediction result is conservative and that the actual renewable energy power is higher than expected.

[0073] Step 403: Determine the energy prediction error range based on the statistical distribution of each energy prediction error sample.

[0074] Among them, the energy forecast error range reflects the degree of uncertainty in the new energy output forecasting process itself.

[0075] First, the server calculates the statistical characteristics of the energy forecast error samples, including the mean, standard deviation, and quantile distribution of the error samples.

[0076] Then, the server determines the upper and lower boundaries of the energy prediction error interval based on a preset confidence level (e.g., 95%). For example, the server can use the quantile method to determine the energy prediction error interval, sorting the energy prediction error samples in ascending order of value, taking the value at the 2.5th quantile as the lower limit of the energy prediction error interval, and taking the value at the 97.5th quantile as the upper limit. Its advantage lies in that it does not require assuming that the error samples follow a specific probability distribution, possessing non-parametric characteristics, and can adapt to the diverse prediction error distributions of different renewable energy power plants.

[0077] In another embodiment, if the energy prediction error samples approximately follow a normal distribution, the server can also use a parameterization method based on the mean and standard deviation to determine the energy prediction error interval. For example, the mean of the error samples is used as the center point, and 1.96 times the standard deviation (corresponding to a 95% confidence level) is used as the half-interval width to obtain the energy prediction error interval.

[0078] Step 404: Determine the load fluctuation range based on the fluctuation of the predicted load power.

[0079] The fluctuation of the predicted load power refers to the change in the actual load power relative to the predicted load power. In actual operation, load demand is affected by various factors such as user electricity consumption behavior, temperature changes, and differences between weekdays and holidays, resulting in a deviation between the actual and predicted values.

[0080] First, the server obtains the predicted load power and the corresponding actual load value of the distribution network at multiple times within a historical period.

[0081] Then, the server calculates the load forecast error at each time point, which is the predicted load power minus the actual load value.

[0082] Next, the server determines the load fluctuation range based on the statistical distribution of the load forecasting error. The method for determining this range is similar to that for determining the energy forecasting error range. Specifically, the server determines the upper and lower boundaries of the load fluctuation range based on the quantiles of the load forecasting error sample at a preset confidence level.

[0083] In another embodiment, if the load forecast accuracy is high, the server can also directly use the historical fluctuation of the load forecast power as the load fluctuation range, that is, determine the fluctuation range based on the statistical distribution of load changes at adjacent times in the historical load data.

[0084] Step 405: Determine the uncertainty range based on the energy forecast error range and the load fluctuation range.

[0085] First, the server integrates the prediction error range on the renewable energy side and the fluctuation range on the load side, ensuring that the uncertainty range simultaneously reflects the combined impact of renewable energy output uncertainty and load demand uncertainty on system operation. Specifically, the server first determines the possible fluctuation range of renewable energy output based on the energy prediction error range, then determines the possible fluctuation range of load demand based on the load fluctuation range. Finally, based on the power balance relationship between renewable energy output and load demand, the possible fluctuation ranges of renewable energy output and load demand are superimposed to obtain the comprehensive fluctuation range of the system's net load (i.e., load demand minus renewable energy output).

[0086] The server then calculates the upward and downward uncertainty boundaries. The upward uncertainty boundary corresponds to situations where the system needs to adjust upwards to cope with insufficient renewable energy output or unexpected load. It is determined by the lower limit of the energy forecast error range (indicating that renewable energy output may be too low) and the upper limit of the load fluctuation range (indicating that load demand may be too high). The downward uncertainty boundary corresponds to situations where the system needs to adjust downwards to cope with situations where renewable energy output exceeds expectations or load is below expectations. It is also determined by the upper limit of the energy forecast error range (indicating that renewable energy output may be too high) and the lower limit of the load fluctuation range (indicating that load demand may be too low).

[0087] Finally, the server determines the flexibility requirement range based on the upward and downward uncertainty boundaries, and outputs the uncertainty range in the form of a downward flexibility requirement range.

[0088] In this embodiment, the estimation results of the uncertainty interval do not depend on prior assumptions about future extreme scenarios. This provides reinforcement learning agents with a quantitative basis for perceiving normal fluctuations that matches the prediction information. Thus, while maintaining economical scheduling under normal operating conditions, it reserves the necessary flexibility to adjust capacity for risk avoidance in extreme scenarios.

[0089] In one exemplary implementation, the method further includes: acquiring energy time-series data and meteorological time-series data of the distribution network within a historical period; inputting the energy time-series data and meteorological time-series data within the historical period into a trained long short-term memory network to obtain the energy prediction error range output by the long short-term memory network.

[0090] Long Short-Term Memory (LSTM) networks are recurrent neural network structures suitable for time-series data modeling. By introducing three gating mechanisms—input gate, forget gate, and output gate—they can capture long-term dependencies in time-series data. In this embodiment, LSM networks are used to establish a mapping relationship between historical energy time-series data, meteorological time-series data, and future energy prediction errors, thereby estimating the range of energy prediction errors for future periods. Energy time-series data refers to the output power records of new energy power generation equipment (including wind turbines and photovoltaic arrays) connected to the distribution network at continuous historical moments, including actual output power and predicted output power, reflecting the temporal variation pattern of new energy output. Meteorological time-series data refers to the records of meteorological variables collected at multiple consecutive moments within a historical period. Meteorological variables include wind speed, vertical wind shear index, wind direction difference, temperature, humidity, etc. These meteorological variables are the main external factors affecting the fluctuation of new energy output, collectively constituting the external causal characteristics of prediction errors.

[0091] First, the server acquires energy time-series data and meteorological time-series data of the distribution network within historical time periods. The server then organizes and aligns the acquired energy time-series data and meteorological time-series data according to a preset time step to ensure that each data sequence corresponds in the time dimension, resulting in organized energy time-series data and meteorological time-series data.

[0092] Then, the server combines the processed energy time-series data and meteorological time-series data into an input feature matrix. Each row of the feature matrix corresponds to a historical time point, and each column corresponds to a feature variable (including energy output power, predicted power, and various meteorological variables). The server inputs the feature matrix into a trained Long Short-Term Memory (LSTM) network. The LSM network extracts temporal features from the input sequence through its internal gating mechanism and memory units, capturing the dynamic correlation between meteorological conditions and new energy prediction errors. The forward propagation process of the LSM network is as follows: at each time step, the network receives the input feature vector at the current time point, as well as the hidden state and cell state passed from the previous time point. The input gate controls how much of the current input information is written into the cell state, the forget gate controls how much of the cell state from the previous time point is retained in the current time point, and the output gate controls how much of the cell state from the current time point is output as a hidden state. Through the above gating mechanism, the network can selectively remember and forget historical information, thereby achieving effective modeling of long sequence dependencies. After iterative calculations at multiple time steps, the output layer of the LSM network produces the prediction result for the current time point. After the output layer undergoes a fully connected transformation, it simultaneously outputs the lower and upper limits of the energy prediction error range, where the lower and upper limits correspond to the lower and upper quantiles of the energy prediction error under a preset confidence level, respectively.

[0093] It should be noted that the Long Short-Term Memory (LSTM) network outputs an interval estimate of the energy prediction error. Its lower and upper bounds represent the lowest and highest possible values ​​that the predicted error of new energy output might fall into in the future, under a preset confidence level. In other words, the LTM network directly outputs the energy prediction error interval; the lower bound is the lower limit of the prediction error, and the upper bound is the upper limit. During the training phase, the LTM network uses historical energy time-series data and meteorological time-series data as input, and historical prediction error values ​​at the same time as the supervision label. A quantile loss function is used for parameter optimization, enabling the trained network to directly output an interval estimate of the prediction error. The server combines the lower and upper bounds output by the LTM network to form the energy prediction error interval, reflecting the normal fluctuation range that the predicted value of new energy output might deviate from the actual value under given meteorological conditions and historical output trends.

[0094] In this embodiment, by using a Long Short-Term Memory (LSTM) network to model historical energy time-series data and meteorological time-series data, an interval estimate of energy prediction error is directly output, fully utilizing the temporal correlation between meteorological factors and new energy output prediction errors. The interval width can be dynamically adjusted according to the current meteorological conditions, allowing the energy prediction error interval to have different estimation results under different meteorological conditions, thus more accurately reflecting the non-stationary characteristics of prediction errors as external conditions change.

[0095] In one exemplary implementation, determining the node voltage and line transmission power of the distribution network based on exploratory scheduling actions, and determining the comprehensive violation degree based on the node voltage and line transmission power, includes: determining the injected power of each node in the distribution network based on the exploratory scheduling actions; determining the voltage of each node and the power of each line based on the injected power of each node; determining the voltage limit violation value based on the deviation of the voltage of each node from a preset voltage threshold; determining the power limit violation value based on the deviation of the power of each line from a preset power threshold; and determining the comprehensive violation degree based on the maximum value between the voltage limit violation value and the power limit violation value.

[0096] The exploration of dispatch actions includes charging and discharging power commands for each node's energy storage system and dispatch quantity commands for each node's transferable load. The energy storage charging and discharging power command can be positive, negative, or zero. A positive value indicates that the energy storage system releases energy to the grid (discharging), a negative value indicates that the energy storage system absorbs energy from the grid (charging), and a zero value indicates that the energy storage system does not operate. The transferable load dispatch quantity command indicates the power value transferred upwards or downwards relative to the baseline electricity consumption plan. A positive value indicates load reduction (reduced electricity consumption), and a negative value indicates load increase (increased electricity consumption).

[0097] First, the server parses the exploration scheduling actions. These actions are multi-dimensional numerical instructions output by the policy network, including charging / discharging power instructions for each node's energy storage system and scheduling instructions for each node's transferable load. The energy storage charging / discharging power instructions can be positive, negative, or zero. A positive value indicates the energy storage system is releasing energy to the grid (discharging), a negative value indicates the energy storage system is absorbing energy from the grid (charging), and a zero value indicates the energy storage system is inactive. The transferable load scheduling instructions represent the power value transferred upwards or downwards relative to the baseline electricity consumption plan. A positive value indicates load reduction (reduced electricity consumption), and a negative value indicates load increase (increased electricity consumption). The server assigns the parsed energy storage charging / discharging power instructions to the corresponding node's energy storage system and the transferable load scheduling instructions to the corresponding node's transferable load.

[0098] Then, based on the calculation method of the injected power for each node, the server aggregates the renewable energy generation power, energy storage charging and discharging power, transferable load power, and fixed load power of each node according to their node affiliation, and calculates the injected power for each node. Specifically, renewable energy generation power comes from energy forecast power, energy storage charging and discharging power comes from energy storage charging and discharging power commands in the exploration dispatch action, transferable load power comes from transferable load dispatch quantity commands in the exploration dispatch action, and fixed load power comes from the non-adjustable portion of the load forecast power. The specific calculation method for the injected power of a node is: the renewable energy generation power connected to the node minus the load power connected to the node (including fixed load and transferable load), plus the power injected into the grid by the energy storage system, plus the power injected into the grid from the upstream grid through the node's interconnection line. In the calculation, energy storage discharging power is taken as positive values, charging power as negative values, and transferable load reduction power as positive values ​​and increase power as negative values. For nodes without any connected equipment, their injected power is zero. Through the above calculation, the server assigns an injected power value to each node in the distribution network.

[0099] Subsequently, the server uses the injected power of each node as a known quantity to perform AC power flow calculations to determine the operating state of the distribution network. After obtaining the injected power of each node, the server uses the Newton-Raphson method for iterative solution. The basic principle of the Newton-Raphson method is to expand the nonlinear equations into Taylor series near the initial solution and retain the first-order terms, transforming the nonlinear problem into an iterative solution process for a linear problem. Specifically, the server first sets an initial estimate of the voltage of each node; in one implementation, the initial estimate is set to 1.0 per unit. Then, it calculates the imbalance of the injected power at each node, which is the difference between the given injected power and the injected power calculated under the current voltage estimate. Next, it corrects the voltage estimate by solving the Jacobian matrix, which is a matrix composed of the partial derivatives of the power balance equation with respect to the voltage of each node. The server repeats the above iterative process until the imbalance is less than a preset convergence tolerance. In one embodiment, the AC power flow calculation is performed by the Pandapower power flow calculation engine in the Python environment. In another embodiment, the server can also perform AC power flow calculations by calling the MATLAB / Matpower engine.

[0100] Furthermore, the server extracts the voltage amplitude of all nodes from the power flow calculation results and identifies the maximum and minimum voltage values. Preset voltage thresholds include an upper voltage limit and a lower voltage limit. The upper voltage limit refers to the highest allowable voltage value of a node, set to 1.05 per unit in this embodiment; the lower voltage limit refers to the lowest allowable voltage value of a node, set to 0.95 per unit in this embodiment. The server calculates the difference between the maximum voltage value and the upper voltage limit. If the difference is greater than zero, it indicates that a node voltage is higher than the upper limit, and this difference is the maximum voltage over-limit value. If the difference is less than or equal to zero, it indicates that all node voltages are within the upper limit, and the maximum voltage over-limit value is set to zero. Similarly, the server calculates the difference between the lower voltage limit and the minimum voltage value. If the difference is greater than zero, it indicates that a node voltage is lower than the lower limit, and this difference is the minimum voltage over-limit value. If the difference is less than or equal to zero, it indicates that all node voltages are within the lower limit, and the minimum voltage over-limit value is set to zero. It should be noted that both the maximum and minimum voltage over-limit values ​​are non-negative numbers. The server compares the maximum and minimum voltage over-limit values ​​and takes the larger value as the voltage over-limit value. This voltage over-limit value reflects the severity of over-limits in both over-voltage and under-voltage directions, ensuring that the most serious voltage safety issues are accurately detected.

[0101] Furthermore, the server extracts the active power values ​​transmitted by all lines from the power flow calculation results. For each line, the server obtains its rated transmission capacity, which refers to the maximum power value that the line can safely transmit over a long period. It then calculates the percentage of active power currently transmitted by the line relative to its rated transmission capacity; this percentage is the current load rate of the line. A preset power threshold is given in the form of the line's maximum permissible load rate. The maximum permissible load rate is a pre-set safe limit for the line's long-term operation, and its value is less than 100% to leave a certain safety margin. For example, the maximum permissible load rate is set to 80%, meaning that the line's transmitted power must not exceed 80% of its rated capacity. The server finds the maximum current load rate among all lines, i.e., the load rate of the most severely congested line in the entire network. Then, the server calculates the difference between the maximum load rate and the maximum permissible load rate. If the difference is greater than zero, it indicates that the load rate of some lines exceeds the preset maximum allowable load rate; this difference is the power over-limit value, also known as the power limit violation value. If the difference is less than or equal to zero, it indicates that the load rate of all lines has not exceeded the maximum allowable load rate; in this case, the power over-limit value is zero. The power over-limit value reflects the safety margin of power transmission in the distribution network. The larger the value, the closer the line is to or exceeds its safe transmission limit, and the higher the risk of line overload tripping in the system. For example, if the maximum line load rate of the entire network is 90% and the maximum allowable load rate is 80%, then the power over-limit value is 0.10 (i.e., 10%); if the maximum line load rate of the entire network is 75%, then the power over-limit value is zero. When the power over-limit value is positive, it indicates that there is line congestion in the distribution network, and dispatching measures need to be taken to reduce the transmission power of the relevant lines.

[0102] Finally, the server compares the voltage and power over-limit values, taking the larger of the two as the final comprehensive violation score. Specifically, if a node in the system experiences a severe voltage over-limit but all line load rates are low, the voltage over-limit value is a large positive number while the power over-limit value is zero. The comprehensive violation score is the voltage over-limit value, accurately reflecting the main safety risks faced by the system. Conversely, if a line is severely overloaded but all node voltages are within the normal range, the comprehensive violation score is the power over-limit value, also accurately reflecting the main safety risks faced by the system. If both over-limit situations exist simultaneously, the comprehensive violation score is the larger of the two, ensuring that the most serious safety risk is not underestimated.

[0103] Figure 5 For a topology and flexibility resource allocation diagram of a distribution network system in one embodiment, please refer to [link / reference]. Figure 5The baseline operating scenario for the distribution network is an IEEE 33-node system, with nodes numbered 1 to 33. Nodes 10, 20, and 30 are connected to energy storage systems (ESS1, ESS2, and ESS3), respectively; nodes 15, 25, and 32 are connected to transferable loads (AL1, AL2, and AL3); nodes 17 and 32 are connected to photovoltaic (PV1 and PV2) power generation; and node 24 is connected to wind power (WT). The upstream grid exchanges power with the distribution network through the root node (node ​​1). This distribution network system encompasses renewable energy generation (PV and wind power), energy storage regulation, and demand-side flexible resources, enabling comprehensive testing of the scheduling effectiveness of this embodiment under the coordinated use of multiple flexible resources.

[0104] In this embodiment, the comprehensive violation degree maps the multi-node voltage security constraints and multi-line transmission power security constraints of the distribution network into a continuous and differentiable scalar index. This provides a clear, continuous, and physically meaningful security feedback signal for the reinforcement learning agent, enabling the agent to perceive the gradual trend of the physical security boundary during the training process, thereby effectively learning a scheduling strategy that balances economy and security.

[0105] In an exemplary embodiment, determining the voltage over-limit value based on the deviation of the voltage of each node from a preset voltage threshold includes: obtaining the maximum voltage and minimum voltage among the voltages of each node; determining the maximum voltage over-limit value based on the positive deviation of the maximum voltage from a preset upper voltage limit; determining the minimum voltage over-limit value based on the positive deviation of the preset lower voltage limit from the minimum voltage; and determining the voltage over-limit value based on the maximum voltage over-limit value and the minimum voltage over-limit value.

[0106] The physical meaning of the maximum voltage over-limit value is the degree to which the most severely overvoltage node in the distribution network exceeds the safe operating limit. The larger the value, the more severe the overvoltage problem. For example, if the voltage of a photovoltaic access node in the distribution network reaches 1.08 per unit at a certain dispatching time, while the voltages of other nodes are all below 1.05 per unit, then the maximum voltage is 1.08, and the maximum voltage over-limit value is 1.08 minus 1.05, which equals 0.03, indicating that there is an overvoltage over-limit of 0.03 per unit in the system. If the voltages of all nodes are below 1.05 per unit, then the maximum voltage over-limit value is zero.

[0107] The physical meaning of the minimum voltage limit is the degree to which the most severely undervoltage node in the distribution network falls below the safe operating lower limit. The larger the value, the more severe the undervoltage problem. For example, if at a certain dispatching time, the voltage of a node at the end of a line in the distribution network is only 0.92 per unit, while the voltages of other nodes are all higher than 0.95 per unit, then the minimum voltage is 0.92, and the minimum voltage limit is 0.95 minus 0.92, which equals 0.03, indicating that the system has an undervoltage limit of 0.03 per unit. If the voltages of all nodes are higher than 0.95 per unit, then the minimum voltage limit is zero.

[0108] First, the server extracts the voltage amplitude of each node from the AC power flow calculation results, forming a set containing the voltage amplitudes of all nodes. Then, the server iterates through this set, comparing the values ​​to identify the highest voltage as the maximum voltage and the lowest voltage as the minimum voltage. The maximum voltage reflects the node with the highest voltage level in the distribution network, typically occurring near nodes where distributed generation (especially photovoltaic power) is concentrated. When renewable energy generation is high and the load is low, the voltage at this node may rise significantly. The minimum voltage reflects the node with the lowest voltage level in the distribution network, typically occurring at the end of a line or near heavily loaded nodes. When the load is heavy and the output of distributed generation is insufficient, the voltage at this node may drop significantly.

[0109] Next, the server calculates the voltage exceedance level from both overvoltage and undervoltage perspectives. Regarding overvoltage, the server determines the maximum voltage exceedance value based on the positive deviation between the maximum voltage and the preset upper voltage limit. The server compares the obtained maximum voltage with the preset upper voltage limit, which is set to 1.05 per unit in this embodiment. The server calculates the difference between the maximum voltage and the upper voltage limit and determines whether the difference is greater than zero. If the difference is greater than zero, it indicates that at least one node in the distribution network has a voltage exceeding the preset upper voltage limit, resulting in an overvoltage exceedance. In this case, the difference is used as the maximum voltage exceedance value, i.e., the maximum voltage exceedance value equals the maximum voltage minus the upper voltage limit. If the difference is less than or equal to zero, it indicates that the voltage of all nodes in the distribution network has not exceeded the upper voltage limit, and there is no overvoltage exceedance. The maximum voltage exceedance value is set to zero. The above process can be expressed by the formula: Maximum voltage exceedance value = max(Maximum voltage - Upper voltage limit, 0), thus ensuring that the maximum voltage exceedance value is a non-negative number.

[0110] Regarding undervoltage, the server determines the minimum voltage exceedance value based on the positive deviation between the preset lower voltage limit and the minimum voltage. The server compares the acquired minimum voltage with the preset lower voltage limit, which in this embodiment is set to 0.95 per unit. The server calculates the difference between the lower voltage limit and the minimum voltage and determines if the difference is greater than zero. If the difference is greater than zero, it indicates that at least one node in the distribution network has a voltage lower than the preset lower voltage limit, resulting in an undervoltage exceedance. In this case, the difference is used as the minimum voltage exceedance value, i.e., the minimum voltage exceedance value equals the lower voltage limit minus the minimum voltage. If the difference is less than or equal to zero, it indicates that the voltage of all nodes in the distribution network is not lower than the lower voltage limit, and there is no undervoltage exceedance. The minimum voltage exceedance value is set to zero. The above process can be expressed by the formula: Minimum voltage exceedance value = max(lower voltage limit - minimum voltage, 0), thus ensuring that the minimum voltage exceedance value is a non-negative number.

[0111] Finally, the server compares the maximum and minimum voltage over-limit values, taking the larger value as the final voltage over-limit value. The formula for calculating the voltage over-limit value is: Voltage Over-limit Value = max(Maximum Voltage Over-limit Value, Minimum Voltage Over-limit Value). By using the maximum value, the voltage over-limit value can simultaneously reflect the severity of over-limits in both overvoltage and undervoltage directions. When both overvoltage and undervoltage nodes exist in the distribution network, the voltage over-limit value is taken as the more severe one, ensuring that the most serious voltage safety issue is accurately detected. For example, if at a certain dispatch time the maximum voltage in the distribution network is 1.06 per unit (at which point the maximum voltage over-limit value is 0.01) and the minimum voltage is 0.94 per unit (at which point the minimum voltage over-limit value is 0.01), then the voltage over-limit value is 0.01; if the maximum voltage is 1.08 per unit (at which point the maximum voltage over-limit value is 0.03) and the minimum voltage is 0.97 per unit (at which point the minimum voltage over-limit value is 0), then the voltage over-limit value is 0.03, which reflects the severity of overvoltage; if the maximum voltage is 1.02 per unit (at which point the maximum voltage over-limit value is 0) and the minimum voltage is 0.91 per unit (at which point the minimum voltage over-limit value is 0.04), then the voltage over-limit value is 0.04, which reflects the severity of undervoltage.

[0112] In another embodiment, the server may also add the maximum and minimum voltage over-limit values ​​to obtain the voltage over-limit value, thus comprehensively reflecting the total over-limit amount from both overvoltage and undervoltage perspectives. However, compared to the summation method, taking the maximum value better highlights the most severe single over-limit risk, preventing minor over-limits in one direction from being masked by adding minor over-limits in another direction. Therefore, this embodiment preferably uses the maximum value method to determine the voltage over-limit value.

[0113] In this embodiment, the voltage over-limit value maps the voltage safety constraints of all nodes in the distribution network to a single continuous value. This value reflects both overvoltage and undervoltage risks and captures the most severe voltage over-limit events, providing accurate and physically meaningful input for subsequent calculations of the overall violation rate. By compressing the voltage states of multiple nodes into a single value, the dimensionality of safety feedback signals during subsequent reinforcement learning training is effectively reduced, enabling the agent to perceive the system's voltage safety status more efficiently.

[0114] In an exemplary embodiment, updating the policy network based on the overall violation rate and a risk tolerance threshold includes: obtaining the cost corresponding to the exploration scheduling action output by the policy network; determining a risk penalty based on the excess portion of the overall violation rate relative to the risk tolerance threshold; generating a reward signal based on the risk penalty and cost; and updating the parameters of the policy network based on the reward signal.

[0115] The policy network is essentially a parameterized deep neural network. Its input is the environment state, and its output is a probability distribution or deterministic action value in the action space. The goal of updating the policy network's parameters is to optimize the network parameters so that the actions output by the policy network continuously approach the optimal policy in terms of long-term cumulative reward.

[0116] Cost refers to the total operating expenses incurred by the distribution network after performing exploratory dispatching actions. In this embodiment, cost includes multiple items such as electricity purchase cost, energy storage charging and discharging loss cost, transferable load response compensation cost, wind and solar curtailment penalty cost, and flexibility deficit penalty cost.

[0117] First, based on the specific content of the exploratory dispatch action and the real-time operating data of the distribution network, various costs are calculated item by item. Specifically, the server obtains the real-time electricity price for the current dispatch period and calculates the product of the active power purchased by the distribution network from the upper-level grid and the real-time electricity price as the electricity purchase cost. The server calculates the energy loss during the charging and discharging process of the energy storage system based on its charging and discharging power and efficiency, and calculates the energy loss cost by multiplying the lost power by the electricity price. The server calculates the transferable load response compensation cost based on the actual dispatch volume of transferable load and the pre-agreed compensation unit price. The server calculates the curtailment penalty cost based on the reduced renewable energy generation power and the penalty unit price for wind and solar curtailment. The server multiplies the gap between flexibility demand and flexibility supply by the gap penalty unit price to obtain the flexibility gap penalty cost. The server sums up all the above costs to obtain the cost corresponding to the exploratory dispatch action. The larger this cost, the less economically viable the dispatch action is.

[0118] Then, the server determines the risk penalty based on the excess portion of the overall violation rate relative to the risk tolerance threshold. The server calculates the difference between the overall violation rate and the risk tolerance threshold; this difference represents the excess portion. If the difference is greater than zero, it indicates that the overall violation rate exceeds the risk tolerance threshold set by the agent, and the system has entered a tail-risk region, requiring amplification of the penalty for the excess portion to obtain the desired risk penalty. If the difference is less than or equal to zero, it indicates that the overall violation rate is within the acceptable range for the agent; in this case, the excess portion is considered zero, and no risk penalty is incurred.

[0119] Next, the server generates a reward signal based on risk penalties and costs. The reward signal is designed following a dual-track principle of decoupling economic and physical multi-objectives. The server first negatively evaluates the cost to obtain an economic reward component, which is negatively correlated with cost; that is, the lower the cost, the larger the economic reward component. Simultaneously, the server determines a physical security penalty component based on the overall violation rate. The physical security penalty component is positively correlated with the overall violation rate; that is, the larger the overall violation rate, the larger the physical security penalty component. In one embodiment, the physical security penalty component equals the overall violation rate multiplied by a preset scaling factor. Then, the server superimposes the physical security penalty component with the risk penalty to obtain an adjusted physical security penalty component. The risk penalty only occurs when the overall violation rate exceeds a risk tolerance threshold; its function is to amplify tail risks on top of the physical security penalty component. The server subtracts the adjusted physical security penalty component from the economic reward component to obtain the final reward signal. The mathematical form of the reward signal can be expressed as: Reward Signal = -Cost - Physical Security Penalty Component - Risk Penalty, where both the physical security penalty component and the risk penalty are non-negative values. In the reward structure, the economic reward component and the physical safety penalty component are structurally decoupled. That is, the economic reward component only reflects the economy of operating costs, while the physical safety penalty component only reflects the safety risks of physical operation.

[0120] Finally, the server feeds back the reward signal to the agent. The agent combines the reward signal with previously stored state and action information to form a complete sample (including state, action, reward, and next state), and stores the sample in the experience replay buffer. During training, the server randomly samples a batch of samples from the experience replay buffer and uses a proximal policy optimization algorithm to update the parameters of the policy network. The core idea of ​​the proximal policy optimization algorithm is to ensure training stability by limiting the step size of policy updates. Specifically, the server first calculates the advantage function, which evaluates the superiority of taking a certain action relative to the average level in a given state. Its physical meaning is the additional reward obtained by that action relative to the average level of the current policy. Then, the server constructs a pruning objective function. The objective function constrains the magnitude of policy updates by limiting the probability ratio of taking the same action in the same state between the old and new policies to within the range of 1 plus or minus a pruning factor, preventing policy collapse caused by excessively large single updates. The server uses stochastic gradient ascent to maximize the pruning objective function, thereby updating the parameters of the policy network to increase the probability of taking actions that yield higher rewards in that state. Simultaneously, the server updates the parameters of the value network, which is used to estimate the state value function. This update method minimizes the mean squared error of the value estimation error. Through multiple iterations of this update process, the parameters of the policy network are gradually optimized, and its output scheduling actions continuously approach the optimal policy while balancing economy and security.

[0121] In this embodiment, the agent is more sensitive to extreme violation scenarios where the overall violation degree exceeds the risk tolerance threshold during the policy update process. This effectively suppresses the agent's tendency to sacrifice physical security in order to reduce costs and alleviates the problem of traditional risk-neutral reinforcement learning algorithms outputting high-risk actions in low-probability severe violation scenarios.

[0122] In one exemplary embodiment, determining a risk penalty based on the excess portion of the overall violation degree relative to the risk tolerance threshold includes: determining the excess portion based on the value by which the overall violation degree exceeds the risk tolerance threshold; determining an amplification factor based on a preset confidence level if the excess portion is greater than zero; determining an amplification value of the excess portion based on the excess portion and the amplification factor; and determining a risk penalty based on the amplification value.

[0123] First, the server calculates the difference between the calculated overall violation score and the risk tolerance threshold synchronously output by the policy network, obtaining the excess portion. The server then checks if the difference is greater than zero. If the difference is greater than zero, it means the overall violation score exceeds the risk tolerance threshold set by the agent, and the system's operating state has entered the tail risk region; in this case, the positive difference is determined as the excess portion. If the difference is less than or equal to zero, it means the overall violation score is below the risk tolerance threshold, and the system's physical operating state is within the agent's acceptable safety range; in this case, there is no excess portion, and the value of the excess portion is set to zero. The above process can be expressed by the formula: Excess portion = max(Overall violation score - Risk tolerance threshold, 0), thus ensuring that the excess portion is a non-negative number.

[0124] Then, when the excess is greater than zero, the server determines the amplification factor based on a preset confidence level. The preset confidence level is a pre-defined probability threshold; in one embodiment, the confidence level is set to 95%. For example, the physical meaning of the confidence level can be understood as the agent focusing on extreme tail events with a probability not exceeding 5% in risk management. The server calculates the amplification factor based on the confidence level, using the formula: Amplification Factor = 1 / (1 - Confidence Level). For example, when the confidence level is 95%, the amplification factor = 1 / (1 - 0.95) = 20. The amplification factor reflects the risk amplification multiple of extreme tail scenarios relative to normal scenarios. The higher the confidence level (i.e., the lower the probability of the tail event being focused on), the larger the amplification factor, and the stronger the penalty for tail risks. It should be noted that the value of the confidence level can be adjusted according to the actual needs of the power grid security level. For example, if the power grid has a high safety level and requires more stringent control of extreme risks, the confidence level can be set to 97.5%, in which case the amplification factor is 40; in another embodiment, if the power grid operating conditions are relatively relaxed, the confidence level can be set to 90%, in which case the amplification factor is 10.

[0125] Next, the server determines the amplification value of the excess portion based on the excess amount and the amplification factor. The server multiplies the excess amount by the amplification factor to obtain the amplification value of the excess amount. For example, when the excess amount is 0.03 per unit and the confidence level is 95% (amplification factor is 20), the amplification value = 0.03 × 20 = 0.60. This amplification value is much larger than the original excess amount, thus significantly amplifying the penalty for tail risk. The above process can be expressed by the formula: Amplification value = Excess amount × Amplification factor = max(Overall violation degree - Risk tolerance threshold, 0) / (1 - Confidence level). This amplification value comprehensively reflects the magnitude of tail risk and the importance of extreme scenarios.

[0126] Finally, the server determines the risk penalty based on the amplification value. The server uses the amplification value calculated in step three as the final value of the risk penalty. In one implementation, the risk penalty is directly equal to the amplification value, i.e., risk penalty = amplification value. In another embodiment, to further balance the proportion of the risk penalty in the overall reward signal, the server can also multiply the amplification value by a preset risk penalty weighting coefficient to obtain the final risk penalty, i.e., risk penalty = risk penalty weighting coefficient × amplification value. The risk penalty weighting coefficient is a preset hyperparameter used to adjust the relative importance of the risk penalty in the total reward signal. The server outputs the finally determined risk penalty, which will be used in the subsequent reward signal generation process.

[0127] It should be noted that when the excess is zero (i.e. the overall violation rate does not exceed the risk tolerance threshold), the server directly sets the risk penalty to zero. At this time, no additional tail risk penalty is generated, and the agent's reward only includes economic costs and regular physical security penalties.

[0128] In this embodiment, by transforming the risk penalty based on conditional risk value into the aforementioned algebraic form, the tail risk penalty can be embedded into the reward signal of deep reinforcement learning without relying on explicit modeling of the probability density function of the overall violation degree or taking the derivative of the risk value quantile. The calculation process is simple and efficient, guiding the policy network to actively reduce the overall violation degree during the update process, keeping it as low as possible below the risk tolerance threshold, thereby effectively reducing the probability of node voltage exceeding limits or line transmission power exceeding limits in distribution networks under extreme scenarios.

[0129] In an exemplary embodiment, generating a reward signal based on risk penalties and costs includes: determining a cost reward component based on the cost corresponding to the exploration scheduling action; wherein the cost reward component is negatively correlated with the cost; determining a physical security penalty component based on the overall violation degree; wherein the physical security penalty component is positively correlated with the overall violation degree; superimposing the physical security penalty component with the risk penalty to obtain an adjusted physical security penalty component; and generating a reward signal based on the cost reward component and the adjusted physical security penalty component.

[0130] First, the server determines the cost reward component based on the cost corresponding to the exploration and scheduling action. Then, the server cost is negative to obtain the cost reward component. Since the cost is a non-negative value, the cost reward component is non-positive after being negatively calculated. When the cost is low (e.g., when renewable energy generation is high and load demand is low, resulting in low electricity purchase costs), the cost reward component is large (e.g., close to zero), meaning the agent performs well economically under this action. When the cost is high (e.g., when renewable energy output is insufficient, requiring large amounts of electricity to be purchased from the upper-level grid), the cost reward component is small (e.g., a large negative value), meaning the agent performs poorly economically under this action. By mapping costs to negative rewards, the agent will tend to choose actions that reduce operating costs during policy updates, thereby achieving the goal of economic optimization.

[0131] Then, the server multiplies the overall violation score by a preset physical security penalty coefficient to obtain the physical security penalty component. The physical security penalty coefficient is a pre-set positive hyperparameter used to control the proportion of the physical security penalty in the overall reward signal. When the overall violation score is zero, the physical security penalty component is zero, indicating that no additional security penalty is imposed on the agent when the system is in a completely safe state. When the overall violation score is greater than zero, the physical security penalty component is positive, and the larger the overall violation score, the larger the physical security penalty component. For example, the physical security penalty coefficient can be set to 10. In this case, if the overall violation score is 0.01, the physical security penalty component is 0.1; if the overall violation score is 0.05, the physical security penalty component is 0.5. In another embodiment, the server can also use a nonlinear mapping function to determine the physical security penalty component, for example, squaring the overall violation score and multiplying it by the penalty coefficient to further increase the penalty intensity.

[0132] Next, the server adds the physical security penalty component to the risk penalty component to obtain the adjusted physical security penalty component. It should be noted that the risk penalty is an additional penalty applied independently of the physical security penalty component. The superposition of the two achieves a secondary reinforcement penalty effect on tail risk. The physical security penalty component penalizes the overall violation throughout the entire process (including regular and tail-end violations), while the risk penalty is only applied when the overall violation exceeds the risk tolerance threshold, and its value is significantly amplified by the amplification factor corresponding to the confidence level. Through superposition, the overall violation will receive a penalty much stronger than that for regular violations after entering the tail-end risk region, effectively guiding the agent to actively control the overall violation below the risk tolerance threshold during policy updates. In one embodiment, when the overall violation does not exceed the risk tolerance threshold, the risk penalty is zero, and the adjusted physical security penalty component is the original physical security penalty component; in this case, the system is only guided by regular physical security penalties. When the overall violation exceeds the risk tolerance threshold, the risk penalty takes a positive value, and the adjusted physical security penalty component is significantly larger than the original physical security penalty component; the system is guided by additional tail-end risk penalties.

[0133] Finally, the server generates a reward signal based on the cost reward component and the adjusted physical security penalty component. The server subtracts the adjusted physical security penalty component from the cost reward component to obtain the final reward signal. In the reward signal, the cost reward component and the physical security penalty component are structurally decoupled. The cost reward component only reflects the economic efficiency of operating costs, while the physical security penalty component only reflects the security risks of physical operation. The two are combined into a unified reward scalar through a subtraction operation. The advantage of this decoupled design is that the agent cannot sacrifice physical security for increased economic benefits, because the physical security penalty component exists independently of the economic reward component, and the two are equal in the reward signal, with no weight offsetting mechanism. When the agent attempts to reduce operating costs at the expense of increasing the overall violation degree, although the cost reward component will increase (because the cost is reduced), the physical security penalty component will also increase simultaneously, and the increase directly corresponds to the increase in the overall violation degree. For extreme tail-end violation scenarios, the additional superposition of risk penalties further increases the adjusted physical security penalty component, thereby effectively suppressing the agent's strategy tendency to sacrifice physical security to reduce costs. After the final generated reward signal is fed back to the agent, the agent uses the near-end policy optimization algorithm to update the parameters of the policy network based on the reward signal, so that the scheduling action output by the policy network continuously approaches the optimal policy in a way that balances economy and safety.

[0134] In this embodiment, the reward signal enables the agent to simultaneously maximize economic benefits and minimize tail physical risks during the policy network update process. This mitigates the problem in traditional risk-neutral reinforcement learning algorithms where the economic objective cancels out the security objective due to the simple linear weighting of economic and security objectives. It also reduces the agent's tendency to pursue local optimal strategies by sacrificing physical security for economic benefits. Thus, while pursuing economic efficiency, the physical operation safety of the distribution network is ensured.

[0135] In an exemplary embodiment, the predicted energy power, predicted load power, and uncertainty interval are input into a policy network to obtain exploratory scheduling actions and risk tolerance thresholds output by the policy network. This includes: constructing an augmented action space, which includes a physical action subspace and a virtual action subspace; inputting the predicted energy power, predicted load power, and uncertainty interval into the policy network so that the policy network outputs exploratory scheduling actions and risk tolerance thresholds within the augmented action space; wherein the exploratory scheduling actions belong to the physical action subspace, and the risk tolerance thresholds belong to the virtual action subspace.

[0136] In the standard Markov decision process framework, the action space only contains physical scheduling actions. To enable the agent to achieve both economic efficiency and the ability to perceive and avoid physical security risks, this embodiment extends the standard action space. First, the server constructs an augmented action space, including two parts: a physical action subspace and a virtual action subspace. Each action dimension in the physical action subspace corresponds to an adjustable physical resource, specifically including the charging and discharging power of all energy storage systems connected to the distribution network and the scheduling amount of all transferable loads connected to the distribution network. In one example, the baseline operating scenario of the distribution network is an IEEE 33-node system, where nodes 10, 20, and 30 are connected to energy storage systems, and nodes 15, 25, and 32 are connected to transferable loads. Therefore, the physical action subspace contains a total of 6 dimensions (3 energy storage charging and discharging powers plus 3 transferable load scheduling amounts). The value range of the physical action subspace is limited by the technical parameters of each physical device. The upper and lower limits of the energy storage charging and discharging power are determined by the rated power of the energy storage device, and the upper and lower limits of the transferable load scheduling amount are determined by the maximum transferable power of the load. The virtual action subspace contains an action dimension, namely the risk tolerance threshold. Unlike physical actions, the risk tolerance threshold does not have a corresponding physical device; its role is to act as an internal variable for risk management during the decision-making process. The range of the risk tolerance threshold is usually set to a non-negative real number interval, with a lower limit of 0 (indicating that the agent has absolutely no tolerance for any physical violations) and an upper limit of a preset large positive number (indicating that the agent has an extremely lenient attitude towards physical violations).

[0137] Then, the server inputs the predicted energy power, predicted load power, and uncertainty interval into the policy network, which outputs exploratory scheduling actions and risk tolerance thresholds within the augmented action space. The server combines the predicted energy power, predicted load power, uncertainty interval, and other necessary state variables (including the state of charge of the energy storage system and the cumulative offset of transferable loads) into an input state vector. After receiving this input state vector, the policy network performs forward propagation calculations using a deep neural network. During forward propagation, the input state vector is first multiplied by the weight matrix from the policy network's input layer to the first hidden layer and a bias term is added. Then, it is transformed using a nonlinear activation function to introduce nonlinear expressive power. The transformed result continues to the next hidden layer, repeating the above weighted summation and nonlinear transformation calculation process. This layer-by-layer propagation process continues until the data is transmitted to the output layer of the policy network. The number of neurons in the output layer is equal to the total dimension of the augmented action space, i.e., the dimension of the physical action subspace plus the dimension of the virtual action subspace. The output layer generates output values ​​using appropriate activation functions based on the value range of each action dimension. For bounded continuous actions such as energy storage charging and discharging power and transferable load scheduling, the Sigmoid activation function can be used to map the output values ​​to the [0, 1] interval, and then scaled to the actual value range through a linear transformation. For risk tolerance thresholds, a linear activation function or a Softplus activation function (ensuring the output value is non-negative) can be used to directly generate the output value. The result generated by the policy network output layer is the complete action vector within the augmented action space, which includes both physical action components and virtual action components.

[0138] During the training phase, the policy network outputs probability distribution parameters in the action space. The server randomly samples from these parameters to obtain an exploratory action sample, which serves as the exploratory scheduling action and its corresponding risk tolerance threshold. The sampled exploratory scheduling action belongs to the physical action subspace and is used to interact with the power grid simulation environment to obtain reward feedback and accumulate training samples. The sampled risk tolerance threshold belongs to the virtual action subspace and is passed to subsequent steps as the basis for calculating risk penalties. By introducing this random sampling mechanism, the agent can fully explore different action combinations in the early stages of training, avoiding premature entrapment in local optima. As training progresses, the policy network gradually converges, the variance of the output distribution gradually decreases, and the agent's behavior tends towards determinism. In the deployment phase, the policy network no longer performs random sampling but directly outputs the mean of the probability distribution as the deterministic action value.

[0139] The exploratory scheduling action and risk tolerance threshold output by the policy network each have different subsequent processing paths. The exploratory scheduling action is passed to the distribution network simulation environment, where the server determines the injected power of each node and performs AC power flow calculations based on this action, thereby calculating the overall violation degree. The risk tolerance threshold is then passed to the reward construction module, where it is used together with the overall violation degree to calculate the risk penalty. The two paths converge in the reward signal generation process in subsequent steps. By using the risk tolerance threshold as a trainable output node of the policy network, the agent, guided by the reward signal, can autonomously learn to set appropriate risk tolerance thresholds under different operating states, achieving dynamic adaptive adjustment of risk preference.

[0140] In another embodiment, if the types of flexible resources connected to the distribution network change, the dimensional structure of the physical action subspace is adjusted accordingly. For example, if the distribution network also includes output limit adjustment of distributed generation, switching of reactive power compensation equipment, or power control of flexible interconnection devices, the physical action subspace can be expanded with corresponding dimensions. If the types of flexible resources in the distribution network are few (e.g., only energy storage systems), the dimensions of the physical action subspace are reduced accordingly. The virtual action subspace remains unchanged, still being a single dimension of risk tolerance threshold. Regardless of how the dimensions of the physical action subspace change, the number of neurons in the policy network output layer always adapts accordingly, and the risk tolerance threshold always exists as a virtual action dimension independent of physical actions.

[0141] To verify the effectiveness of the above-mentioned augmented action space design and policy network training method, the following explanation is based on simulation results of specific scheduling periods. Figure 6 This is a schematic diagram illustrating the power allocation results of various resources during a typical scheduling period in one embodiment, including time-series curves for baseline load, grid power purchase, load reduction / increase, photovoltaic output, energy storage charging and discharging, grid backfeed, and wind power output. It can be seen that during peak photovoltaic and wind power output periods, the energy storage system charges to absorb excess renewable energy, while simultaneously increasing the load transfer capacity. During periods of insufficient renewable energy output, the energy storage system discharges and reduces some load, effectively smoothing net load fluctuations, and the overall grid power purchase remains at a low level.

[0142] While focusing on resource power allocation, the system's flexibility and supply-demand balance are also indicators for verifying the effectiveness of scheduling strategies. Figure 7 This is a timeline diagram illustrating the balance between flexibility requirements and supply in one embodiment. Figure 7 This demonstrates the system's up / down flexibility requirements within the same scheduling day, as well as the flexibility supply provided by energy storage systems and transferable loads. Figure 7As can be seen, the upward flexibility demand (the system needs to increase power) and the downward flexibility demand (the system needs to decrease power) change over time. The energy storage system provides most of the flexibility adjustment capacity with its rapid response capability, and the transferable load serves as a supplement. Together, they meet the flexibility demand, and the supply is always no less than the demand, which verifies the effectiveness of this embodiment in ensuring the flexibility and reliability of system operation.

[0143] In this embodiment, the risk tolerance threshold, as a trainable and differentiable network output node, mitigates the gradient blocking problem caused by the non-differentiability of risk value quantiles in conditional risk value optimization by transforming the task of estimating risk value quantiles into a neural network regression task. Guided by the reward signal, the risk tolerance threshold autonomously learns the appropriate risk tolerance threshold for each operating state, enabling the agent to adaptively adjust its tolerance for physical violations based on wind and solar load forecasts and grid safety margins. This maintains economic efficiency under normal operating scenarios and proactively tightens safety boundaries under extreme disturbance scenarios, thereby mitigating tail-end physical risks.

[0144] In this embodiment, the risk tolerance threshold, as a trainable and differentiable network output node, mitigates the gradient blocking problem caused by the non-differentiability of risk value quantiles in conditional risk value optimization by transforming the task of estimating risk value quantiles into a neural network regression task. Guided by the reward signal, the risk tolerance threshold autonomously learns the appropriate risk tolerance threshold for each operating state, enabling the agent to adaptively adjust its tolerance for physical violations based on wind and solar load forecasts and grid safety margins. This maintains economic efficiency under normal operating scenarios and proactively tightens safety boundaries under extreme disturbance scenarios, thereby mitigating tail-end physical risks.

[0145] In a detailed embodiment, the server first obtains the predicted energy power and predicted load power of the distribution network at the current moment. Specifically, the server can use a long short-term memory network to make predictions based on historical wind and solar power output time-series data, load time-series data, and meteorological variable data, or it can directly receive prediction data from an external system and perform normalization processing.

[0146] Then, the server determines the uncertainty interval characterizing the normal fluctuation range based on the predicted energy power and the predicted load power. The server acquires the predicted energy power and actual output power at multiple times within a historical period, calculates the prediction error samples at each time point, and determines the energy prediction error interval under a preset confidence level based on their statistical distribution; simultaneously, it determines the load fluctuation interval based on the load prediction error; combining the two, it obtains the upward and downward flexibility demand intervals, which serve as the final expression of the uncertainty interval. In another embodiment, the server can input historical energy time-series data and meteorological time-series data into a trained Long Short-Term Memory (LSTM) network, which directly outputs the energy prediction error interval.

[0147] Next, the server inputs the predicted energy power, predicted load power, and uncertainty interval into the trained policy network to obtain exploratory scheduling actions and risk tolerance thresholds. The policy network employs an Actor network with a near-end policy optimization algorithm, and the risk tolerance threshold is equivalent to the risk value quantile of the system's overall violation rate at a given confidence level. The server constructs an augmented action space containing physical action subspaces and virtual action subspaces. The policy network outputs energy storage charging / discharging and transferable load commands in the former and the risk tolerance threshold in the latter. During the training phase, exploratory scheduling actions are obtained through random sampling, while during the deployment phase, deterministic actions are directly output.

[0148] Then, the server performs AC power flow calculations based on the exploration scheduling actions, determines the node voltage and line transmission power, and determines the overall violation degree accordingly. After parsing the scheduling actions, the server calculates the injected power of each node and uses the Newton-Raphson method to solve the power flow; if the power flow does not converge, a circuit breaker mechanism is triggered and a maximum penalty is returned. In the case of convergence, the server determines the voltage limit violation value based on the deviation between the maximum node voltage and the upper and lower limits, and determines the power limit violation value based on the deviation between the maximum line load rate and the preset maximum allowable load rate, taking the larger of the two values ​​as the overall violation degree.

[0149] Subsequently, the server updates the policy network based on the overall violation rate and risk tolerance threshold. The server calculates the operating costs corresponding to exploration scheduling actions, including electricity purchase costs, energy storage loss costs, transferable load compensation costs, wind and solar curtailment penalties, and flexibility deficit penalties. Simultaneously, the server calculates the excess portion of the overall violation rate exceeding the risk tolerance threshold, determines an amplification factor based on a pre-set confidence level, and multiplies the excess portion by the amplification factor to obtain the risk penalty. The server negates the cost to obtain the economic reward component, multiplies the overall violation rate by the penalty factor to obtain the physical security penalty component, and subtracts the sum of the physical security penalty component and the risk penalty component from the economic reward component to generate the final reward signal. The server combines the reward signal with environmental interaction data to form a sample, uses a near-end policy optimization algorithm to prune the objective function of the policy network, and simultaneously updates the value network. Through multiple iterations, the policy network parameters gradually converge.

[0150] Finally, the server re-inputs the predicted energy power, predicted load power, and uncertainty interval into the updated policy network, and schedules distribution network resources based on the deterministic scheduling actions output by the network. During the training phase, the server iteratively updates the policy network using both standard scenarios and additionally constructed extreme distribution scenarios. During the validation phase, typical and extreme day scenarios not involved in the training are used to evaluate economic costs, risk indicators, and the number of circuit breakers. During the online deployment phase, the scheduling server distributes the scheduling instructions output by the policy network to field devices such as energy storage and transferable loads after power flow verification and circuit breaker checks, completing the actual scheduling control.

[0151] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0152] Based on the same inventive concept, this application also provides a distribution network dispatching device for implementing the distribution network dispatching method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more distribution network dispatching device embodiments provided below can be found in the limitations of the distribution network dispatching method described above, and will not be repeated here.

[0153] In one exemplary embodiment, such as Figure 8 As shown, a power distribution network dispatching device 800 is provided, comprising:

[0154] The acquisition module 801 is used to acquire the predicted energy power and predicted load power of the distribution network at the current moment;

[0155] The first determining module 802 is used to determine the uncertainty interval based on the predicted energy power and the predicted load power; the uncertainty interval characterizes the normal fluctuation range of energy and load in the distribution network.

[0156] The input module 803 is used to input the energy forecast power, load forecast power and uncertainty interval into the trained policy network to obtain the exploratory scheduling action and risk tolerance threshold output by the policy network; the risk tolerance threshold represents the tolerance boundary of the comprehensive violation degree.

[0157] The second determining module 804 is used to determine the node voltage and line transmission power of the distribution network based on the exploration and scheduling actions, and to determine the comprehensive violation degree based on the node voltage and line transmission power; the comprehensive violation degree characterizes the degree of exceeding the limit of the node voltage and / or line transmission power of the distribution network.

[0158] The update module 805 is used to update the policy network based on the overall violation rate and risk tolerance threshold to obtain the updated policy network.

[0159] The scheduling module 806 is used to re-input the predicted energy power, predicted load power, and uncertainty interval into the updated strategy network, and to schedule the resources of the distribution network based on the deterministic scheduling actions output by the updated strategy network.

[0160] In an exemplary embodiment, the first determining module 803 is configured to: acquire the energy prediction power and corresponding actual output power of the distribution network at multiple times within a historical period; determine energy prediction error samples at multiple times based on the energy prediction power and actual output power at each time; determine the energy prediction error range based on the statistical distribution of each energy prediction error sample; determine the load fluctuation range based on the fluctuation of the load prediction power; and determine the uncertainty range based on the energy prediction error range and the load fluctuation range.

[0161] In one exemplary embodiment, the apparatus further includes:

[0162] The data acquisition module is used to acquire energy time-series data and meteorological time-series data of the distribution network within historical periods;

[0163] The network input module is used to input energy time-series data and meteorological time-series data within historical periods into a trained long short-term memory network to obtain the energy prediction error range output by the long short-term memory network.

[0164] In an exemplary embodiment, the second determining module 804 is configured to: determine the injected power of each node in the distribution network based on the exploration scheduling action; determine the voltage of each node and the power of each line based on the injected power of each node; determine the voltage limit violation value based on the deviation of the voltage of each node from a preset voltage threshold; determine the power limit violation value based on the deviation of the power of each line from a preset power threshold; and determine the comprehensive violation degree based on the maximum value of the voltage limit violation value and the power limit violation value.

[0165] In an exemplary embodiment, the second determining module 804 is configured to: obtain the maximum voltage and minimum voltage of each node; determine the maximum voltage limit value based on the positive deviation between the maximum voltage and a preset upper voltage limit value; determine the minimum voltage limit value based on the positive deviation between a preset lower voltage limit value and the minimum voltage; and determine the voltage limit value based on the maximum voltage limit value and the minimum voltage limit value.

[0166] In an exemplary embodiment, the update module 805 is configured to: obtain the cost corresponding to the exploration scheduling action output by the policy network; determine the risk penalty based on the excess portion of the comprehensive violation degree relative to the risk tolerance threshold; generate a reward signal based on the risk penalty and cost; and update the parameters of the policy network based on the reward signal.

[0167] In an exemplary embodiment, the update module 805 is configured to: determine the excess portion based on the value of the comprehensive violation exceeding the risk tolerance threshold; determine an amplification factor based on a preset confidence level if the excess portion is greater than zero; determine an amplification value of the excess portion based on the excess portion and the amplification factor; and determine a risk penalty based on the amplification value.

[0168] In an exemplary embodiment, the update module 805 is configured to: determine a cost reward component based on the cost corresponding to the exploration scheduling action; wherein the cost reward component is negatively correlated with the cost; determine a physical security penalty component based on the comprehensive violation degree; wherein the physical security penalty component is positively correlated with the comprehensive violation degree; superimpose the physical security penalty component with the risk penalty to obtain an adjusted physical security penalty component; and generate a reward signal based on the cost reward component and the adjusted physical security penalty component.

[0169] In an exemplary embodiment, the input module 803 is used to: construct an augmented action space, which includes a physical action subspace and a virtual action subspace; input the energy forecast power, load forecast power, and uncertainty interval to the policy network, so that the policy network outputs exploratory scheduling actions and risk tolerance thresholds within the augmented action space; wherein, the exploratory scheduling actions belong to the physical action subspace, and the risk tolerance thresholds belong to the virtual action subspace.

[0170] Each module in the aforementioned power distribution network dispatching device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0171] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores historical operating data of the distribution network, predicted and measured data of new energy output, predicted and measured load data, meteorological time-series data, energy storage system status data, transferable load records, and training samples and model parameters for the policy network and value network. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a distribution network scheduling method.

[0172] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0173] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0174] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0175] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0176] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A distribution network dispatching method, characterized in that, include: Obtain the predicted energy power and predicted load power of the distribution network at the current moment; The uncertainty range is determined based on the predicted energy power and the predicted load power. The uncertainty interval characterizes the normal fluctuation range of energy and load in the distribution network; The predicted energy power, the predicted load power, and the uncertainty interval are input into a trained policy network to obtain the exploratory scheduling action and risk tolerance threshold output by the policy network; the risk tolerance threshold represents the tolerance boundary of the comprehensive violation degree. The node voltage and line transmission power of the distribution network are determined based on the exploration and scheduling actions, and the comprehensive violation degree is determined based on the node voltage and the line transmission power; the comprehensive violation degree characterizes the degree to which the node voltage and / or line transmission power of the distribution network exceeds the limit. The policy network is updated based on the overall violation rate and the risk tolerance threshold to obtain the updated policy network. The predicted energy power, the predicted load power, and the uncertainty interval are re-inputted into the updated strategy network, and the resources of the distribution network are scheduled based on the deterministic scheduling actions output by the updated strategy network.

2. The method according to claim 1, characterized in that, Based on the predicted energy power and the predicted load power, an uncertainty range is determined, including: Obtain the predicted energy power and corresponding actual output power of the power distribution network at multiple points in a historical period; Based on the predicted energy power and the actual output power at each time point, determine energy prediction error samples for multiple time points; Based on the statistical distribution of each energy prediction error sample, the energy prediction error range is determined; The load fluctuation range is determined based on the fluctuation of the predicted load power. The uncertainty range is determined based on the energy forecast error range and the load fluctuation range.

3. The method according to claim 2, characterized in that, Also includes: Acquire the energy time-series data and meteorological time-series data of the power distribution network during the historical period; The energy time-series data and meteorological time-series data within the historical period are input into a trained long short-term memory network to obtain the energy prediction error range output by the long short-term memory network.

4. The method according to claim 1, characterized in that, The node voltages and line transmission power of the distribution network are determined based on the exploratory scheduling actions, and the overall violation degree is determined based on the node voltages and line transmission power, including: The injected power of each node in the distribution network is determined based on the exploratory scheduling action; Based on the injected power of each node, determine the voltage of each node and the power of each line; The voltage limit is determined based on the deviation between the voltage of each node and the preset voltage threshold. The power limit is determined based on the deviation between the power of each line and the preset power threshold. The overall violation level is determined based on the maximum value between the voltage and power over-limit values.

5. The method according to claim 4, characterized in that, Based on the deviation of the voltage at each node from a preset voltage threshold, the voltage exceedance value is determined, including: Obtain the maximum and minimum voltages among the voltages of each node; The maximum voltage exceedance value is determined based on the positive deviation between the maximum voltage and the preset upper voltage limit. The minimum voltage exceedance value is determined based on the positive deviation between the preset lower voltage limit and the minimum voltage. The voltage over-limit value is determined based on the maximum voltage over-limit value and the minimum voltage over-limit value.

6. The method according to claim 1, characterized in that, The policy network is updated based on the overall violation rate and the risk tolerance threshold, including: Obtain the cost corresponding to the exploration scheduling action output by the policy network; Risk penalties are determined based on the portion of the overall violation rate exceeding the risk tolerance threshold. A reward signal is generated based on the risk penalty and the cost, and the parameters of the policy network are updated based on the reward signal.

7. The method according to claim 6, characterized in that, Based on the excess portion of the overall violation rate relative to the risk tolerance threshold, a risk penalty is determined, including: The excess portion is determined based on the value by which the overall violation exceeds the risk tolerance threshold; When the excess portion is greater than zero, the amplification factor is determined according to a preset confidence level; The amplification value of the excess portion is determined based on the excess portion and the amplification factor; The risk penalty is determined based on the amplification value.

8. The method according to claim 6, characterized in that, Generate a reward signal based on the risk penalty and the cost, including: Based on the cost corresponding to the exploration and scheduling action, a cost reward component is determined; wherein, the cost reward component is negatively correlated with the cost; Based on the overall violation rate, a physical security penalty component is determined; wherein, the physical security penalty component is positively correlated with the overall violation rate. The physical security penalty component is superimposed with the risk penalty component to obtain the adjusted physical security penalty component; A reward signal is generated based on the cost reward component and the adjusted physical security penalty component.

9. The method according to claim 1, characterized in that, The predicted energy power, the predicted load power, and the uncertainty interval are input into the policy network to obtain the exploratory scheduling actions and risk tolerance thresholds output by the policy network, including: Construct an augmented action space, which includes a physical action subspace and a virtual action subspace; The predicted energy power, the predicted load power, and the uncertainty interval are input into the policy network, so that the policy network outputs an exploration scheduling action and a risk tolerance threshold within the augmented action space; wherein the exploration scheduling action belongs to the physical action subspace, and the risk tolerance threshold belongs to the virtual action subspace.

10. A power distribution network dispatching device, characterized in that, include: The acquisition module is used to acquire the energy forecast power and load forecast power of the distribution network at the current moment; The first determining module is used to determine the uncertainty range based on the predicted energy power and the predicted load power; The uncertainty interval characterizes the normal fluctuation range of energy and load in the distribution network; The input module is used to input the predicted energy power, the predicted load power, and the uncertainty interval into the trained policy network to obtain the exploratory scheduling action and risk tolerance threshold output by the policy network; the risk tolerance threshold represents the tolerance boundary of the comprehensive violation degree. The second determining module is used to determine the node voltage and line transmission power of the distribution network based on the exploration and scheduling action, and to determine the comprehensive violation degree based on the node voltage and the line transmission power; the comprehensive violation degree characterizes the degree to which the node voltage and / or line transmission power of the distribution network exceeds the limit; The update module is used to update the policy network based on the comprehensive violation rate and the risk tolerance threshold to obtain the updated policy network. The scheduling module is used to re-input the predicted energy power, the predicted load power, and the uncertainty interval into the updated strategy network, and to schedule the resources of the distribution network based on the deterministic scheduling actions output by the updated strategy network.

Images