A power distribution network dispatching optimization method based on exploration-doubt iteration optimization

Abstract

This invention provides a distribution network scheduling optimization method based on exploration-questioning iterative optimization, belonging to the field of distribution network scheduling technology. It includes: S1, Exploration model training based on situation confidence: Constructing a distribution network scheduling optimization mechanism model containing long-term and short-term constraints based on expert experience, building a deep reinforcement learning scheduling optimization model containing exploration neurons, adaptively adjusting exploration coefficients, and optimizing the exploration model; S2, Distribution network scheduling optimization based on cross-questioning of exploration decisions and mechanism models: Cross-questioning decisions based on situation similarity and safety distance, dual-strategy verification and challenge judgment, and model aggregation optimization based on performance gap and safety distance. This invention significantly improves the proactive adaptability and optimization efficiency of distribution network scheduling models in the face of unknown and complex operating conditions, realizes the continuous evolution and self-improvement of scheduling strategies, and effectively balances the safety and economy of distribution network scheduling.

Description

Technical Field

[0001] This invention provides a distribution network scheduling optimization method based on exploration-questioning iterative optimization, belonging to the field of distribution network scheduling technology. Background Technology

[0002] Currently, distribution network dispatching mainly relies on mathematical models built upon physical mechanisms and expert experience. These models rigorously model long-term security constraints (such as energy storage balance) and short-term security constraints (such as line thermal stability), and then use optimization algorithms to solve for dispatching strategies. This experience-based mechanistic model is widely used in traditional power grids, providing deterministic security boundaries for the system. It serves as the fundamental guarantee for current distribution network dispatching and control, ensuring the stable operation of the power grid under known operating conditions.

[0003] However, existing expert-based scheduling optimization methods exhibit a series of limitations when faced with high uncertainty in source loads. First, expert experience often tends to be conservative, leading to excessively large safety domains in the constructed mechanistic models. While ensuring safety, this severely sacrifices operational economy and limits asset utilization efficiency. Second, traditional optimization methods suffer from low computational efficiency when dealing with a vast array of dynamic operating scenarios, making it difficult to meet real-time requirements. More importantly, existing experience struggles to cover all extreme or unknown emergencies, causing models to often fail to output optimal strategies when facing new operating conditions without prior experience. They may even produce suboptimal or no solutions due to insufficient scenario coverage, making it difficult to adapt to the growing intelligent demands of distribution networks. Summary of the Invention

[0004] This invention provides a distribution network scheduling optimization method based on exploration-questioning iterative optimization, which solves the following technical problems:

[0005] 1. To address the problems of insufficient scenario coverage and limited optimization for unknown operating conditions caused by the reliance on expert experience in existing scheduling models, this invention proposes an exploration model training method based on situation confidence. First, a scheduling mechanism model that distinguishes between long-term safety constraints and short-term hard constraints is constructed based on expert experience. A deep reinforcement learning model with exploration neurons in the input layer is established, and output safety is ensured through mechanism verification in the early stages of model shaping. Second, training is conducted in a simulation environment, using clustering algorithms to classify historical operating conditions. The exploration coefficients are adaptively adjusted based on the similarity between the current condition and typical scenarios, as well as historical performance. Finally, during exploration strategy verification, only short-term hard constraints are locked, ignoring long-term constraints to expand the search space, and model parameters are updated through simulation feedback. This invention effectively overcomes the shortcomings of traditional expert experience models, such as excessively large safety domains and insufficient scenario coverage. By using situation confidence to guide the model to conduct targeted and efficient exploration within the safety boundary, it significantly improves the proactive adaptability and optimization efficiency of distribution network scheduling models in the face of unknown and complex operating conditions.

[0006] 2. To address the issues of poor economic efficiency and lack of evolutionary capability in traditional mechanistic models due to conservative safety domains, this invention proposes a distribution network scheduling optimization method based on exploratory decision-making and cross-questioning of the mechanistic model. First, the same grid condition is simultaneously input into both the basic model and the exploratory model, and the strategy deviation is calculated. If the deviation exceeds a threshold, a challenge score is calculated based on condition similarity, historical challenge success rate, and hard constraint safety distance to determine whether to initiate a challenge. Second, after triggering the challenge, both strategies are substituted into a high-precision simulation environment for full-cycle verification. If the exploratory strategy outperforms the basic strategy while satisfying all long-term and short-term constraints, the challenge is considered successful. Finally, aggregate weights are calculated based on performance improvement and hard constraint safety margin, and the mechanistic model is updated with weighted averages. This invention, through a closed-loop iterative mechanism, can break the conservatism of traditional mechanistic models while strictly ensuring system operational safety, achieving continuous evolution and self-improvement of the scheduling strategy, and effectively balancing the safety and economy of distribution network scheduling.

[0007] This invention provides a distribution network scheduling optimization method based on exploration-questioning iterative optimization. First, an expert mechanism model incorporating long-term and short-term constraints is constructed, and a deep reinforcement learning model with exploration neurons in the input layer is established. Mechanism verification assists in the initial model shaping. Second, in a simulation environment, exploration coefficients are adaptively adjusted based on situation clustering similarity, and efficient exploration training is performed only for short-term hard constraints with relaxation. Finally, a cross-questioning mechanism between exploration decisions and the mechanism model is established. Simulation challenges are initiated for situations with large policy deviations. If the exploration strategy performs better while satisfying all constraints, the model parameters are updated based on safety margin aggregation. Through this process, this invention achieves autonomous evolution of the scheduling model under strict safety assurance, effectively solving the problem of conservative and inefficient traditional models.

[0008] The specific technical solution is as follows:

[0009] A distribution network scheduling optimization method based on exploration-questioning iterative optimization includes the following steps:

[0010] S1. Training of the Exploratory Model Based on State Confidence

[0011] First, based on expert experience, a scheduling mechanism model is constructed to distinguish between long-term safety constraints and short-term hard constraints. A deep reinforcement learning model with exploratory neurons in the input layer is also established to ensure output safety through mechanism verification in the early stage of shaping.

[0012] Secondly, training is conducted in a simulation environment, and clustering algorithms are used to classify historical operating conditions. The exploration coefficient is adaptively adjusted based on the similarity between the current condition and typical scenarios, as well as historical performance.

[0013] Finally, when exploring strategy verification, only short-term hard constraints are locked, while long-term constraints are ignored to expand the search space, and model parameters are updated through simulation feedback.

[0014] S2. Distribution network dispatch optimization based on cross-questioning of exploratory decision-making and mechanism models

[0015] First, the same power grid conditions are simultaneously input into the basic model and the exploration model to calculate the strategy deviation. If the deviation exceeds the threshold, the challenge score is calculated by combining the similarity of conditions, the historical challenge success rate, and the hard constraint safety distance, and a decision is made on whether to initiate a challenge.

[0016] Secondly, after the challenge is triggered, the two strategies are substituted into a high-precision simulation environment for full-cycle verification. If the exploration strategy is more economical than the basic strategy under the premise of satisfying all long-term and short-term constraints, the challenge is judged to be successful.

[0017] Finally, the aggregate weights are calculated based on the performance improvement and the safety margin of hard constraints, and the mechanistic model is updated in a weighted manner.

[0018] Further:

[0019] S1. Training the exploration model based on situation confidence, specifically including the following sub-steps:

[0020] S1.1 Constructing a distribution network dispatch optimization mechanism model based on expert experience, incorporating both long-term and short-term constraints.

[0021] For the distribution network source-grid-load-storage integration scenario, a scheduling optimization mechanism model is constructed based on expert experience. This model takes minimizing the overall system operating cost within the scheduling cycle as its objective function and strictly divides the constraints into short-term and long-term security constraints. Short-term security constraints refer to the physical hard limitations that must be met at any single time point, while long-term security constraints refer to limitations involving time coupling or cumulative effects over the entire cycle. The mathematical model is established as follows:

[0022] (1)

[0023] In the formula, scheduling period Total cost within; It is a collection of generator sets, nodes, lines, energy storage devices, and on-load tap-changing transformers; This is a time-period index. For the first The power generation cost coefficient of the Taiwanese generator unit; For the first Time period The active power output of the Taiwanese generator set; For the first The operation and maintenance costs of the unit; For the first The unit price of electricity purchased from the grid during the specified time period; For the first Power exchanged with the upstream power grid during specific time periods; This is the network loss penalty coefficient; For the first System network loss during certain periods; For the first Lifetime loss cost coefficient of an energy storage device; For the first Time period The charging and discharging power of an energy storage device;

[0024] In short-run constraints: For the first The lower and upper limits of the unit's output; For the first Time period The voltage amplitude at each node, For the first The lower and upper limits of the voltage safety threshold for each node; For the first Time period The current amplitude of the line, For the first The thermal stability limit of the line; For the first Time period The load power of each node; For the first Time period The state of charge of an energy storage device; For the first The self-discharge rate of the energy storage unit; The first The charging and discharging efficiency of an energy storage device; The first The charging and discharging power components of an energy storage device; For the first The lower and upper limits of the State of Charge (SOC) of an energy storage battery; For the first The initial and final SOC states of an energy storage system.

[0025] In long-term constraints: For the first Time period The tap position of the transformer; For the first The maximum number of permissible operations for a transformer throughout its entire lifecycle.

[0026] S1.2 Constructing a deep reinforcement learning scheduling optimization model with exploratory neurons

[0027] Based on historical operational data, a distribution network scheduling optimization model, or basic model, based on deep reinforcement learning is constructed. An independent exploration neuron unit is set in the input layer of the model, with its value corresponding to the exploration weight. In the early stages of model shaping, the value of the exploration neuron is forcibly set to 0. At this time, the output result must pass the safety verification of the S1.1 mechanism model; if the constraints are violated, rule correction is applied. After the basic model training converges, the network structure and parameters of this model are copied to generate the exploration model. The network forward propagation and exploration mechanism are defined as follows:

[0028] (2)

[0029] (3)

[0030] In the formula, For the first The scheduling action vector output by the time-segment model; The activation function for the output layer; The output layer weight matrix and bias vector; This is the feature extraction function for the hidden layer of a neural network. For the first The performance of the time-period distribution network scheduling optimization model is defined in S1.3; Network parameters of the basic model; For the first Random noise signals generated over a period of time; The weight values ​​of neurons are determined during the scheduling phase to explore their potential. The adaptive exploration coefficients are calculated based on the state during the exploration phase; Phase indicates the current training phase of the model, Shaping indicates the model shaping phase, and Exploring indicates the exploration phase.

[0031] S1.3 Adaptive adjustment of exploration coefficient

[0032] The exploratory model was trained in a distribution network dispatch optimization simulation environment. A clustering algorithm was used to analyze the historical distribution network operation status. Divided into A typical scenario, among which For the first The first of the typical scenario's state characteristics Dimensional value. Define the exploration coefficient. Its value depends on the similarity between the current situation and the center of a typical scene, as well as the model's historical performance in the corresponding scene. Higher similarity and better historical performance result in a greater exploration weight. For the current... Feature extraction is performed on the operating status of the distribution network during different time periods, and the first feature of the status is... Dimensional numerical representation is .but The calculation is as follows:

[0033] (4)

[0034] (5)

[0035] (6)

[0036] In the formula, For the first Current status of the time period With the Cluster centers The weighted Euclidean distance; The dimension of the state vector; For the first Weights of state features; For the current state Dimensional value; For the first The first cluster center Dimensional value. To represent the scenario that most closely resembles the typical case obtained from clustering, For the first Time period The weighted Euclidean distance between the cluster centers. The baseline exploration coefficient; This is the distance attenuation parameter; As a performance incentive factor; To explore the model in the first Historical average reward value in similar scenarios; The baseline reward value for the basic model; To prevent small quantities with a denominator of zero.

[0037] S1.4 Exploration Model Optimization

[0038] When performing safety checks and gradient updates, the exploration strategy only considers the short-term safety constraints defined in S1.1; the loss function of the exploration model is expressed as:

[0039] (7)

[0040] (8)

[0041] In the formula, To explore the loss function of the model; To explore model parameters; Indicates the expected value; Economic rewards for environmental feedback; The penalty weight for violating short-term hard constraints; This is the function for penalizing exceeding the limit; For the first The first period The physical quantity values ​​corresponding to each short-term safety constraint. This represents the total number of short-term constraints in the model. These correspond to the upper and lower limits of the constraints in S1.1.

[0042] S2. Distribution network dispatch optimization based on cross-questioning of exploratory decision-making and mechanism models, specifically including the following sub-steps:

[0043] S2.1 Cross-questioning decision based on situation similarity and safety distance

[0044] The same power grid operation status Simultaneously, the inputs are fed into both the basic scheduling optimization model and the exploration model. The policy deviation between the two is calculated. When the deviation exceeds a preset threshold, the exploration model calculates a challenge score based on situation similarity, historical challenge success rate, and hard constraint safety distance, and decides whether to initiate a challenge, expressed as:

[0045] (9)

[0046] (10)

[0047] In the formula, This is a strategy normalization bias; For the action dimension; The basic model and the exploratory model are respectively in the first two stages. Time period Dimensional action output. To score points for the challenge; These are the weighting coefficients; The maximum cluster distance; For the first Success rate of historical challenges in similar scenarios; This represents the absolute margin of the current physical quantity from the nearest hard constraint boundary. This is the nominal value of the physical quantity. Only when... and If all values ​​exceed the threshold, a challenge is initiated.

[0048] S2.2 Dual-Strategy Verification and Challenge Judgment

[0049] After the challenge is initiated, the two strategies are substituted into a high-precision distribution network dispatch optimization simulation environment. If the objective function value of the exploration model strategy is better than that of the mechanistic model, provided that all constraints described in S1.1 are satisfied, then the challenge is considered successful. The determination logic is as follows:

[0050] (11)

[0051] In the formula, The sign indicating a successful challenge; The total cost objective function defined for S1.1; These are the exploration strategy and the mechanism model strategy vectors, respectively. The set of short-run constraint violations is defined if and only if all The value is 0 when the constraint is satisfied. For a set of long-term constraint violations, if and only if all The value is 0 when the constraint is satisfied.

[0052] S2.3 Model Aggregation Optimization Based on Performance Gap and Safety Distance

[0053] After successfully completing the challenge, the performance gap between the exploratory model and the mechanistic model is calculated, along with the hard constraint safety distance of the exploratory model. Based on this, aggregation weights are set to update the mechanistic model. For safety reasons, a weight cap is set. The aggregation update formula is expressed as:

[0054] (12)

[0055] (13)

[0056] In the formula, Aggregate weights for the model; This represents the upper limit of the weight. This represents the performance improvement value. To explore the normalized minimum distance of the strategy to the nearest short-term hard constraint boundary in the simulation; Impact factor weights; This is the sensitivity coefficient. For the updated mechanistic model parameters; The parameters of the mechanistic model before the update; These are the parameters for the current exploration model.

[0057] The beneficial effects of the technical solution of this invention are as follows:

[0058] 1. This invention proposes a training method for an exploration model based on situation confidence. First, a scheduling mechanism model that distinguishes between long-term safety constraints and short-term hard constraints is constructed based on expert experience. A deep reinforcement learning model with exploration neurons in the input layer is established, and output safety is ensured through mechanism verification in the early stages of model shaping. Second, training is conducted in a simulation environment. Historical operating conditions are classified using a clustering algorithm, and the exploration coefficients are adaptively adjusted based on the similarity between the current condition and typical scenarios, as well as historical performance. Finally, during exploration strategy verification, only short-term hard constraints are locked, ignoring long-term constraints to expand the search space, and model parameters are updated through simulation feedback. This invention effectively overcomes the shortcomings of traditional expert experience models, such as excessively large safety domains and insufficient scenario coverage. By using situation confidence to guide the model to conduct targeted and efficient exploration within the safety boundary, it significantly improves the proactive adaptability and optimization efficiency of the distribution network scheduling model in the face of unknown and complex operating conditions.

[0059] 2. This invention proposes a distribution network scheduling optimization method based on cross-questioning of exploratory decision-making and mechanistic models. First, the same grid condition is simultaneously input into both the basic model and the exploratory model, and the strategy deviation is calculated. If the deviation exceeds a threshold, a challenge score is calculated based on condition similarity, historical challenge success rate, and hard constraint safety distance to determine whether to initiate a challenge. Second, after triggering the challenge, both strategies are substituted into a high-precision simulation environment for full-cycle verification. If the exploratory strategy is more economical than the basic strategy while satisfying all long-term and short-term constraints, the challenge is considered successful. Finally, aggregate weights are calculated based on performance improvement and hard constraint safety margin, and the mechanistic model is updated with weighted averages. This invention, through a closed-loop iterative mechanism, can break the conservatism of traditional mechanistic models while strictly ensuring system operational safety, achieving continuous evolution and self-improvement of scheduling strategies, and effectively balancing the safety and economy of distribution network scheduling. Attached Figure Description

[0060] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0061] The specific technical solution of the present invention will be described in conjunction with the accompanying drawings.

[0062] This invention proposes a distribution network scheduling optimization method based on exploration-challenge iterative optimization. The method includes training a deep reinforcement learning model with adaptive exploration coefficients to generate new strategies, constructing a cross-challenge mechanism between the new strategy and the underlying mechanism model, and aggregating and updating the underlying mechanism model after a successful challenge. This achieves safe iteration and continuous optimization of the scheduling strategy. The specific process is as follows: Figure 1 As shown.

[0063] S1 Exploratory Model Training Method Based on State Confidence

[0064] This invention constructs a deep reinforcement learning model containing exploratory neurons and trains the exploratory model in a simulation environment using adaptive exploratory coefficients based on situation clustering similarity and historical performance feedback. This enables autonomous strategy exploration under the premise of ensuring short-term hard constraints, effectively solving the problems of insufficient coverage of expert experience scheduling scenarios and low optimization efficiency.

[0065] S1.1 Constructing a distribution network dispatch optimization mechanism model based on expert experience, incorporating both long-term and short-term constraints.

[0066] For distribution network source-grid-load-storage integration scenarios, a scheduling optimization mechanism model is constructed based on expert experience. This model takes minimizing the overall system operating cost within the scheduling cycle as its objective function and strictly divides the constraints into short-term and long-term security constraints. Short-term security constraints refer to the physical hard limitations that must be met at any single time point, while long-term security constraints refer to limitations involving time coupling or cumulative effects over the entire cycle. For example, the following mathematical model can be established, expressed as:

[0067] (1)

[0068] In the formula, scheduling period Total cost within; It is a collection of generator sets, nodes, lines, energy storage devices, and on-load tap-changing transformers; This is a time-period index. For the first The power generation cost coefficient of the Taiwanese generator unit; For the first Time period The active power output of the Taiwanese generator set; For the first The operation and maintenance costs of the unit; For the first The unit price of electricity purchased from the grid during the specified time period; For the first Power exchanged with the upstream power grid during specific time periods; This is the network loss penalty coefficient; For the first System network loss during certain periods; For the first Lifetime loss cost coefficient of an energy storage device; For the first Time period The charging and discharging power of an energy storage device (charging is positive, discharging is negative).

[0069] In short-run constraints: For the first The lower and upper limits of the unit's output; For the first Time period The voltage amplitude at each node, For the first The lower and upper limits of the voltage safety threshold for each node; For the first Time period The current amplitude of the line, For the first The thermal stability limit of the line; For the first Time period The load power of each node; For the first Time period The state of charge of an energy storage device; For the first The self-discharge rate of the energy storage unit; The first The charging and discharging efficiency of an energy storage device; The first The charging and discharging power components of an energy storage device; For the first The lower and upper limits of the State of Charge (SOC) of an energy storage battery; For the first The initial and final SOC states of an energy storage system.

[0070] In long-term constraints: For the first Time period The tap position of the transformer; For the first The maximum number of permissible operations for a transformer throughout its entire lifecycle.

[0071] This invention does not limit the objective function, constraints, and network topology in the above model. Technical personnel can flexibly expand or adjust the model according to the actual form of the distribution network (such as radial, ring, or independent microgrids) and specific business needs. For example, for scenarios where high-proportion distributed photovoltaic access leads to fluctuations on both the source and load sides, constraints related to preventing node voltage overruns and backflow can be added; for low-carbon-oriented dispatching needs, carbon emission indicators can be incorporated into the objective function; for vehicle-grid interaction or demand response scenarios, the adjustable capacity constraints of the electric vehicle fleet and the interruption characteristic constraints of flexible loads can be further expanded.

[0072] S1.2 Constructing a deep reinforcement learning scheduling optimization model with exploratory neurons

[0073] Based on historical operational data, a distribution network scheduling optimization model (basic model) based on deep reinforcement learning is constructed. An independent exploratory neuron unit is set in the input layer of the model, with its value corresponding to the exploratory weight. In the early stages of model shaping, the value of the exploratory neuron is forcibly set to 0. At this time, the output result must pass the safety verification of the mechanism model described in S1.1; if the constraints are violated, rule correction is applied. After the basic model training converges, the network structure and parameters of this model are copied to generate the exploratory model. The network forward propagation and exploration mechanism are defined as follows:

[0074] (2)

[0075] (3)

[0076] In the formula, For the first The scheduling action vector output by the time-segment model; The activation function for the output layer; The output layer weight matrix and bias vector; This is the feature extraction function for the hidden layer of a neural network. For the first The performance of the time-period distribution network scheduling optimization model is defined in S1.3; Network parameters of the basic model; For the first Random noise signals generated over a period of time; The weight values ​​of neurons are determined during the scheduling phase to explore their potential. The adaptive exploration coefficients are calculated based on the state during the exploration phase; Phase indicates the current training phase of the model, Shaping indicates the model shaping phase, and Exploring indicates the exploration phase.

[0077] S1.3 Adaptive adjustment of exploration coefficient

[0078] The exploratory model was trained in a distribution network dispatch optimization simulation environment. A clustering algorithm was used to analyze the historical distribution network operation status. Divided into A typical scenario, among which For the first The first of the typical scenario's state characteristics Dimensional value. Define the exploration coefficient. Its value depends on the similarity between the current situation and the center of a typical scene, as well as the model's historical performance in the corresponding scene. Higher similarity and better historical performance result in a greater exploration weight. For the current... Feature extraction is performed on the operating status of the distribution network during different time periods, and the first feature of the status is... Dimensional numerical representation is .but The calculation is as follows:

[0079] (4)

[0080] (5)

[0081] (6)

[0082] In the formula, For the first Current status of the time period With the Cluster centers The weighted Euclidean distance; The dimension of the state vector; For the first Weights of state features; For the current state Dimensional value; For the first The first cluster center Dimensional value. To represent the scenario that most closely resembles the typical case obtained from clustering, For the first Time period The weighted Euclidean distance between the cluster centers. The baseline exploration coefficient; This is the distance attenuation parameter; As a performance incentive factor; To explore the model in the first Historical average reward value in similar scenarios; The baseline reward value for the basic model; To prevent small quantities with a denominator of zero.

[0083] S1.4 Exploration Model Optimization

[0084] The exploration strategy, when performing safety checks and gradient updates, only considers short-term safety constraints (such as voltage, current, and power balance) defined in S1.1, temporarily ignoring long-term coupling constraints to expand the search space and find potential global optima. The loss function of the exploration model is expressed as:

[0085] (7)

[0086] (8)

[0087] In the formula, To explore the loss function of the model; To explore model parameters; Indicates the expected value; Economic rewards for environmental feedback; The penalty weight for violating short-term hard constraints; The penalty function is for exceeding the limit. The specific form of the penalty function varies depending on the constraints. In actual engineering construction, it can be expressed differently according to the physical characteristics of the controlled object (such as transformer, line, inverter, etc.). For example, it can be defined as a quantitative value reflecting the risk of insulation aging cost caused by short-term overload operation, depreciation acceleration cost caused by line heating, or potential fault shutdown. This invention does not strictly limit the specific mathematical structure of the penalty function, and it can be defined according to actual operation needs and equipment physical characteristics. For the first The first period The physical quantity values ​​corresponding to each short-term safety constraint. This represents the total number of short-term constraints in the model. This corresponds to the upper and lower bounds of the constraints in S1.1. This step ensures that the strategy generated by the exploration model is instantaneously feasible at the physical level.

[0088] S2 A Distribution Network Dispatch Optimization Method Based on Cross-Questioning of Exploratory Decision-Making and Mechanism Models

[0089] This invention constructs a cross-questioning mechanism between exploratory decision-making and mechanistic models. When the strategy deviation is large, simulation challenges are triggered based on situation similarity. After the challenge is successfully completed, the mechanistic model is updated using aggregated weights based on performance gap and safety margin. This achieves safe iterative optimization of the distribution network scheduling model and breaks through the conservative limitations of traditional mechanistic models.

[0090] S2.1 Cross-questioning decision based on situation similarity and safety distance

[0091] The same power grid operation status Simultaneously, the inputs are fed into both the basic scheduling optimization model and the exploration model. The policy deviation between the two is calculated. When the deviation exceeds a preset threshold, the exploration model calculates a challenge score based on situation similarity, historical challenge success rate, and hard constraint safety distance, and decides whether to initiate a challenge, expressed as:

[0092] (9)

[0093] (10)

[0094] In the formula, This is a strategy normalization bias; For the action dimension; The basic model and the exploratory model are respectively in the first two stages. Time period Dimensional action output. To score points for the challenge; These are the weighting coefficients; The maximum cluster distance; For the first Success rate of historical challenges in similar scenarios; This represents the absolute margin of the current physical quantity from the nearest hard constraint boundary. This is the nominal value of the physical quantity. Only when... and If all values ​​exceed the threshold, a challenge is initiated.

[0095] S2.2 Dual-Strategy Verification and Challenge Judgment

[0096] After the challenge is initiated, the two strategies are substituted into a high-precision distribution network dispatch optimization simulation environment. If the objective function value of the exploration model strategy is better than that of the mechanistic model, provided that all constraints described in S1.1 (including short-term hard constraints and long-term safety constraints) are satisfied, the challenge is considered successful. The determination logic is as follows:

[0097] (11)

[0098] In the formula, The sign indicating a successful challenge; The total cost objective function defined for S1.1; These are the exploration strategy and the mechanism model strategy vectors, respectively. The set of short-run constraint violations is defined if and only if all The value is 0 when the constraint is satisfied. For a set of long-term constraint violations, if and only if all The value is 0 when the constraints are met. This step ensures that the exploration strategy is not only more economical but also meets the safety requirements throughout the entire lifecycle.

[0099] S2.3 Model Aggregation Optimization Based on Performance Gap and Safety Distance

[0100] After successfully completing the challenge, the performance gap between the exploratory model and the mechanistic model is calculated, along with the hard constraint safety distance of the exploratory model. Based on this, aggregation weights are set to update the mechanistic model. For safety reasons, a weight cap is set. The aggregation update formula is expressed as:

[0101] (12)

[0102] (13)

[0103] In the formula, Aggregate weights for the model; This represents the upper limit of the weight. This represents the performance improvement value. To explore the normalized minimum distance of the strategy to the nearest short-term hard constraint boundary in the simulation; Impact factor weights; This is the sensitivity coefficient. For the updated mechanistic model parameters; The parameters of the mechanistic model before the update; These are the parameters for the current exploratory model. This step enables the gradual optimization of the mechanistic model while ensuring safety.

Claims

1. A distribution network dispatch optimization method based on exploration-questioning iterative optimization, characterized in that, Includes the following steps: S1. Training of the Exploratory Model Based on State Confidence First, for the distribution network source-grid-load-storage access scenario, a scheduling optimization mechanism model is constructed based on expert experience. This model takes minimizing the overall system operating cost within the scheduling cycle as the objective function and strictly divides the constraints into short-term security constraints and long-term security constraints. Furthermore, a deep reinforcement learning model with exploratory neurons in the input layer is established, and the output safety is ensured through mechanism verification in the early stage of shaping; Secondly, training is conducted in a simulation environment, and clustering algorithms are used to classify historical operating conditions. Based on the similarity between the current condition and typical scenarios and historical performance, the exploration coefficient is adaptively adjusted. Finally, when exploring model strategy verification, only short-term security constraints are locked and long-term security constraints are ignored to expand the search space, and model parameters are updated through simulation feedback. S2. Distribution network dispatch optimization based on cross-questioning of exploratory decision-making and mechanism models First, the same power grid condition is simultaneously input into the basic model and the exploration model to calculate the strategy deviation. If the deviation exceeds the threshold, the challenge score is calculated by combining the condition similarity, historical challenge success rate and short-term safety constraint safety distance, and then it is decided whether to initiate a challenge. Secondly, after the challenge is triggered, the dual strategies will be substituted into the high-precision distribution network scheduling optimization simulation environment for full-cycle verification. If the economic efficiency of the exploration model strategy is better than that of the basic strategy under the premise of satisfying all long-term and short-term security constraints, the challenge will be judged as successful. Finally, after successfully completing the challenge, the performance gap between the exploration model and the mechanism model is calculated, as well as the short-term safety constraint safety distance of the exploration model. Based on this, the aggregation weight is set, and the mechanism model is updated with weight.

2. The distribution network dispatch optimization method based on exploration-questioning iterative optimization according to claim 1, characterized in that, S1. Training the exploration model based on situation confidence, specifically including the following sub-steps: S1.1 Constructing a distribution network dispatch optimization mechanism model based on expert experience, incorporating both long-term and short-term security constraints. For the distribution network source-grid-load-storage integration scenario, a scheduling optimization mechanism model is constructed based on expert experience. This model takes minimizing the overall system operating cost within the scheduling cycle as its objective function and strictly divides the constraints into short-term and long-term security constraints. Short-term security constraints refer to the physical hard limitations that must be met at any single time point, while long-term security constraints refer to limitations involving time coupling or full-cycle accumulation. The following mathematical model is established, expressed as: （1） In the formula, scheduling period Total cost within; It is a collection of generator sets, nodes, lines, energy storage devices, and on-load tap-changing transformers; Indexed by time period; For the first The power generation cost coefficient of the Taiwanese generator unit; For the first Time period The active power output of the Taiwanese generator set; For the first The operation and maintenance costs of the unit; For the first The unit price of electricity purchased from the grid during the specified time period; For the first Power exchanged with the upstream power grid during specific time periods; This is the network loss penalty coefficient; For the first System network loss during certain periods; For the first Lifetime loss cost coefficient of an energy storage device; For the first Time period The charging and discharging power of an energy storage device; In short-term security constraints: For the first The lower and upper limits of the unit's output; For the first Time period The voltage amplitude at each node, For the first The lower and upper limits of the voltage safety threshold for each node; For the first Time period The current amplitude of the line, For the first The thermal stability limit of the line; For the first Time period The load power of each node; For the first Time period The state of charge of an energy storage device; For the first The self-discharge rate of the energy storage unit; The first The charging and discharging efficiency of an energy storage device; The first The charging and discharging power components of an energy storage device; For the first The lower and upper limits of the State of Charge (SOC) of an energy storage battery; For the first The initial and final SOC states of an energy storage device; In long-term security constraints: For the first Time period The tap position of the transformer; For the first The maximum permissible number of operations for a transformer throughout its entire lifecycle; S1.2 Constructing a deep reinforcement learning scheduling optimization model with exploratory neurons Based on historical operational data, a distribution network scheduling optimization model based on deep reinforcement learning, i.e., the basic model, is constructed. An independent exploration neuron unit is set in the input layer of the model, with its value corresponding to the exploration weight. In the early stage of model shaping, the value of the exploration neuron is forcibly set to 0. At this time, the output result must pass the safety verification of the S1.1 mechanism model; if the constraint is violated, rule correction is adopted. After the basic model training converges, the network structure and parameters of the model are copied to generate the exploration model. The network forward propagation and exploration mechanism are defined as follows: （2） （3） In the formula, For the first The scheduling action vector output by the time-segment model; The activation function for the output layer; The output layer weight matrix and bias vector; This is the feature extraction function for the hidden layer of a neural network. For the first The performance of the time-period distribution network scheduling optimization model is defined in S1.3; Network parameters of the basic model; For the first Random noise signals generated over a period of time; The weight values ​​of neurons are determined during the scheduling phase to explore their potential. For the adaptive exploration coefficients calculated based on the state during the exploration phase; Phase indicates the current training phase of the model, Shaping indicates the model shaping phase, and Exploring indicates the exploration phase; S1.3 Adaptive adjustment of exploration coefficient The exploratory model was trained in a power distribution network dispatch optimization simulation environment; Using clustering algorithms to analyze the historical operation status of the power distribution network Divided into A typical scenario, among which For the first The first of the typical scenario's state characteristics Dimensional value; defining the exploration coefficient Its value depends on the similarity between the current situation and the center of a typical scene, as well as the model's historical performance in the corresponding scene; the higher the similarity and the better the historical performance, the greater the exploration weight is assigned; for the current... Feature extraction is performed on the operating status of the distribution network during different time periods, and the first feature of the status is... Dimensional numerical representation is ;but The calculation is as follows: （4） （5） （6） In the formula, For the first Current status of the time period With the Cluster centers The weighted Euclidean distance; The dimension of the state vector; For the first Weights of state features; For the current state Dimensional value; For the first The first cluster center Dimensional value; To represent the scenario that most closely resembles the typical case obtained from clustering, For the first Time period The weighted Euclidean distance between the cluster centers; The baseline exploration coefficient; This is the distance attenuation parameter; As a performance incentive factor; To explore the model in the first Historical average reward value in similar scenarios; The baseline reward value for the basic model; To prevent small quantities with a denominator of zero; S1.4 Exploration Model Optimization When performing safety checks and gradient updates, the exploration model strategy only considers the short-term safety constraints defined in S1.1; the loss function of the exploration model is expressed as: （7） （8） In the formula, To explore the loss function of the model; To explore model parameters; Indicates the expected value; Economic rewards for environmental feedback; The penalty weight for violating short-term security constraints; This is the function for penalizing exceeding the limit; For the first The first period The physical quantity values ​​corresponding to each short-term safety constraint. This represents the total number of short-term safety constraints in the model; These correspond to the upper and lower limits of the constraints in S1.

1.

3. The distribution network dispatch optimization method based on exploration-questioning iterative optimization according to claim 2, characterized in that, S2. Distribution network dispatch optimization based on cross-questioning of exploratory decision-making and mechanism models, specifically including the following sub-steps: S2.1 Cross-questioning decision based on situation similarity and safety distance The same power grid operation status Simultaneously, the data is input into both the basic scheduling optimization model and the exploration model; the policy deviation between the two is calculated. When the deviation exceeds a preset threshold, the exploration model calculates the challenge score based on situation similarity, historical challenge success rate, and short-term safety constraint distance, and decides whether to initiate a challenge, as shown below: （9） （10） In the formula, This is a strategy normalization bias; For the action dimension; The basic model and the exploratory model are respectively in the first two stages. Time period Dimensional action output; To score points for the challenge; These are the weighting coefficients; The maximum cluster distance; For the first Success rate of historical challenges in similar scenarios; This represents the absolute margin of the current physical quantity from the nearest short-term safety constraint boundary; This is the nominal value of the physical quantity; only when and If all values ​​exceed the threshold, initiate a challenge; S2.2 Dual-Strategy Verification and Challenge Judgment After the challenge is initiated, the two strategies are substituted into the high-precision distribution network dispatch optimization simulation environment. If the objective function value of the exploration model strategy is better than that of the mechanistic model under the premise of satisfying all the constraints described in S1.1, the challenge is considered successful. The determination logic is as follows: （11） In the formula, The sign indicating a successful challenge; The total cost objective function defined for S1.1; These are the strategy vectors for the exploration model and the strategy vector for the mechanism model, respectively. The set of short-term safety constraint violations is defined if and only if all The value is 0 when the constraint is satisfied. For the set of long-term security constraint violations, if and only if all The value is 0 when the constraint is satisfied. S2.3 Model Aggregation Optimization Based on Performance Gap and Safety Distance After successfully completing the challenge, the performance gap between the exploration model and the mechanism model is calculated, as well as the short-term safety constraint safety distance of the exploration model. Based on this, the aggregation weight is set, and the mechanism model is updated. For safety reasons, a weight cap is set. The aggregation update formula is expressed as: （12） （13） In the formula, Aggregate weights for the model; This represents the upper limit of the weight. This represents the performance improvement value. To explore the normalized minimum distance of the model strategy to the nearest short-term safety constraint boundary in simulation; Impact factor weights; This is the sensitivity coefficient; For the updated mechanistic model parameters; The parameters of the mechanistic model before the update; These are the parameters for the current exploration model.

Images