Self-adaptive scheduling method and system for power distribution area
By constructing a reward function and an improved deep reinforcement learning algorithm, combined with a Markov decision process model and a dual Q-network architecture, the problem of insufficient scheduling flexibility in distribution substations was solved, enabling adaptive scheduling of high proportions of renewable energy and improving the stability and responsiveness of the power grid.
Patent Information
- Application Number
- CN202511927935.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-19
- Publication Date
- 2026-03-27
AI Technical Summary
Traditional distribution area dispatching methods are ill-suited to the uncertainties and volatility of high-proportion renewable energy sources, resulting in insufficient dispatching flexibility, unstable dispatching actions, and impacting grid stability and economic efficiency.
A reward function is constructed based on the Markov decision process model and an improved deep reinforcement learning algorithm. By utilizing multi-source states and scheduling instructions, the scheduling strategy is optimized through state normalization and feature expansion. Adaptive scheduling is achieved by combining a dual Q-network architecture and a time-dependent noise mechanism.
It improves the global optimization capability of the scheduling strategy, enhances the stability and real-time response capability of the power grid, overcomes the impact of random fluctuations in source and load, and achieves smoothness and stability of high-frequency and fast scheduling.
Smart Images
Figure CN121749134A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical field of distribution radio area scheduling, and more specifically, to an adaptive scheduling method and system for distribution radio areas. Background Technology
[0002] As the final link in the power system, distribution substations directly face users, and the balance of their power supply and demand is crucial to the safe and stable operation of the entire power grid. Furthermore, with the integration of a high proportion of renewable energy, traditional power dispatching methods are increasingly unable to cope with the uncertainties, volatility, and uneven load distribution brought about by this integration. Therefore, researching adaptive dispatching methods for distribution substations based on wind, solar, energy storage, and load response can improve energy utilization efficiency, reduce operating costs, and enhance system flexibility and stability, which has significant practical implications.
[0003] Traditional distribution area scheduling methods focus on building mathematical models and using optimization algorithms to find the optimal scheduling scheme for the system. The accuracy of this approach depends on precise mathematical models and is difficult to handle the uncertainties and fluctuations brought by renewable energy sources such as wind and solar power. To reduce the reliance on precise models, heuristic algorithms based on genetic algorithms and ant colony algorithms have been applied to distribution area scheduling. These methods do not require complex mathematical solutions and can handle nonlinear scheduling problems. However, these methods mainly rely on massive random trial and error to find the answer, requiring a long iteration time and relatively slow convergence speed, especially when dealing with large-scale problems, resulting in low computational efficiency.
[0004] With the development of big data and artificial intelligence technologies, scheduling methods based on deep reinforcement learning and other AI technologies are gradually being applied to distribution substations. This method continuously optimizes scheduling strategies through training on historical and real-time data, exhibiting strong adaptability. However, it is prone to falling into the trap of "local optima" during the adjustment process, and its adjustment actions oscillate due to noise disturbances, making it unable to adapt to sudden changes in operating conditions and directly affecting grid stability, economy, and equipment safety. Summary of the Invention
[0005] To address the issues of insufficient scheduling flexibility and unstable scheduling actions in existing distribution area scheduling methods, this invention proposes an adaptive scheduling method for distribution areas. This method improves scheduling flexibility, performs high-frequency and rapid scheduling through a pre-trained model, and also ensures the smoothness and stability of the scheduling process.
[0006] To achieve the above-mentioned technical effects, the technical solution of the present invention is as follows: Firstly, this application proposes an adaptive scheduling method for distribution radio areas, comprising the following steps: S1. Construct a reward function based on the distribution area scheduling cost and distribution area line loss cost; S2. Using the multi-source states of the distribution substation as the state space of the Markov decision process, and the multi-source control quantities of the distribution substation as the action space of the Markov decision process, a transfer function is fitted based on the current state of the multi-source distribution substation in the state space, the action in the action space, and the next state of the multi-source distribution substation; the multi-source states of the distribution substation integrate the states of new energy, distribution substation load, and energy storage. S3. Construct a Markov decision process model based on the state space, action space, transition function, and reward function; S4. Improve the deep reinforcement learning algorithm based on the Markov decision process model, train the adaptive scheduling strategy, and obtain the trained adaptive scheduling strategy. S5. Input the real-time multi-source status data of the distribution radio area into the trained adaptive scheduling strategy to obtain the current distribution radio area scheduling decision.
[0007] In this technical solution, a reward function is first constructed based on cost. A Markov decision process model is constructed using the multi-source state integrating new energy, load, and energy storage data as the state space and scheduling instructions as the action space, combined with the reward function. An improved deep reinforcement learning algorithm is then used to iteratively train the agent. During training, state normalization and feature expansion enhance the agent's perception of environmental randomness, and a dual Q-network architecture and time-series correlated noise mechanism optimize the learning of scheduling strategies. The trained distribution area scheduling agent outputs the optimal distribution area scheduling instructions in real time based on the state input, achieving adaptive scheduling of distribution areas. The use of the improved deep reinforcement learning algorithm overcomes the impact of random fluctuations in source and load, improves the global optimization capability of the scheduling strategy, and enhances the stability and real-time response capability of the power grid operation.
[0008] Preferably, the construction of the reward function The process is as follows: Construct a distribution area scheduling cost reward function The expression is:
[0009] in, This represents the dispatch cost of wind power generation. This represents the dispatch cost of photovoltaic power generation. Indicates load dispatching cost, Indicates the cost of energy storage dispatch; Construct a reward function for distribution transformer area line loss costs The expression is:
[0010] in, Indicates the total number of branches. Indicates the conductance of the branch circuit. Indicates the starting node of the branch. voltage, Indicates the end node of the branch. voltage, Indicates the starting node of the branch. voltage phase angle, Indicates the end node of the branch. The voltage phase angle; Based on the distribution area scheduling cost reward function and distribution area line network loss cost reward function Construct a reward function The expression is: .
[0011] Preferably, the multi-source status of the distribution substation includes: wind power generation status, photovoltaic power generation status, load status, energy storage status, and environmental status; The multi-source control quantities of the distribution substation include: wind power output control quantity, photovoltaic power output control quantity, load control quantity, and energy storage charging and discharging power; The transfer function is fitted using a pre-defined long short-term memory network.
[0012] Preferably, when improving the deep reinforcement learning algorithm based on Markov decision process, the wind power generation state, photovoltaic power generation state, load state, energy storage state and environmental state in the state space are normalized.
[0013] Preferably, external environmental prediction variables are introduced to expand the state space, including wind speed, irradiance, temperature, and load prediction data.
[0014] Preferably, the improved deep reinforcement learning algorithm employs a dual Q-network architecture and utilizes improved temporally correlated noise for action space exploration.
[0015] Preferably, the adaptive scheduling strategy is based on a deep reinforcement learning network framework, which includes a target action network and a target evaluation network. The process of training the adaptive scheduling strategy is as follows: S41. Initialize the action network, evaluation network, and their respective target action network and target evaluation network. The action network is used to generate an adaptive scheduling strategy, and the evaluation network is used to evaluate the quality of the adaptive strategy. Initialize the experience replay pool and set relevant parameters. S42. At each time step Obtain the current state in the state space of the Markov decision process model. Based on the current state, scheduling actions are generated through an action network and improved time-dependent noise. ; S43. Schedule the action The input is fed into a Markov decision process model, a scheduling action is executed, and the next state is obtained based on the transition function. and instant rewards ; S44. Change the status Scheduling actions Next state and instant rewards As experience samples, they are stored in the experience replay pool. The evaluation network and action network are updated by sampling from the experience replay pool according to the priority of temporal difference error. S45. Iterate through S42~S44 until the Q-value of the state-action value function of the reinforcement learning algorithm converges, and obtain the target action network and the target evaluation network as the trained adaptive scheduling strategy.
[0016] Preferably, a timing smoothing term is introduced into the improved timing-dependent noise, making the scheduling action... Satisfying the expression:
[0017]
[0018] in, Represents a time-series index. Represents the policy function. Indicates timing smoothing noise, Indicates the noise smoothing coefficient. This represents Gaussian noise.
[0019] Preferably, the two Q-network architectures in the dual Q-network architecture have different structures and parameters. During iterative execution S42~S44, the output of the network with the lower Q-value is used to obtain the target value in the training iteration, expressed as:
[0020] in, Represents the discount factor. Indicates the Q network index. This represents the Q-network function value; Update and evaluate network parameters based on minimizing the mean squared error loss function. The expression satisfies:
[0021] in, Indicates the number of samples; Updating Action Network Parameters Based on Gradient Ascent Method The gradient of the gradient ascent method Satisfying the expression:
[0022] in, Action networks are based on The output action.
[0023] Secondly, this application also proposes a distribution area adaptive scheduling system, the system comprising: The reward function construction unit is used to construct a reward function based on the distribution area scheduling cost and the distribution area line loss cost. The Markov decision process model pre-construction unit is used to take the multi-source state of the distribution substation as the state space of the Markov decision process and the multi-source control quantity of the distribution substation as the action space of the Markov decision process. Based on the current state of the multi-source distribution substation in the state space, the action in the action space, and the next state of the multi-source distribution substation, a transfer function is fitted. The multi-source state of the distribution substation integrates the states of new energy, distribution substation load, and energy storage. The Markov decision process model building unit is used to construct Markov decision process models based on state space, action space, transition function, and reward function. The distribution area scheduling agent training unit is used to improve the deep reinforcement learning algorithm based on the Markov decision process model, train the adaptive scheduling strategy, and obtain the trained adaptive scheduling strategy. The distribution area scheduling unit is used to input the real-time multi-source status data of the distribution area into the trained adaptive scheduling strategy to obtain the current distribution area scheduling decision.
[0024] Compared with the prior art, the beneficial effects of the present invention are: This invention proposes an adaptive dispatching method and system for distribution substations. First, a reward function is constructed based on cost. The state space is defined by integrating multi-source states from renewable energy sources, loads, and energy storage data, and the action space is defined by dispatching instructions. A Markov decision process model is then constructed using the reward function. An improved deep reinforcement learning algorithm is used to iteratively train the agent. During training, state normalization and feature expansion enhance the agent's perception of environmental randomness. A dual-Q network architecture and a time-dependent noise mechanism are used to optimize the learning of the dispatching strategy. The trained distribution substation dispatching agent outputs the optimal dispatching instructions for the distribution substation in real time based on the state input, achieving adaptive dispatching of the distribution substation. The improved deep reinforcement learning algorithm overcomes the impact of random fluctuations in source and load while improving the global optimization capability of the dispatching strategy, thus enhancing the stability and real-time response capability of the power grid operation. Attached Figure Description
[0025] Figure 1 A flowchart illustrating the adaptive scheduling method for distribution radio stations proposed in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram illustrating the training process of the adaptive scheduling strategy proposed in Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of the structure of the distribution radio area adaptive scheduling system proposed in Embodiment 3 of the present invention. Detailed Implementation
[0026] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent. To better illustrate this embodiment, some parts of the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions; It is understandable to those skilled in the art that some well-known details may be omitted from the accompanying drawings.
[0027] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.
[0028] The positional relationships depicted in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent. Example 1 This embodiment proposes an adaptive scheduling method for distribution radio areas. A flowchart illustrating this method can be found here. Figure 1 This includes the following steps: S1. Construct a reward function based on the distribution area scheduling cost and distribution area line loss cost; S2. Using the multi-source states of the distribution substation as the state space of the Markov decision process, and the multi-source control quantities of the distribution substation as the action space of the Markov decision process, a transfer function is fitted based on the current state of the multi-source distribution substation in the state space, the action in the action space, and the next state of the multi-source distribution substation; the multi-source states of the distribution substation integrate the states of new energy, distribution substation load, and energy storage. S3. Construct a Markov decision process model based on the state space, action space, transition function, and reward function; S4. Improve the deep reinforcement learning algorithm based on the Markov decision process model, train the adaptive scheduling strategy, and obtain the trained adaptive scheduling strategy. S5. Input the real-time multi-source status data of the distribution radio area into the trained adaptive scheduling strategy to obtain the current distribution radio area scheduling decision.
[0029] In this embodiment, a reward function is first constructed based on cost. A Markov decision process model is constructed using the multi-source state integrating new energy, load, and energy storage data as the state space and scheduling instructions as the action space, combined with the reward function. An improved deep reinforcement learning algorithm is then used to iteratively train the agent. During training, state normalization and feature expansion enhance the agent's perception of environmental randomness, and a dual Q-network architecture and time-dependent noise mechanism optimize the learning of the scheduling strategy. The trained distribution area scheduling agent outputs the optimal distribution area scheduling instructions in real time based on the state input, achieving adaptive scheduling of distribution areas. The improved deep reinforcement learning algorithm overcomes the impact of random fluctuations in source and load while improving the global optimization capability of the scheduling strategy, thus enhancing the stability and real-time response capability of the power grid operation.
[0030] Example 2 In this embodiment, the construction of the reward function The process is as follows: Construct a distribution area scheduling cost reward function The expression is:
[0031] in, This represents the dispatch cost of wind power generation. This represents the dispatch cost of photovoltaic power generation. Indicates load dispatching cost, Indicates the cost of energy storage dispatch; Construct a reward function for distribution transformer area line loss costs The expression is:
[0032] in, Indicates the total number of branches. Indicates the conductance of the branch circuit. Indicates the starting node of the branch. voltage, Indicates the end node of the branch. voltage, Indicates the starting node of the branch. voltage phase angle, Indicates the end node of the branch. The voltage phase angle; Based on the distribution area scheduling cost reward function and distribution area line network loss cost reward function Construct a reward function The expression is: .
[0033] Specifically, the dispatch cost of wind power includes the operation and maintenance costs of wind power generation as well as the penalty costs incurred due to wind curtailment. Intelligent agents aim to reduce the curtailment rate and thus decrease this cost by adjusting wind power output control commands. The dispatch cost of photovoltaic (PV) power generation includes the operation and maintenance costs of PV power generation as well as the penalty costs for solar curtailment. Similar to wind power, reducing solar curtailment can lower this cost. Load dispatch costs include the compensation costs or electricity purchase costs incurred when peak shaving or power interruption for controllable loads (such as flexible loads) during demand-side response. Energy storage dispatch costs mainly include the lifespan depreciation costs of energy storage batteries during charging and discharging (related to the depth and number of charge / discharge cycles) and the operation and maintenance costs of charging and discharging.
[0034] In this embodiment, the multi-source status of the distribution area includes: wind power generation status, photovoltaic power generation status, load status, energy storage status, and environmental status; The multi-source control quantities of the distribution substation include: wind power output control quantity, photovoltaic power output control quantity, load control quantity, and energy storage charging and discharging power; The transfer function is fitted using a pre-defined long short-term memory network.
[0035] Specifically, the wind power generation status includes the current wind speed data, the real-time output power of the wind turbine, and the wind turbine's on / off status; the photovoltaic power generation status includes the current solar irradiance, ambient temperature, photovoltaic panel backsheet temperature, and the real-time output power of the photovoltaic inverter; the load status includes the current total active load demand, reactive load demand, and proportion of important loads in the distribution area; the energy storage status includes the current state of charge (SOC) of the energy storage battery, the number of charge / discharge cycles, and the maximum charge / discharge power limit; and the environmental status includes the current timestamp information, used to characterize the periodic characteristics of the load and photovoltaics, such as the number of hours and seasonal information, as well as historical power time-series data of wind-solar-load.
[0036] Specifically, the wind power output control instructions and photovoltaic power output control instructions include: the reduction amount of active power (i.e., the amount of wind / solar curtailment) or the amount of reactive power support; the load control instructions include: the amount of power reduction or power transfer of flexible loads; the energy storage control instructions include: the set value of the charging and discharging power of energy storage batteries.
[0037] Specifically, when constructing the transfer function of the Markov decision process model, a pre-defined long short-term memory network is also used for data-driven modeling. The specific steps include: The historical state sequence of the distribution area is used as the input of the long short-term memory network to complete the construction of the input layer of the long short-term memory network. The input vector in the input layer of the long short-term memory network contains wind speed, irradiance, temperature, load power and historical action sequence of several past time steps. The input sequence is processed using forget gates, input gates, and output gates within the Long Short-Term Memory (LSTM) network units. The forget gates determine which outdated environmental information (such as sudden weather changes from a few hours ago) is discarded from the cell state. The input gates determine which new wind and solar fluctuation information is updated to the cell state at the current moment. Through this gating mechanism, the network can capture long-term periodicity (such as day-night cycle) and short-term fluctuation (such as cloud cover) in wind, solar, and load data. Construct a fully connected layer and an output layer for a Long Short-Term Memory (LSTM) network. The output layer of the LSM network is mapped through the fully connected layer to output the predicted environmental state for the next time step (i.e., predicted wind speed, irradiance, and base load). By combining the environmental prediction state output by the Long Short-Term Memory network with the scheduling actions of the agent (such as energy storage charging and discharging), the complete system state at the next moment is calculated together through the physical constraint equations of the distribution radio station.
[0038] In this embodiment, when improving the deep reinforcement learning algorithm based on Markov decision process, the wind power generation state, photovoltaic power generation state, load state, energy storage state and environmental state in the state space are normalized.
[0039] Specifically, due to the inconsistent numerical ranges of multi-source state data (wind, solar, load, and storage) in distribution substations—for example, power values might be hundreds of kilowatts, while the state of charge (SOC) of energy storage is only a decimal between 0 and 1—direct input can lead to significant scale differences that negatively impact the gradient updates of neural networks. Therefore, it is necessary to use the min-max normalization method to normalize the states, bringing them to a more uniform level. Within the interval, the difference in units is eliminated to prevent large numerical features from masking small numerical features, thereby ensuring the training effect.
[0040] In this embodiment, external environmental prediction variables are introduced to expand the state space. These external environmental prediction variables include wind speed, irradiance, temperature, and load prediction data.
[0041] In this embodiment, the improved deep reinforcement learning algorithm adopts a dual Q-network architecture and utilizes improved temporally correlated noise for action space exploration.
[0042] In this embodiment, the adaptive scheduling strategy is based on a deep reinforcement learning network framework, which includes a target action network and a target evaluation network. The process of training the adaptive scheduling strategy is as follows: S41. Initialize the action network, evaluation network, and their respective target action network and target evaluation network. The action network is used to generate an adaptive scheduling strategy, and the evaluation network is used to evaluate the quality of the adaptive strategy. Initialize the experience replay pool and set relevant parameters. S42. At each time step Obtain the current state in the state space of the Markov decision process model. Based on the current state, scheduling actions are generated through an action network and improved time-dependent noise. ; S43. Schedule the action The input is fed into a Markov decision process model, a scheduling action is executed, and the next state is obtained based on the transition function. and instant rewards ; S44. Change the status Scheduling actions Next state and instant rewards As experience samples, they are stored in the experience replay pool. The evaluation network and action network are updated by sampling from the experience replay pool according to the priority of temporal difference error. S45. Iterate through S42~S44 until the Q-value of the state-action value function of the reinforcement learning algorithm converges, and obtain the target action network and the target evaluation network as the trained adaptive scheduling strategy.
[0043] In this embodiment, a timing smoothing term is introduced into the improved timing-dependent noise, making the scheduling action... Satisfying the expression:
[0044]
[0045] in, Represents a time-series index. Represents the policy function. Indicates timing smoothing noise, Indicates the noise smoothing coefficient. This represents Gaussian noise.
[0046] Specifically, the introduction of a noise smoothing coefficient in the motion generation stage The time-dependent noise formula makes the generated scheduling exploration actions autocorrelated in the time dimension. The technical effects of this design are twofold: firstly, it enables smooth transitions in scheduling instructions (such as energy storage charging and discharging) during the exploration process, avoiding equipment losses and grid voltage fluctuations caused by drastic action jumps, thus improving the physical feasibility of the scheduling scheme; secondly, it guides the agent to conduct strategy exploration with time inertia, which is more in line with the continuous dynamic characteristics of the power system, thereby significantly improving the convergence speed and optimization efficiency of deep reinforcement learning algorithms in complex continuous action spaces.
[0047] In this embodiment, the two Q-network architectures in the dual Q-network architecture have different structures and parameters. During iterative execution S42~S44, the output of the network with the lower Q-value is used to obtain the target value in the training iteration, expressed as:
[0048] in, Represents the discount factor. Indicates the Q network index. This represents the Q-network function value; Update and evaluate network parameters based on minimizing the mean squared error loss function. The expression satisfies:
[0049] in, Indicates the number of samples; Updating Action Network Parameters Based on Gradient Ascent Method The gradient of the gradient ascent method Satisfying the expression:
[0050] in, Action networks are based on The output action.
[0051] Specifically, the gradient is obtained. Then, it also includes according to the learning rate. The action that updates the action network output is expressed as:
[0052] in, This indicates the updated action network output action.
[0053] Specifically, during the training process of the evaluation network, a dual Q-network architecture combined with a minimum target calculation strategy was creatively applied. By taking the smaller value of the outputs of two independent evaluation networks as the training target, the error of overestimating Q-values, which is common in deep reinforcement learning, was effectively eliminated. In the scenario of distribution area scheduling, this mechanism can prevent the agent from choosing high-risk scheduling actions due to environmental noise or initial errors, significantly enhance the algorithm's resistance to random fluctuations in wind and solar loads, and ensure the safety of the scheduling strategy and the stability of algorithm convergence.
[0054] Specifically, a deterministic policy gradient algorithm was employed in the training of the action network. Through the chain rule, the gradient of the evaluation network's assessment of the action's value was passed back to the action network, thus providing clear directional guidance for updating the action network parameters. The beneficial effects of this technical design are: it overcomes the limitations of traditional methods in handling continuous action spaces, achieving high-precision continuous control of the energy storage power and load regulation of distribution substations; simultaneously, the gradient-based directional update mechanism avoids blind searching in high-dimensional spaces, significantly improving the convergence speed and optimization accuracy of the agent in complex and variable source-load environments.
[0055] Example 3 This embodiment proposes an adaptive scheduling system for distribution radio areas. In this embodiment, the system is used to implement an adaptive scheduling method for distribution radio areas. The structural diagram is shown below. Figure 3 As shown, it includes: The reward function construction unit is used to construct a reward function based on the distribution area scheduling cost and the distribution area line loss cost. The Markov decision process model pre-construction unit is used to take the multi-source state of the distribution substation as the state space of the Markov decision process and the multi-source control quantity of the distribution substation as the action space of the Markov decision process. Based on the current state of the multi-source distribution substation in the state space, the action in the action space, and the next state of the multi-source distribution substation, a transfer function is fitted. The multi-source state of the distribution substation integrates the states of new energy, distribution substation load, and energy storage. The Markov decision process model building unit is used to construct Markov decision process models based on state space, action space, transition function, and reward function. The distribution area scheduling agent training unit is used to improve the deep reinforcement learning algorithm based on the Markov decision process model, train the adaptive scheduling strategy, and obtain the trained adaptive scheduling strategy. The distribution area scheduling unit is used to input the real-time multi-source status data of the distribution area into the trained adaptive scheduling strategy to obtain the current distribution area scheduling decision.
[0056] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.
Claims
1. An adaptive scheduling method for distribution radio areas, characterized in that, Includes the following steps: S1. Construct a reward function based on the distribution area scheduling cost and distribution area line loss cost; S2. Using the multi-source states of the distribution substation as the state space of the Markov decision process, and the multi-source control quantities of the distribution substation as the action space of the Markov decision process, a transfer function is fitted based on the current state of the multi-source distribution substation in the state space, the action in the action space, and the next state of the multi-source distribution substation; the multi-source states of the distribution substation integrate the states of new energy, distribution substation load, and energy storage. S3. Construct a Markov decision process model based on the state space, action space, transition function, and reward function; S4. Improve the deep reinforcement learning algorithm based on the Markov decision process model, train the adaptive scheduling strategy, and obtain the trained adaptive scheduling strategy. S5. Input the real-time multi-source status data of the distribution radio area into the trained adaptive scheduling strategy to obtain the current distribution radio area scheduling decision.
2. The adaptive scheduling method for distribution radio areas according to claim 1, characterized in that, The construction of the reward function The process is as follows: Construct a distribution area scheduling cost reward function The expression is: in, This represents the dispatch cost of wind power generation. This represents the dispatch cost of photovoltaic power generation. Indicates load dispatching cost, Indicates the cost of energy storage dispatch; Construct a reward function for distribution transformer area line loss costs The expression is: in, Indicates the total number of branches. Indicates the conductance of the branch circuit. Indicates the starting node of the branch. voltage, Indicates the end node of the branch. voltage, Indicates the starting node of the branch. voltage phase angle, Indicates the end node of the branch. The voltage phase angle; Based on the distribution area scheduling cost reward function and distribution area line network loss cost reward function Construct a reward function The expression is: 。 3. The adaptive scheduling method for distribution radio areas according to claim 2, characterized in that, The multi-source status of the distribution area includes: wind power generation status, photovoltaic power generation status, load status, energy storage status, and environmental status; The multi-source control quantities of the distribution substation include: wind power output control quantity, photovoltaic power output control quantity, load control quantity, and energy storage charging and discharging power; The transfer function is fitted using a pre-defined long short-term memory network.
4. The adaptive scheduling method for distribution radio areas according to claim 3, characterized in that, When improving the deep reinforcement learning algorithm based on Markov decision process, the wind power generation state, photovoltaic power generation state, load state, energy storage state and environmental state in the state space are normalized.
5. The adaptive scheduling method for distribution radio areas according to claim 4, characterized in that, The state space is expanded by introducing external environmental predictive variables, which include wind speed, irradiance, temperature, and load prediction data.
6. The adaptive scheduling method for distribution radio areas according to claim 4 or 5, characterized in that, The improved deep reinforcement learning algorithm employs a dual Q-network architecture and utilizes improved temporally correlated noise for action space exploration.
7. The adaptive scheduling method for distribution radio areas according to claim 6, characterized in that, The adaptive scheduling strategy is based on a deep reinforcement learning network framework, which includes a target action network and a target evaluation network. The process of training the adaptive scheduling strategy is as follows: S41. Initialize the action network, evaluation network, and their respective target action network and target evaluation network. The action network is used to generate an adaptive scheduling strategy, and the evaluation network is used to evaluate the quality of the adaptive strategy. Initialize the experience replay pool and set relevant parameters. S42. At each time step Obtain the current state in the state space of the Markov decision process model. Based on the current state, scheduling actions are generated through an action network and improved time-dependent noise. ; S43. Schedule the action The input is fed into a Markov decision process model, a scheduling action is executed, and the next state is obtained based on the transition function. and instant rewards ; S44. Change the status Scheduling actions Next state and instant rewards As experience samples, they are stored in the experience replay pool. The evaluation network and action network are updated by sampling from the experience replay pool according to the priority of temporal difference error. S45. Iterate through S42~S44 until the Q-value of the state-action value function of the reinforcement learning algorithm converges, and obtain the target action network and the target evaluation network as the trained adaptive scheduling strategy.
8. The adaptive scheduling method for distribution radio areas according to claim 7, characterized in that, The improved time-dependent noise incorporates a time-smoothing term, which improves scheduling actions. Satisfying the expression: in, Represents a time-series index. Represents the policy function. Indicates timing smoothing noise, Indicates the noise smoothing coefficient. This represents Gaussian noise.
9. The adaptive scheduling method for distribution radio areas according to claim 7, characterized in that, The dual Q-network architecture used has two different Q-network structures and parameters. During iterations S42 to S44, the output of the network with the lower Q-value is used to obtain the target value in the training iteration, expressed as: in, Represents the discount factor. Indicates the Q network index. This represents the Q-network function value; Update and evaluate network parameters based on minimizing the mean squared error loss function. The expression satisfies: in, Indicates the number of samples; Updating Action Network Parameters Based on Gradient Ascent Method The gradient of the gradient ascent method Satisfying the expression: in, Action networks are based on The output action.
10. A distribution area adaptive scheduling system, characterized in that, include: The reward function construction unit is used to construct a reward function based on the distribution area scheduling cost and the distribution area line loss cost. The Markov decision process model pre-construction unit is used to take the multi-source state of the distribution substation as the state space of the Markov decision process and the multi-source control quantity of the distribution substation as the action space of the Markov decision process. Based on the current state of the multi-source distribution substation in the state space, the action in the action space, and the next state of the multi-source distribution substation, a transfer function is fitted. The multi-source state of the distribution substation integrates the states of new energy, distribution substation load, and energy storage. The Markov decision process model building unit is used to construct Markov decision process models based on state space, action space, transition function, and reward function. The distribution area scheduling agent training unit is used to improve the deep reinforcement learning algorithm based on the Markov decision process model, train the adaptive scheduling strategy, and obtain the trained adaptive scheduling strategy. The distribution area scheduling unit is used to input the real-time multi-source status data of the distribution area into the trained adaptive scheduling strategy to obtain the current distribution area scheduling decision.