A method and system for dispatching a power grid by clusters

Abstract

The application discloses a kind of cluster power grid dispatching method and system, belong to power grid dispatching technical field, including: based on cluster benefit acquisition cluster local target, according to local target and the historical data of cluster the local training of corresponding agent of cluster is carried out process agent;Based on the overall benefit of power grid acquisition global target, using partially observable Markov decision process to obtain the final contribution of local target to global target;According to global target, obtain global reward function, according to final contribution and global reward function, the global training of process agent is carried out to obtain optimized agent;Through PPO algorithm, construct global scheduling strategy update rule, under the constraint of global scheduling strategy update rule, obtain global scheduling strategy through optimized agent, through global scheduling strategy, carry out dispatching to power grid.The technical problem that present technical difficulty to consider local and global dispatching demand is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power grid dispatching, in particular to a method and system for dispatching a power grid in clusters. BACKGROUND

[0002] With the large-scale grid connection of distributed renewable energy, energy storage devices and flexible loads, the county-level power grid presents the characteristics of strong source-load fluctuation, high resource heterogeneity and diversified edge control. The traditional centralized power grid dispatching architecture gradually exposes bottlenecks such as slow response and long control path in the face of uneven temporal and spatial distribution of resources, increasing real-time requirements and enhanced autonomous control capabilities. To break through the above bottlenecks, the existing technology introduces reinforcement learning technology to configure an independent agent for each cluster in the power grid, and generates an independent local dispatching strategy through the agent to dispatch the corresponding cluster. However, since the agent can only obtain and process local observation data within its jurisdiction, it lacks the ability to perceive the overall operating state of the power grid, which may lead to conflicts between the local decisions of each agent and the overall decisions of the power grid. Therefore, it is difficult to balance the local and global dispatching requirements by configuring an independent agent for each cluster. SUMMARY

[0003] In view of the technical problem that the existing technology is difficult to balance the local and global dispatching requirements, the present application provides a method and system for dispatching a power grid in clusters, which combines the contribution mapping of a partially observable Markov decision process with the safety constraints of a proximal policy optimization algorithm to construct a learning mechanism that aligns with the global target under local information and ensures the safety of the dispatching strategy, ultimately achieving the overall improvement of cluster benefits and the safe and stable operation of the power grid. The technical problem that the existing technology is difficult to balance the local and global dispatching requirements is solved.

[0004] To solve the above technical problem, the present application provides a method for dispatching a power grid in clusters, comprising the following steps:

[0005] Obtain the local target of the cluster based on the cluster benefits, and locally train the agent corresponding to the cluster based on the local target and the historical data of the cluster to obtain a process agent;

[0006] Obtain the global target based on the comprehensive benefits of the power grid, and use a partially observable Markov decision process to obtain the final contribution of the local target to the global target;

[0007] Obtain a global reward function based on the global target, and globally train the process agent based on the final contribution and the global reward function to obtain an optimized agent;

[0008] Construct a global dispatching strategy update rule through a PPO algorithm, obtain a global dispatching strategy through the optimized agent under the constraint of the global dispatching strategy update rule, and dispatch the power grid through the global dispatching strategy.

[0009] The application has the following advantages:

[0010] Considering that the agent can only obtain and process local observation data within its jurisdiction, lacks the perception ability of the overall operation state of the power grid, and the partially observable Markov decision process has the partially observable characteristic, the local target of the cluster can be accurately mapped to the contribution to the global target of the power grid through the partially observable Markov decision process, thereby breaking through the information barrier that the local agent is difficult to accurately observe the global and the global is difficult to accurately perceive the local influence, the process agent is globally trained through the final contribution and the global reward function, so that the agent can accurately learn the influence of the change of the local target on the global target, thereby aligning the optimization direction of the local target with the optimization direction of the global target under the guidance of the global reward function, so as to balance the scheduling requirements of the local and the global. The technical problem that the prior art is difficult to balance the scheduling requirements of the local and the global is solved, thereby improving the overall benefit of the power grid while improving the overall benefit of the power grid;

[0011] The global scheduling strategy update rule is constructed through the PPO algorithm to constrain the acquisition of the global scheduling strategy, so as to prevent the global scheduling strategy from exceeding the safety limit value of the device or the system stability margin, thereby solving the technical problem that the prior art lacks a scheduling strategy update constraint mechanism, and ensuring the safe and stable operation of the power grid;

[0012] Through the combination of the partially observable Markov decision process and the PPO algorithm, the safe and stable operation of the power grid is ensured, and the scheduling efficiency of the cluster and the overall operation benefit of the power grid are improved.

[0013] Preferably, the process agent is obtained by locally training the agent corresponding to the cluster according to the local target and the historical data of the cluster, and the process agent comprises the following steps:

[0014] The agent corresponding to the cluster is obtained according to reinforcement learning, and the local reward function is obtained according to the local target;

[0015] The next state of the cluster in the historical data of the cluster is input into the local reward function to obtain a first reward value, and the local training data set is obtained according to the current state of the cluster in the historical data of the cluster, the historical local scheduling strategy, the next state of the cluster and the first reward value.

[0016] The local value function of the agent is trained through the local training data set to obtain an updated agent, the current state of the cluster is input into the updated agent to obtain a second reward value, if the second reward value meets a preset condition, the updated agent is taken as the process agent, otherwise, the local value function of the updated agent is trained according to the local training data set to obtain the process agent.

[0017] In the scheme, the local value function of the agent is trained through the current state of the cluster, the historical local scheduling strategy, the next state of the cluster and the first reward value in the historical data of the cluster, so that the agent can learn the rule that "after taking a specific local scheduling strategy in a certain current state, what kind of next state change will be triggered and the corresponding local reward", and according to the learned rule, the agent can accurately obtain the local reward, thereby autonomously selecting the local scheduling strategy that can maximize the local reward, so that the agent has the ability to independently generate a local scheduling strategy that can optimize the local target, thereby improving the adaptability of the local scheduling strategy to the cluster benefit.

[0018] Preferably, the global target is obtained based on the comprehensive benefit of the power grid, and the comprehensive benefit of the power grid is obtained according to the energy consumption efficiency of the power grid, the voltage deviation penalty and the regulation and control cost.

[0019] The global target is obtained based on the comprehensive benefit of the power grid, and the comprehensive benefit of the power grid is obtained according to the energy consumption efficiency of the power grid, the voltage deviation penalty and the regulation and control cost.

[0020] Preferably, the final contribution of the local target to the global target is obtained using a partially observable Markov decision process, and the initial contribution of the local target to the global target is obtained according to historical experience in the historical data of the power grid.

[0021] The initial contribution of the local target to the global target is obtained according to historical experience in the historical data of the power grid, and the initial achievement probability of the local state to the local target is obtained according to the historical experience.

[0022] Any historical global scheduling strategy in the historical data of the power grid is executed to obtain an observation result, and the initial achievement probability is corrected using the Bayesian rule based on the observation result to obtain a first corrected achievement probability.

[0023] The initial contribution is corrected according to the first corrected achievement probability to obtain a corrected contribution, if the corrected contribution meets the preset requirement, the corrected contribution is taken as the final contribution, otherwise, the achievement probability difference and the achievement probability correction direction are obtained according to the initial achievement probability and the first corrected achievement probability, the contribution difference and the contribution correction direction are obtained according to the initial contribution and the corrected contribution, the ratio of the achievement probability difference and the contribution difference is obtained, the achievement probability correction value is obtained according to the preset requirement and the ratio, the process achievement probability is obtained according to the achievement probability correction value, the first corrected achievement probability and the correlation coefficient representing the correlation between the achievement probability correction direction and the contribution correction direction correction trend, the first corrected achievement probability is taken as the initial achievement probability, the initial achievement probability is corrected using the Bayesian rule to obtain a second corrected achievement probability, the average achievement probability is obtained according to the second corrected achievement probability and the process achievement probability, and the final contribution is obtained according to the average achievement probability.

[0024] In the scheme, the local state is a key scene variable that affects the contribution of the local target to the global target. By obtaining the initial achievement probability of the local state, the contribution is linked to the possibility of the cluster state. The initial achievement probability is corrected using Bayesian rule to obtain the corrected achievement probability according to the observation result. The initial contribution is corrected by the corrected achievement probability of the fitting cluster scene to obtain the corrected contribution, which can make the corrected contribution deeply match the power grid scene. On this basis, when the corrected contribution meets the preset requirement, it means that the corrected contribution accurately reflects the real contribution of the local target to the global target under the power grid scene, thereby improving the scene adaptability of the final contribution. When the corrected contribution does not meet the preset requirement, the average achievement probability of the process achievement probability and the second corrected achievement probability is obtained, and the final contribution is obtained according to the average achievement probability, thereby considering the internal linkage between the achievement probability and the contribution, thereby making up for the lack of association of the Bayesian rule and improving the acquisition efficiency of the final contribution. By accurately mapping the local target of the cluster to the contribution to the global target of the power grid, the information barrier that the local intelligent agent is difficult to accurately observe the global and the global is difficult to accurately perceive the local impact is broken through.

[0025] Preferably, the global training of the process intelligent agent according to the final contribution and the global reward function to obtain the optimized intelligent agent, comprising:

[0026] According to the final contribution and the local target value, a global target value is obtained, the global target value is input into the global reward function to obtain a third reward value, and the global training data set is obtained according to the third reward value and the local target value.

[0027] The global value function of the process intelligent agent is trained by the global training data set to obtain the optimized intelligent agent.

[0028] In the scheme, the global training data set is obtained according to the third reward value and the local target value, and the global value function of the process intelligent agent is trained, so that the process intelligent agent can learn the rule that "after taking a certain local target value, the global reward obtained", according to the learned rule, the process intelligent agent can accurately obtain the global reward, thereby autonomously selecting the global scheduling strategy that can maximize the global reward, so that the process intelligent agent can obtain the global scheduling strategy in the subsequent direction that can maximize the global reward, thereby improving the adaptability of the global scheduling strategy to the comprehensive benefit of the power grid.

[0029] Preferably, the global scheduling strategy update rule is constructed by the PPO algorithm, comprising:

[0030] The preset tolerance is taken as a shear constraint, the global scheduling switching rate is sheared according to a shear function of the PPO algorithm to obtain a sheared scheduling switching rate under the shear constraint, the smaller value between the global scheduling switching rate and the sheared scheduling switching rate is taken as a scheduling strategy optimization function by using the power grid comprehensive benefit, and the global scheduling strategy updating rule is obtained according to the scheduling strategy optimization function.

[0031] Preferably, the global scheduling strategy updating rule obtained according to the scheduling strategy optimization function comprises:

[0032] The first scheduling strategy constraint is obtained according to the scheduling strategy optimization function, the power grid state change constraint is obtained according to the power grid safety requirement, the second scheduling strategy constraint is obtained according to the power grid state change constraint, and the global scheduling strategy updating rule is obtained according to the first scheduling strategy constraint and the second scheduling strategy constraint.

[0033] Preferably, the global scheduling strategy is obtained by optimizing the agent under the constraint of the global scheduling strategy updating rule, and the global scheduling strategy updating rule comprises:

[0034] The candidate local scheduling strategy of the cluster is obtained by optimizing the local value function of the agent, the candidate global scheduling strategy of the power grid is obtained according to the global value function of the agent under the constraint of the global scheduling strategy updating rule, the candidate local target value corresponding to the candidate local scheduling strategy of the cluster is compared with the desired local target value corresponding to the candidate global scheduling strategy of the power grid, if the comparison is successful, the candidate local scheduling strategy of the comparison is taken as the global scheduling strategy, otherwise, the local value function or the global value function of the agent is modified according to the difference between the candidate local target value and the desired local target value, so as to obtain the global scheduling strategy.

[0035] In the scheme, the candidate global scheduling strategy specifically refers to the desired local target value in order to achieve the global target. The candidate local target value corresponding to the candidate local scheduling strategy is compared with the desired local target value, and the candidate local scheduling strategy of the comparison is taken as the global scheduling strategy, which not only takes into account the scheduling requirements of the local and global, but also guarantees the scheduling efficiency of the cluster.

[0036] Preferably, the method further comprises:

[0037] The feedback result of the power grid scheduled according to the global scheduling strategy is obtained, it is judged whether the feedback result meets the trigger condition, if yes, the optimal agent is obtained by updating the agent according to the feedback result, the optimal global scheduling strategy is obtained by using the optimal agent, and the power grid is scheduled by using the optimal global scheduling strategy.

[0038] The application further provides a power grid scheduling system for clusters, which is suitable for the power grid scheduling method for clusters and comprises:

[0039] a local training module configured to obtain a local target of the cluster based on a cluster benefit, and to perform local training on the agent corresponding to the cluster based on the local target and historical data of the cluster to obtain a process agent;

[0040] a final contribution obtaining module configured to obtain a global target based on a power grid comprehensive benefit, and to obtain a final contribution of the local target to the global target using a partially observable Markov decision process;

[0041] a global training module configured to obtain a global reward function based on the global target, and to perform global training on the process agent based on the final contribution and the global reward function to obtain an optimized agent;

[0042] a scheduling module configured to construct a global scheduling strategy update rule by using a PPO algorithm, to obtain a global scheduling strategy by using the optimized agent under the constraint of the global scheduling strategy update rule, and to schedule the power grid by using the global scheduling strategy.

[0043] By using the above technical solution, the present application has the following advantages:

[0044] Considering that the agent can only obtain and process local observation data within its jurisdiction and lacks the ability to perceive the overall operation state of the power grid, and the partially observable Markov decision process has the partially observable characteristic, the local target of the cluster can be accurately mapped to the contribution to the global target of the power grid by using the partially observable Markov decision process, thereby breaking through the information barrier that the local agent is difficult to accurately observe the global and the global is difficult to accurately perceive the local influence, the process agent is globally trained by using the final contribution and the global reward function, the agent can accurately learn the influence of the change of the local target on the global target, thereby aligning the optimization direction of the local target with the optimization direction of the global target under the guidance of the global reward function, and the scheduling demands of the local and the global are taken into account, thereby solving the technical problem that the prior art is difficult to take into account the scheduling demands of the local and the global, improving the cluster benefit, and improving the comprehensive benefit of the power grid;

[0045] Specifically, the candidate local scheduling strategy of the cluster is obtained by using the local value function of the optimized agent, the candidate global scheduling strategy of the power grid is obtained based on the global value function of the optimized agent, the candidate global scheduling strategy specifically refers to the desired local target value in order to achieve the global target, the candidate local target value corresponding to the candidate local scheduling strategy is compared with the desired local target value, the candidate local scheduling strategy that passes the comparison is taken as the global scheduling strategy, the scheduling demands of the local and the global are taken into account, and the scheduling efficiency of the cluster is ensured;

[0046] The global scheduling strategy update rule is constructed by the PPO algorithm to constrain the acquisition of the global scheduling strategy, so as to prevent the global scheduling strategy from exceeding the device safety limit or the system stability margin, thereby solving the technical problem that the prior art lacks a scheduling strategy update constraint mechanism, and ensuring the safe and stable operation of the power grid.

[0047] By combining the partially observable Markov decision process with the PPO algorithm, the safety and stability of the power grid are ensured, and the scheduling efficiency of the cluster and the overall operation benefit of the power grid are improved. BRIEF DESCRIPTION OF DRAWINGS

[0048] Other features, objects, and advantages of the application will become more apparent from the following detailed description of non-limiting embodiments, read in conjunction with the accompanying drawings. The drawings are only for the purpose of illustrating preferred embodiments and are not to be considered as limiting the application. Throughout the drawings, like referenced numerals are used to designate like parts throughout the several views.

[0049] Figure 1 A flowchart of a sub-cluster power grid scheduling method of the application;

[0050] Figure 2 A flowchart of the generation of the final contribution in a sub-cluster power grid scheduling method of the application. DETAILED DESCRIPTION

[0051] To make the objectives, technical solutions, and advantages of the application clearer, further detailed description will be made of the application in conjunction with the drawings and examples. It should be understood that the specific embodiments described herein are only the best mode of the application, which are used to explain the application and do not limit the protection scope of the application. All other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the application.

[0052] Before discussing the example embodiments in more detail, it should be mentioned that some example embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts depict the operations (or steps) as sequential processes, many of the operations (or steps) can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the operations can be rearranged. The processes can be terminated when their operations are completed, but can also have additional steps not included in the figures; the processes can correspond to methods, functions, routines, subroutines, subprograms, etc.

[0053] Example 1:

[0054] As shown in Figure 1 A sub-cluster power grid scheduling method includes the following steps:

[0055] S1: obtain a local target of the cluster based on a cluster benefit, and perform local training on an agent corresponding to the cluster according to the local target and historical data of the cluster to obtain a process agent.

[0056] Specifically, the local training on the agent corresponding to the cluster according to the local target and the historical data of the cluster to obtain the process agent includes:

[0057] obtaining a local reward function according to the local target and an agent corresponding to the cluster according to reinforcement learning;

[0058] inputting a next state of the cluster in the historical data of the cluster into the local reward function to obtain a first reward value, and obtaining a local training data set according to a current state of the cluster in the historical data of the cluster, a historical local scheduling strategy, the next state of the cluster, and the first reward value;

[0059] training a local value function of the agent through the local training data set to obtain an updated agent, inputting the current state of the cluster into the updated agent to obtain a second reward value, if the second reward value meets a preset condition, taking the updated agent as the process agent, otherwise, training the local value function of the updated agent through the local training data set to obtain the process agent.

[0060] In the embodiment, the local target is specifically a highest scheduling efficiency of the cluster, a highest safety of the cluster, etc. The local reward function is obtained by weighted summation of the local target, wherein the weight of the safety is greater than the weight of the scheduling efficiency. The calculation method of the partial reward in the local reward function can be: if all line load rates are less than or equal to 80% and the absolute value of bus voltage deviation is less than or equal to 5%, the safety reward is 10, and every time a line load rate exceeds 80% or the absolute value of voltage deviation exceeds 5%, 3 points are deducted, and the minimum is -5; if the scheduling instruction execution delay is less than or equal to 30 seconds, the scheduling efficiency reward is 8, and every 10 seconds of delay deducts 2 points, and the minimum is 0. The current state and the next state of the cluster both include active load data, photovoltaic output, energy storage state of charge, node voltage, line load rate, bus voltage, etc. of the cluster.

[0061] The local value function is a predictor of the cumulative return of the agent in the future, and is used to measure the pros and cons of taking a certain local scheduling strategy in a certain local state. The standard for measuring the pros and cons is how much local reward can be obtained from the current local state in the future. The local value function of the agent is trained through the local training data set, so that the agent learns the rule that "after taking a specific local scheduling strategy in a certain current state, what kind of next state change will be triggered, and what reward feedback the change corresponds to". When the learned rule is input into the updated agent to obtain a second reward value greater than or equal to a preset value, it indicates that the second reward value meets the preset condition, otherwise, it indicates that the learning of the agent to the rule is not sufficient or there is a deviation, so the local value function continues to be trained. Through training, the agent can obtain a local scheduling strategy in the subsequent direction to maximize the local reward value, thereby improving the adaptability of the local scheduling strategy to the cluster benefit.

[0062] S2: Obtain a global target based on the comprehensive benefit of the power grid, and use a partially observable Markov decision process to obtain the final contribution of the local target to the global target.

[0063] The global target based on the comprehensive benefit of the power grid includes:

[0064] The global target is obtained based on the comprehensive benefit of the power grid, the energy consumption efficiency of the power grid, the voltage deviation penalty, and the regulation and control cost.

[0065] In this embodiment, the expression of the global target, i.e., the joint target function, is:

[0066] ;

[0067] Wherein, represents the policy network parameter, and the global scheduling strategy is learned by optimizing the policy network parameter, represents the global scheduling strategy, i.e., the total benefit expected to be obtained by the global scheduling action in the entire decision process when the policy network parameter is , represents the expectation operation, represents the global scheduling period, represents the policy probability of the global scheduling strategy, represents the global scheduling strategy, such as the load transfer amount, the energy storage start-stop signal, represents the current state of the power grid, represents the comprehensive benefit of the power grid, i.e., the comprehensive benefit corresponding to the global scheduling strategy, , , , respectively represent the empirical weights of the energy consumption efficiency of the power grid, the voltage deviation penalty, and the regulation cost, which can be dynamically adjusted, represents the local renewable energy consumption amount after scheduling, represents the node voltage fluctuation reduction amount after scheduling, represents the cost paid for scheduling, such as energy storage conversion loss, device start-stop overhead, etc. represents the comprehensive benefit of the power grid in the current state adopting a global scheduling strategy .

[0068] In some embodiments, as shown in Figure 2 the use of the partially observable Markov decision process to obtain the final contribution of the local target to the global target includes:

[0069] S2a: obtaining an initial contribution of the local target to the global target according to historical experience in historical data of the power grid, and obtaining an initial achievement probability of the local state to the local target according to the historical experience;

[0070] S2b: executing any historical global scheduling strategy in the historical data of the power grid to obtain an observation result, and correcting the initial achievement probability to obtain a first corrected achievement probability based on the observation result using the Bayes rule;

[0071] S2c: correcting the initial contribution to obtain a corrected contribution according to the first corrected achievement probability, if the corrected contribution meets a preset requirement, the corrected contribution is taken as the final contribution, otherwise, an achievement probability difference value and an achievement probability correction direction are obtained according to the initial achievement probability and the first corrected achievement probability, a contribution difference value and a contribution correction direction are obtained according to the initial contribution and the corrected contribution, a ratio of the achievement probability difference value and the contribution difference value is obtained, an achievement probability correction value is obtained according to the preset requirement and the ratio, a process achievement probability is obtained according to the achievement probability correction value, the first corrected achievement probability, and a correlation coefficient representing the correlation between the achievement probability correction direction and the contribution correction direction correction trend, the first corrected achievement probability is taken as the initial achievement probability, the initial achievement probability is corrected to obtain a second corrected achievement probability using the Bayes rule, an average achievement probability is obtained according to the second corrected achievement probability and the process achievement probability, and the final contribution is obtained according to the average achievement probability.

[0072] The partially observable Markov decision process is a mathematical framework for describing the sequential decision-making of an intelligent agent in an environment with incomplete state information. In this embodiment, the final contribution of the local target to the global target is obtained through the partially observable Markov decision process, which breaks through the information barrier that the local intelligent agent cannot accurately observe the global and the global cannot accurately perceive the local impact.

[0073] In this embodiment, a certain city core area power grid is taken as an example, which contains two key clusters. The commercial center cluster covers 3 110KV substations, 8 10KV feeders, and serves 5 large commercial complexes with a peak load of 80MW. The residential community cluster covers 2 110KV substations, 12 10KV feeders, and serves 12 residential communities with a peak load of 60MW. The global target is to optimize the overall benefit of the power grid, and the quantitative indicators of the global target are: power supply reliability rate ≥ 99.99%, comprehensive network loss rate ≤ 3.5%, and new energy consumption rate ≥ 92%. The local targets are as follows: the commercial center cluster has the highest dispatching efficiency (load adjustment response delay ≤ 25 seconds) and the highest safety (line load rate ≤ 82%, and the absolute value of bus voltage deviation ≤ 4%); the residential community cluster has the highest dispatching efficiency (energy storage charging and discharging response delay ≤ 30 seconds) and the highest safety (line load rate ≤ 80%, and the absolute value of bus voltage deviation ≤ 5%). The local state is defined as the key scenario variable that affects the contribution. The local state of the commercial center cluster is the real-time load fluctuation amplitude and the feeder load balancing degree. The local state of the residential community cluster is the energy storage SOC level and the peak-valley load difference. The initial contribution is obtained by calculating the improvement of the global target when the cluster achieves the local target, that is, initial contribution = single indicator improvement × indicator weight. If the weights of power supply reliability rate, network loss rate, and consumption rate are 0.4, 0.3, and 0.3 respectively, when the commercial center cluster achieves the local target, the power supply reliability rate improves by 0.01% and the network loss rate decreases by 0.2%, then the initial contribution of the commercial center cluster C1 = 0.01% × 0.4 + 0.2% × 0.3 = 0.01. When the residential community cluster achieves the local target, the new energy consumption rate improves by 1.5% and the network loss rate decreases by 0.1%, then the initial contribution of the residential community cluster C2 = 1.5% × 0.3 + 0.1% × 0.3 = 0.0048. The initial achievement probability is determined according to the frequency of the local state supporting the achievement of the local target in the historical data. If in 1000 events, there are 650 events in which the local state "load fluctuation amplitude ≤ 10%, feeder load balancing degree ≥ 0.8" supports the achievement of the local target, then the initial achievement probability of "load fluctuation amplitude ≤ 10%, feeder load balancing degree ≥ 0.8" is 65%. If in 1000 events, there are 700 events in which the local state "energy storage SOC ≥ 50%, peak-valley load difference ≤ 40MW" supports the achievement of the local target, then the initial achievement probability of "energy storage SOC ≥ 50%, peak-valley load difference ≤ 40MW" is 70%.After selecting and executing the historical global scheduling strategy, the observation results are collected through the cluster monitoring system. The observation results of the commercial center cluster can be "the load fluctuation amplitude is 8%, the feeder load balancing degree is 0.85 (both meet the support state), the scheduling delay is 22 seconds, and the load rate is 78% (the local target is achieved)"; the observation results of the residential community cluster can be "the energy storage SOC is 62%, the peak-valley load difference is 35 MW (both meet the support state), the charging and discharging delay is 28 seconds, and the load rate is 76% (the local target is achieved)". The initial achievement probability is corrected using the Bayesian rule, including: defining the observation event O as the local state meeting the support condition and the local target being achieved, if P(O|local state meeting the support condition) = 98%, P(O|local state not meeting the support condition) = 15%, then the first corrected achievement probability P(local state meeting the support condition|O) = [P(O|local state meeting the support condition) * initial achievement probability] / [P(O|local state meeting the support condition) * initial achievement probability + P(O|local state not meeting the support condition) * (1-initial achievement probability)], so the first corrected achievement probability P1 of the local state in the commercial center cluster is (0.98 * 0.65) / (0.98 * 0.65 + 0.15 * 0.35) ≈ 0.92; similarly, the first corrected achievement probability P2 of the local state in the residential community cluster is (0.98 * 0.7) / (0.98 * 0.7 + 0.15 * 0.3) ≈ 0.94, the correction contribution is obtained by the calculation method of correction contribution = initial contribution * (first corrected achievement probability / initial achievement probability), at this time, the correction contribution C11 of the local target of the commercial center cluster is 0.01 * (0.92 / 0.65) ≈ 0.0142, and the correction contribution C21 of the local target of the residential community cluster is 0.0048 * (0.94 / 0.7) ≈ 0.0065. It can be understood that when the absolute value of the deviation between the correction contribution and the initial contribution is less than or equal to 0.005, it means that the correction contribution meets the preset requirement. In this embodiment, by obtaining the initial achievement probability of the local state, the contribution is linked to the possibility of the cluster state, the initial achievement probability is corrected by using the Bayesian rule to obtain the first corrected achievement probability through the observation results, the initial contribution is corrected by the first corrected achievement probability which is close to the cluster scene to obtain the correction contribution, which can make the correction contribution deeply match the power grid scene, and when the correction contribution meets the preset requirement, it means that the correction contribution has accurately reflected the real contribution of the local target to the global target under the power grid scene, thereby improving the scene adaptability of the final contribution. By mapping the local target of the cluster to the quantitative contribution degree of the global target of the power grid, the information barrier that the local intelligent agent is difficult to accurately observe the global and the global is difficult to accurately perceive the local influence is broken through.

[0074] In the embodiment, when the achievement probability correction direction is consistent with the contribution correction direction, the correlation coefficient is 1, when the achievement probability correction direction is inconsistent with the contribution correction direction, the correlation coefficient is -1, when the achievement probability increases, the contribution also increases or the achievement probability decreases, the contribution also decreases, which indicates that the achievement probability correction direction is consistent with the contribution correction direction, otherwise, it indicates that the achievement probability correction direction is inconsistent with the contribution correction direction. The achievement probability correction value is obtained according to the preset requirement and the proportion, specifically: the initial difference value is obtained according to the preset requirement and the initial contribution, and the achievement probability correction value is obtained according to the initial difference value and the proportion. The process achievement probability is obtained according to the achievement probability correction value, the first corrected achievement probability and the correlation coefficient representing the correlation between the achievement probability correction direction and the contribution correction direction correction trend, specifically: process achievement probability = correlation coefficient × achievement probability correction value + first corrected achievement probability. The final contribution is obtained according to the average achievement probability, including: taking the corrected contribution as the initial contribution, taking the average achievement probability as the first corrected achievement probability, and iteratively executing S2c. In the embodiment, the final contribution is obtained according to the average achievement probability of the process achievement probability and the second corrected achievement probability, so that the internal linkage between the achievement probability and the contribution is considered, so as to make up for the lack of correlation of the Bayesian rule, significantly reduce the iteration convergence time of the final contribution obtained by only relying on the Bayesian rule to correct the achievement probability, and avoid one-sidedness caused by only relying on observation, so as to improve the accuracy of the final contribution while improving the efficiency of obtaining the final contribution.

[0075] S3: obtaining a global reward function according to the global target, and globally training the process agent according to the final contribution and the global reward function to obtain an optimized agent.

[0076] The global training of the process agent according to the final contribution and the global reward function to obtain an optimized agent includes:

[0077] The global target value is obtained according to the final contribution and the local target value, the third reward value is obtained by inputting the global target value into the global reward function, and the global training data set is obtained according to the third reward value and the local target value.

[0078] The global value function of the process agent is trained through the global training data set to obtain an optimized agent.

[0079] In the embodiment, the global value function is a predictor of the cumulative return of the agent in the future, which is used to measure the pros and cons of taking a certain global scheduling strategy in a certain global state. The standard for measuring the pros and cons is how much global reward can be obtained from the current global state. The global value function of the process agent is trained according to the third reward value and the local target value, so that the process agent can learn the rule that "after taking a certain local target value, the global reward obtained", and through training, the process agent can obtain the global scheduling strategy in the subsequent direction of maximizing the global reward value, thereby improving the adaptability of the global scheduling strategy to the comprehensive benefit of the power grid.

[0080] S4: constructing a global scheduling strategy updating rule through a PPO algorithm, and obtaining a global scheduling strategy through an optimization agent under the constraint of the global scheduling strategy updating rule, and scheduling the power grid through the global scheduling strategy.

[0081] The global scheduling strategy updating rule is constructed through the PPO algorithm, and includes:

[0082] The global scheduling switching rate is sheared to obtain a sheared scheduling switching rate according to a shearing function of the PPO algorithm under the shearing constraint of the preset tolerance, and a smaller value between the global scheduling switching rate and the sheared scheduling switching rate is used to obtain a scheduling strategy optimization function together with the comprehensive benefit of the power grid, and the global scheduling strategy updating rule is obtained according to the scheduling strategy optimization function.

[0083] The core goal of the PPO algorithm (proximal policy optimization) is to efficiently and stably optimize the decision-making strategy of the agent. The PPO algorithm limits the step length of policy updating to ensure that the difference between the new policy and the old policy is not too large, thereby preventing the global scheduling strategy from exceeding the safety limit of the device or the stability margin of the system. By solving the technical problem of the lack of scheduling strategy updating constraint mechanism in the prior art, the safe and stable operation of the power grid is ensured. The expression of the scheduling strategy optimization function is specifically:

[0084]

[0085] Among them, represents the scheduling strategy optimization value under the policy network parameter , represents the global scheduling switching rate under the policy network parameter , represents the shearing function, represents the preset tolerance, and through the constraint of the shearing function, the problem of frequency fluctuation and voltage disturbance caused by the oscillation of the global scheduling strategy is prevented, , represents the probability of selecting the global scheduling strategy under the current state of the power grid by the old global scheduling strategy,​ represents a new global dispatching strategy in the current state of the power grid selects a global dispatching strategy probability.

[0086] In some embodiments, the global dispatching strategy update rule is obtained according to the dispatching strategy optimization function, comprising:

[0087] The first dispatching strategy constraint is obtained according to the dispatching strategy optimization function, the power grid state change constraint is obtained according to the power grid safety requirement, the second dispatching strategy constraint is obtained according to the power grid state change constraint, and the global dispatching strategy update rule is obtained according to the first dispatching strategy constraint and the second dispatching strategy constraint.

[0088] In this embodiment, the first dispatching strategy constraint specifically refers to the range of the global dispatching switching rate , and the global dispatching strategy is always updated around the global target. The second dispatching strategy constraint specifically includes: the frequency fluctuation is not more than 50 MHz, the voltage offset rate is not more than 5%, and the load switching in any cluster is not more than 10%. If any threshold is violated, the global dispatching strategy is rejected, and the last stable global dispatching strategy is rolled back. By constructing a double-layer global dispatching strategy screening mechanism, on the one hand, the clip function is used to limit the update amplitude of the global dispatching strategy to avoid global dispatching strategy oscillation, and on the other hand, the threshold of physical indicators such as frequency, voltage, and load switching is set as a safety supervisor to perform post-filtering and rollback judgment on the dispatching result. If it is detected that any indicator exceeds the safety domain during dispatching, the protection mode is automatically triggered and the dispatching authority is degraded, thereby ensuring the safe and stable operation of the power grid.

[0089] In another embodiment, the global dispatching strategy is obtained by optimizing the agent under the constraint of the global dispatching strategy update rule, comprising:

[0090] The candidate local dispatching strategy of the cluster is obtained by optimizing the local value function of the agent, the candidate global dispatching strategy of the power grid is obtained according to the global value function of the agent under the constraint of the global dispatching strategy update rule, the candidate local target value corresponding to the candidate local dispatching strategy of the cluster is compared with the desired local target value corresponding to the candidate global dispatching strategy of the power grid, if the comparison is successful, the candidate local dispatching strategy of the comparison is taken as the global dispatching strategy, otherwise, the local value function or the global value function of the agent is modified according to the difference between the candidate local target value and the desired local target value, thereby obtaining the global dispatching strategy.

[0091] In this embodiment, the candidate global scheduling strategy specifically refers to the desired local target value for achieving the global target. By comparing the candidate local target value corresponding to the candidate local scheduling strategy with the desired local target value, the candidate local scheduling strategy that passes the comparison is taken as the global scheduling strategy, which not only takes into account the scheduling needs of the local and global, but also guarantees the scheduling efficiency of the cluster. Under the constraint of the global scheduling strategy updating rule, the global scheduling strategy is obtained, and the safety of the power grid is also guaranteed.

[0092] Further comprising:

[0093] The feedback result of scheduling the power grid according to the global scheduling strategy is obtained, and it is determined whether the feedback result meets the trigger condition. If it meets, the optimal agent is obtained by updating the optimization agent according to the feedback result, the optimal global scheduling strategy is obtained through the optimal agent, and the power grid is scheduled through the optimal global scheduling strategy.

[0094] The feedback result specifically refers to the active power output difference, the energy storage SOC offset, and the voltage deviation rate. The trigger condition includes: 1. In three consecutive scheduling periods The advantage function descent amplitude is greater than 30%; 2. The energy storage SOC trend reverses (i.e., repeated charging and discharging) for two consecutive periods; 3. The single-area photovoltaic curtailment ratio is higher than the set threshold (such as 30%). Once the trigger condition is met, the feedback result is returned to the cluster scheduling layer, and after being summarized and coded by the local model, it is sent to the central control layer. If the current scheduling strategy causes the target benefit function to decrease significantly, the fast retraining mode is entered, the central control layer loads the historical training model and supplements the recent feedback data, and performs strategy fine-tuning to complete a fast strategy repair process. The feedback mechanism can also trigger the main model to re-fit or only modify the optimization agent strategy. In addition, it can also call the historical 30-day data for sample expansion training, use a short window model (such as 10 iterations) for fast convergence, and be suitable for seasonal or event-type scenario changes. Through the real-time feedback of the strategy running effect and the fast retraining mechanism, and according to the key operation indicators (such as advantage function descent, SOC reversal, and curtailment rate surge), abnormality is identified, and local fine-tuning of the strategy or re-fitting of the historical model is automatically triggered, forming an online adaptive scheduling closed loop of strategy-execution-feedback-relearning. It also effectively solves the problem of slow response of traditional global scheduling strategies to seasonal, event-type sudden working conditions, greatly improving the robustness and evolution ability of long-term system operation.

[0095] Embodiment 2:

[0096] The embodiment also provides a sub-cluster power grid scheduling system, which is suitable for the sub-cluster power grid scheduling method and includes:

[0097] a local training module configured to obtain a local target of the cluster based on a cluster benefit, and to perform local training on an agent corresponding to the cluster based on the local target and historical data of the cluster to obtain a process agent;

[0098] a final contribution obtaining module configured to obtain a global target based on a power grid comprehensive benefit, and to obtain a final contribution of the local target to the global target using a partially observable Markov decision process;

[0099] a global training module configured to obtain a global reward function based on the global target, and to perform global training on the process agent based on the final contribution and the global reward function to obtain an optimized agent;

[0100] a scheduling module configured to construct a global scheduling strategy update rule by using a PPO algorithm, to obtain a global scheduling strategy by using the optimized agent under the constraint of the global scheduling strategy update rule, and to schedule the power grid by using the global scheduling strategy.

[0101] The final contribution of the local target to the global target is obtained using the partially observable Markov decision process, including: obtaining an initial contribution of the local target to the global target according to historical experience in historical data of the power grid, and obtaining an initial achievement probability of the local state to the local target according to the historical experience; any historical global scheduling strategy in the historical data of the power grid is executed to obtain an observation result, and based on the observation result, the initial achievement probability is corrected using the Bayesian rule to obtain a first corrected achievement probability; the initial contribution is corrected according to the first corrected achievement probability to obtain a corrected contribution, if the corrected contribution meets a preset requirement, the corrected contribution is taken as the final contribution, otherwise, the achievement probability difference and the achievement probability correction direction are obtained according to the initial achievement probability and the first corrected achievement probability, the contribution difference and the contribution correction direction are obtained according to the initial contribution and the corrected contribution, the ratio of the achievement probability difference and the contribution difference is obtained, the achievement probability correction value is obtained according to the preset requirement and the ratio, the process achievement probability is obtained according to the achievement probability correction value, the first corrected achievement probability and the correlation coefficient representing the correlation between the achievement probability correction direction and the contribution correction direction correction trend, the first corrected achievement probability is taken as the initial achievement probability, the initial achievement probability is corrected using the Bayesian rule to obtain a second corrected achievement probability, the average achievement probability is obtained according to the second corrected achievement probability and the process achievement probability, and the final contribution is obtained according to the average achievement probability. The process agent is globally trained according to the final contribution and the global reward function to obtain an optimized agent, including: obtaining a global target value according to the final contribution and a local target value, inputting the global target value into the global reward function to obtain a third reward value, and obtaining a global training data set according to the third reward value and the local target value; the global value function of the process agent is trained through the global training data set to obtain the optimized agent. The global scheduling strategy update rule is constructed through the PPO algorithm, including: taking a preset tolerance as a clipping constraint, under the clipping constraint, the global scheduling switching rate is clipped according to the clipping function of the PPO algorithm to obtain a clipped scheduling switching rate, the global scheduling switching rate and the clipped scheduling switching rate are used. The smaller value and the overall benefit of the power grid are used to obtain a scheduling strategy optimization function, and the global scheduling strategy update rule is obtained according to the scheduling strategy optimization function.

[0102] In the embodiment, by considering that the agent can only obtain and process local observation data within its jurisdiction, lacks the perception ability of the overall operation state of the power grid, and the partially observable Markov decision process has the partially observable characteristic, the local target of the cluster can be accurately mapped to the contribution to the global target of the power grid through the partially observable Markov decision process, thereby breaking through the information barrier that the local agent is difficult to accurately observe the global, and the global is difficult to accurately perceive the local influence. The process agent is globally trained through the final contribution and the global reward function, so that the agent can accurately learn the influence of the change of the local target on the global target, thereby aligning the optimization direction of the local target with the optimization direction of the global target under the guidance of the global reward function, so as to balance the scheduling requirements of the local and the global. The technical problem that the prior art is difficult to balance the scheduling requirements of the local and the global is solved, thereby improving the cluster benefit while improving the overall benefit of the power grid; the acquisition of the global scheduling strategy is constrained by constructing the global scheduling strategy update rule through the PPO algorithm, thereby preventing the global scheduling strategy from exceeding the device safety limit value or the system stability margin, thereby solving the technical problem that the prior art lacks a scheduling strategy update constraint mechanism, thereby ensuring the safe and stable operation of the power grid; by combining the partially observable Markov decision process and the PPO algorithm, the safe and stable operation of the power grid is ensured while the scheduling efficiency of the cluster and the overall operation benefit of the power grid are improved.

[0103] The specific implementation mode described above is a preferred implementation mode of the cluster-based power grid scheduling method and system of the present application, and is not limited to the specific implementation range of the present application. The scope of the present application includes but is not limited to the specific implementation mode. Any equivalent changes made in accordance with the shape and structure of the present application are within the scope of protection of the present application.

Claims

1. A method for dispatching power grids by cluster, characterized in that, Includes the following steps: Based on the cluster effect, the local target of the cluster is obtained, and the agent corresponding to the cluster is locally trained based on the local target and the historical data of the cluster to obtain the process agent; The global objective is obtained based on the comprehensive benefits of the power grid, and the final contribution of local objectives to the global objective is obtained using a partially observable Markov decision process. Obtain the global reward function based on the global objective, and perform global training on the process agent based on the final contribution and the global reward function to obtain the optimized agent; A global scheduling strategy update rule is constructed using the PPO algorithm. Under the constraint of the global scheduling strategy update rule, the global scheduling strategy is obtained by optimizing the agent, and the power grid is scheduled using the global scheduling strategy. The use of partially observable Markov decision processes to obtain the final contribution of local objectives to the global objective includes: The initial contribution of local targets to the global target is obtained based on historical experience from historical data of the power grid, and the initial probability of achieving local targets is obtained based on historical experience. Execute any historical global scheduling strategy from the historical data of the power grid to obtain the observation results. Based on the observation results, use Bayesian rules to correct the initial achievement probability to obtain the first corrected achievement probability. The initial contribution is corrected based on the first corrected achievement probability to obtain a corrected contribution. If the corrected contribution meets the preset requirements, it is taken as the final contribution. Otherwise, the difference between the achievement probability and the direction of achievement probability correction are obtained based on the initial achievement probability and the first corrected achievement probability. The difference between the contribution and the direction of contribution correction are obtained based on the initial contribution and the corrected contribution. The ratio between the difference between the achievement probability and the contribution difference is obtained. The corrected achievement probability value is obtained based on the preset requirements and the ratio. The process achievement probability is obtained based on the corrected achievement probability value, the first corrected achievement probability, and the correlation coefficient representing the correlation between the correction trend of the achievement probability correction direction and the direction of contribution correction. The first corrected achievement probability is taken as the initial achievement probability. The initial achievement probability is corrected using Bayesian rules to obtain the second corrected achievement probability. The average achievement probability is obtained based on the second corrected achievement probability and the process achievement probability. The final contribution is obtained based on the average achievement probability.

2. The method for dispatching a power grid cluster according to claim 1, characterized in that, The process of acquiring intelligent agents by locally training the agents corresponding to the cluster based on historical data of local targets and the cluster includes: The agent corresponding to the cluster is obtained based on reinforcement learning, and the local reward function is obtained based on the local objective. The next state of the cluster from the historical data of the cluster is input into the local reward function to obtain the first reward value. The local training dataset is obtained based on the current state of the cluster, the historical local scheduling strategy, the next state of the cluster, and the first reward value from the historical data of the cluster. The local value function of the agent is trained using a local training dataset to obtain an updated agent. The current state of the cluster is input into the updated agent to obtain a second reward value. If the second reward value meets a preset condition, the updated agent is used as the process agent. Otherwise, the local value function of the updated agent is trained using the local training dataset to obtain the process agent.

3. The method for dispatching a power grid by sub-cluster according to claim 1, characterized in that, The method of obtaining the overall objective based on the comprehensive benefits of the power grid includes: The overall benefits of the power grid are obtained by considering its energy absorption efficiency, voltage deviation penalties, and regulation costs, and the overall objectives are derived from these benefits.

4. The method for dispatching a power grid by sub-cluster according to claim 1, characterized in that, The step of globally training the process agent based on the final contribution and the global reward function to obtain an optimized agent includes: The global target value is obtained based on the final contribution and the local target value. The global target value is then input into the global reward function to obtain the third reward value. The global training dataset is obtained based on the third reward value and the local target value. The global value function of the process agent is trained using a global training dataset to obtain an optimized agent.

5. The method for dispatching a power grid by sub-cluster according to claim 1, characterized in that, The construction of global scheduling policy update rules using the PPO algorithm includes: Using a preset tolerance as a shearing constraint, the global scheduling switching rate is sheared according to the shearing function of the PPO algorithm to obtain the sheared scheduling switching rate. The smaller value between the global scheduling switching rate and the sheared scheduling switching rate is used with the comprehensive benefits of the power grid to obtain the scheduling strategy optimization function. The global scheduling strategy update rule is obtained based on the scheduling strategy optimization function.

6. The method for dispatching a power grid by sub-cluster according to claim 5, characterized in that, The step of obtaining the global scheduling policy update rule based on the scheduling policy optimization function includes: The first scheduling strategy constraint is obtained based on the scheduling strategy optimization function, the power grid state change constraint is obtained based on the power grid security requirements, the second scheduling strategy constraint is obtained based on the power grid state change constraint, and the global scheduling strategy update rule is obtained based on the first scheduling strategy constraint and the second scheduling strategy constraint.

7. The method for dispatching a power grid by sub-cluster according to claim 1, characterized in that, The step of obtaining the global scheduling policy by optimizing the agent under the constraints of the global scheduling policy update rules includes: Candidate local scheduling strategies for the cluster are obtained by optimizing the local value function of the intelligent agent. Under the constraint of the global scheduling strategy update rule, candidate global scheduling strategies for the power grid are obtained based on the global value function of the optimizing intelligent agent. The candidate local target value corresponding to the candidate local scheduling strategy of the cluster is compared with the desired local target value corresponding to the candidate global scheduling strategy of the power grid. If the comparison is successful, the candidate local scheduling strategy that has been successfully compared is adopted as the global scheduling strategy. Otherwise, the local value function or global value function of the optimizing intelligent agent is modified based on the difference between the candidate local target value and the desired local target value, thereby obtaining the global scheduling strategy.

8. The method for dispatching a power grid by sub-cluster according to claim 1, characterized in that, Also includes: Obtain feedback results of power grid scheduling based on the global scheduling strategy, determine whether the feedback results meet the triggering conditions, and if so, update the optimization agent based on the feedback results to obtain the optimal agent. Obtain the optimal global scheduling strategy through the optimal agent, and schedule the power grid through the optimal global scheduling strategy.

9. A clustered power grid dispatching system, applicable to the clustered power grid dispatching method according to any one of claims 1-8, characterized in that, include: The local training module is used to obtain local targets of the cluster based on cluster benefits, and to obtain process agents by locally training the agents corresponding to the cluster based on the local targets and the historical data of the cluster. The final contribution acquisition module is used to obtain the global objective based on the comprehensive benefits of the power grid, and to obtain the final contribution of local objectives to the global objective using a partially observable Markov decision process. The global training module is used to obtain the global reward function based on the global objective, and to perform global training on the process agent based on the final contribution and the global reward function to obtain the optimized agent. The scheduling module is used to construct global scheduling policy update rules through the PPO algorithm. Under the constraints of the global scheduling policy update rules, the module optimizes the agent to obtain the global scheduling policy and then schedules the power grid using the global scheduling policy.

Images