Power dispatching auxiliary decision-making agent interaction method and system

Abstract

The invention provides a power dispatching auxiliary decision-making agent interaction method and system. The method comprises the following steps: constructing a digital mirror image model, obtaining real-time operation information of a power grid, determining a dispatching task, decomposing the dispatching task into a plurality of dispatching sub-tasks, and copying the digital mirror image model to obtain a plurality of copy digital mirror image models; judging whether the sub-scheduling task control strategy meets a preset credibility requirement or not based on an auxiliary decision verification agent; outputting a first scheduling task scheduling scheme based on the aid decision reinforcement learning agent and the aid decision scheduling agent; and outputting a second scheduling task scheduling scheme based on the aid decision reinforcement learning agent. The problems that in the prior art, data are not uniform, historical faults occur or extreme operation states are few, an intelligent agent is difficult to output a valuable scheduling control strategy, and consequently the credibility of the scheduling control strategy is low are solved. According to the method, the intelligent agent interaction environment is simulated more accurately, and the reliability of intelligent agent strategy generation is improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent agent interaction, and in particular to an intelligent agent interaction method and system for auxiliary decision-making in power dispatching. Background Art

[0002] In new power systems, artificial intelligence technologies such as deep reinforcement learning models form implicit knowledge through massive experience sample inputs generated by interaction with the environment and deep learning. Based on the experience samples, an automated decision model is formed through training. When new situations occur and new data is input, the system can use the previously established model to make decisions directly. This traditional model-driven method has been widely used.

[0003] At present, the traditional pure data-driven decision-making intelligence technology in the existing technology has a typical small sample problem due to data imbalance. Most of the historical measurement samples are normal state data, and there are very few historical failures or extreme operating states. When encountering small sample scenarios during operation, it is difficult for the intelligent agent to output valuable scheduling and control strategies. Therefore, there are problems of insufficient security and credibility. At the same time, due to the lack of historical experience or prior knowledge, it is difficult to judge the credibility of the scheduling and control strategy, resulting in the problem of low credibility of the scheduling and control strategy. Summary of the invention

[0004] In order to solve the above problems, the present invention proposes an intelligent agent interaction method and system for power dispatching auxiliary decision-making, which fully considers the various states of the intelligent agent operation to output valuable dispatching control strategies, while considering historical experience or prior knowledge to improve the credibility of the dispatching control strategy.

[0005] To achieve the above objectives, an embodiment of the present invention provides a method for interactive intelligent agent for auxiliary decision-making in power dispatching, comprising:

[0006] Constructing a digital mirror model and obtaining real-time operation information of the power grid, determining a scheduling task and decomposing the scheduling task into a number of scheduling subtasks;

[0007] Based on the requirements corresponding to the plurality of scheduling subtasks, the digital mirror model is copied to obtain a plurality of copy digital mirror models;

[0008] Based on the auxiliary decision verification agent and the auxiliary decision reinforcement learning agent, determine whether the sub-scheduling task control strategy meets the preset credibility requirements;

[0009] When the sub-scheduling task control strategy does not meet the preset credibility requirement, outputting a first scheduling task scheduling plan based on the auxiliary decision reinforcement learning agent and the auxiliary decision scheduling agent;

[0010] When the sub-scheduling task control strategy meets the preset credibility requirement, a second scheduling task scheduling plan is output based on the auxiliary decision-making reinforcement learning agent.

[0011] Furthermore, based on the digital mirror model and the real-time operation information of the power grid, a scheduling task is determined and the scheduling task is decomposed into a plurality of scheduling subtasks, including:

[0012] Based on the digital mirror model, situation awareness and risk assessment are performed on the real-time operation information of the power grid, and events that meet preset assessment requirements are marked as risk accidents;

[0013] Based on the risk accident, matching the corresponding fault scenario to obtain an accident matching result;

[0014] Based on the accident matching result, generating a corresponding scheduling task;

[0015] According to preset different performance index solution requirements, the control action corresponding to the scheduling task is solved by a preset parallel solution algorithm to obtain a number of scheduling subtasks.

[0016] Furthermore, based on the plurality of scheduling subtasks, the digital mirror model is replicated to obtain a plurality of replica digital mirror models, including:

[0017] Based on the plurality of scheduling subtasks, different performance indicators are obtained;

[0018] Based on the different performance indicators, the digital mirror model is copied to obtain a plurality of copy digital mirror models corresponding to each indicator.

[0019] Furthermore, based on the auxiliary decision verification agent and the auxiliary decision reinforcement learning agent, it is judged whether the sub-scheduling task control strategy meets the preset credibility requirements, including:

[0020] Based on the auxiliary decision-making reinforcement learning agent, each sub-scheduling task is input into the corresponding replica digital mirror model for interactive operation analysis, and several operation status characteristics and sub-scheduling task control strategies are obtained;

[0021] Based on the auxiliary decision verification agent, the residual values ​​of the several operating state characteristics and the preset training experience are calculated to obtain the credibility evaluation parameters corresponding to each sub-scheduling task;

[0022] Based on the credibility evaluation parameters corresponding to each of the sub-scheduling tasks, calculate the corresponding credibility score;

[0023] Based on the corresponding credibility score, it is determined whether the sub-scheduling task control strategy meets the preset credibility requirement.

[0024] Furthermore, when the sub-scheduling task control strategy does not meet the preset credibility requirement, based on the auxiliary decision-making reinforcement learning agent and the auxiliary decision-making scheduling agent, a first scheduling task scheduling scheme is output, including:

[0025] When the sub-scheduling task control strategy does not meet the preset credibility requirement, the sub-scheduling task control strategy is decided and regulated by the auxiliary decision-making scheduling agent, a priority experience pool is constructed, and the first sub-scheduling task control strategy is output;

[0026] The priority experience pool is sampled by a preset sampling algorithm to obtain an experience sample; wherein the experience sample includes a first experience sample and a second experience sample; the first experience sample is a sample obtained by extracting a common experience pool; the common experience pool is constructed by making decisions and regulating the sub-scheduling task control strategy through an auxiliary decision-making reinforcement learning agent; the second experience sample is a sample obtained by extracting a priority experience pool;

[0027] Based on the first sub-scheduling task control strategy, the experience sample is input into the auxiliary decision-making reinforcement learning agent, model learning is performed by the auxiliary decision-making reinforcement learning agent, and a first scheduling task scheduling plan is output.

[0028] Furthermore, when the sub-scheduling task control strategy meets the preset credibility requirement, a second scheduling task scheduling scheme is output based on the auxiliary decision-making reinforcement learning agent, including:

[0029] When the sub-scheduling task control strategy meets the preset credibility requirement, based on the auxiliary decision-making reinforcement learning agent, the first local state of each sub-scheduling task is obtained and the corresponding second sub-scheduling task control strategy is output;

[0030] Based on the digital mirror models of each replica, the first local state of each corresponding sub-scheduling task is executed to obtain a second local state and a reward;

[0031] Building a common experience pool based on the first local state of each sub-scheduling task, the corresponding second sub-scheduling task control strategy, the second local state and the reward;

[0032] Based on the common experience pool, the auxiliary decision-making reinforcement learning agent is updated until the preset number of iterations is met, and a number of reward values ​​and a third sub-scheduling task control strategy are obtained;

[0033] Based on the several reward values ​​and the third sub-scheduling task control strategy, a second scheduling task scheduling plan is output.

[0034] Furthermore, it also includes:

[0035] Based on the requirements corresponding to the several scheduling subtasks, construct an auxiliary decision-making intelligent agent;

[0036] Based on the historical operation data and the auxiliary decision-making agent, generate several operation scenarios;

[0037] Based on the auxiliary decision-making agent and the several operation scenarios, obtaining a third local state of each sub-scheduling task and outputting a fourth sub-scheduling task control strategy;

[0038] Based on the digital mirror models of each replica, execute the third local state of each corresponding sub-scheduling task to obtain a fourth local state and a reward;

[0039] The third local state of each sub-scheduling task, the corresponding fourth sub-scheduling task control strategy, the fourth local state and the reward are saved in the common experience pool;

[0040] Based on the common experience pool, the auxiliary decision-making reinforcement learning agent is updated until it meets the preset model fitting requirements to obtain a pre-trained auxiliary decision-making agent.

[0041] The embodiment of the present invention also provides an intelligent interactive system for auxiliary decision-making of power dispatching, including: a task acquisition and decomposition module, a model replication module, a credibility judgment module, a first output module and a second output module;

[0042] The task acquisition and decomposition module is used to build a digital mirror model and obtain real-time operation information of the power grid, determine the scheduling task and decompose the scheduling task into several scheduling subtasks;

[0043] The model replication module is used to replicate the digital mirror model based on the requirements corresponding to the plurality of scheduling subtasks to obtain a plurality of replica digital mirror models;

[0044] The credibility judgment module is used to judge whether the sub-scheduling task control strategy meets the preset credibility requirements based on the auxiliary decision verification agent and the auxiliary decision reinforcement learning agent;

[0045] The first output module is used to output a first scheduling task scheduling plan based on the auxiliary decision reinforcement learning agent and the auxiliary decision scheduling agent when the sub-scheduling task control strategy does not meet the preset credibility requirement;

[0046] The second output module is used to output a second scheduling task scheduling plan based on the auxiliary decision-making reinforcement learning agent when the sub-scheduling task control strategy meets the preset credibility requirement.

[0047] Furthermore, the first output module is used to output a first scheduling task scheduling solution based on the auxiliary decision reinforcement learning agent and the auxiliary decision scheduling agent when the sub-scheduling task control strategy does not meet the preset credibility requirement, including:

[0048] A first experience pool construction unit, a sample extraction unit and a first agent output unit;

[0049] The first experience pool construction unit is used to make decisions and adjust the sub-scheduling task control strategy through the auxiliary decision-making scheduling agent when the sub-scheduling task control strategy does not meet the preset credibility requirement, construct a priority experience pool and output the first sub-scheduling task control strategy;

[0050] The sample extraction unit is used to sample the priority experience pool through a preset sampling algorithm to obtain an experience sample; wherein the experience sample includes a first experience sample and a second experience sample; the first experience sample is a sample obtained by extracting a common experience pool; the common experience pool is constructed by making decisions and regulating the sub-scheduling task control strategy through an auxiliary decision-making reinforcement learning agent; the second experience sample is a sample obtained by extracting a priority experience pool;

[0051] The first agent output unit is used to input the experience sample into the auxiliary decision-making reinforcement learning agent based on the first sub-scheduling task control strategy, perform model learning through the auxiliary decision-making reinforcement learning agent, and output a first scheduling task scheduling plan.

[0052] Furthermore, the second output module is used to output a second scheduling task scheduling scheme based on the auxiliary decision-making reinforcement learning agent when the sub-scheduling task control strategy meets the preset credibility requirement, including:

[0053] A first local state acquisition unit, a second local state acquisition unit, a second experience pool construction unit, an iterative update unit, and a second agent output unit;

[0054] The first local state acquisition unit is used to acquire the first local state of each sub-scheduling task and output the corresponding second sub-scheduling task control strategy based on the auxiliary decision-making reinforcement learning agent when the sub-scheduling task control strategy meets the preset credibility requirement;

[0055] The second local state acquisition unit is used to execute the first local state of each corresponding sub-scheduling task based on each replica digital mirror model to obtain a second local state and a reward;

[0056] The second experience pool construction unit is used to construct a common experience pool based on the first local state of each sub-scheduling task, the corresponding second sub-scheduling task control strategy, the second local state and the reward;

[0057] The iterative updating unit is used to update the auxiliary decision reinforcement learning agent based on the common experience pool until the preset number of iterations is met, and obtain a number of reward values ​​and a third sub-scheduling task control strategy;

[0058] The second agent output unit is used to output a second scheduling task scheduling plan based on the several reward values ​​and the third sub-scheduling task control strategy.

[0059] Beneficial effects:

[0060] (1) By determining the scheduling task and decomposing it into several scheduling subtasks, multiple types of operating scenarios can be mapped to improve the credibility of subsequent intelligent agent strategy generation; based on the needs of several scheduling subtasks, a digital mirror model and a replica digital mirror model are constructed, which can reflect the changing laws of the new power system in the digital space and can perform deduction and trend prediction of the operating status, more accurately simulate the intelligent agent interaction environment, and improve the reliability of intelligent agent strategy generation;

[0061] (2) The credibility of the sub-scheduling task control strategy generated by the agent is judged. By intervening in the auxiliary decision-making scheduling agent to provide historical experience or prior knowledge, the auxiliary decision-making reinforcement learning agent, after model learning, will eventually form a scheduling task control strategy composed of multiple sub-scheduling task control strategies, and use one of them as the final scheduling task scheduling plan. In this way, the auxiliary decision-making agent can independently judge the credibility of the scheduling control strategy, greatly improving the credibility of the output scheduling plan. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 A schematic diagram of a process flow of an intelligent agent interaction method for power dispatching auxiliary decision-making provided in an embodiment of the present invention Figure 1 ;

[0063] Figure 2 A schematic diagram of a process flow of an intelligent agent interaction method for power dispatching auxiliary decision-making provided in an embodiment of the present invention Figure 2 ;

[0064] Figure 3 A schematic diagram of the module structure of an intelligent interactive system for auxiliary decision-making in power dispatching provided by an embodiment of the present invention;

[0065] Figure 4 A schematic diagram of the structure of a first output module of an electric power dispatch auxiliary decision-making intelligent agent interactive system provided by a certain embodiment of the present invention;

[0066] Figure 5 A schematic diagram of the structure of a second output module of an electric power dispatching auxiliary decision-making intelligent agent interaction system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] Due to the data imbalance problem in the current existing technology, the power system pursues safe and reliable operation. Most of the historical measurement samples are normal state data, and there are very few historical faults or extreme operating states. This is a typical small sample problem. When encountering a small sample scenario during operation, it is difficult for the intelligent agent to output valuable dispatching and control strategies; in addition, reinforcement learning adopts an "end-to-end" approach in the training, deployment and application links. There is a lack of active intervention from a third party, and it is difficult to judge the credibility of the dispatching and control strategy, and thus it is impossible to actively seek third-party intervention and verification when the credibility of the dispatching and control results is low. The third party can be a model with previous knowledge or historical experience. Therefore, the embodiment of the present invention proposes a method and system for interactive intelligent agents for power dispatching and auxiliary decision-making to solve the problems existing in the prior art.

[0069] In this embodiment of the invention, the auxiliary decision-making agent includes: an auxiliary decision-making reinforcement learning agent, an auxiliary decision-making verification agent and an auxiliary decision-making scheduling agent. The auxiliary decision-making reinforcement learning agent, the auxiliary decision-making verification agent and the auxiliary decision-making scheduling agent are all pre-trained neural network models; the auxiliary decision-making reinforcement learning agent is a fully connected neural network model trained using a deep reinforcement learning algorithm (such as SAC and PPO algorithms); the auxiliary decision-making scheduling agent is a neural network model that is trained through prior knowledge and historical experience and can simulate human scheduling experience. The auxiliary decision-making agent is deployed in the central controller (dispatching control center).

[0070] Example 1

[0071] See also Figure 1 , Figure 1 A schematic diagram of a process flow of an intelligent agent interaction method for power dispatching auxiliary decision-making provided in an embodiment of the present invention Figure 1 .like Figure 1 As shown, the embodiment of the present invention proposes an intelligent agent interaction method for power dispatch auxiliary decision-making, including steps 101 to 105. It is worth mentioning that, see Figure 2 , Figure 2 A schematic diagram of a process flow of an intelligent agent interaction method for power dispatching auxiliary decision-making provided in an embodiment of the present invention Figure 2In the specific explanation of this embodiment, Figure 2 The flowchart specifically explains each step, which will not be repeated below. The specific steps are as follows:

[0072] Step 101, constructing a digital mirror model and obtaining real-time operation information of the power grid, determining a scheduling task and decomposing the scheduling task into a plurality of scheduling subtasks;

[0073] As an example of this embodiment, based on the digital mirror model, situation awareness and risk assessment are performed on the real-time operation information of the power grid, and events that meet preset assessment requirements are marked as risk accidents; based on the risk accidents, corresponding fault scenarios are matched to obtain accident matching results; based on the accident matching results, corresponding scheduling tasks are generated; according to preset different performance indicator solution requirements, the control actions corresponding to the scheduling tasks are solved by a preset parallel solution algorithm to obtain a number of scheduling subtasks.

[0074] A specific possible implementation method is to build a real-time benchmark digital mirror model based on the new power system in actual operation. The benchmark digital mirror model includes: a power flow equation model, a sensitivity analysis model, a time domain simulation model and a small disturbance stability analysis model, etc. It is a digital space model that can reflect the changing laws of the new power system in the digital space and can deduce and trend the operating status; then read the power grid operation information from the new power system, such as power grid topology, load forecast and maintenance plan data; based on the real-time digital mirror model and the power grid operation information, situation awareness and risk assessment are carried out to detect risk accidents in advance and generate corresponding fault scenarios and scheduling tasks. For abnormal data in the power grid operation information, the data with abnormal risk is marked as a risk event accident, and each risk accident is matched with its corresponding fault scenario, and a corresponding scheduling task is generated for each fault scenario, and the scheduling task is further decomposed into sub- Scheduling task; specifically, for the same control problem, multiple performance indicators are used to simultaneously evaluate the pros and cons of the control strategy, that is, a multi-Markov problem parallel solution control strategy, so that multiple performance indicators can meet the requirements; the multi-Markov problem parallel solution control strategy refers to selecting a preset Markov problem according to the decomposed sub-scheduling task, and using an auxiliary decision-making reinforcement learning agent to solve it. Different Markov problems have different local states, reward functions, action spaces and replica digital mirror models. For example, for the voltage stability problem, the voltage stability can be decomposed into control problems of performance indicators such as node voltage and power distribution, sensitivity of natural resources (wind speed and light, etc.) to unit output, dynamic changes of electronic component state variables and small disturbance stability. According to the solution requirements of different performance indicators, the corresponding Markov problem is called to solve the control actions of each control unit, that is, the scheduling task is decomposed into sub-scheduling tasks.

[0075] Step 102, based on the requirements corresponding to the plurality of scheduling subtasks, the digital mirror model is copied to obtain a plurality of copy digital mirror models;

[0076] As an example of this embodiment, different performance indicators are obtained based on the plurality of scheduling subtasks; based on the different performance indicators, the digital mirror model is replicated to obtain a plurality of replica digital mirror models corresponding to each indicator.

[0077] A specific feasible implementation method is that, based on the requirements of the sub-scheduling task, the auxiliary decision-making intelligent agent uses the baseline digital mirror model to copy and generate multiple replica digital mirror models as the sub-scheduling task interaction environment; the replica digital mirror model needs to be selected according to different performance indicators, such as calling the power flow equation model when paying attention to node voltage and power distribution, calling the sensitivity analysis model when paying attention to the sensitivity of natural resources and unit output, and calling the time domain simulation model when paying attention to the dynamic changes of electronic component state variables.

[0078] Step 103, based on the auxiliary decision verification agent and the auxiliary decision reinforcement learning agent, determine whether the sub-scheduling task control strategy meets the preset credibility requirement;

[0079] As an example of this embodiment, based on the auxiliary decision-making reinforcement learning agent, each sub-scheduling task is input into the corresponding copy digital mirror model for interactive operation analysis to obtain a number of operating state characteristics and sub-scheduling task control strategies; based on the auxiliary decision-making verification agent, the residual values ​​of the several operating state characteristics and the preset training experience are calculated to obtain the credibility evaluation parameters corresponding to each sub-scheduling task; based on the credibility evaluation parameters corresponding to each sub-scheduling task, the corresponding credibility score is calculated; based on the corresponding credibility score, it is determined whether the sub-scheduling task control strategy meets the preset credibility requirements.

[0080] A specific possible implementation method is that the auxiliary decision reinforcement learning agent reads the local state required by the sub-scheduling task from the replica digital mirror model and outputs the corresponding sub-scheduling task control strategy. The auxiliary decision verification agent is a deep network model that can distinguish the differences in the operating state characteristics of the new power system; such as using a recurrent neural network model (LSTM model, GRU model, etc.) trained with data with time series characteristics such as voltage and voltage sensitivity, and distinguishing the changes in the operating state characteristics of the new power system by the residual between the predicted data and the actual data. For each credibility evaluation parameter corresponding to the scheduling subtask, the corresponding credibility score is calculated, and the residual is within a certain range. If it is shown that the state characteristics are consistent with the training experience, then the credibility score corresponding to the scheduling subtask is high, and the sub-scheduling task control strategy output by the auxiliary decision reinforcement learning agent is highly credible. Otherwise, if the current state is far from the pre-training working condition or is a difficult working condition, then the credibility score corresponding to the scheduling subtask is low, and the sub-scheduling task control strategy output by the auxiliary decision reinforcement learning agent is low in credibility; wherein, the specific process of the pre-training is described below and will not be repeated here.

[0081] Step 104: when the sub-scheduling task control strategy does not meet the preset credibility requirement, outputting a first scheduling task scheduling plan based on the auxiliary decision reinforcement learning agent and the auxiliary decision scheduling agent;

[0082] As an example of this embodiment, when the sub-scheduling task control strategy does not meet the preset credibility requirement, the sub-scheduling task control strategy is decided and regulated by the auxiliary decision-making scheduling agent, a priority experience pool is constructed and the first sub-scheduling task control strategy is output; the priority experience pool is sampled by a preset sampling algorithm to obtain experience samples; wherein the experience samples include a first experience sample and a second experience sample; the first experience sample is a sample obtained by extracting from a common experience pool; the common experience pool is obtained by constructing the sub-scheduling task control strategy through decision-making and regulation by an auxiliary decision-making reinforcement learning agent; the second experience sample is a sample obtained by extracting from the priority experience pool; based on the first sub-scheduling task control strategy, the experience sample is input into the auxiliary decision-making reinforcement learning agent, model learning is performed by the auxiliary decision-making reinforcement learning agent, and the first scheduling task scheduling plan is output.

[0083] For a specific implementation method, please refer to Figure 2 , the auxiliary decision scheduling agent outputs a scheduling plan based on the sub-scheduling task control strategy, and then performs model learning through the auxiliary decision reinforcement learning agent; the sub-scheduling task control strategy is The final sub-scheduling task control strategy is a subset of the sub-scheduling task control strategy, namely The scheduling task control strategy is the intersection of the sub-scheduling task control strategies, that is, Π = π1* ∩π 2* ...∩π j* , the scheduling scheme is one of the feasible strategies in the scheduling task control strategy, where j is the number of sub-scheduling tasks, i is the number of control strategies required for each sub-scheduling task, and in addition, the experience pool described in the embodiment of the present invention includes: an ordinary experience pool and a priority experience pool; wherein the ordinary experience pool is used to store the experience generated by the interaction of the auxiliary decision-making reinforcement learning agent; the priority experience pool is used to store the scheduling control decisions output by the auxiliary decision-making scheduling agent in a small sample scenario. During the reinforcement learning training and model learning, the selected experience samples include samples from the priority experience pool and samples from the ordinary experience pool at the same time. The sample sampling method of the priority experience pool is based on the sampling method of the priority value, and the ordinary experience pool adopts a random sampling method.

[0084] As another example of an embodiment of the present invention, in the process of training the auxiliary decision-making scheduling agent, experts and dispatchers can also be involved in training and regulation, and the expert experience can be added to the experience sample of the auxiliary decision-making agent, that is, the experts and dispatchers select the scheduling task control strategy according to human experience during the model training process for output, so that the auxiliary decision-making scheduling agent can output an effective scheduling task control strategy in a small sample scenario, and input it into the auxiliary decision-making reinforcement learning agent for model learning;

[0085] As another example, in addition to training by using prior knowledge and historical experience and training with the intervention of experts and dispatchers, when the reliability of the dispatch control decision output by the auxiliary decision-making dispatching agent in the face of a small sample scenario is not high, experts and dispatchers can directly intervene in the output of the dispatching task control strategy, and use the expert intervention criteria to determine whether the dispatcher needs to take over. If necessary, the dispatcher outputs the dispatching task control strategy, and then the dispatching task control strategy output by the dispatcher is input into the auxiliary decision-making reinforcement learning agent for model learning, wherein the expert intervention criteria is a human-computer active interaction method, which enables the auxiliary decision-making agent to independently judge the credibility of the dispatching control strategy and realize the function of allowing the agent to actively seek help from the dispatcher.

[0086] Step 105: When the sub-scheduling task control strategy meets the preset credibility requirement, a second scheduling task scheduling plan is output based on the auxiliary decision-making reinforcement learning agent.

[0087] As an example of this embodiment, when the sub-scheduling task control strategy meets the preset credibility requirement, based on the auxiliary decision-making reinforcement learning agent, the first local state of each sub-scheduling task is obtained and the corresponding second sub-scheduling task control strategy is output; based on the digital mirror models of each copy, the first local state of each corresponding sub-scheduling task is executed to obtain the second local state and reward; based on the first local state of each sub-scheduling task, the corresponding second sub-scheduling task control strategy, the second local state and reward, a common experience pool is constructed; based on the common experience pool, the auxiliary decision-making reinforcement learning agent is updated until the preset number of iterations is met, and a number of reward values ​​and a third sub-scheduling task control strategy are obtained; based on the several reward values ​​and the third sub-scheduling task control strategy, a second scheduling task scheduling plan is output.

[0088] A specific implementation method is that the auxiliary decision reinforcement learning agent reads the local state required for each sub-scheduling task (equivalent to the first local state) from each replica digital mirror model, outputs the corresponding sub-scheduling task control strategy (equivalent to the second sub-scheduling task control strategy), and each replica digital mirror model executes the corresponding sub-scheduling task control strategy to obtain a new local state and reward; the local state, sub-scheduling task control strategy, reward and new local state (equivalent to the first local state of each sub-scheduling task, the corresponding second sub-scheduling task control strategy, the second local state and reward) are taken as a set of experience and saved in the experience pool, and the auxiliary decision reinforcement learning agent is updated using the saved experience, and the auxiliary decision reinforcement learning agent is updated until the number of iterations reaches a predetermined number. Output several sub-scheduling task control strategies with the best or sub-optimal reward values ​​from the experience (equivalent to several reward values ​​and the third sub-scheduling task control strategy), select the final sub-scheduling task control strategy according to the actual operating conditions, and multiple final sub-scheduling task control strategies together constitute the scheduling task control strategy, and one of them is used as the scheduling scheme of the new power system.

[0089] As another example of this embodiment, based on the requirements corresponding to the several scheduling subtasks, an auxiliary decision-making agent is constructed; based on the historical operation data and the auxiliary decision-making agent, several operation scenarios are generated; based on the auxiliary decision-making agent and the several operation scenarios, the third local state of each sub-scheduling task is obtained and the fourth sub-scheduling task control strategy is output; based on the respective copy digital mirror models, the corresponding third local state of each sub-scheduling task is executed to obtain the fourth local state and reward; the third local state of each sub-scheduling task, the corresponding fourth sub-scheduling task control strategy, the fourth local state and the reward are saved to the common experience pool; based on the common experience pool, the auxiliary decision-making reinforcement learning agent is updated until the preset model fitting requirements are met to obtain a pre-trained auxiliary decision-making agent. Specifically, the pre-training includes the following steps: (1) Based on the sub-dispatching task requirements, a corresponding new power system mathematical model is constructed, and an auxiliary decision reinforcement learning agent and an auxiliary decision verification agent are constructed respectively; (2) Historical operation data are collected and combined with the mathematical model to generate a typical operation scenario of the new power system; (3) The auxiliary decision reinforcement learning agent reads the local state required by the sub-dispatching task from the replica digital mirror model (equivalent to the third local state) and outputs the corresponding sub-dispatching task control strategy (equivalent to the fourth sub-dispatching task control strategy); (4) Each replica digital mirror model executes the corresponding sub-dispatching task control strategy The local state, sub-scheduling task control strategy, reward and new local state (equivalent to the third local state of each sub-scheduling task, the corresponding fourth sub-scheduling task control strategy, fourth local state and reward) are taken as a set of experience and saved, and the auxiliary decision reinforcement learning agent is reinforced by the saved experience; (5) the above steps (1) to (4) are repeated to update the auxiliary decision reinforcement learning agent until the number of training times reaches a predetermined number; (6) the auxiliary decision verification agent is trained by using the saved experience until the model is just fitted to obtain a pre-trained auxiliary decision agent.

[0090] An interactive method for an auxiliary decision-making agent for power dispatching proposed in an embodiment of the present invention can map multiple types of operating scenarios by determining a dispatching task and decomposing it to obtain a number of dispatching subtasks, thereby improving the credibility of subsequent agent strategy generation; a digital mirror model and a copy digital mirror model are constructed based on the requirements of a number of dispatching subtasks, which can reflect the changing laws of the new power system in the digital space and can deduce and trend the operating status, more accurately simulate the agent interaction environment, and improve the reliability of agent strategy generation; the credibility of the sub-dispatching task control strategy generated by the agent is judged, and historical experience or prior knowledge is provided by intervening in the auxiliary decision-making dispatching agent, so that after the auxiliary decision-making reinforcement learning agent performs exemplary learning, the dispatching task control strategy is finally composed of multiple sub-dispatching task control strategies, and one of them is used as the final dispatching task scheduling plan, thereby enabling the auxiliary decision-making agent to autonomously judge the credibility of the dispatching control strategy, thereby greatly improving the credibility of the output scheduling plan.

[0091] Example 2

[0092] See also Figure 3 , Figure 3 A schematic diagram of the module structure of an intelligent interactive system for auxiliary decision-making of power dispatching provided by an embodiment of the present invention; Figure 3 As shown, an embodiment of the present invention proposes an intelligent interactive system for auxiliary decision-making of power dispatching, including:

[0093] Task acquisition and decomposition module 301, model replication module 302, credibility judgment module 303, first output module 304 and second output module 305;

[0094] The task acquisition and decomposition module 301 is used to build a digital mirror model and obtain real-time operation information of the power grid, determine the scheduling task and decompose the scheduling task into several scheduling subtasks;

[0095] The model replication module 302 is used to replicate the digital mirror model based on the requirements corresponding to the plurality of scheduling subtasks to obtain a plurality of replica digital mirror models;

[0096] The credibility judgment module 303 is used to judge whether the sub-scheduling task control strategy meets the preset credibility requirements based on the auxiliary decision verification agent and the auxiliary decision reinforcement learning agent;

[0097] The first output module 304 is used to output a first scheduling task scheduling solution based on the auxiliary decision reinforcement learning agent and the auxiliary decision scheduling agent when the sub-scheduling task control strategy does not meet the preset credibility requirement;

[0098] As an example of this embodiment, see Figure 4 , Figure 4 A schematic diagram of the structure of a first output module of an electric power dispatch auxiliary decision-making intelligent agent interactive system provided in one embodiment of the present invention; Figure 4 As shown, the first output module 304 is used to output a first scheduling task scheduling solution based on the auxiliary decision reinforcement learning agent and the auxiliary decision scheduling agent when the sub-scheduling task control strategy does not meet the preset credibility requirement, including:

[0099] A first experience pool construction unit 401, a sample extraction unit 402 and a first agent output unit 403;

[0100] The first experience pool construction unit 401 is used to make decisions and adjust the sub-scheduling task control strategy through the auxiliary decision-making scheduling agent when the sub-scheduling task control strategy does not meet the preset credibility requirement, construct a priority experience pool and output the first sub-scheduling task control strategy;

[0101] The sample extraction unit 402 is used to sample the priority experience pool through a preset sampling algorithm to obtain an experience sample; wherein the experience sample includes a first experience sample and a second experience sample; the first experience sample is a sample obtained by extracting a common experience pool; the common experience pool is obtained by constructing a decision-making and regulating sub-scheduling task control strategy through an auxiliary decision-making reinforcement learning agent; the second experience sample is a sample obtained by extracting a priority experience pool;

[0102] The first agent output unit 403 is used to input the experience sample into the auxiliary decision-making reinforcement learning agent based on the first sub-scheduling task control strategy, perform model learning through the auxiliary decision-making reinforcement learning agent, and output the first scheduling task scheduling plan.

[0103] The second output module 305 is used to output a second scheduling task scheduling plan based on the auxiliary decision-making reinforcement learning agent when the sub-scheduling task control strategy meets the preset credibility requirement.

[0104] As an explanation of this embodiment, see Figure 5 , Figure 5 A schematic diagram of the structure of the second output module of a power dispatch auxiliary decision-making intelligent agent interactive system provided by a certain embodiment of the present invention. Figure 5 As shown, the second output module 305 is used to output a second scheduling task scheduling scheme based on the auxiliary decision-making reinforcement learning agent when the sub-scheduling task control strategy meets the preset credibility requirement, including:

[0105] A first local state acquisition unit 501, a second local state acquisition unit 502, a second experience pool construction unit 503, an iterative update unit 504, and a second agent output unit 505;

[0106] The first local state acquisition unit 501 is used to acquire the first local state of each sub-scheduling task and output the corresponding second sub-scheduling task control strategy based on the auxiliary decision-making reinforcement learning agent when the sub-scheduling task control strategy meets the preset credibility requirement;

[0107] The second local state acquisition unit 502 is used to execute the first local state of each corresponding sub-scheduled task based on each replica digital mirror model to obtain a second local state and a reward;

[0108] The second experience pool construction unit 503 is used to construct a common experience pool based on the first local state of each sub-scheduling task, the corresponding second sub-scheduling task control strategy, the second local state and the reward;

[0109] The iterative updating unit 504 is used to update the auxiliary decision reinforcement learning agent based on the common experience pool until the preset number of iterations is met, and obtain a number of reward values ​​and a third sub-scheduling task control strategy;

[0110] The second agent output unit 505 is used to output a second scheduling task scheduling plan based on the multiple reward values ​​and the third sub-scheduling task control strategy.

[0111] An interactive system for an auxiliary decision-making agent for power dispatching proposed in an embodiment of the present invention determines a dispatching task through a task acquisition and decomposition module and decomposes it to obtain a number of dispatching subtasks, which can map to obtain multiple types of operating scenarios, thereby improving the credibility of subsequent agent strategy generation; a digital mirror model and a copy digital mirror model are constructed based on the needs of a number of dispatching subtasks through a model replication module, which can reflect the changing law of a new power system in a digital space and can perform deduction and trend prediction of the operating state, more accurately simulate the agent interaction environment, and improve the reliability of agent strategy generation; the credibility of the sub-dispatching task control strategy generated by the agent is judged through a credibility judgment module, and historical experience or prior knowledge is provided by intervening in the auxiliary decision-making dispatching agent, and the first output module and the second output module are used to enable the auxiliary decision-making reinforcement learning agent to finally form a dispatching task control strategy composed of multiple sub-dispatching task control strategies after model learning, and use one of them as the final dispatching task scheduling plan, thereby enabling the auxiliary decision-making agent to autonomously judge the credibility of the dispatching control strategy, thereby greatly improving the credibility of the output scheduling plan.

[0112] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

[0113] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0114] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

Claims

1. A method for interactive intelligent agent for auxiliary decision making in power dispatching, characterized in that: include: Constructing a digital mirror model and obtaining real-time operation information of the power grid, determining a scheduling task and decomposing the scheduling task into a number of scheduling subtasks; Based on the requirements corresponding to the plurality of scheduling subtasks, the digital mirror model is copied to obtain a plurality of copy digital mirror models; Based on the auxiliary decision verification agent and the auxiliary decision reinforcement learning agent, it is judged whether the sub-scheduling task control strategy meets the preset credibility requirements; When the sub-scheduling task control strategy does not meet the preset credibility requirement, outputting a first scheduling task scheduling plan based on the auxiliary decision reinforcement learning agent and the auxiliary decision scheduling agent; When the sub-scheduling task control strategy meets the preset credibility requirement, a second scheduling task scheduling plan is output based on the auxiliary decision-making reinforcement learning agent.

2. The power dispatching auxiliary decision-making intelligent agent interaction method according to claim 1, characterized in that: Based on the digital mirror model and the real-time operation information of the power grid, a scheduling task is determined and the scheduling task is decomposed into a plurality of scheduling subtasks, including: Based on the digital mirror model, situation awareness and risk assessment are performed on the real-time operation information of the power grid, and events that meet preset assessment requirements are marked as risk accidents; Based on the risk accident, matching the corresponding fault scenario to obtain an accident matching result; Based on the accident matching result, generating a corresponding scheduling task; According to preset different performance index solution requirements, the control action corresponding to the scheduling task is solved by a preset parallel solution algorithm to obtain a number of scheduling subtasks.

3. The power dispatching auxiliary decision-making intelligent agent interaction method according to claim 1, characterized in that: Based on the plurality of scheduling subtasks, the digital mirror model is replicated to obtain a plurality of replica digital mirror models, including: Based on the plurality of scheduling subtasks, different performance indicators are obtained; Based on the different performance indicators, the digital mirror model is copied to obtain a plurality of copy digital mirror models corresponding to each indicator.

4. The method for interactive power dispatching auxiliary decision-making agent according to claim 1, characterized in that: Based on the auxiliary decision verification agent and the auxiliary decision reinforcement learning agent, it is judged whether the sub-scheduling task control strategy meets the preset credibility requirements, including: Based on the auxiliary decision-making reinforcement learning agent, each sub-scheduling task is input into the corresponding replica digital mirror model for interactive operation analysis, and several operation status characteristics and sub-scheduling task control strategies are obtained; Based on the auxiliary decision verification agent, the residual values ​​of the several operating state characteristics and the preset training experience are calculated to obtain the credibility evaluation parameters corresponding to each sub-scheduling task; Based on the credibility evaluation parameters corresponding to each of the sub-scheduling tasks, calculate the corresponding credibility score; Based on the corresponding credibility score, it is determined whether the sub-scheduling task control strategy meets the preset credibility requirement.

5. The method for interactive power dispatching auxiliary decision-making agent according to claim 1, characterized in that: When the sub-scheduling task control strategy does not meet the preset credibility requirement, based on the auxiliary decision reinforcement learning agent and the auxiliary decision scheduling agent, a first scheduling task scheduling scheme is output, including: When the sub-scheduling task control strategy does not meet the preset credibility requirement, the sub-scheduling task control strategy is decided and regulated by the auxiliary decision-making scheduling agent, a priority experience pool is constructed, and the first sub-scheduling task control strategy is output; The priority experience pool is sampled by a preset sampling algorithm to obtain an experience sample; wherein the experience sample includes a first experience sample and a second experience sample; the first experience sample is a sample obtained by extracting a common experience pool; the common experience pool is constructed by making decisions and regulating the sub-scheduling task control strategy through an auxiliary decision-making reinforcement learning agent; the second experience sample is a sample obtained by extracting a priority experience pool; Based on the first sub-scheduling task control strategy, the experience sample is input into the auxiliary decision-making reinforcement learning agent, model learning is performed by the auxiliary decision-making reinforcement learning agent, and a first scheduling task scheduling plan is output.

6. The power dispatching auxiliary decision-making intelligent agent interaction method according to claim 5, characterized in that: When the sub-scheduling task control strategy meets the preset credibility requirement, a second scheduling task scheduling scheme is output based on the auxiliary decision-making reinforcement learning agent, including: When the sub-scheduling task control strategy meets the preset credibility requirement, based on the auxiliary decision-making reinforcement learning agent, the first local state of each sub-scheduling task is obtained and the corresponding second sub-scheduling task control strategy is output; Based on the digital mirror models of each replica, execute the first local state of each corresponding sub-scheduling task to obtain the second local state and reward; Building a common experience pool based on the first local state of each sub-scheduling task, the corresponding second sub-scheduling task control strategy, the second local state and the reward; Based on the common experience pool, the auxiliary decision-making reinforcement learning agent is updated until the preset number of iterations is met, and a number of reward values ​​and a third sub-scheduling task control strategy are obtained; Based on the several reward values ​​and the third sub-scheduling task control strategy, a second scheduling task scheduling plan is output.

7. A method for interactive power dispatching auxiliary decision-making agent according to any one of claim 6, characterized in that: Also includes: Based on the requirements corresponding to the several scheduling subtasks, construct an auxiliary decision-making intelligent agent; Based on the historical operation data and the auxiliary decision-making agent, generate several operation scenarios; Based on the auxiliary decision-making agent and the several operation scenarios, obtaining a third local state of each sub-scheduling task and outputting a fourth sub-scheduling task control strategy; Based on the digital mirror models of each replica, execute the third local state of each corresponding sub-scheduling task to obtain a fourth local state and a reward; The third local state of each sub-scheduling task, the corresponding fourth sub-scheduling task control strategy, the fourth local state and the reward are saved in the common experience pool; Based on the common experience pool, the auxiliary decision-making reinforcement learning agent is updated until it meets the preset model fitting requirements to obtain a pre-trained auxiliary decision-making agent.

8. An intelligent interactive system for auxiliary decision-making of power dispatching, characterized in that: Executing a power dispatch auxiliary decision-making intelligent agent interaction method as described in any one of claims 1 to 7, comprising: Task acquisition and decomposition module, model replication module, credibility judgment module, first output module and second output module; The task acquisition and decomposition module is used to build a digital mirror model and obtain real-time operation information of the power grid, determine the scheduling task and decompose the scheduling task into several scheduling subtasks; The model replication module is used to replicate the digital mirror model based on the requirements corresponding to the plurality of scheduling subtasks to obtain a plurality of replica digital mirror models; The credibility judgment module is used to judge whether the sub-scheduling task control strategy meets the preset credibility requirements based on the auxiliary decision verification agent and the auxiliary decision reinforcement learning agent; The first output module is used to output a first scheduling task scheduling plan based on the auxiliary decision reinforcement learning agent and the auxiliary decision scheduling agent when the sub-scheduling task control strategy does not meet the preset credibility requirement; The second output module is used to output a second scheduling task scheduling plan based on the auxiliary decision-making reinforcement learning agent when the sub-scheduling task control strategy meets the preset credibility requirement.

9. The electric power dispatching auxiliary decision-making intelligent agent interactive system according to claim 8, characterized in that: The first output module is used to output a first scheduling task scheduling solution based on the auxiliary decision reinforcement learning agent and the auxiliary decision scheduling agent when the sub-scheduling task control strategy does not meet the preset credibility requirement, including: A first experience pool construction unit, a sample extraction unit and a first agent output unit; The first experience pool construction unit is used to make decisions and adjust the sub-scheduling task control strategy through the auxiliary decision-making scheduling agent when the sub-scheduling task control strategy does not meet the preset credibility requirement, construct a priority experience pool and output the first sub-scheduling task control strategy; The sample extraction unit is used to sample the priority experience pool through a preset sampling algorithm to obtain an experience sample; wherein the experience sample includes a first experience sample and a second experience sample; the first experience sample is a sample obtained by extracting a common experience pool; the common experience pool is constructed by making decisions and regulating the sub-scheduling task control strategy through an auxiliary decision-making reinforcement learning agent; the second experience sample is a sample obtained by extracting a priority experience pool; The first agent output unit is used to input the experience sample into the auxiliary decision-making reinforcement learning agent based on the first sub-scheduling task control strategy, perform model learning through the auxiliary decision-making reinforcement learning agent, and output a first scheduling task scheduling plan.

10. The electric power dispatching auxiliary decision-making intelligent agent interactive system according to claim 9, characterized in that: The second output module is used to output a second scheduling task scheduling plan based on the auxiliary decision-making reinforcement learning agent when the sub-scheduling task control strategy meets the preset credibility requirement, including: A first local state acquisition unit, a second local state acquisition unit, a second experience pool construction unit, an iterative update unit, and a second agent output unit; The first local state acquisition unit is used to acquire the first local state of each sub-scheduling task and output the corresponding second sub-scheduling task control strategy based on the auxiliary decision-making reinforcement learning agent when the sub-scheduling task control strategy meets the preset credibility requirement; The second local state acquisition unit is used to execute the first local state of each corresponding sub-scheduling task based on each replica digital mirror model to obtain a second local state and a reward; The second experience pool construction unit is used to construct a common experience pool based on the first local state of each sub-scheduling task, the corresponding second sub-scheduling task control strategy, the second local state and the reward; The iterative updating unit is used to update the auxiliary decision reinforcement learning agent based on the common experience pool until the preset number of iterations is met, and obtain a number of reward values ​​and a third sub-scheduling task control strategy; The second agent output unit is used to output a second scheduling task scheduling plan based on the several reward values ​​and the third sub-scheduling task control strategy.

Images