Multi-agent collaborative optimization scheduling method and device of power system and electronic equipment

Abstract

The invention relates to the technical field of novel power system dispatching, in particular to a multi-agent collaborative optimization dispatching method and device of a power system and electronic equipment. The method comprises the following steps: acquiring current operation section data of a power system; performing scene identification on the current operation section data by using a preset scene identification model to obtain an operation scene identification result, the preset scene identification model being obtained by training historical operation section data of the power system; and according to the operation scene identification result and a plurality of preset sub-scheduling intelligent agents, generating a current optimization scheduling strategy, and adjusting the operation state of the power system based on the current optimization scheduling strategy. Therefore, the scene to which the real-time operation section belongs is sensed in advance, and the corresponding sub-scheduling agent is coordinated to quickly solve the real-time scheduling task, so that the problem that the flexibility and the stability of a real-time scheduling model are insufficient due to the fact that the difference of the operation scene is not considered in the existing centralized single-agent scheduling modeling technology is solved.

Description

Technical Field

[0001] The present application relates to the field of novel power system dispatching technology, and in particular to a multi-agent collaborative optimization dispatching method, device and electronic equipment for a power system. Background Art

[0002] Power system optimization and dispatch involves formulating reasonable unit startup and shutdown and power generation plans within a specific dispatch cycle, while meeting power system security constraints and minimizing costs. However, with the increasing integration of high-proportion renewable energy, the uncertainty, volatility, and intermittency of renewable energy output are making power system optimization and dispatch increasingly difficult.

[0003] In related technologies, reinforcement learning (RL) technology is usually combined to use powerful learning and logical reasoning capabilities to provide more possibilities for optimizing power system scheduling.

[0004] However, this method mainly learns a single dispatching agent based on a complete data set, ignoring the differences in typical operating scenarios caused by source-load uncertainty or emergency operation of the power grid, resulting in insufficient flexibility and stability of the real-time dispatching model, which needs to be urgently addressed. Summary of the Invention

[0005] The present application provides a multi-agent collaborative optimization scheduling method, device and electronic equipment for an electric power system to solve the problem that the existing centralized, single-agent scheduling modeling technology lacks flexibility and stability of the real-time scheduling model due to the lack of consideration of the differences in operating scenarios, so as to improve the operating efficiency and stability of the entire electric power system.

[0006] To achieve the above objectives, the first embodiment of the present application proposes a multi-agent collaborative optimization scheduling method for a power system, comprising the following steps:

[0007] Obtain current operating section data of the power system;

[0008] Performing scene recognition on the current operating section data using a preset scene recognition model to obtain an operating scene recognition result, wherein the preset scene recognition model is trained using historical operating section data of the power system;

[0009] According to the operation scenario identification result and multiple preset sub-dispatching agents, a current optimization dispatching strategy is generated, and the operation state of the power system is adjusted based on the current optimization dispatching strategy.

[0010] According to one embodiment of the present application, generating a current optimized scheduling strategy based on the operating scenario identification result and a plurality of preset sub-scheduling agents includes:

[0011] Determine a target preset sub-scheduling agent from the multiple preset sub-scheduling agents according to the identified result of the operating scenario;

[0012] Observe the current operating environment state of the power system, and based on the current operating environment state, use the target preset sub-scheduling agent to generate the current optimal scheduling strategy.

[0013] According to an embodiment of the present application, before using the preset scenario identification model to perform scenario identification on the current operating section data to obtain the identified result of the operating scenario, it further includes:

[0014] Obtain historical operating section data of different operating scenarios of the power system, where the historical operating section data includes scenario true labels;

[0015] Perform normalization processing on the historical operating section data, and divide the normalized historical operating section data into a training set and a verification set;

[0016] Input the training set into a preset neural network, use the forward propagation algorithm to calculate the first scenario prediction label to obtain an initial scenario identification model, and input the verification set into the initial scenario identification model to obtain a second scenario prediction label;

[0017] Perform cross-entropy loss calculation based on the second scenario prediction label and the scenario true label to obtain a calculation result, and determine whether the calculation result meets a preset standard;

[0018] If the calculation result meets the preset standard, end the iterative training of the preset neural network, and use the initial scenario identification model as the preset scenario identification model; otherwise, use the backpropagation algorithm to calculate the gradient of the loss function with respect to the model parameters, and update the model parameters using the gradient descent algorithm based on the gradient of the loss function with respect to the model parameters, and then continue the iterative training.

[0019] According to an embodiment of the present application, before generating the current optimal scheduling strategy according to the identified result of the operating scenario and the multiple preset sub-scheduling agents, it further includes:

[0020] Perform in-depth time series aggregation analysis on the historical operating section data based on the feature iterative aggregation algorithm to obtain different types of operating scenario sets;

[0021] Use the entropy regularization deep reinforcement learning algorithm to construct the multiple preset sub-scheduling agents according to the different types of operating scenario sets.

[0022] According to an embodiment of the present application, the construction of the multiple preset sub-scheduling agents based on the different types of operation scenario sets by using the entropy regularization deep reinforcement learning algorithm includes:

[0023] Determine an initial sub-scheduling agent corresponding to each type of operation scenario set according to the different types of operation scenario sets;

[0024] Construct a power system simulation environment model, and based on the operation environment state, power flow distribution algorithm, and preset constraint conditions of the power system simulation environment model, use the initial sub-scheduling agent corresponding to each type of operation scenario set to generate a corresponding optimal scheduling strategy;

[0025] Use a reward function to perform reward feedback on the corresponding optimal scheduling strategy, and use the reward feedback as the update guidance for model parameters until the initial sub-scheduling agents corresponding to each type of operation scenario set converge to a preset maximum cumulative reward, obtaining the multiple preset sub-scheduling agents.

[0026] According to the multi-agent collaborative optimal scheduling method of the power system proposed in the embodiment of the present application, by acquiring the real-time operation data of the power system, using the trained scenario identification model to analyze the real-time data to identify the current operation scenario, and formulating the current optimal scheduling strategy according to the identified current operation scenario and the corresponding sub-scheduling agent, so as to adjust the operation state of the power system according to the optimal scheduling strategy. Thus, the problem that the existing centralized and single-agent scheduling modeling technologies lack consideration of operation scenario differences, resulting in insufficient flexibility and stability of the real-time scheduling model, is solved.

[0027] To achieve the above object, an embodiment of the second aspect of the present application proposes a multi-agent collaborative optimal scheduling device for a power system, including:

[0028] An acquisition module, configured to acquire the current operation section data of the power system;

[0029] An identification module, configured to perform scenario identification on the current operation section data by using a preset scenario identification model to obtain a running scenario identification result, where the preset scenario identification model is trained by the historical operation section data of the power system;

[0030] A generation module, configured to generate a current optimal scheduling strategy according to the running scenario identification result and multiple preset sub-scheduling agents, and adjust the operation state of the power system based on the current optimal scheduling strategy.

[0031] According to an embodiment of the present application, the generation module is specifically configured to:

[0032] Determine a target preset sub-scheduling agent from the multiple preset sub-scheduling agents according to the operation scenario identification result;

[0033] Observe the current operating environment state of the power system, and based on the current operating environment state, use the target preset sub-scheduling agent to generate the current optimal scheduling strategy.

[0034] According to an embodiment of the present application, before using the preset scenario identification model to perform scenario identification on the current operating section data to obtain the operation scenario identification result, the identification module is further configured to:

[0035] Obtain historical operating section data of different operating scenarios of the power system, where the historical operating section data includes scenario true labels;

[0036] Perform standardization processing on the historical operating section data, and divide the standardized historical operating section data into a training set and a validation set;

[0037] Input the training set into a preset neural network, use the forward propagation algorithm to calculate the first scenario prediction label to obtain an initial scenario identification model, and input the validation set into the initial scenario identification model to obtain a second scenario prediction label;

[0038] Perform cross-entropy loss calculation based on the second scenario prediction label and the scenario true label to obtain a calculation result, and determine whether the calculation result meets a preset standard;

[0039] If the calculation result meets the preset standard, end the iterative training of the preset neural network, and use the initial scenario identification model as the preset scenario identification model; otherwise, calculate the gradient of the loss function with respect to the model parameters using the backpropagation algorithm, and update the model parameters using the gradient descent algorithm based on the gradient of the loss function with respect to the model parameters, and then continue the iterative training.

[0040] According to an embodiment of the present application, before generating the current optimal scheduling strategy according to the operation scenario identification result and the multiple preset sub-scheduling agents, the generating module further includes:

[0041] An aggregation analysis unit for performing in-depth time-series aggregation analysis on the historical operating section data based on the feature iterative aggregation algorithm to obtain different types of operating scenario sets;

[0042] A construction unit for constructing the multiple preset sub-scheduling agents according to the different types of operating scenario sets using the entropy regularization deep reinforcement learning algorithm.

[0043] According to an embodiment of the present application, the construction unit is specifically configured to:

[0044] Determine initial sub-scheduling agents corresponding to each type of operating scenario set according to the different types of operating scenario sets;

[0045] Construct a power system simulation environment model, and based on the operating environment state, power flow distribution algorithm, and preset constraint conditions of the power system simulation environment model, use the initial sub-scheduling agents corresponding to each type of operating scenario set to generate corresponding optimal scheduling strategies;

[0046] Use a reward function to perform reward feedback on the corresponding optimal scheduling strategies, and use the reward feedback as the update guidance for model parameters until the initial sub-scheduling agents corresponding to each type of operating scenario set converge to a preset maximum cumulative reward, obtaining the multiple preset sub-scheduling agents.

[0047] The multi-agent collaborative optimal scheduling device for a power system proposed in the embodiment of the present application analyzes real-time data using a trained scenario identification model by obtaining real-time operation data of the power system to identify the current operating scenario, and formulates the current optimal scheduling strategy according to the identified current operating scenario and the corresponding sub-scheduling agent, so as to adjust the operating state of the power system according to the optimal scheduling strategy. Thus, the problem that the existing centralized and single-agent scheduling modeling technologies lack flexibility and stability of the real-time scheduling model due to lack of consideration of operating scenario differences is solved.

[0048] To achieve the above object, an embodiment of the third aspect of the present application proposes an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the program to implement the multi-agent collaborative optimal scheduling method for a power system as described in the above embodiment.

[0049] To achieve the above object, an embodiment of the fourth aspect of the present application proposes a computer-readable storage medium, on which a computer program is stored, and the program is executed by a processor to be used to implement the multi-agent collaborative optimal scheduling method for a power system as described in the above embodiment.

[0050] To achieve the above object, an embodiment of the fifth aspect of the present application proposes a computer program product, which includes a computer program, and when the computer program is executed by a processor, it is used to implement the multi-agent collaborative optimal scheduling method for a power system as described in the above embodiment.

[0051] Additional aspects and advantages of the present application will be given in part in the following description, become apparent in part from the following description, or be understood through the practice of the present application. Description of the Drawings

[0052] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description of embodiments in conjunction with the drawings, where:

[0053] Figure 1 It is a flowchart of a multi-agent collaborative optimization scheduling method for a power system provided according to an embodiment of the present application;

[0054] Figure 2 It is a schematic diagram of the stage division of a multi-agent collaborative optimization scheduling method for a power system according to an embodiment of the present application;

[0055] Figure 3 It is a schematic diagram of the construction process of a sub-scheduling agent based on the entropy-regularized DRL (Deep Reinforcement Learning) algorithm according to an embodiment of the present application;

[0056] Figure 4 It is a schematic diagram of the pseudocode of the entropy-regularized DRL algorithm according to an embodiment of the present application;

[0057] Figure 5 It is a flowchart of the execution of a multi-agent scheduling coordination algorithm according to an embodiment of the present application;

[0058] Figure 6 It is a schematic diagram of the confusion matrix of a preset scenario identification model test set according to an embodiment of the present application;

[0059] Figure 7 It is a schematic diagram of the network topology of an experimental power grid according to an embodiment of the present application;

[0060] Figure 8 It is a schematic diagram of an experiment on the stability analysis of multi-agent collaborative optimization scheduling according to an embodiment of the present application;

[0061] Figure 9 It is a block diagram of a multi-agent collaborative optimization scheduling device for a power system provided according to an embodiment of the present application;

[0062] Figure 10 It is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. Detailed Description of the Embodiments

[0063] The embodiments of the present application will be described in detail below. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals denote the same or similar elements or elements with the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary and are intended to explain the present application, and should not be construed as limiting the present application.

[0064] The multi-agent collaborative optimal scheduling method, device and electronic device of the power system according to the embodiments of the present application will be described below with reference to the accompanying drawings.

[0065] Figure 1 It is a flowchart of the multi-agent collaborative optimal scheduling method of the power system according to an embodiment of the present application.

[0066] Before introducing the multi-agent collaborative optimal scheduling method of the power system proposed by the embodiments of the present application, the relevant technical background will be introduced first.

[0067] It can be understood that the optimal scheduling of the power system needs to combine the load demand and the available generating unit capacity to determine the optimal generating unit combination and power generation amount to meet the load demand, and consider economic, safety and environmental factors, etc.

[0068] Traditional typical optimal scheduling methods can be summarized into the following three categories: the deterministic optimization (DO) method. Since it requires all variables in the system to be known and without uncertainty, it is difficult to ensure system stability and the operating economy of the generating units; the stochastic optimization (SO) and robust optimization (RO) and their extended methods are more commonly used, and each has its own advantages in dealing with uncertainty problems. However, in large-scale real-time scheduling problems, they cannot immediately obtain results and the calculation time is long.

[0069] In recent years, the learning-based optimization method has been the most widely used. This type of method makes full use of the experience in historical operation data to learn strategies. In particular, its potential in dealing with sequential decision-making problems such as near-real-time scheduling has been recognized by the research community. As an important scientific paradigm for solving multi-stage sequential decision-making problems, RL technology can assist the power system scheduling to realize self-exploration and update of scheduling strategies. In particular, the deep reinforcement learning technology combined with deep learning provides more possibilities for power grid optimal scheduling with its powerful learning and logical deduction capabilities, and significantly improves the speed of decision-making.

[0070] However, in terms of scheduling modeling and solving, the existing RL algorithms mainly learn a single scheduling agent based on a complete data set. Although the single-task modeling method can learn the variation law of historical operation sections and generate scheduling strategies in a timely manner, it ignores the differences in typical operation scenarios caused by source-load uncertainty or emergency operation of the power grid, resulting in insufficient adaptability of the scheduling agent to complex and changeable operation scenarios, and it is difficult to ensure the stable strategy optimality (low system operation cost, high new energy consumption) of the decision results generated.

[0071] It is precisely based on the above problems that the embodiment of the present application proposes a multi-agent collaborative optimization scheduling method for the power system. By pre-perceiving the scenario to which the real-time operation section belongs, the corresponding sub-scheduling agent is coordinated to quickly solve the real-time scheduling task, thereby solving the problem of insufficient flexibility and stability of the real-time scheduling model due to the lack of consideration of the differences in operating scenarios in the existing centralized and single-agent scheduling modeling technology.

[0072] Further, combined with Figure 2 As shown, the embodiment of the present application can be divided into three stages, namely, extraction of differentiated operation scenario sets, multi-task optimization scheduling modeling and multi-agent coordination. First, the differentiated operation scenario set extraction stage belongs to the offline learning stage. In this stage, different typical scenario data sets can be extracted based on historical operation section data (the collection interval can be 5 minutes). Each typical scenario data set is used as the initial state of the interaction between the sub-dispatching agent and the power system simulation environment to construct a training data set for the subsequent multi-agent. In the multi-task optimization scheduling modeling stage, by using different operation scenario data sets to independently train each sub-dispatching agent, a multi-task optimization scheduling model with integrated scenario differentiation can be constructed. The multi-agent coordination stage is the online decision-making stage. In this stage, the scenario category to which the scheduling task belongs can be identified based on the real-time operation section changes (the time interval can be 5 minutes), and the mature sub-dispatching agents can be coordinated and matched, so as to provide targeted optimization scheduling strategies.

[0073] Next, the multi-agent collaborative optimization scheduling method of the power system will be explained in detail.

[0074] For example, Figure 1 As shown in FIG, the multi-agent collaborative optimization scheduling method of the power system includes the following steps:

[0075] In step S101 , current operation section data of the power system is acquired.

[0076] It's understood that operational cross-sectional data refers to information about the power system's operating status at a specific moment. This data can include parameters such as power generation, transmission, power consumption, grid frequency, voltage level, and equipment status. This data can be collected through various sensors, measuring devices, and data acquisition systems. By acquiring the power system's current operational cross-sectional data, we can understand the system's real-time operating status.

[0077] In step S102 , a preset scene recognition model is used to perform scene recognition on the current operating section data to obtain an operating scene recognition result, wherein the preset scene recognition model is trained by historical operating section data of the power system.

[0078] That is to say, the current operating section data of the power system (i.e., the change of real-time operating section data with a time interval of 5 minutes) is input into the already trained scenario identification model (i.e., the preset scenario identification model). This model can output the operation scenario identification result for the current operating section data by deeply analyzing the current operating section data. In real-time scheduling tasks, operation scenario identification is the key to transforming traditional single-task scheduling modeling into multi-agent collaborative scheduling decision-making. Among them, the preset scenario identification model is formed through training and optimization with a large amount of historical data. They can capture various typical patterns and characteristics in the operation of the power system, thus providing strong support for real-time monitoring and decision-making.

[0079] In step S103, based on the operation scenario identification result and multiple preset sub-scheduling agents, a current optimized scheduling strategy is generated, and the operation state of the power system is adjusted based on the current optimized scheduling strategy.

[0080] That is to say, through the operation scenario identification result and multiple preset sub-scheduling agents, an optimized scheduling strategy most suitable for the current operation scenario can be formulated. Subsequently, based on this optimized scheduling strategy, the operation state of the power system can be adjusted and optimized accordingly to ensure the efficient and stable operation of the power system.

[0081] As a possible implementation method, in some embodiments, generating a current optimized scheduling strategy based on the operation scenario identification result and multiple preset sub-scheduling agents includes: determining a target preset sub-scheduling agent from multiple preset sub-scheduling agents according to the operation scenario identification result; observing the current operation environment state of the power system, and generating a current optimized scheduling strategy by using the target preset sub-scheduling agent based on the current operation environment state.

[0082] Specifically, since each preset sub-scheduling agent focuses on the optimized scheduling of a specific operation scenario, according to the operation scenario identification result in step S102, a preset sub-scheduling agent corresponding to the current operation scenario can be selected as the target preset sub-scheduling agent. By observing the current operation environment state of the power system (such as factors like grid load, power generation capacity, energy storage situation, and possible external disturbances), the target preset sub-scheduling agent can quickly generate an optimized scheduling strategy for the current operation scenario. This strategy aims to maximize the operation efficiency of the power system while ensuring the reliability and economy of power supply, as well as meeting environmental and policy requirements.

[0083] To achieve the identification of dispatching tasks based on the operation sections in time series, the embodiments of the present application can construct a multi-layer perceptron network to adaptively learn the complex non-linear relationship between the cross-time operation section data and the typical operation scenario labels, so as to flexibly cope with the complex and changeable operation scenario switching during the operation of the power system. For ease of understanding, the following will introduce in detail how to obtain the preset scenario identification model.

[0084] In some embodiments, before using the preset scenario identification model to identify the current operation section data and obtain the operation scenario identification result, it further includes: obtaining the historical operation section data of different operation scenarios of the power system, where the historical operation section data includes the true scenario labels; performing standardization processing on the historical operation section data, and dividing the standardized historical operation section data into a training set and a validation set; inputting the training set into the preset neural network, calculating the first scenario prediction label using the forward propagation algorithm to obtain the initial scenario identification model, and inputting the validation set into the initial scenario identification model to obtain the second scenario prediction label; calculating the cross-entropy loss based on the second scenario prediction label and the true scenario label to obtain the calculation result, and determining whether the calculation result meets the preset standard; if the calculation result meets the preset standard, end the iterative training of the preset neural network, and use the initial scenario identification model as the preset scenario identification model, otherwise calculate the gradient of the loss function with respect to the model parameters using the backpropagation algorithm, and continue the iterative training after updating the model parameters based on the gradient of the loss function with respect to the model parameters using the gradient descent algorithm.

[0085] Specifically, first, the embodiments of the present application can select the "snapshot" of one or several time points from a "data archive" that records the operation states at different times to view the operation status of the power system, that is, obtain the time series historical operation section data of different operation scenarios of the power system in a long time period. Among them, the historical operation section data of each operation scenario contains the corresponding scenario label i, that is:

[0086]

[0087] Among them, is all conventional units at time t, is all new energy units at time t, is the active power of all loads at time t, N Tgen 、N Rgen 、N Ld are the maximum values of their quantities respectively.

[0088] The following formula is used to perform standardization processing on the historical operation section data to eliminate the dimension difference between different features and accelerate the convergence of the model, that is:

[0089]

[0090] Among them, is the original eigenvalue of the historical operation section data after standardization, X is the original eigenvalue of the historical operation section data, X μ is the mean value of the corresponding historical operation section data, X σ is the standard deviation of the corresponding historical operation section data.

[0091] Next, construct a Multi-Layer Perceptron (MLP) model for the operation scenario identification task. First, define the network structure (i.e., the number of neurons), including the input layer (the number of neurons in the input layer corresponds to the feature dimension of the input operation section data), the first hidden layer (including 200 neurons), the second hidden layer (including 75 neurons), and the output layer (the number of neurons corresponds to the total number of operation scenario types l). Second, in the neural network, the activation function is used to introduce non-linearity, enabling the model to learn and simulate more complex function mappings. In the embodiments of the present application, the ReLU (Rectified Linear Unit) activation function is adopted. This function is a commonly used activation function that can help alleviate the vanishing gradient problem. Finally, the weight and bias parameters of the network can be set by random initialization, laying the foundation for the subsequent training process, thereby obtaining a preset neural network.

[0092] After setting the network structure, divide the standardized historical operation section data into a training set and a validation set, and input the training set into the preset neural network. Calculate the scenario prediction label (i.e., the first scenario prediction label) of the model through the forward propagation algorithm to learn the non-linear relationship between the historical operation section data and the typical operation scenario labels, obtaining an initial scenario identification model. Then, input the validation set into the initial scenario identification model, and the scenario prediction label (i.e., the second scenario prediction label) of the model based on the validation set can be output, and calculate the cross-entropy loss between the second scenario prediction label and the scenario true label (a method to measure the difference between the predicted probability distribution and the true label probability distribution). The loss function is defined as:

[0093]

[0094] where y ti is the true label (i.e., the scenario true label) of the operation section data at time t belonging to scenario i, is the predicted probability that the initial scenario identification model assigns the operation section data at time t to scenario i. Since the scenario category labels are in one-hot encoding form, y ti and are both vectors of length l.

[0095] According to the obtained cross-entropy loss calculation result, subsequently, a judgment is made on this calculation result to determine whether it meets the preset standard. If the calculation result meets the preset standard, then the iterative training process of the preset neural network can be stopped, and the initial scene recognition model can be determined as the final scene recognition model, that is, the preset scene recognition model. However, if the calculation result does not meet the preset standard, then further steps can be taken. Specifically, the embodiments of the present application can use the backpropagation algorithm to calculate the gradients of the loss function with respect to the model parameters (such as weights and biases). By calculating these gradients, it can be understood how to adjust the parameters to reduce the error. Based on this gradient information, the gradient descent algorithm (used to update the model parameters through an iterative process with the aim of minimizing the loss function) is used to update the parameters of the model. After completing the parameter update, continue with the iterative training. Each iteration will update the parameters in the direction where the loss function decreases fastest (i.e., the opposite direction of the gradient), thereby continuously reducing the error of the model until the performance of the model meets the preset standard. The trained multi-layer perceptron scene recognition model (i.e., the preset scene recognition model) can identify the operation scene category of the current moment's operation section data, laying a foundation for subsequently matching the preset sub-scheduling agents according to the scene labels and quickly solving the real-time scheduling task.

[0096] Thus, it can be seen that the preset scene recognition model can simulate the thinking and judgment of dispatchers in reality, pre-perceive the scene labels to which the real-time operation section belongs, and actively coordinate the corresponding sub-scheduling agents to solve the real-time scheduling task. The scene recognition model proposed based on the multi-layer perceptron has a high typical operation scene recognition ability after differential training, and can effectively improve the solution efficiency of multi-agent coordination.

[0097] Next, a detailed description will be given on how to construct sub-scheduling agents that can meet different operation scenarios.

[0098] In some embodiments, before generating the current optimal scheduling strategy according to the operation scene recognition result and multiple preset sub-scheduling agents, it further includes: performing in-depth temporal aggregation analysis on the historical operation section data based on the feature iterative aggregation algorithm to obtain different types of operation scene sets; using the entropy regularization deep reinforcement learning algorithm to construct multiple preset sub-scheduling agents according to different types of operation scene sets.

[0099] Specifically, in order to achieve cross-time-scale aggregation while ensuring the integrity of power system operation section data, a differential operation scenario generation algorithm based on feature iterative aggregation (aggregating and processing data features through an iterative method) can be used to conduct in-depth time-series aggregation analysis on the historical operation section data of time series within a long time period, mainly focusing on the changes in load fluctuations (i.e., electricity demand) during the operation of the power system, as well as the instability of new energy power generation, to achieve aggregation in the feature space dimension and obtain different types of operation scenario sets.

[0100] Thus, by capturing the spatio-temporal characteristics of time-series operation section data and achieving aggregation in the feature space dimension, classifying historical data on a long time scale into a set of differential operation scenarios with typical distribution characteristics can effectively solve the problem of difficult fine control of complex and changeable operation scenarios dominated by new energy.

[0101] Considering the characteristics of the new power system, such as nonlinearity, uncertainty, and strong randomness, as well as the requirement for the continuity of the action space, the embodiment of this application proposes a deep reinforcement learning algorithm based on entropy regularization. According to different types of operation scenario sets, each sub-scheduling agent is trained to obtain multiple preset sub-scheduling agents.

[0102] As a possible implementation, in some embodiments, using the entropy regularization deep reinforcement learning algorithm, multiple preset sub-scheduling agents are constructed according to different types of operation scenario sets, including: determining the initial sub-scheduling agent corresponding to each type of operation scenario set according to different types of operation scenario sets; constructing a power system simulation environment model, and based on the operation environment state, power flow distribution algorithm, and preset constraint conditions of the power system simulation environment model, using the initial sub-scheduling agent corresponding to each type of operation scenario set to generate the corresponding optimal scheduling strategy; using the reward function to perform reward feedback on the corresponding optimal scheduling strategy, and using the reward feedback as the update guidance for model parameters until the initial sub-scheduling agent corresponding to each type of operation scenario set converges to the preset maximum cumulative reward, obtaining multiple preset sub-scheduling agents.

[0103] It can be understood that the embodiment of this application trains each sub-scheduling agent from the data source to the state-action differentiation. Each sub-agent interacts with the power system simulation environment model, observes the operation environment state, takes the optimal action at the current training time step, and obtains the corresponding feedback reward result as the update guidance for model parameters until the sub-scheduling agent can stably converge to the maximum cumulative reward. Among them, considering various situations that the power system may face in actual operation and ensuring that the trained sub-scheduling agents have strong stability and robustness, typical operation conditions such as severe fluctuations in source and load, N-1 faults in tie lines, tie line congestion, and curtailment of new energy are introduced into the power system simulation environment model.

[0104] As Figure 3 shown, the training process of the sub-scheduling agent can be described as a Markov decision process, centered around the interaction between the agent and the power system simulation environment model. By observing the environmental state space s t at time t, the sub-scheduling agent can output the corresponding action space a t . After receiving the executed action, the environment model can generate a new environmental state s t+1 according to the state transition probability function. At the same time, the environment model will perform a power flow calculation on s t+1 in order to solve the corresponding reward value r t for the executed action space a t .

[0105] Next, the power system simulation environment model, state space, action space, reward function, and entropy regularization DRL algorithm involved will be elaborated in detail respectively.

[0106] (1) Power system simulation environment model: Different from supervised learning and unsupervised learning, reinforcement learning does not require any data to be given in advance, but obtains learning information and updates the DRL decision model parameters by receiving the rewards (feedback) of the environment for actions. Therefore, for the autonomous exploration and decision-making of the power system scheduling strategy in a certain region, the construction of its power system simulation environment model is one of the key points of the maturity of the sub-scheduling agent.

[0107] 1) The power flow distribution algorithm is an important link in power system analysis, involving the calculation of the voltage of each node, the current of each branch, and the power distribution in the power system under stable operating conditions. Given the power system network topology, component parameters, and load conditions, the nonlinear algebraic equations describing the steady-state operating characteristics of the system are solved by numerical calculation methods, so as to obtain the steady-state operating data such as the voltage amplitude and phase angle of the entire power system, the branch power distribution, and the network loss. The entire calculation process needs to comprehensively consider various factors such as generator output, load demand, and the impedance of transmission lines, and establish corresponding mathematical models to describe them. For example, for each node, the difference between the injection and consumption of active power and reactive power should be equal to the total power flowing in and out through the connecting lines. Mathematically, it can be expressed as:

[0108]

[0109] where P i , Q i are the active power and reactive power (the difference between injection and consumption) of node i respectively, V i , V j are the voltage amplitudes of node i and node j respectively, θ ij is the voltage phase difference between node i and node j, Gij is the real part of the network admittance matrix, and B ij is the imaginary part of the network admittance matrix, and n is the total number of nodes.

[0110] As can be seen from the above, power flow calculation can assist in determining the voltage magnitudes and phases of each node in the power system and the power flow on each transmission line after implementing the scheduling strategy, and evaluating the security of the strategy, that is, explaining the feasibility of the strategy from a mechanism perspective.

[0111] In the multi-agent collaborative optimization scheduling of the embodiments of the present application, the power flow distribution algorithm (power flow calculation) plays a core role, providing the necessary power system state information and physical constraint conditions for the learning of multiple scheduling agents. Whenever a sub-scheduling agent takes an action, the power flow distribution algorithm can provide the immediate result of its action and the new state of the operating environment, and check for voltage and line power over-limit conditions, calculate network losses, etc. These information are beneficial for the agent to learn how to make better decisions. Considering power flow calculation during the training process of the sub-scheduling agent, the sub-scheduling agent can receive accurate feedback based on the physical constraints of the power grid after each decision, ensuring that each scheduling strategy takes into account the real-time state and constraint conditions of the power system. In this way, each strategy and result are closely connected to the actual physical laws of the power system, not only enhancing the interpretability of the decision-making process of the sub-scheduling agent, but also effectively improving the security of the scheduling scheme because all decisions learned by the policy network of the sub-scheduling agent are based on real data and models that ensure the stability and security of the power grid. Therefore, the deep reinforcement learning scheduling agent integrating power flow calculation can significantly improve the interpretability of decision-making intelligence technology and the security of power scheduling strategies, thus better serving complex and dynamic power systems.

[0112] Therefore, based on Equation (5), the power flow equation in the power system simulation environment model is:

[0113]

[0114] where P Tgen,m,t is the active power of the conventional unit m at time t, Q Tgen,m,t is the reactive power of the conventional unit m at time t, P Rgen,m,t is the active power of the new energy unit m at time t, P Sto,m,t is the charging (discharging) electric power of the energy storage unit m at time t, P Ld,m,t is the active load at time t, Q Ld,m,t is the reactive load at time t, U m,t is the voltage magnitude of node m at time t, θ mn,t is the phase angle difference between node m and node n, G mn is the conductance between node m and node n, B mnis the susceptance between node m and node n.

[0115] 2) The voltage upper and lower limit constraints (as shown in Equation (8)) can ensure that the voltage level during the operation of the power system remains within a safe and effective range. If the node voltage exceeds its upper and lower limits, the environment gives a negative reward as feedback.

[0116] Specifically, the magnitude of the voltage of each node (or bus) needs to be maintained between the predetermined upper and lower limits. This is to ensure the stability and reliability of the power system, and at the same time to guarantee the normal operation and service life of electrical equipment. If the voltage exceeds this range, it may lead to equipment damage, reduced operating efficiency, or other safety problems. Therefore, in the construction of the power system simulation environment model, the voltage upper and lower limit constraints are incorporated as basic conditions to ensure that all scheduling decisions do not cause the voltage of any part of the power system to exceed its safe operating range. In this way, the scheduling agent can balance system safety and service quality during training.

[0117]

[0118] where is the lower voltage limit of node m, is the upper voltage limit of node m.

[0119] 3) The balancing machine upper and lower limit constraints (as shown in Equation (9)) refer to the restrictions on the output power of the balancing machine in the power system, to ensure that the actual output power of the generator can be adjusted within its minimum and maximum production capacity ranges, neither exceeding the maximum safe output of the unit nor being lower than the minimum output required to ensure stable operation. From a safety and technical perspective, the limitation of the balancing machine output can prevent the unit from overloading, ensuring the stability of the equipment and extending its service life. From an economic perspective, by adjusting the output of the balancing machine to cope with load changes, the operating cost of the system can be optimized and the power generation efficiency can be improved. In addition, the balancing machine upper and lower limit constraints are also crucial for the frequency modulation and peak shaving capabilities of the power system, maintaining the balance between power supply and demand in the face of load fluctuations, and ensuring the reliability and stability of the power grid. Therefore, incorporating the balancing machine upper and lower limit constraints into the environment model, where the balancing machine is used to share the system unbalanced power caused by the deviation of the agent control strategy, is one of the basic requirements for realizing system operation optimization, maintaining grid stability and economic operation during the training of the scheduling agent.

[0120]

[0121] where P Bal,m,t is the active power output of the balancing machine m at time t, is the lower limit of the active power output of the balancing machine m, is the upper limit of the active power output of the balancing machine m.

[0122] 4) Branch overload constraint (as shown in Equation (10))

[0123] If the current value of a certain branch exceeds its thermal stability limit value, it indicates that the branch current is overloaded. Among them, 1.35 is the tolerance coefficient for short-term overload of the branch. If the branch current is overloaded but does not exceed 1.35 times the thermal stability limit value, it means that the branch has a "soft overload"; if the branch current exceeds 1.35 times the thermal stability limit value, it means that the branch has a "hard overload".

[0124]

[0125] Among them, I Bra,m,t is the current value of branch m at time t, is the thermal stability limit value of branch m. If any branch has a "soft overload" for consecutive time steps, then the branch is taken out of service; if there is a "hard overload", then the branch is immediately shut down, and the branch is put back into service time steps later.

[0126] 5) The random fault constraint is to train the robustness and adaptability of the intelligent agent in the face of sudden situations in the power system. In each time step, the power system simulation environment model is configured with a 1% probability of outage of the tie-line branch, that is, each tie-line branch has a 1% probability of an outage event. This constraint condition is used to simulate the unexpected situations that may be encountered during the operation of the power system in the real world, such as the temporary inavailability of transmission lines due to equipment failures, maintenance work, or natural disasters. Similarly, the out-of-service branch is put back into service after time steps, indicating that once a tie-line is forced to be out of service for some reason, it will return to the operating state at the th time step after the outage. This setting helps to simulate the equipment maintenance or fault repair process in the power system, as well as the impact of such random fault constraints on the overall operation and scheduling strategy of the power system.

[0127] Specifically, this constraint is used to simulate and train the power system dispatching intelligent agent, enabling it to learn how to quickly and effectively adjust the scheduling strategy when a tie-line branch fails or is out of service, ensuring the stability and reliable power supply of the power system. This configuration can increase the complexity and realism of the training, enabling the intelligent agent to learn how to maintain the balance and stability of the power system in the dynamic changes of equipment outages and reconnections, thus helping the intelligent agent to make optimal decisions in an uncertain environment.

[0128] At each moment when the agent executes an action, it is necessary to calculate the power flow distribution of the system and judge the constraint conditions (i.e., the preset constraint conditions, including upper and lower voltage limits, upper and lower limits of the balancing machine, branch overlimit constraints, and random fault constraints) to standardize and test the feasibility of the action, and correspondingly feedback the reward function value.

[0129] (2) Agent state space

[0130] When designing the state space of the sub-scheduling agent, the characteristics and objectives of the power system need to be considered. The observed operating environment state should be able to reflect all important variables and parameters in the power system so that the sub-scheduling agent can make effective decisions. For the source-grid-load-storage power system targeted by the embodiments of the present application, the state space of the sub-scheduling agent mainly includes the following four aspects:

[0131] 1) Generation state: including the current output of conventional units and new energy units, and the maximum output of new energy units in the next time step.

[0132] 2) Grid state: including branch state, branch power flow, grid load level, and grid loss.

[0133] 3) Load state: including current and predicted power demands.

[0134] 4) Energy storage state: including the charge and discharge levels of the energy storage system.

[0135] Therefore, the state space s of the sub-agent in a certain type of operating scenario t includes the grid measurements at the current moment t, the active load in the next moment, and the maximum active output of new energy, etc. The specific state space observation variables and descriptions are shown in Table 1.

[0136] Table 1

[0137]

[0138]

[0139] (3) Agent action space

[0140] In the operating scenario category i, according to the state space s observed at time t t , the action space a of each sub-scheduling agent t includes the active power output adjustment values ΔP Tgen,t and ΔP Rgen,t , the voltage adjustment value ΔU of the unit Gen,t , the active power adjustment value ΔP of the adjustable load Adjld,t and the active power adjustment value ΔP of the energy storage Sto,t , that is:

[0141] a t={ΔP Tgen,t ,ΔP Rgen,t ,ΔU Gen,t ,ΔP Adjld,t ,ΔP Sto,t}; (11)

[0142] 1) Upper and lower limits of active power output of the unit

[0143] The active output injection value of any unit (except the balancing unit) in the power system cannot be greater than the upper limit of active output, nor less than the lower limit of active output. Among them, the active output injection value is equal to the actual active output value of the observed quantity in the previous time step plus the active output adjustment value in the current action, that is:

[0144]

[0145] in, are the upper and lower limits of the active power output of conventional units, respectively. Unlike the upper limit of the active power output of conventional units, the maximum output value of the new energy unit will change with time steps. Therefore, the maximum output constraint of the new energy unit is taken as the maximum active power output prediction value of the new energy unit at the current moment.

[0146] 2) Unit climbing constraints

[0147] The active output adjustment value of any conventional unit must be less than the ramp rate.

[0148]

[0149] Where ΔP Tgen,m,t is the active output adjustment value of unit m at time t, These are the limits for the upward and downward climbing rates of conventional units respectively.

[0150] In addition, the start-up and shutdown constraints of conventional units are equally important. When the unit is started, it is required to adjust the active power output from the current value to the output lower limit before the unit starts. Once the unit is restarted, it will The shutdown rule requires that the active power output be adjusted from the current value to the output lower limit before the unit is shut down, and then adjusted to zero. Once the unit is shut down, it will Restarts are not allowed within a time step.

[0151] 3) Constraints on adjustable load up and down adjustment capabilities

[0152] The adjustable load is adjusted based on the predicted value at the next moment. In a time step, the cumulative value of the adjustment amount of any adjustable load cannot be greater than the positive adjustable load adjustment capacity, nor less than the negative adjustable load adjustment capacity, that is:

[0153]

[0154] Among them, is the absolute value of the cumulative value of the adjustment amount of the adjustable load m, is the upper limit of the positive adjustable load regulation ability, is the lower limit of the negative adjustable load regulation ability. At the same time, in any time step, the active power adjustment value of any adjustable load must satisfy the adjustable load ramp constraint, that is:

[0155]

[0156] Among them, ΔP Adjld,m,t is the active power output adjustment value of the adjustable load m at time t, are the upper and lower limits of the adjustable load active power regulation rates respectively.

[0157] 4) Energy storage operation constraints

[0158] The energy storage is always adjusted based on zero, and the adjustment value is the actual charge and discharge power at the next moment. At the current time step t cur , the remaining power E Sto,m,t of any energy storage cannot be greater than the energy storage capacity and cannot be less than zero, that is:

[0159]

[0160] Among them, E ES,m,max is the rated capacity of the energy storage m; ξ Sto,m,t is the charge and discharge state of the energy storage m at time t, 1 means charging, 0 means discharging; is the charging power of the energy storage m at time t, is the discharging power of the energy storage m at time t; μ ch,m is the energy storage charging efficiency, μ dis,m is the energy storage discharging efficiency; the state of charge SOC ES,m,t of the energy storage satisfies:

[0161]

[0162] At the same time, in any time step, the charging power of the energy storage cannot be greater than the charging upper limit, and the discharging power of the energy storage cannot be greater than the discharging upper limit, that is:

[0163]

[0164] Among them, is the upper limit of the charging power of the energy storage m, is the upper limit of the discharging power of the energy storage m.

[0165] (4) Agent reward function

[0166] In the embodiments of the present application, the reward function is used to evaluate the quality of different actions taken by each sub-scheduling agent in the environment, and guide the agent to learn how to make optimal decisions in different operation scenarios. For the source-grid-load-storage collaborative system with a high proportion of new energy, the reward function designed in the embodiments of the present application quantitatively evaluates the decision execution effect from three aspects: grid economy, low carbon, and safety. The specific scoring includes the following six items:

[0167] 1) The operating cost of the power system includes the operating cost of the units, the regulation cost of the adjustable load, and the regulation cost of the energy storage. By introducing negative reward modeling, that is:

[0168]

[0169] Where N Tgen 、N Adjld 、N Sto are the total numbers of conventional units, adjustable loads participating in regulation, and energy storage participating in charging in the power system respectively; is the operating cost of the conventional unit set at time t; is the coefficient of the consumption characteristic function of the conventional unit m; is the operating cost of the adjustable load set at time t; is the coefficient of the active power regulation amount of the adjustable load m, indicating the cost corresponding to each unit of active power regulation amount; is a constant term, indicating the basic cost when ΔP Adjld,m,t = 0; is the operating cost of the energy storage set at time t; ΔP Sto,m,t is the charge and discharge power of the energy storage m at time t; is the coefficient of the charge and discharge power of the energy storage m, indicating the cost corresponding to each unit of charge and discharge power; is a constant term, indicating the basic cost when ΔP Sto,m,t = 0.

[0170]

[0171] Among them, R1 is the negative reward corresponding to the operating cost of the power system at the t-th time step, indicating the reward value to be feedback by the environment when performing the action a t in the state s[[ID=4′7]] t ; N Ld is the total number of loads in the power system; α, β, δ are the coefficients of this reward function, used to suppress the agent from using the adjustable load and energy storage as appropriate when performing the action a t .

[0172] 2) Absorption of new energy units

[0173] <I

[0174] Among them, R2 is the reward value obtained by the power system for accommodating new energy at the t-th time step; N Rgen is the total number of new energy units; P Rgen,m,t is the actual active power output of new energy unit m at time t; is the maximum active power output of new energy unit m at time t.

[0175] 3) Node voltage violation

[0176]

[0177] Among them, ΔU m,t is the over-limit voltage amount of node m at the t-th time step.

[0178]

[0179] Among them, R3 is the negative reward feedback by the node voltage violation environment in the power system at the t-th time step, N Bus is the total number of grid nodes.

[0180] 4) Reactive power output violation

[0181] When the intelligent agent adjusts the terminal voltage of the generator set, if the reactive power output value of the generator set exceeds its upper and lower limits, a negative reward is obtained, that is:

[0182]

[0183] Among them, R4 is the negative reward value feedback by the environment when the reactive power output of the generator set in the power system exceeds the limit at the t-th time step, N Gen is the total number of generator sets, are the upper and lower limits of the reactive power output of generator set m, respectively.

[0184] 5) Branch violation

[0185]

[0186] Among them, R5 is the negative reward value feedback by the environment under the thermal limit of the power system branch at the t-th time step, N Bra is the total number of grid branches.

[0187] 6) Balancing machine power violation

[0188] If the active power output of the balancing machine is greater than the upper limit but less than 1.1 times the upper limit, or less than the lower limit but greater than 0.9 times the lower limit, the environment model will feedback a negative reward, that is:

[0189]

[0190] Among them, N Bal is the total number of balancing machines in the system.

[0191] In the embodiments of the present application, the training process of deep neural networks often faces the problems of gradient explosion and gradient disappearance. By normalizing the reward terms, the value range differences and scale differences of each reward term can be reduced, which helps to avoid an unstable training process and makes it easy to find a suitable learning rate, accelerating the convergence speed. Therefore, normalize the R3 and R4 reward terms, that is:

[0192]

[0193] In the subsequent training process of the intelligent agent, for each action executed at each time step, the power flow calculation and out-of-limit judgment will be performed on the action environment model, and then the reward function will be constructed with the above-mentioned reward terms as components:

[0194]

[0195] where ε j is the coefficient of each reward term, which affects the degree of emphasis on different goals during the learning process of the intelligent agent and guides the learning direction; R(s t , a t ) is the comprehensive reward feedback by the environment model at time t, which is used as the optimization target of the DRL algorithm; through learning, the intelligent agent attempts to maximize its cumulative reward, that is, minimize the negative reward terms (R1, R5, R6) and maximize the positive reward term R2, so as to ensure the stable operation of the power system while reducing the operation cost of the power system and improving the new energy consumption.

[0196] (5) Entropy-regularized DRL algorithm and execution process

[0197] The algorithm of the embodiments of the present application includes 1 Actor network (which realizes the distribution parameters of the optimal action output by the policy network π θ at the corresponding state for sampling specific actions), 2 V Critic networks (used to estimate the value of the current state), and 2 Q Critic networks (used to estimate the value of the action-state pair). The learning process of each sub-scheduling intelligent agent can be regarded as a process of gradually converging the neural network parameters by applying data samples in different scenarios. Among them, while satisfying the maximization of the cumulative reward, the Actor network makes the scheduling policy more random through entropy regularization, so that the output actions of the sub-scheduling intelligent agent during the training stage can balance exploration and exploitation. Therefore, the training objective is:

[0198]

[0199] where π * is the optimal policy to be found, ρ π is the state-action trajectory distribution formed by the policy π, is the probability distribution ρ πExpected value of the sampling space is the regularization coefficient, used to control the importance of entropy, and H[(·|s t )] is the entropy value of the policy π taking actions in the state s t .

[0200] The definition of entropy is as follows:

[0201] H(X) = E x～p [-logp(x)]; (30)

[0202] where X is a random variable, corresponding to the control variable in the action space, following the distribution of the probability density function p; the magnitude of the entropy value H(X) reflects the randomness of the random variable.

[0203] The pseudo-code of the entropy regularization DRL algorithm of the embodiments of this application is as Figure 4 shown.

[0204] The multi-agent collaborative optimization scheduling mode considering the differentiation of operation scenarios can effectively cope with the switching of operation scenarios of highly random scheduling services at adjacent moments, ensuring that the power system can quickly respond to scenario changes and the optimality of scheduling decisions. The execution process of the multi-agent scheduling coordination method can also be as Figure 5 shown.

[0205] To verify the effectiveness of the multi-agent collaborative optimization scheduling method for the power system provided by the embodiments of this application, the optimization effect of the collaborative control strategy is analyzed using a provincial 126-node power system as an example. The topology of the power system is as Figure 6 shown, where the new energy installed capacity accounts for 41.7%, there are a total of 185 branches, the unit at node 40 is the balancing machine. During the training process, the initial state of the power system simulation environment model adopts 106,820 consecutive converged AC power flow sections and the maximum output prediction of new energy units in a year of this example, and the acquisition interval is 5 minutes. The experimental environment is carried out using python 3.7 on a standard PC, and the hardware configuration can adopt a CPU Intel Xeon Gold-5118 48-core processor, with a memory of 128GB and a graphics card of NVIDIA Quadro P5000.

[0206] According to various typical scenario samples, train the corresponding scheduling operation scenario identification model, where the splitting ratio of the training set and the test set can be 8:2, as Figure 7 shown, and the normalized confusion matrix of the scenario identification model in the test stage is given.

[0207] From Figure 7It can be seen that the scenario identification model constructed based on the multi-layer perceptron in the embodiments of the present application has a high typical operation scenario identification ability after differential training, so as to ensure that the "mature" sub-scheduling agents trained by each typical scenario can be directly called during subsequent real-time scheduling online decision-making, realizing the rapid output of scheduling strategies.

[0208] To further verify the decision-making efficiency and new energy consumption stability of the proposed method, a single operation section is randomly selected from each scenario test set, and the method based on the embodiments of the present application is compared with the traditional single-agent (trained using the standard SCA (Sine Cosine Algorithm) algorithm), the stochastic economic dispatch based on the scenario method, and the scheduling decision results of the optimal power flow (OPF for short). The comparison results of the decision-making time and the total system operation cost are shown in Table 2.

[0209] Table 2

[0210]

[0211]

[0212] As can be seen from Table 2, in terms of operation cost, the method of the embodiments of the present application has the lowest system operation cost for the scheduling decision of each section through the targeted training of the previous operation scenarios and the cooperation of multiple agents. In addition, in terms of scheduling decision-making efficiency, the solution time of the DRL learning method is much lower than that of model-driven methods such as stochastic economic dispatch and optimal power flow dispatch, which has significant advantages for adapting to the real-time scheduling of high-randomness and uncertainty systems. And the decision-making time of the DRL learning method is not affected by the complexity of the operation section, but there is a defect of long pre-training time. At the same time, compared with the traditional single-agent, the calculation decision-making time of the method of the embodiments of the present application slightly increases. The reason is mainly that the decision-making time of the method of the embodiments of the present application consists of two parts: the scenario identification model and the solution of the sub-scheduling agent, and there is a situation of two iterative solutions for the sections with fuzzy scenario features.

[0213] As Figure 8 shown, the scheduling decision results of the traditional single-agent and multi-agent cooperation for continuous operation sections are given. As Figure 8 can be seen, compared with the single-agent decision-making, the multi-agent cooperation mode can better grasp the new energy change characteristics, and the decision-making output is more in line with the upper limit of the new energy predicted output, consuming wind power in the form of maximizing economic benefits. Its stability benefits from the entropy regularization term introduced in the algorithm training objective function, which enables each agent to explore a larger decision space during the training process, so as to better solve the high-dimensional non-convex and non-linear optimal scheduling problem.

[0214] In summary, in the embodiments of the present application, by interacting and iterating the traditional power flow algorithm with deep reinforcement learning, the adaptive scenario recognition of the operating section and the collaborative control decision-making of the source-network-load-storage are realized. The multi-agent collaborative scheduling mode can pre-perceive the typical scenario features of the real-time operating section through the scenario identification model, accurately call the sub-scheduling agents to solve the scheduling tasks, effectively reduce the system operation cost while improving the new energy consumption rate. The proposed training objective function of deep reinforcement learning based on entropy regularization can more flexibly cope with the uncertainty of the source and load, ensuring the stability of multi-agent decision-making.

[0215] According to the multi-agent collaborative optimization scheduling method for a power system proposed in the embodiments of the present application, by obtaining the real-time operation data of the power system, using the trained scenario identification model to analyze the real-time data to identify the current operating scenario, and according to the identified current operating scenario and the corresponding sub-scheduling agents, formulating the current optimal scheduling strategy to adjust the operating state of the power system according to the optimal scheduling strategy. Thus, the problems of insufficient flexibility and stability of the real-time scheduling model caused by the existing centralized and single-agent scheduling modeling techniques due to the lack of consideration of the differences in operating scenarios are solved.

[0216] Next, refer to the drawings to describe the multi-agent collaborative optimization scheduling device for a power system proposed in the embodiments of the present application.

[0217] Figure 9 It is a block diagram of a multi-agent collaborative optimization scheduling device for a power system according to an embodiment of the present application.

[0218] As Figure 9 shown, the multi-agent collaborative optimization scheduling device 10 for the power system includes: an acquisition module 100, an identification module 200, and a generation module 300.

[0219] Among them, the acquisition module 100 is used to acquire the current operating section data of the power system;

[0220] The identification module 200 is used to perform scenario recognition on the current operating section data by using a preset scenario identification model to obtain a scenario recognition result, wherein the preset scenario identification model is trained by the historical operating section data of the power system;

[0221] The generation module 300 is used to generate the current optimal scheduling strategy according to the scenario recognition result and a plurality of preset sub-scheduling agents, and adjust the operating state of the power system based on the current optimal scheduling strategy.

[0222] Further, in some embodiments, the generation module 300 is specifically used for:

[0223] Determine a target preset sub-scheduling agent from multiple preset sub-scheduling agents according to the operation scenario identification result;

[0224] Observe the current operating environment state of the power system, and based on the current operating environment state, use the target preset sub-scheduling agent to generate the current optimal scheduling strategy.

[0225] Further, in some embodiments, before using the preset scenario identification model to perform scenario identification on the current operating section data to obtain the operation scenario identification result, the identification module 200 is further configured to:

[0226] Obtain the historical operating section data of different operating scenarios of the power system, where the historical operating section data includes scenario true labels;

[0227] Perform standardization processing on the historical operating section data, and divide the standardized historical operating section data into a training set and a validation set;

[0228] Input the training set into the preset neural network, use the forward propagation algorithm to calculate the first scenario prediction label to obtain the initial scenario identification model, and input the validation set into the initial scenario identification model to obtain the second scenario prediction label;

[0229] Perform cross-entropy loss calculation based on the second scenario prediction label and the scenario true label to obtain the calculation result, and determine whether the calculation result meets the preset standard;

[0230] If the calculation result meets the preset standard, end the iterative training of the preset neural network, and use the initial scenario identification model as the preset scenario identification model; otherwise, use the backpropagation algorithm to calculate the gradient of the loss function with respect to the model parameters, and based on the gradient of the loss function with respect to the model parameters, use the gradient descent algorithm to update the model parameters and then continue the iterative training.

[0231] Further, in some embodiments, before generating the current optimal scheduling strategy according to the operation scenario identification result and multiple preset sub-scheduling agents, the generating module 300 further includes:

[0232] An aggregation analysis unit, configured to perform in-depth time-series aggregation analysis on the historical operating section data based on the feature iterative aggregation algorithm to obtain different types of operating scenario sets;

[0233] A construction unit, configured to use the entropy regularization deep reinforcement learning algorithm to construct multiple preset sub-scheduling agents according to different types of operating scenario sets.

[0234] Further, in some embodiments, the construction unit is specifically configured to:

[0235] Determine the initial sub-scheduling agents corresponding to each type of operating scenario set according to different types of operating scenario sets;

[0236] Construct a power system simulation environment model, and based on the operating environment state, power flow distribution algorithm and preset constraint conditions of the power system simulation environment model, use the initial sub-scheduling agents corresponding to each type of operating scenario set to generate corresponding optimal scheduling strategies;

[0237] Use the reward function to perform reward feedback on the corresponding optimal scheduling strategy, and use the reward feedback as the update guidance for the model parameters until the initial sub-scheduling agents corresponding to each type of operating scenario set converge to the preset maximum cumulative reward, and obtain multiple preset sub-scheduling agents.

[0238] It should be noted that the foregoing explanation of the embodiments of the multi-agent collaborative optimal scheduling method for the power system also applies to the multi-agent collaborative optimal scheduling device for the power system in this embodiment, and will not be elaborated here.

[0239] According to the multi-agent collaborative optimal scheduling device for the power system proposed in the embodiments of the present application, by obtaining the real-time operation data of the power system, using the trained scenario identification model to analyze the real-time data to identify the current operating scenario, and formulating the current optimal scheduling strategy according to the identified current operating scenario and the corresponding sub-scheduling agent, so as to adjust the operating state of the power system according to the optimal scheduling strategy. Thus, the problem that the existing centralized and single-agent scheduling modeling technologies lack consideration of the differences in operating scenarios, resulting in insufficient flexibility and stability of the real-time scheduling model, is solved.

[0240] Figure 10 The structural schematic diagram of the electronic device provided in the embodiments of the present application. The electronic device may include:

[0241] A memory 1001, a processor 1002, and a computer program stored on the memory 1001 and executable on the processor 1002.

[0242] When the processor 1002 executes the program, it implements the multi-agent collaborative optimal scheduling method for the power system provided in the above embodiments.

[0243] Further, the electronic device further includes:

[0244] A communication interface 1003 for communication between the memory 1001 and the processor 1002.

[0245] The memory 1001 is used to store a computer program executable on the processor 1002.

[0246] The memory 1001 may include a high-speed RAM (Random Access Memory) memory and may also include non-volatile memory, such as at least one disk memory.

[0247] If the memory 1001, the processor 1002, and the communication interface 1003 are implemented independently, the communication interface 1003, the memory 1001, and the processor 1002 can be interconnected through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity in representation, Figure 10 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0248] Optionally, in a specific implementation, if the memory 1001, the processor 1002, and the communication interface 1003 are integrated on a single chip, the memory 1001, the processor 1002, and the communication interface 1003 can communicate with each other through an internal interface.

[0249] The processor 1002 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0250] The embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the multi-agent collaborative optimization scheduling method of the power system as described above is implemented.

[0251] The embodiments of the present application also provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the multi-agent collaborative optimization scheduling method of the power system as described above is implemented.

[0252] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In the description of this application, "a plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0253] In the description of this specification, the description with reference to terms such as "an embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0254] Although the embodiments of this application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting this application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A multi-agent collaborative optimization scheduling method for a power system, characterized in that Including the following steps: Obtain the current operating section data of the power system; Use a preset scenario identification model to perform scenario identification on the current operating section data to obtain an operating scenario identification result, where the preset scenario identification model is trained by the historical operating section data of the power system; Generate a current optimal scheduling strategy according to the operating scenario identification result and multiple preset sub-scheduling agents, and adjust the operating state of the power system based on the current optimal scheduling strategy.

2. The method according to claim 1, wherein The generating of the current optimal scheduling strategy according to the operating scenario identification result and multiple preset sub-scheduling agents includes: Determine a target preset sub-scheduling agent from the multiple preset sub-scheduling agents according to the operating scenario identification result; Observe the current operating environment state of the power system, and generate the current optimal scheduling strategy by using the target preset sub-scheduling agent based on the current operating environment state.

3. The method according to claim 1, wherein Before using the preset scenario identification model to perform scenario identification on the current operating section data to obtain the operating scenario identification result, it further includes: Obtain the historical operating section data of different operating scenarios of the power system, where the historical operating section data includes scenario true labels; Perform standardization processing on the historical operating section data, and divide the standardized historical operating section data into a training set and a validation set; Input the training set into a preset neural network, calculate a first scenario prediction label by using the forward propagation algorithm to obtain an initial scenario identification model, and input the validation set into the initial scenario identification model to obtain a second scenario prediction label; Perform cross-entropy loss calculation based on the second scenario prediction label and the scenario true label to obtain a calculation result, and determine whether the calculation result meets a preset standard; If the calculation result meets the preset standard, end the iterative training of the preset neural network, and use the initial scenario identification model as the preset scenario identification model, otherwise calculate the gradient of the loss function with respect to the model parameters by using the backpropagation algorithm, and continue the iterative training after updating the model parameters by using the gradient descent algorithm based on the gradient of the loss function with respect to the model parameters.

4. The method according to claim 3, wherein Before generating the current optimal scheduling strategy according to the operating scenario identification result and the multiple preset sub-scheduling agents, it further includes: Perform in-depth time-series aggregation analysis on the historical operating section data based on the feature iterative aggregation algorithm to obtain different types of operating scenario sets; Use the entropy-regularized deep reinforcement learning algorithm to construct the multiple preset sub-scheduling agents according to the different types of operating scenario sets.

5. The method according to claim 4, characterized in that, The using of the entropy-regularized deep reinforcement learning algorithm to construct the multiple preset sub-scheduling agents according to the different types of operating scenario sets includes: Determine an initial sub-scheduling agent corresponding to each type of operating scenario set according to the different types of operating scenario sets; Construct a power system simulation environment model, and based on the operating environment state, power flow distribution algorithm, and preset constraint conditions of the power system simulation environment model, use the initial sub-scheduling agents corresponding to each type of operating scenario set to generate corresponding optimal scheduling strategies; Use a reward function to perform reward feedback on the corresponding optimal scheduling strategies, and use the reward feedback as a guidance for updating model parameters until the initial sub-scheduling agents corresponding to each type of operating scenario set converge to a preset maximum cumulative reward, and obtain the multiple preset sub-scheduling agents.

6. A multi-agent collaborative optimization scheduling device for a power system, characterized in that, It includes: An acquisition module for acquiring the current operating section data of the power system; An identification module for using a preset scenario identification model to identify the current operating section data to obtain an operating scenario identification result, where the preset scenario identification model is trained from the historical operating section data of the power system; A generation module for generating a current optimal scheduling strategy based on the operating scenario identification result and multiple preset sub-scheduling agents, and adjusting the operating state of the power system based on the current optimal scheduling strategy.

7. The device according to claim 6, characterized in that, The generation module is specifically used for: Determine a target preset sub-scheduling agent from the multiple preset sub-scheduling agents according to the operating scenario identification result; Observe the current operating environment state of the power system, and based on the current operating environment state, use the target preset sub-scheduling agent to generate the current optimal scheduling strategy.

8. An electronic device, characterized in that, It includes: A memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the program to implement the multi-agent collaborative optimal scheduling method of the power system according to any one of claims 1-5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to be used to implement the multi-agent collaborative optimal scheduling method of the power system according to any one of claims 1-5.

10. A computer program product, characterized in that, It includes a computer program, and when the computer program is executed by the processor, it is used to implement the multi-agent collaborative optimal scheduling method of the power system according to any one of claims 1-5.

Images