A power dispatching planning method and system, an electronic device, and a storage medium

Abstract

The application provides a power dispatch planning method and system, an electronic device and a storage medium. The power dispatch planning method comprises the following steps: acquiring basic information of power dispatch planning and future electricity demand prediction data; constructing a deep reinforcement learning power dispatch simulation environment according to the basic information of power dispatch planning and the future electricity demand prediction data; defining an original state, an original action and an original reward through the deep reinforcement learning power dispatch simulation environment; interacting with the deep reinforcement learning power dispatch simulation environment through a first deep deterministic policy gradient model, generating a state transition sample, and training the first deep deterministic policy gradient model multiple times using the state transition sample until the model converges, thereby obtaining an optimized use strategy optimization network model; and inputting a real-time state of a power consumption unit into the optimized use strategy optimization network model to generate an optimal power dispatch planning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power dispatch planning, and particularly relates to a power dispatch planning method and system, an electronic device and a storage medium. BACKGROUND

[0002] In modern society, a power distribution network is a key infrastructure for maintaining the operation of the national economy and the daily life of residents. With the continuous growth of social electricity demand, the large-scale grid connection of new energy represented by wind and solar energy, and the popularity of distributed power supply, the operating environment of modern power distribution networks presents unprecedented complexity and uncertainty. On the one hand, the demand on the user side presents certain volatility and randomness, and external characteristics such as weather, holidays and other influencing factors are not fully taken into account in dispatching. On the other hand, the intermittency and unpredictability of new energy bring great challenges to the stability of balancing supply and demand in the power distribution network.

[0003] The existing technology based on the two-stage traditional method of "prediction-optimization" lacks adaptability and robustness, completely decouples prediction and decision-making, and thus prediction errors directly affect the optimization results. SUMMARY

[0004] Embodiments of the present application provide a power dispatch planning method and system, an electronic device and a storage medium to solve the problems in the related art. The technical solutions are as follows:

[0005] In a first aspect, the embodiments of the present application provide a power dispatch planning method, comprising:

[0006] obtaining basic information of power dispatch planning and future electricity demand prediction data;

[0007] constructing a deep reinforcement learning power dispatch simulation environment according to the basic information of power dispatch planning and the future electricity demand prediction data, defining an original state, an original action and an original reward through the deep reinforcement learning power dispatch simulation environment;

[0008] interacting with the first deep deterministic policy gradient model and the power dispatch simulation environment to generate state transition samples, and training the first deep deterministic policy gradient model multiple times using the state transition samples until the first deep deterministic policy gradient model converges, to obtain an optimized usage strategy optimization network model;

[0009] inputting the real-time state of the electricity unit into the optimized usage strategy optimization network model to generate an optimal power dispatch planning.

[0010] In an embodiment of the present application, obtaining the future electricity demand prediction data comprises:

[0011] acquire historical power consumption demand data of each power consumption unit, historical external feature data of each power consumption unit and an initial neural network model;

[0012] preprocess the historical power consumption demand data of each power consumption unit and the historical external feature data of each power consumption unit to generate preprocessing data;

[0013] divide the preprocessing data to obtain a training set and a test set;

[0014] train the initial neural network model using a gradient descent method and the training set, and calculate a mean square error loss function to update model parameters until the MSE loss converges or a maximum number of iterations is reached, to obtain a neural network prediction model;

[0015] test the neural network prediction model according to the test set to obtain a final prediction model;

[0016] input the training set into an input layer of the final prediction model to obtain output data of the input layer;

[0017] input the output data of the input layer into a long short-term memory network layer of the final prediction model to obtain output data of the long short-term memory network layer;

[0018] input the output data of the long short-term memory network layer into a fully connected layer of the final prediction model to obtain output data of the fully connected layer;

[0019] input the output data of the fully connected layer into an output layer of the final prediction model to obtain future power consumption demand prediction data of each power consumption unit.

[0020] In an embodiment of the present application, the original state, the original action and the original reward are defined according to the basic information of the power dispatching plan and the future power consumption demand prediction data, including:

[0021] The basic information of the power dispatching plan includes time encoding of the power consumption unit, temperature, weather type encoding, date type encoding, peak type encoding, instantaneous available capacity of the power distribution station, instantaneous capacity utilization rate of the power distribution station, instantaneous schedulable capacity between the power distribution stations, average scheduling capacity between the power distribution stations in the previous H time points and average scheduling data of each power consumption unit;

[0022] The original state is defined according to the time encoding of the power consumption unit, the temperature, the weather type encoding, the date type encoding, the peak type encoding, the instantaneous available capacity of the power distribution station, the instantaneous capacity utilization rate of the power distribution station, the instantaneous schedulable capacity between the power distribution stations, the average scheduling capacity between the power distribution stations in the previous H time points and the average scheduling data of each power consumption unit;

[0023] The basic information of the power dispatch plan further includes power dispatch plans between the power distribution stations and power dispatch plans between the power distribution stations and the power consumption units.

[0024] An original action is defined according to the power dispatch plans between the power distribution stations and the power dispatch plans between the power distribution stations and the power consumption units.

[0025] A capacity reasonable allocation reward, a fixed dispatch behavior cost, a line loss cost, an unmet demand penalty and a cross-station dispatch behavior reward are determined according to the basic information of the power dispatch plan.

[0026] An original reward is defined according to the capacity reasonable allocation reward, the fixed dispatch behavior cost, the line loss cost, the unmet demand penalty and the cross-station dispatch behavior reward.

[0027] In an embodiment of the present application, the first deep deterministic policy gradient model includes a first use policy optimization network, a first use value optimization network, a first target value optimization network and a first target policy optimization network, the first deep deterministic policy gradient model is used to interact with the power dispatch simulation environment to generate state transition samples, and the first deep deterministic policy gradient model is trained multiple times using the state transition samples until the model converges, and an optimized use policy optimization network model is obtained, which includes:

[0028] An original state is obtained according to the power dispatch simulation environment.

[0029] The original state is input into the first use policy optimization network to generate a first action.

[0030] The first action is physically constrained and corrected to generate a second action.

[0031] The second action is executed in the power dispatch simulation environment to obtain a first reward and a first state.

[0032] A first state transition sample is determined according to the original state, the second action, the first reward and the first state, and the first state transition sample is stored in an experience replay pool, and if the experience replay pool is full, the earliest stored state transition sample in the experience replay pool is removed.

[0033] The above steps are repeated until the number of first state transition samples in the experience replay pool reaches a preset threshold number, a batch of first state transition samples is extracted from the experience replay pool as second state transition samples, and the second state transition samples include a second state, a third action, a second reward and a third state.

[0034] The third state in the second state transition sample is input into the first target policy optimization network to obtain a fourth action.

[0035] The fourth action is corrected by physical constraint to obtain a fifth action;

[0036] The third state in the second state transition sample and the fifth action are input into the first target value optimization network to obtain a first target future value estimation;

[0037] The first target future value estimation, the second reward in the second state transition sample and the discount factor are used to obtain a first target value;

[0038] The second state in the second state transition sample and the third action are input into the first use value optimization network to obtain a first use value;

[0039] The first loss function is constructed based on the first target value and the first use value;

[0040] The first use value optimization network is adjusted according to the first loss function to obtain an adjusted first use value optimization network;

[0041] The second state is input into the first use policy optimization network to obtain a sixth action;

[0042] The sixth action is corrected by physical constraint to obtain a seventh action;

[0043] The second state and the seventh action are input into the adjusted first use value optimization network, and the adjusted first use policy optimization network is updated by using a policy gradient ascent algorithm and a gradient back propagation method to obtain an updated first use policy optimization network;

[0044] The above steps are repeatedly executed to update the first use value optimization network and the first use policy optimization network multiple times until the model converges, and an optimized use value optimization network and an optimized use policy optimization network are obtained.

[0045] In an embodiment of the present application, the first deep deterministic policy gradient model is interacted with a power dispatch simulation environment to generate a state transition sample, and the first deep deterministic policy gradient model is trained multiple times using the state transition sample until the model converges, and an optimized use policy optimization network model is obtained, and the method further comprises:

[0046] After each update of the use value optimization network, the updated use value optimization network parameters are used to soft update the target value optimization network parameters;

[0047] After each update of the use policy optimization network, the updated use policy optimization network parameters are used to soft update the target policy optimization network parameters.

[0048] In an embodiment of the present application, the method for constructing the first deep deterministic policy gradient model comprises:

[0049] constructing an initial deep deterministic policy gradient model, the initial deep deterministic policy gradient model comprising an initial use policy optimization network, an initial use value optimization network, an initial target value optimization network, and an initial target policy optimization network; constructing an experience replay pool and a physical constraint correction module, the experience replay pool being used to store serialized historical experience records; the physical constraint correction module being used to correct the output of the use policy optimization network so that the output meets the physical constraints in reality;

[0050] initializing the initial use policy optimization network, the initial use value optimization network, the initial target value optimization network, and the initial target policy optimization network to obtain the first deep deterministic policy gradient model, wherein in the first deep deterministic policy gradient model, the structure of the first use policy optimization network is the same as that of the first target policy optimization network, the parameters of the first use policy optimization network are the same as those of the first target policy optimization network, the structure of the first use value optimization network is the same as that of the first target value optimization network, and the parameters of the first use value optimization network are the same as those of the first target value optimization network.

[0051] In an embodiment of the present application, the historical power consumption demand data of each power consumption unit and the historical external feature data of each power consumption unit are preprocessed to generate preprocessed data, which comprises:

[0052] The historical power consumption demand data of each power consumption unit and the historical external feature data of each power consumption unit are subjected to interpolation processing and normalization processing to generate preprocessed data.

[0053] In a second aspect, the embodiments of the present application provide a power dispatch planning system, which comprises:

[0054] a first obtaining module configured to obtain basic information of power dispatch planning and future power consumption demand prediction data;

[0055] a first constructing module configured to construct a deep reinforcement learning power dispatch simulation environment according to the basic information of power dispatch planning and the future power consumption demand prediction data, and define an original state, an original action, and an original reward through the deep reinforcement learning power dispatch simulation environment;

[0056] a first obtaining module configured to obtain basic information of power dispatch planning and future power consumption demand prediction data;

[0057] The first generation module is used to optimize the network model of the optimized usage strategy and input the real-time status of the power consumption unit to generate the optimal power dispatch plan.

[0058] Thirdly, embodiments of this application provide an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to execute the above-described power dispatch planning method.

[0059] Fourthly, embodiments of this application provide a computer-readable storage medium that stores computer instructions, wherein when the computer instructions are executed on a computer, the methods in any of the above-described embodiments are performed.

[0060] The advantages or beneficial effects of the above technical solutions include at least the following:

[0061] In this embodiment, the power dispatch planning method integrates basic information on power dispatch planning and future electricity demand forecast data into a first deep deterministic strategy gradient model for training, resulting in an optimized strategy model. By inputting the real-time status of electricity users into the optimized strategy model, the training of the first deep deterministic strategy gradient model considers the power dispatch decisions of multiple substations and user units, as well as future electricity demand, thus making the power dispatch planning decisions more forward-looking and accurate. Furthermore, by combining the ability to predict the future with the ability to optimize dispatch, prediction and decision-making are integrated, improving the adaptability and robustness of power planning strategies. This makes the power dispatch planning strategies more accurate and effectively solves the problems of existing two-stage traditional methods based on "prediction-optimization," which lack adaptability and robustness, completely decouple prediction and decision-making, and cause prediction errors to directly propagate and affect the optimization results.

[0062] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of this application will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0063] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments disclosed in this application and should not be construed as limiting the scope of this application.

[0064] Figure 1 This is a flowchart of a power dispatch planning method according to an embodiment of this application.

[0065] Figure 2 A block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0066] Hereinafter, certain example embodiments are described simply. As can be appreciated by those skilled in the art, the described embodiments can be modified in various different ways without departing from the spirit or scope of the present application. Therefore, the drawings and descriptions are to be regarded as illustrative in nature rather than restrictive.

[0067] The existing invention mainly adopts two method paths:

[0068] The first is a two-stage traditional method based on "prediction-optimization". First, it uses time series analysis (such as ARIMA) or machine learning models (such as support vector machines) to predict future loads and new energy output. Then, the predicted value is used as a deterministic input to establish a mathematical optimization model with the goal of minimizing operating costs. Finally, the scheduling plan for one or more future periods is obtained by solving the optimization model.

[0069] However, the two-stage traditional method based on "prediction-optimization" lacks adaptability and robustness, completely decouples prediction and decision-making, resulting in prediction errors directly affecting the optimization results, and unable to dynamically adjust according to real-time feedback of system state, poor adaptability to sudden changes in new energy output or power load. In addition, this method relies on precise mathematical modeling. As the scale and complexity of the power grid increase, it becomes extremely difficult to establish and solve such precise models, and any simplification of the model can lead to sub-optimal decision-making.

[0070] The second is a method based on traditional reinforcement learning (such as Q-Learning). Such methods usually discretize the power grid state and actions. For example, actions are discretized into fixed instructions, and the agent learns a Q-value table through trial-and-error interactions with the power grid environment to guide the selection of a discrete action in a specific state to achieve long-term optimal returns.

[0071] However, the traditional reinforcement learning-based method has limited control accuracy. Real power dispatching is a process that requires precise control in continuous space. Discretizing it into a few fixed gears not only may cause some precision loss, but also makes the system unable to find the true optimal operating point, essentially limiting the upper limit of optimization effect. In addition, such methods are difficult to handle high-dimensional state and action spaces, and are difficult to apply in real large-scale power grid scenarios.

[0072] Artificial intelligence technology represented by deep reinforcement learning provides a new solution to such complex dynamic optimization problems due to its strong autonomous learning and sequential decision-making capabilities. A deep reinforcement learning method for microgrid energy scheduling considering source-load uncertainty is provided in related technology. The deep deterministic policy gradient (DDPG) algorithm is used to avoid the impact of continuous scheduling action discretization on the scheduling strategy, but there are still some limitations. First, although neural networks and other technologies are used to improve prediction capabilities, external features such as weather, temperature, and their impact on load, historical scheduling data, and line losses during scheduling are not fully considered in prediction and scheduling, resulting in limited prediction and scheduling accuracy, making it difficult to respond to demand fluctuations and unexpected events. Second, the deep deterministic policy gradient algorithm is only used to solve the energy scheduling of a single microgrid, while the real power grid is a complex network composed of a large number of interconnected sites and users. Therefore, the correlation between each distribution side is not fully explored, which leads to insufficient adaptability of the load and scheduling prediction model, and the scheduling scheme is difficult to achieve balance between supply and demand in reality.

[0073] Therefore, in order to solve the above problems, the present application provides a power scheduling planning method, system, electronic device and storage medium.

[0074] Figure 1 A flowchart of a power scheduling planning method according to an embodiment of the present application is shown. As shown in Figures 1-2 The power scheduling planning method can include:

[0075] S110: obtaining basic information of power scheduling planning and future electricity demand prediction data;

[0076] S120: constructing a deep reinforcement learning power scheduling simulation environment according to the basic information of power scheduling planning and the future electricity demand prediction data, defining the original state, original action and original reward through the deep reinforcement learning power scheduling simulation environment;

[0077] S130: interacting with the power scheduling simulation environment through the first deep deterministic policy gradient model, generating state transition samples, and training the first deep deterministic policy gradient model multiple times using the state transition samples until the first deep deterministic policy gradient model converges, obtaining an optimized usage strategy optimization network model;

[0078] S140: inputting the real-time state of the electricity consumption unit to the optimized usage strategy optimization network model to generate the optimal power scheduling planning.

[0079] In the embodiment, the basic information of power dispatch planning and the future power demand prediction data are integrated into the first deep deterministic policy gradient model for training to obtain an optimized policy model. By inputting the real-time state of the power consumption unit into the optimized policy model, the power dispatch planning decisions are obtained with more foresight and accuracy by considering the power dispatch decisions of multiple distribution stations and user units in the training of the first deep deterministic policy gradient model, and by considering the future power demand.

[0080] In step S110, the basic information of power dispatch planning and the future power demand prediction data are obtained.

[0081] In the embodiment, the basic information of power dispatch planning includes power supply structure information, power consumption unit environment information, power consumption unit time characteristic information, distribution station state information, and historical dispatch information.

[0082] The power supply structure information includes the number m of distribution stations, the number n of power consumption units, distribution network topology structure data, and a relative importance weight vector of each power consumption unit. In the embodiment, the distribution network topology structure data stores the connection relationship between the distribution stations and the power consumption units in the form of a matrix, i.e., whether a distribution station distributes power to a power consumption unit.

[0083] The power consumption unit environment information includes temperature and weather type, both of which are real-time data.

[0084] The power consumption unit time characteristic information includes date type and peak type, both of which are real-time data of the current day.

[0085] The distribution station state information includes the available capacity of each distribution station, the capacity utilization rate of each distribution station, and the dispatchable capacity between distribution stations, all of which are real-time data. In the embodiment, the capacity utilization rate of each distribution station is the ratio of the capacity being used to the maximum capacity, and is stored in the form of a vector.

[0086] The historical dispatch information includes the average dispatchable capacity between distribution stations and the corresponding average line loss rate for the previous H time points, the average data allocated to each power consumption unit and the corresponding average line loss rate.

[0087] In the embodiments of this application, future electricity demand forecast data can be obtained by inputting historical electricity demand data and historical external feature data into a trained predictive neural network model, and then outputting future electricity demand forecast data through the trained predictive neural network model.

[0088] For multiple electricity consumption units, historical electricity demand data and historical external characteristic data are collected for each unit. The original predictive neural network model is trained using these data to obtain a trained model. The historical electricity demand data and historical external characteristic data are then input into the trained model for prediction, yielding the predicted future electricity demand for each unit. Among them, the future electricity demand forecast data for each electricity consumption unit. Taking the prediction of demand at time t as an example, , This represents the predicted electricity demand data for the x-th electricity-consuming unit at time t.

[0089] Specifically, historical electricity demand data is presented as a serialized demand record, represented as: {electricity unit, time, electricity demand}; historical external characteristic data refers to multi-dimensional data obtained by observation or collection, which includes weather, temperature, date type, and peak type of the area where the electricity unit is located at a specified historical time, represented as {weather, temperature, date type, peak type}.

[0090] In this embodiment, the weather is divided into: sunny, rainy, snowy, and extreme. The weather is represented by a code according to its type, specifically, it can be represented as: :clear"," :rain"," :Snow"," The system categorizes dates into two types: weekdays and holidays. Dates are represented by codes based on their type, specifically "0: weekday" and "1: holiday". Peak periods are categorized into peak and off-peak periods, with peak periods represented by codes based on their type, specifically "0: off-peak" and "1: peak". To better reflect the impact of external characteristics on electricity demand and improve prediction accuracy, the initial prediction neural network model is trained using historical electricity demand data and historical external characteristic data. This combined data is then used to predict the future electricity demand of each electricity consumption unit.

[0091] In step S120, a deep reinforcement learning-based power dispatch simulation environment is constructed based on the basic information of power dispatch planning and the forecast data of future electricity demand. The original state, original action and original reward are defined through the deep reinforcement learning-based power dispatch simulation environment.

[0092] In this embodiment, a deep reinforcement learning-based power dispatch simulation environment is constructed using the aforementioned basic information on power dispatch planning and future electricity demand forecast data. This simulation environment then defines the initial state, initial action, and initial reward. Specifically, the power dispatch simulation environment is a digital model that follows the physical operating laws of the power grid to simulate the real power grid's response to different dispatch commands. This simulation environment provides state observation, action execution, and reward feedback; that is, it constructs the initial state and generates its reward and next state based on the input action. This provides a stable, reliable, and realistically physical interactive platform for training deep deterministic policy gradient models. The initial state... It contains multiple state information at time t, specifically including the time code of the power-consuming unit. ,temperature Weather type codes Date type encoding Peak type coding The immediate available capacity of the substation Real-time capacity utilization rate of substations Real-time dispatchable capacity between substations Average dispatch capacity among substations in the previous H time periods and average dispatch data for each power consumption unit , is represented as:

[0093]

[0094] Among them, for The encoding method used is periodic two-dimensional encoding, that is, the encoding corresponding to time t is as follows:

[0095]

[0096] in, It represents 180°.

[0097] At time t, the action can be represented by the power dispatch plan made by each substation, the original action. This represents the power dispatching plan among various substations at time t. and power dispatch planning between power distribution stations and power consumption units , original action Represented as:

[0098]

[0099] wherein, , represents the capacity dispatched by the ith power distribution station to the jth power distribution station at time t, and the diagonal elements are 0; , represents the capacity finally dispatched by the ith power distribution station to the xth power consumption unit at time t.

[0100] Original reward contains multiple components for evaluating actions in the state . The setting of the reward can reduce the probability of making wrong choices when exploring the environment and improve optimization efficiency. Specifically:

[0101] Original reward is composed of capacity reasonable allocation reward , fixed scheduling behavior cost , line loss cost , unsatisfied demand penalty and cross-site scheduling behavior reward .

[0102] Capacity reasonable allocation reward is defined as follows:

[0103]

[0104] wherein, is a relative importance weight vector of each power consumption unit, indicating the priority of meeting the demand of each power consumption unit, which can be set according to the actual situation; is an economic conversion coefficient for converting capacity allocation value into economic value, helps to guide to meet the electricity demand of the power consumption unit with greater needs first, represents the electricity demand prediction data of the xth power consumption unit at time t.

[0105] Fixed scheduling behavior cost is defined as follows:

[0106]

[0107] wherein, represents the fixed cost of the ith power distribution station to the xth power consumption unit, represents the fixed cost of the ith power distribution station to the jth power distribution station; is an indicator function, when , , otherwise 0, By analogy. Helps to guide the reduction of unnecessary scheduling paths.

[0108] Line loss, i.e. line loss, is a common phenomenon of power loss in power distribution. In this embodiment, the line loss rate is used to calculate the specific line loss, and the line loss cost is defined as follows:

[0109]

[0110] wherein, , represents the average line loss rate between the output party (i.e. the i-th power distribution station) and the receiving party (i.e. the x-th power consumption unit), represents the average line loss rate between the output party (i.e. the i-th power distribution station) and the receiving party (i.e. the j-th power distribution station), and can be calculated based on historical scheduling information using the data of the previous H time points. is an economic conversion coefficient for converting the value of lost power into economic value, and setting the line loss cost helps to increase the authenticity.

[0111] Unmet demand penalty is defined as follows:

[0112]

[0113] wherein, is an economic conversion coefficient for converting the value of uncompleted power consumption demand into economic value, helps to promote power distribution scheduling to meet demand as much as possible, represents the power consumption demand prediction data of the x-th power consumption unit at time t.

[0114] Cross-station scheduling behavior reward is defined as follows:

[0115]

[0116] wherein, represents the instantaneous capacity utilization rate of the output party (i.e. the i-th power distribution station), represents the instantaneous capacity utilization rate of the receiving party (i.e. the j-th power distribution station), is an economic conversion coefficient for converting the value of beneficial power capacity scheduling into economic value, helps to guide the scheduling of power from low utilization rate power distribution stations to high utilization rate power distribution stations to maintain the stability of the power distribution network, while supporting new energy consumption.

[0117] wherein, the economic conversion coefficient The multiplied values all belong to demand values or capacity values of power, and thus the values can refer to real-time power transmission and distribution prices or sales prices.

[0118] Original reward The expression is as follows:

[0119]

[0120] Each element in the weight combination vector is a value greater than 0, and by adjusting the weight combination , the importance of each component can be controlled, and the reward value can be controlled within a reasonable range, which helps to improve the training effect.

[0121] In step S130, the first deep deterministic policy gradient model is interacted with the power dispatch simulation environment to generate state transition samples, and the first deep deterministic policy gradient model is trained multiple times using the state transition samples until the first deep deterministic policy gradient model converges, and an optimized use strategy optimization network model is obtained.

[0122] In the embodiments of the present application, a first deep deterministic policy gradient model is first constructed, including a first use strategy optimization network, a first use value optimization network, a first target strategy optimization network, a first target value optimization network, and an experience replay pool, and a physical constraint correction module is set.

[0123] The structures of the first use strategy optimization network and the first use value optimization network are both deep neural networks, including an input layer, at least one hidden layer, and an output layer.

[0124] The first use strategy optimization network is used to calculate an original state and output a first action ; the physical constraint correction module is used to receive the first action and calculate a second action according to the physical constraint; the first use value optimization network is used to calculate the value of the current combination , The first target strategy optimization network and the first target value optimization network are used to update parameters to stabilize the training process. The experience replay pool is used to store serialized historical experience records.

[0125] According to the original state , the first action is output by the use strategy optimization network model, the first action is output as the second action after being corrected by the physical constraint, the first reward is obtained through the original reward after interacting with the power dispatch simulation environment, and the first state at the next moment Transition sample from first state Stored in the experience replay pool; a small batch of samples is randomly sampled multiple times from the experience replay pool as the second state transition samples. The parameters of the first use strategy optimization network and the first use value optimization network are updated by using the second state transition samples, respectively, to obtain the optimized use strategy optimization network model.

[0126] In step S140, the real-time status of the power consumption unit is input into the optimized network model of the usage strategy to generate the optimal power dispatch plan.

[0127] In this embodiment, the optimized usage strategy optimization network model is input with the real-time state of the power consumption unit, and outputs the optimal power dispatch plan to achieve optimization of the power distribution network dispatch plan. The parameters of the optimized usage strategy optimization network model can be used in subsequent training to increase training efficiency. That is, when planning for the next state, the parameters of the usage strategy optimization network model are initialized with the optimized usage strategy optimization network model parameters, and all steps above except parameter initialization are executed to obtain the plan for the next time step.

[0128] In one embodiment of this application, obtaining future electricity demand forecast data includes:

[0129] Acquire historical electricity demand data, historical external characteristic data, and initial neural network model for each electricity consumption unit;

[0130] The historical electricity demand data and historical external characteristic data of each power consumption unit are preprocessed to generate preprocessed data;

[0131] The preprocessed data is divided into training and testing sets;

[0132] The initial neural network model is trained using gradient descent and the training set, and the mean squared error loss function is calculated to update the model parameters until the MSE loss converges or the maximum number of iterations is reached, thus obtaining the neural network prediction model.

[0133] The neural network prediction model is tested using a test set to obtain the final prediction model.

[0134] The training set is input into the input layer of the final prediction model to obtain the output data of the input layer;

[0135] The output data of the input layer is fed into the long short-term memory network layer of the final prediction model to obtain the output data of the long short-term memory network layer.

[0136] The output data of the Long Short-Term Memory network layer is input into the fully connected layer of the final prediction model to obtain the output data of the fully connected layer.

[0137] The output data of the fully connected layer is input into the output layer of the final prediction model to obtain the future electricity demand prediction data for each electricity consumption unit.

[0138] In this embodiment, based on the periodic characteristics of the power demand of the power consumption unit, a Long Short-Term Memory (LSTM) network is selected as the initial neural network model for training and prediction.

[0139] Historical electricity demand data and historical external characteristic data of each electricity consumption unit are acquired. Preprocessing of this data involves handling missing values. In this embodiment, to ensure the temporal attributes of the original historical electricity demand data and historical external characteristic data of each electricity consumption unit are not corrupted, mean interpolation is used to handle missing values. Then, the historical electricity demand data and historical external characteristic data of each electricity consumption unit are normalized; in this embodiment, min-max normalization is used to obtain preprocessed data.

[0140] Based on the preprocessed data, the data was split, with 80% used as the training set and 20% as the test set. The dataset includes historical electricity demand data and historical external characteristic data for the same time period.

[0141] The MSE loss function is constructed, and the initial neural network model is trained using gradient descent. The MSE loss function is calculated to update the model parameters until the MSE loss converges or the maximum number of iterations is reached. Then, the training stops, and the model's performance is evaluated using test set data. Finally, the optimal future electricity demand prediction model corresponding to the optimal parameters is obtained, which is the final prediction model, and predictions are made.

[0142] The initial neural network model includes an input layer, an LSTM layer, a fully connected layer, and an output layer.

[0143] Specifically, the input layer is used to input the preprocessed training set data.

[0144] The LSTM layer is used to extract dependency information in the time dimension. The LSTM layer has k layers, and each layer performs the following operation:

[0145]

[0146] in, This represents the hidden state of the LSTM layer. This represents the parameters of the LSTM model. This represents the output data of the input layer, which is also the input data of the LSTM layer.

[0147] The fully connected layer receives the output of the k-th LSTM layer. And, using it as input, perform the following operations:

[0148]

[0149] in, For the output of the fully connected layer, The neuron parameter matrix, is the bias term, and ReLU is the activation function.

[0150] The output layer receives the output from the fully connected layer and ultimately outputs the future electricity demand forecast data for each power consumption unit. Taking the prediction of demand at time t as an example, ,but This represents the predicted electricity demand data for the x-th electricity-consuming unit at time t.

[0151] In one embodiment of this application, based on basic information of power dispatch planning and future electricity demand forecast data, an initial state, an initial action, and an initial reward are defined; including:

[0152] The basic information for power dispatch planning includes the time code of the power consumption unit, temperature, weather type code, date type code, peak type code, the real-time available capacity of the substation, the real-time capacity utilization rate of the substation, the real-time dispatchable capacity between substations, the average dispatchable capacity between substations in the previous H time periods, and the average dispatch data of each power consumption unit.

[0153] The original state is defined based on the time code, temperature, weather type code, date type code, peak type code, real-time available capacity of the substation, real-time capacity utilization rate of the substation, real-time dispatchable capacity between substations, average dispatchable capacity between substations in the previous H time periods, and average dispatch data of each power consumption unit.

[0154] The basic information for power dispatch planning also includes power dispatch planning between various substations and power dispatch planning between substations and power-consuming units;

[0155] Define the original actions based on the power dispatching plans between each substation and between the substation and the power consumption unit;

[0156] Based on the basic information of power dispatch planning, determine the incentives for reasonable capacity allocation, fixed dispatching behavior costs, line loss costs, penalties for unmet demand, and rewards for cross-site dispatching behavior.

[0157] The original reward is defined based on the capacity allocation reward, fixed scheduling behavior cost, line loss cost, unmet demand penalty, and cross-site scheduling behavior reward.

[0158] In the embodiments of this application, the original state It contains multiple state information at time t, specifically including the time code of the power-consuming unit. ,temperature Weather type codes Date type encoding Peak type coding The immediate available capacity of the substation Real-time capacity utilization rate of substations Real-time dispatchable capacity between substations Average dispatch capacity among substations in the previous H time periods And average dispatch data for each power consumption unit , is represented as:

[0159]

[0160] Among them, for The encoding method used is periodic two-dimensional encoding, that is, the encoding corresponding to time t is as follows:

[0161]

[0162] in, It represents 180°.

[0163] At time t, the original action The power dispatching plan that can be made by each substation represents the original action. This represents the power dispatching plan among various substations at time t. and power dispatch planning between power distribution stations and power consumption units , original action Represented as:

[0164]

[0165] in, , This represents the capacity dispatched from the i-th substation to the j-th substation at time t, with the diagonal elements being 0; , This represents the capacity that the i-th substation ultimately dispatches to the x-th power-consuming unit at time t.

[0166] Original reward It contains multiple components for evaluating actions. In state The overall benefits are as follows. The reward system reduces the probability of making incorrect choices while exploring the environment, thus improving optimization efficiency. Specifically:

[0167] Original reward Rewards should be allocated reasonably based on capacity. Fixed scheduling costs Line loss cost Penalties for unmet needs And cross-site scheduling behavior rewards composition.

[0168] Reasonable capacity allocation reward The definition is as follows:

[0169]

[0170] in, This is a weight vector representing the relative importance of each power consumption unit, indicating the priority of meeting the needs of each power consumption unit. This vector can be set according to the actual situation. It is an economic conversion factor used to convert capacity allocation values ​​into economic value. This helps to guide the prioritization of meeting the electricity needs of units that have greater need for electricity. This represents the predicted electricity demand data for the x-th electricity-consuming unit at time t.

[0171] Fixed scheduling cost The definition is as follows:

[0172]

[0173] in, Let represent the fixed cost of dispatching power from the i-th substation to the x-th power-consuming unit. This represents the fixed cost of dispatching between the i-th substation and the j-th substation; For indicator functions, when hour, Otherwise, it is 0. Similarly. This helps to guide and reduce unnecessary scheduling paths.

[0174] Line loss, also known as line loss, is a common phenomenon of electrical energy loss in power distribution. In this embodiment, the line loss rate is used to calculate specific line losses and line loss costs. The definition is as follows:

[0175]

[0176] in, , This represents the average line loss rate between the output side (i.e., the i-th substation) and the receiving side (i.e., the x-th power consumption unit). This represents the average line loss rate between the output (i.e., the i-th substation) and the receiving (i.e., the j-th substation). and It can be calculated using data from the previous H time points based on historical scheduling information. It is an economic conversion factor used to convert the value of lost electricity into economic value. Setting line loss costs helps to increase realism.

[0177] Penalty for unmet needs The definition is as follows:

[0178]

[0179] in, It is an economic conversion factor used to convert unmet electricity demand into economic value. This helps to ensure that power distribution dispatch meets demand as much as possible. This represents the predicted electricity demand data for the x-th electricity-consuming unit at time t.

[0180] Cross-site dispatching behavior rewards The definition is as follows:

[0181]

[0182] in, This represents the instantaneous capacity utilization rate of the output side (i.e., the i-th substation). This represents the instantaneous capacity utilization rate of the receiving station (i.e., the j-th substation). It is an economic conversion factor used to convert favorable power capacity dispatch values ​​into economic value. It helps guide the dispatch of electricity from low-utilization substations to high-utilization substations to maintain the stability of the distribution network, while supporting the consumption of new energy sources.

[0183] Among them, the economic conversion coefficient The values ​​multiplied are all electricity demand or capacity values, so the values ​​can be taken with reference to the real-time transmission and distribution price or sales price.

[0184] Original reward The expression is as follows:

[0185]

[0186] Among them, the weight combination vector Each element in the set is a value greater than 0, and the weights are adjusted to combine them. This allows for control over the importance of each component and keeps the reward value within a reasonable range, which helps improve training effectiveness.

[0187] In some embodiments of this application, the first deep deterministic policy gradient model includes a first usage policy optimization network, a first usage value optimization network, a first target value optimization network, and a first target policy optimization network. The first deep deterministic policy gradient model interacts with a power dispatching simulation environment to generate state transition samples. These state transition samples are then used multiple times to train the first deep deterministic policy gradient model until the model converges, resulting in an optimized usage policy optimization network model, including:

[0188] Based on the power dispatch simulation environment, obtain the original state;

[0189] The original state is input into the first policy optimization network to generate the first action;

[0190] The first action is modified with physical constraints to generate the second action;

[0191] In the power dispatch simulation environment, the second action is performed to obtain the first reward and the first state;

[0192] Based on the original state, the second action, the first reward, and the first state, determine the first state transition sample and store it in the experience replay pool. If the experience replay pool is full, remove the earliest state transition sample stored in the experience replay pool.

[0193] Repeat the above steps until the number of first state transition samples in the experience replay pool reaches the preset threshold number. Then, extract a batch of first state transition samples from the experience replay pool and determine them as second state transition samples. The second state transition samples include the second state, the third action, the second reward, and the third state.

[0194] The third state from the second state transition sample is input into the first objective policy optimization network to obtain the fourth action;

[0195] By applying physical constraints to the fourth action, the fifth action is obtained.

[0196] The third state and the fifth action in the second state transition sample are input into the first target value optimization network to obtain the future value estimate of the first target;

[0197] The value of the first objective is obtained based on the estimated future value of the first objective, the second reward and the discount factor in the second state transition sample;

[0198] The second state and the third action in the second state transition sample are input into the first use value optimization network to obtain the first use value;

[0199] Based on the primary objective value and primary use value, a primary loss function is constructed;

[0200] The first use value optimization network is adjusted according to the first loss function to obtain the adjusted first use value optimization network;

[0201] The second state is input into the adjusted first use strategy optimization network to obtain the sixth action;

[0202] By applying physical constraints to the sixth action, the seventh action is obtained.

[0203] The second state and the seventh action are input into the adjusted first use value optimization network, and the first use policy optimization network is updated using the policy gradient ascent algorithm and the gradient backpropagation method to obtain the updated first use policy optimization network.

[0204] Repeat the above steps to update the first use value optimization network and the first use strategy optimization network multiple times until the model converges, resulting in the optimized use value optimization network and the optimized use strategy optimization network.

[0205] In the embodiments of this application, the original state This refers to the environmental state data at any given time t, based on the original state. The initial state is input into the network using the first strategy optimization method. Then, the output is obtained and exploration noise is added to obtain the first action. The first action This refers to the action instructions generated by the first policy network based on the original state. As shown in the following formula:

[0206]

[0207] in, Represents the original state The exploration noise added below.

[0208] The first action to optimize network output using the first strategy. This may not meet the physical constraints of real-world situations. In this embodiment, in order to ensure that the first action of the network output optimized by the first strategy during training satisfies the physical constraints and conforms to real-world conditions, the first action is... The second action is output after the physical constraint correction module. As shown in the following formula:

[0209]

[0210] in, Denotes the 2-norm of a matrix. Indicates finding Minimal action , This represents the available capacity of the i-th substation at time t. This is used to simulate the minimum power demand of each power consumption unit, and its value can be selected according to the actual situation. Among the constraints, the first constraint is a capacity constraint, that is, the total output of each substation shall not exceed its available capacity plus the input capacity. The second constraint is a minimum demand constraint, that is, the total dispatch volume for each power consumption unit should at least meet its minimum demand.

[0211] Due to the calculation The Karush-Kuhn-Tucker (KKT) conditions are satisfied, therefore it can be based on the first action. With the second action Calculate the first action Compared to the second action The gradient, i.e., the first action Compared to the second action The Jacobian matrix is ​​represented as .

[0212] The original state of the power dispatch simulation environment Second action combination After interaction, update the state to the first state. And generate the first reward. The first state transition sample that records the complete experience formed by this interaction. The above steps are performed on all basic information of power dispatch planning and future electricity demand forecast data, and the first state transition sample obtained after each execution is stored in the experience replay pool. If the number of experience records in the experience replay pool is less than a preset threshold, the process is repeated multiple times until a sufficient number of experience records are obtained in the experience replay pool; if the experience replay pool is full, the earliest state transition sample stored in the experience replay pool is removed.

[0213] A small batch is randomly selected from the experience replay pool. A historical experience record, represented as a second-state transition sample. Transition from the second state to the third state in the sample Input the target policy to optimize the network and obtain the fourth action. Then, following the same steps as above, perform the fourth action. Perform physical constraint corrections to obtain the fifth action as the output after physical constraint correction. Combine the third state and the fifth action. By using a common input target value optimization network, the value of the combination is obtained, which is the estimated future value of the first target. ;

[0214] The second state and The combination of the third action Inputting the data into the use value optimization network yields the value of the combination, i.e., the first use value. Then, the first target value is calculated based on the following formula. and the first loss function of the first use value optimization network :

[0215]

[0216] in, As a discount factor, This indicates the expectation calculation, based on The parameters of the first value-optimized network are updated by backpropagating the gradient of this loss. As shown in the following formula:

[0217]

[0218] in, Indicates to Find the gradient. This represents the learning rate of the network used to optimize value in the first application.

[0219] Update the parameters of the first policy-optimized network using the policy gradient ascent algorithm. This causes the first-use value optimization network to tend to output actions that yield higher value. Specifically, it adjusts the first-use policy optimization network... For the sixth action The gradient is backpropagated to the seventh action. The policy gradient is calculated as follows:

[0220]

[0221] Among them, the sixth action Indicates the second state Input the output of the network after optimization using the first strategy and correction for physical constraints, the seventh action. Indicates the second state Input the output obtained after optimizing the network using the first strategy.

[0222] Based on the calculated policy gradient, the parameters of the first policy optimization network are updated to obtain the updated parameters of the first policy optimization network. As shown in the following formula:

[0223]

[0224] in, This indicates that the first strategy used is to optimize the learning rate of the network.

[0225] Based on the above steps, after multiple training iterations to bring the model to convergence, the real-time state of the power consumption unit will be recorded. The first optimized strategy is used as input to optimize the network, and the optimal power dispatch plan is output. This enables the optimization of power distribution network scheduling and planning. The network parameters trained at the current moment can be used in subsequent training sessions to increase training efficiency. This involves monitoring the real-time status of power-consuming units. The next state During the planning process, the parameters of the deep deterministic policy gradient model are initialized using the parameters of the currently saved predictive neural network model, and all steps except parameter initialization are executed to obtain the planning for the next time step.

[0226] In one embodiment of this application, the optimized policy optimization network model is further obtained by training the first deep deterministic policy gradient model multiple times using state transition samples.

[0227] Each time the use value optimization network is updated, the target value optimization network parameters are softly updated based on the updated use value optimization network parameters.

[0228] Each time the network is updated using the strategy optimization method, the target strategy optimization method network parameters are softly updated based on the updated network parameters.

[0229] In this embodiment, after each update of the first usage strategy optimization network and the first usage value optimization network, the parameters of the updated first usage strategy optimization network and the updated first usage value optimization network are periodically synchronized to the first target value optimization network and the first target strategy optimization network via a soft update method, as follows:

[0230]

[0231] in, For update rate, and .

[0232] In one embodiment of this application, the method for constructing a first deep deterministic policy gradient model includes:

[0233] An initial deep deterministic policy gradient model is constructed, which includes an initial use policy optimization network, an initial use value optimization network, an initial target value optimization network, and an initial target policy optimization network. An experience replay pool and a physical constraint correction module are also constructed. The experience replay pool is used to store serialized historical experience records. The physical constraint correction module is used to correct the output of the use policy optimization network so that its output meets the physical constraints under real-world conditions.

[0234] The initial use policy optimization network, initial use value optimization network, initial target value optimization network, and initial target policy optimization network are initialized to obtain a first deep deterministic policy gradient model. In the first deep deterministic policy gradient model, the structure of the first use policy optimization network is the same as that of the first target policy optimization network, the parameters of the first use policy optimization network are the same as those of the first target policy optimization network, the structure of the first use value optimization network is the same as that of the first target value optimization network, and the parameters of the first use value optimization network are the same as those of the first target value optimization network.

[0235] In the embodiments of this application, an initial deep deterministic policy gradient model network (initial DDPG network) is constructed, including: an initial use policy optimization network (initial Actor network), an initial use value optimization network (initial Critic network), an initial target value optimization network (initial target Actor network), and an initial target policy optimization network (initial target Critic network), as well as an experience replay pool, and a physical constraint correction module is set up to initialize the initial deep deterministic policy gradient model (initial DDPG network).

[0236] Both the initial use strategy optimization network and the initial use value optimization network are deep neural networks, containing an input layer, at least one hidden layer, and an output layer.

[0237] The initial strategy optimizes the network to receive the raw state. And output the first action. The physical constraint correction module is used to receive the first action. And calculate and output the second action based on the physical constraints of the physical constraint correction module. The initial value optimization network is used to calculate the value of the current combination. The first-objective policy optimization network and the first-objective value optimization network are used to update parameters to stabilize the training process. The experience replay pool is used to store serialized historical experience records.

[0238] The initial deep deterministic policy gradient model network (initial DDPG network) is initialized, resulting in: the first policy optimization network. With the first objective strategy optimization network With the same structure and parameter settings, the first use is a value optimization network. With the first objective value optimization network They have the same structure and parameter settings. , , among which, among which, , Let these represent the parameters in the first use strategy optimization network and the first use value optimization network, respectively. , denoted as parameters in the first objective policy optimization network and the first objective value optimization network, respectively.

[0239] In one embodiment of this application, historical electricity demand data and historical external characteristic data of each electricity consumption unit are preprocessed to generate preprocessed data, including:

[0240] Interpolation and normalization are performed on the historical electricity demand data and historical external characteristic data of each electricity consumption unit to generate preprocessed data.

[0241] In the embodiments of this application, the historical electricity demand data and historical external characteristic data of each electricity consumption unit are preprocessed. Specifically, missing value processing is performed on the collected historical electricity demand data and historical external characteristic data of each electricity consumption unit. In this embodiment, to ensure that the temporal attributes of the original historical electricity demand data and historical external characteristic data of each electricity consumption unit are not destroyed, the mean interpolation method is used to process missing values. Then, the historical electricity demand data and historical external characteristic data of each electricity consumption unit are normalized; in this embodiment, the min-max normalization method is used for normalization to obtain preprocessed data.

[0242] Secondly, embodiments of this application provide a power dispatching and planning system, including:

[0243] The first acquisition module is used to acquire basic information on power dispatch planning and future electricity demand forecast data.

[0244] The first construction module is used to build a deep reinforcement learning power dispatch simulation environment based on the basic information of power dispatch planning and the forecast data of future electricity demand. The original state, original action and original reward are defined through the deep reinforcement learning power dispatch simulation environment.

[0245] The first module is used to interact with the power dispatch simulation environment through the first deep deterministic policy gradient model to generate state transition samples, and to train the first deep deterministic policy gradient model multiple times using the state transition samples until the first deep deterministic policy gradient model converges, thereby obtaining the optimized policy optimization network model.

[0246] The first generation module is used to optimize the network model of the optimized usage strategy and input the real-time status of the power consumption unit to generate the optimal power dispatch plan.

[0247] In this embodiment, basic information on power dispatch planning and future electricity demand forecast data are integrated into a first deep deterministic strategy gradient model for training, resulting in an optimized strategy model. By inputting the real-time status of power-consuming units into the optimized strategy model, the training of the first deep deterministic strategy gradient model considers the power dispatch decisions of multiple substations and user units, as well as future electricity demand, thus making the power dispatch planning decisions more forward-looking and accurate. Furthermore, by combining the ability to predict the future with the ability to optimize dispatch, prediction and decision-making are integrated, improving the adaptability and robustness of power planning strategies. This makes the power dispatch planning strategies more accurate and effectively solves the problems of existing two-stage traditional methods based on "prediction-optimization," which lack adaptability and robustness, completely decouple prediction and decision-making, causing prediction errors to directly propagate and affect the optimization results.

[0248] In one embodiment of this application, obtaining future electricity demand forecast data includes:

[0249] Acquire historical electricity demand data, historical external characteristic data, and initial neural network model for each electricity consumption unit;

[0250] The historical electricity demand data and historical external characteristic data of each power consumption unit are preprocessed to generate preprocessed data;

[0251] The preprocessed data is divided into training and testing sets;

[0252] The initial neural network model is trained using gradient descent and the training set, and the mean squared error loss function is calculated to update the model parameters until the MSE loss converges or the maximum number of iterations is reached, thus obtaining the neural network prediction model.

[0253] The neural network prediction model is tested using a test set to obtain the final prediction model.

[0254] The training set is input into the input layer of the final prediction model to obtain the output data of the input layer;

[0255] The output data of the input layer is fed into the long short-term memory network layer of the final prediction model to obtain the output data of the long short-term memory network layer.

[0256] The output data of the Long Short-Term Memory network layer is input into the fully connected layer of the final prediction model to obtain the output data of the fully connected layer.

[0257] The output data of the fully connected layer is input into the output layer of the final prediction model to obtain the future electricity demand prediction data for each electricity consumption unit.

[0258] In one embodiment of this application, based on basic information of power dispatch planning and future electricity demand forecast data, an initial state, an initial action, and an initial reward are defined; including:

[0259] The basic information for power dispatch planning includes the time code of the power consumption unit, temperature, weather type code, date type code, peak type code, the real-time available capacity of the substation, the real-time capacity utilization rate of the substation, the real-time dispatchable capacity between substations, the average dispatchable capacity between substations in the previous H time periods, and the average dispatch data of each power consumption unit.

[0260] The original state is defined based on the time code, temperature, weather type code, date type code, peak type code, real-time available capacity of the substation, real-time capacity utilization rate of the substation, real-time dispatchable capacity between substations, average dispatchable capacity between substations in the previous H time periods, and average dispatch data of each power consumption unit.

[0261] The basic information for power dispatch planning also includes power dispatch planning between various substations and power dispatch planning between substations and power-consuming units;

[0262] Define the original actions based on the power dispatching plans between each substation and between the substation and the power consumption unit;

[0263] Based on the basic information of power dispatch planning, determine the incentives for reasonable capacity allocation, fixed dispatching behavior costs, line loss costs, penalties for unmet demand, and rewards for cross-site dispatching behavior.

[0264] The original reward is defined based on the capacity allocation reward, fixed scheduling behavior cost, line loss cost, unmet demand penalty, and cross-site scheduling behavior reward.

[0265] In one embodiment of this application, the first deep deterministic policy gradient model includes a first usage policy optimization network, a first usage value optimization network, a first target value optimization network, and a first target policy optimization network. The first deep deterministic policy gradient model interacts with a power dispatching simulation environment to generate state transition samples. These state transition samples are then used multiple times to train the first deep deterministic policy gradient model until the model converges, resulting in an optimized usage policy optimization network model, including:

[0266] Based on the power dispatch simulation environment, obtain the original state;

[0267] The original state is input into the first policy optimization network to generate the first action;

[0268] The first action is modified with physical constraints to generate the second action;

[0269] In the power dispatch simulation environment, the second action is performed to obtain the first reward and the first state;

[0270] Based on the original state, the second action, the first reward, and the first state, determine the first state transition sample and store it in the experience replay pool. If the experience replay pool is full, remove the earliest stored state transition sample according to the first-in-first-out principle.

[0271] Repeat the above steps until the number of first state transition samples in the experience replay pool reaches the preset threshold number. Then, extract a batch of first state transition samples from the experience replay pool and determine them as second state transition samples. The second state transition samples include the second state, the third action, the second reward, and the third state.

[0272] The third state from the second state transition sample is input into the first objective policy optimization network to obtain the fourth action;

[0273] By applying physical constraints to the fourth action, the fifth action is obtained.

[0274] The third state and the fifth action in the second state transition sample are input into the first target value optimization network to obtain the future value estimate of the first target;

[0275] The value of the first objective is obtained based on the estimated future value of the first objective, the second reward and the discount factor in the second state transition sample;

[0276] The second state and the third action in the second state transition sample are input into the first use value optimization network to obtain the first use value;

[0277] Based on the primary objective value and primary use value, a primary loss function is constructed;

[0278] The first use value optimization network is adjusted according to the first loss function to obtain the adjusted first use value optimization network;

[0279] The second state is input into the adjusted first use strategy optimization network to obtain the sixth action;

[0280] The sixth action is modified by physical constraints to obtain the seventh action;

[0281] The second state and the seventh action are input into the adjusted first use value optimization network, and the adjusted first use policy optimization network is updated using the policy gradient ascent algorithm and the gradient backpropagation method to obtain the updated first use policy optimization network.

[0282] Repeat the above steps to update the first use value optimization network and the first use strategy optimization network multiple times until the model converges, resulting in the optimized use value optimization network and the optimized use strategy optimization network.

[0283] In one embodiment of this application, the first deep deterministic policy gradient model interacts with the power dispatch simulation environment to generate state transition samples, and the first deep deterministic policy gradient model is trained multiple times using the state transition samples until the model converges, resulting in an optimized policy optimization network model. This further includes:

[0284] Each time the use value optimization network is updated, the target value optimization network parameters are softly updated based on the updated use value optimization network parameters.

[0285] Each time the network is updated using the strategy optimization method, the target strategy optimization method network parameters are softly updated based on the updated network parameters.

[0286] In one embodiment of this application, the method for constructing a first deep deterministic policy gradient model includes:

[0287] An initial deep deterministic policy gradient model is constructed, which includes an initial use policy optimization network, an initial use value optimization network, an initial target value optimization network, and an initial target policy optimization network. An experience replay pool and a physical constraint correction module are also constructed. The experience replay pool is used to store serialized historical experience records. The physical constraint correction module is used to correct the output of the use policy optimization network so that its output meets the physical constraints under real-world conditions.

[0288] The initial use policy optimization network, initial use value optimization network, initial target value optimization network, and initial target policy optimization network are initialized to obtain a first deep deterministic policy gradient model. In the first deep deterministic policy gradient model, the structure of the first use policy optimization network is the same as that of the first target policy optimization network, the parameters of the first use policy optimization network are the same as those of the first target policy optimization network, the structure of the first use value optimization network is the same as that of the first target value optimization network, and the parameters of the first use value optimization network are the same as those of the first target value optimization network.

[0289] In one embodiment of this application, historical electricity demand data and historical external characteristic data of each electricity consumption unit are preprocessed to generate preprocessed data, including:

[0290] Interpolation and normalization are performed on the historical electricity demand data and historical external characteristic data of each electricity consumption unit to generate preprocessed data.

[0291] The functions of each module in each device in the embodiments of this application can be found in the corresponding descriptions in the above methods, and will not be repeated here.

[0292] Figure 2 A structural block diagram of an electronic device according to an embodiment of this application is shown. Figure 2 As shown, the electronic device includes a memory 410 and a processor 420. The memory 410 stores instructions that can be executed on the processor 420. When the processor 420 executes the instructions, it implements the power dispatch planning method in the above embodiments. The number of memories 410 and processors 420 can be one or more. This electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0293] The electronic device may also include a communication interface 430 for communicating with external devices and exchanging data. The devices are interconnected using different buses and can be mounted on a common motherboard or otherwise as needed. The processor 420 can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). The bus can be divided into address buses, data buses, control buses, etc. For ease of illustration, Figure 2 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0294] Optionally, in a specific implementation, if the memory 410, processor 420 and communication interface 430 are integrated on a single chip, the memory 410, processor 420 and communication interface 430 can communicate with each other through an internal interface.

[0295] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0296] This application provides a computer-readable storage medium (such as the memory 410 described above) that stores computer instructions, which, when executed by a processor, implement the method provided in this application.

[0297] Optionally, memory 410 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device, etc. Furthermore, memory 410 may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 410 may optionally include memory remotely located relative to processor 420, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0298] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0299] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the order or number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0300] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more (two or more) executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0301] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0302] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0303] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0304] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A power dispatching planning method, characterized in that, include: The basic information for power dispatch planning and future electricity demand forecast data are obtained. The basic information for power dispatch planning includes power supply structure information, power consumption unit environmental information, power consumption unit time characteristic information, substation status information, and historical dispatch information. Based on the basic information of the power dispatch plan and the predicted future electricity demand, a deep reinforcement learning power dispatch simulation environment is constructed, and the original state, original action and original reward are defined through the deep reinforcement learning power dispatch simulation environment. The first deep deterministic policy gradient model interacts with the power dispatch simulation environment to generate state transition samples. These state transition samples are then used to train the first deep deterministic policy gradient model multiple times until it converges, resulting in an optimized usage policy optimization network model. The first deep deterministic policy gradient model includes a first usage policy optimization network and a first value optimization network. This model learns and forms a deterministic mapping policy from the current state of the power dispatch simulation environment to the optimal action. The state transition samples are empirical data recording a single interaction between the first deep deterministic policy gradient model and the power dispatch simulation environment. These samples provide reward feedback to the first deep deterministic policy gradient model to adjust and improve it. The optimized power dispatch plan is generated by taking the real-time status of the power consumption unit into the optimized network model of the usage strategy.

2. The method according to claim 1, characterized in that, Obtaining the future electricity demand forecast data includes: Acquire historical electricity demand data, historical external characteristic data, and initial neural network model for each electricity consumption unit; The historical electricity demand data and historical external characteristic data of each power consumption unit are preprocessed to generate preprocessed data; The preprocessed data is divided into a training set and a test set; The initial neural network model is trained using gradient descent and the training set, and the mean squared error loss function is calculated to update the model parameters until the MSE loss converges or the maximum number of iterations is reached, thus obtaining the neural network prediction model. The neural network prediction model is tested based on the test set to obtain the final prediction model; The training set is input into the input layer of the final prediction model to obtain the output data of the input layer; The output data of the input layer is input into the long short-term memory network layer of the final prediction model to obtain the output data of the long short-term memory network layer. The output data of the Long Short-Term Memory network layer is input into the fully connected layer of the final prediction model to obtain the output data of the fully connected layer. The output data of the fully connected layer is input into the output layer of the final prediction model to obtain the future electricity demand prediction data for each electricity consumption unit.

3. The method according to claim 2, characterized in that, Based on the basic information of the power dispatch plan and the predicted future electricity demand data, a deep reinforcement learning-based power dispatch simulation environment is constructed. This deep reinforcement learning-based power dispatch simulation environment defines the initial state, initial action, and initial reward, including: The basic information for the power dispatch planning includes the time code of the power consumption unit, temperature, weather type code, date type code, peak type code, the real-time available capacity of the substation, the real-time capacity utilization rate of the substation, the real-time dispatchable capacity between substations, the average dispatchable capacity between substations in the previous H time periods, and the average dispatch data of each power consumption unit. The original state is defined based on the time code of the power consumption unit, the temperature, the weather type code, the date type code, the peak type code, the real-time available capacity of the substation, the real-time capacity utilization rate of the substation, the real-time dispatchable capacity between the substations, the average dispatchable capacity between the substations in the previous H time periods, and the average dispatch data of each power consumption unit. The basic information of the power dispatching plan also includes the power dispatching plan between each substation and the power dispatching plan between the substation and the power consumption unit. Define the original actions based on the power dispatching plans between the various power distribution stations and the power dispatching plans between the power distribution stations and the power consumption units; Based on the basic information of the power dispatch plan, determine the incentive for reasonable capacity allocation, the cost of fixed dispatch behavior, the cost of line loss, the penalty for unmet demand, and the incentive for cross-site dispatch behavior. The original reward is defined based on the capacity allocation reward, the fixed scheduling behavior cost, the line loss cost, the unmet demand penalty, and the cross-site scheduling behavior reward.

4. The method according to claim 3, characterized in that, The first deep deterministic policy gradient model includes a first usage policy optimization network, a first usage value optimization network, a first target value optimization network, and a first target policy optimization network. The process involves interacting with the power dispatch simulation environment through the first deep deterministic policy gradient model to generate state transition samples, and repeatedly training the first deep deterministic policy gradient model using these state transition samples until the model converges, resulting in an optimized usage policy optimization network model. Based on the power dispatch simulation environment, obtain the original state; The original state is input into the first policy optimization network to generate the first action; The first action is modified with physical constraints to generate the second action; The second action is performed in the power dispatch simulation environment to obtain a first reward and a first state; Based on the original state, the second action, the first reward, and the first state, a first state transition sample is determined and stored in the experience replay pool. If the experience replay pool is full, the earliest state transition sample stored in the experience replay pool is removed. Repeat the above steps until the number of first state transition samples in the experience replay pool reaches a preset threshold number. Then, extract a batch of first state transition samples from the experience replay pool and determine them as second state transition samples. The second state transition sample includes a second state, a third action, a second reward, and a third state. The third state in the second state transition sample is input into the first target policy optimization network to obtain the fourth action; The fourth action is modified by physical constraints to obtain the fifth action; The third state and the fifth action in the second state transition sample are input into the first target value optimization network to obtain the first target future value estimate; The first target value is obtained based on the first target future value estimate, the second reward and discount factor in the second state transition sample; The second state and the third action in the second state transition sample are input into the first use value optimization network to obtain the first use value; Based on the first target value and the first use value, a first loss function is constructed; The first use value optimization network is adjusted according to the first loss function to obtain the adjusted first use value optimization network. The second state is input into the adjusted first usage strategy optimization network to obtain the sixth action; The sixth action is modified by physical constraints to obtain the seventh action; The second state and the seventh action are input into the adjusted first use value optimization network, and the adjusted first use policy optimization network is updated using the policy gradient ascent algorithm and the gradient backpropagation method to obtain the updated first use policy optimization network. Repeat the above steps to adjust and update the first use value optimization network and the first use strategy optimization network multiple times until the first deep deterministic policy gradient model converges, resulting in the optimized use value optimization network and the optimized use strategy optimization network.

5. The method according to claim 4, characterized in that, The process of interacting with the power dispatch simulation environment through a first deep deterministic policy gradient model to generate state transition samples, and repeatedly training the first deep deterministic policy gradient model using these state transition samples until the model converges, to obtain an optimized policy optimization network model, further includes: Each time the use value optimization network is updated, the target value optimization network parameters are updated based on the updated use value optimization network parameters. Each time the usage strategy optimization network is updated, the target strategy optimization network parameters are updated based on the updated usage strategy optimization network parameters.

6. The method according to claim 5, characterized in that, The method for constructing the first deep deterministic policy gradient model includes: An initial deep deterministic policy gradient model is constructed, which includes an initial use policy optimization network, an initial use value optimization network, an initial target value optimization network, and an initial target policy optimization network. An experience replay pool and a physical constraint correction module are constructed. The experience replay pool is used to store serialized historical experience records. The physical constraint correction module is used to correct the output of the use policy optimization network so that its output meets the physical constraints under real conditions. The initial use policy optimization network, the initial use value optimization network, the initial target value optimization network, and the initial target policy optimization network are initialized to obtain a first deep deterministic policy gradient model. In the first deep deterministic policy gradient model, the structure of the first use policy optimization network is the same as that of the first target policy optimization network, the parameters of the first use policy optimization network are the same as those of the first target policy optimization network, the structure of the first use value optimization network is the same as that of the first target value optimization network, and the parameters of the first use value optimization network are the same as those of the first target value optimization network.

7. The method according to claim 6, characterized in that, The step of preprocessing the historical electricity demand data and historical external characteristic data of each electricity unit to generate preprocessed data includes: The historical electricity demand data and historical external characteristic data of each power consumption unit are interpolated and normalized to generate preprocessed data.

8. A power dispatching and planning system, characterized in that, include: The first acquisition module is used to acquire basic information of power dispatch planning and future electricity demand forecast data. The basic information of power dispatch planning includes power supply structure information, power consumption unit environment information, power consumption unit time characteristic information, substation status information and historical dispatch information. The first construction module is used to construct a deep reinforcement learning power dispatch simulation environment based on the basic information of the power dispatch plan and the future power demand forecast data, and to define the original state, original action and original reward through the deep reinforcement learning power dispatch simulation environment. The first obtaining module is used to interact with the power dispatch simulation environment through a first deep deterministic policy gradient model, generate state transition samples, and train the first deep deterministic policy gradient model multiple times using the state transition samples until the first deep deterministic policy gradient model converges, thereby obtaining an optimized usage policy optimization network model. The first deep deterministic policy gradient model includes a first usage policy optimization network and a first value optimization network. The first deep deterministic policy gradient model is used to learn and form a deterministic mapping policy from the current state of the power dispatch simulation environment to the optimal action. The state transition samples are empirical data recording a single interaction process between the first deep deterministic policy gradient model and the power dispatch simulation environment. The state transition samples are used to provide reward feedback to the first deep deterministic policy gradient model to adjust and improve the first deep deterministic policy gradient model. The first generation module is used to input the real-time status of the power consumption unit into the optimized network model of the usage strategy optimization, and generate the optimal power dispatch plan.

9. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method as described in any one of claims 1-7.

Images