Micro-grid multi-energy optimization method and device integrated with AI algorithm and medium
Through the microgrid multi-energy optimization method with integrated AI algorithms, the graph attention near-end strategy optimization algorithm and graph attention technology are used to build a dynamic graph model and strategy optimization model, which solves the problem that microgrids are difficult to achieve efficient and stable operation in a multi-energy environment, and achieves efficient and economical energy scheduling and system stability.
Patent Information
- Application Number
- CN202510106398.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-23
- Publication Date
- 2025-05-23
AI Technical Summary
It is difficult for microgrids to achieve efficient and stable operation in a multi-energy environment, mainly due to the random output characteristics of distributed energy, which makes it difficult to maintain power balance, which in turn causes voltage fluctuations, affecting power supply quality and system stability.
The microgrid multi-energy optimization method is adopted with an integrated AI algorithm. The microgrid node correlation is modeled through the graph attention near-end strategy optimization algorithm, a dynamic graph model is constructed, and the graph structural features are extracted through graph attention technology, which is transformed into the Markov decision-making process, a strategy optimization model is constructed, and the microgrid optimal strategy is finally output.
It realizes accurate capture of the relationship between each node in the microgrid, improves the efficiency and stability of energy scheduling, significantly reduces operating costs, improves energy utilization efficiency, and enhances the reliability of the microgrid.
Smart Images
Figure CN120031195A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of microgrid technology, and in particular to a microgrid multi-energy optimization method, device and medium integrating an AI algorithm. Background Art
[0002] At present, microgrids are usually integrated with multiple distributed power sources (such as solar photovoltaic panels, wind turbines, etc.), loads (including households, commercial and industrial users, etc.) and energy storage devices (such as battery energy storage systems), forming a relatively independent and smaller-scale new power system, representing an innovative energy supply and management model.
[0003] However, with the rapid development of renewable energy technology, especially the widespread application of distributed photovoltaic power generation, the installed capacity of microgrids has increased significantly year by year. At the same time, with the popularization of electric vehicles, the sharp increase in electric vehicle loads has also posed new challenges to the stable operation of microgrids. These changes have greatly affected the supply and demand balance of low-voltage microgrids, making the management of microgrids increasingly complex. The traditional method of relying on continuous monitoring and management by manual experts in the power control room can no longer meet the current needs of efficient and stable operation of multi-energy microgrids.
[0004] In the management of multi-energy microgrids, optimal scheduling is a core issue. Although the introduction of distributed energy has improved the diversity and flexibility of energy utilization, its small-scale volatility and intermittent characteristics have brought great difficulties to the optimal allocation and scheduling of microgrids. The random output characteristics of distributed energy make it difficult to maintain the power balance within the microgrid, which in turn causes voltage fluctuations, affects the power supply quality and system stability, and may not only cause equipment damage, but also affect the normal electricity demand of users, reducing the energy efficiency and economy of the entire microgrid. Summary of the invention
[0005] The embodiments of the present application provide a microgrid multi-energy optimization method, device and medium integrating AI algorithm to solve the above-mentioned technical problems.
[0006] On the one hand, an embodiment of the present application provides a microgrid multi-energy optimization method integrating an AI algorithm, including:
[0007] Through the graph attention proximal strategy optimization algorithm, the node correlation in the microgrid is modeled to obtain the corresponding dynamic graph model;
[0008] By using the graph attention technology, feature extraction is performed in the dynamic graph model to obtain graph structure features, and the operation process of the microgrid is converted into a Markov decision process to construct a corresponding microgrid strategy optimization model;
[0009] The graph structure features are input into the microgrid strategy optimization model, and the final action is output to obtain the optimal strategy of the microgrid, and the multi-energy optimization of the microgrid is realized according to the optimal strategy of the microgrid.
[0010] In one implementation of the present application, the node correlation in the microgrid is modeled through the graph attention proximal strategy optimization algorithm to obtain the corresponding dynamic graph model, which specifically includes:
[0011] Combine the proximal strategy optimization algorithm with the graph attention technology to obtain the graph attention proximal strategy optimization algorithm;
[0012] Through the graph attention proximal strategy optimization algorithm, the nodes in the microgrid and the association relationships between different nodes are modeled to project the grid state information of the microgrid into node embedding vectors to obtain the corresponding dynamic graph model.
[0013] In one implementation of the present application, it also includes:
[0014] When modeling the node correlation in the microgrid, the attention coefficient corresponding to each neighborhood node in the dynamic graph model is calculated; wherein the attention coefficient is used to indicate the influence degree of the corresponding neighborhood node on the central node;
[0015] The feature vectors of the neighborhood nodes of the central node are aggregated through an attention mechanism to obtain the feature vector of the central node.
[0016] In one implementation of the present application, the attention coefficient corresponding to each neighborhood node in the dynamic graph model is calculated, specifically including:
[0017] The attention coefficient is calculated by the following formula:
[0018]
[0019] Among them, h i represents the central node feature vector, h j represents the neighborhood node feature vector, e ij Represents the attention weight coefficient, that is, the neighborhood node h j For the central node h i Relu() represents the activation function, W represents the trainable coefficient, ∥ represents the matrix concatenation operator, a represents the parameter for the dot product operation, exp() represents the exponential function, Represents the summation expression, which is used to calculate the energy sum of the central node, N i Indicates the natural growth number.
[0020] In one implementation of the present application, before inputting the graph structure feature into the microgrid strategy optimization model and outputting the final action to obtain the optimal strategy of the microgrid, the method further includes:
[0021] According to the attention coefficient corresponding to each neighborhood node, weighted summation is performed on the graph structure features;
[0022] The weighted sum is calculated by the following formula:
[0023]
[0024] Among them, h , i Represents the weighted sum of the central node feature vectors.
[0025] In one implementation of the present application, the graph structure feature is input into the microgrid strategy optimization model, and the final action is output to obtain the optimal strategy of the microgrid, which specifically includes:
[0026] Inputting the graph structure features into the microgrid strategy optimization model to extract the weighted summed input features through a graph convolutional network;
[0027] The extracted features are activated through the activation function, and the corresponding final actions are output through the fully connected layer to obtain the optimal strategy for the microgrid.
[0028] In one implementation of the present application, before outputting the final action to obtain the optimal strategy for the microgrid, the method further includes:
[0029] The environmental state in the microgrid is collected by a preset collection device, and the environmental state is input into the Actor network of the graph attention proximal strategy optimization algorithm;
[0030] Outputting the action of the microgrid through the Actor network, and generating a trajectory of the microgrid according to the action; wherein the trajectory is used to represent the trajectory corresponding to the action, state and reward;
[0031] Inputting the trajectory into the Critic network of the graph attention proximal strategy optimization algorithm to output a state value estimate of the microgrid;
[0032] The state value estimation is input into the action advantage function to update the network parameters corresponding to the Actor network and the Critic network, thereby optimizing the graph attention proximal strategy optimization algorithm.
[0033] In one implementation of the present application, feature extraction is performed in the dynamic graph model through graph attention technology to obtain graph structure features, and the operation process of the microgrid is converted into a Markov decision process to construct a corresponding microgrid strategy optimization model, which specifically includes:
[0034] By using graph attention technology, the local neighbor information of each node in the microgrid is aggregated in the dynamic graph model to capture the association relationship between the nodes in the microgrid and obtain the corresponding graph structure features;
[0035] The graph structure features are used as input, and a state space is constructed according to the real-time state of the microgrid, and an action space is constructed based on the energy scheduling strategy, so as to transform the operation process of the microgrid into a Markov decision process and obtain the corresponding microgrid strategy optimization model.
[0036] On the other hand, an embodiment of the present application further provides a microgrid multi-energy optimization device integrating an AI algorithm, the device comprising:
[0037] at least one processor;
[0038] and, a memory communicatively coupled to the at least one processor;
[0039] Among them, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a microgrid multi-energy optimization method integrating an AI algorithm as described above.
[0040] On the other hand, an embodiment of the present application also provides a non-volatile computer storage medium storing computer executable instructions, which, when executed, implement a microgrid multi-energy optimization method integrating an AI algorithm as described above.
[0041] The present application provides a microgrid multi-energy optimization method, device and medium integrating AI algorithm, which at least have the following beneficial effects:
[0042] By modeling the node correlation in the microgrid through the graph attention proximal strategy optimization algorithm, the complex correlation relationship between the nodes in the microgrid can be accurately captured to form a dynamic graph model, which not only reflects the current state of the microgrid, but also predicts future changes, providing a solid foundation for subsequent energy optimization; using graph attention technology to extract features in the dynamic graph model can efficiently obtain the graph structure characteristics of the microgrid, accurately reflect the topological structure and energy flow of the microgrid, and provide key information for building a microgrid strategy optimization model; by converting the operation process of the microgrid into a Markov decision process, the advantages of reinforcement learning can be used to achieve strategy optimization and iteration, ensuring the efficiency and stability of the microgrid in multi-energy scheduling; by inputting the extracted graph structure features into the microgrid strategy optimization model, the final optimization action, that is, the optimal strategy of the microgrid, can be output, which can significantly reduce the operation cost of the microgrid, improve energy utilization efficiency, and enhance the reliability of the microgrid. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0044] Figure 1 A schematic diagram of a flow chart of a microgrid multi-energy optimization method integrating an AI algorithm provided in an embodiment of the present application;
[0045] Figure 2 A schematic diagram of an optimization learning process of a graph attention proximal strategy optimization algorithm provided in an embodiment of the present application;
[0046] Figure 3 A schematic diagram of an improved microgrid IEEE 33-node system provided in an embodiment of the present application;
[0047] Figure 4 A schematic diagram of the internal structure of a microgrid multi-energy optimization device integrating an AI algorithm provided in an embodiment of the present application. DETAILED DESCRIPTION
[0048] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0049] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.
[0050] Figure 1 A flowchart of a microgrid multi-energy optimization method integrating an AI algorithm provided in an embodiment of the present application.
[0051] The analysis method involved in the embodiments of the present application can be implemented by a terminal device or a server, and the present application does not impose any special restrictions on this. For the convenience of understanding and description, the following embodiments are described in detail by taking a server as an example.
[0052] It should be noted that the server may be a single device or a system consisting of multiple devices, that is, a distributed server, and this application does not make any specific limitation on this.
[0053] like Figure 1 As shown, a microgrid multi-energy optimization method integrating an AI algorithm provided in an embodiment of the present application includes:
[0054] 101. Through the graph attention proximal strategy optimization algorithm, the node correlation in the microgrid is modeled and the corresponding dynamic graph model is obtained.
[0055] In order to better consider the impact of the interaction between different nodes in the microgrid on the operation of the multi-energy microgrid, a dynamic graph model structure is constructed for the nodes in the area and their interaction relationships, and the graph attention technology is used to extract features from the graph model data of the non-Euclidean structure. The extracted features can be further used by PPO to formulate the optimal operation strategy.
[0056] Since different types of neural networks have different features in extracting input data, it is necessary to consider data features and task types when selecting a neural network structure. The attention mechanism is introduced to utilize the graph structure features of the power grid. The correlation between each power device node and the adjacent device nodes is calculated based on the connection status of the power equipment, and the overall network structure information of the microgrid is further obtained.
[0057] Specifically, in one embodiment of the present application, the node correlation in the microgrid is modeled by the graph attention proximal strategy optimization algorithm to obtain the corresponding dynamic graph model, which specifically includes:
[0058] Combine the proximal strategy optimization algorithm with the graph attention technology to obtain the graph attention proximal strategy optimization algorithm;
[0059] Through the graph attention proximal strategy optimization algorithm, the nodes in the microgrid and the association relationships between different nodes are modeled, so as to project the grid state information of the microgrid into node embedding vectors and obtain the corresponding dynamic graph model.
[0060] In one embodiment, first, the proximal policy optimization algorithm is selected as the basic framework because it has good stability and convergence and is suitable for dealing with problems in continuous action spaces and complex environments. Then, the graph attention mechanism is integrated into the proximal policy optimization (PPO) algorithm to enhance the algorithm's ability to process graph structure data, and the graph attention proximal policy optimization (GT-PPO) algorithm is obtained.
[0061] In the graph attention proximal strategy optimization algorithm, a graph attention layer is defined, which can receive the node feature matrix and edge relationship matrix of the microgrid as input. The node feature matrix contains the initial features of each node, such as power, voltage, etc.; the edge relationship matrix describes the connection relationship between nodes, such as transmission lines, transformer connections, etc.
[0062] Through the graph attention layer, the algorithm can calculate the attention weights of each node to its neighboring nodes and aggregate the features of the neighboring nodes according to these weights to update the feature representation of each node. Multiple iterations are performed until a stable node embedding vector is obtained.
[0063] Next, these node embedding vectors are used as the input state of the PPO algorithm to train the Actor network and the Critic network. The Actor network is responsible for outputting the action probability distribution based on the current state, while the Critic network evaluates the value of a given state and action. By alternately optimizing these two networks, the algorithm is able to learn strategies that perform well in a microgrid environment.
[0064] Finally, the trained graph attention proximal strategy optimization algorithm is used to model the nodes in the microgrid and the relationship between different nodes. The algorithm projects the grid state information of the microgrid (such as node power, voltage, line flow, etc.) into node embedding vectors, thereby constructing a dynamic graph model that can reflect the real-time operation status of the microgrid.
[0065] In one embodiment, the correlation between microgrid nodes is related to their location in the grid. In a radial microgrid, the voltage of each node is affected by all other nodes, but the influence decreases with distance. Therefore, the agent needs to be aware of the relationship between nodes in the grid in order to make collaborative control decisions. In order to exploit the correlation between grid nodes, GT-PPO uses graph attention (GAT) to model the correlation of nodes within the microgrid, and the grid state information is projected into a node embedding vector and then input into the PPO network.
[0066] In one embodiment of the present application, it includes:
[0067] When modeling the node correlation in the microgrid, the attention coefficient corresponding to each neighborhood node in the dynamic graph model is calculated; wherein the attention coefficient is used to indicate the influence degree of the corresponding neighborhood node on the central node;
[0068] The feature vectors of the neighborhood nodes of the central node are aggregated through the attention mechanism to obtain the feature vector of the central node.
[0069] In one embodiment, first, a dynamic graph model is constructed based on the physical structure and operating status of the microgrid. The nodes in the model represent the various devices or energy points in the microgrid, and the edges represent the connection relationship between the devices or the energy flow path. Each node has an initial feature vector that contains basic information such as power, voltage, and current of the node.
[0070] Next, the attention mechanism is used to calculate the attention coefficient corresponding to the neighboring nodes of each central node, which is used to represent the degree of influence of the neighboring nodes on the central node. Specifically, an attention function is designed, which receives the feature vectors of the central node and the neighboring nodes as input and outputs an attention weight vector, in which each element corresponds to the attention coefficient of a neighboring node to the central node.
[0071] The calculation of the attention coefficient usually involves measuring the similarity or correlation between the feature vectors of the central node and the neighboring nodes. The dot product attention mechanism is used, that is, the dot product between the feature vector of the central node and the feature vector of the neighboring nodes is calculated, and normalized by the softmax function to obtain the final attention coefficient.
[0072] After obtaining the attention coefficients, these coefficients are used to aggregate the feature vectors of the neighborhood nodes of the central node. Specifically, the feature vector of each neighborhood node is multiplied by the corresponding attention coefficient, and then these weighted feature vectors are added together to obtain the aggregated feature vector of the central node. This aggregated feature vector not only contains the information of the central node itself, but also integrates the information of its neighborhood nodes, thereby more comprehensively reflecting the status and role of the central node in the microgrid.
[0073] Finally, the obtained dynamic graph model is applied to the operation analysis and optimization decision of the microgrid. By real-time monitoring and analyzing the status information of each node in the microgrid, the node feature vector and attention coefficient in the graph model can be dynamically updated, thereby realizing real-time tracking and prediction of the microgrid status. At the same time, the node characteristics and neighborhood relationships in the graph model can also be used to perform energy scheduling, fault diagnosis and optimization design of the microgrid.
[0074] In one embodiment of the present application, calculating the attention coefficients corresponding to each neighborhood node in the dynamic graph model specifically includes:
[0075] Calculate the attention coefficient through the following formula:
[0076]
[0077] where h i represents the central node feature vector, h j represents the neighborhood node feature vector, e ij represents the attention weight coefficient, that is, the influence degree of the neighborhood node h j on the central node h i , Relu() represents the activation function, W represents the trainable coefficient, ∥ represents the matrix concatenation symbol, a represents the parameter for dot product operation, exp() represents the exponential function, represents the summation expression, used to calculate the energy sum of the central node, N i represents the natural growth number.
[0078] In one embodiment, in feature calculation for each node, it is necessary to first count the attention coefficients of the neighborhood nodes that affect the central node, so as to make a weight division for each neighborhood node, so that the nodes with greater target effects obtain higher attention weights, while ignoring the nodes with smaller target effects. For the central node feature vector hi, use the attention mechanism to aggregate the feature vectors h1,..., hj of the adjacent nodes of the node.
[0079] 102. Through graph attention technology, perform feature extraction in the dynamic graph model to obtain graph structure features, and transform the operation process of the microgrid into a Markov decision process to construct a corresponding microgrid policy optimization model.
[0080] After transforming the multi - energy microgrid operation optimization strategy problem into a Markov decision process model, use the improved GT - PPO algorithm to solve the reinforcement learning problem.
[0081] Specifically, in one embodiment of the present application, through graph attention technology, perform feature extraction in the dynamic graph model to obtain graph structure features, and transform the operation process of the microgrid into a Markov decision process to construct a corresponding microgrid policy optimization model, which specifically includes:
[0082] Through graph attention technology, aggregate the local neighbor information of each node in the microgrid in the dynamic graph model to capture the correlation relationship between each node in the microgrid and obtain the corresponding graph structure features;
[0083] The graph structure features are taken as input, and the state space is constructed according to the real-time state of the microgrid, and the action space is constructed based on the energy scheduling strategy, so as to transform the operation process of the microgrid into a Markov decision process and obtain the corresponding microgrid strategy optimization model.
[0084] In one embodiment, first, a dynamic graph model is constructed based on the physical structure and operating characteristics of the microgrid. In this model, each node represents a key component in the microgrid (such as a generator, load, energy storage device, etc.), and the edge represents the connection relationship between the components (such as electrical connection, energy flow, etc.).
[0085] Next, the graph attention technique is used to aggregate the local neighbor information of each node. The graph attention technique can adaptively assign different weights to different neighbor nodes, thereby more accurately capturing the association between nodes. Specifically, an attention coefficient is calculated for each node, which reflects the relative importance between the node and its neighbors. Then, the information of the neighbor nodes is weighted and summed according to these attention coefficients to obtain the aggregated features of each node. Through the aggregation operation of the graph attention technique, the graph structure features of each node in the microgrid are obtained. These features not only contain the information of the node itself, but also incorporate the information of its neighbor nodes, thereby more comprehensively reflecting the operating status of the microgrid.
[0086] Next, the graph structure features are used as input to construct the state space according to the real-time state of the microgrid. The state space contains all possible state combinations, and each state corresponds to a specific operating condition of the microgrid. At the same time, the action space is also constructed based on the energy scheduling strategy. The action space contains all possible action combinations, and each action represents an energy scheduling solution.
[0087] With the state space and action space, the operation process of the microgrid is transformed into a Markov decision process (MDP). In this process, the microgrid will select an action to execute based on the current state at each time step and transfer to the next state. At the same time, corresponding rewards or penalties will be given according to the quality of the action.
[0088] Finally, a microgrid strategy optimization model is constructed using reinforcement learning algorithms (such as proximal policy optimization, deep deterministic policy gradient, etc.). This model can learn an optimal strategy based on the definition of the Markov decision process, so that the microgrid can obtain the maximum cumulative reward during long-term operation.
[0089] 103. Input the graph structure features into the microgrid strategy optimization model, output the final action to obtain the optimal strategy of the microgrid, and realize the multi-energy optimization of the microgrid based on the optimal strategy of the microgrid.
[0090] In one embodiment of the present application, before inputting the graph structure feature into the microgrid strategy optimization model and outputting the final action to obtain the optimal strategy of the microgrid, the method further includes:
[0091] According to the attention coefficient corresponding to each neighborhood node, the graph structure features are weighted summed;
[0092] The weighted sum is calculated by the following formula:
[0093]
[0094] Among them, h , i Represents the weighted sum of the central node feature vectors.
[0095] In one embodiment of the present application, the graph structure features are input into the microgrid strategy optimization model, and the final action is output to obtain the optimal strategy of the microgrid, which specifically includes:
[0096] The graph structure features are input into the microgrid strategy optimization model to extract the weighted summed input features through the graph convolutional network;
[0097] The extracted features are activated through the activation function, and the corresponding final actions are output through the fully connected layer to obtain the optimal strategy for the microgrid.
[0098] In one embodiment, first, the graph structure features of the microgrid are extracted from the dynamic graph model, including the state information of each node in the microgrid and the relationship between the nodes. These features are passed as input to the microgrid strategy optimization model. In the microgrid strategy optimization model, a graph convolutional network (GCN) is used to extract the input graph structure features. The graph convolutional network is a neural network specially used to process graph structure data. It can effectively extract the local features and global structure information of graph nodes through convolution operations.
[0099] The weighted sum of the input features is used as the input of the graph convolutional network. The weighted sum operation is to consider the importance of different nodes and edges, so that the graph convolutional network can extract key information more accurately. Then, the graph convolutional network extracts the deep feature representation of the graph nodes layer by layer through multi-layer convolution operations. After the deep feature representation is extracted, the features are nonlinearly transformed through an activation function (such as ReLU) to enhance the expressive power of the model. The choice of activation function has an important impact on the performance of the model. The ReLU function is widely used because of its simplicity and effectiveness.
[0100] After being processed by the activation function, the features are passed to the fully connected layer. The fully connected layer is responsible for mapping the extracted features to the action space and outputting the corresponding final action. The final action represents the scheduling strategy or energy allocation scheme of each device in the microgrid. Finally, the microgrid policy optimization model is trained using a reinforcement learning algorithm (such as the proximal policy optimization algorithm). During the training process, the model continuously adjusts the policy parameters according to the current grid state and action feedback (such as rewards or costs) to maximize the long-term cumulative rewards or minimize the long-term cumulative costs.
[0101] After multiple iterations of training, the model converges to a stable strategy, namely the optimal strategy for the microgrid, which can automatically adjust the operating status and energy allocation plan of each device according to the current grid status and energy demand to achieve efficient and stable operation of the microgrid.
[0102] In one embodiment of the present application, before outputting the final action to obtain the optimal strategy of the microgrid, the method further includes:
[0103] The environmental status in the microgrid is collected through the preset collection device, and the environmental status is input into the Actor network of the graph attention proximal strategy optimization algorithm;
[0104] Output the action of the microgrid through the Actor network, and generate the trajectory of the microgrid based on the action; the trajectory is used to represent the trajectory corresponding to the action, state and reward;
[0105] The trajectory is input into the Critic network of the graph attention proximal strategy optimization algorithm to output the state value estimation of the microgrid;
[0106] The state value estimation is input into the action advantage function to update the network parameters corresponding to the Actor network and the Critic network, thereby optimizing the graph attention proximal strategy optimization algorithm.
[0107] In one embodiment, first, preset collection devices, such as sensors and measuring instruments, are installed at key locations of the microgrid. These devices can monitor the environmental status within the microgrid in real time, including key parameters such as voltage, current, power, and temperature. Then, these environmental status data are input into the Actor network of the GA-PPO algorithm as the input state of the algorithm.
[0108] After receiving the input state, the Actor network outputs the action of the microgrid according to the current strategy. These actions may include adjusting the output power of the generator, switching the load, adjusting the charging and discharging state of the energy storage device, etc. Then, based on the output action and the current environmental state, the trajectory of the microgrid is generated. The trajectory is a sequence containing the corresponding relationship between actions, states, and rewards, which is used to describe the operating state and decision-making process of the microgrid over a period of time.
[0109] Next, the generated trajectory is input into the Critic network of the GA-PPO algorithm. The Critic network is responsible for estimating the state value of the microgrid, that is, the expected sum of rewards that can be obtained from the current state under the current strategy. Then, the state value estimate is input into the action advantage function, which is used to measure the pros and cons of the current action relative to other actions.
[0110] Based on the action advantage function, the loss function of the Actor network and the Critic network can be calculated, and the network parameters can be updated through the back propagation algorithm. This process will be iterated continuously until the network parameters converge to a stable state, that is, the algorithm has found an optimal strategy.
[0111] By continuously updating network parameters, the GA-PPO algorithm can gradually optimize the control strategy of the microgrid. The optimized strategy can automatically adjust the actions of the microgrid according to the real-time environmental status and reward feedback to achieve the goal of maximizing long-term cumulative rewards. In this process, the algorithm will automatically weigh the pros and cons of different actions and select the optimal combination of actions to ensure the efficient and stable operation of the microgrid.
[0112] like Figure 2 As shown in the figure, given the system operation mode in a certain period of time, the environment state is obtained through data collection, and the agent takes action according to the environment state. When the next period comes, the environment generates state according to the action and data such as grid load and new energy generation, and subsequent interactions are carried out in this process. At the same time, the agent itself includes the Actor network and the Critic network. The Actor network and the Critic network share the lower layer, which consists of GNN and linear layers. The Actor network outputs actions, generates trajectories of actions, states, and rewards based on the actions, and inputs the Critic network to estimate the advantage function, update the network parameters of the two networks, and finally solve the optimal strategy.
[0113] like Figure 3 As shown in Figure 1, the improved microgrid IEEE 33 and IEEE 118 node systems are used for implementation. The IEEE 33 node system topology is shown in Figure 1. Figure 2 As shown, the energy storage devices are connected to nodes 8, 14, 24, and 30 respectively, and the wind turbines and photovoltaic generators are connected to nodes 5, 10, 16, 20, 26 and 32, 35, and 36 respectively. Node 1 is assumed to be a balancing node, and the rest are PQ nodes. Each grid has a set of scenarios, and each scenario specifies the changes in the simulation, such as power supply and demand at each time step.
[0114] The GT-PPO algorithm, PPO algorithm, and DDQN algorithm are used to solve the microgrid optimization operation scheme. In terms of algorithm structure setting, the network structure of the PPO algorithm and the GT-PPO algorithm is basically the same, with a hidden layer of 3 fully connected layers with 128 neurons; the output layer sets 4 neurons as the number of action outputs. The number of update iterations during training is set to 10 times, and the specific parameter settings during the training process are shown in Table 1.
[0115] Table 1 Training parameter settings
[0116] parameter Parameter Value Number of training sets N 400 Strategy trajectory cache size D 3000 Update times I 10 Mini-batch data capacity B 200 Reward discount factor γ 0.99
[0117] The microgrid multi-energy comprehensive optimization dispatching method integrating AI algorithm can reduce the voltage fluctuation and operation loss of the daily operation of the microgrid multi-energy microgrid by learning and obtaining the optimal operation strategy of the microgrid. The GT-PPO algorithm can effectively solve the problem of optimal operation of multi-energy microgrids. At the algorithm level, the GT-PPO algorithm inherits the advantages of the PPO algorithm, has better adaptability, and is less sensitive to hyperparameters. At the same time, it combines the graph self-attention network and makes full use of the interconnected structure of the microgrid, which is better than the previous DDQN algorithm. It can effectively solve the problem of optimal operation of multi-energy microgrids, and reasonably optimize the operation strategy of microgrids, which is of great significance to the stability of multi-energy microgrids.
[0118] The above is an embodiment of the method proposed in this application. Based on the same inventive concept, this application embodiment also provides a microgrid multi-energy optimization device integrating an AI algorithm, whose structure is as follows: Figure 4 shown.
[0119] Figure 4 A schematic diagram of the internal structure of a microgrid multi-energy optimization device with an integrated AI algorithm provided in an embodiment of the present application. Figure 4 As shown, the device includes:
[0120] at least one processor;
[0121] and, a memory communicatively coupled to the at least one processor;
[0122] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor to enable the at least one processor to:
[0123] Through the graph attention proximal strategy optimization algorithm, the node correlation in the microgrid is modeled to obtain the corresponding dynamic graph model;
[0124] Through graph attention technology, feature extraction is performed in the dynamic graph model to obtain graph structure features, and the operation process of the microgrid is converted into a Markov decision process to build a corresponding microgrid strategy optimization model;
[0125] The graph structure features are input into the microgrid strategy optimization model, and the final action is output to obtain the optimal strategy of the microgrid. Based on the optimal strategy of the microgrid, multi-energy optimization of the microgrid is achieved.
[0126] The present application also provides a non-volatile computer storage medium storing computer executable instructions. When the computer executable instructions are executed, they can:
[0127] Through the graph attention proximal strategy optimization algorithm, the node correlation in the microgrid is modeled to obtain the corresponding dynamic graph model;
[0128] Through graph attention technology, feature extraction is performed in the dynamic graph model to obtain graph structure features, and the operation process of the microgrid is converted into a Markov decision process to build a corresponding microgrid strategy optimization model;
[0129] The graph structure features are input into the microgrid strategy optimization model, and the final action is output to obtain the optimal strategy of the microgrid. Based on the optimal strategy of the microgrid, multi-energy optimization of the microgrid is achieved.
[0130] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.
[0131] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0132] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.
[0133] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0134] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0135] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0136] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0137] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0138] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0139] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0140] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0141] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A microgrid multi-energy optimization method integrating AI algorithm, characterized in that: The method comprises: Through the graph attention proximal strategy optimization algorithm, the node correlation in the microgrid is modeled to obtain the corresponding dynamic graph model; By using the graph attention technology, feature extraction is performed in the dynamic graph model to obtain graph structure features, and the operation process of the microgrid is converted into a Markov decision process to construct a corresponding microgrid strategy optimization model; The graph structure features are input into the microgrid strategy optimization model, and the final action is output to obtain the optimal strategy of the microgrid, and the multi-energy optimization of the microgrid is realized according to the optimal strategy of the microgrid.
2. According to claim 1, a microgrid multi-energy optimization method integrating AI algorithm is characterized in that: Through the graph attention proximal strategy optimization algorithm, the node correlation in the microgrid is modeled to obtain the corresponding dynamic graph model, which includes: Combine the proximal strategy optimization algorithm with the graph attention technology to obtain the graph attention proximal strategy optimization algorithm; Through the graph attention proximal strategy optimization algorithm, the nodes in the microgrid and the association relationships between different nodes are modeled to project the grid state information of the microgrid into node embedding vectors to obtain the corresponding dynamic graph model.
3. According to the microgrid multi-energy optimization method integrating AI algorithm according to claim 1, it is characterized in that: The method further comprises: When modeling the node correlation in the microgrid, the attention coefficient corresponding to each neighborhood node in the dynamic graph model is calculated; wherein the attention coefficient is used to indicate the influence degree of the corresponding neighborhood node on the central node; The feature vectors of the neighborhood nodes of the central node are aggregated through an attention mechanism to obtain the feature vector of the central node.
4. According to claim 3, a microgrid multi-energy optimization method integrating AI algorithm is characterized in that: Calculate the attention coefficient corresponding to each neighborhood node in the dynamic graph model, including: The attention coefficient is calculated by the following formula: Among them, h i represents the central node feature vector, h j represents the neighborhood node feature vector, e ij Represents the attention weight coefficient, that is, the neighborhood node h j For the central node h i Relu() represents the activation function, W represents the trainable coefficient, ∥ represents the matrix concatenation operator, a represents the parameter for the dot product operation, exp() represents the exponential function, Represents the summation expression, which is used to calculate the energy sum of the central node, N i Indicates the natural growth number.
5. According to claim 1, a microgrid multi-energy optimization method integrating AI algorithm is characterized in that: Before inputting the graph structure feature into the microgrid strategy optimization model and outputting the final action to obtain the optimal strategy of the microgrid, the method further includes: According to the attention coefficient corresponding to each neighborhood node, weighted summation is performed on the graph structure features; The weighted sum is calculated by the following formula: Among them, h , i Represents the weighted sum of the central node feature vectors.
6. According to the microgrid multi-energy optimization method integrating AI algorithm according to claim 1, it is characterized in that: The graph structure features are input into the microgrid strategy optimization model, and the final action is output to obtain the optimal strategy of the microgrid, which specifically includes: Inputting the graph structure features into the microgrid strategy optimization model to extract the weighted summed input features through a graph convolutional network; The extracted features are activated through the activation function, and the corresponding final actions are output through the fully connected layer to obtain the optimal strategy for the microgrid.
7. The microgrid multi-energy optimization method integrating AI algorithm according to claim 1 is characterized in that: Before outputting the final action to obtain the optimal strategy of the microgrid, the method further includes: The environmental state in the microgrid is collected by a preset collection device, and the environmental state is input into the Actor network of the graph attention proximal strategy optimization algorithm; Outputting the action of the microgrid through the Actor network, and generating a trajectory of the microgrid according to the action; wherein the trajectory is used to represent the trajectory corresponding to the action, state and reward; Inputting the trajectory into the Critic network of the graph attention proximal strategy optimization algorithm to output a state value estimate of the microgrid; The state value estimation is input into the action advantage function to update the network parameters corresponding to the Actor network and the Critic network, thereby optimizing the graph attention proximal strategy optimization algorithm.
8. The microgrid multi-energy optimization method integrating AI algorithm according to claim 1 is characterized in that: Through the graph attention technology, feature extraction is performed in the dynamic graph model to obtain graph structure features, and the operation process of the microgrid is converted into a Markov decision process to construct a corresponding microgrid strategy optimization model, which specifically includes: By using graph attention technology, the local neighbor information of each node in the microgrid is aggregated in the dynamic graph model to capture the association relationship between the nodes in the microgrid and obtain the corresponding graph structure features; The graph structure features are used as input, and a state space is constructed according to the real-time state of the microgrid, and an action space is constructed based on the energy scheduling strategy, so as to transform the operation process of the microgrid into a Markov decision process and obtain the corresponding microgrid strategy optimization model.
9. A microgrid multi-energy optimization device integrating AI algorithm, characterized in that: The device comprises: at least one processor; and, a memory communicatively coupled to the at least one processor; In which, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a microgrid multi-energy optimization method integrating an AI algorithm as described in any one of claims 1-8.
10. A non-volatile computer storage medium storing computer executable instructions, characterized in that: When the computer executable instructions are executed, a microgrid multi-energy optimization method integrating an AI algorithm as described in any one of claims 1 to 8 is implemented.