Distribution network source network load storage collaborative optimization method, device, equipment and medium

Abstract

The invention discloses a distribution network source network load storage collaborative optimization method and device, equipment and a medium. The method comprises the following specific steps: constructing a node data sample set containing power data of a distribution network under different topological structures; representing the power distribution network system as graph structure data according to the node data sample set under each topological structure; constructing a graph neural network model, and extracting topology context information of each node in the graph structure data; constructing a source network load storage collaborative optimization element reinforcement learning model; training a source network load storage collaborative optimization element reinforcement learning model; and quickly adapting to a new task by using the trained meta-reinforcement learning model. According to the method, the source network load storage can be quickly responded and adapted under the scene of frequent topology change, the meta parameters are optimized through meta reinforcement learning, the requirement of the model for training from zero on a new task is reduced, and the training efficiency and the generalization performance of the model are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of distribution network source-grid-load-storage, and in particular to a distribution network source-grid-load-storage collaborative optimization method, device, equipment and medium. Background Art

[0002] With the continuous access of distributed power sources (such as wind power, photovoltaics), energy storage equipment and dynamic loads, the complexity and dynamic changes of modern distribution networks have increased significantly. These changes have brought about frequent topology adjustments, such as planned maintenance, fault recovery and flexible scheduling. This dynamicity poses severe challenges to the reliability, economy and adaptability of power grid operation. However, traditional optimization methods are slow to respond to complex topological changes and often cannot quickly generate effective strategies. In addition, the synergy between distributed resources has not been fully explored, resulting in low overall operating efficiency. Traditional methods also lack generalization capabilities and need to be trained from scratch for new topological tasks, which is time-consuming and inefficient. Artificial intelligence technology, especially reinforcement learning (RL), provides new ideas for power grid optimization. Reinforcement learning can effectively learn optimization strategies through the interaction between intelligent agents and the environment without relying on explicit system models. However, standard reinforcement learning has significant limitations, such as a large demand for training samples and difficulty in quickly adapting to topological changes in scenarios with high real-time requirements. Especially in the scenario of rapidly changing distribution networks, standard reinforcement learning requires a lot of time to retrain and cannot meet the real-time needs of power grid operation. Meta-reinforcement learning (MRL) is an enhanced reinforcement learning method that aims to extract general strategies or knowledge through training in a multi-task environment so that the model can quickly adapt to new tasks. In practical applications, meta-reinforcement learning uses the commonalities between tasks to quickly adjust strategies to adapt to new environmental changes. It is particularly suitable for dynamic scenarios and fields with a wide distribution of tasks. Combined with other technologies (such as neural networks), meta-reinforcement learning can further improve the understanding and adaptability of complex environments, and is an important tool for meeting real-time and high-efficiency requirements. Although meta-reinforcement learning can significantly improve the adaptability and generalization ability of strategies, its application in complex power grid optimization still faces some challenges. For example, how to effectively integrate the topological characteristics of the distribution network and the operating status of distributed resources, and how to quickly generate efficient collaborative optimization strategies, these problems need to be further addressed. Summary of the invention

[0003] The present invention provides a distribution network source-grid-load-storage coordinated optimization method, the specific steps are as follows:

[0004] Construct a node data sample set containing power data of distribution networks under different topological structures;

[0005] The distribution network system is represented as graph structure data according to the node data sample set under each topological structure;

[0006] Build a graph neural network model to extract the topological context information of each node in the graph structure data, and then obtain the node features;

[0007] A meta-reinforcement learning model for the coordinated optimization of power generation, grid, load and storage is constructed, and the node features extracted by the graph neural network model are used as the initial state input of the meta-reinforcement learning model.

[0008] The source-grid-load-storage collaborative optimization meta-reinforcement learning model is trained through inner and outer loop operations;

[0009] Use trained meta-reinforcement learning models to quickly adapt to new tasks.

[0010] Preferably, the method for constructing a node data sample set containing power data of a distribution network under different topological structures is as follows:

[0011] Collect power data of multiple time sections under each topological structure of the distribution network. The power data includes the connection relationship between the distribution network nodes, the operating status of the load unit, the power generation unit and the energy storage device;

[0012] Preprocess the power data to remove missing values ​​and outliers;

[0013] According to the preprocessed power data, node data sample sets under different topological structures are constructed, and the node data sample sets are divided into training set, verification set and test set in the ratio of 8:1:1.

[0014] Preferably, the method for representing the distribution network system as graph structure data according to the node data sample set under each topological structure is as follows:

[0015] The real physical nodes in the distribution network are used as graph nodes, and the power connections between physical nodes are used as edges to obtain the topological graph of the distribution network. The graph structure data of the n-node distribution network system can be expressed as G(A,V);

[0016] Among them, A is an nxn matrix, which is the connection matrix of graph structure data;

[0017] V={v0,v1,…,v i ,…v N-1} is an nxf matrix, which is the characteristic matrix of graph structure data;

[0018] f is the number of features per node, v i Represents the electrical characteristics of the ith physical node; if v i and v j If there is an edge connection, let the element a in the i-th row and j-th column beij =1; otherwise, let the element a in the i-th row and j-th column ij = 0, thus obtaining the N×N order adjacency matrix A of G = {a ij |i∈[0,N-1], j∈[0,N-1]};

[0019] The electrical quantity of the physical node is used as the graph node feature, and each node feature is expressed as:

[0020]

[0021] in are the load active and reactive power, are the active power of MT units, The amount of electricity stored in the energy storage system, t is the time; if the node does not contain an element, the eigenvalue is set to 0.

[0022] Preferably, the method for constructing a graph neural network model is as follows:

[0023] The graph neural network model includes a preprocessing layer, L graph convolution layers and a post-processing layer;

[0024] The preprocessing layer is used to perform preliminary processing on the input node features and map the original input features to a higher-dimensional feature space to facilitate processing by the subsequent graph convolutional network layer. The method is to transform the input node features through the fully connected layer and then generate a new node feature representation through the activation function. The calculation method is as follows:

[0025] X=σ(W·V+b)

[0026] Among them, V represents the node features of the original input, W is the learnable weight matrix, b is the bias term, σ is the activation function, and X is the new node feature representation;

[0027] The graph convolution layer is used to aggregate the information of each node and its neighboring nodes. For each node i, its features are updated to graph structure data through the graph convolution layer for feature extraction and expressed as:

[0028]

[0029] Where D is the degree matrix of A, Z (l) The node feature matrix of the lth layer, W (l) is the weight of the lth layer, σ is the activation function; the node features are extracted layer by layer through the stacking of multiple layers of graph convolutional layers, while capturing the global topological information of the system;

[0030] The post-processing layer first maps the node feature Y output by the graph convolution layer into topological context information through a fully connected layer. The calculation method is as follows:

[0031] Z=σ(W·Y+b)

[0032] Among them, Z represents the topological context information mapped by the fully connected layer, W is the learnable weight matrix, b is the bias term, σ is the activation function, and Y is the node feature output by the graph convolution layer.

[0033] Preferably, the method for constructing a source-grid-load-storage collaborative optimization meta-reinforcement learning model is as follows:

[0034] Define the reinforcement learning elements for the coordinated optimization of power generation, grid, load and storage, including state space, action space and reward function;

[0035] State space S: The system state s∈S describes the current physical state of the system, including load, unit output, new energy output, and energy storage state, which can be expressed as:

[0036]

[0037] in The active and reactive power of the load of the i-th node at time t, is the active power of the i-th MT unit at time t, is the maximum active power of the i-th RES unit at the t-th moment, The power stored for the i-th node at time t;

[0038] Action space A: Action a includes the output change of MT unit New energy output and energy storage output

[0039] Taking into account the operating constraints of each unit, the action space is processed as follows:

[0040]

[0041] Where: i∈N MT ,j∈N RES ,k∈N ESS ; are the upper and lower limits of the climbing rate of the i-th MT; are the upper and lower limits of charging / discharging power of the i-th ESS respectively; and are the MT, RES and ESS decision action quantities given by GRL respectively;

[0042] Reward function R: for the current state s t and action a t The immediate reward r represents the reward value received by the agent immediately after executing the current action, which is described as the negative value of the running cost:

[0043] r(t)=-(F MT (t)+F RES (t)+F ESS (t)+F trans (t)).

[0044] Among them, F MT (t) is the electricity generation cost in MT, F RES (t) represents the cost of abandoning wind and solar power, F ESS (t) represents the charging and discharging cost of ESS, F Trans (t) represents the cost of purchasing electricity from the main grid through the tie line;

[0045] A meta-reinforcement learning model is constructed. The node features containing topological context information are extracted through the graph neural network model and used as the initial state input of the meta-reinforcement learning model. The parameters of the policy network π are divided into two parts: context parameters φ and shared parameters θ. The policy gradient method is used to maximize the expected reward. The objective function is:

[0046]

[0047] in: represents the expected value of the agent's cumulative discounted reward within a finite number of iterations H under strategy π, γ∈(0,1) is the discount factor, γ t is the tth power of the discount factor, r(s t ,a t ,s t+1 ) means taking action a t After that, the system is in state s t Transfer to s t+1 Rewards;

[0048] Preferably, the method for training the source-grid-load-storage collaborative optimization meta-reinforcement learning model is as follows:

[0049] It includes inner loop and outer loop, which are performed alternately. The inner loop updates the context parameters of the policy network for the collaborative optimization task of each topological scenario to ensure that the model can adapt to the tasks of a specific topological scenario. The outer loop is used to optimize shared parameters to ensure that the model can quickly adapt to different topological scenarios.

[0050] The inner loop performs the following operations for each topology scenario task:

[0051] Task T in each topology scenario i At the beginning, the context parameter φ is randomly initialized to φ0 and the current policy network is used Execute in the task's environment, collecting a trajectory:

[0052]

[0053] Among them, s j ,a j ,r j They represent the state, action and reward of the system at the i-th iteration of the agent, respectively, and H represents the number of iterations of the agent;

[0054]

[0055] Among them, α is the learning rate, Is the use of strategy The resulting trajectory To update the objective function of the policy parameters, is the gradient of the objective function with respect to the parameter φ;

[0056] Use the updated policy network Execute in the environment of the task and collect a trajectory Used to evaluate the quality of the task update and adjust π or the update rule accordingly;

[0057] The outer loop optimizes shared parameters so that the model can achieve better generalization in different topological scenarios. The specific process is as follows:

[0058] Sampling multiple tasks: Sample a batch of tasks from the task distribution p(T) And execute the inner loop to update the context parameters of each task;

[0059] Meta-loss calculation: In the outer loop, the context parameters φ and shared parameters θ updated in the inner loop are used to calculate the average reward or loss of the model over all topological scenarios to form the meta-loss;

[0060]

[0061] in, Is to use the strategy in the i-th topology scenario The resulting trajectory To update the objective function of the shared parameter θ;

[0062] Shared parameter update: The gradient of the meta-loss with respect to the shared parameters is calculated, and the shared parameters are optimized using the gradient update method. The updated shared parameters are more generalizable in the task distribution and can be quickly adapted to new tasks.

[0063]

[0064] Where β is the learning rate, The gradient of the objective function with respect to the shared parameters θ.

[0065] Preferably, the method of using the trained meta-reinforcement learning model to quickly adapt to new tasks is as follows:

[0066] New task initialization: Load the trained shared parameters θ to provide a good initial strategy for the new topology task.

[0067] Perform inner layer learning: Use GNN to extract new topological features and make a small amount of gradient updates to the context parameters of the policy network to obtain an optimized policy adapted to the new task.

[0068] Strategy execution and evaluation: Use the updated strategy to make optimization decisions and evaluate the adaptability and generalization of the model under the new topology to ensure efficient collaborative optimization in new scenarios.

[0069] A distribution network source-grid-load-storage collaborative optimization device, comprising:

[0070] A node data sample set building module is used to collect node data of power data in distribution networks under different topological structures;

[0071] The data processing module is used to represent the node data under each topological structure as graph structure data, and also includes a graph neural network model and a source-grid-load-storage collaborative optimization meta-reinforcement learning model;

[0072] The graph neural network model is used to extract the topological context information of each node in the structural data, and the extracted node features are used as the initial state input of the meta-reinforcement learning model;

[0073] The meta-reinforcement learning model for source-grid-load-storage collaborative optimization is used to enhance the adaptability of distribution network source-grid-load-storage collaborative optimization devices under new tasks.

[0074] An electronic device comprises: at least one processor; and a memory connected to the at least one processor in communication; wherein:

[0075] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a distribution network source-grid-load-storage collaborative optimization method.

[0076] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements a distribution network source-grid-load-storage collaborative optimization method.

[0077] Through the above steps, the present invention realizes a distribution network source-grid-load-storage collaborative optimization method based on meta-reinforcement learning, which can quickly respond and adapt to scenarios with frequent topology changes, and has the following advantages:

[0078] Fast adaptation: Ability to quickly adapt to new topology changes through small updates to meet real-time requirements.

[0079] Efficient training: Optimize meta-parameters through meta-reinforcement learning, reduce the need to train the model from scratch on new tasks, and improve training efficiency.

[0080] Strong generalization ability: Models trained in a multi-task environment can also show good adaptability to out-of-distribution tasks, improving the generalization performance of the model.

[0081] Adapt to topology change scenarios: Designed specifically for topology changes in distribution networks, it adapts to the operating requirements of distribution systems in planned maintenance or failure situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 The present invention provides a method flow for collaborative optimization of distribution network source-grid-load-storage based on meta-reinforcement learning;

[0083] Figure 2 The distribution network source-grid-load-storage collaborative optimization meta-reinforcement learning model training and rapid adaptation process of the present invention;

[0084] Figure 3 Schematic diagram of IEEE33 node distribution network in an embodiment of the present invention;

[0085] Figure 4 It is a schematic diagram of the structure of the electronic device of the present invention;

[0086] Figure 5 It is a schematic diagram of the computer-readable storage medium structure of the present invention. DETAILED DESCRIPTION

[0087] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art may think of other obvious variations.

[0088] See also Figures 1 to 5 The present invention provides a technical solution: a distribution network source-grid-load-storage collaborative optimization method, which is performed in the following steps:

[0089] Step 1: Construct a node data sample set, which includes power data of the distribution network under different topological structures;

[0090] Step 1.1: The schematic diagram of the IEEE 33 node distribution network connected to the distributed power source in this embodiment is as follows: Figure 3 As shown in the figure, power data of multiple time sections are collected under each topological structure of the distribution network. The power data includes the following: the connection relationship between the nodes of the distribution network, the operating status of the load unit, the power generation unit and the energy storage device (such as power, charging status, load demand, etc.)

[0091] Step 1.2: Preprocess the data. First, clean the data to remove missing values ​​and outliers. Since the range of node feature values ​​varies greatly (such as different ranges of load demand and energy storage status), the node feature matrix X is normalized to facilitate model training. The normalization formula is:

[0092]

[0093] Among them, x min and x max are the minimum and maximum eigenvalues, respectively.

[0094] Step 1.3: Construct node data sample sets under different topological structures. Divide the node data sample sets into training set, validation set and test set in a ratio of 8:1:1.

[0095] Step 2: Represent the distribution network system as graph structure data;

[0096] Step 2.1: Take the real physical nodes in the distribution network as graph nodes and the power connections between physical nodes as edges to obtain the topological graph of the distribution network. The graph structure data of the n-node distribution network system can be expressed as G(A,V), where A is an nxn matrix, which is the connection matrix of the graph structure data: V = {v0,v1,…,v i ,…v N-1} is an nxf matrix, which is the feature matrix of the graph structure data, and f is the number of features of each node v i Represents the electrical characteristics of the ith physical node; if v i and v j If there is an edge connection, let the element a in the i-th row and j-th column be ij =1; otherwise, let the element a in the i-th row and j-th column ij = 0, thus obtaining the N×N order adjacency matrix A of G = {a ij |i∈[0,N-1], j∈[0,N-1]};

[0097] Step 2.2: Take the electrical quantity of the physical node as the graph node feature, and each node feature is represented as:

[0098]

[0099] in are the load active and reactive power, are the active powers of MT units and RES units respectively, The amount of electricity stored in the energy storage system, t is the time; if the node does not contain an element, the eigenvalue is set to 0.

[0100] Step 3: Build a graph neural network model to extract distribution network topology context information;

[0101] Step 3.1: Construct a graph neural network model for extracting distribution network topology context information, including a preprocessing layer, L graph convolution layers and a post-processing layer;

[0102] The main purpose of the preprocessing layer is to perform preliminary processing on the input node features and map the original input features to a higher-dimensional feature space to facilitate processing by the subsequent graph convolutional network layer. Specifically, the input node features are transformed by the fully connected layer, and then a new node feature representation is generated through the activation function. The calculation method is as follows:

[0103] X=σ(W·V+b)

[0104] Among them, V represents the node features of the original input, W is the learnable weight matrix, b is the bias term, σ is the activation function, and X is the new node feature representation;

[0105] Step 3.2: In order to extract the features of graph structured data more efficiently, a graph convolutional neural network (GCN) is introduced. GCN can aggregate the information of each node and its neighboring nodes, so as to reflect the local environment of the node when the feature is updated. After multiple layers of graph convolution operations, the features of each node not only contain its own state, but also incorporate the information of neighboring nodes. This method has significant advantages in improving the optimality and efficiency of scheduling optimization, and at the same time provides feasible technical support for coping with larger-scale distribution network systems. For each node i, its features are updated to graph structured data through the graph convolution layer for feature extraction and expressed as:

[0106]

[0107] Where D is the degree matrix of A, Z (l) The node feature matrix of the lth layer, W (l) is the weight of the lth layer, and σ is the activation function. By stacking multiple layers of GCN, node features are extracted layer by layer, while capturing the global topological information of the system.

[0108] Step 3.3: The post-processing layer first maps the node feature Y output by the graph convolution layer into topological context information through a fully connected layer, and the calculation method is as follows:

[0109] Z=σ(W·Y+b)

[0110] Among them, Z represents the topological context information mapped by the fully connected layer, W is the learnable weight matrix, b is the bias term, σ is the activation function, and Y is the node feature output by the graph convolution layer.

[0111] Step 4: Construct a meta-reinforcement learning model for coordinated optimization of power generation, grid, load and storage

[0112] Step 4.1: Define the reinforcement learning elements for source-grid-load-storage collaborative optimization, including state space, action space, and reward function;

[0113] State space S: The system state s∈S describes the current physical state of the system, including load, unit output, new energy output, and energy storage state, which can be expressed as

[0114]

[0115] in The active and reactive power of the load of the i-th node at time t, is the active power of the i-th MT unit at time t, is the maximum active power of the i-th RES unit at the t-th moment, The power stored for the i-th node at time t;

[0116] Action space A: Action a includes the output change of MT unit New energy output and energy storage output Taking into account the operating constraints of each unit, the action space is processed as follows:

[0117]

[0118] Where: i∈N MT ,j∈N RES ,k∈N ESS ; are the upper and lower limits of the climbing rate of the i-th MT; are the upper and lower limits of charging / discharging power of the i-th ESS respectively; and are the MT, RES and ESS decision action quantities given by GRL respectively;

[0119] Reward function R: is the current state s t and action a t The immediate reward r represents the reward value received by the agent immediately after executing the current action, which is described as the negative value of the running cost:

[0120] r(t)=-(F MT (t)+F RES (t)+F ESS (t)+F trans (t)).

[0121] Among them, F MT (t) is the electricity generation cost in MT, F RES (t) represents the cost of abandoning wind and solar power, F ESS(t) represents the charging and discharging cost of ESS, F Trans (t) represents the cost of purchasing electricity from the main grid through the tie line;

[0122] Step 4.2: Build a meta-reinforcement learning model, extract node features containing topological context information through the graph neural network model, and use it as the initial state input of the meta-reinforcement learning model. Divide the parameters of the policy network π into two parts: context parameters φ and shared parameters θ. Use the policy gradient method to maximize the expected reward. The objective function is:

[0123]

[0124] in: represents the expected value of the agent's cumulative discounted reward within a finite number of iterations H under strategy π, γ∈(0,1) is the discount factor, γ t is the tth power of the discount factor, r(s t ,a t ,s t+1 ) means taking action a t After that, the system is in state s t Transfer to s t+1 Rewards;

[0125] Step 5: Train the meta-reinforcement learning model for source-grid-load-storage collaborative optimization;

[0126] Step 5.1 Figure 2 As shown in the figure, during the whole training process, the inner loop and the outer loop are performed alternately: the inner loop updates the context parameters of the policy network for the collaborative optimization task of each topological scenario to ensure that the model can adapt to the tasks of a specific topological scenario; the outer loop optimizes the shared parameters to ensure that the model can quickly adapt to different topological scenarios;

[0127] The inner loop of step 5.2 performs the following operations for each topology scenario task:

[0128] Task T in each topology scenario i At the beginning, the context parameter φ is randomly initialized to φ0 and the current policy network is used Execute in the task's environment, collecting a trajectory:

[0129]

[0130] Among them, s j ,a j ,r j Represent the state, action and reward of the system at the i-th iteration of the agent, respectively, and H represents the number of iterations of the agent; according to the collected trajectory data, the policy gradient method is used to update the context parameter φ of the policy network in the i-th topological scenario i :

[0131]

[0132] Among them, α is the learning rate, Is the use of strategy The resulting trajectory To update the objective function of the policy parameters, is the gradient of the objective function with respect to the parameter φ;

[0133] Use the updated policy network Execute in the environment of the task and collect a trajectory Used to evaluate the quality of the task update and adjust π or the update rule accordingly;

[0134] Step 5.3: The outer loop optimizes shared parameters so that the model can achieve better generalization in different topological scenarios. The specific process is as follows:

[0135] Sampling multiple tasks: Sample a batch of tasks from the task distribution p(T) And execute the inner loop to update the context parameters of each task;

[0136] Meta-loss calculation: In the outer loop, the context parameters φ and shared parameters θ updated in the inner loop are used to calculate the average reward or loss of the model over all topological scenarios to form the meta-loss;

[0137]

[0138] in, Is to use the strategy in the i-th topology scenario The resulting trajectory To update the objective function of the shared parameter θ;

[0139] Shared parameter update: The gradient of the meta-loss with respect to the shared parameters is calculated, and the shared parameters are optimized using the gradient update method. The updated shared parameters are more generalizable in the task distribution and can be quickly adapted to new tasks.

[0140]

[0141] Where β is the learning rate, The gradient of the objective function with respect to the shared parameters θ;

[0142] Step 6: Meta-RL model quickly adapts to new tasks

[0143] New task initialization: Load the trained shared parameters θ to provide a good initial strategy for the new topology task.

[0144] Perform inner layer learning: Use GNN to extract new topological features and make a small amount of gradient updates to the context parameters of the policy network to obtain an optimized policy adapted to the new task.

[0145] Strategy execution and evaluation: Use the updated strategy to make optimization decisions and evaluate the adaptability and generalization of the model under the new topology to ensure efficient collaborative optimization in new scenarios.

[0146] Furthermore, the method according to the embodiment of the present application can also be performed by Figure 4 The electronic device architecture shown in FIG. Figure 4 As shown, the electronic device 500 may include a bus 501, one or more CPUs 502, a read-only memory (ROM) 503, a random access memory (RAM) 504, a communication port 505 connected to a network, an input / output component 506, a hard disk 507, etc. The storage device in the electronic device 500, such as ROM 503 or hard disk 507, may store a distribution network source-grid-load-storage collaborative optimization method provided in the present application. The electronic device 500 may also include a user interface 508. Of course, Figure 4 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Figure 4 One or more components of an electronic device are shown.

[0147] Figure 5 Schematic diagram of a computer-readable storage medium structure provided by an embodiment of the present application. Figure 5 As shown, a computer-readable storage medium 600 according to an embodiment of the present application is shown. Computer-readable instructions are stored on the computer-readable storage medium 600. When the computer-readable instructions are executed by the processor, a distribution network source-grid-load-storage collaborative optimization method according to an embodiment of the present application described with reference to the above figures can be executed. The storage medium 600 includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0148] In summary, the advantages of the present invention are: it enables the source, grid, load and storage to respond and adapt quickly in scenarios with frequent topology changes, optimizes meta-parameters through meta-reinforcement learning, reduces the need to train the model from scratch on new tasks, and improves training efficiency and the generalization performance of the model.

[0149] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A distribution network source-grid-load-storage collaborative optimization method, characterized in that: The specific steps are as follows: Construct a node data sample set containing power data of distribution networks under different topological structures; The distribution network system is represented as graph structure data according to the node data sample set under each topological structure; Build a graph neural network model to extract the topological context information of each node in the graph structure data, and then obtain the node features; A meta-reinforcement learning model for the coordinated optimization of power generation, grid, load and storage is constructed, and the node features extracted by the graph neural network model are used as the initial state input of the meta-reinforcement learning model. The source-grid-load-storage collaborative optimization meta-reinforcement learning model is trained through inner and outer loop operations; Use trained meta-reinforcement learning models to quickly adapt to new tasks.

2. A distribution network source-grid-load-storage collaborative optimization method according to claim 1, characterized in that: The method for constructing a node data sample set containing power data of the distribution network under different topological structures is as follows: Collect power data of multiple time sections under each topological structure of the distribution network. The power data includes the connection relationship between the distribution network nodes, the operating status of the load unit, the power generation unit and the energy storage device; Preprocess the power data to remove missing values ​​and outliers; According to the preprocessed power data, node data sample sets under different topological structures are constructed, and the node data sample sets are divided into training set, verification set and test set in the ratio of 8:1:

1.

3. A distribution network source-grid-load-storage collaborative optimization method according to claim 1, characterized in that: The method for representing the distribution network system as graph structure data according to the node data sample set under each topological structure is as follows: The real physical nodes in the distribution network are used as graph nodes, and the power connections between physical nodes are used as edges to obtain the topological graph of the distribution network. The graph structure data of the n-node distribution network system is expressed as G(A,V); Among them, A is an nxn matrix, which is the connection matrix of graph structure data; V={v0,v1,…,v i ,…v N-1 } is an nxf matrix, which is the characteristic matrix of graph structure data; f is the number of features per node, v i represents the electrical characteristics of the ith physical node; if v i and v j If there is an edge connection, let the element a in the i-th row and j-th column be ij =1; otherwise, let the element a in the i-th row and j-th column ij = 0, thus obtaining the N×N order adjacency matrix A of G = {a ij |i∈[0,N-1], j∈[0,N-1]}; The electrical quantity of the physical node is used as the graph node feature, and each node feature is expressed as: in are the load active and reactive power, respectively. are the active power of MT units, The amount of electricity stored in the energy storage system, t is the time; if the node does not contain an element, the eigenvalue is set to 0.

4. A distribution network source-grid-load-storage collaborative optimization method according to claim 1, characterized in that: The method for constructing the graph neural network model is as follows: The graph neural network model includes a preprocessing layer, L graph convolution layers and a post-processing layer; The preprocessing layer is used to perform preliminary processing on the input node features and map the original input features to a higher-dimensional feature space. The method is to transform the input node features through the fully connected layer and then generate a new node feature representation through the activation function. The calculation method is as follows: X=σ(W·V+b) Among them, V represents the node features of the original input, W is the learnable weight matrix, b is the bias term, σ is the activation function, and X is the new node feature representation; The graph convolution layer is used to aggregate the information of each node and its neighboring nodes. For each node i, its features are updated to graph structure data through the graph convolution layer, and feature extraction is expressed as: Where D is the degree matrix of A, Z (l) The node feature matrix of the lth layer, W (l) is the weight of the lth layer, σ is the activation function; the node features are extracted layer by layer through the stacking of multiple layers of graph convolutional layers, while capturing the global topological information of the system; The post-processing layer maps the node feature Y output by the graph convolution layer into topological context information through the fully connected layer. The calculation method is as follows: Z=σ(W·Y+b) Among them, Z represents the topological context information mapped by the fully connected layer, W is the learnable weight matrix, b is the bias term, σ is the activation function, and Y is the node feature output by the graph convolution layer.

5. A distribution network source-grid-load-storage collaborative optimization method according to claim 1, characterized in that: The method for constructing the source-grid-load-storage collaborative optimization meta-reinforcement learning model is as follows: Define the reinforcement learning elements for the coordinated optimization of power generation, grid, load and storage, including state space, action space and reward function; State space S: The system state s∈S describes the current physical state of the system, including load, unit output, new energy output, and energy storage state, expressed as: in The active and reactive power of the load of the i-th node at time t, is the active power of the i-th MT unit at time t, is the maximum active power of the i-th RES unit at the t-th moment, The power stored for the i-th node at time t; Action space A: Action a includes the output change of MT unit New energy output and energy storage output Taking into account the operating constraints of each unit, the action space is processed as follows: Where: i∈N MT ,j∈N RES ,k∈N ESS ; are the upper and lower limits of the climbing rate of the i-th MT; are the upper and lower limits of charging / discharging power of the i-th ESS respectively; are the MT, RES and ESS decision action quantities given by GRL respectively; Reward function R: for the current state s t and action a t The immediate reward r represents the reward value received by the agent immediately after executing the current action, which is described as the negative value of the running cost: r(t)=-(F MT (t)+F RES (t)+F ESS (t)+F trans (t)). Among them, F MT (t) is the electricity generation cost in MT, F RES (t) represents the penalty for abandoning wind and solar power, F ESS (t) represents the charging and discharging cost of ESS, F Trans (t) represents the cost of purchasing electricity from the main grid through the tie line; A meta-reinforcement learning model is constructed. The node features containing topological context information are extracted through the graph neural network model and used as the initial state input of the meta-reinforcement learning model. The parameters of the policy network are divided into two parts: context parameters φ and shared parameters θ. The policy gradient method is used to maximize the expected reward. The objective function is: in: represents the expected value of the agent's cumulative discounted reward within a finite number of iterations H under strategy π, γ∈(0,1) is the discount factor, γ t is the tth power of the discount factor, r(s t ,a t ,s t+1 ) means taking action a t After that, the system is in state s t Transfer to s t+1 Reward.

6. A distribution network source-grid-load-storage coordinated optimization method according to claim 5, characterized in that: The method for training the source-grid-load-storage collaborative optimization meta-reinforcement learning model through inner and outer loop operations is as follows: The inner loop and the outer loop are performed alternately; the inner loop updates the strategy network for the collaborative optimization task of each topological scenario to ensure that the model can adapt to the tasks of a specific topological scenario, and the outer loop is used to optimize the shared parameters to ensure that the model can quickly adapt to different topological scenarios; The inner loop performs the following operations for each topology scenario task: Task T in each topology scenario i At the beginning, the context parameter φ is randomly initialized to φ0 and the current policy network is used Execute in the task's environment, collecting a trajectory: Among them, s j ,a j ,r j They represent the state, action and reward of the system at the i-th iteration of the agent, respectively, and H represents the number of iterations of the agent; According to the collected trajectory data, the policy gradient method is used to update the context parameter φ of the policy network in the i-th topological scenario i : Among them, α is the learning rate, Is the use of strategy The resulting trajectory To update the objective function of the policy parameters, is the gradient of the objective function with respect to the parameter φ; Use the updated policy network Execute in the environment of the task and collect a trajectory Used to evaluate the quality of the task update and adjust π or the update rule accordingly; The outer loop optimizes shared parameters so that the model can achieve better generalization in different topological scenarios. The specific process is as follows: Sampling multiple tasks: Sample a batch of tasks containing N topological scenarios from the task distribution p(T) And execute the inner loop to update the context parameter φ of each topology scenario task i ; Meta-loss calculation: In the outer loop, the context parameters φ and shared parameters θ updated in the inner loop are used to calculate the average reward or loss of the model over all topological scenarios to form the meta-loss; in, Is to use the strategy in the i-th topology scenario The resulting trajectory To update the objective function of the shared parameter θ; Shared parameter update: The gradient of the meta-loss with respect to the shared parameters is calculated, and the shared parameters are optimized using the gradient update method. The updated shared parameters are more generalizable in the task distribution and can be quickly adapted to new tasks. Where β is the learning rate, The gradient of the objective function with respect to the shared parameters θ.

7. A distribution network source-grid-load-storage coordinated optimization method according to claim 6, characterized in that: The method of using the trained meta-reinforcement learning model to quickly adapt to new tasks is as follows: New task initialization: load the trained shared parameters θ to provide a good initial strategy for the new topology task; Perform inner layer learning: use GNN to extract new topological features and make a small gradient update to the context parameters of the policy network to obtain an optimized strategy adapted to the new task; Strategy execution and evaluation: Use the updated strategy to make optimization decisions and evaluate the adaptability and generalization of the model under the new topology to ensure efficient collaborative optimization in new scenarios.

8. A distribution network source-grid-load-storage collaborative optimization device, characterized in that: include: A node data sample set building module is used to collect node data of power data in distribution networks under different topological structures; The data processing module is used to represent the node data under each topological structure as graph structure data, and also includes a graph neural network model and a source-grid-load-storage collaborative optimization meta-reinforcement learning model; The graph neural network model is used to extract the topological context information of each node in the structural data, and the extracted node features are used as the initial state input of the meta-reinforcement learning model; The meta-reinforcement learning model for source-grid-load-storage collaborative optimization is used to enhance the adaptability of distribution network source-grid-load-storage collaborative optimization devices under new tasks.

9. An electronic device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a distribution network source-grid-load-storage collaborative optimization method as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, a distribution network source-grid-load-storage collaborative optimization method according to any one of claims 1 to 7 is implemented.

Images