A Grid Line Loss Optimization Method and System Based on Graph Attention Perception and Reinforcement Learning Decision-Making

Abstract

The present invention provides a power grid line loss optimization method and system based on graph attention network and reinforcement learning decision-making. First, real-time measurement and historical operation data of the power grid are collected, and the data is preprocessed, labeled and partitioned. Based on the global graph attention network and the deep autoencoder, an intelligent perception model of the new energy high-penetration power grid is constructed. Then, the intelligent perception model is trained using historical data to realize the intelligent online perception of the power grid line loss under the condition of high proportion of new energy access. Secondly, based on the global graph attention network model and the proximal policy optimization algorithm, an intelligent decision-making model of the new energy high-penetration power grid is constructed. Finally, the power grid line loss optimization problem of the new energy high-penetration power grid is transformed into a Markov decision-making process, and the intelligent decision-making model for loss reduction is trained to obtain the adaptive online loss reduction strategy of the power grid. The present invention can make optimal decisions based on the current power grid state, maximize the reduction of line losses, and realize the adaptive online loss reduction strategy.

Description

Technical Field

[0001] The present invention relates to the technical field of power grid line loss optimization control, and particularly to a power grid line loss optimization method and system based on graph attention perception and reinforcement learning decision-making. Background Art

[0002] With the continuous increase in the penetration rate of renewable energy sources (such as photovoltaic power generation and battery energy storage) in the distribution network, the problem of line loss in power grid operation has become increasingly prominent. The intermittency and volatility of renewable energy sources make the power flow in the power grid complex, which further exacerbates the line loss. How to minimize the line loss while ensuring the safe and stable operation of the power grid has become an important challenge in power system optimization.

[0003] Currently, many methods have been proposed to optimize power grid operation to reduce line loss. The stochastic programming (SP) method is a commonly used approach to deal with the uncertainty of renewable energy generation and load demand. However, the SP method relies on the probability distribution of random variables, which is often difficult to obtain in practical applications. Another commonly used method is robust optimization (RO), which addresses uncertainty by finding a solution that is robust to all cases within an uncertainty set. However, robust optimization usually solves the worst-case optimization problem, resulting in solutions that are often too conservative to flexibly handle the rapid fluctuations of photovoltaic power generation. In addition, both SP and RO methods rely on accurate physical models, which often show significant limitations in actual distribution networks due to the existence of system model errors.

[0004] In contrast, machine learning methods shift the burden of online calculation to the offline training process, so they can make decisions within milliseconds based on the latest observed data. Among many machine learning methods, deep reinforcement learning (DRL) has been widely applied to the optimization control of power systems. However, many DRL-based methods still assume the existence of an accurate physical model to generate training data. To reduce the dependence on accurate physical models, researchers have proposed some optimization methods based on model-free physics. These methods use historical operation data to train DRL agents, avoiding the dependence on an accurate model of the distribution network.

[0005] In recent years, graph neural networks (GNNs), as a deep learning method capable of learning node relationships from graph-structured data, have gradually attracted attention. Graph reinforcement learning methods based on GNNs have been applied to address power grid topology changes, and graph learning methods for distribution network restoration have been proposed.

[0006] However, existing methods still rely on the accurate line parameters of the power grid during the training process and generally assume that global observation information of the entire network can be obtained. In practical applications, due to problems such as data acquisition errors, communication failures, or measurement noise, only partial observation information of the power grid can be obtained, which poses a huge challenge to the line loss optimization method based on DRL. Summary of the Invention

[0007] Objective of the Invention: To solve the problems arising from the above-mentioned existing technologies, the present invention provides a power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making, which solves the problems of excessive line losses in a new energy high-penetration power grid, inaccurate perception of the uncertainty of new energy power, and insufficient real-time performance of optimization strategy calculation. The present invention also provides a power grid line loss optimization system based on graph attention perception and reinforcement learning decision-making.

[0008] Technical Solution: According to the first aspect of the present invention, there is provided a power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making, the method comprising the following steps:

[0009] S1 Collect real-time measurement and historical operation data of the power grid, and perform preprocessing and annotation division on the data; the real-time measurement and historical operation data include the voltage of nodes, active load power, reactive load power, line current, active and reactive power of photovoltaic power, energy storage charge and discharge records, and line loss conditions, and the annotation division includes converting the preprocessed data into graph data containing node features, edge features, and topological structures;

[0010] S2 Construct an intelligent perception model, the intelligent perception model includes a global graph attention network, a one-dimensional convolutional neural network, and a fully connected network, transmit the input data to the global graph attention network to obtain the hidden state of each node, and then sequentially process with the one-dimensional convolutional neural network and the fully connected network, and finally output high-dimensional node features;

[0011] S3 Compress the high-dimensional node features into a low-dimensional representation by using a deep autoencoder;

[0012] S4 Use the training set to train the intelligent perception model and the deep autoencoder to obtain a trained intelligent perception model;

[0013] S5 Construct an intelligent decision-making model, the intelligent decision-making model includes two neural networks: an Actor network and a Critic network, the Actor network receives the data output by the trained intelligent perception model, and then obtains the action distribution of each node; the Critic network receives the data output by the trained intelligent perception model, and then the Critic network outputs a value function through a fully connected layer to evaluate the value of the current state and action;

[0014] S6 transforms the line loss optimization problem of the new - energy high - penetration power grid into a Markov decision process, thereby training the intelligent decision - making model according to the obtained action distribution of each node and the value of the current state and action, and then obtaining the adaptive online loss - reduction strategy of the new - energy high - penetration power grid.

[0015] Further, it includes:

[0016] In the step S1, the graph data is represented as:

[0017] G=(V, E, X), where V is the set of nodes, representing each node in the power grid, E is the set of edges, representing the connection relationship between nodes, and X is the attribute matrix containing node features and edge features, including the voltage of the node, active power of the load, reactive power of the load, line current, active power and reactive power of the photovoltaic, charge - discharge record of the energy storage, and the loss situation of the line.

[0018] Further, it includes:

[0019] In the step S2, constructing the intelligent perception model specifically includes:

[0020] The formula for calculating the hidden state of each node through the global graph attention network is as follows:

[0021]

[0022] where h att,i is the feature representation of node i, α ij is the attention score measuring the importance of node j to node i, N(i) represents the set of adjacent nodes of node i, g j (x j ) is the embedding function of node j composed of a single - layer neural network, x j is the input feature of node j, σ is the activation function, and the attention score α ij is calculated by the following formula:

[0023]

[0024] In the formula, L1 represents the number of layers of the global graph attention network, g i (x i ) is the embedding function of node i composed of a single - layer neural network, represents the concatenation of two node embeddings, w is the weight vector, LeakyReLU is the activation function, e ij represents the attention weight. In order to extract the common features of the whole system, after the graph attention network, a one - dimensional convolutional neural network is used, which is represented as:

[0025]

[0026] Wherein, W k is the weight of the k-th one-dimensional convolutional neural network layer, and b k represents the bias of the k-th neural network layer. F represents the number of layers of the one-dimensional convolutional neural network, σ is the activation function, and h cov,k is the output of the feature quantity passing through the convolutional layer;

[0027] Then, the feature h cov,k is further embedded using a fully connected network, and the final output layer is the representation h g of the high-level node features.

[0028] Furthermore, it includes:

[0029] The high-dimensional node features are compressed into a low-dimensional representation by using a deep autoencoder, and the encoding process is as follows:

[0030] h e = f m […f1(h g )],(5)

[0031] Wherein, f m represents the fully connected network of the m-th layer in the deep autoencoder. Through the non-linear mapping of the multi-layer fully connected network, the original high-dimensional feature h g is converted into a low-dimensional representation h e , that is, compressed from a high-dimensional space to a low-dimensional space.

[0032] Furthermore, it includes:

[0033] The specific steps of step S4 include:

[0034] S41 Randomly initialize the parameters θ ggat and θ dae of the global graph attention network and the deep autoencoder;

[0035] S42 Input the data after annotation division in step S1 into the global graph attention network;

[0036] S43 Use historical data and labels to perform supervised training on the global graph attention network, calculate the actual line loss of each line and the line loss predicted by the model, and use the loss function Loss(θ ggat ) to perform gradient descent. During the optimization process, the model parameters will be gradually adjusted to enable accurate prediction of the line loss;

[0037] S44 Use unsupervised learning to train the deep autoencoder, and minimize the reconstruction loss function Loss(θ dae) to optimize the parameters of the encoder. Finally, by optimizing this loss function, a low-dimensional representation is extracted from the high-dimensional features; the loop iterates through steps S42 - S44 until the corresponding loss function gradually converges, thereby predicting the line loss and adapting to the grid operation conditions with a high proportion of new energy access.

[0038] Furthermore, it includes:

[0039] In step S43, the loss function Loss(θ ggat ) is expressed as:

[0040]

[0041] where B is the sampling batch, is the true line loss of the q-th line, is the line loss predicted by the model for the q-th line;

[0042] Use the gradient descent method to update the parameters θ of the perception model ggat , and optimize the model by minimizing the loss function:

[0043]

[0044] where η is the learning rate.

[0045] Furthermore, it includes:

[0046] The loss function Loss(θ dae ) is specifically expressed as:

[0047]

[0048] where L2 represents the number of layers of the deep autoencoder, h g is the high-dimensional feature generated by the global graph attention network, is the feature reconstructed by the deep auto-decoder.

[0049] Furthermore, it includes:

[0050] In step S5, the objective function of the intelligent decision-making model is expressed as:

[0051]

[0052] where θ are the parameters of the policy network, r t (θ) is the ratio of the new and old policies, that is, it represents the change amplitude of the current policy relative to the old policy, A t is the advantage function, which is used to measure the quality of a certain action relative to the average performance. ∈ is a hyperparameter that controls the policy update amplitude, which prevents excessive updates. clip() is a truncation function, Indicates expectation.

[0053] Furthermore, it includes:

[0054] In step S6, the line loss optimization problem of the new - energy high - penetration power grid is transformed into a Markov decision process, which is described as follows:

[0055] DRL agent: Regarding the dispatching decision - making center as an agent to control PV and BSS in the power grid to achieve intelligent decision - making for power grid loss reduction;

[0056] Environment: The environment for agent interaction is the intelligent perception model in step S2. When the agent makes a decision, the intelligent perception model feeds back an optimized line loss value and voltage value according to the decision variables, and then calculates the reward value based on the line loss and voltage values;

[0057] State space: When using PV inverters and BSS to reduce line loss, the state at each time step is defined as s t =(m t , A), where m t represents the observed characteristics of the power grid, including: node net active power and reactive power, voltage amplitude and current amplitude, and A represents the adjacency matrix;

[0058] Action space: The action of the agent at each time step is a t =(a pv,t , a bss,t ), where a pv,t represents the reactive power output of the PV inverter at time t, and a bss,t represents the active power output of the BSS at time t;

[0059] Reward function: The reward function is defined as minimizing the power grid line loss. The line loss at time t is represented as p loss,t , and the reward r t =-p loss,t +η, where η is the voltage violation penalty value.

[0060] Furthermore, it includes:

[0061] In step S6, the training process of the intelligent decision - making model includes:

[0062] S61 Model parameter loading: Loading the parameters of the intelligent perception model trained in step S4, including the parameters θ ggat and θ dae , and randomly initializing the parameters of the Actor network and the Critic network;

[0063] S62 initializes grid state observation: For each iteration, the initial state s0 of the grid is extracted from historical data or real-time data, representing the current grid node status and line information. Each iteration consists of multiple time steps, and the agent interacts with the perception model at each time step to generate an experience trajectory.

[0064] S63 agent performs actions and calculates state transitions and reward values: The agent extracts the feature state s at time step t. t Generate action a through the Actor network t , the state of the power grid changes as the action is executed, and it transfers to the next state s t+1 ,Based on the executed actions and state changes, the perception model calculates the line loss changes and voltage conditions brought about by the current action, and generates the corresponding reward value. The reward value represents the impact of the current action on the line loss. The goal is to maximize the cumulative reward by minimizing the line loss;

[0065] S64 stores the experience track to the experience replay pool: the current state s t 、Action a t , reward r t and the new state s t+1 Stored in the experience replay pool for subsequent use in training models. If the replay pool is full, delete the earliest experience data to keep the capacity of the experience pool fixed.

[0066] S65 samples training data from the experience replay pool: When enough samples have accumulated in the experience replay pool, a small batch of data is randomly sampled from it, containing multiple state-action-reward-new state combinations.

[0067] S66 updates the Critic network and Actor network;

[0068] After multiple rounds of iterations in S67, the Actor network and Critic network gradually converged, and the intelligent agent was able to make optimal decisions based on the current grid status, maximize the reduction of line losses, and implement an adaptive online loss reduction strategy.

[0069] Further, including:

[0070] In step S66, updating the critic network specifically includes:

[0071] The critic network uses data sampled from the experience pool to update its parameters. The goal of the critic network is to minimize the estimation error of the state-action pair. The update rule is as follows:

[0072]

[0073] Where, Losscritic is the loss function of the Critic network, which measures the gap between the actual state value and the estimated value, θ critic and η critic are the parameters and learning rate of the Critic network.

[0074] Furthermore, it includes:

[0075] In the step S66, updating the Actor network specifically includes:

[0076] The Actor network uses the advantage function A calculated from the Critic network t , and the update rule is as follows:

[0077]

[0078] where θ actor and η actor are the parameters and learning rate of the Actor network.

[0079] On the second aspect, the present invention also provides a power grid line loss optimization system based on graph attention perception and reinforcement learning decision-making, and the system includes:

[0080] A data preprocessing module, which is used to collect real-time measurement and historical operation data of the power grid, and preprocess and label and divide the data; the real-time measurement and historical operation data include the voltage of the node, the active power of the load, the reactive power of the load, the line current, the active power and reactive power of the photovoltaic, the charge and discharge record of the energy storage, and the loss situation of the line, and the label division includes converting the preprocessed data into graph data including node features, edge features and topological structures;

[0081] An intelligent perception model construction module, which is used to construct an intelligent perception model, and the intelligent perception model includes a global graph attention network, a one-dimensional convolutional neural network and a fully connected network, and transmits the input data to the global graph attention network to obtain the hidden state of each node, and then processes it successively by the one-dimensional convolutional neural network and the fully connected network, and finally outputs high-dimensional node features;

[0082] A feature dimensionality reduction module, which is used to compress the high-dimensional node features into a low-dimensional representation by using a deep autoencoder;

[0083] An intelligent perception model training module, which is used to train the intelligent perception model and the deep autoencoder by using a training set to obtain a trained intelligent perception model;

[0084] The intelligent decision-making model construction module is used to construct an intelligent decision-making model. The intelligent decision-making model includes two neural networks: the Actor network and the Critic network. The Actor network receives the data output by the trained intelligent perception model, and then obtains the action distribution of each node. The Critic network receives the data output by the trained intelligent perception model, and then the Critic network outputs a value function through a fully connected layer to evaluate the value of the current state and action.

[0085] The intelligent decision-making model training module is used to transform the line loss optimization problem of the new energy high-penetration power grid into a Markov decision process, so as to train the intelligent decision-making model according to the obtained action distribution of each node and the value of the current state and action, and then obtain the adaptive online loss reduction strategy of the new energy high-penetration power grid.

[0086] In a third aspect, the present invention also provides a computer-readable storage medium, on which computer instructions are stored. When the computer instructions run, they execute the above-mentioned power grid line loss optimization based on graph attention perception and reinforcement learning decision-making.

[0087] In a fourth aspect, the present invention also provides a risk assessment device based on task indicators, characterized in that the risk assessment device includes: a memory, a processor, and a power grid line loss optimization program based on graph attention perception and reinforcement learning decision-making stored on the memory and executable on the processor. When the power grid line loss optimization program based on graph attention perception and reinforcement learning decision-making is executed by the processor, it realizes the steps of the above-mentioned power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making.

[0088] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0089] (1) Efficient and accurate intelligent perception of power grid line loss: The present invention uses a line loss intelligent perception model that combines the global graph attention network GGAT and the deep autoencoder DAE to accurately perceive the line loss of each line in the power grid. This model utilizes the topological structure and real-time operation data of the power grid to be able to predict the line loss in real time and accurately, enhancing the transparency of the power grid operation and providing key basic data support for subsequent optimization decisions.

[0090] (2) Adaptive line loss optimization strategy: By introducing PPO and combining with an intelligent perception model, the present invention constructs an intelligent decision-making model for a new energy high-penetration power grid. Through continuous interaction with the power grid environment, this model learns how to adaptively adjust the operation strategies of distributed energy sources such as photovoltaic power generation and energy storage systems in the power grid according to different operating states, thereby minimizing line losses to the greatest extent. Compared with traditional physical model-based optimization methods, this model does not rely on precise physical parameters and has higher flexibility and adaptability.

[0091] (3) Enhancement of power grid stability under new energy access: Aiming at the power grid operation fluctuations caused by high proportion of new energy access, the present invention can achieve automatic regulation and optimization of the power grid through a reinforcement learning intelligent decision-making model, enhancing the stability of the power grid while reducing line losses. This model can handle the power fluctuations brought by new energy access and ensure the efficient and stable operation of the power grid in a complex dynamic environment by adaptively adjusting the output strategies of distributed energy sources. Description of the Drawings

[0092] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0093] Figure 1 Flowchart of the power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making according to the embodiments of the present invention;

[0094] Figure 2 Framework diagram of the power grid intelligent perception model based on GGAT and DAE according to the embodiments of the present invention;

[0095] Figure 3 Flowchart of the GGAT-PPO method according to the embodiments of the present invention;

[0096] Figure 4 Schematic structural diagram of the power grid line loss optimization system based on graph attention perception and reinforcement learning decision-making according to the embodiments of the present invention;

[0097] Figure 5 Schematic diagram of the terminal hardware structure of each embodiment of the power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making according to the embodiments of the present invention. Detailed Embodiments

[0098] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0099] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0100] Embodiment 1

[0101] Since the idea of the reinforcement learning decision-making scheme is that the agent and the environment continuously interact and learn the best strategy from historical experience, it is possible to capture the uncertainty of the new type of source-load power from the data and solve it online in real time. However, current reinforcement learning methods all assume that the agent interacts with a simulated environment built on software, which is divorced from the perception of the real power grid environment, and it is impossible to train the interactive agent in the actual power grid environment.

[0102] Therefore, the present invention innovatively proposes a line loss intelligent perception model. The line loss perception model for the interaction of the reinforcement learning agent is trained through actual historical data, and then a new reinforcement learning algorithm is proposed to train the loss reduction intelligent decision-making model, thus realizing the perception of the uncertainty of the real power grid power and power flow and the intelligent loss reduction decision-making.

[0103] Specifically, this embodiment discloses a power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making. The method includes the following steps:

[0104] S1 Collect real-time measurement and historical operation data of the power grid, and perform preprocessing and annotation division on the data; the real-time measurement and historical operation data include the voltage of nodes, active power of loads, reactive power of loads, line current, active power and reactive power of photovoltaic, charge and discharge records of energy storage, and line loss conditions. The annotation division includes converting the preprocessed data into graph data including node features, edge features, and topological structures;

[0105] S2 Construct an intelligent perception model. The intelligent perception model includes a global graph attention network, a one-dimensional convolutional neural network, and a fully connected network. Transmit the input data into the global graph attention network to obtain the hidden state of each node, and then process it successively by the one-dimensional convolutional neural network and the fully connected network, and finally output high-dimensional node features;

[0106] S3 Use a deep autoencoder to compress the high-dimensional node features into a low-dimensional representation;

[0107] S4 uses the training set to train the intelligent perception model and the deep autoencoder to obtain the trained intelligent perception model;

[0108] S5 constructs an intelligent decision-making model, which includes two neural networks: an Actor network and a Critic network. The Actor network receives the data output by the trained intelligent perception model to obtain the action distribution of each node; the Critic network receives the data output by the trained intelligent perception model, and then the Critic network outputs a value function through a fully connected layer to evaluate the value of the current state and action;

[0109] S6 transforms the line loss optimization problem of the new energy high-penetration power grid into a Markov decision process, and then trains the intelligent decision-making model according to the obtained action distribution of each node and the value of the current state and action, so as to obtain the adaptive online loss reduction strategy of the new energy high-penetration power grid.

[0110] In a specific implementation manner in this embodiment, it includes:

[0111] Step 1: Collect the real-time measurement and historical operation data of the power grid, and perform preprocessing and annotation partitioning on the data;

[0112] This application collects the voltage of the node, the active power and reactive power of the load, the line current, the active power and reactive power of the photovoltaic, the charge and discharge records of the energy storage, the line loss situation and the power grid topology structure data; after collecting a large amount of data, due to possible sensor failures, data transmission errors or other reasons during the data collection process resulting in data anomalies, first clean and fill in the missing data, and standardize the data.

[0113] Then, annotate the data, convert the input data into graph data containing node features, edge features and topology structure. The power grid is represented as a graph G=(V, E, X), where V is the node set, representing each node in the power grid, E is the edge set, representing the connection relationship between nodes, and X is the attribute matrix containing node features and edge features, including information such as the voltage, power of the node, and the current and loss of the line. This graph structure not only reflects the physical topology relationship of the power grid, but also contains real-time electrical parameters, which helps the model capture the global information in the power grid.

[0114] Finally, label the line loss data with the label of the supervised learning model, and divide the data set into a training set and a test set, with a division ratio of 80% for the training set and 20% for the test set. The training set is used for the training of the perception model, and the test set is used for evaluating the final performance of the model.

[0115] Step 2: Construct an intelligent perception model for line losses in a new - energy - highly - penetrated power grid based on an improved deep neural network;

[0116] The framework of the improved deep neural network is as Figure 2 shown. Among them, the global graph attention network GGAT is the core part of the model. It captures the complex relationships between various nodes through the graph attention mechanism, including the mutual influence between electrical states such as voltage and power and the topological structure. In GGAT, the feature vector of each node not only depends on its own state information but also combines the information of neighboring nodes for update. Through the attention mechanism, the model can adaptively learn the influence degree of neighboring nodes on the target node.

[0117] The formula for deriving the hidden state of each node through the global graph attention network GGAT is as follows:

[0118]

[0119] In the formula, h att,i is the feature representation of node i, α ij is the attention score measuring the importance of node j to node i, N(i) represents the set of neighboring nodes of node i, g j (x j ) is the embedding function of node j composed of a single - layer neural network, x j is the input feature of node j, σ is the activation function, and the attention score α ij can be calculated by the following formula:

[0120]

[0121] In the formula, represents the concatenation of the embeddings of two nodes, w is the weight vector, LeakyReLU is the activation function, and e ij represents the attention weight. In order to extract the common features of the entire system, after the graph attention network, a one - dimensional convolutional neural network CNN is used:

[0122]

[0123] In the formula, W i is the weight of the i - th one - dimensional convolutional neural network layer, b i represents the bias of the i - th neural network layer, F represents the number of layers of the one - dimensional convolutional neural network, σ is the activation function, and h cov,i is the output of the feature quantity after passing through the convolutional layer. Then, a fully - connected network FCN is used to further embed the features, and the final output layer is the representation h g of the high - level node features.

[0124] After the GGAT module processes the real-time and pseudo-measurement data, the model generates a high-dimensional feature representation h g , in order to reduce the computational burden while maintaining important feature information, dimensionality reduction processing is required, so the deep autoencoder DAE is proposed. DAE consists of two parts: an encoder and a decoder. First, the encoder compresses the high-dimensional feature h g into a low-dimensional representation h e , and the encoding process is as follows:

[0125] h e = f l […f1(h g )],(4)

[0126] In the formula, f l represents the l-th layer FCN in the encoder. Through the non-linear mapping of multiple layers of FCN, the original feature h g is converted into a low-dimensional representation h e , that is, compressed from a high-dimensional space to a low-dimensional space.

[0127] The main challenges faced by GGAT in high-dimensional data processing in the prior art are: redundancy and noise of high-dimensional feature data and computational complexity.

[0128] In the combination of GGAT, DAE and CNN of the present invention, a new processing method is proposed to deal with the complex spatio-temporal data characteristics in the new energy high-penetration power grid. The specific improvements and innovations are as follows:

[0129] Hierarchical feature processing: First, GGAT captures the power grid topology relationship and the mutual influence between nodes, then CNN extracts the global system pattern, and finally DAE reduces the dimension, compresses and filters noise. This hierarchical processing can optimize the expression and processing of features in stages, ensuring that the output of each layer of the network makes a key contribution to the final optimization goal (such as minimizing line losses). This scheme is more efficient and targeted than the application of a single model.

[0130] Systematic modeling combining local and global features: GGAT is used to model local topology relationships, CNN is used to identify global features of the system, and DAE further compresses and enhances the robustness of information. At the same time, for the DAE part, we only retain the encoding part of DAE and remove the decoding part, such a design simplifies the model computational complexity.

[0131] Multi-layer network collaborative optimization: This innovative combination not only uses advanced neural network models in each layer of the network, but also ensures collaborative optimization between different network layers. The high-dimensional features output by the GGAT layer are processed by the CNN and DAE, reducing information redundancy and computational complexity, and ensuring the efficiency and reliability of the final low-dimensional features input into the decision network.

[0132] Step 3: Consider a training method that combines supervised and unsupervised learning, and train the line loss intelligent perception model in Step 2 with historical data to achieve intelligent online perception of the line loss of the power grid under high-proportion new energy access.

[0133] The training process of the model is as follows:

[0134] Step 3.1: Initialize the parameters of the intelligent perception model. Before training the intelligent perception model, randomly initialize the parameters θ ggat and θ dae of GGAT and DAE, and use the Xavier initialization method to avoid the problem of gradient disappearance or explosion.

[0135] Step 3.2: Represent the power grid node features, edge features, and topological structure collected and preprocessed in Step 1 in the form of a graph, and input them into the global graph attention network.

[0136] Step 3.3: Conduct supervised training on the GGAT network based on CNN using historical data and labels, calculate the actual line loss of each line and the line loss predicted by the model, and perform gradient descent using the loss function Loss(θ ggat ).

[0137] In Step 3.3, the loss function Loss(θ ggat ) of the perception model is calculated as follows:

[0138]

[0139] where B is the sampling batch, is the actual line loss of the i-th line, is the line loss predicted by the model.

[0140] Update the parameters θ ggat of the perception model using the gradient descent method, and optimize the model by minimizing the loss function:

[0141]

[0142] where η is the learning rate. This optimization process will gradually adjust the model parameters to enable accurate prediction of line losses.

[0143] Step 3.4: Train the DAE parameters θ dae;

[0144] After the initial training of GGAT, unsupervised learning is used to train the DAE to reduce the dimension of high-dimensional features while ensuring the retention of information. The DAE optimizes the parameters of the encoder by minimizing the reconstruction error, and the reconstruction loss function Loss(θ dae ) is calculated as follows:

[0145]

[0146] where h g is the high-dimensional feature generated by GGAT, is the feature reconstructed by the DAE decoder. Finally, by optimizing this loss function, the model can extract a more robust low-dimensional representation from the high-dimensional features.

[0147] Step 3.5: Optimization and evaluation of the model convergence effect;

[0148] After multiple rounds of training, the intelligent perception model gradually converges, can accurately predict the line loss, and adapt to the operation of the power grid with a high proportion of new energy access. The trained perception model is verified using the test set to evaluate its ability to perceive the line loss of the power grid online, and further optimize the parameters and structure of the model to ensure its effectiveness and robustness in practical applications.

[0149] In this application, supervised learning is used for the line loss prediction task in Step 3.3, and the model is trained through the known labels in the historical data, that is, the actual line loss value. This method can directly optimize the prediction ability of the model, enabling it to accurately predict the line loss of each line, thereby helping the power grid operator make better decisions in daily operation.

[0150] Unsupervised learning is used to train the deep autoencoder DAE in Step 3.4, aiming to reduce the redundant information of the input data through dimensionality reduction and extract representative low-dimensional features. In this case, there is no need to rely on labeled data, and the focus can be on improving the feature representation ability of the model to ensure that the features can still be effectively compressed and retained in the case of a large dataset and complex information.

[0151] This joint training method improves the generalization ability and stability of the model; reduces the dependence on labeled data through feature compression and denoising; reduces the complexity of the model and the risk of overfitting.

[0152] Step 4: Construct a new energy high-penetration power grid loss reduction intelligent decision-making model based on the global graph attention proximal policy optimization algorithm;

[0153] The present invention proposes an intelligent decision-making model of the global graph attention proximal policy optimization algorithm (GGAT-PPO). The algorithm framework of the GGAT-PPO model is asFigure 3 as shown;

[0154] The GGAT-PPO algorithm is optimized by defining an objective function. The most core objective is to ensure that the new policy does not deviate too far from the old policy through policy clipping and limitation. The objective function of GGAT-PPO is:

[0155]

[0156] In the formula, θ is the parameter of the policy network of the GGAT-PPO algorithm, r t (θ) is the ratio of the new and old policies, indicating the change range of the current policy relative to the old policy. A t is the advantage function, which is used to measure the quality of a certain action relative to the average performance. ∈ is a hyperparameter that controls the policy update amplitude to prevent excessive updates.

[0157] The GGAT-PPO algorithm consists of two neural networks: the Actor network, the Critic network, and the feature extraction network. The Actor network is responsible for generating policies, that is, selecting corresponding actions (such as adjusting photovoltaic power generation, charging and discharging of energy storage systems, etc.) given the power grid state. The Critic network is responsible for evaluating the value of the current state-action pair and is used to assist the policy update of the Actor network. The feature extraction network is to combine the GGAT network and the DAE network in step 2, aiming to enhance the DRL agent's perception ability of power grid graph data. The features output by the deep autoencoder DAE are fed into the fully connected layer of the Actor to generate the final policy distribution. The output of the Actor network is the action distribution π θ (s|a) of each node. The features output by the deep autoencoder DAE are fed into the fully connected layer of the Critic network to extract the global feature h g , and the Critic network outputs the value function V(s) through the fully connected layer to evaluate the value of the current state s and action a. The PPO algorithm improved by GGAT is the GGAT-PPO algorithm.

[0158] Step 5: Convert the line loss optimization problem of the new energy high-penetration power grid into a Markov decision process, and use the historical operation data and the line loss perception model trained in step 3 to train the loss reduction intelligent decision-making model in step 4 to obtain the adaptive online line loss reduction strategy of the power grid;

[0159] The Markov decision process (MDP) representation of the line loss optimization problem is as follows:

[0160] DRL agent: Regard the dispatching decision center as the agent to control the PV and BSS in the power grid to achieve intelligent decision-making for power grid loss reduction;

[0161] Environment: The environment for agent interaction is the intelligent perception model in Step 3. The agent makes a decision, and the perception model feedbacks an optimized line loss value and voltage value according to the decision quantity, and then calculates the reward value based on the line loss and voltage values.

[0162] State space: When using PV inverters and BSS to reduce line losses, the state at each time step is defined as s t =(m t , A), where m t represents the observed characteristics of the power grid, including: node net active power and reactive power, voltage amplitude and current amplitude, and A represents the adjacency matrix.

[0163] Action space: The action of the agent at each time step is a t =(a pv,t , a bss,t ), where a pv,t represents the reactive power output of the PV inverter at time t, and a bss,t represents the active power output of the BSS at time t.

[0164] Reward function: The reward function is defined as minimizing the line loss of the power grid. The line loss at time t is expressed as p loss,t , and the reward r t =-p loss,t +η, where η is the voltage violation penalty value.

[0165] The training process of the GGAT-PPO algorithm is as follows:

[0166] (1) Loading of GGAT-PPO model parameters: Load the parameters of the trained intelligent perception model in Step 3, including the parameters θ ggat and θ dae of GGAT and DAE, and randomly initialize the parameters of the control models (Actor and Critic networks) in the PPO algorithm.

[0167] (2) Initializing the power grid state observation: For each iteration (episode), the initial state s0 of the power grid is extracted from historical data or real-time data, representing the current power grid node state and line information; each iteration includes multiple time steps, and the agent interacts with the perception model at each time step to generate an experience trajectory;

[0168] (3) The agent executes actions and calculates state transitions and reward values: The agent generates an action a t through the Actor network based on the extracted feature state s t at time step t, and the power grid state changes accordingly after executing this action, transferring to the next state s t+1, according to the executed actions and state changes, the perception model calculates the line loss changes and voltage conditions brought by the current action, generates corresponding reward values, and the reward value represents the impact of the current action on the line loss. The goal is to maximize the cumulative reward by minimizing the line loss;

[0169] (4) Store the experience trajectory in the experience replay pool: Store the current state s t , action a t , reward r t and the new state s t+1 in the experience replay pool replay buffer for subsequent model training. If the replay pool is full, delete the earliest experience data to keep the capacity of the experience pool fixed.

[0170] (5) Sample training data from the experience replay pool: When enough samples have accumulated in the experience replay pool, randomly sample a small batch of mini-batch data, which contains multiple state-action-reward-new state combinations.

[0171] (6) Update the Critic network: The Critic network uses the data sampled from the experience pool to update its parameters; the goal of the Critic network is to minimize the estimation error of the state-action pair, and the update rule is as follows:

[0172]

[0173] In the formula, Loss critic is the loss function of the Critic network, which measures the gap between the actual state value and the estimated value, θ critic and η critic are the parameters and learning rate of the Critic network;

[0174] (7) Update the Actor network: The Actor network uses the advantage function A t calculated from the Critic network and is optimized through the objective function of PPO. The update rule is as follows:

[0175]

[0176] In the formula, L PPO is the objective function of PPO, which ensures the stability of the policy update by restricting the amplitude of the policy update, θ actor and η actor are the parameters and learning rate of the Actor network;

[0177] After multiple rounds of iteration, the Actor and Critic networks gradually converge, and the agent can make optimal decisions based on the current power grid state, maximize the reduction of line loss, and implement an adaptive online loss reduction strategy.

[0178] Traditional PPO algorithms are mainly used for optimizing standard state-action spaces. However, in complex power grid optimization problems, the states of the power grid have obvious spatio-temporal characteristics. The state of the power grid not only depends on the voltage or power information of a single node but also involves the complex topological structure and interaction relationships among the nodes in the power grid. Therefore, traditional PPO algorithms cannot fully utilize these graph structure features, resulting in low efficiency and limited optimization effects when dealing with power grid problems.

[0179] Therefore, the following improvements are made to the present invention based on the prior art:

[0180] 1. The present invention embeds a Global Graph Attention Network (GGAT) into the Actor network and the Critic network of the PPO algorithm to process the graph structure features of the power grid. GGAT can capture the complex interaction relationships among different nodes in the power grid and generate high-dimensional feature representations for each node through the graph attention mechanism, greatly enhancing the state perception ability of the PPO algorithm in the power grid environment.

[0181] 2. Specific improvements to the Actor and Critic networks. The present invention inputs the high-dimensional graph feature representations extracted by GGAT into the Actor network and the Critic network, enabling the Actor network and the Critic network to fully utilize the global graph information of the power grid when generating strategies.

[0182] 3. The present invention combines a Deep Autoencoder (DAE) to reduce the dimension and filter noise of the high-dimensional features extracted by GGAT, thereby generating a more compact low-dimensional representation.

[0183] In summary, in a power grid with high penetration of new energy, the interaction relationships among nodes are complex, and multiple factors such as voltage, power, and load simultaneously affect line losses. This scenario is significantly different from the traditional PPO application scenarios. Therefore, the present invention significantly improves the performance of the PPO algorithm in optimizing line losses in a power grid with high penetration of new energy by introducing the Global Graph Attention Network GGAT into the Actor and Critic networks of the PPO algorithm and combining the Deep Autoencoder DAE for feature dimension reduction. This technical improvement in this specific scenario not only enhances the adaptability of the PPO algorithm in a complex power grid environment but also provides an efficient and intelligent power grid optimization solution through innovative feature processing and policy update mechanisms.

[0184] Embodiment 2

[0185] As Figure 4 shown, the present invention also provides a power grid line loss optimization system based on graph attention perception and reinforcement learning decision-making, which includes:

[0186] A data preprocessing module for collecting real-time grid measurement and historical operation data, and preprocessing and annotating and partitioning the data; the real-time measurement and historical operation data include the voltage of nodes, active load power, reactive load power, line current, active and reactive power of photovoltaic, energy storage charge and discharge records, and line loss conditions, and the annotation partitioning includes converting the preprocessed data into graph data containing node features, edge features, and topological structures.

[0187] Specifically, this application collects data including the voltage of nodes, active and reactive load power, line current, active and reactive power of photovoltaic, energy storage charge and discharge records, line loss conditions, and grid topological structure data; after collecting a large amount of data, due to possible sensor failures, data transmission errors, or other reasons leading to data anomalies during the data collection process, the data is first cleaned and filled for vacancies, and the data is standardized.

[0188] Then, the data is annotated, and the input data is converted into graph data containing node features, edge features, and topological structures. The power grid is represented as a graph G=(V, E, X), where V is the set of nodes representing each node in the power grid, E is the set of edges representing the connection relationships between nodes, and X is the attribute matrix containing node features and edge features, including information such as the voltage, power of nodes, and current and loss of lines. This graph structure not only reflects the physical topological relationship of the power grid but also contains real-time electrical parameters, which helps the model capture global information in the power grid.

[0189] Finally, the line loss data is labeled with the labels of the supervised learning model, and the data set is divided into a training set and a test set, with a division ratio of 80% for the training set and 20% for the test set. The training set is used for training the perception model, and the test set is used for evaluating the final performance of the model.

[0190] An intelligent perception model construction module for constructing an intelligent perception model. The intelligent perception model includes a global graph attention network, a one-dimensional convolutional neural network, and a fully connected network. The input data is transmitted to the global graph attention network to obtain the hidden state of each node, and then processed by the one-dimensional convolutional neural network and the fully connected network in sequence, and finally high-dimensional node features are output.

[0191] In this embodiment, the global graph attention network GGAT is the core part of the model. It captures the complex relationships between each node through the graph attention mechanism, including the mutual influence between electrical states such as voltage and power and the topological structure. In GGAT, the feature vector of each node not only depends on its own state information but also combines the information of neighboring nodes for update. Through the attention mechanism, the model can adaptively learn the influence degree of neighboring nodes on the target node.

[0192] The formula for deriving the hidden state of each node through the global graph attention network GGAT is as follows:

[0193]

[0194] In the formula, h att,i is the feature representation of node i, α ij is the attention score measuring the importance of node j to node i, N(i) represents the set of adjacent nodes of node i, g j (x j ) is the embedding function of node j composed of a single-layer neural network, x j is the input feature of node j, σ is the activation function, and the attention score α ij can be calculated by the following formula:

[0195]

[0196] In the formula, represents the concatenation of the embeddings of two nodes, w is the weight vector, LeakyReLU is the activation function, and e ij represents the attention weight. To extract the common features of the entire system, after the graph attention network, a one-dimensional convolutional neural network CNN is used:

[0197]

[0198] In the formula, W i is the weight of the i-th neural network layer, b i represents the bias of the i-th neural network layer, F represents the number of neural network layers, σ is the activation function, and h cov,i is the output of the feature quantity after passing through the convolutional layer. Then, the fully connected network FCN is used to further embed the features, and the final output layer is the representation h g of the high-level node features.

[0199] A feature dimensionality reduction module for compressing the high-dimensional node features into a low-dimensional representation using a deep autoencoder.

[0200] After the GGAT module processes the real-time and pseudo-measurement data, the model generates a high-dimensional feature representation h g . To reduce the computational burden while maintaining important feature information, it is necessary to perform dimensionality reduction on it. Therefore, a deep autoencoder DAE is proposed. DAE consists of two parts: an encoder and a decoder. First, the encoder compresses the high-dimensional feature h g into a low-dimensional representation h e , and the encoding process is as follows:

[0201] h e = f l […f1(h g)],

[0202] In the formula, f l represents the l-th layer FCN in the encoder. Through the non-linear mapping of multiple layers of FCN, the original feature h g is converted into a low-dimensional representation h e , that is, compressed from a high-dimensional space to a low-dimensional space.

[0203] An intelligent perception model training module, which is used to train the intelligent perception model and the deep autoencoder using a training set to obtain a trained intelligent perception model.

[0204] In this embodiment, the training process of the intelligent perception model is as follows:

[0205] Parameter initialization of the intelligent perception model. Before training the intelligent perception model, randomly initialize the parameters θ ggat and θ dae of GGAT and DAE, and use the Xavier initialization method to avoid the problem of gradient disappearance or explosion;

[0206] Represent the collected and preprocessed power grid node features, edge features, and topological structure in the form of a graph and input them into the global graph attention network;

[0207] Use historical data and labels to perform supervised training on the CNN-based GGAT network, calculate the actual line loss of each line and the line loss predicted by the model, and use the loss function Loss(θ ggat ) for gradient descent;

[0208] Among them, the loss function Loss(θ ggat ) of the perception model is calculated as follows:

[0209]

[0210] In the formula, B is the sampling batch, is the actual line loss of the i-th line, is the line loss predicted by the model.

[0211] Use the gradient descent method to update the parameters θ ggat of the perception model, and optimize the model by minimizing the loss function:

[0212]

[0213] Among them, η is the learning rate. This optimization process will gradually adjust the model parameters to enable it to accurately predict the line loss.

[0214] Train the DAE parameter θ dae in an unsupervised manner;

[0215] After the initial training of GGAT, unsupervised learning is used to train the DAE to reduce the dimension of high-dimensional features while ensuring the retention of information. The DAE optimizes the parameters of the encoder by minimizing the reconstruction error, and the reconstruction loss function Loss(θ dae ) is calculated as follows:

[0216]

[0217] In the formula, h g is the high-dimensional feature generated by GGAT, is the feature reconstructed by the DAE decoder. Finally, by optimizing this loss function, the model can extract a more robust low-dimensional representation from the high-dimensional features.

[0218] Optimization and evaluation of model convergence effect;

[0219] After multiple rounds of training, the intelligent perception model gradually converges, can accurately predict line losses, and adapt to the operation of the power grid with a high proportion of new energy access. The trained perception model is verified using the test set to evaluate its ability to online perceive power grid line losses, and further optimize the parameters and structure of the model to ensure its effectiveness and robustness in practical applications.

[0220] The intelligent decision-making model construction module is used to construct an intelligent decision-making model. The intelligent decision-making model includes two neural networks: the Actor network and the Critic network. The Actor network receives the data output by the trained intelligent perception model, and then obtains the action distribution of each node; the Critic network receives the data output by the trained intelligent perception model, and then the Critic network outputs a value function through a fully connected layer to evaluate the value of the current state and action.

[0221] The intelligent decision-making model is an intelligent decision-making model based on the global graph attention proximal policy optimization algorithm GGAT-PPO. The GGAT-PPO algorithm is optimized by defining an objective function. The most core goal is to ensure that the new policy does not deviate too far from the old policy through policy clipping and limitation. The objective function of GGAT-PPO is:

[0222]

[0223] In the formula, θ is the parameter of the GGAT-PPO algorithm policy network, r t (θ) is the ratio of the new and old policies, indicating the change range of the current policy relative to the old policy. A t is the advantage function, which is used to measure the quality of a certain action relative to the average performance. ∈ is a hyperparameter that controls the policy update amplitude to prevent excessive updates.

[0224] The GGAT-PPO algorithm consists of two neural networks: the Actor network, the Critic network, and the feature extraction network. The Actor network is responsible for generating policies, that is, selecting corresponding actions (such as adjusting photovoltaic power generation, charging and discharging of energy storage systems, etc.) given the power grid state. The Critic network is responsible for evaluating the value of the current state-action pair and is used to assist in updating the policy of the Actor network. The feature extraction network is the GGAT network and the DAE network in step 2, aiming to enhance the DRL agent's perception ability of power grid graph data. The features output by the deep autoencoder DAE are fed into the fully connected layer of the Actor to generate the final policy distribution, and the output of the Actor network is the action distribution π θ (s|a). The features output by the deep autoencoder DAE are fed into the fully connected layer of the Critic network to extract the global feature h g , and the Critic network outputs the value function V(s) through the fully connected layer to evaluate the value of the current state s and action a. The PPO algorithm improved by GGAT is the GGAT-PPO algorithm.

[0225] The intelligent decision-making model training module is used to transform the line loss optimization problem of the new energy high-penetration power grid into a Markov decision-making process, so as to train the intelligent decision-making model according to the obtained action distribution of each node and the value of the current state and action, and then obtain the adaptive online loss reduction strategy of the new energy high-penetration power grid.

[0226] The Markov decision-making process (MDP) of the line loss optimization problem is expressed as follows:

[0227] DRL agent: Regarding the dispatching decision center as an agent to control PV and BSS in the power grid to achieve intelligent decision-making for power grid loss reduction;

[0228] Environment: The environment for the agent to interact is the intelligent perception model. The agent makes a decision, and the perception model feeds back an optimized line loss value and voltage value according to the decision quantity, and then calculates the reward value according to the line loss and voltage values.

[0229] State space: When using PV inverters and BSS to reduce line loss, the state at each time step is defined as s t =(m t ,A), where m t represents the observed features of the power grid, including: node net active power and reactive power, voltage amplitude and current amplitude, and A represents the adjacency matrix.

[0230] Action space: The action of the agent at each time step is a t =(a pv,t ,a bss,t ), where a pv,tDenote the reactive power output of the PV inverter at time t as a bss,t Denote the active power output of the BSS at time t

[0231] Reward function: The reward function is defined as minimizing the grid line loss. The line loss at time t is denoted as p loss,t , and the reward r t =-p loss,t +η, where η is the voltage violation penalty value.

[0232] The training process of the GGAT-PPO algorithm is as follows:

[0233] (1)GGAT-PPO model parameter loading: Load the parameters of the trained intelligent perception model, including the parameters θ ggat of GGAT and DAE dae , and randomly initialize the parameters of the control models (Actor and Critic networks) in the PPO algorithm.

[0234] (2)Initializing the grid state observation: For each iteration (episode), the initial state s0 of the grid is extracted from historical data or real-time data, representing the current grid node state and line information; each iteration consists of multiple time steps, and the agent interacts with the perception model at each time step to generate an experience trajectory;

[0235] (3)Agent executes actions and calculates state transitions and reward values: At time step t, the agent generates an action a t through the Actor network based on the extracted feature state s t , and when this action is executed, the grid state changes and transfers to the next state s t+1 . According to the executed action and state change, the perception model calculates the line loss change and voltage situation brought by the current action, and generates the corresponding reward value. The reward value represents the impact of the current action on the line loss, and the goal is to maximize the cumulative reward by minimizing the line loss;

[0236] (4)Store the experience trajectory in the experience replay pool: Store the current state s t , action a t , reward r t and the new state s t+1 in the experience replay pool replay buffer for subsequent model training. If the replay pool is full, delete the earliest experience data to keep the capacity of the experience pool fixed.

[0237] (5)Sample training data from the experience replay pool: When enough samples have accumulated in the experience replay pool, randomly sample a small batch of mini-batch data from it, which contains multiple combinations of state-action-reward-new state.

[0238] (6) Update the Critic network: The Critic network uses the data sampled from the experience pool to update its parameters; the goal of the Critic network is to minimize the estimation error of the state-action pair, and the update rule is as follows:

[0239]

[0240] In the formula, Loss critic is the loss function of the Critic network, which measures the gap between the actual state value and the estimated value, θ critic and η critic are the parameters and learning rate of the Critic network;

[0241] (7) Update the Actor network: The Actor network uses the advantage function A t calculated from the Critic network and is optimized through the objective function of PPO. The update rule is as follows:

[0242]

[0243] In the formula, L PPO is the objective function of PPO, which ensures the stability of the policy update by restricting the amplitude of the policy update. θ actor and η actor are the parameters and learning rate of the Actor network;

[0244] After multiple rounds of iteration, the Actor and Critic networks gradually converge, and the agent can make optimal decisions based on the current power grid state, maximizing the reduction of line losses and implementing an adaptive online loss reduction strategy.

[0245] Example 3

[0246] Refer to Figure 5 , Figure 5 which is a schematic diagram of the terminal structure of the hardware operating environment involved in the solution of the embodiment of the present invention.

[0247] As Figure 5As shown in the figure, the terminal may include: a processor 1001, such as a Central Processing Unit (CPU), a communication bus 1002, a network interface 1003, and a memory 1004. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The network interface 1003 may optionally include a standard wired interface and a wireless interface (such as a Wireless-Fidelity (WI-FI) interface). The memory 1004 may be a high-speed Random Access Memory (RAM) or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1004 may also be an independent storage device independent of the aforementioned processor 1001.

[0248] Those skilled in the art can understand that Figure 5 the terminal structure shown in the figure does not constitute a limitation on the terminal, and it may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements.

[0249] As Figure 5 shown in the figure, the memory 1004, as a computer storage medium, may include an operating system, a data storage module, a network communication module, and a power grid line loss optimization program based on graph attention perception and reinforcement learning decision-making.

[0250] In Figure 5 the terminal shown in the figure, the network interface 1003 is mainly used to connect to the background server and communicate with the background server for data; the processor 1001 can call the power grid line loss optimization program stored in the memory 1004 and perform the following operations:

[0251] S1: Collect real-time measurement and historical operation data of the power grid, and preprocess and label and divide the data; the real-time measurement and historical operation data include the voltage of nodes, active load power, reactive load power, line current, active and reactive power of photovoltaic power, charge and discharge records of energy storage, and line loss conditions, and the label division includes converting the preprocessed data into graph data containing node features, edge features, and topological structures;

[0252] S2: Construct an intelligent perception model, the intelligent perception model includes a global graph attention network, a one-dimensional convolutional neural network, and a fully connected network, transmit the input data to the global graph attention network to obtain the hidden state of each node, and then process it sequentially using the one-dimensional convolutional neural network and the fully connected network, and finally output high-dimensional node features;

[0253] S3 uses a deep autoencoder to compress the high-dimensional node features into a low-dimensional representation;

[0254] S4 uses a training set to train the intelligent perception model and the deep autoencoder to obtain a trained intelligent perception model;

[0255] S5 constructs an intelligent decision-making model, which includes two neural networks: an Actor network and a Critic network. The Actor network receives the data output by the trained intelligent perception model to obtain the action distribution of each node; the Critic network receives the data output by the trained intelligent perception model, and then the Critic network outputs a value function through a fully connected layer to evaluate the value of the current state and action;

[0256] S6 transforms the line loss optimization problem of the new energy high-penetration power grid into a Markov decision process, thereby training the intelligent decision-making model according to the obtained action distribution of each node and the value of the current state and action, and then obtaining the adaptive online loss reduction strategy of the new energy high-penetration power grid.

[0257] Further, the processor 1001 can call the power grid line loss optimization program stored in the memory 1004 based on graph attention perception and reinforcement learning decision-making, and also perform the following operations:

[0258] In the step S1, the graph data is represented as:

[0259] G=(V, E, X), where V is the set of nodes, representing each node in the power grid, E is the set of edges, representing the connection relationship between nodes, and X is an attribute matrix containing node features and edge features, including the voltage of the node, active power of the load, reactive power of the load, line current, active power and reactive power of the photovoltaic, charge and discharge records of the energy storage, and the loss situation of the line.

[0260] Further, the processor 1001 can call the power grid line loss optimization program stored in the memory 1004 based on graph attention perception and reinforcement learning decision-making, and also perform the following operations:

[0261] In the step S2, constructing the intelligent perception model specifically includes:

[0262] The formula for calculating the hidden state of each node through the global graph attention network is as follows:

[0263]

[0264] where h att,i is the feature representation of node i, α ijThe attention score for measuring the importance of node j to node i, where N(i) represents the set of adjacent nodes of node i, and g j (x j ) is the embedding function of node j composed of a single-layer neural network, and x j is the input feature of node j, σ is the activation function, and the attention score α ij is calculated by the following formula:

[0265]

[0266] In the formula, L represents the number of layers of the global graph attention network, and g i (x i ) is the embedding function of node i composed of a single-layer neural network, represents the concatenation of the embeddings of two nodes, w is the weight vector, LeakyReLU is the activation function, and e ij represents the attention weight. To extract the common features of the entire system, after the graph attention network, a one-dimensional convolutional neural network is used, denoted as:

[0267]

[0268] In the formula, W i is the weight of the i-th one-dimensional convolutional neural network layer, b i represents the bias of the i-th one-dimensional convolutional neural network layer, F represents the number of layers of the one-dimensional convolutional neural network, σ is the activation function, and h cov,i is the output of the feature quantity after passing through the convolutional layer;

[0269] Then, the fully connected network is used to further embed the feature h cov,i , and the final output layer is the representation h g of the high-level node features.

[0270] Furthermore, the processor 1001 can call the grid line loss optimization program stored in the memory 1004 based on graph attention perception and reinforcement learning decision-making, and also perform the following operations:

[0271] The high-dimensional node features are compressed into a low-dimensional representation by using a deep autoencoder, and the encoding process is as follows:

[0272] h e = f l […f1(h g )],(5)

[0273] In the formula, f l represents the fully connected network of the l-th layer in the encoder. Through the non-linear mapping of the multi-layer fully connected network, the original high-dimensional feature h g is converted into a low-dimensional representation h e, that is, compressing from a high-dimensional space to a low-dimensional space.

[0274] Furthermore, the processor 1001 can call the grid line loss optimization program stored in the memory 1004 based on graph attention perception and reinforcement learning decision-making, and also perform the following operations:

[0275] The step S4 specifically includes:

[0276] S41 Randomly initialize the parameters θ of the global graph attention network and the deep autoencoder ggat and θ dae ;

[0277] S42 Input the data after annotation division in step S1 into the global graph attention network;

[0278] S43 Use historical data and labels to perform supervised training on the global graph attention network, calculate the actual line loss of each line and the line loss predicted by the model, and use the loss function Loss(θ ggat ) to perform gradient descent. During the optimization process, the model parameters will be gradually adjusted to enable accurate prediction of line losses;

[0279] S44 Use unsupervised learning to train the deep autoencoder, and optimize the parameters of the encoder by minimizing the reconstruction loss function Loss(θ dae ). Finally, by optimizing this loss function, extract low-dimensional representations from high-dimensional features; S45 Loop and iterate steps S42 - S44 until the corresponding loss function gradually converges, thereby predicting line losses and adapting to the operation of the power grid with high-proportion new energy access.

[0280] Furthermore, the processor 1001 can call the grid line loss optimization program stored in the memory 1004 based on graph attention perception and reinforcement learning decision-making, and also perform the following operations:

[0281] In the step S43, the loss function Loss(θ ggat ) is expressed as:

[0282]

[0283] where B is the sampling batch, is the true line loss of the i-th line, is the line loss predicted by the model for the i-th line;

[0284] Update the parameters θ of the perception model using the gradient descent method ggat , and optimize the model by minimizing the loss function:

[0285]

[0286] Among them, η is the learning rate.

[0287] Furthermore, it includes:

[0288] The loss function Loss(θ dae ) is specifically expressed as:

[0289]

[0290] Among them, L represents the number of layers of the DAE neural network, h g is the high-dimensional feature generated by the global graph attention network, is the feature reconstructed by the deep auto-decoder.

[0291] Furthermore, the processor 1001 can call the grid line loss optimization program stored in the memory 1004 based on graph attention perception and reinforcement learning decision-making, and also perform the following operations:

[0292] In the step S5, the objective function of the intelligent decision-making model is expressed as:

[0293]

[0294] Among them, θ is the parameter of the policy network, r t (θ) is the ratio of the new and old policies, that is, it represents the change range of the current policy relative to the old policy, A t is the advantage function, which is used to measure the quality of a certain action relative to the average performance. ∈ is a hyperparameter that controls the update range of the policy to prevent excessive update, and clip() is a truncation function, represents the expectation.

[0295] Furthermore, the processor 1001 can call the grid line loss optimization program stored in the memory 1004 based on graph attention perception and reinforcement learning decision-making, and also perform the following operations:

[0296] In the step S6, the line loss optimization problem of the new energy high-penetration power grid is transformed into a Markov decision process, which is described as follows:

[0297] DRL agent: Regarding the dispatching decision center as the agent to control the PV and BSS in the power grid to achieve intelligent decision-making for power grid loss reduction;

[0298] Environment: The environment for agent interaction is the intelligent perception model in step S2. When the agent makes a decision, the intelligent perception model feeds back an optimized line loss value and voltage value according to the decision-making quantity, and then calculates the reward value according to the line loss and voltage values;

[0299] State space: When using PV inverters and BSS to reduce line loss, the state at each time step is defined as st =(m t , A), where m t represents the observed characteristics of the power grid, including: node net active power and reactive power, voltage amplitude, and current amplitude, and A represents the adjacency matrix;

[0300] Action space: The action of the agent at each time step is a t =(a pv,t , a bss,t ), where a pv,t represents the reactive power output of the PV inverter at time t, and a bss,t represents the active power output of the BSS at time t;

[0301] Reward function: The reward function is defined as minimizing the power grid line loss. The line loss at time t is expressed as p loss,t , and the reward r t =-p loss,t +η, where η is the voltage violation penalty value.

[0302] Furthermore, the processor 1001 can call the power grid line loss optimization program stored in the memory 1004 based on graph attention perception and reinforcement learning decision-making, and also perform the following operations:

[0303] In the step S6, the intelligent decision-making model training process includes:

[0304] S61 Model parameter loading: Load the parameters of the intelligent perception model trained in step S4, including the parameters θ ggat and θ dae of GGAT and DAE, and randomly initialize the parameters of the Actor network and the Critic network;

[0305] S62 Initializing the power grid state observation: For each round of iteration, the initial state s0 of the power grid is extracted from historical data or real-time data, representing the current power grid node state and line information; each iteration includes multiple time steps, and the agent interacts with the perception model at each time step to generate an experience trajectory;

[0306] S63 The agent executes actions and calculates the state transition and reward value: The agent generates an action a t through the Actor network according to the extracted feature state s t at time step t. Executing this action causes the power grid state to change and transfer to the next state s t+1 . According to the executed action and state change, the perception model calculates the line loss change and voltage situation brought by the current action, and generates a corresponding reward value. The reward value represents the impact of the current action on the line loss, and the goal is to maximize the cumulative reward by minimizing the line loss;

[0307] S64 Store the experience trajectory in the experience replay pool: Store the current state s t , action a t , reward r t and the new state s t+1 in the experience replay pool for subsequent model training. If the replay pool is full, delete the earliest experience data to keep the capacity of the experience pool fixed.

[0308] S65 Sample training data from the experience replay pool: When enough samples have accumulated in the experience replay pool, randomly sample a small batch of data containing multiple state-action-reward-new state combinations.

[0309] S66 Update the Critic network and the Actor network;

[0310] S67 After multiple rounds of iteration, the Actor network and the Critic network gradually converge, and the agent can make optimal decisions based on the current power grid state to maximize the reduction of line loss and implement an adaptive online line loss reduction strategy.

[0311] Furthermore, the processor 1001 can call the power grid line loss optimization program stored in the memory 1004 based on graph attention perception and reinforcement learning decision-making, and also perform the following operations:

[0312] In the step S66, updating the Critic network specifically includes:

[0313] The Critic network uses the data sampled from the experience pool to update its parameters; the goal of the Critic network is to minimize the estimation error of the state-action pair, and the update rule is as follows:

[0314]

[0315] In the formula, Loss critic is the loss function of the Critic network, which measures the gap between the actual state value and the estimated value, θ critic and η critic are the parameters and learning rate of the Critic network.

[0316] Furthermore, the processor 1001 can call the power grid line loss optimization program stored in the memory 1004 based on graph attention perception and reinforcement learning decision-making, and also perform the following operations:

[0317] In the step S66, updating the Actor network specifically includes:

[0318] The Actor network uses the advantage function A calculated from the Critic network t , and the update rule is as follows:

[0319]

[0320] where θ actor and η actor are the parameters and learning rate of the Actor network.

[0321] In addition, those of ordinary skill in the art can understand that all or part of the processes in the methods for implementing the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the control terminal to implement the flow steps of the embodiments of the above methods.

[0322] Therefore, the present invention also provides a computer-readable storage medium storing a power grid line loss optimization program based on graph attention perception and reinforcement learning decision-making. When the power grid line loss optimization program based on graph attention perception and reinforcement learning decision-making is executed by a processor, it implements each step of the power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making as described in the above embodiments.

[0323] It should be noted that since the storage medium provided in the embodiments of the present application is the storage medium used for implementing the methods of the embodiments of the present application, based on the methods introduced in the embodiments of the present application, those skilled in the art can understand the specific structure and variations of the storage medium, so it will not be elaborated here. Any storage medium used in the methods of the embodiments of the present application falls within the scope to be protected by the present application.

[0324] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0325] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can also be implemented. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in one block or multiple blocks.

[0326] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implement the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in one block or multiple blocks.

[0327] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in one block or multiple blocks.

[0328] It should be noted that in the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" placed before a component does not exclude the presence of a plurality of such components. The present invention can be implemented by means of hardware including several different components and by means of a properly programmed computer. In the unit claims listing several apparatuses, several of these apparatuses can be embodied by the same hardware item. The use of the words first, second, and third, etc. does not denote any order. These words can be interpreted as names.

[0329] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concept. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0330] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

[0331] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied to other related technical fields, shall be included in the patent protection scope of the present invention by the same token.

Claims

1. A power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making, characterized in that The method includes the following steps: S1 Collect real-time measurement and historical operation data of the power grid, and perform preprocessing and labeled partitioning on the data; the real-time measurement and historical operation data include the voltage of nodes, active power of loads, reactive power of loads, line current, active power and reactive power of photovoltaic power, charge and discharge records of energy storage, and line loss conditions, and the labeled partitioning includes converting the preprocessed data into graph data containing node features, edge features, and topological structures; S2 Construct an intelligent perception model, the intelligent perception model includes a global graph attention network, a one-dimensional convolutional neural network, and a fully connected network, transmit the input data to the global graph attention network to obtain the hidden state of each node, and then process it successively with the one-dimensional convolutional neural network and the fully connected network, and finally output high-dimensional node features; S3 Use a deep autoencoder to compress the high-dimensional node features into a low-dimensional representation; S4 Use the training set to train the intelligent perception model and the deep autoencoder to obtain the trained intelligent perception model; S5 Construct an intelligent decision-making model, the intelligent decision-making model includes two neural networks: an Actor network and a Critic network, the Actor network receives the data output by the trained intelligent perception model, and then obtains the action distribution of each node; the Critic network receives the data output by the trained intelligent perception model, and then the Critic network outputs a value function through a fully connected layer to evaluate the value of the current state and action; S6 Convert the line loss optimization problem of the new energy high-penetration power grid into a Markov decision process, so as to train the intelligent decision-making model according to the obtained action distribution of each node and the value of the current state and action, and then obtain the adaptive online loss reduction strategy of the new energy high-penetration power grid.

2. The grid line loss optimization method based on graph attention perception and reinforcement learning decision-making according to claim 1, wherein In the step S1, the graph data is represented as: G=(V,E,X), where V is the node set, representing each node in the power grid, E is the edge set, representing the connection relationship between nodes, and X is an attribute matrix containing node features and edge features, including the voltage of nodes, active power of loads, reactive power of loads, line current, active power and reactive power of photovoltaic power, charge and discharge records of energy storage, and line loss conditions.

3. The grid line loss optimization method based on graph attention perception and reinforcement learning decision-making according to claim 2, wherein, In the step S2, the construction of the intelligent perception model specifically includes: The formula for calculating the hidden state of each node through the global graph attention network is as follows: Among them, h att,i is the feature representation of node i, α ij is the attention score measuring the importance of node j to node i, N(i) represents the set of adjacent nodes of node i, g j (x j ) is the embedding function of node j composed of a single-layer neural network, x j is the input feature of node j, σ is the activation function, and the attention score α ij is calculated by the following formula: where L1 represents the number of layers of the global graph attention network, and g i (x i ) is the embedding function of node i composed of a single-layer neural network, represents the connection of two node embeddings, w is the weight vector, LeakyReLU is the activation function, and e ij represents the attention weight. To extract the common features of the entire system, after the graph attention network, a one-dimensional convolutional neural network is used, denoted as: Wherein, W k is the weight of the k-th one-dimensional convolutional neural network layer, b k represents the bias of the k-th neural network layer, F represents the number of layers of the one-dimensional convolutional neural network, σ is the activation function, and h cov,k is the output of the feature quantity passing through the convolutional layer; Then, the fully-connected network is used to further embed the feature h cov,k , and the final output layer is the representation h of the high-level node features g .

4. The power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making according to claim 3, characterized in that, The process of using the deep autoencoder to compress the high-dimensional node features into a low-dimensional representation is as follows: h e = f m […f1(h g )],(5) where f m represents the fully connected network of the m-th layer in the deep autoencoder. Through the non-linear mapping of the multi-layer fully connected network, the original high-dimensional feature h g is transformed into the low-dimensional representation h e , that is, compressed from the high-dimensional space to the low-dimensional space.

5. The grid line loss optimization method based on graph attention perception and reinforcement learning decision-making according to claim 4, characterized in that The step S4 specifically includes: S41 Randomly initialize the parameters θ of the global graph attention network and the deep autoencoder ggat and θ dae ; S42 Input the data after labeled partitioning in step S1 into the global graph attention network; S43 uses historical data and labels to perform supervised training on the global graph attention network, calculates the actual line loss of each line and the line loss predicted by the model, and uses the loss function Loss(θ ggat ) to perform gradient descent. During the optimization process, the model parameters will be gradually adjusted to enable accurate prediction of line losses; S44 trains a deep autoencoder using unsupervised learning and optimizes the parameters of the encoder by minimizing the reconstruction loss function Loss(θ dae ) Finally, by optimizing this loss function, it extracts a low-dimensional representation in high-dimensional features; S45 Loop and iterate steps S42 - S44 until the corresponding loss function gradually converges, so as to predict the line loss and adapt to the operation of the power grid with high proportion of new energy access.

6. The power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making according to claim 5, wherein In the step S43, the loss function Loss(θ ggat ) is expressed as: where B is the sampling batch, is the true line loss of the q-th line, is the line loss predicted by the model for the q-th line; Update the parameters θ of the perceptron model using the gradient descent method ggat , and optimize the model by minimizing the loss function: Among them, η is the learning rate.

7. The grid line loss optimization method based on graph attention perception and reinforcement learning decision-making according to claim 5, characterized in that, The loss function Loss(θ dae ) is specifically expressed as: where L2 represents the number of layers of the deep autoencoder, h g is the high-dimensional feature generated by the global graph attention network, and is the feature reconstructed by the deep auto-decoder.

8. The power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making according to claim 7, characterized in that, In the step S5, the objective function of the intelligent decision-making model is expressed as: where θ are the parameters of the policy network, and r t (θ) is the ratio of the new and old policies, which represents the change magnitude of the current policy relative to the old policy. A t is the advantage function at time t, which is used to measure the quality of a certain action relative to the average performance. ∈ is a hyperparameter that controls the policy update magnitude, and clip() is a truncation function, denotes the expectation.

9. The grid line loss optimization method based on graph attention perception and reinforcement learning decision-making according to claim 8, wherein, In the step S6, converting the line loss optimization problem of the new energy high-penetration power grid into a Markov decision process is described as follows: DRL Agent: Regarding the dispatching decision center as an agent to control PV and BSS in the power grid to achieve intelligent decision-making for power grid loss reduction; Environment: The environment for agent interaction is the intelligent perception model in step S2. When the agent makes a decision, the intelligent perception model feeds back an optimized line loss value and voltage value according to the decision-making quantity, and then calculates the reward value based on the line loss and voltage values; State space: When using a PV inverter and a BSS to reduce line losses, the state at each time step is defined as s t =(m t , A), where m t represents the observed characteristics of the power grid, including: node net active power and reactive power, voltage amplitude, and current amplitude, and A represents the adjacency matrix; Action space: The action of the agent at each time step is a t =(a pv,t , a bss,t ), where a pv,t represents the reactive power output of the PV inverter at time t, and a bss,t represents the active power output of the BSS at time t; Reward function: The reward function is defined as minimizing the power loss of the power grid. The power loss at time t is denoted as p loss,t , and the reward r t = -p loss,t + η, where η is the penalty value for voltage violation.

10. The grid line loss optimization method based on graph attention perception and reinforcement learning decision-making according to claim 9, characterized in that, In step S6, the training process of the intelligent decision-making model includes: S61 Model parameter loading: Load the parameters of the intelligent perception model trained in step S4, including the parameters θ of GGAT and DAE ggat and θ dae , and randomly initialize the parameters of the Actor network and the Critic network; S62 Initialize the power grid state observation: For each iteration, the initial state s0 of the power grid is extracted from historical data or real-time data, representing the current power grid node state and line information; Each iteration consists of multiple time steps, and the agent interacts with the perception model at each time step to generate an experience trajectory; The S63 agent executes actions and calculates state transitions and reward values: At time t, the agent generates an action a based on the extracted feature state s t through the Actor network t , and executes this action, causing the power grid state to change and transfer to the next state s t+1 . According to the executed action and state change, the perception model calculates the line loss change and voltage situation brought about by the current action, and generates a corresponding reward value. The reward value represents the impact of the current action on the line loss. The goal is to maximize the cumulative reward by minimizing the line loss; S64 Store the experience trajectory in the experience replay pool: Store the current state s t , action a t , reward r t and the new state s t+1 in the experience replay pool for subsequent use in training the model. If the replay pool is full, delete the earliest experience data to keep the capacity of the experience pool fixed; S65 Sample training data from the experience replay pool: When enough samples have been accumulated in the experience replay pool, randomly sample a small batch of data, including multiple state-action-reward-new state combinations; S66 Update the Critic network and the Actor network; S67 After multiple iterations, the Actor network and the Critic network gradually converge to achieve an adaptive online loss reduction strategy.

11. The power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making as claimed in claim 10, wherein In step S66, updating the Critic network specifically includes: The Critic network uses the data sampled from the experience pool to update its parameters; The goal of the Critic network is to minimize the estimation error of the state-action pair, and the update rule is as follows: where Loss critic is the loss function of the Critic network, which measures the gap between the actual state value and the estimated value, θ critic and η critic are the parameters and learning rate of the Critic network.

12. The power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making according to claim 10, characterized in that In step S66, updating the Actor network specifically includes: The Actor network uses the advantage function A calculated from the Critic network t , and the update rule is as follows: where θ actor and η actor are the parameters and learning rate of the Actor network.

13. A power grid line loss optimization system based on graph attention perception and reinforcement learning decision-making, characterized in that, The system includes: A data preprocessing module, which is used to collect real-time measurements and historical operation data of the power grid, and preprocess and label and divide the data; The real-time measurements and historical operation data include the voltage of nodes, active power of loads, reactive power of loads, line current, active power and reactive power of photovoltaic, energy storage charge and discharge records, and line loss conditions. The label division includes converting the preprocessed data into graph data containing node features, edge features, and topological structures; An intelligent perception model construction module, which is used to construct an intelligent perception model. The intelligent perception model includes a global graph attention network, a one-dimensional convolutional neural network, and a fully connected network. The input data is transmitted to the global graph attention network to obtain the hidden state of each node, and then processed by the one-dimensional convolutional neural network and the fully connected network in sequence, and finally high-dimensional node features are output; A feature dimensionality reduction module, which is used to compress the high-dimensional node features into a low-dimensional representation by using a deep autoencoder; An intelligent perception model training module, which is used to train the intelligent perception model and the deep autoencoder with a training set to obtain a trained intelligent perception model; An intelligent decision-making model construction module, which is used to construct an intelligent decision-making model. The intelligent decision-making model includes two neural networks: an Actor network and a Critic network. The Actor network receives the data output by the trained intelligent perception model, and then obtains the action distribution of each node. The Critic network receives the data output by the trained intelligent perception model, and then the Critic network outputs a value function through a fully connected layer to evaluate the value of the current state and action. An intelligent decision-making model training module, which is used to transform the line loss optimization problem of a new energy high-penetration power grid into a Markov decision process, so as to train the intelligent decision-making model according to the obtained action distribution of each node and the value of the current state and action, and then obtain the adaptive online loss reduction strategy of the new energy high-penetration power grid.

Images