Power grid line loss optimization method and system based on graph attention perception and reinforcement learning decision

Abstract

The invention provides a power grid line loss optimization method and system based on a graph attention network and reinforcement learning decision, and the method comprises the steps: collecting the real-time measurement and historical operation data of a power grid, carrying out the preprocessing and marking division of the data, and carrying out the marking division of the data based on a global graph attention network and a depth automatic encoder; constructing an intelligent sensing model of the new energy high-permeability power grid; then, training an intelligent sensing model by using historical data, and realizing intelligent online sensing of the line loss of the power grid under high-proportion new energy access; secondly, based on the global graph attention network model and a near-end strategy optimization algorithm, constructing a new energy high-permeability power grid intelligent decision model; and finally, converting the line loss optimization problem of the new energy high-permeability power grid into a Markov decision process, and training a loss reduction intelligent decision model to obtain a self-adaptive online loss reduction strategy of the power grid. According to the method, the optimal decision can be made based on the current power grid state, the reduction of the line loss is maximized, and a self-adaptive online loss reduction strategy is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of power grid line loss optimization control, and in particular to a power grid line loss optimization method and system based on graph attention perception and reinforcement learning decision-making. Background Art

[0002] As the penetration rate of renewable energy (such as photovoltaic power generation and battery energy storage) in distribution networks continues to increase, the problem of line loss in grid operation has become increasingly prominent. The intermittent and volatile nature of renewable energy complicates the flow of power in the grid, which in turn exacerbates line losses. How to minimize line losses while ensuring safe and stable operation of the grid has become an important challenge in power system optimization.

[0003] Currently, many methods have been proposed to optimize grid operation to reduce line losses. Stochastic programming (SP) is a commonly used method to deal with the uncertainty of renewable energy generation and load demand, but the SP method relies on the probability distribution of random variables, which is often difficult to obtain in practical applications. Another commonly used method is robust optimization (RO), which deals with uncertainty by finding a solution that is robust to all situations in the uncertainty set. However, robust optimization usually solves the optimization problem in the worst case, resulting in its solution being too conservative and unable to flexibly respond to rapidly changing photovoltaic power generation fluctuations. In addition, both SP and RO methods rely on accurate physical models, which often show great limitations in actual distribution networks due to the existence of system model errors.

[0004] In contrast, machine learning methods shift the burden of online computation to the offline training process, so they are able to make decisions within milliseconds based on the latest observation data. Among the many machine learning methods, deep reinforcement learning (DRL) is widely used in the optimal control of power systems. However, many DRL-based methods still assume an accurate physical model to generate data for training. In order to reduce the dependence on accurate physical models, researchers have proposed some optimization methods based on physical-free models. These methods use historical operating data to train DRL agents, avoiding the dependence on accurate models of distribution networks.

[0005] In recent years, graph neural networks (GNNs) have gradually attracted attention as a deep learning method that can learn node relationships from graph structured data. GNN-based graph reinforcement learning methods have been applied to respond to power grid topology changes, and graph learning methods for distribution network restoration have been proposed.

[0006] However, existing methods still rely on the precise line parameters of the power grid during training, and usually assume that global observation information of the entire network can be obtained. In practical applications, due to problems such as data acquisition errors, communication failures or measurement noise, only partial observation information of the power grid can be obtained, which brings great challenges to the line loss optimization method based on DRL. Summary of the invention

[0007] Purpose of the invention: In order to solve the problems arising from the above-mentioned prior art, the present invention provides a grid line loss optimization method based on graph attention perception and reinforcement learning decision-making, which solves the problems of excessive line loss in new energy high-penetration power grids, inaccurate perception of new energy power uncertainty and insufficient real-time calculation of optimization strategy calculations. The present invention also provides a grid line loss optimization system based on graph attention perception and reinforcement learning decision-making.

[0008] Technical solution: According to a first aspect of the present invention, a method for optimizing power line loss based on graph attention perception and reinforcement learning decision-making is provided, and the method comprises the following steps:

[0009] S1 collects real-time measurement and historical operation data of the power grid, and preprocesses and annotates the data; the real-time measurement and historical operation data include node voltage, load active power, load reactive power, line current, photovoltaic active power and reactive power, energy storage charging and discharging records, and line loss; the annotated division includes converting the preprocessed data into graph data containing node features, edge features, and topological structure;

[0010] S2 builds an intelligent perception model, which includes a global graph attention network, a one-dimensional convolutional neural network and a fully connected network, transmits input data to the global graph attention network to obtain the hidden state of each node, and then uses the one-dimensional convolutional neural network and the fully connected network for processing in turn, and finally outputs high-dimensional node features;

[0011] S3 uses a deep autoencoder to compress the high-dimensional node features into a low-dimensional representation;

[0012] S4 uses the training set to train the intelligent perception model and the deep autoencoder to obtain a trained intelligent perception model;

[0013] S5 builds an intelligent decision model, which includes two neural networks: an Actor network and a Critic network. The Actor network receives the data output by the trained intelligent perception model, and then obtains the action distribution of each node; the Critic network receives the data output by the trained intelligent perception model, and then the Critic network outputs a value function through a fully connected layer to evaluate the value of the current state and action;

[0014] S6 converts the line loss optimization problem of the new energy high penetration power grid into a Markov decision process, thereby training the intelligent decision model according to the obtained action distribution of each node and the current state and action value, and then obtaining the adaptive online loss reduction strategy of the new energy high penetration power grid.

[0015] Further, including:

[0016] In step S1, the graph data is represented as:

[0017] G = (V, E, X), where V is the node set, representing each node in the power grid; E is the edge set, representing the connection relationship between nodes; and X is the attribute matrix containing node features and edge features, including node voltage, load active power, load reactive power, line current, photovoltaic active power and reactive power, energy storage charging and discharging records, and line loss.

[0018] Further, including:

[0019] In step S2, constructing the intelligent perception model specifically includes:

[0020] The formula for calculating the hidden state of each node through the global graph attention network is as follows:

[0021]

[0022] Among them, h att,i is the feature representation of node i, α ij is the attention score that measures the importance of node j to node i, N(i) represents the set of neighboring nodes of node i, g j (x j ) is the embedding function of node j composed of a single-layer neural network, x j is the input feature of node j, σ is the activation function, and the attention score α ij Calculated by the following formula:

[0023]

[0024] Where, L 1 represents the number of layers of the global graph attention network, g i (x i ) is the embedding function of node i consisting of a single-layer neural network, represents the connection between two node embeddings, w is the weight vector, LeakyReLU is the activation function, e ij Represents the attention weight. In order to extract the common features of the entire system, a one-dimensional convolutional neural network is used after the graph attention network, which is expressed as:

[0025]

[0026] Where W k is the weight of the kth 1D convolutional neural network layer, b k represents the bias of the kth neural network layer, F represents the number of layers of the one-dimensional convolutional neural network, σ is the activation function, and h cov,k It is the output of the feature quantity after the convolution layer;

[0027] Then, the fully connected network is used to further embed the feature h cov,k , the final output layer is the representation of high-level node features h g .

[0028] Further, including:

[0029] The deep autoencoder is used to compress the high-dimensional node features into a low-dimensional representation, and the encoding process is as follows:

[0030] h e =f m […f 1 (h g )],(5)

[0031] In the formula, f m Represents the fully connected network of the mth layer in the deep autoencoder. Through the nonlinear mapping of the multi-layer fully connected network, the original high-dimensional feature h g is converted into a low-dimensional representation h e , that is, compressing from high-dimensional space to low-dimensional space.

[0032] Further, including:

[0033] The step S4 specifically includes:

[0034] S41 randomly initializes the parameters θ of the global graph attention network and deep autoencoder ggat and θ dae ;

[0035] S42 inputs the data marked and divided in step S1 into the global graph attention network;

[0036] S43 uses historical data and labels to perform supervised training on the global graph attention network, calculates the actual line loss of each line and the line loss predicted by the model, and uses the loss function Loss(θ ggat ) performs gradient descent, and the model parameters will be gradually adjusted during the optimization process so that it can accurately predict line losses;

[0037] S44 uses unsupervised learning to train a deep autoencoder and minimizes the reconstruction loss function Loss(θ dae) to optimize the parameters of the encoder, and finally, by optimizing the loss function, it is possible to extract low-dimensional representations from high-dimensional features; S45 loops through steps S42-S44 until the corresponding loss function gradually converges, thereby predicting line losses and adapting to the operation of a power grid with a high proportion of new energy access.

[0038] Further, including:

[0039] In step S43, the loss function Loss(θ ggat ) is expressed as:

[0040]

[0041] Among them, B is the sampling batch, is the actual line loss of the qth line, The line loss predicted by the model for the qth line;

[0042] Use gradient descent to update the parameters θ of the perception model ggat , optimize the model by minimizing the loss function:

[0043]

[0044] Among them, η is the learning rate.

[0045] Further, including:

[0046] The loss function Loss(θ dae ) is specifically expressed as:

[0047]

[0048] Among them, L 2 represents the number of layers of the deep autoencoder, h g The high-dimensional features generated by the global graph attention network, are the features reconstructed by the deep auto-decoder.

[0049] Further, including:

[0050] In step S5, the objective function of the intelligent decision-making model is expressed as:

[0051]

[0052] Among them, θ is the parameter of the policy network, r t (θ) is the ratio of the new and old strategies, which means the change of the current strategy relative to the old strategy. t is the advantage function, which is used to measure the performance of an action relative to the average performance. ∈ is a hyperparameter that controls the update amplitude of the policy to prevent the update from being too large. clip() is the truncation function. Express expectations.

[0053] Further, including:

[0054] In step S6, the line loss optimization problem of the new energy high penetration power grid is transformed into a Markov decision process, which is expressed as follows:

[0055] DRL agent: The dispatch decision center is regarded as an agent to control PV and BSS in the power grid, achieving intelligent decision-making for power grid loss reduction;

[0056] Environment: The environment in which the agent interacts is the intelligent perception model of step S2. The agent makes a decision. The intelligent perception model feeds back an optimized line loss value and voltage value according to the decision amount, and then calculates the reward value according to the line loss and voltage values.

[0057] State space: When using PV inverters and BSS to reduce line losses, define the state of each time step as s t =(m t ,A), where m t represents the observed characteristics of the power grid, including: node net active power and reactive power, voltage amplitude and current amplitude, A represents the adjacency matrix;

[0058] Action space: The action of the agent at each time step is a t =(a pv,t ,a bss,t ), where a pv,t represents the reactive power output of the PV inverter at time t, a bss,t represents the active output of BSS at time t;

[0059] Reward function: The reward function is defined as minimizing the power grid line loss. The line loss at time t is represented by p loss,t , reward r t =-p loss,t +η, where η is the voltage over-limit penalty value.

[0060] Further, including:

[0061] In step S6, the intelligent decision model training process includes:

[0062] S61 Model parameter loading: Load the parameters of the intelligent perception model trained in step S4, including the parameters θ of GGAT and DAE ggat and θ dae , randomly initialize the parameters of the Actor network and the Critic network;

[0063] S62 Initialize grid state observation: For each iteration, the initial state of the grid s 0It will be extracted from historical data or real-time data to represent the current grid node status and line information; each iteration includes multiple time steps, and the intelligent agent interacts with the perception model at each time step to generate an experience trajectory;

[0064] S63 Agent performs actions and calculates state transitions and reward values: The agent extracts the feature state s at time step t. t Generate action a through the Actor network t , the state of the power grid changes accordingly when this action is executed, and it transfers to the next state s t+1 ,According to the executed actions and state changes, the perception model calculates the line loss changes and voltage conditions caused by the current action, and generates the corresponding reward value. The reward value represents the impact of the current action on the line loss. The goal is to maximize the cumulative reward by minimizing the line loss;

[0065] S64 stores the experience track to the experience replay pool: the current state s t 、Action a t , Reward t and the new state s t+1 Stored in the experience replay pool for subsequent use in training models. If the replay pool is full, delete the earliest experience data to keep the capacity of the experience pool fixed.

[0066] S65 samples training data from the experience replay pool: When enough samples have accumulated in the experience replay pool, a small batch of data is randomly sampled from it, containing multiple state-action-reward-new state combinations.

[0067] S66 updates the Critic network and Actor network;

[0068] After multiple rounds of iterations of S67, the Actor network and the Critic network gradually converged, and the intelligent agent was able to make the best decision based on the current grid status, maximize the reduction of line losses, and implement an adaptive online loss reduction strategy.

[0069] Further, including:

[0070] In step S66, updating the Critic network specifically includes:

[0071] The Critic network uses data sampled from the experience pool to update its parameters; the goal of the Critic network is to minimize the estimation error of the state-action pair, and the update rule is as follows:

[0072]

[0073] In the formula, Loss criticis the loss function of the Critic network, which measures the gap between the actual state value and the estimated value, θ critic and η critic are the parameters and learning rate of the Critic network.

[0074] Further, including:

[0075] In step S66, updating the Actor network specifically includes:

[0076] The Actor network uses the advantage function A calculated from the Critic network t , the update rules are as follows:

[0077]

[0078] Among them, θ actor and η actor are the parameters and learning rate of the Actor network.

[0079] In a second aspect, the present invention also provides a power grid line loss optimization system based on graph attention perception and reinforcement learning decision-making, the system comprising:

[0080] A data preprocessing module is used to collect real-time measurement and historical operation data of the power grid, and to preprocess and annotate the data; the real-time measurement and historical operation data include node voltage, load active power, load reactive power, line current, photovoltaic active power and reactive power, energy storage charging and discharging records, and line loss; the annotating and partitioning includes converting the preprocessed data into graph data containing node features, edge features, and topological structure;

[0081] An intelligent perception model construction module is used to construct an intelligent perception model, which includes a global graph attention network, a one-dimensional convolutional neural network and a fully connected network. The input data is transmitted to the global graph attention network to obtain the hidden state of each node, and then the one-dimensional convolutional neural network and the fully connected network are used for processing in turn to finally output high-dimensional node features.

[0082] A feature dimensionality reduction module, used for compressing the high-dimensional node features into a low-dimensional representation using a deep autoencoder;

[0083] An intelligent perception model training module, used to train the intelligent perception model and the deep autoencoder using a training set to obtain a trained intelligent perception model;

[0084] An intelligent decision model construction module is used to construct an intelligent decision model, wherein the intelligent decision model includes two neural networks: an Actor network and a Critic network. The Actor network receives the data output by the trained intelligent perception model, and then obtains the action distribution of each node; the Critic network receives the data output by the trained intelligent perception model, and then the Critic network outputs a value function through a fully connected layer to evaluate the value of the current state and action;

[0085] The intelligent decision-making model training module is used to convert the line loss optimization problem of the renewable energy high-penetration power grid into a Markov decision process, so as to train the intelligent decision-making model according to the obtained action distribution of each node and the current state and action value, and then obtain the adaptive online loss reduction strategy of the renewable energy high-penetration power grid.

[0086] In a third aspect, the present invention also provides a computer-readable storage medium having computer instructions stored thereon, which, when the computer instructions are executed, execute the above-mentioned grid line loss optimization based on graph attention perception and reinforcement learning decision-making.

[0087] In a fourth aspect, the present invention also provides a risk assessment device based on task indicators, characterized in that the risk assessment device comprises: a memory, a processor, and a power grid line loss optimization program based on graph attention perception and reinforcement learning decision-making stored in the memory and executable on the processor, wherein the power grid line loss optimization program based on graph attention perception and reinforcement learning decision-making implements the steps of the power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making as described above when executed by the processor.

[0088] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0089] (1) Efficient and accurate intelligent perception of power grid line loss: The present invention uses a line loss intelligent perception model that combines the global graph attention network (GGAT) with the deep autoencoder (DAE) to accurately perceive the line loss of each line in the power grid. The model uses the topological structure and real-time operation data of the power grid to accurately predict line loss in real time, enhance the transparency of power grid operation, and provide key basic data support for subsequent optimization decisions.

[0090] (2) Adaptive line loss optimization strategy: This paper introduces PPO and combines it with an intelligent perception model to build an intelligent decision-making model for a new energy high-penetration power grid. The model continuously interacts with the power grid environment and learns how to adaptively adjust the operation strategies of distributed energy such as photovoltaic power generation and energy storage systems in the power grid according to different operating states, thereby minimizing line losses. Compared with traditional physical model-based optimization methods, this model does not rely on precise physical parameters and has higher flexibility and adaptability.

[0091] (3) Improvement of grid stability under new energy access: In response to the fluctuations in grid operation caused by the high proportion of new energy access, the present invention can realize automatic adjustment and optimization of the grid through a reinforcement learning intelligent decision-making model, reduce line losses and enhance the stability of the grid. The model can handle the power fluctuations caused by the access of new energy, and through adaptive adjustment of the output strategy of distributed energy, ensure that the grid can still operate efficiently and stably in a complex dynamic environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0093] Figure 1 A flow chart of a method for optimizing power grid line losses based on graph attention perception and reinforcement learning decision-making according to an embodiment of the present invention;

[0094] Figure 2 This is a framework diagram of a power grid intelligent perception model based on GGAT and DAE according to an embodiment of the present invention;

[0095] Figure 3 This is a flow chart of the GGAT-PPO method described in an embodiment of the present invention;

[0096] Figure 4 A schematic diagram of the structure of a power grid line loss optimization system based on graph attention perception and reinforcement learning decision-making according to an embodiment of the present invention;

[0097] Figure 5 A schematic diagram of the terminal hardware structure of each embodiment of the power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making described in an embodiment of the present invention. DETAILED DESCRIPTION

[0098] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0099] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0100] Example 1

[0101] Since the idea of ​​the reinforcement learning decision-making solution is that the intelligent agent and the environment interact continuously and learn the best strategy from historical experience, the uncertainty of the power of new sources and loads can be captured from the data and solved online in real time. However, the current reinforcement learning methods all assume that the intelligent agent interacts with the simulation environment built on the software, which is separated from the perception of the actual power grid environment, and it is impossible to train interactive intelligent agents in the actual power grid environment.

[0102] Therefore, the present invention innovatively proposes a line loss intelligent perception model, which uses actual historical data to train the line loss perception model for reinforcement learning agent interaction, and then proposes a new reinforcement learning algorithm to train the loss reduction intelligent decision-making model, thereby realizing the perception of real grid power and flow uncertainty and intelligent loss reduction decision-making.

[0103] Specifically, this example discloses a method for optimizing power grid line losses based on graph attention perception and reinforcement learning decision-making, which includes the following steps:

[0104] S1 collects real-time measurement and historical operation data of the power grid, and preprocesses and annotates the data; the real-time measurement and historical operation data include node voltage, load active power, load reactive power, line current, photovoltaic active power and reactive power, energy storage charging and discharging records, and line loss; the annotated division includes converting the preprocessed data into graph data containing node features, edge features, and topological structure;

[0105] S2 builds an intelligent perception model, which includes a global graph attention network, a one-dimensional convolutional neural network and a fully connected network, transmits input data to the global graph attention network to obtain the hidden state of each node, and then uses the one-dimensional convolutional neural network and the fully connected network for processing in turn, and finally outputs high-dimensional node features;

[0106] S3 uses a deep autoencoder to compress the high-dimensional node features into a low-dimensional representation;

[0107] S4 uses the training set to train the intelligent perception model and the deep autoencoder to obtain a trained intelligent perception model;

[0108] S5 builds an intelligent decision model, which includes two neural networks: an Actor network and a Critic network. The Actor network receives the data output by the trained intelligent perception model, and then obtains the action distribution of each node; the Critic network receives the data output by the trained intelligent perception model, and then the Critic network outputs a value function through a fully connected layer to evaluate the value of the current state and action;

[0109] S6 converts the line loss optimization problem of the new energy high penetration power grid into a Markov decision process, thereby training the intelligent decision model according to the obtained action distribution of each node and the current state and action value, and then obtaining the adaptive online loss reduction strategy of the new energy high penetration power grid.

[0110] A specific implementation method in this embodiment includes:

[0111] Step 1: Collect real-time measurement and historical operation data of the power grid, and pre-process and mark the data;

[0112] This application collects data including node voltage, load active power and load reactive power, line current, photovoltaic active power and reactive power, energy storage charging and discharging records, line loss and grid topology data; after collecting a large amount of data, due to sensor failure, data transmission errors or other reasons that may cause data anomalies during the data collection process, the data is first cleaned and gaps are filled, and the data is standardized.

[0113] Then, the data is labeled and the input data is converted into graph data containing node features, edge features and topological structure. The power grid is represented as a graph G = (V, E, X), where V is a node set representing each node in the power grid, E is an edge set representing the connection relationship between nodes, and X is an attribute matrix containing node features and edge features, including node voltage, power, line current, loss and other information. This graph structure not only reflects the physical topological relationship of the power grid, but also contains real-time electrical parameters, which helps the model capture global information in the power grid.

[0114] Finally, the line loss data is annotated with the labels of the supervised learning model, and the dataset is divided into a training set and a test set with a ratio of 80% training set and 20% test set. The training set is used to train the perception model, and the test set is used to evaluate the final performance of the model.

[0115] Step 2: Build an intelligent perception model of power line loss in high-penetration renewable energy 2 grid based on improved deep neural network;

[0116] Improved deep neural network frameworks such as Figure 2 As shown in the figure, the global graph attention network GGAT is the core part of the model. It uses the graph attention mechanism to capture the complex relationship between nodes, including the mutual influence between electrical states such as voltage and power and topological structures. In GGAT, the feature vector of each node not only depends on its own state information, but also updates it in combination with the information of neighboring nodes. Through the attention mechanism, the model can adaptively learn the degree of influence of neighboring nodes on the target node.

[0117] The formula for pushing the hidden state of each node through the global graph attention network GGAT is as follows:

[0118]

[0119] In the formula, h att,i is the feature representation of node i, α ij is the attention score that measures the importance of node j to node i, N(i) represents the set of neighboring nodes of node i, g j (x j ) is the embedding function of node j composed of a single-layer neural network, x j is the input feature of node j, σ is the activation function, and the attention score α ij It can be calculated by the following formula:

[0120]

[0121] In the formula, represents the connection between two node embeddings, w is the weight vector, LeakyReLU is the activation function, e ij Represents the attention weight. In order to extract the common features of the entire system, a one-dimensional convolutional neural network CNN is used after the graph attention network:

[0122]

[0123] Where W i is the weight of the i-th 1D convolutional neural network layer, b i represents the bias of the i-th neural network layer, F represents the number of layers of the one-dimensional convolutional neural network, σ is the activation function, and h cov,i It is the output of the feature quantity after the convolution layer. Then the fully connected network FCN is used to further embed the features, and the final output layer is the representation of the high-level node features h g .

[0124] After the GGAT module processes the real-time and pseudo-measurement data, the model generates a high-dimensional feature representation h g In order to reduce the computational burden and keep important feature information, it is necessary to reduce the dimension, so a deep autoencoder DAE is proposed. DAE consists of two parts: encoder and decoder. First, the encoder converts the high-dimensional feature h g Compressed into a low-dimensional representation h e , the encoding process is as follows:

[0125] h e =f l […f 1 (h g )],(4)

[0126] In the formula, f l represents the lth layer FCN in the encoder. Through the nonlinear mapping of multiple layers of FCN, the original feature h g is converted into a low-dimensional representation h e , that is, compressing from high-dimensional space to low-dimensional space.

[0127] The main challenges faced by GGAT in high-dimensional data processing in the existing technology are: redundancy and noise of high-dimensional feature data and computational complexity.

[0128] In the combination of GGAT, DAE and CNN, this paper proposes a new processing method to deal with the complex spatiotemporal data characteristics in the power grid with high penetration of new energy. The specific improvements and innovations are as follows:

[0129] Hierarchical feature processing: GGAT is used to first capture the topological relationship of the power grid and the mutual influence between nodes, CNN is used to extract the global mode of the system, and DAE is used to reduce the dimension, compress and filter the noise. This hierarchical processing can optimize the expression and processing of features in stages, ensuring that the output of each layer of the network makes a key contribution to the final optimization goal (such as minimizing line loss). This solution is more efficient and targeted than the application of a single model.

[0130] Systematic modeling that combines local and global features: GGAT is used to model local topological relationships, CNN is used to identify the global features of the system, and DAE further compresses and enhances the robustness of information. At the same time, for the DAE part, we only retain the encoding part of DAE and remove the decoding part. This design simplifies the computational complexity of the model.

[0131] Multi-layer network collaborative optimization: This innovative combination not only uses advanced neural network models in each layer of the network, but also ensures collaborative optimization between different network layers. The high-dimensional features output by the GGAT layer are processed by CNN and DAE, which reduces information redundancy and computational complexity, and ensures that the final low-dimensional features are input into the decision network with high efficiency and reliability.

[0132] Step 3: Considering the combined training methods of supervised and unsupervised training, the line loss intelligent perception model in step 2 is trained through historical data to achieve intelligent online perception of line loss in the power grid with a high proportion of new energy access;

[0133] The training process of the model is as follows:

[0134] Step 3.1: Parameter initialization of the intelligent perception model. Before training the intelligent perception model, the parameters θ of GGAT and DAE are randomly initialized. ggat and θ dae , and use the Xavier initialization method to avoid the problem of gradient disappearance or explosion;

[0135] Step 3.2: The grid node features, edge features and topological structures collected and preprocessed in step 1 are represented in the form of a graph and input into the global graph attention network;

[0136] Step 3.3: Use historical data and labels to perform supervised training on the CNN-based GGAT network, calculate the actual line loss of each line and the line loss predicted by the model, and use the loss function Loss(θ ggat ) performs gradient descent;

[0137] In step 3.3, the loss function Loss(θ ggat ) is calculated as follows:

[0138]

[0139] In the formula, B is the sampling batch, is the actual line loss of the ith line, is the line loss predicted by the model.

[0140] Use gradient descent to update the parameters θ of the perception model ggat , and optimize the model by minimizing the loss function:

[0141]

[0142] Where η is the learning rate. This optimization process will gradually adjust the model parameters so that it can accurately predict line losses.

[0143] Step 3.4: Train the DAE parameters θ in an unsupervised manner dae;

[0144] After the initial training of GGAT, DAE is trained using unsupervised learning to reduce the dimensionality of high-dimensional features while ensuring information preservation. DAE optimizes the parameters of the encoder by minimizing the reconstruction error, the reconstruction loss function Loss(θ dae ) is calculated as follows:

[0145]

[0146] In the formula, h g High-dimensional features generated by GGAT, The features reconstructed by the DAE decoder are finally optimized, and the model can extract more robust low-dimensional representations from high-dimensional features.

[0147] Step 3.5: Optimize and evaluate the model convergence effect;

[0148] After multiple rounds of training, the intelligent perception model gradually converged and was able to accurately predict line losses and adapt to the operation of power grids with a high proportion of new energy access. The trained perception model was verified using a test set to evaluate its ability to perceive power grid line losses online, and the model's parameters and structure were further optimized to ensure its effectiveness and robustness in practical applications.

[0149] In this application, supervised learning is used for the line loss prediction task in step 3.3, and the model is trained with known labels in historical data, i.e., actual line loss values. This approach can directly optimize the prediction ability of the model, enabling it to accurately predict the line loss of each line, thereby helping power grid operators make better decisions in daily operations.

[0150] Unsupervised learning is used to train the deep autoencoder DAE in step 3.4, with the goal of reducing the redundant information of the input data through dimensionality reduction and extracting representative low-dimensional features. In this case, there is no need to rely on labeled data, and you can focus on improving the feature representation ability of the model to ensure that the features can still be effectively compressed and retained when the data set is large and the information is complex.

[0151] This joint training method improves the generalization ability and stability of the model; reduces the dependence on labeled data through feature compression and denoising; reduces the complexity of the model and reduces the risk of overfitting.

[0152] Step 4: Construct an intelligent decision-making model for loss reduction in new energy high-penetration power grid based on the global graph attention proximal strategy optimization algorithm;

[0153] This paper proposes an intelligent decision model of the global graph attention proximal strategy optimization algorithm (GGAT-PPO). The algorithm framework of the GGAT-PPO model is as follows: Figure 3 As shown;

[0154] The GGAT-PPO algorithm is optimized by defining an objective function. The core goal is to ensure that the new strategy does not deviate too far from the old strategy through strategy cutting and restriction. The objective function of GGAT-PPO is:

[0155]

[0156] In the formula, θ is the parameter of the GGAT-PPO algorithm policy network, r t (θ) is the ratio of the new and old strategies, indicating the magnitude of change of the current strategy relative to the old strategy. t is an advantage function, which is used to measure the performance of an action relative to the average performance, and ∈ is a hyperparameter that controls the update amplitude of the policy to prevent the update from being too large.

[0157] The GGAT-PPO algorithm consists of two neural networks: the Actor network, the Critic network, and the feature extraction network. The Actor network is responsible for generating strategies, that is, selecting corresponding actions given the state of the power grid (such as adjusting photovoltaic power generation, charging and discharging of energy storage systems, etc.), and the Critic network is responsible for evaluating the value of the current state-action pair to assist the Actor network in updating its strategies. The feature extraction network is the GGAT network and DAE network in step 2, which aims to enhance the DRL agent's perception of power grid data. The features output by the deep autoencoder DAE are passed into the fully connected layer of the Actor to generate the final strategy distribution. The output of the Actor network is the action distribution π of each node. θ (s|a). The features output by the deep autoencoder DAE are passed into the fully connected layer of the Critic network to extract the global feature h g , the Critic network outputs the value function V(s) through the fully connected layer to evaluate the value of the current state s and action a. The PPO algorithm improved by GGAT is the GGAT-PPO algorithm.

[0158] Step 5: Convert the line loss optimization problem of the renewable energy high penetration power grid into a Markov decision process. Use the historical operation data and the line loss perception model trained in step 3 to train the loss reduction intelligent decision model in step 4 to obtain the adaptive online loss reduction strategy of the power grid.

[0159] The Markov decision process (MDP) for the line loss optimization problem is expressed as follows:

[0160] DRL agent: The dispatch decision center is regarded as an agent to control PV and BSS in the power grid, achieving intelligent decision-making for power grid loss reduction;

[0161] Environment: The environment in which the agent interacts is the intelligent perception model in step 3. The agent makes a decision, and the perception model feeds back an optimized line loss value and voltage value based on the decision amount, and then calculates the reward value based on the line loss and voltage values.

[0162] State space: When using PV inverters and BSS to reduce line losses, define the state of each time step as s t =(m t ,A), where m t It represents the observed characteristics of the power grid, including: node net active power and reactive power, voltage amplitude and current amplitude, and A represents the adjacency matrix.

[0163] Action space: The action of the agent at each time step is a t =(a pv,t ,a bss,t ), where a pv,t represents the reactive power output of the PV inverter at time t, a bss,t represents the active output of BSS at time t,

[0164] Reward function: The reward function is defined as minimizing the power grid line loss. The line loss at time t is represented by p loss,t , reward r t =-p loss,t +η, where η is the voltage over-limit penalty value.

[0165] The training process of the GGAT-PPO algorithm is as follows:

[0166] (1) GGAT-PPO model parameter loading: Load the parameters of the intelligent perception model trained in step 3, including the parameters θ of GGAT and DAE ggat and θ dae , randomly initialize the parameters of the control model (Actor and Critic networks) in the PPO algorithm.

[0167] (2) Initialization of grid state observation: For each iteration (episode), the initial state of the grid s 0 It will be extracted from historical data or real-time data to represent the current grid node status and line information; each iteration includes multiple time steps, and the intelligent agent interacts with the perception model at each time step to generate an experience trajectory;

[0168] (3) The agent performs actions and calculates state transitions and reward values: The agent extracts the feature state s at time step t. t Generate action a through the Actor network t , the state of the power grid changes accordingly when this action is executed, and it transfers to the next state s t+1,According to the executed actions and state changes, the perception model calculates the line loss changes and voltage conditions caused by the current action, and generates the corresponding reward value. The reward value represents the impact of the current action on the line loss. The goal is to maximize the cumulative reward by minimizing the line loss;

[0169] (4) Store the experience trajectory to the experience replay pool: store the current state s t 、Action a t , Reward t and the new state s t+1 Stored in the experience replay buffer for subsequent use in training the model. If the replay pool is full, delete the earliest experience data to keep the capacity of the experience pool fixed.

[0170] (5) Sampling training data from the experience replay pool: When enough samples have accumulated in the experience replay pool, a mini-batch of data is randomly sampled from it, which contains multiple state-action-reward-new state combinations.

[0171] (6) Update the Critic network: The Critic network uses data sampled from the experience pool to update its parameters. The goal of the Critic network is to minimize the estimation error of the state-action pair. The update rule is as follows:

[0172]

[0173] In the formula, Loss critic is the loss function of the Critic network, which measures the gap between the actual state value and the estimated value, θ critic and η critic are the parameters and learning rate of the Critic network;

[0174] (7) Update the Actor network: The Actor network uses the advantage function A calculated from the Critic network t , optimized through the objective function of PPO, the update rules are as follows:

[0175]

[0176] Where, L PPO is the objective function of PPO, which ensures the stability of strategy update by limiting the amplitude of strategy update, θ actor and η actor are the parameters and learning rate of the Actor network;

[0177] After multiple rounds of iterations, the Actor and Critic networks gradually converge, and the intelligent agent can make the best decision based on the current grid status, maximize the reduction of line losses, and implement an adaptive online loss reduction strategy.

[0178] The traditional PPO algorithm is mainly used for standard state-action space optimization. However, in complex power grid optimization problems, the state of the power grid has obvious spatiotemporal characteristics. The state of the power grid not only depends on the voltage or power information of a single node, but also involves the complex topological structure and interaction relationship between the nodes in the power grid. Therefore, the traditional PPO algorithm cannot make full use of these graph structure features, resulting in low efficiency in dealing with power grid problems and limited optimization effect.

[0179] Therefore, based on the prior art, the present invention makes the following improvements:

[0180] 1. The present invention embeds the global graph attention network (GGAT) into the Actor network and Critic network of the PPO algorithm to process the graph structure characteristics of the power grid. GGAT can capture the complex interaction between different nodes in the power grid, and generate high-dimensional feature representations of each node through the graph attention mechanism, which greatly enhances the state perception ability of the PPO algorithm in the power grid environment.

[0181] 2. Specific improvements to the Actor and Critic networks. The present invention inputs the high-dimensional graph feature representation extracted by GGAT into the Actor network and the Critic network, which enables the Actor network and the Critic network to fully utilize the global graph information of the power grid when generating strategies.

[0182] 3. The present invention combines a deep autoencoder (DAE) to perform dimensionality reduction and noise filtering on the high-dimensional features extracted by GGAT, thereby generating a more compact low-dimensional representation.

[0183] In summary, in a power grid with high penetration of new energy, the interaction between nodes is complex, and multiple factors such as voltage, power, and load affect line loss at the same time. This scenario is significantly different from the traditional PPO application scenario. Therefore, the present invention introduces the global graph attention network GGAT in the Actor and Critic networks of the PPO algorithm, and combines the deep autoencoder DAE for feature dimensionality reduction, which significantly improves the performance of the PPO algorithm in line loss optimization of power grids with high penetration of new energy. The technical improvement in this specific scenario not only enhances the adaptability of the PPO algorithm in complex power grid environments, but also provides an efficient and intelligent power grid optimization solution through innovative feature processing and strategy update mechanisms.

[0184] Example 2

[0185] like Figure 4 As shown, the present invention also provides a power grid line loss optimization system based on graph attention perception and reinforcement learning decision-making, the system comprising:

[0186] The data preprocessing module is used to collect real-time measurement and historical operation data of the power grid, and preprocess and mark the data; the real-time measurement and historical operation data include node voltage, load active power, load reactive power, line current, photovoltaic active power and reactive power, energy storage charging and discharging records and line loss. The marking and division includes converting the preprocessed data into graph data containing node features, edge features and topological structure.

[0187] Specifically, the application collects data including node voltage, load active power and load reactive power, line current, photovoltaic active power and reactive power, energy storage charging and discharging records, line losses and grid topology data; after collecting a large amount of data, due to data anomalies caused by sensor failure, data transmission errors or other reasons during the data collection process, the data is first cleaned and gaps are filled, and the data is standardized.

[0188] Then, the data is labeled and the input data is converted into graph data containing node features, edge features and topological structure. The power grid is represented as a graph G = (V, E, X), where V is a node set representing each node in the power grid, E is an edge set representing the connection relationship between nodes, and X is an attribute matrix containing node features and edge features, including node voltage, power, line current, loss and other information. This graph structure not only reflects the physical topological relationship of the power grid, but also contains real-time electrical parameters, which helps the model capture global information in the power grid.

[0189] Finally, the line loss data is annotated with the labels of the supervised learning model, and the dataset is divided into a training set and a test set with a ratio of 80% training set and 20% test set. The training set is used to train the perception model, and the test set is used to evaluate the final performance of the model.

[0190] The intelligent perception model construction module is used to construct an intelligent perception model, which includes a global graph attention network, a one-dimensional convolutional neural network and a fully connected network. The input data is transmitted to the global graph attention network to obtain the hidden state of each node, and then the one-dimensional convolutional neural network and the fully connected network are used for processing in turn to finally output high-dimensional node features.

[0191] In this embodiment, the global graph attention network GGAT is the core part of the model, which captures the complex relationship between nodes through the graph attention mechanism, including the mutual influence between electrical states such as voltage and power and topological structures. In GGAT, the feature vector of each node not only depends on its own state information, but also updates it in combination with the information of neighboring nodes. Through the attention mechanism, the model can adaptively learn the degree of influence of neighboring nodes on the target node.

[0192] The formula for pushing the hidden state of each node through the global graph attention network GGAT is as follows:

[0193]

[0194] In the formula, h att,i is the feature representation of node i, α ij is the attention score that measures the importance of node j to node i, N(i) represents the set of neighboring nodes of node i, g j (x j ) is the embedding function of node j composed of a single-layer neural network, x j is the input feature of node j, σ is the activation function, and the attention score α ij It can be calculated by the following formula:

[0195]

[0196] In the formula, represents the connection between two node embeddings, w is the weight vector, LeakyReLU is the activation function, e ij Represents the attention weight. In order to extract the common features of the entire system, a one-dimensional convolutional neural network CNN is used after the graph attention network:

[0197]

[0198] Where W i is the weight of the ith neural network layer, b i represents the bias of the ith neural network layer, F represents the number of neural network layers, σ is the activation function, and h cov,i It is the output of the feature quantity after the convolution layer. Then the fully connected network FCN is used to further embed the features, and the final output layer is the representation of the high-level node features h g .

[0199] The feature dimension reduction module is used to compress the high-dimensional node features into a low-dimensional representation using a deep autoencoder.

[0200] After the GGAT module processes the real-time and pseudo-measurement data, the model generates a high-dimensional feature representation h g In order to reduce the computational burden and keep important feature information, it is necessary to reduce the dimension, so a deep autoencoder DAE is proposed. DAE consists of two parts: encoder and decoder. First, the encoder converts the high-dimensional feature h g Compressed into a low-dimensional representation h e , the encoding process is as follows:

[0201] h e =f l […f 1 (hg )],

[0202] In the formula, f l represents the lth layer FCN in the encoder. Through the nonlinear mapping of multiple layers of FCN, the original feature h g is converted into a low-dimensional representation h e , that is, compressing from high-dimensional space to low-dimensional space.

[0203] The intelligent perception model training module is used to train the intelligent perception model and the deep automatic encoder using a training set to obtain a trained intelligent perception model.

[0204] In this embodiment, the training process of the intelligent perception model is as follows:

[0205] Parameter initialization of the intelligent perception model. Before training the intelligent perception model, the parameters θ of GGAT and DAE are randomly initialized. ggat and θ dae , and use the Xavier initialization method to avoid the problem of gradient disappearance or explosion;

[0206] The collected and preprocessed grid node features, edge features and topological structures are represented in the form of graphs and input into the global graph attention network;

[0207] The CNN-based GGAT network is supervised and trained using historical data and labels to calculate the actual line loss of each line and the line loss predicted by the model. ggat ) performs gradient descent;

[0208] Among them, the loss function Loss(θ ggat ) is calculated as follows:

[0209]

[0210] In the formula, B is the sampling batch, is the actual line loss of the ith line, is the line loss predicted by the model.

[0211] Use gradient descent to update the parameters θ of the perception model ggat , and optimize the model by minimizing the loss function:

[0212]

[0213] Where η is the learning rate. This optimization process will gradually adjust the model parameters so that it can accurately predict line losses.

[0214] The DAE parameters θ are trained in an unsupervised manner. dae ;

[0215] After the initial training of GGAT, DAE is trained using unsupervised learning to reduce the dimensionality of high-dimensional features while ensuring information preservation. DAE optimizes the parameters of the encoder by minimizing the reconstruction error, the reconstruction loss function Loss(θ dae ) is calculated as follows:

[0216]

[0217] In the formula, h g High-dimensional features generated by GGAT, The features reconstructed by the DAE decoder are finally optimized, and the model can extract more robust low-dimensional representations from high-dimensional features.

[0218] Optimization and evaluation of model convergence effect;

[0219] After multiple rounds of training, the intelligent perception model gradually converged and was able to accurately predict line losses and adapt to the operation of power grids with a high proportion of new energy access. The trained perception model was verified using a test set to evaluate its ability to perceive power grid line losses online, and the model's parameters and structure were further optimized to ensure its effectiveness and robustness in practical applications.

[0220] The intelligent decision model construction module is used to construct an intelligent decision model, which includes two neural networks: an Actor network and a Critic network. The Actor network receives the data output by the trained intelligent perception model to obtain the action distribution of each node; the Critic network receives the data output by the trained intelligent perception model, and then the Critic network outputs a value function through a fully connected layer to evaluate the value of the current state and action.

[0221] The intelligent decision model is based on the global graph attention proximal policy optimization algorithm GGAT-PPO. The GGAT-PPO algorithm is optimized by defining an objective function. The core goal is to ensure that the new strategy does not deviate too far from the old strategy through policy cutting and restriction. The objective function of GGAT-PPO is:

[0222]

[0223] In the formula, θ is the parameter of the GGAT-PPO algorithm policy network, r t (θ) is the ratio of the new and old strategies, indicating the magnitude of change of the current strategy relative to the old strategy. t is an advantage function, which is used to measure the performance of an action relative to the average performance, and ∈ is a hyperparameter that controls the update amplitude of the policy to prevent the update from being too large.

[0224] The GGAT-PPO algorithm consists of two neural networks: the Actor network, the Critic network, and the feature extraction network. The Actor network is responsible for generating strategies, that is, selecting corresponding actions given the state of the power grid (such as adjusting photovoltaic power generation, charging and discharging of energy storage systems, etc.), and the Critic network is responsible for evaluating the value of the current state-action pair to assist the Actor network in updating its strategies. The feature extraction network is the GGAT network and DAE network in step 2, which aims to enhance the DRL agent's perception of power grid data. The features output by the deep autoencoder DAE are passed into the fully connected layer of the Actor to generate the final strategy distribution. The output of the Actor network is the action distribution π of each node. θ (s|a). The features output by the deep autoencoder DAE are passed into the fully connected layer of the Critic network to extract the global feature h g , the Critic network outputs the value function V(s) through the fully connected layer to evaluate the value of the current state s and action a. The PPO algorithm improved by GGAT is the GGAT-PPO algorithm.

[0225] The intelligent decision-making model training module is used to convert the line loss optimization problem of the renewable energy high-penetration power grid into a Markov decision process, so as to train the intelligent decision-making model according to the obtained action distribution of each node and the current state and action value, and then obtain the adaptive online loss reduction strategy of the renewable energy high-penetration power grid.

[0226] The Markov decision process (MDP) for the line loss optimization problem is expressed as follows:

[0227] DRL agent: The dispatch decision center is regarded as an agent to control PV and BSS in the power grid, achieving intelligent decision-making for power grid loss reduction;

[0228] Environment: The environment in which the agent interacts is the intelligent perception model. The agent makes a decision, and the perception model feeds back an optimized line loss value and voltage value based on the decision amount, and then calculates the reward value based on the line loss and voltage values.

[0229] State space: When using PV inverters and BSS to reduce line losses, define the state of each time step as s t =(m t ,A), where m t It represents the observed characteristics of the power grid, including: node net active power and reactive power, voltage amplitude and current amplitude, and A represents the adjacency matrix.

[0230] Action space: The action of the agent at each time step is a t =(a pv,t ,a bss,t ), where apv,t represents the reactive power output of the PV inverter at time t, a bss,t represents the active output of BSS at time t,

[0231] Reward function: The reward function is defined as minimizing the power grid line loss. The line loss at time t is represented by p loss,t , reward r t =-p loss,t +η, where η is the voltage over-limit penalty value.

[0232] The training process of the GGAT-PPO algorithm is as follows:

[0233] (1) GGAT-PPO model parameter loading: Load the parameters of the trained intelligent perception model, including the parameters θ of GGAT and DAE ggat and θ dae , randomly initialize the parameters of the control model (Actor and Critic networks) in the PPO algorithm.

[0234] (2) Initialization of grid state observation: For each iteration (episode), the initial state of the grid s 0 It will be extracted from historical data or real-time data to represent the current grid node status and line information; each iteration includes multiple time steps, and the intelligent agent interacts with the perception model at each time step to generate an experience trajectory;

[0235] (3) The agent performs actions and calculates state transitions and reward values: The agent extracts the feature state s at time step t. t Generate action a through the Actor network t , the state of the power grid changes accordingly when this action is executed, and it transfers to the next state s t+1 ,According to the executed actions and state changes, the perception model calculates the line loss changes and voltage conditions caused by the current action, and generates the corresponding reward value. The reward value represents the impact of the current action on the line loss. The goal is to maximize the cumulative reward by minimizing the line loss;

[0236] (4) Store the experience trajectory to the experience replay pool: store the current state s t 、Action a t , Reward t and the new state s t+1 Stored in the experience replay buffer for subsequent use in training the model. If the replay pool is full, delete the earliest experience data to keep the capacity of the experience pool fixed.

[0237] (5) Sampling training data from the experience replay pool: When enough samples have accumulated in the experience replay pool, a mini-batch of data is randomly sampled from it, which contains multiple state-action-reward-new state combinations.

[0238] (6) Update the Critic network: The Critic network uses data sampled from the experience pool to update its parameters. The goal of the Critic network is to minimize the estimation error of the state-action pair. The update rule is as follows:

[0239]

[0240] In the formula, Loss critic is the loss function of the Critic network, which measures the gap between the actual state value and the estimated value, θ critic and η critic are the parameters and learning rate of the Critic network;

[0241] (7) Update the Actor network: The Actor network uses the advantage function A calculated from the Critic network t , optimized through the objective function of PPO, the update rules are as follows:

[0242]

[0243] Where, L PPO is the objective function of PPO, which ensures the stability of strategy update by limiting the amplitude of strategy update, θ actor and η actor are the parameters and learning rate of the Actor network;

[0244] After multiple rounds of iterations, the Actor and Critic networks gradually converge, and the intelligent agent can make the best decision based on the current grid status, maximize the reduction of line losses, and implement an adaptive online loss reduction strategy.

[0245] Example 3

[0246] Reference Figure 5 , Figure 5 The figure is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present invention.

[0247] like Figure 5As shown, the terminal may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a network interface 1003, and a memory 1004. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The network interface 1003 may optionally include a standard wired interface, a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1004 may be a high-speed RAM memory (Random Access Memory, RAM) or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1004 may optionally be a storage device independent of the aforementioned processor 1001.

[0248] Those skilled in the art will understand that Figure 5 The terminal structure shown in the figure does not constitute a limitation on the terminal, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0249] like Figure 5 As shown, the memory 1004 as a computer storage medium may include an operating system, a data storage module, a network communication module, and a power grid line loss optimization program based on graph attention perception and reinforcement learning decision-making.

[0250] exist Figure 5 In the terminal shown, the network interface 1003 is mainly used to connect to the backend server and perform data communication with the backend server; the processor 1001 can call the power grid line loss optimization program based on graph attention perception and reinforcement learning decision-making stored in the memory 1004, and perform the following operations:

[0251] S1 collects real-time measurement and historical operation data of the power grid, and preprocesses and annotates the data; the real-time measurement and historical operation data include node voltage, load active power, load reactive power, line current, photovoltaic active power and reactive power, energy storage charging and discharging records, and line loss; the annotated division includes converting the preprocessed data into graph data containing node features, edge features, and topological structure;

[0252] S2 builds an intelligent perception model, which includes a global graph attention network, a one-dimensional convolutional neural network and a fully connected network, transmits input data to the global graph attention network to obtain the hidden state of each node, and then uses the one-dimensional convolutional neural network and the fully connected network for processing in turn, and finally outputs high-dimensional node features;

[0253] S3 uses a deep autoencoder to compress the high-dimensional node features into a low-dimensional representation;

[0254] S4 uses the training set to train the intelligent perception model and the deep autoencoder to obtain a trained intelligent perception model;

[0255] S5 builds an intelligent decision model, which includes two neural networks: an Actor network and a Critic network. The Actor network receives the data output by the trained intelligent perception model, and then obtains the action distribution of each node; the Critic network receives the data output by the trained intelligent perception model, and then the Critic network outputs a value function through a fully connected layer to evaluate the value of the current state and action;

[0256] S6 converts the line loss optimization problem of the new energy high penetration power grid into a Markov decision process, thereby training the intelligent decision model according to the obtained action distribution of each node and the current state and action value, and then obtaining the adaptive online loss reduction strategy of the new energy high penetration power grid.

[0257] Further, the processor 1001 may call the power grid line loss optimization program based on graph attention perception and reinforcement learning decision-making stored in the memory 1004, and further perform the following operations:

[0258] In step S1, the graph data is represented as:

[0259] G = (V, E, X), where V is the node set, representing each node in the power grid; E is the edge set, representing the connection relationship between nodes; and X is the attribute matrix containing node features and edge features, including node voltage, load active power, load reactive power, line current, photovoltaic active power and reactive power, energy storage charging and discharging records, and line loss.

[0260] Further, the processor 1001 may call the power grid line loss optimization program based on graph attention perception and reinforcement learning decision-making stored in the memory 1004, and further perform the following operations:

[0261] In step S2, constructing the intelligent perception model specifically includes:

[0262] The formula for calculating the hidden state of each node through the global graph attention network is as follows:

[0263]

[0264] Among them, h att,i is the feature representation of node i, α ijis the attention score that measures the importance of node j to node i, N(i) represents the set of neighboring nodes of node i, g j (x j ) is the embedding function of node j composed of a single-layer neural network, x j is the input feature of node j, σ is the activation function, and the attention score α ij Calculated by the following formula:

[0265]

[0266] Where L represents the number of layers of the global graph attention network, g i (x i ) is the embedding function of node i consisting of a single-layer neural network, represents the connection between two node embeddings, w is the weight vector, LeakyReLU is the activation function, e ij Represents the attention weight. In order to extract the common features of the entire system, a one-dimensional convolutional neural network is used after the graph attention network, which is expressed as:

[0267]

[0268] Where W i is the weight of the i-th 1D convolutional neural network layer, b i represents the bias of the i-th one-dimensional convolutional neural network layer, F represents the number of layers of the one-dimensional convolutional neural network, σ is the activation function, and h cov,i It is the output of the feature quantity after the convolution layer;

[0269] Then, the fully connected network is used to further embed the feature h cov,i , the final output layer is the representation of high-level node features h g .

[0270] Further, the processor 1001 may call the power grid line loss optimization program based on graph attention perception and reinforcement learning decision-making stored in the memory 1004, and further perform the following operations:

[0271] The deep autoencoder is used to compress the high-dimensional node features into a low-dimensional representation, and the encoding process is as follows:

[0272] h e =f l […f 1 (h g )],(5)

[0273] In the formula, f l represents the fully connected network of the lth layer in the encoder. Through the nonlinear mapping of the multi-layer fully connected network, the original high-dimensional feature h g is converted into a low-dimensional representation he , that is, compressing from high-dimensional space to low-dimensional space.

[0274] Further, the processor 1001 may call the power grid line loss optimization program based on graph attention perception and reinforcement learning decision-making stored in the memory 1004, and further perform the following operations:

[0275] The step S4 specifically includes:

[0276] S41 randomly initializes the parameters θ of the global graph attention network and deep autoencoder ggat and θ dae ;

[0277] S42 inputs the data marked and divided in step S1 into the global graph attention network;

[0278] S43 uses historical data and labels to perform supervised training on the global graph attention network, calculates the actual line loss of each line and the line loss predicted by the model, and uses the loss function Loss(θ ggat ) performs gradient descent, and the model parameters will be gradually adjusted during the optimization process so that it can accurately predict line losses;

[0279] S44 uses unsupervised learning to train a deep autoencoder and minimizes the reconstruction loss function Loss(θ dae ) to optimize the parameters of the encoder, and finally, by optimizing the loss function, it is possible to extract low-dimensional representations from high-dimensional features; S45 loops through steps S42-S44 until the corresponding loss function gradually converges, thereby predicting line losses and adapting to the operation of a power grid with a high proportion of new energy access.

[0280] Further, the processor 1001 may call the power grid line loss optimization program based on graph attention perception and reinforcement learning decision-making stored in the memory 1004, and further perform the following operations:

[0281] In step S43, the loss function Loss(θ ggat ) is expressed as:

[0282]

[0283] Among them, B is the sampling batch, is the actual line loss of the ith line, is the line loss predicted by the model for the i-th line;

[0284] Use gradient descent to update the parameters θ of the perception model ggat , optimize the model by minimizing the loss function:

[0285]

[0286] Among them, η is the learning rate.

[0287] Further, including:

[0288] The loss function Loss(θ dae ) is specifically expressed as:

[0289]

[0290] Among them, L represents the number of layers of the DAE neural network, h g The high-dimensional features generated by the global graph attention network, are the features reconstructed by the deep auto-decoder.

[0291] Further, the processor 1001 may call the power grid line loss optimization program based on graph attention perception and reinforcement learning decision-making stored in the memory 1004, and further perform the following operations:

[0292] In step S5, the objective function of the intelligent decision-making model is expressed as:

[0293]

[0294] Among them, θ is the parameter of the policy network, r t (θ) is the ratio of the new and old strategies, which means the change of the current strategy relative to the old strategy. t is the advantage function, which is used to measure the performance of an action relative to the average performance. ∈ is a hyperparameter that controls the update amplitude of the policy to prevent the update from being too large. clip() is the truncation function. Express expectations.

[0295] Further, the processor 1001 may call the power grid line loss optimization program based on graph attention perception and reinforcement learning decision-making stored in the memory 1004, and further perform the following operations:

[0296] In step S6, the line loss optimization problem of the new energy high penetration power grid is transformed into a Markov decision process, which is expressed as follows:

[0297] DRL agent: The dispatch decision center is regarded as an agent to control PV and BSS in the power grid, achieving intelligent decision-making for power grid loss reduction;

[0298] Environment: The environment in which the agent interacts is the intelligent perception model of step S2. The agent makes a decision. The intelligent perception model feeds back an optimized line loss value and voltage value according to the decision amount, and then calculates the reward value according to the line loss and voltage values.

[0299] State space: When using PV inverters and BSS to reduce line losses, define the state of each time step as s t =(m t ,A), where m t represents the observed characteristics of the power grid, including: node net active power and reactive power, voltage amplitude and current amplitude, A represents the adjacency matrix;

[0300] Action space: The action of the agent at each time step is a t =(a pv,t ,a bss,t ), where a pv,t represents the reactive power output of the PV inverter at time t, a bss,t represents the active output of BSS at time t;

[0301] Reward function: The reward function is defined as minimizing the power grid line loss. The line loss at time t is represented by p loss,t , reward r t =-p loss,t +η, where η is the voltage over-limit penalty value.

[0302] Further, the processor 1001 may call the power grid line loss optimization program based on graph attention perception and reinforcement learning decision-making stored in the memory 1004, and further perform the following operations:

[0303] In step S6, the intelligent decision model training process includes:

[0304] S61 Model parameter loading: Load the parameters of the intelligent perception model trained in step S4, including the parameters θ of GGAT and DAE ggat and θ dae , randomly initialize the parameters of the Actor network and the Critic network;

[0305] S62 Initialize grid state observation: For each iteration, the initial state of the grid s 0 It will be extracted from historical data or real-time data to represent the current grid node status and line information; each iteration includes multiple time steps, and the intelligent agent interacts with the perception model at each time step to generate an experience trajectory;

[0306] S63 Agent performs actions and calculates state transitions and reward values: The agent extracts the feature state s at time step t. t Generate action a through the Actor network t , the state of the power grid changes accordingly when this action is executed, and it transfers to the next state s t+1,According to the executed actions and state changes, the perception model calculates the line loss changes and voltage conditions caused by the current action, and generates the corresponding reward value. The reward value represents the impact of the current action on the line loss. The goal is to maximize the cumulative reward by minimizing the line loss;

[0307] S64 stores the experience track to the experience replay pool: the current state s t 、Action a t , Reward t and the new state s t+1 Stored in the experience replay pool for subsequent use in training models. If the replay pool is full, delete the earliest experience data to keep the capacity of the experience pool fixed.

[0308] S65 samples training data from the experience replay pool: When enough samples have accumulated in the experience replay pool, a small batch of data is randomly sampled from it, containing multiple state-action-reward-new state combinations.

[0309] S66 updates the Critic network and Actor network;

[0310] After multiple rounds of iterations of S67, the Actor network and the Critic network gradually converged, and the intelligent agent was able to make the best decision based on the current grid status, maximize the reduction of line losses, and implement an adaptive online loss reduction strategy.

[0311] Further, the processor 1001 may call the power grid line loss optimization program based on graph attention perception and reinforcement learning decision-making stored in the memory 1004, and further perform the following operations:

[0312] In step S66, updating the Critic network specifically includes:

[0313] The Critic network uses data sampled from the experience pool to update its parameters; the goal of the Critic network is to minimize the estimation error of the state-action pair, and the update rule is as follows:

[0314]

[0315] In the formula, Loss critic is the loss function of the Critic network, which measures the gap between the actual state value and the estimated value, θ critic and η critic are the parameters and learning rate of the Critic network.

[0316] Further, the processor 1001 may call the power grid line loss optimization program based on graph attention perception and reinforcement learning decision-making stored in the memory 1004, and further perform the following operations:

[0317] In step S66, updating the Actor network specifically includes:

[0318] The Actor network uses the advantage function A calculated from the Critic network t , the update rules are as follows:

[0319]

[0320] Among them, θ actor and η actor are the parameters and learning rate of the Actor network.

[0321] In addition, it can be understood by a person skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the control terminal to implement the process steps of the embodiment of the above method.

[0322] Therefore, the present invention also provides a computer-readable storage medium, which stores a power grid line loss optimization program based on graph attention perception and reinforcement learning decision-making. When the power grid line loss optimization program based on graph attention perception and reinforcement learning decision-making is executed by a processor, it implements the various steps of the power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making described in the above embodiment.

[0323] It should be noted that since the storage medium provided in the embodiment of the present application is the storage medium used to implement the method of the embodiment of the present application, based on the method introduced in the embodiment of the present application, the person skilled in the art can understand the specific structure and deformation of the storage medium, so it is not repeated here. All storage media used in the method of the embodiment of the present application belong to the scope of protection of this application.

[0324] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0325] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0326] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0327] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0328] It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claim. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0329] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0330] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0331] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making, characterized in that: The method comprises the following steps: S1 collects real-time measurement and historical operation data of the power grid, and preprocesses and annotates the data; the real-time measurement and historical operation data include node voltage, load active power, load reactive power, line current, photovoltaic active power and reactive power, energy storage charging and discharging records, and line loss; the annotated division includes converting the preprocessed data into graph data containing node features, edge features, and topological structure; S2 builds an intelligent perception model, which includes a global graph attention network, a one-dimensional convolutional neural network and a fully connected network, transmits input data to the global graph attention network to obtain the hidden state of each node, and then uses the one-dimensional convolutional neural network and the fully connected network for processing in turn, and finally outputs high-dimensional node features; S3 uses a deep autoencoder to compress the high-dimensional node features into a low-dimensional representation; S4 uses the training set to train the intelligent perception model and the deep autoencoder to obtain a trained intelligent perception model; S5 builds an intelligent decision model, which includes two neural networks: an Actor network and a Critic network. The Actor network receives the data output by the trained intelligent perception model, and then obtains the action distribution of each node; the Critic network receives the data output by the trained intelligent perception model, and then the Critic network outputs a value function through a fully connected layer to evaluate the value of the current state and action; S6 converts the line loss optimization problem of the new energy high penetration power grid into a Markov decision process, thereby training the intelligent decision model according to the obtained action distribution of each node and the current state and action value, and then obtaining the adaptive online loss reduction strategy of the new energy high penetration power grid.

2. The power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making according to claim 1 is characterized in that: In step S1, the graph data is represented as: G = (V, E, X), where V is the node set, representing each node in the power grid; E is the edge set, representing the connection relationship between nodes; and X is the attribute matrix containing node features and edge features, including node voltage, load active power, load reactive power, line current, photovoltaic active power and reactive power, energy storage charging and discharging records, and line loss.

3. The power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making as claimed in claim 2, characterized in that: In step S2, constructing the intelligent perception model specifically includes: The formula for calculating the hidden state of each node through the global graph attention network is as follows: Among them, h att,i is the feature representation of node i, α ij is the attention score that measures the importance of node j to node i, N(i) represents the set of neighboring nodes of node i, g j (x j ) is the embedding function of node j composed of a single-layer neural network, x j is the input feature of node j, σ is the activation function, and the attention score α ij Calculated by the following formula: Where L1 represents the number of layers of the global graph attention network, g i (x i ) is the embedding function of node i consisting of a single-layer neural network, represents the connection between two node embeddings, w is the weight vector, LeakyReLU is the activation function, e ij Represents the attention weight. In order to extract the common features of the entire system, a one-dimensional convolutional neural network is used after the graph attention network, which is expressed as: Where W k is the weight of the kth 1D convolutional neural network layer, b k represents the bias of the kth neural network layer, F represents the number of layers of the one-dimensional convolutional neural network, σ is the activation function, and h cov,k It is the output of the feature quantity after the convolution layer; Then, the fully connected network is used to further embed the feature h cov,k , the final output layer is the representation of high-level node features h g .

4. The power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making as claimed in claim 3 is characterized in that: The deep autoencoder is used to compress the high-dimensional node features into a low-dimensional representation, and the encoding process is as follows: h e =f m […f1(h g )],(5) In the formula, f m Represents the fully connected network of the mth layer in the deep autoencoder. Through the nonlinear mapping of the multi-layer fully connected network, the original high-dimensional feature h g is converted into a low-dimensional representation h e , that is, compressing from high-dimensional space to low-dimensional space.

5. The power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making as claimed in claim 4, characterized in that: The step S4 specifically includes: S41 randomly initializes the parameters θ of the global graph attention network and deep autoencoder ggat and θ dae ; S42 inputs the data marked and divided in step S1 into the global graph attention network; S43 uses historical data and labels to perform supervised training on the global graph attention network, calculates the actual line loss of each line and the line loss predicted by the model, and uses the loss function Loss(θ ggat ) performs gradient descent, and the model parameters will be gradually adjusted during the optimization process so that it can accurately predict line losses; S44 uses unsupervised learning to train a deep autoencoder and minimizes the reconstruction loss function Loss(θ dae ) to optimize the encoder parameters, and finally, by optimizing the loss function, it can extract low-dimensional representation from high-dimensional features; S45 loops through steps S42-S44 until the corresponding loss function gradually converges, thereby predicting line losses and adapting to the operation of a power grid with a high proportion of new energy access.

6. The power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making as claimed in claim 5, characterized in that: In step S43, the loss function Loss(θ ggat ) is expressed as: Among them, B is the sampling batch, is the actual line loss of the qth line, The line loss predicted by the model for the qth line; Use gradient descent to update the parameters θ of the perception model ggat , optimize the model by minimizing the loss function: Among them, η is the learning rate.

7. The power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making as claimed in claim 5, characterized in that: The loss function Loss(θ dae ) is specifically expressed as: Among them, L2 represents the number of layers of the deep autoencoder, h g The high-dimensional features generated by the global graph attention network, are the features reconstructed by the deep auto-decoder.

8. The power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making according to claim 7, characterized in that: In step S5, the objective function of the intelligent decision-making model is expressed as: Among them, θ is the parameter of the policy network, r t (θ) is the ratio of the new and old strategies, which means the change of the current strategy relative to the old strategy. t is the advantage function at time t, which is used to measure the performance of an action relative to the average performance, ∈ is a hyperparameter that controls the update amplitude of the policy, clip() is the truncation function, Express expectations.

9. The power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making as claimed in claim 8, characterized in that: In step S6, the line loss optimization problem of the new energy high penetration power grid is transformed into a Markov decision process, which is expressed as follows: DRL agent: The dispatch decision center is regarded as an agent to control PV and BSS in the power grid, achieving intelligent decision-making for power grid loss reduction; Environment: The environment in which the agent interacts is the intelligent perception model of step S2. The agent makes a decision. The intelligent perception model feeds back an optimized line loss value and voltage value according to the decision amount, and then calculates the reward value according to the line loss and voltage values. State space: When using PV inverters and BSS to reduce line losses, define the state of each time step as s t =(m t ,A), where m t represents the observed characteristics of the power grid, including: node net active power and reactive power, voltage amplitude and current amplitude, A represents the adjacency matrix; Action space: The action of the agent at each time step is a t =(a pv,t ,a bss,t ), where a pv,t represents the reactive power output of the PV inverter at time t, a bss,t represents the active output of BSS at time t; Reward function: The reward function is defined as minimizing the power grid line loss. The line loss at time t is represented by p loss,t , reward r t =-p loss,t +η, where η is the voltage over-limit penalty value.

10. The power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making according to claim 9, characterized in that: In step S6, the intelligent decision model training process includes: S61 Model parameter loading: Load the parameters of the intelligent perception model trained in step S4, including the parameters θ of GGAT and DAE ggat and θ dae , randomly initialize the parameters of the Actor network and the Critic network; S62 initializes grid state observation: For each round of iteration, the initial state s0 of the grid is extracted from historical data or real-time data to represent the current grid node state and line information; each iteration includes multiple time steps, and the agent interacts with the perception model at each time step to generate an experience trajectory; S63 Agent performs actions and calculates state transitions and reward values: The agent extracts the feature state s at time t. t Generate action a through the Actor network t , the state of the power grid changes accordingly when this action is executed, and it transfers to the next state s t+1 ,According to the executed actions and state changes, the perception model calculates the line loss changes and voltage conditions caused by the current action, and generates the corresponding reward value. The reward value represents the impact of the current action on the line loss. The goal is to maximize the cumulative reward by minimizing the line loss; S64 stores the experience track to the experience replay pool: the current state s t 、Action a t , Reward t and the new state s t+1 Store in the experience replay pool for subsequent use in training models. If the replay pool is full, delete the earliest experience data to keep the capacity of the experience pool fixed. S65 samples training data from the experience replay pool: When enough samples have accumulated in the experience replay pool, randomly sample a small batch of data from it, containing multiple state-action-reward-new state combinations; S66 updates the Critic network and Actor network; After multiple rounds of iterations of S67, the Actor network and the Critic network gradually converged, realizing an adaptive online loss reduction strategy.

11. The power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making according to claim 10, characterized in that: In step S66, updating the Critic network specifically includes: The Critic network uses data sampled from the experience pool to update its parameters; the goal of the Critic network is to minimize the estimation error of the state-action pair, and the update rule is as follows: In the formula, Loss critic is the loss function of the Critic network, which measures the gap between the actual state value and the estimated value, θ critic and η critic are the parameters and learning rate of the Critic network.

12. The power grid line loss optimization method based on graph attention perception and reinforcement learning decision-making according to claim 10, characterized in that: In step S66, updating the Actor network specifically includes: The Actor network uses the advantage function A calculated from the Critic network t , the update rules are as follows: Among them, θ actor and η actor are the parameters and learning rate of the Actor network.

13. A power grid line loss optimization system based on graph attention perception and reinforcement learning decision-making, characterized in that: The system includes: A data preprocessing module is used to collect real-time measurement and historical operation data of the power grid, and to preprocess and annotate the data; the real-time measurement and historical operation data include node voltage, load active power, load reactive power, line current, photovoltaic active power and reactive power, energy storage charging and discharging records, and line loss; the annotating and partitioning includes converting the preprocessed data into graph data containing node features, edge features, and topological structure; An intelligent perception model construction module is used to construct an intelligent perception model, which includes a global graph attention network, a one-dimensional convolutional neural network and a fully connected network. The input data is transmitted to the global graph attention network to obtain the hidden state of each node, and then the one-dimensional convolutional neural network and the fully connected network are used for processing in turn to finally output high-dimensional node features. A feature dimensionality reduction module, used for compressing the high-dimensional node features into a low-dimensional representation using a deep autoencoder; An intelligent perception model training module, used to train the intelligent perception model and the deep autoencoder using a training set to obtain a trained intelligent perception model; An intelligent decision model construction module is used to construct an intelligent decision model, wherein the intelligent decision model includes two neural networks: an Actor network and a Critic network. The Actor network receives the data output by the trained intelligent perception model, and then obtains the action distribution of each node; the Critic network receives the data output by the trained intelligent perception model, and then the Critic network outputs a value function through a fully connected layer to evaluate the value of the current state and action; The intelligent decision-making model training module is used to convert the line loss optimization problem of the renewable energy high-penetration power grid into a Markov decision process, so as to train the intelligent decision-making model according to the obtained action distribution of each node and the current state and action value, and then obtain the adaptive online loss reduction strategy of the renewable energy high-penetration power grid.

Images