A method and system for intelligent fault monitoring in power distribution networks with distributed energy resources

Abstract

A method and system for intelligent fault monitoring in distribution networks containing distributed energy resources are disclosed. The method first acquires multi-source information data of the distribution network containing distributed energy resources and extracts features to generate a node feature matrix. Then, based on the node feature matrix, a Markov decision model for fault detection in the distribution network containing distributed energy resources is constructed. The system's main body is used as the agent, and the agent is trained using an improved D4PG algorithm based on the Markov decision model. After training, the optimal fault detection strategy for the distribution network containing distributed energy resources is derived and deployed in the fault detection task for intelligent fault monitoring. This invention, through a deep reinforcement learning framework, transforms the complex fault detection task of the distribution network containing distributed energy resources into an optimal decision problem, successfully establishing a direct, fast, and reliable logical mapping relationship from the grid fault state to the optimal diagnostic action, providing a more robust and adaptive fault detection method.

Description

Technical Field

[0001] This invention belongs to the field of distribution network fault detection, specifically relating to an intelligent monitoring method and system for distribution network faults including distributed energy resources. Background Technology

[0002] With the rapid increase in the penetration rate of distributed generation systems (DERs) and renewable energy sources (such as photovoltaics and wind power), traditional unidirectional radial distribution networks are evolving into complex active distribution networks. While this transformation improves energy efficiency and power supply flexibility, it also brings severe challenges to the operation and management of power systems. In active distribution networks, rapid and accurate fault identification and location are crucial to ensuring the safe and stable operation of the power grid. However, the intermittent and random nature of distributed generation, the bidirectional nature of power flow, and the dynamic changes in distribution network topology make fault characteristics more complex and difficult to capture. Traditional fault detection methods for active distribution networks often rely on single information sources from local measurements or static topology analysis, which are insufficient to effectively address issues such as relay protection maloperation and failure to operate caused by bidirectional power flow. Furthermore, the power grid contains a large amount of multi-source heterogeneous data generated by sensors and smart devices, including topology data, voltage / current time-series data, and equipment status data. How to effectively integrate this complex information and extract robust fault characteristics from the highly coupled power grid topology has become a pressing problem in the field of fault detection.

[0003] In the field of intelligent fault detection in power distribution networks, research mainly focuses on data-driven fault feature extraction and fault location decision-making. For feature extraction, convolutional neural networks (CNNs) are widely used due to their powerful ability to capture local features. A fault selection method for power distribution networks based on image fusion and dual-channel CNNs improves the accuracy of fault selection under complex operating conditions by converting current signals into two-dimensional images, achieving an accuracy of up to 99.97%. However, this method transforms the complex topological structure information of the power distribution network into image features, destroying the original connection relationships between nodes and making it difficult to directly capture the spatiotemporal coupling characteristics of the power grid. A fault location method for power distribution networks based on graph attention networks improves the discriminative power of fault features under complex topologies by adaptively allocating connection weights between nodes. However, this method only focuses on feature extraction, and its model design emphasizes the static output of fault location or classification, making it difficult to directly apply to dynamic decision optimization after fault detection. Furthermore, these models are mostly trained on existing real data, failing to effectively address the problem of the scarcity of high-risk fault samples in practical applications.

[0004] In fault decision-making, reinforcement learning (RL) methods have been introduced in recent years to achieve adaptive decision-making, including deep reinforcement learning-based distribution network fault recovery methods based on distributed generation sources. These methods construct Markov decision process models and train agents using deep reinforcement learning (DRL). However, these methods primarily focus on fault recovery decisions rather than early-stage intelligent fault detection, and the algorithms employed have limitations in handling continuous action spaces and ensuring policy convergence.

[0005] In summary, at the feature extraction level, existing research remains at the level of simple feature learning, lacking effective enhancement mechanisms for sparse fault data. At the fault decision-making level, existing algorithms, when applied to distribution network fault problems, often employ basic models, making it difficult to adapt to the complex and dynamic environmental changes of power distribution networks with multiple sources. Therefore, researching an intelligent fault detection method driven by multi-source information fusion to overcome the limitations of traditional methods in complex power distribution networks with multiple sources has become a hot topic and core requirement for ensuring the high reliability of power systems. Summary of the Invention

[0006] The purpose of this invention is to address the aforementioned problems in the existing technology by providing a method and system for intelligent monitoring of faults in distribution networks containing distributed energy resources.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows:

[0008] In a first aspect, the present invention proposes an intelligent fault monitoring method for distribution networks containing distributed energy resources, comprising:

[0009] S1. Obtain multi-source information data of the distribution network containing distributed energy resources, extract features, and generate a node feature matrix;

[0010] S2. Based on the node feature matrix, construct a Markov decision model for fault detection in power distribution networks with sources;

[0011] S3. Using the main body of the fault intelligent detection system as the intelligent agent, the intelligent agent is trained based on the Markov decision model and using the improved D4PG algorithm.

[0012] S4. After training, export the optimal fault detection strategy for power distribution networks with active sources and deploy it in the fault detection task of power distribution networks with active sources to carry out intelligent monitoring of power distribution network faults.

[0013] S3 includes:

[0014] S31. Based on the state space, action space, and reward function defined by the Markov decision model, the agent selects actions according to the current policy, records the current observations, actions, and rewards, and stores these experiences in the experience pool.

[0015] S32. Use the multi-scale nearest neighbor algorithm to sample the experience most relevant to the current state from the experience pool, and train the Q network and policy network.

[0016] S33. Design a dynamic feedback mechanism to dynamically adjust the noise constant of the action selection by the strategy network according to changes in environmental conditions;

[0017] S34. Design a generalized Huber loss function for updating the Q-network; update the policy network based on the policy gradient;

[0018] S35. Repeat steps S31-S34 until the model performance is stable or the maximum number of training rounds is reached, then training ends.

[0019] In S32, the multi-scale nearest neighbor algorithm is as follows:

[0020] ;

[0021] ;

[0022] In the above formula, The most relevant subset of experiences in the replay pool for the current state. For the first in the playback pool One experience The collection of all experiences in the replay pool. For the current experience and replay pool, the first Multiscale distance between experiences To consider the number of scales, For the first Weighting coefficients for each scale For the current experience in the first eigenvectors at various scales For the first in the playback pool The first experience in Feature vectors at various scales;

[0023] In S33, the dynamic feedback mechanism is as follows:

[0024] ;

[0025] ;

[0026] ;

[0027] In the above formula, For a moment The space of motion For a moment Deterministic actions obtained through policy network computation For a moment The noise constant, For a moment OU noise, The initial noise constant is... For the total number of training rounds, This is the coefficient representing the influence of environmental complexity on the noise constant. For a moment State space entropy, For a moment The state space, For smoothing parameters, For state The probability of occurrence;

[0028] In S34, the generalized Huber loss function is:

[0029] ;

[0030] ;

[0031] ;

[0032] ;

[0033] ;

[0034] ;

[0035] In the above formula, For the generalized Huber loss function, For the target Q-value distribution, To predict the Q-value distribution, For the threshold, For the selected set of quantiles, Here is the Huber loss function. for and Wasserstein distance between them For joint distribution, , Let each be a sample point in the target Q-value distribution and the predicted Q-value distribution, respectively. For a moment The state space, For a moment The space of motion For quantiles, The total dimension of the sample points. , Sample points , of Dimensional components.

[0036] In S2, the state space of the Markov decision model is:

[0037] ;

[0038] The action space of the Markov decision model is:

[0039] ;

[0040] The reward function of the Markov decision model is:

[0041] ;

[0042] ;

[0043] ;

[0044] In the above formula, For a moment The state space, For the first The feature vector of each node This represents the total number of nodes in the distribution network. For a moment The space of motion This indicates that the device is in a normal state. This indicates that the equipment is in a minor fault condition. This indicates that the equipment is in a critical fault condition. This indicates that the device is in a faulty state. For a moment The reward function, As a time-sensitive reward, For accuracy rewards, , These are the reward weights for timely responses and slightly slower responses within the threshold range. Penalty weighting for delayed response, The time difference for fault response. It is a time constant. To accurately detect the reward weight for faults, , These are the penalty weights for false alarms and false alarms, respectively. To accurately detect the proportion of faults, This indicates the penalty for a false alarm. This indicates the penalty for failing to report.

[0045] S1 includes:

[0046] S11. Represent the multi-source information data of the distribution network containing distributed energy as a graph structure;

[0047] S12. Input the graph structure into the graph generative adversarial network, and use the generator and discriminator of the graph generative adversarial network to gradually optimize the generated graph data and output a new graph structure.

[0048] S13. Use graph convolutional networks to extract node features from the new graph structure and generate a node feature matrix.

[0049] Secondly, this invention proposes an intelligent fault monitoring system for distribution networks containing distributed energy resources, including a feature matrix generation module, a decision model construction module, an agent training module, and an intelligent fault monitoring module.

[0050] The feature matrix generation module is used to acquire multi-source information data of the distribution network containing distributed energy, extract features, and generate a node feature matrix.

[0051] The decision model construction module is used to construct a Markov decision model for source distribution network fault detection based on the node feature matrix.

[0052] The agent training module is used to train the agent using the main body of the fault intelligent detection system as the agent, based on the Markov decision model and using the improved D4PG algorithm.

[0053] The intelligent fault monitoring module is used to export the optimal fault detection strategy for power distribution networks with active sources after training, and to deploy it in the fault detection task of power distribution networks with active sources for intelligent fault monitoring of power distribution networks.

[0054] The agent training module includes an experience storage unit, a network training unit, a dynamic adjustment unit, a network update unit, and a repetitive iteration unit;

[0055] The experience storage unit is used to define the state space, action space, and reward function based on the Markov decision model. The agent selects actions according to the current policy, records the current observations, actions, and rewards, and stores these experiences in the experience pool.

[0056] The network training unit is used to sample the experience most relevant to the current state from the experience pool using a multi-scale nearest neighbor algorithm to train the Q network and the policy network.

[0057] The dynamic adjustment unit is used to design a dynamic feedback mechanism to dynamically adjust the noise constant of the action selected by the strategy network according to changes in environmental conditions.

[0058] The network update unit is used to design the generalized Huber loss function for updating the Q-network; and to update the policy network based on the policy gradient.

[0059] The repeated iteration unit is used to repeat the experience storage unit-network update unit until the model performance is stable or the maximum number of training rounds is reached, at which point training ends.

[0060] In the network training unit, the multi-scale nearest neighbor algorithm is as follows:

[0061] ;

[0062] ;

[0063] In the above formula, The most relevant subset of experiences in the replay pool for the current state. For the first in the playback pool One experience The collection of all experiences in the replay pool. For the current experience and replay pool, the first Multiscale distance between experiences To consider the number of scales, For the first Weighting coefficients for each scale For the current experience in the first eigenvectors at various scales For the first in the playback pool The first experience in Feature vectors at various scales;

[0064] The dynamic feedback mechanism in the dynamic adjustment unit is as follows:

[0065] ;

[0066] ;

[0067] ;

[0068] In the above formula, For a moment The space of motion For a moment Deterministic actions obtained through policy network computation For a moment The noise constant, For a moment OU noise, The initial noise constant is... For the total number of training rounds, This is the coefficient representing the influence of environmental complexity on the noise constant. For a moment State space entropy, For a moment The state space, For smoothing parameters, For state The probability of occurrence;

[0069] In the network update unit, the generalized Huber loss function is:

[0070] ;

[0071] ;

[0072] ;

[0073] ;

[0074] ;

[0075] ;

[0076] In the above formula, For the generalized Huber loss function, For the target Q-value distribution, To predict the Q-value distribution, For the threshold, For the selected set of quantiles, Here is the Huber loss function. for and Wasserstein distance between them For joint distribution, , Let each be a sample point in the target Q-value distribution and the predicted Q-value distribution, respectively. For a moment The state space, For a moment The space of motion For quantiles, The total dimension of the sample points. , Sample points , of Dimensional components.

[0077] In the decision model construction module, the state space of the Markov decision model is:

[0078] ;

[0079] The action space of the Markov decision model is:

[0080] ;

[0081] The reward function of the Markov decision model is:

[0082] ;

[0083] ;

[0084] ;

[0085] In the above formula, For a moment The state space, For the first The feature vector of each node This represents the total number of nodes in the distribution network. For a moment The space of motion This indicates that the device is in a normal state. This indicates that the equipment is in a minor fault condition. This indicates that the equipment is in a critical fault condition. This indicates that the device is in a faulty state. For a moment The reward function, As a time-sensitive reward, For accuracy rewards, , These are the reward weights for timely responses and slightly slower responses within the threshold range. Penalty weighting for delayed response, The time difference for fault response. It is a time constant. To accurately detect the reward weight for faults, , These are the penalty weights for false alarms and false alarms, respectively. To accurately detect the proportion of faults, This indicates the penalty for a false alarm. This indicates the penalty for failing to report.

[0086] The feature matrix generation module includes a graph structure forming unit, a graph data generation unit, and a node feature extraction unit;

[0087] The graph structure forming unit is used to represent multi-source information data of a distribution network containing distributed energy resources as a graph structure;

[0088] The graph data generation unit is used to input the graph structure into the graph generative adversarial network, and use the generator and discriminator of the graph generative adversarial network to gradually optimize the generated graph data and output a new graph structure.

[0089] The node feature extraction unit is used to extract node features from a new graph structure using a graph convolutional network, and generate a node feature matrix.

[0090] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0091] 1. This invention proposes an intelligent monitoring method and system for distribution network faults containing distributed energy sources. The method first acquires multi-source information data of the distribution network containing distributed energy sources and extracts features to generate a node feature matrix. Then, based on the node feature matrix, a Markov decision model for fault detection in the distribution network containing distributed energy sources is constructed. The main body of the intelligent fault detection system is used as the agent, and the agent is trained using an improved D4PG algorithm based on the Markov decision model. After training, the optimal fault detection strategy for the distribution network containing distributed energy sources is derived and deployed in the fault detection task of the distribution network containing distributed energy sources for intelligent monitoring of distribution network faults. On the one hand, this method transforms the complex task of fault detection in power distribution networks into an optimal decision problem through a deep reinforcement learning framework. It successfully establishes a direct, fast, and reliable logical mapping relationship from the fault state of the power grid to the optimal diagnostic action, enhancing the convergence speed and providing a more robust and adaptive fault decision-making capability. On the other hand, this method constructs a framework combining graph generative adversarial networks and graph convolutional networks to generate a node feature matrix. It integrates local neighborhood information and global power grid topology information, effectively solving the problem of scarce power grid fault data. By enhancing the diversity of the training set, the agent can perceive the environment more quickly and accelerate the convergence speed.

[0092] 2. This invention proposes an intelligent fault monitoring method and system for distribution networks containing distributed energy resources. Based on the D4PG algorithm, three groundbreaking targeted improvements are made. Experience replay optimization and dynamic feedback are used to address dynamic drift in operating conditions and topology, while the generalized Huber loss function addresses outliers and uncertainties in real-world data. Experience replay optimization using a multi-scale nearest neighbor algorithm ensures the optimal deterministic strategy has stronger generalization ability during training, avoiding deviations from the training distribution in real-world data. A dynamic feedback mechanism is designed to enable the optimal deterministic strategy to quickly adapt to changes in the real-world environment, preventing strategy rigidity. The generalized Huber loss function, combining Huber loss (anti-outlier) and Wasserstein distance (anti-distribution bias), ensures that strategy evaluation is based on the most stable and noise-resistant Q-value distribution, preventing the optimal deterministic strategy from being misled in high-noise environments. The improved D4PG ensures the agent can stably estimate the Q-value distribution during training, significantly improving the stability and reliability of the algorithm in real-world distribution network environments with high noise and high outliers, resulting in more accurate and reliable fault detection results.

[0093] 3. This invention proposes an intelligent monitoring method and system for distribution network faults containing distributed energy resources. When constructing a Markov decision model for fault detection in a distribution network containing energy resources, this method integrates timeliness rewards and accuracy control into a single reward function to improve the intelligence of decision-making. This not only considers the accuracy of fault detection but also the timeliness of detection and the penalties for false alarms and missed alarms, thus more comprehensively optimizing the fault detection system and reducing unnecessary fault responses. Attached Figure Description

[0094] Figure 1 This is an overall flowchart of the method described in this invention.

[0095] Figure 2 This is the log-scale mixed matrix diagram described in Example 1.

[0096] Figure 3 This is the average reward convergence curve described in Example 1.

[0097] Figure 4 This is a structural diagram of the system described in this invention. Detailed Implementation

[0098] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings.

[0099] This invention proposes an intelligent monitoring method and system for distribution network faults involving distributed energy resources. It constructs a framework combining Generative Graph Networks (GGN) and Generative Graph Convolutional Networks (GCN). GGN generates effective data, while GCN extracts robust graph structure features from the complex power grid topology, which serve as input to the D4PG algorithm. A Markov decision model for fault detection in distribution networks with distributed energy resources is constructed based on the D4PG algorithm. Multiple improvements are made to the D4PG algorithm, including: designing a generalized Huber loss function to enhance the robustness of Q-value distribution estimation; designing a dynamic feedback mechanism to dynamically adjust the noise constant according to changes in environmental conditions, thus balancing exploration and utilization in a more flexible and adaptive manner; and introducing a multi-scale nearest neighbor algorithm into the experience pool to optimize experience selection and ensure the diversity and representativeness of replay experiences. Finally, the agent is trained based on the improved D4PG algorithm to achieve optimal intelligent fault detection in distribution networks with distributed energy resources.

[0100] Example 1:

[0101] like Figure 1 As shown, a method for intelligent fault monitoring in a distribution network containing distributed energy resources is carried out in the following steps:

[0102] 1. Obtain multi-source information data of the distribution network containing distributed energy resources, extract features, and generate a node feature matrix;

[0103] Acquire multi-source information data of distribution networks containing distributed energy sources, including grid topology (such as equipment connection relationships), equipment status (normal, minor fault, serious fault, failure), real-time current, voltage, power, load data, and distributed energy output (such as solar energy, wind energy, etc.).

[0104] First, the multi-source information data of the distribution network containing distributed energy resources is represented as a graph structure;

[0105] In the graph structure, nodes represent equipment such as transformers, circuit breakers, and meters, as well as additional distributed power sources (solar, wind power); node characteristics include equipment status adjacency matrix state data (voltage, current, load, equipment status, etc.), and for distributed power sources, it also includes their status (such as power generation, frequency fluctuation, etc.); edges represent power transmission paths between equipment (cables, substation connections, etc.); edge characteristics include transmission capacity, transmission status, distance, etc.

[0106] The node characteristic matrix in the distribution network is represented as follows , This represents the total number of nodes in the distribution network. The feature number of each node; the connection relationship between nodes in the distribution network is represented by an adjacency matrix. If node and nodes If there is an edge between them, then Otherwise, it is 0.

[0107] Then, the graph structure is input into the graph generative adversarial network (GAN). The generator and discriminator of the GAN are used to gradually optimize the generated graph data, output a new graph structure, and generate a realistic power grid topology.

[0108] Generative Graph Networks (GGNs) are implemented using Generative Generative Adversarial Networks (GraphGANs). Based on the real topology of the power grid, GGNs generate realistic synthetic topology and multi-modal fault graph data. This effectively addresses the problem of scarce power grid fault data by enhancing the diversity and robustness of the training set, providing high-quality input for subsequent fault detection.

[0109] Graph generative adversarial networks are mainly composed of generators. and discriminator The system is composed of components trained through adversarial learning. The generator produces a new graph structure based on random noise or a partial power grid graph structure, aiming to generate graph data with realistic power grid characteristics (including the electrical states of nodes and the connections between devices). The discriminator receives the real power grid graph and the generated graph, determining which graph comes from the real data and which is a fabricated graph created by the generator. Through this process, the generator and discriminator compete against each other, progressively optimizing the generated graph data.

[0110] The generator takes random noise or a partial graph structure as input and outputs a new graph structure, where the generator loss function is:

[0111] ;

[0112] In the above formula, Let the loss function of the generator be... It is random noise. For random noise The probability distribution, For the discriminator to determine the generated image, The graph generated by the generator;

[0113] The discriminator takes a graph structure as input and determines whether it is a true power grid graph structure. The discriminator loss function is:

[0114] ;

[0115] In the above formula, Let the loss function of the discriminator be , This is real power grid diagram data. For real power grid diagram data The probability distribution, To determine the authenticity of the image;

[0116] After training with a graph generative adversarial network, the generator can produce new graph data with similar topology and grid characteristics to the original power grid data. This generated data can be used to augment the training set, helping to better learn fault modes in the power grid during fault detection.

[0117] Finally, a graph convolutional network is used to extract node features from the new graph structure and generate a node feature matrix.

[0118] After the graph data (including real data and data generated by graph generative adversarial networks) is prepared, feature extraction is performed using graph convolutional networks (GCNs). Graph convolutional networks can automatically learn and enhance high-level spatiotemporal features of nodes and edges from complex power grid graph structures, and are particularly adept at capturing the interactions and dynamic impacts between devices in distribution networks containing distributed power sources. Through graph convolution, the feature vector of each node is updated, fusing local neighborhood information and global power grid topology information. The final generated node feature matrix serves as input for downstream fault detection.

[0119] Each layer of the graph convolutional network performs a convolution operation on the node features and the features of neighboring nodes, and then passes them to the next layer through an activation function;

[0120] The formula for extracting node features using a graph convolutional network is:

[0121] ;

[0122] ;

[0123] ;

[0124] In the above formula, For the first The node feature matrix of the layer For activation function, To standardize the adjacency matrix, For the first The node feature matrix of the layer For the first The weight matrix of the layer, For the first Layer Time Node eigenvectors, For nodes The set of neighboring nodes, For nodes and nodes The adjacency relationship between them For the first Layer Time Node eigenvectors.

[0125] To address data sparsity and measurement noise, a framework combining Graph Generative Adversarial Networks (GraphGAN) and Graph Convolutional Networks (GCN) is constructed. This framework denoises or completes noisy data, uses GraphGAN to generate effective data, standardizes or normalizes numerical data (such as voltage, current, and power), and produces synthetic data that covers boundary conditions. This ensures that the optimal deterministic strategy generalizes to unseen fault modes. Furthermore, the Graph Convolutional Network extracts robust graph structure features from the complex power grid topology, focusing on stable power grid topology features rather than volatile raw measurements, forming the final state representation as input for subsequent algorithms.

[0126] 2. Based on the node feature matrix, construct a Markov decision model for fault detection in power distribution networks with active nodes based on the D4PG algorithm;

[0127] Assume that there are power sources in the distribution network. If we take the node features extracted by the graph convolutional network as the state space, then the state space of the Markov decision model can be represented as:

[0128] ;

[0129] For a fault detection task, the action space includes the agent's classification judgments on the device state. Therefore, the action space of the Markov decision model is represented as:

[0130] ;

[0131] By integrating timeliness rewards and accuracy control into a single reward function, this approach considers not only the accuracy of fault detection but also the timeliness of detection and penalties for false alarms and false negatives. The final reward function of the Markov decision model is expressed as follows:

[0132] ;

[0133] To improve the intelligence of decision-making, by comprehensively considering timeliness and accuracy through the reward function, the intelligent agent can more comprehensively optimize the fault detection system and reduce unnecessary fault responses;

[0134] The time-weighted reward can be expressed as:

[0135] Timeliness rewards are not limited to positive rewards for early fault detection; weighted rewards can also be designed based on the time difference in fault detection. The shorter the fault response time, the greater the reward weight, thus incentivizing the agent to complete detection as quickly as possible after a fault occurs. Assume... The time when the fault was detected. The response time difference is the time when the fault occurs. Used to measure the timeliness of an agent's response;

[0136] ;

[0137] The weighted penalty for false positives and false negatives can be expressed as:

[0138] To control accuracy, a weighted penalty for false positives and false negatives is introduced. By defining false positive weights and false negative weights, the agent becomes more sensitive to the control of false positives and false negatives, thereby improving accuracy.

[0139] ;

[0140] In the above formula, For a moment The state space, For the first The feature vector of each node includes the device status (normal, minor fault, severe fault, failure) and power information (voltage, current, power, load, distributed power output, etc.) of that node. This represents the total number of nodes in the distribution network. For a moment The space of motion This indicates that the device is in a normal state. This indicates that the equipment is in a minor fault condition. This indicates that the equipment is in a critical fault condition. This indicates that the device is in a faulty state. For a moment The reward function, As a time-sensitive reward, it is calculated based on the time difference in detection. For accuracy rewards, calculations are based on accurate detection, false alarms, and missed detections. , These are the reward weights for timely responses and slightly slower responses within the threshold range. Penalty weighting for delayed response, The time difference for fault response. This is a time constant used to control the decay rate of time-sensitive rewards. To accurately detect the reward weight for faults, , These are the penalty weights for false alarms and false alarms, respectively. To accurately detect the proportion of faults, This indicates the penalty for a false alarm. This indicates the penalty for failing to report.

[0141] 3. Using the main body of the fault intelligent detection system as the intelligent agent, the agent is trained based on the Markov decision model and using the improved D4PG algorithm;

[0142] In this technical solution, the "intelligent agent" specifically refers to the core decision-making module built based on the improved D4PG deep reinforcement learning algorithm, which is the main body of the intelligent fault detection system. In layman's terms, the "main body" of the intelligent fault detection system is a high-performance computer or server installed in the distribution network dispatch center; it can be understood as the physical "brain" of the entire system. It receives real-time voltage and current data from the power grid through a hardware interface, uses its internal CPU and memory to run our designed improved D4PG algorithm program, automatically calculates and analyzes the data, and directly displays the result of "where the fault occurred" on the screen or sends it to automated equipment. Through continuous interaction with the distribution network environment, and based on improvements such as reward feedback, the generalized Huber loss function, and dynamic feedback mechanisms, the intelligent agent continuously optimizes its decision-making strategy, ultimately achieving optimal intelligent fault detection.

[0143] Traditional deep reinforcement learning (DRL) algorithms, such as DQN and basic A2C / A3C, often face challenges such as inaccurate Q-value estimation and unstable policy convergence when applied to environments like distribution networks with active grids, especially in scenarios with frequent grid topology changes and sparse fault samples. Therefore, D4PG (Distributed Deterministic Policy Gradient) is chosen as the fault detection algorithm. The core advantage of D4PG lies in its introduction of a distributed review network, which can predict the complete distribution of Q-values ​​rather than a single expected value. This distribution-based learning approach significantly enhances the robustness and reliability of Q-value estimation in dynamic environments with sparse reward signals, such as distribution networks. Simultaneously, D4PG's distributed architecture and efficient experience replay mechanism improve data utilization efficiency and exploration diversity, enabling it to learn the optimal fault detection decision policy faster and more stably than other traditional DRL algorithms.

[0144] D4PG is a reinforcement learning algorithm, an extension of DDPG. It combines distributed deep learning and distributed value estimation, employing deterministic policy gradients and distributed value networks. Its aim is to accelerate the learning process by estimating the value function in a distributed manner and to handle reinforcement learning tasks in complex environments. The basic principles of its algorithm include:

[0145] (1) Deterministic policy gradient;

[0146] D4PG uses deterministic policy gradients for policy updates. Specifically, it assumes a policy... It is deterministic, meaning that for each state... The strategy will give a definite action. Instead of a probability distribution, the parameters of the policy network are used to maximize the cumulative reward. The update is performed along the direction of the gradient of the value function, and the formula is as follows:

[0147] ;

[0148] ;

[0149] In the above formula, For the gradient of the policy, This represents the parameters of the current policy network. For experience replay pool, This indicates that the value network output corresponds to the action. gradient, For a moment The state space, For a moment The space of motion The parameters of the current value network, The policy network output is related to the parameters. gradient, For policy networks, based on state Select Action .

[0150] (2) Distributed value estimation;

[0151] D4PG employs distributed value estimation to improve the accuracy of Q-value estimation. Traditional Q-learning uses a single Q-value to evaluate each state-action pair, while distributed estimation uses a probability distribution. The reward is represented by this formula. By estimating the probability distribution, the agent can capture the uncertainty and multimodal characteristics of the environment, as shown in the formula:

[0152] ;

[0153] In the above formula, The distribution of the random variable representing the returns, This represents the total number of nodes in the distribution network. This represents the corresponding probability output by the network. These are the pre-defined support set atoms.

[0154] (3) Distributed Bellman Update and Return Estimation;

[0155] To train the value network, a target distribution needs to be constructed. D4PG uses the Bellman operator to update the distribution for the next time step, which serves as the learning target for the current time step. The update process can be represented as follows:

[0156] ;

[0157] In the above formula, For the distributed Bellman operator, This indicates the like-distribution relationship. For the reward, As a discount factor, For the target policy network, For the parameters of the target policy network, These are the parameters of the target value network.

[0158] (4) Target network and soft update;

[0159] To maintain training stability, D4PG employs a target network and a soft update strategy. Soft updates avoid oscillations during training by slowly updating the weights of the target network using a moving average. The soft update formula for the target network is:

[0160] ;

[0161] ;

[0162] In the above formula, This is the soft update coefficient.

[0163] (5) Experience playback and distributed data collection;

[0164] D4PG uses a distributed experience replay pool to accelerate the learning process. Multiple agents can interact with the environment simultaneously, and then store the collected experiences (state, action, reward, next state) into the experience replay pool for subsequent sampling and training. The formula is:

[0165] ;

[0166] In the above formula, For a moment Collected experience pool data, Indicates the next state.

[0167] Based on the fundamental principles of the D4PG algorithm, targeted improvements are made in three aspects: First, a generalized Huber loss function is designed by combining Wasserstein distance and Huber loss to enhance the robustness of Q-value distribution estimation. Second, a dynamic feedback mechanism is designed to dynamically adjust the noise constant according to changes in environmental conditions, allowing for a more flexible and adaptive balance between exploration and utilization. Third, a multi-scale nearest neighbor algorithm is introduced into the experience pool to optimize experience selection and ensure the diversity and representativeness of replay experiences.

[0168] Training the agent using an improved D4PG algorithm:

[0169] First, based on the state space, action space, and reward function defined by the Markov decision model, the agent selects actions according to the current policy, records the current observations, actions, and rewards, and stores these experiences in the experience pool;

[0170] Improvement 1: Use a multi-scale nearest neighbor algorithm to sample the experience most relevant to the current state from the experience pool and train the Q-network and policy network;

[0171] Improvement point 2: Design a dynamic feedback mechanism to dynamically adjust the noise constant of the action selection by the strategy network according to changes in environmental conditions;

[0172] Improvement 3: Design a generalized Huber loss function for updating the Q-network; update the policy network based on policy gradients;

[0173] Repeat the above steps until the model performance is stable or the maximum number of training epochs is reached, at which point training ends.

[0174] In Improvement 1, a multi-scale nearest neighbor algorithm is introduced into the experience pool to optimize experience selection and ensure the diversity and representativeness of replay experiences.

[0175] In the replay pool, the similarity between the current state and samples in each replay pool is calculated using a multi-scale nearest neighbor algorithm. Multi-scale metrics are then used to select the experience most relevant to the current state, ensuring that the experience in the replay pool is both diverse and representative, thereby improving the quality of the training samples. The specific steps are as follows:

[0176] First, the current experience sample Represented as a quadruple ;

[0177] Then, the multi-scale nearest neighbor algorithm is used to measure the current experience. and all other experiences in the replay pool The similarity between them can be calculated using Euclidean distance:

[0178] ;

[0179] Finally, selective playback is performed: After calculating the multi-scale distance, the most relevant empirical playback can be selected based on the current state. The formula for selective playback is:

[0180] ;

[0181] In the above formula, For the first in the playback pool One experience The collection of all experiences in the replay pool. For the current experience and replay pool, the first Multiscale distance between experiences To consider the number of scales, For the first The weighting coefficients for each scale are used to adjust the influence of each scale in the total distance calculation. For the current experience in the first eigenvectors at various scales For the first in the playback pool The first experience in eigenvectors at various scales for and Euclidean distance, The most relevant subset of experiences in the replay pool for the current state;

[0182] Experience replay is a crucial step in reinforcement learning to improve sample utilization. However, standard experience replay (including priority replay) can lead to locality and repetitiveness issues in experience samples, affecting the model's generalization ability. This patent introduces a multi-scale nearest neighbor algorithm to optimize sample selection in the experience pool. The innovation of this mechanism lies in its focus not only on the priority of experiences but also on their diversity and representativeness: by measuring the similarity between experiences at different "scales," it ensures that the replayed experiences cover a broad state space, avoiding repeated training of the agent on only a few similar experiences. This optimization significantly improves the quality and diversity of replayed experiences, thereby enhancing the generalization ability of the improved D4PG algorithm, enabling it to better handle failure modes that did not appear or were rare during training.

[0183] In improvement point 2, the dynamic feedback mechanism dynamically adjusts the noise constant according to changes in environmental conditions, so as to adjust the balance between exploration and utilization in a more flexible and adaptive way;

[0184] In D4PG, the policy network is used to select actions. Furthermore, in the continuous action space, noise is typically added to the selected actions to guide the exploration. This noise is usually generated using Ornstein-Uhlenbeck noise (OU noise), as shown in the formula:

[0185] ;

[0186] Traditional D4PG algorithms use OU noise for exploration, but the exploration intensity is fixed and difficult to adaptively adjust based on environmental stability and uncertainty. This makes it difficult for the algorithm to effectively balance exploration and exploitation in complex or changing environments. To address this issue, this paper introduces a dynamic feedback mechanism. This mechanism determines the noise adjustment magnitude by considering the stability of the current environmental state, enabling the algorithm to adjust the balance between exploration and exploitation in a more flexible and adaptive manner, thus better adapting to environmental changes.

[0187] The formula for the dynamic feedback mechanism is as follows:

[0188] ;

[0189] ;

[0190] In the above formula, For a moment The space of motion For a moment Deterministic actions obtained through policy network computation For a moment The noise constant is used to control the magnitude of the noise. For a moment OU noise, The initial noise constant is... For the total number of training rounds, This is the coefficient representing the influence of environmental complexity on the noise constant. For a moment The state-space entropy represents the environmental complexity. For a moment The state space, These are smoothing parameters used to adjust for the effects of state changes. This represents the change in state, used to measure the change between the current state and the previous state. For state The probability of occurrence;

[0191] In distribution network environments containing distributed energy resources, operating states and fault characteristics are highly dynamic. Traditional D4PG algorithms typically employ fixed or linearly decaying exploration noise strategies, failing to perceive real-time environmental changes. This patent designs a dynamic feedback mechanism that dynamically adjusts the exploration noise constant based on real-time changes in environmental states (e.g., load surges, distributed energy resource integration / disconnection). The advantage of this mechanism lies in achieving an intelligent adaptive balance between exploration and utilization: when the environmental state is stable, noise decreases, allowing the agent to focus on efficient detection using existing strategies; when the environment undergoes drastic changes, noise increases, prompting the agent to actively explore new decision spaces and quickly adapt to new grid topologies and operating modes. This flexibility ensures that the algorithm possesses faster adaptation speed and more stable decision-making performance in complex and constantly changing distribution network scenarios.

[0192] In improvement point 3, the generalized Huber loss function is designed by combining Wasserstein distance and Huber loss to enhance the robustness of Q-value distribution estimation;

[0193] Traditional D4PG algorithms estimate Q-value distributions and optimize strategies using KL divergence. However, the sensitivity of KL divergence to outliers and extreme cases makes it prone to instability in high-noise environments. Therefore, noise and sudden anomalies from sensor data and distributed generation information pose significant challenges to D4PG in distribution network fault detection. To enhance the fault tolerance of Q-value estimation to these outliers and noise, a generalized quantile Huber loss function is introduced to replace the KL divergence in D4PG. This improvement aims to enhance the robustness and performance of D4PG in practical tasks such as fault detection in distribution networks with active sources.

[0194] The traditional Huber loss function effectively reduces the impact of outliers on regression tasks because it uses mean squared error (MSE) for small errors and linear loss for large errors, avoiding over-penalization. Building upon this, the generalized quantile Huber loss constructs a generalized Huber loss by incorporating Wasserstein distance into the loss function: it uses Wasserstein distance to measure the difference between the target and predicted quantiles, thus more accurately capturing and accommodating noise and outliers in the distribution. The specific steps are as follows:

[0195] First, define the Q-value distributions of the target and predicted quantiles:

[0196] ;

[0197] ;

[0198] In the above formula, Let be the distribution of the target Q-value, representing the th digit of the target Q-network computation. Q value of quantile, To predict the Q-value distribution, the network for predicting the Q-value is calculated using the _____th _____. Q value of quantile, For quantiles;

[0199] Then, the Wasserstein distance is used to measure the difference between the target and predicted quantiles, where the Wasserstein distance can be expressed as:

[0200] ;

[0201] ;

[0202] In the above formula, for and The Wasserstein distance between the two quantiles is used to measure the difference between them. This is a joint distribution used to measure the difference between the target Q-distribution and the predicted Q-distribution. , Let each be a sample point in the target Q-value distribution and the predicted Q-value distribution, respectively. This represents the Euclidean distance between the target Q-value and the predicted Q-value. The total dimension of the sample points. , Sample points , of The goal of this formula is to minimize the minimum movement cost from the target distribution to the predicted distribution by selecting the optimal joint distribution, thus obtaining the sample points. , Minimize the distance between them, thereby minimizing the Wasserstein distance between the two distributions;

[0203] The impact of the difference is calculated using Huber loss. The basic logic is to use Wasserstein distance to represent the error, and then use Huber loss to judge whether the error is too large. The Huber loss function is expressed as:

[0204] ;

[0205] In the above formula, The Huber loss function penalizes larger errors, ensuring it is not excessively affected by outliers. The threshold is used to control the penalty for large errors:

[0206] Finally, combining the Wasserstein distance and the Huber loss function, we obtain the final generalized Huber loss function as follows:

[0207] ;

[0208] In the above formula, For the generalized Huber loss function, For the target Q-value distribution, To predict the Q-value distribution, For the threshold, For the selected set of quantiles, for example ;

[0209] The generalized Huber loss function combines Wasserstein distance and Huber loss. It controls the impact of large errors by calculating the difference between the target Q distribution and the predicted Q distribution, thereby optimizing the estimation of Q value and reducing noise interference.

[0210] Traditional reinforcement learning algorithms, even those using the Q-distribution D4PG, still have room for improvement in robustness when dealing with complex transient noise and sensor anomalies in power grids, particularly in real-world engineering data. This patent addresses this issue by designing a generalized Huber loss function that combines Wasserstein distance and Huber loss. This innovation achieves dual robustness: Wasserstein distance is more sensitive and stable to distribution changes when measuring Q-distribution differences; while Huber loss effectively suppresses the impact of outliers. This ensures that the agent can stably estimate the Q-value distribution during training, significantly improving the stability and reliability of the algorithm in high-noise, high-outlier real-world power grid environments, resulting in more accurate and reliable fault detection results.

[0211] 4. After training, export the optimal fault detection strategy for power distribution networks with active sources, perform offline verification, and deploy it in fault detection tasks for power distribution networks with active sources to perform intelligent monitoring of power distribution network faults, ensuring its effectiveness in real-world environments.

[0212] By employing a deep reinforcement learning (DRL) framework, the complex task of fault detection in power distribution networks with sources is transformed into an optimal decision problem. The completion of agent training signifies the successful establishment of an optimal mapping logic from power grid state to diagnostic actions. The process of establishing and implementing this logical relationship is as follows:

[0213] Logical basis: Transforming the detection task into decision elements;

[0214] First, the fault detection task is broken down into three core elements of reinforcement learning: State: robust features extracted by the (GraphGAN) + (GCN) framework, which integrates multi-source information and power grid topology, representing the agent's perception of the current power grid operating condition; Action: explicit diagnostic instructions output by the agent, such as "judge fault" or "confirm normal operation"; Reward: feedback signals obtained by the agent based on the accuracy and real-time performance of its diagnosis, which defines our "optimal" goal.

[0215] Training process: Determine the optimal decision rule;

[0216] During training, the agent continuously interacts with the power distribution network environment, and its policy network is continuously optimized. Utilizing the robust loss function and dynamic feedback mechanism unique to this patent, it learns the following key rules: under what power grid conditions, which diagnostic action should be performed to obtain the highest cumulative reward. The final result of training is the solidification of the policy network parameters, thus forming an optimal deterministic policy. This policy contains the most efficient set of decision rules for fault detection summarized by the agent from massive amounts of experience.

[0217] Detection is achieved through direct execution of the policy network.

[0218] Once training is complete, the agent switches from learning mode to inference mode and can be directly used for actual detection. When a power grid fault occurs, its characteristic information is immediately input into the agent to generate a state. This state is then directly processed by a pre-defined optimal deterministic policy, without hesitation or exploration, instantly outputting diagnostic actions based on optimal decision rules.

[0219] Therefore, the trained agent itself is a highly optimized and robust fault diagnosis expert system. It achieves intelligent fault detection precisely because it successfully establishes a direct, fast, and reliable logical mapping from power grid fault states to optimal diagnostic actions. Although there may be discrepancies between training data and actual data, the innovation of this patent is precisely to bridge this gap between simulation and reality, ensuring the robustness of the optimal deterministic strategy.

[0220] The derived optimal fault detection strategy for power distribution networks with active faults was verified offline, and the results are as follows: Figure 2 The log-scale mixing matrix diagram is shown, and the key indicators are shown in Table 1.

[0221] Table 1 Key Indicators

[0222] ;

[0223] In Table 1, TP stands for: the number of samples correctly predicted as belonging to the currently analyzed category; FN stands for: the number of samples that actually belong to the current category but were incorrectly predicted as belonging to another category by the model (i.e., false negatives); TP+FN stands for: the total number of samples that actually belong to the current category; Recall stands for: the ability of the model to avoid false negatives, i.e., the proportion of samples that actually belong to the current category that the model successfully retrieves; Precision stands for: the ability of the model to avoid false positives, i.e., the proportion of samples that the model predicts to belong to the current category that actually belong to the current category; F1-Score stands for: the harmonic mean of precision and recall, a comprehensive indicator used to simultaneously evaluate the model's recall and precision capabilities. A high F1-Score is only achieved when both precision and recall are high.

[0224] Figure 2Table 1 powerfully demonstrates the superior performance and reasonable decision-making of this invention in extremely imbalanced multi-class classification tasks. The confusion matrix intuitively reflects that the model maintains extremely high accuracy on a large sample size (9,500 cases), resulting in an overall accuracy of 98.88%. However, the true value of the model lies in its handling of fault categories: by achieving extremely high recall rates for failures (98.67%) and critical faults (93.33%), the model effectively reduces the false negative rate of high-risk faults to an extremely low level (failure FN=2, critical fault FN=10). Although this conservative strategy results in a slightly lower precision for the critical fault category (74.87%), meaning that the model tends to sacrifice some precision to avoid high-risk false negatives, this trade-off is highly consistent with engineering practice requirements in applications like power systems where safety redundancy is paramount. In contrast, the recall rate for minor faults was 75.00%, indicating that the model still faces challenges in distinguishing edge fault states. However, overall, the tables and graphs together confirm that the combination of GNN / GGN feature enhancement and the improved D4PG algorithm successfully provides a safe, efficient and robust solution for intelligent fault detection in power distribution networks with active faults.

[0225] To verify the effectiveness of the proposed scheme (GraphGAN) + (GCN) + improved D4PG, a comparative ablation experiment was conducted using base D4PG and (GGN) + (GCN) + base D4PG. The average reward convergence curves of the three models are shown below. Figure 3 As shown in the diagram, a comparison of the basic D4PG (light gray) and (GGN) + (GCN) + basic D4PG (dark gray) reveals that the (GGN) + (GCN) framework significantly improves learning efficiency through its superior feature enhancement. The robust graph structure features provided by GGN / GCN enable the agent to perceive the environment more quickly, significantly accelerating the convergence speed. Further comparison with our proposed solution (black curve) shows that our solution rapidly reaches its performance peak at approximately 500 iterations, with a convergence speed several times faster than other models, demonstrating superior performance. More importantly, its final convergence reward is approximately 510, significantly higher than that of feature enhancement. However, the final convergence reward of the unimproved (GGN) + (GCN) + basic D4PG is approximately 480. This strongly demonstrates that the triple improvements of generalized Huber loss, dynamic feedback mechanism, and multi-scale experience pool optimization not only accelerate training but also successfully push the policy quality for source-based power grid fault detection to a higher performance limit. Furthermore, the black curve exhibits the smoothest shape and least fluctuation after reaching the high-reward plateau, indicating that the improved D4PG algorithm possesses stronger robustness and stability. It effectively avoids severe oscillations during training, ensuring that the agent outputs stable and optimal fault detection decisions in complex power grid environments. In summary, the simulation results fully demonstrate the accelerating effect of GGN / GCN feature enhancement and the outstanding contributions of the improved D4PG algorithm to final decision performance and training stability.

[0226] Example 2:

[0227] like Figure 4 As shown, a power distribution network fault intelligent monitoring system with distributed energy resources includes a feature matrix generation module, a decision model construction module, an agent training module, and a fault intelligent monitoring module.

[0228] The feature matrix generation module is used to acquire multi-source information data of the distribution network containing distributed energy, extract features, and generate a node feature matrix.

[0229] The decision model construction module is used to construct a Markov decision model for source distribution network fault detection based on the node feature matrix.

[0230] The agent training module is used to train the agent using the main body of the fault intelligent detection system as the agent, based on the Markov decision model and using the improved D4PG algorithm.

[0231] The intelligent fault monitoring module is used to export the optimal fault detection strategy for power distribution networks with active sources after training, and to deploy it in the fault detection task of power distribution networks with active sources for intelligent fault monitoring of power distribution networks.

[0232] The feature matrix generation module includes a graph structure forming unit, a graph data generation unit, and a node feature extraction unit;

[0233] The graph structure forming unit is used to represent multi-source information data of a distribution network containing distributed energy resources as a graph structure;

[0234] The graph data generation unit is used to input the graph structure into the graph generative adversarial network, and use the generator and discriminator of the graph generative adversarial network to gradually optimize the generated graph data and output a new graph structure.

[0235] The node feature extraction unit is used to extract node features from a new graph structure using a graph convolutional network, and generate a node feature matrix.

[0236] In the decision model construction module, the state space of the Markov decision model is:

[0237] ;

[0238] The action space of the Markov decision model is:

[0239] ;

[0240] The reward function of the Markov decision model is:

[0241] ;

[0242] ;

[0243] ;

[0244] In the above formula, For a moment The state space, For the first The feature vector of each node This represents the total number of nodes in the distribution network. For a moment The space of motion This indicates that the device is in a normal state. This indicates that the equipment is in a minor fault condition. This indicates that the equipment is in a critical fault condition. This indicates that the device is in a faulty state. For a moment The reward function, As a time-sensitive reward, For accuracy rewards, , These are the reward weights for timely responses and slightly slower responses within the threshold range. Penalty weighting for delayed response, The time difference for fault response. It is a time constant. To accurately detect the reward weight for faults, , These are the penalty weights for false alarms and false alarms, respectively. To accurately detect the proportion of faults, This indicates the penalty for a false alarm. This indicates the penalty for failing to report.

[0245] The agent training module includes an experience storage unit, a network training unit, a dynamic adjustment unit, a network update unit, and a repetitive iteration unit;

[0246] The experience storage unit is used to define the state space, action space, and reward function based on the Markov decision model. The agent selects actions according to the current policy, records the current observations, actions, and rewards, and stores these experiences in the experience pool.

[0247] The network training unit is used to sample the experience most relevant to the current state from the experience pool using a multi-scale nearest neighbor algorithm to train the Q network and the policy network.

[0248] The dynamic adjustment unit is used to design a dynamic feedback mechanism to dynamically adjust the noise constant of the action selected by the strategy network according to changes in environmental conditions.

[0249] The network update unit is used to design the generalized Huber loss function for updating the Q-network; and to update the policy network based on the policy gradient.

[0250] The repeated iteration unit is used to repeat the experience storage unit-network update unit until the model performance is stable or the maximum number of training rounds is reached, at which point training ends.

[0251] In the network training unit, the multi-scale nearest neighbor algorithm is as follows:

[0252] ;

[0253] ;

[0254] In the above formula, The most relevant subset of experiences in the replay pool for the current state. For the first in the playback pool One experience The collection of all experiences in the replay pool. For the current experience and replay pool, the first Multiscale distance between experiences To consider the number of scales, For the first Weighting coefficients for each scale For the current experience in the first eigenvectors at various scales For the first in the playback pool The first experience in Feature vectors at various scales;

[0255] The dynamic feedback mechanism in the dynamic adjustment unit is as follows:

[0256] ;

[0257] ;

[0258] ;

[0259] In the above formula, For a moment The space of motion For a moment Deterministic actions obtained through policy network computation For a moment The noise constant, For a moment OU noise, The initial noise constant is... For the total number of training rounds, This is the coefficient representing the influence of environmental complexity on the noise constant. For a moment State space entropy, For a moment The state space, For smoothing parameters, For state The probability of occurrence;

[0260] In the network update unit, the generalized Huber loss function is:

[0261] ;

[0262] ;

[0263] ;

[0264] ;

[0265] ;

[0266] ;

[0267] In the above formula, For the generalized Huber loss function, For the target Q-value distribution, To predict the Q-value distribution, For the threshold, For the selected set of quantiles, Here is the Huber loss function. for and Wasserstein distance between them For joint distribution, , Let each be a sample point in the target Q-value distribution and the predicted Q-value distribution, respectively. For a moment The state space, For a moment The space of motion For quantiles, The total dimension of the sample points. , Sample points , of Dimensional components.

Claims

1. A method for intelligent fault monitoring in a distribution network containing distributed energy resources, characterized in that, The method includes: S1. Obtain multi-source information data of the distribution network containing distributed energy resources, extract features, and generate a node feature matrix; S2. Based on the node feature matrix, construct a Markov decision model for fault detection in power distribution networks with sources; S3. Using the main body of the fault intelligent detection system as the intelligent agent, the intelligent agent is trained based on the Markov decision model and using the improved D4PG algorithm. S4. After training, export the optimal fault detection strategy for power distribution networks with active sources and deploy it in the fault detection task of power distribution networks with active sources to carry out intelligent monitoring of power distribution network faults.

2. The intelligent fault monitoring method for a distribution network containing distributed energy resources according to claim 1, characterized in that, S3 includes: S31. Based on the state space, action space, and reward function defined by the Markov decision model, the agent selects actions according to the current policy, records the current observations, actions, and rewards, and stores these experiences in the experience pool. S32. Use the multi-scale nearest neighbor algorithm to sample the experience most relevant to the current state from the experience pool, and train the Q network and policy network. S33. Design a dynamic feedback mechanism to dynamically adjust the noise constant of the action selection by the strategy network according to changes in environmental conditions; S34. Design a generalized Huber loss function for updating the Q-network; update the policy network based on the policy gradient; S35. Repeat steps S31-S34 until the model performance is stable or the maximum number of training rounds is reached, then training ends.

3. The intelligent fault monitoring method for a distribution network containing distributed energy resources according to claim 2, characterized in that, In S32, the multi-scale nearest neighbor algorithm is as follows: ； ； In the above formula, The most relevant subset of experiences in the replay pool for the current state. For the first in the playback pool One experience The collection of all experiences in the replay pool. For the current experience and replay pool, the first Multiscale distance between experiences To consider the number of scales, For the first Weighting coefficients for each scale For the current experience in the first eigenvectors at various scales For the first in the playback pool The first experience in Feature vectors at various scales; In S33, the dynamic feedback mechanism is as follows: ； ； ； In the above formula, For a moment The space of motion For a moment Deterministic actions obtained through policy network computation For a moment The noise constant, For a moment OU noise, The initial noise constant is... For the total number of training rounds, This is the coefficient representing the influence of environmental complexity on the noise constant. For a moment State space entropy, For a moment The state space, For smoothing parameters, For state The probability of occurrence; In S34, the generalized Huber loss function is: ； ； ； ； ； ； In the above formula, For the generalized Huber loss function, For the target Q-value distribution, To predict the Q-value distribution, For the threshold, For the selected set of quantiles, Here is the Huber loss function. for and Wasserstein distance between them For joint distribution, , Let each be a sample point in the target Q-value distribution and the predicted Q-value distribution, respectively. For a moment The state space, For a moment The space of motion For quantiles, The total dimension of the sample points. , Sample points , of Dimensional components.

4. The intelligent fault monitoring method for a distribution network containing distributed energy resources according to claim 1, characterized in that, In S2, the state space of the Markov decision model is: ； The action space of the Markov decision model is: ； The reward function of the Markov decision model is: ； ； ； In the above formula, For a moment The state space, For the first The feature vector of each node This represents the total number of nodes in the distribution network. For a moment The space of motion This indicates that the device is in a normal state. This indicates that the equipment is in a minor fault condition. This indicates that the equipment is in a critical fault condition. This indicates that the device is in a faulty state. For a moment The reward function, As a time-sensitive reward, For accuracy rewards, , These are the reward weights for timely responses and slightly slower responses within the threshold range. Penalty weighting for delayed response, The time difference for fault response. It is a time constant. To accurately detect the reward weight for faults, , These are the penalty weights for false alarms and false alarms, respectively. To accurately detect the proportion of faults, This indicates the penalty for a false alarm. This indicates the penalty for failing to report.

5. The intelligent fault monitoring method for a distribution network containing distributed energy resources according to claim 1, characterized in that, S1 includes: S11. Represent the multi-source information data of the distribution network containing distributed energy as a graph structure; S12. Input the graph structure into the graph generative adversarial network, and use the generator and discriminator of the graph generative adversarial network to gradually optimize the generated graph data and output a new graph structure. S13. Use graph convolutional networks to extract node features from the new graph structure and generate a node feature matrix.

6. A smart fault monitoring system for a distribution network incorporating distributed energy resources, characterized in that, The system includes a feature matrix generation module, a decision model construction module, an agent training module, and a fault intelligent monitoring module. The feature matrix generation module is used to acquire multi-source information data of the distribution network containing distributed energy, extract features, and generate a node feature matrix. The decision model construction module is used to construct a Markov decision model for source distribution network fault detection based on the node feature matrix. The agent training module is used to train the agent using the main body of the fault intelligent detection system as the agent, based on the Markov decision model and using the improved D4PG algorithm. The intelligent fault monitoring module is used to export the optimal fault detection strategy for power distribution networks with active sources after training is completed, and then deploy it in the fault detection task of power distribution networks with active sources to carry out intelligent fault monitoring of power distribution networks.

7. The intelligent fault monitoring system for a distribution network containing distributed energy resources according to claim 6, characterized in that, The agent training module includes an experience storage unit, a network training unit, a dynamic adjustment unit, a network update unit, and a repetitive iteration unit; The experience storage unit is used for the state space, action space and reward function defined based on the Markov decision model. The agent selects actions according to the current policy, records the current observations, actions and rewards, and stores these experiences in the experience pool. The network training unit is used to sample the experience most relevant to the current state from the experience pool using a multi-scale nearest neighbor algorithm to train the Q network and the policy network. The dynamic adjustment unit is used to design a dynamic feedback mechanism to dynamically adjust the noise constant of the action selected by the strategy network according to changes in environmental conditions. The network update unit is used to design the generalized Huber loss function for updating the Q-network; and to update the policy network based on the policy gradient. The repeated iteration unit is used to repeat the experience storage unit-network update unit until the model performance is stable or the maximum number of training rounds is reached, at which point training ends.

8. The intelligent fault monitoring system for a distribution network containing distributed energy resources according to claim 7, characterized in that, In the network training unit, the multi-scale nearest neighbor algorithm is as follows: ； ； In the above formula, The most relevant subset of experiences in the replay pool for the current state. For the first in the playback pool One experience The collection of all experiences in the replay pool. For the current experience and replay pool, the first Multiscale distance between experiences To consider the number of scales, For the first Weighting coefficients for each scale For the current experience in the first eigenvectors at various scales For the first in the playback pool The first experience in Feature vectors at various scales; The dynamic feedback mechanism in the dynamic adjustment unit is as follows: ； ； ； In the above formula, For a moment The space of motion For a moment Deterministic actions obtained through policy network computation For a moment The noise constant, For a moment OU noise, The initial noise constant is... For the total number of training rounds, This is the coefficient representing the influence of environmental complexity on the noise constant. For a moment State space entropy, For a moment The state space, For smoothing parameters, For state The probability of occurrence; In the network update unit, the generalized Huber loss function is: ； ； ； ； ； ； In the above formula, For the generalized Huber loss function, For the target Q-value distribution, To predict the Q-value distribution, For the threshold, For the selected set of quantiles, Here is the Huber loss function. for and Wasserstein distance between them For joint distribution, , Let each be a sample point in the target Q-value distribution and the predicted Q-value distribution, respectively. For a moment The state space, For a moment The space of motion For quantiles, The total dimension of the sample points. , Sample points , of Dimensional components.

9. The intelligent fault monitoring system for a distribution network containing distributed energy resources according to claim 6, characterized in that, In the decision model construction module, the state space of the Markov decision model is: ； The action space of the Markov decision model is: ； The reward function of the Markov decision model is: ； ； ； In the above formula, For a moment The state space, For the first The feature vector of each node This represents the total number of nodes in the distribution network. For a moment The space of motion This indicates that the device is in a normal state. This indicates that the equipment is in a minor fault condition. This indicates that the equipment is in a critical fault condition. This indicates that the device is in a faulty state. For a moment The reward function, As a time-sensitive reward, For accuracy rewards, , These are the reward weights for timely responses and slightly slower responses within the threshold range. Penalty weighting for delayed response, The time difference for fault response. It is a time constant. To accurately detect the reward weight for faults, , These are the penalty weights for false alarms and false alarms, respectively. To accurately detect the proportion of faults, This indicates the penalty for a false alarm. This indicates the penalty for failing to report.

10. The intelligent fault monitoring system for a distribution network containing distributed energy resources according to claim 6, characterized in that, The feature matrix generation module includes a graph structure forming unit, a graph data generation unit, and a node feature extraction unit; The graph structure forming unit is used to represent multi-source information data of a distribution network containing distributed energy resources as a graph structure; The graph data generation unit is used to input the graph structure into the graph generative adversarial network, and use the generator and discriminator of the graph generative adversarial network to gradually optimize the generated graph data and output a new graph structure. The node feature extraction unit is used to extract node features from a new graph structure using a graph convolutional network, and generate a node feature matrix.

Images