Power distribution network relay protection artificial intelligence setting method based on reinforcement learning algorithm

Abstract

The invention relates to the technical field of power system relay protection, particularly discloses a power distribution network relay protection artificial intelligence setting method based on a reinforcement learning algorithm, and aims to solve the technical problem that a traditional off-line setting method is difficult to adapt to dynamic operation characteristics of a complex power distribution network containing a high-proportion distributed power supply. According to the method, a reinforcement learning model for power distribution network relay protection setting is constructed, a power grid operation state is mapped into a state space, and setting parameter adjustment is mapped into an action space; training an intelligent agent through a depth deterministic strategy gradient algorithm; and online dynamic optimization of the setting parameters is realized. According to the technical scheme, the adaptive capacity, the action selectivity and the fault removal speed of relay protection in the complex power distribution network can be remarkably improved, and the engineering feasibility and the long-term stability of a setting strategy are guaranteed.

Description

Technical Field

[0001] This invention belongs to the field of power system relay protection technology, specifically relating to an artificial intelligence setting method for distribution network relay protection based on reinforcement learning algorithm. Background Technology

[0002] With the deepening of smart grid construction, the accuracy and adaptability of distribution network relay protection settings are receiving increasing attention as a key link in ensuring the safe and stable operation of the power system. Traditional relay protection setting methods mainly rely on manual experience and offline calculations, setting parameters based on fixed operating modes and preset fault scenarios. These methods are insufficient to cope with the increasingly complex topological changes in distribution networks, the bidirectional power flow brought about by distributed power source access, and the dynamic operating characteristics such as load fluctuations.

[0003] Among them, the AI-based setting method for distribution network relay protection based on reinforcement learning algorithms aims to learn the optimal setting strategy autonomously through continuous interaction between the agent and the power grid environment in order to adapt to changing operating conditions.

[0004] However, existing technologies face multiple challenges in applying reinforcement learning to relay protection setting.

[0005] First, the state space of the distribution network is large and continuous, making it difficult for traditional reinforcement learning algorithms to efficiently explore and converge to the global optimal strategy. They are prone to getting stuck in local optima or unstable training.

[0006] Secondly, protection settings must strictly meet power system security constraints (such as selective coordination and sensitivity verification), but existing methods lack an effective embedding mechanism for rigid engineering constraints, which may lead to the generation of infeasible or non-compliant setting schemes during the learning process.

[0007] Furthermore, real-world distribution networks are subject to numerous uncertainties and disturbances (such as communication delays, measurement noise, and equipment aging). Existing models typically assume a completely observable and deterministic environment, making it difficult to maintain the reliability and generalization ability of the setting strategy in real-world scenarios. Therefore, there is an urgent need for a reinforcement learning-based artificial intelligence setting method for distribution network relay protection that can balance learning efficiency, constraint satisfaction, and environmental stability. Summary of the Invention

[0008] The purpose of this invention is to provide an artificial intelligence setting method for distribution network relay protection based on reinforcement learning algorithm, which can effectively solve the problems in the background art.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0010] An artificial intelligence setting method for distribution network relay protection based on reinforcement learning algorithm includes the following specific steps:

[0011] Step 1: Construct a reinforcement learning model for the setting problem of relay protection in distribution networks. Map the operating state of the distribution network to the state space of the reinforcement learning agent, map the setting parameter adjustment actions of the relay protection device to the action space, and quantify the correctness, selectivity, speed and system stability of the protection actions as reward functions.

[0012] Step 2: Design a state-space representation mechanism, collect real-time operation data of the distribution network, including node voltage, branch current, power flow, distributed generation output and load level, and construct a multi-dimensional continuous state vector through feature extraction and normalization.

[0013] Step 3: Design an action space constraint mechanism, define the adjustable range of relay protection setting parameters, including the action current setting value, time setting value and action boundary of directional element of overcurrent protection, and embed hard constraints for safe operation of power system.

[0014] Step 4: Design a reward function optimization mechanism, comprehensively considering the accuracy of protection actions, the timing of coordination with upstream and downstream protection, fault clearing speed, and the impact on system transient stability, and construct a multi-objective weighted reward function;

[0015] Step 5: Train the reinforcement learning agent using the deep deterministic policy gradient algorithm or its improved variant. Through continuous interaction between the agent and the power distribution network simulation environment, iteratively update the parameters of the policy network and the value network until the policy converges.

[0016] Step 6: Deployment and online setting. Integrate the trained reinforcement learning agent into the distribution network relay protection setting system, and dynamically output the optimal relay protection setting parameters based on the real-time collected power grid status data.

[0017] Preferably, in step 1, the reinforcement learning model adopts the Markov decision process framework, the state space dimension is adaptively adjusted according to the distribution network scale, the minimum dimension is 128, and the action space is defined as a continuous space, the dimension of which is proportional to the number of protection devices to be tuned.

[0018] Preferably, the state-space representation mechanism in step 2 specifically includes: acquiring measurement data in real time through a power distribution network monitoring and data acquisition system, with a sampling frequency of not less than 2 kHz; performing wavelet transform denoising on the original data to retain the main fault characteristic frequency bands; using principal component analysis algorithm to reduce the dimensionality of high-dimensional features and retaining principal components with a variance contribution rate greater than or equal to 95%; and finally constructing a state vector that includes key operating indicators such as voltage deviation, current phase, power factor, frequency fluctuation, and distributed power source penetration rate.

[0019] Preferably, the action space constraint mechanism in step 3 is implemented through a projection layer, which is integrated into the output of the strategy network and maps the original action vector output by the strategy network to the range of executable setting parameters that meet the actual engineering requirements. The adjustable range is set according to the technical specifications of the relay protection device and the operation procedures of the distribution network. The overcurrent protection action current setting adjustment range is 1.2 times to 10 times the rated current, and the time setting adjustment range is 0.1 seconds to 2 seconds.

[0020] Preferably, the specific components of the reward function optimization mechanism in step 4 include: a basic reward item, used to reward correct identification and fault clearing, with a reward value of +10 for correct action and a penalty value of -20 for false action or failure to act; a selective reward item, used to reward coordinated protection actions, with a reward of +5 for successful regional selective isolation and a penalty of -15 for over-tripping; a speed reward item, rewarding rapid fault clearing, with a reward of +1 for every 0.01 seconds reduction in action time and a penalty of -10 for exceeding the maximum allowable time limit; and a stability reward item, rewarding positive impact on system transient stability, with a reward of +1 to +5 based on the system voltage recovery and frequency stability after the fault.

[0021] Preferably, in step 5, the training of the reinforcement learning agent adopts an improved deep deterministic policy gradient algorithm. The improvement is that priority sampling with experience replay mechanism is introduced, and a dual value network structure is combined to alleviate the overestimation problem. Both the policy network and the value network adopt a deep neural network structure with three hidden layers, with 512, 256 and 128 neurons in each layer, respectively. The activation function adopts the modified linear unit. During training, the learning rate is set to 0.001, the discount factor is set to 0.99, and the batch size is set to 64.

[0022] Preferably, in step 5, the power distribution network simulation environment is built based on an electromagnetic transient simulation program, which can simulate various fault types including three-phase short circuit, two-phase short circuit, and single-phase ground fault, and consider the fault characteristics of distributed power sources and the dynamic changes of load; the simulation step size is no more than 50 microseconds to ensure the fidelity of the training data.

[0023] Preferably, the deployment and online setting process in step 6 includes: deploying a reinforcement learning agent inference module in the distribution network control center; the module is connected to the protection devices of each substation through a high-speed communication network; real-time status data is preprocessed and input into the trained strategy network, and the network outputs setting parameter suggestions within 10 milliseconds; after confirmation by the safety verification module, the new setting parameters are sent to the corresponding protection devices through the protection setting management system.

[0024] Preferably, it also includes a model update and adaptive learning module; this module continuously monitors the online tuning performance of the agent, and triggers the model retraining process when the cumulative performance index drops by more than 5% of a preset threshold; the retraining uses the latest power grid operation data and fault cases to fine-tune the agent to adapt to long-term changes in power grid structure or operating characteristics.

[0025] Preferably, the method also integrates a simulation and testing platform for closed-loop testing before strategy deployment; the platform can reproduce historical failure scenarios and generate extreme operating conditions to verify the reliability and stability of the agent tuning strategy under various boundary conditions, with a test pass rate of over 99.9%.

[0026] In summary, this application has the following beneficial technical effects compared to the prior art:

[0027] By constructing a reinforcement learning model that integrates the dynamic operating characteristics of the distribution network and designing refined state representations, action constraints, and multi-objective reward mechanisms, adaptive and online optimization adjustment of relay protection setting parameters is achieved. This method effectively overcomes the dependence of traditional offline setting methods on fixed operating modes and significantly improves the adaptability, selectivity, and speed of the protection system in complex distribution networks with a high proportion of distributed power sources. Meanwhile, embedded engineering constraints ensure the feasibility of the setting scheme, while the continuous online learning mechanism ensures the long-term stability of the strategy in the face of grid evolution and uncertain disturbances. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the overall technical solution architecture of the artificial intelligence setting method for distribution network relay protection based on reinforcement learning algorithm proposed in this invention;

[0029] Figure 2 This is a schematic diagram of the core principle framework of reinforcement learning-driven relay protection setting in this invention; Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0031] Currently, in the field of distribution network relay protection setting, traditional methods rely on manual experience and offline calculations, which are difficult to adapt to the dynamic operating characteristics of modern distribution networks caused by distributed power source access, frequent topology changes and load fluctuations. Especially in scenarios with a high proportion of renewable energy penetration, existing setting strategies have significant shortcomings in terms of selectivity, speed and stability.

[0032] To address the aforementioned technical problems, this invention proposes an artificial intelligence setting method for distribution network relay protection based on reinforcement learning algorithms. By constructing a reinforcement learning framework that integrates state perception, action constraints, and multi-objective reward mechanisms, adaptive online optimization of setting parameters is achieved, and this method is applied to the artificial intelligence setting method for distribution network relay protection based on reinforcement learning algorithms.

[0033] refer to Figure 1 The overall technical architecture of the distribution network relay protection artificial intelligence setting method based on reinforcement learning algorithm in this application includes a distribution network real-time data acquisition layer, a state representation and preprocessing module, a reinforcement learning intelligent agent core engine, an action constraint and safety verification unit, a setting parameter distribution and execution layer, and a model update and adaptive learning feedback loop.

[0034] Among them, the real-time data acquisition layer of the distribution network obtains raw measurement data such as node voltage, branch current, power flow, distributed power output and load level through the distribution network monitoring and data acquisition system;

[0035] The state representation and preprocessing module performs denoising, normalization, and dimensionality reduction on the original data to generate standardized state vectors;

[0036] The core engine of the reinforcement learning agent outputs tuning action suggestions based on this state vector;

[0037] The action constraint and safety verification unit ensures that the output parameters meet the power system engineering specifications;

[0038] The setting parameters are sent to the execution layer, which then writes the verified parameters into the protection device.

[0039] The model update and adaptive learning feedback loop continuously monitors online performance and triggers a retraining mechanism to maintain the long-term effectiveness of the strategy.

[0040] refer to Figure 2 This diagram illustrates the core principle framework of reinforcement learning-driven relay protection setting in this invention. The core principle framework of reinforcement learning-driven relay protection setting in this invention is modeled using a Markov decision process. Specifically, it includes a state space S, an action space A, a state transition function T, a reward function R, and a policy π.

[0041] At each decision time t, the reinforcement learning agent observes the grid state s_t∈S, selects an action a_t∈A (i.e., the adjustment amount of the tuning parameters) according to the policy π(a_t|s_t). After the environment (i.e., the distribution network simulation or the actual system) executes the action, it transitions to the new state s_{t+1} and feeds back the reward r_t=R(s_t,a_t,s_{t+1}). By maximizing the expected cumulative discounted reward, the agent learns the optimal policy π*.

[0042] Furthermore, the logical process of co-optimization of state representation, action constraints, and multi-objective rewards is as follows: the state vector is constructed through feature engineering and then input into the policy network; the policy network outputs the original action vector; this vector is mapped to the action space through the projection layer; after the action is executed, multi-dimensional reward signals are calculated and fed back synchronously; the experience data is stored in the priority experience replay buffer; the training process uses a dual-value network structure to jointly optimize the policy and value functions.

[0043] The above-mentioned artificial intelligence setting method for distribution network relay protection based on reinforcement learning algorithm includes the following steps: Step 1: Construct a reinforcement learning model for the setting problem of distribution network relay protection, map the operation state of the distribution network to the state space of the reinforcement learning agent, map the setting parameter adjustment action of the relay protection device to the action space, and quantify the performance index of the protection action into a reward function.

[0044] Step 2: Design a state-space representation mechanism, collect real-time operation data of the power distribution network, and construct a multi-dimensional continuous state vector through feature extraction and normalization.

[0045] Step 3: Design an action space constraint mechanism, define the adjustable range of relay protection setting parameters, including the action current setting value, time setting value and action boundary of directional element of overcurrent protection, and embed hard constraints for safe operation of power system.

[0046] Step 4: Design a reward function optimization mechanism and construct a multi-objective weighted reward function;

[0047] Step 5: Train the reinforcement learning agent using the deep deterministic policy gradient algorithm or its improved variant. Through continuous interaction between the agent and the power distribution network simulation environment, iteratively update the parameters of the policy network and the value network until the policy converges.

[0048] Step 6: Deployment and online setting. Integrate the trained reinforcement learning agent into the distribution network relay protection setting system, and dynamically output the optimal relay protection setting parameters based on the real-time collected power grid status data.

[0049] In step 1, a reinforcement learning model is constructed for the setting problem of distribution network relay protection. The operating state of the distribution network is mapped to the state space of the reinforcement learning agent, the setting parameter adjustment action of the relay protection device is mapped to the action space, and the correctness, selectivity, speed and system stability of the protection action are quantified into reward functions.

[0050] Specifically, in step 1, the reinforcement learning model adopts the Markov decision process framework. The state space dimension is adaptively adjusted according to the distribution network scale, with a minimum dimension of 128. The action space is defined as a continuous space, and its dimension is proportional to the number of protection devices to be tuned.

[0051] For example, in a typical 10 kV distribution network containing 20 feeder output protection devices, the operating space dimension is 60, and each protection device corresponds to three adjustable parameters: operating current setting, time setting, and directional element boundary angle.

[0052] The state space is constructed based on observable variables of the power grid to ensure the Markov property of state transitions. At the same time, the dimensionality adaptation mechanism avoids the decrease in training efficiency caused by state redundancy in small-scale networks or the loss of information caused by insufficient dimensions in large-scale networks.

[0053] In step 2 of the above-mentioned artificial intelligence setting method for distribution network relay protection based on reinforcement learning algorithm, a state space representation mechanism is designed to collect real-time operation data of the distribution network, including node voltage, branch current, power flow, distributed power output and load level. Through feature extraction and normalization processing, a multi-dimensional continuous state vector is constructed.

[0054] Specifically, the state-space representation mechanism in step 2 includes: acquiring measurement data in real time through the distribution network monitoring and data acquisition system, with a sampling frequency of not less than 2 kHz; performing wavelet transform denoising on the original data to retain the main fault characteristic frequency bands; using principal component analysis algorithm to reduce the dimensionality of high-dimensional features and retaining principal components with a variance contribution rate greater than or equal to 95%; and finally constructing a state vector that includes key operating indicators such as voltage deviation, current phase, power factor, frequency fluctuation, and distributed power source penetration rate.

[0055] Extreme simulations show that the original measurement data is first synchronously acquired at a frequency of 2 kHz, forming a time series matrix X∈R^{N×T}, where N is the number of monitoring points and T is the sampling window length (usually 200 milliseconds, i.e., 400 sampling points). A 5-level decomposition is performed on each channel signal x_i(t) using the db4 wavelet basis. During reconstruction, only the detail coefficients from the 3rd to 5th levels are retained, effectively filtering out high-frequency noise and low-frequency drift.

[0056] Subsequently, statistical characteristics within a sliding window are calculated for the denoised data, including mean, variance, peak value, kurtosis, and harmonic distortion rate, forming an initial feature set F∈R^{N×15}. This feature set is then normalized by Z-score and input into the principal component analysis model, retaining principal components with a cumulative variance contribution rate ≥95%, ultimately outputting a state vector s∈R^{d}, where d≥128. For example, in a test system containing 8 distributed photovoltaic power stations and 12 load nodes, the initial feature dimension is 300. After principal component analysis, 132 principal components are retained, forming a 132-dimensional state vector. The first 50 principal components primarily characterize voltage stability, the middle 50 characterize power flow directionality, and the last 32 characterize the output fluctuation of distributed power sources.

[0057] In step 3 of the above-mentioned artificial intelligence setting method for distribution network relay protection based on reinforcement learning algorithm, an action space constraint mechanism is designed to define the adjustable range of relay protection setting parameters, including the action current setting value, time setting value and action boundary of directional element of overcurrent protection, and embed hard constraints for safe operation of power system.

[0058] Specifically, the action space constraint mechanism in step 3 is implemented through a projection layer, which is integrated into the output of the strategy network. The projection layer maps the original action vector output by the strategy network to the range of executable setting parameters that meet the actual engineering requirements. The adjustable range is set according to the technical specifications of the relay protection device and the operation procedures of the distribution network. The overcurrent protection action current setting adjustment range is 1.2 times to 10 times the rated current, and the time setting adjustment range is 0.1 seconds to 2 seconds.

[0059] The mathematical implementation of the projection layer is an element-wise linear transformation and truncation operation. Let the policy network output the original action vector a_raw∈R^{m}, where m is the action dimension. For the i-th action component a_raw,i, if its corresponding action current is constant, then the mapping formula is:

[0060] a i =clip(I rated,i ×(1.2+8.8×σ(a raw,i )),I rated,i ×1.2,I rated,i ×10)

[0061] Where σ(·) is the Sigmoid function, and I_{\text{rated},i} is the rated current corresponding to the i-th protection device. For the time setpoint component, the mapping formula is:

[0062] a j =clip(0.1+1.9×σ(a) raw,j ),0.1,2.0)

[0063] The boundary angles of the orientation elements are mapped to the [-90°, 90°] interval using the arctangent function. This projection layer ensures that all output actions are strictly within the engineering feasible region, eliminating illegal tuning schemes that violate selective matching or sensitivity verification at the source. Furthermore, the projection layer uses a straight-through estimator to process gradients during backpropagation, guaranteeing the differentiability of the training process.

[0064] In step 4 of the above-mentioned artificial intelligence setting method for distribution network relay protection based on reinforcement learning algorithm, a reward function optimization mechanism is designed to comprehensively consider the accuracy of protection actions, the timing of coordination with upstream and downstream protection, the fault clearing speed, and the impact on system transient stability, and to construct a multi-objective weighted reward function.

[0065] Specifically, the reward function optimization mechanism in step 4 includes: a basic reward item, which is used to reward the correct identification and removal of faults. The reward value for correct action is positive 10, and the penalty value for false action or refusal to act is negative 20.

[0066] Selective reward items are used to reward the coordination of protection actions. Successfully achieving selective isolation of a region earns a reward of +5, while an over-level trip is penalized with a penalty of -15.

[0067] The speed bonus rewards rapid fault clearing. A bonus of +1 is awarded for every 0.01 seconds reduction in action time, and a penalty of -10 is imposed for exceeding the maximum allowable time limit.

[0068] The stability bonus rewards positive impacts on system transient stability, with a reward of +1 to +5 based on the system voltage recovery and frequency stability after a fault. The complete expression for the bonus function R is the sum of all sub-items:

[0069] R = R base +R selectivity +R speed +R stability

[0070] R_base is determined based on whether the protection device is within the fault area and operates correctly; R_selectivity determines whether there is an over-level trip by comparing the actual trip sequence with the preset coordination logic diagram; R_speed is calculated based on the difference between the fault clearing time t_clear and the preset time limit t_max, specifically R_speed = max(0,(t_max-t_clear) / 0.01) - 10 × I(t_clear>t_max), where I(·) is the indicator function; R_stability is calculated based on the voltage recovery rate V_rec and frequency deviation Δf within 200 milliseconds after fault clearing. If V_rec ≥ 90% and |Δf| ≤ 0.2 Hz, then R_stability = 5; if V_rec ≥ 85% and |Δf| ≤ 0.5 Hz, then R_stability = 3; otherwise, it is 1.

[0071] This multi-objective reward mechanism guides the agent to consider multiple engineering performance aspects during the exploration process through quantitative indicators, avoiding policy bias caused by optimizing a single objective.

[0072] In step 5 of the above-mentioned artificial intelligence setting method for distribution network relay protection based on reinforcement learning algorithm, a reinforcement learning agent is trained. The deep deterministic policy gradient algorithm or its improved variant is used. Through continuous interaction between the agent and the distribution network simulation environment, the parameters of the policy network and value network are iteratively updated until the policy converges.

[0073] Specifically, in step 5, the reinforcement learning agent is trained using an improved deep deterministic policy gradient algorithm. The improvement is that priority sampling with an experience replay mechanism is introduced, and a dual-value network structure is combined to alleviate the overestimation problem. Both the policy network and the value network use a deep neural network structure with three hidden layers, with 512, 256 and 128 neurons per layer, respectively. The activation function is a modified linear unit.

[0074] During training, the learning rate was set to 0.001, the discount factor to 0.99, and the batch size to 64. In step 5, the distribution network simulation environment was built based on an electromagnetic transient simulation program, capable of simulating various fault types, including three-phase short circuits, two-phase short circuits, and single-phase ground faults, while also considering the fault characteristics of distributed power sources and dynamic load changes; the simulation step size was no greater than 50 microseconds to ensure the fidelity of the training data.

[0075] The training process is as follows: First, initialize the policy network μ(s|θ^μ) and two value networks Q1(s,a|θ^{Q1}) and Q2(s,a|θ^{Q2}). In each training cycle, the agent interacts with the simulation environment to generate an experience tuple (s_t,a_t,r_t,s_{t+1}). This tuple is stored in a priority experience replay buffer with a priority p_t = |r_t + γmin(Q1',Q2') - Q(s_t,a_t)| + ε, where Q1' and Q2' are the outputs of the target network, and ε is a small constant.

[0076] Sample batch data from the buffer according to priority; calculate the target value y_t=r_t+γmin(Q1'(s_{t+1},μ'(s_{t+1})),Q2'(s_{t+1},μ'(s_{t+1}))); update the two value networks respectively to minimize the mean square error; update the policy network to maximize Q1(s,μ(s)); periodically soft update the target network parameters.

[0077] The simulation environment is built using the PSCAD / EMTDC platform, and includes detailed primary equipment models (transformers, lines, switches), secondary protection logic, and distributed power inverter control models. Fault types are randomly generated according to probability distribution: three-phase short circuits account for 30%, two-phase short circuits account for 25%, and single-phase grounding accounts for 45%.

[0078] The load model employs a hybrid model combining dynamic induction motors and constant impedance, with distributed power sources providing reactive power support during fault periods according to low-voltage ride-through requirements. The simulation step size is set to 50 microseconds to ensure accurate capture of transient processes. Training continues until the policy's average cumulative reward fluctuation across 1000 consecutive test scenarios is less than 1%, and the selectivity accuracy stabilizes above 99%.

[0079] In step 6 of the above-mentioned artificial intelligence setting method for distribution network relay protection based on reinforcement learning algorithm, the training reinforcement learning agent is integrated into the distribution network relay protection setting system, and the optimal relay protection setting parameters are dynamically output according to the real-time collected power grid status data.

[0080] Specifically, step 6, the deployment and online setting process, includes: deploying a reinforcement learning agent inference module in the distribution network control center; this module is connected to the protection devices of each substation through a high-speed communication network; real-time status data is preprocessed and then input into the trained strategy network, which outputs setting parameter suggestions within 10 milliseconds; after confirmation by the safety verification module, the new setting parameters are sent to the corresponding protection devices through the protection setting management system.

[0081] The inference module employs a lightweight neural network inference engine (such as TensorRT) to convert the trained policy network into an optimized inference model, ensuring that the latency for a single inference operation does not exceed 8 milliseconds on a standard industrial server (such as an Intel Xeon Silver 4210 CPU). The safety verification module performs dual verification: first, it checks whether the output parameters are within the feasible region defined by the projection layer (redundancy verification); second, it verifies through fast power flow calculation whether the new tuning values ​​will cause malfunctions under normal operating conditions. Parameters are only allowed to be issued if both verifications pass.

[0082] The communication network adopts the IEC61850-9-2LE protocol and uses fiber optic Ethernet to achieve low-latency (<5 milliseconds) data transmission between the control center and the substation. Setting parameters are issued using an incremental update mode, transmitting only the changed values ​​to reduce communication load.

[0083] Furthermore, this invention includes a model update and adaptive learning module. This module continuously monitors the online tuning performance of the agent, and triggers a model retraining process when the cumulative performance index drops by more than 5% of a preset threshold. The retraining process uses the latest power grid operation data and fault cases to fine-tune the agent to adapt to long-term changes in the power grid structure or operating characteristics. Performance indicators include the accuracy of protection actions after online tuning, average action time, and system stability score.

[0084] This module summarizes the daily operation logs and calculates the average performance over a sliding window (e.g., 30 days). If the current average drops by more than 5% compared to the historical best average, retraining is initiated. Retraining employs a transfer learning strategy, using the original model weights as initial values, and fine-tuning is performed in an augmented simulation environment that includes the latest power grid topology, equipment parameters, and historical fault data. The amount of training data is typically 20% of the original training set, and the training period is shortened to 30% of the original period.

[0085] Furthermore, the method described in this invention also integrates a simulation and testing platform for conducting closed-loop testing before strategy deployment. This platform can reproduce historical failure scenarios and generate extreme operating conditions to verify the reliability and stability of the agent tuning strategy under various boundary conditions, with a test pass rate requirement of over 99.9%.

[0086] The platform includes a historical fault database, an extreme scenario generator, and automated test scripts. The historical fault scenarios cover all recorded faults from the past 5 years, totaling over 1200 cases. Extreme scenarios include N-2 faults, islanded operation of high-penetration distributed power sources, and local tuning under complete communication interruption. The test process is automated, running each scenario 10 times to evaluate policy stability. The passing standard is defined as: a correct protection action rate ≥ 99.9% across all test scenarios, no cascading tripping, and an average action time not exceeding 0.3 seconds. Policy versions that fail the test must not be deployed.

[0087] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A power distribution network relay protection artificial intelligence setting method based on a reinforcement learning algorithm, characterized in that: The method comprises the following specific steps: Step 1: constructing a reinforcement learning model for the setting problem of the relay protection of a power distribution network, mapping the operating state of the power distribution network to the state space of a reinforcement learning agent, mapping the adjustment action of the setting parameter of the relay protection device to the action space, and quantifying the performance index of the protection action as a reward function; Step 2: designing a state space representation mechanism, collecting real-time operating data of the power distribution network, and constructing a multi-dimensional continuous state vector through feature extraction and normalization processing; Step 3: designing an action space constraint mechanism, defining the adjustable range of the setting parameters of the relay protection, including the action current setting value of the overcurrent protection, the time setting value, and the action boundary of the directional element, and embedding the hard constraints for safe operation of the power system; Step 4: designing a reward function optimization mechanism, constructing a multi-objective weighted reward function; Step 5: training the reinforcement learning agent, using the deep deterministic policy gradient algorithm or its improved variants, iteratively updating the parameters of the policy network and the value network through continuous interaction between the agent and the simulation environment of the power distribution network until the policy converges; Step 6: deploying and online setting, integrating the trained reinforcement learning agent into the relay protection setting system of the power distribution network, and dynamically outputting the optimal setting parameters of the relay protection according to the real-time collected state data of the power grid.

2. The power distribution network relay protection artificial intelligence setting method based on a reinforcement learning algorithm according to claim 1, characterized in that: The reinforcement learning model adopts a Markov decision process framework, the dimension of the state space is adaptively adjusted according to the scale of the power distribution network, the minimum dimension is 128, and the action space is defined as a continuous space, the dimension of which is proportional to the number of protection devices to be set.

3. The power distribution network relay protection artificial intelligence setting method based on a reinforcement learning algorithm according to claim 1, characterized in that: The state space representation mechanism comprises: obtaining measurement data at a sampling frequency of not less than 2 kHz through a power distribution network monitoring and data acquisition system; performing wavelet transform denoising processing on the original data to retain the main fault feature frequency band; using a principal component analysis algorithm to reduce the dimension of the high-dimensional features, and retaining principal components with a variance contribution rate of greater than or equal to 95%; and finally constructing a state vector containing voltage deviation, current phase, power factor, frequency fluctuation, and distributed power penetration rate.

4. The power distribution network relay protection artificial intelligence setting method based on a reinforcement learning algorithm according to claim 1, characterized in that: The action space constraint mechanism is realized through a projection layer integrated at the output end of the policy network, which maps the original action vector output by the policy network to the executable setting parameter range that meets the engineering practice; the action current setting value of the overcurrent protection is adjusted in the range of 1.2 times to 10 times of the rated current, and the time setting value is adjusted in the range of 0.1 seconds to 2 seconds.

5. The power distribution network relay protection artificial intelligence setting method based on a reinforcement learning algorithm according to claim 1, characterized in that: The reward function optimization mechanism comprises: a basic reward item, with a positive reward value of 10 for correct action and a negative penalty value of 20 for misoperation or refusal to operate; a selective reward item, with a positive reward of 5 for successfully achieving regional selective isolation and a negative penalty of 15 for over-level tripping; a speed reward item, with a positive reward of 1 for each 0.01 second reduction in action time, and a negative penalty of 10 for exceeding the maximum allowed time limit; a stability reward item, giving a reward of 1 to 5 according to the voltage recovery and frequency stability of the system after the fault.

6. The power distribution network relay protection artificial intelligence setting method based on a reinforcement learning algorithm according to claim 1, characterized in that: The improved deep deterministic policy gradient algorithm is used to train the reinforcement learning agent, and the improvements include the introduction of a priority experience replay sampling mechanism and a double value network structure. Both policy network and value network adopt deep neural networks with 3 hidden layers, each with 512, 256 and 128 neurons respectively, and rectified linear unit as activation function; training parameters include learning rate 0.001, discount factor 0.99, batch size 64.

7. The power distribution network relay protection artificial intelligence setting method based on a reinforcement learning algorithm according to claim 1, characterized in that: The power distribution network simulation environment is based on electromagnetic transient simulation program, can simulate three-phase short circuit, two-phase short circuit and single-phase ground fault types, and consider the fault characteristics of distributed power supply and dynamic changes of load; the simulation step is not greater than 50 microseconds.

8. The power distribution network relay protection artificial intelligence setting method based on a reinforcement learning algorithm according to claim 1, characterized in that: The deployment and online setting process comprises: Deploying the reinforcement learning intelligent agent reasoning module in the power distribution network control center and connecting with each substation protection device through high-speed communication network; Real-time state data is input into the policy network after preprocessing, and the setting parameter suggestion is output within 10 milliseconds; After being confirmed by the security verification module, the new setting parameters are issued to the corresponding protection device through the protection setting value management system.

9. The power distribution network relay protection artificial intelligence setting method based on a reinforcement learning algorithm according to claim 1, characterized in that: It also includes a model updating and adaptive learning module, which continuously monitors the online setting performance of the intelligent agent, and triggers the model retraining process when the cumulative performance index decreases by more than 5% of the preset threshold; The retraining uses the latest power grid operation data and fault cases to fine-tune the intelligent agent.

10. The power distribution network relay protection artificial intelligence setting method based on a reinforcement learning algorithm according to claim 1, characterized in that: It also includes a simulation and test platform for closed-loop testing before policy deployment, which can reproduce historical fault scenarios and generate extreme operating conditions, and verify the reliability and stability of the setting strategy under boundary conditions.

Images