An active power distribution network voltage control method against distributed photovoltaic stealth attack

Abstract

The application belongs to the field of power system automation and network security, and specifically discloses an active distribution network voltage control method resisting distributed photovoltaic concealment attack. The application firstly constructs a Markov process considering concealment attack used for training and control of deep reinforcement learning agent. Secondly, various abnormal detection methods such as spatial anomaly detection, time series anomaly detection and outlier anomaly detection are used for detection respectively, and a comprehensive anomaly score is obtained through weighting, so as to realize accurate identification of attack data and spatiotemporal correlation correction. Finally, the experience replay mechanism of sampling probability correction is used to reduce the sampling weight of the corrected sample in DRL training, and to guide the agent to preferentially learn the real physical dynamic. The application can effectively cope with concealment attack, improve the convergence stability and voltage control precision of the DRL agent, and guarantee the safe and robust operation of the active distribution network under continuous network attack.

Description

Technical Field

[0001] This invention belongs to the field of power system automation and network security technology, and specifically relates to an active distribution network voltage control method to resist covert attacks by distributed photovoltaic systems. Background Technology

[0002] While the widespread deployment of distributed photovoltaic (PV) power has promoted the development of green electricity, PV inverters suffer from multiple vulnerabilities in design, communication, and operation and maintenance. For example, the use of unencrypted traditional industrial protocols like Modbus / DNP3 and exposed physical interfaces make them highly susceptible to cyberattacks. False Data Injection Attacks (FDIA) manipulate PV active power measurements, causing deep reinforcement learning (DRL) agents to make incorrect decisions based on flawed states, leading to voltage fluctuations or even voltage exceeding limits.

[0003] Deep reinforcement learning, due to its ability to generate control strategies quickly without requiring precise physical models, has become a research hotspot for voltage / reactive power control in active power distribution networks with high photovoltaic penetration. Existing research primarily targets explicit spoofing attacks, using statistical thresholds to detect and correct abnormal deviations in measurement data.

[0004] However, stealth attacks (SA), as a more threatening form of attack, are gradually attracting attention. Stealth attackers carefully design attack vectors to simultaneously satisfy the following constraints: (1) Statistical stealth: the data after the attack falls within the historical distribution confidence interval of the normal data; (2) Temporal smoothness: the time series after the attack maintains a smooth transition characteristic similar to the normal series; (3) Physical feasibility: the attack value does not exceed the physical output limit of the photovoltaic inverter.

[0005] When an attack satisfies the above constraints, detection mechanisms based on a single statistical threshold or residual analysis will completely fail. The corrupted measurement data will be stored in an experience replay pool, continuously interfering with the subsequent training process of the DRL agent, causing a gradual degradation of the control strategy. Currently, research on DRL voltage control against such highly stealthy attacks is lacking, and there is a lack of a resilient control framework capable of effectively identifying and correcting stealthy attacks while ensuring training effectiveness. Therefore, there is an urgent need for a voltage control method that can integrate spatiotemporal feature recognition, possess attack perception and data correction capabilities, and guide the DRL agent to prioritize learning real physical dynamics, in order to improve the security robustness of active distribution networks under network attacks.

[0006] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art. Summary of the Invention

[0007] The purpose of this invention is to propose an active distribution network voltage control method to resist covert attacks on distributed photovoltaic systems. This method uses deep reinforcement learning voltage control based on spatiotemporal feature fusion identification and sampling probability correction through empirical replay, aiming to effectively resist covert attacks on distributed photovoltaic systems and thus ensure the voltage stability of the active distribution network.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: An active distribution network voltage control method for resisting covert attacks by distributed photovoltaic systems includes the following steps: Step 1. Construct a Markov process that considers covert attacks for training and controlling the deep reinforcement learning agent; Step 2. For the distributed photovoltaic measurement data at each time step, perform spatial anomaly detection, temporal anomaly detection, and outlier anomaly detection, respectively. The three anomaly detection methods are as follows: The covariance matrix is ​​calculated based on historical normal data to measure the Mahalanobis distance of the active power vector of all current distributed photovoltaic nodes, and the normalized Mahalanobis distance is defined as the spatial anomaly score. For each distributed photovoltaic node, a multi-scale time series feature set is constructed, including mean, standard deviation, skewness, kurtosis, first-order difference autocorrelation coefficient, and reconstruction error, and a comprehensive time series anomaly score is calculated. Outlier score is obtained by performing outlier anomaly detection on all distributed photovoltaic node data measured at a certain time. A comprehensive anomaly score is obtained by weighted fusion of the three anomaly detection results. Set a preset threshold for determining whether a covert attack has occurred, and determine and locate the attack based on the comprehensive anomaly score obtained by weighted fusion calculation and the preset threshold. For distributed photovoltaic nodes identified as being attacked, data correction is performed using spatiotemporal correlation. Step 3. Train the deep reinforcement learning agent using the double-delay deep deterministic policy gradient algorithm with an empirical replay mechanism that corrects sampling probability; For samples that have been corrected and stored in the experience replay pool, a low sampling probability is assigned during the initial sampling process to reduce their probability of use during updates of the Critic and Actor networks. Set an importance recovery rate so that the sampling probability of the corrected experience recovers as the number of samplings increases; Step 4. Perform voltage / reactive power control online based on the trained agent.

[0009] Furthermore, based on the above-mentioned active distribution network voltage control method for resisting distributed photovoltaic covert attacks, the present invention also proposes a computer device, including a memory and one or more processors; executable code is stored in the memory; when the processor executes the executable code, it is used to implement the steps of the above-mentioned active distribution network voltage control method for resisting distributed photovoltaic covert attacks.

[0010] Furthermore, based on the above-mentioned active distribution network voltage control method for resisting distributed photovoltaic covert attacks, the present invention also proposes a computer-readable storage medium storing a program thereon, which, when executed by a processor, is used to implement the steps of the above-mentioned active distribution network voltage control method for resisting distributed photovoltaic covert attacks.

[0011] The present invention has the following advantages: As described above, this invention proposes a voltage control method for Active Distribution Networks (ADNs) to resist stealth attacks (SA) on distributed photovoltaic (DPV) systems, addressing the voltage / var control (VVC) problem. This method employs a triple detection mechanism integrating Mahalanobis distance (spatial), multi-scale temporal features (temporal), and outlier degree to effectively identify well-constructed and disguised stealth attacks. Instead of relying on a single statistical threshold, this method mines subtle traces of attacks from multiple dimensions, overcoming the complete failure of traditional methods under SA and demonstrating high robustness against stealth attacks. Furthermore, this invention utilizes the spatial correlation between DPV nodes (similar output of neighboring nodes) and their own historical temporal correlation (periodic daily output patterns), corrected through weighted median fusion, enabling accurate data recovery. Median estimation is inherently robust to outliers, allowing contaminated data to be restored to a state closer to physical reality, providing an accurate state perception foundation for the agent. Furthermore, to improve the training effect and policy quality of deep reinforcement learning, this invention proposes an experience replay mechanism with sampling probability correction. This innovatively reduces the training weight of corrected samples, solving the problem that traditional experience replay can lead to policies deviating from real physical dynamics due to learning too much "manually corrected" data. This mechanism ensures that the agent prioritizes learning natural, unattacked interaction experiences, thus enabling it to converge to a high-performance control policy even under covert attack environments. Simultaneously, the importance recovery mechanism prevents some samples from being permanently ignored, ensuring the diversity of the experience replay pool. Through the combined effect of these mechanisms, this invention ensures that the deep reinforcement learning agent can still make correct reactive power voltage control decisions even under high-intensity, highly covert network attacks. Simulation results show that after adopting the method of this invention, the node voltage is effectively controlled within the safe range of 0.95-1.05 pu, voltage deviation and the number of voltage exceedances are significantly reduced, network active power loss is also optimized, and grid voltage stability is guaranteed. The integrated detection and data correction module proposed in this invention can be deployed as a pre-plugin within existing deep reinforcement learning control frameworks without modifying the core agent algorithm, exhibiting good compatibility and scalability. Furthermore, this method is data-driven and does not rely on a precise physical model of the power distribution network, making it particularly suitable for practical engineering scenarios with complex topologies and difficult-to-obtain model parameters. Attached Figure Description

[0012] Figure 1This is a flowchart of an active distribution network voltage control method for resisting covert attacks from distributed photovoltaic systems, as described in an embodiment of the present invention; the flowchart shows the training process of the method of the present invention. Figure 2 This is a flowchart of an active distribution network voltage control method for resisting covert attacks by distributed photovoltaic systems, as described in an embodiment of the present invention; the flowchart illustrates the control flow of the method of the present invention. Figure 3 This is a comparison chart of the reward convergence curves of the present invention and the comparative scheme under covert attacks; Figure 4 This is a three-dimensional distribution diagram of the voltage amplitude of the present invention (Solution 4) under covert attack; Figure 5 This is a voltage distribution histogram of the present invention (Solution 4) under covert attack. Detailed Implementation

[0013] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments: Example 1 To address the problems of traditional statistical detection mechanisms failing, data correctors misjudging, and DRL policy progressively degrading in existing Deep Reinforcement Learning (DRL) voltage control methods when facing carefully designed stealth attacks (SA), this invention proposes an active distribution network voltage control method to resist distributed photovoltaic (PV) stealth attacks. This method proposes an improved TD3 control framework that integrates spatiotemporal feature recognition and empirical replay with sampling probability correction. First, an SA mathematical model considering statistical stealth, temporal smoothness, and physical feasibility constraints is constructed. Second, an ensemble detection based on Mahalanobis distance, multi-scale temporal features, and dispersion is designed to achieve accurate identification and spatiotemporal correlation correction of attack data. Finally, an empirical replay mechanism with sampling probability correction is proposed to reduce the sampling weight of corrected samples in DRL training, guiding the agent to prioritize learning real physical dynamics. Figure 1 and Figure 2 As shown in this embodiment, the active distribution network voltage control method for resisting covert attacks by distributed photovoltaic systems includes the following steps: Step 1. Construct a Markov process that considers covert attacks for training and controlling deep reinforcement learning agents.

[0014] A Markov decision process model considering covert attacks is constructed, defining the agent's state space, action space, and reward function; the state space includes the tampered observations of the distributed photovoltaic active power.

[0015] This observation is composed of the actual power and an attack vector satisfying the constraints of statistical concealment, temporal smoothness, and physical feasibility. Considering the case of a concealed attack on a photovoltaic system in an active distribution network, it is assumed that each photovoltaic node... At any moment The actual active power is The attacker manipulated the measurement data, causing the system to observe a power value of: ;in This is the amount injected for the attack.

[0016] To achieve stealth, attackers need to satisfy the following constraints: I. Statistical concealment constraint.

[0017] The observed values ​​are within the confidence interval of normal data, as shown in the following formula: ;in, Let be the photovoltaic power observed at node i at time t. These are the mean and standard deviation of the historical data, respectively. Here is the confidence coefficient. Usually, 2 or 3 is chosen.

[0018] II. Temporal smoothness constraints.

[0019] The sequence retains temporal correlation after the attack, as shown in the following formula: .

[0020] in , Let be the actual photovoltaic power at node i at time t+1 and time t, respectively. This is the smoothness tolerance threshold, and its value ranges from 0.02 to 0.01; Let be the photovoltaic power observed at node i at time t+1.

[0021] III. Physical feasibility constraints.

[0022] The attack value does not exceed the physical limit of photovoltaics, as shown in the following formula: ;in Let be the rated photovoltaic capacity of node i.

[0023] Under the influence of the aforementioned attacks, the agent's observation state space for: .

[0024] in For the number of photovoltaic nodes, This represents the total number of nodes. For node voltage, For node load.

[0025] The action space is the reactive power output command of each photovoltaic inverter.

[0026] reward function Designed as follows: .in For reference voltage, For network active power loss, These are the weighting coefficients. , This indicates the maximum and minimum values ​​of the specified node voltage.

[0027] Step 2. For the distributed photovoltaic measurement data at each time step, spatial anomaly detection, temporal anomaly detection, and outlier anomaly detection are performed respectively. Then, a weighted fusion of the three anomaly detection results is performed to obtain a comprehensive anomaly score.

[0028] A preset threshold is set to determine whether a covert attack has occurred, and the attack is determined and located based on the comprehensive anomaly score obtained by weighted fusion calculation and the preset threshold.

[0029] For distributed photovoltaic nodes identified as being attacked, data correction is performed using spatiotemporal correlation.

[0030] Step 2.1. Spatial anomaly detection.

[0031] The covariance matrix is ​​calculated based on historical normal data to measure the Mahalanobis distance of the active power vector of all current distributed photovoltaic nodes, identify outliers at the spatial level, and define the normalized Mahalanobis distance as the spatial anomaly score.

[0032] Assuming that under normal operating conditions, the active power of each photovoltaic unit within the same time step follows a multivariate normal distribution.

[0033] For observation vector Its distance from the normally distributed Mahalanobis is defined as: .

[0034] in, This represents the active power observation vector for all distributed photovoltaic nodes at the current time step. and These are from historical normal data (normal training data) The mean vector and covariance matrix estimated in ( ).

[0035] .

[0036] Under the assumption of a multivariate normal distribution, the square of the Mahalanobis distance follows a sequence with degrees of freedom of . The chi-square distribution.

[0037] Spatial anomaly score Defined as the normalized Mahalanobis distance, the formula is: .in This represents the total number of distributed photovoltaic nodes. For degrees of freedom The 99th quantile of the chi-square distribution. When This indicates the existence of anomalies at the spatial level.

[0038] Step 2.2. Timing anomaly detection.

[0039] For each distributed photovoltaic node, a multi-scale time series feature set is constructed, including mean, standard deviation, skewness, kurtosis, first-order difference autocorrelation coefficient, and reconstruction error, and a comprehensive time series anomaly score is calculated.

[0040] To capture anomalous patterns in attacks over time, multi-scale temporal features are defined. Let the set of time scales be... (For example (This corresponds to 2 hours, 4 hours, and 8 hours, assuming a time resolution of 10 minutes).

[0041] For the i-th distributed photovoltaic node and the time scale Multi-scale temporal feature vectors It includes mean characteristics, standard deviation characteristics, skewness characteristics, kurtosis characteristics, first-order difference autocorrelation coefficient, and reconstruction error characteristics.

[0042] Mean characteristics The formula is as follows: ;in This represents the actual photovoltaic power at node i at time t-k+1.

[0043] Standard deviation characteristics The formula is as follows: .

[0044] Skewness characteristics This is used to measure the asymmetry of the distribution, and its formula is as follows: .

[0045] Kurtosis characteristics The formula for , used to measure the tail thickness of the distribution, is as follows: .

[0046] First-order difference autocorrelation coefficient This is used to reflect the smoothness of a time series, and its formula is as follows: .

[0047] definition ; Represents the difference sequence The mean of is given by the formula: .

[0048] Reconstruction error characteristics It is used for linear prediction, and its formula is as follows: ;in For the basis of the previous The predicted value at time 1, which uses the mean feature. replace. This represents the measured photovoltaic power at node i at time t.

[0049] Integrate the time series anomaly score Defined as the weighted sum of the feature vectors of each node at each scale, the formula is as follows: ; ;in To represent different time scales Represents all time scales The set, This is the scale weight vector. For normalization function, This indicates the number of distributed photovoltaic (PV) systems.

[0050] Step 2.3. Outlier anomaly detection.

[0051] Spatial and temporal features are concatenated and input into the isolated forest model to calculate outlier scores.

[0052] Isolation forest is an anomaly detection method based on tree ensembles. Its core idea is that anomalous samples are more easily isolated in the feature space, meaning they have shorter paths in randomly partitioned decision trees. For each time step... Construct global feature vector .

[0053] .

[0054] in For spatial anomaly scores Comprehensive time series anomaly score Mean Mahalanobis distance Multi-scale average power Multiscale power standard deviation The concatenated global feature vector.

[0055] ; ; .

[0056] in To adjust the sliding window size, This represents the actual photovoltaic power at time tk.

[0057] For the sample Outlier score Defined as: .

[0058] in Let be the expected value of the path length of the sample in the isolated forest. The total number of training samples, This is the normalization factor for the average path length. ;in For the first A harmonic number, , It is the Euler-Macheroni constant. is the Euler-Marschroni constant.

[0059] Outlier score The range of values ​​is The closer the value is to 1, the more abnormal it is.

[0060] Step 2.4. Attack detection and location.

[0061] The spatial anomaly score, the comprehensive temporal anomaly score, and the outlier anomaly score obtained from steps 2.1 to 2.3 are fused together to obtain the comprehensive anomaly score.

[0062] A preset threshold is set to determine whether a covert attack has occurred. When the overall anomaly score exceeds the preset threshold, it is determined that a covert attack has occurred, and the specific distributed photovoltaic node being attacked is located using the 3σ principle.

[0063] Comprehensive abnormal score The calculation formula is: .

[0064] in For spatial outlier scores, To synthesize the time series anomaly score, The outlier score is the score indicating the degree of abnormality. It is a minimum-maximum normalization function. Let be the weighting coefficient, satisfying This was determined through cross-validation.

[0065] The attack determination criteria are: .

[0066] in This represents an attack detection flag; it is 1 if an attack is detected, and 0 if no attack is detected. The overall threshold is the combined anomaly score of the samples that were not attacked. The 99th percentile.

[0067] Set of indexes of attacked nodes for: ;in and They are nodes exist Historical mean and standard deviation at any given time.

[0068] Step 2.5. For distributed photovoltaic nodes identified as being attacked, perform data correction using spatiotemporal correlation.

[0069] Among them, based on the power output data of other distributed photovoltaic nodes spatially related to the node and the power output data of the node at the same historical moment, a corrected value is generated by weighted fusion to replace the original observation value.

[0070] Upon detection of an attack, a data correction mechanism is activated to recover an estimated value of the true photovoltaic active power from the observed data. The formula for calculating the correction value based on spatiotemporal correlation proposed in this invention is as follows: .

[0071] in For nodes exist The power correction at any given moment; For nodes Other spatially related distributed photovoltaic node sets, A collection of historical moments. These are weighting coefficients. ; This indicates taking the median.

[0072] in, The value of can be dynamically adjusted according to the strength of spatial and temporal correlation.

[0073] Corrected observation status for: .

[0074] The expected deviation between the corrected state and the true state satisfies: .

[0075] Step 3. Set up an experience replay mechanism with sampling probability correction, and train the deep reinforcement learning agent using a dual-delay deep deterministic policy gradient algorithm with the experience replay mechanism with sampling probability correction. Figure 1 As shown.

[0076] During training, samples that have been corrected in step 2 and stored in the experience replay pool are given a low sampling probability during the initial sampling process to reduce their probability of use in the updates of the Critic and Actor networks.

[0077] At the same time, an importance recovery rate is set so that the sampling probability of the corrected experience recovers as the number of samplings increases.

[0078] The Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm is a deep reinforcement learning method that employs an Actor-Critic architecture and is specifically designed to handle continuous action space problems.

[0079] This invention introduces an empirical replay mechanism based on sampling probability correction.

[0080] Let the empirical sample be The state at the time of collection is: .in, These are the values ​​corrected by the detector. Because the corrected values ​​replace the actual attack values, they deviate from the true state.

[0081] If all samples are learned with equal weights, the agent's policy will shift towards the corrected state distribution, reducing its adaptability to the real environment. Therefore, this invention defines empirical samples. Importance weights are .

[0082] in The penalty coefficient is for low importance. ; For indicator functions; if the sample If the state contains the data corrected in step 2, then Otherwise It is 0.

[0083] sample sampling probability for: .

[0084] in As a priority index, For the sample The number of times samples have been taken.

[0085] In this embodiment, the recovery rate coefficient is used. For example, the possible values ​​are (0, 0.3).

[0086] This design results in low-importance samples having a low initial sampling probability. As the number of samplings increases, their importance gradually recovers, eventually leading to all samples approaching equal probability sampling. The importance recovery rate is expressed as: .

[0087] in, This represents the total number of samples in the experience replay pool.

[0088] In addition, the actions of the agent are specially processed during the training process. By decoupling the reactive power output of the distributed photovoltaic inverter into two independent positive and negative stages, the action space of the agent is mapped to a fixed range of feasible reactive power. Then, the active distribution network voltage control is realized by combining the dual-delay deep deterministic strategy gradient algorithm.

[0089] Meanwhile, the specific measures to decouple the reactive power output of the distributed photovoltaic inverter into two independent positive and negative stages, and map the action space of the intelligent agent to a fixed feasible range of reactive power, are as follows: , Indicates the upper limit of time; .

[0090] in This represents the reactive power injected into the active power distribution network. Represents the control variables of the intelligent agent. This indicates the rated reactive power of the DPV. This indicates the apparent power of the photovoltaic inverter. This represents the active power output of the inverter at time t.

[0091] By decoupling the reactive power output of the DPV inverter into two independent positive and negative stages, the action space of the agent is mapped to a fixed feasible range of reactive power. Then, the voltage control of the active distribution network is achieved by combining the Double-Delay Deep Deterministic Policy Gradient Algorithm (TD3).

[0092] The present invention employs the aforementioned methods to significantly improve the training efficiency and control stability of the intelligent agent in the method of the present invention. Then, the traditional network parameter training process is introduced, which will not be elaborated here as it is not an innovation of the present invention.

[0093] Step 4. Perform online voltage / reactive power control based on the trained agent, such as... Figure 2 As shown.

[0094] Specifically, firstly, real-time operating data of the active distribution network is collected. After detection and correction in step 2, this data is input into a trained dual-delay deep deterministic policy gradient agent. The agent outputs reactive power control commands for each distributed photovoltaic inverter, achieving real-time regulation of the distribution network voltage. Based on the trained agent, the following control flow is executed online: I. Collect real-time operating data of the active power distribution network, including voltage amplitude at each node, photovoltaic output, and load demand; II. Input the collected data into the spatiotemporal feature fusion detection and correction module in step B to obtain the corrected state. ; III. The corrected state Input the trained Actor network To obtain normalized control actions ; IV. Map the control actions to the actual reactive power output commands of each photovoltaic inverter: ;in, and These are the forward and reverse reactive capacities of the inverter, respectively. , These represent the control commands given by the intelligent agent, corresponding to forward and reverse actions, respectively.

[0095] V. The instructions are sent to each distributed photovoltaic inverter for execution, thereby achieving real-time adjustment of the distribution network voltage.

[0096] Meanwhile, the actions of the intelligent agent are also specially processed in the control process. By decoupling the reactive power output of the distributed photovoltaic inverter into two independent positive and negative stages, the action space of the intelligent agent is mapped to a fixed feasible range of reactive power. Then, the active distribution network voltage control is realized by combining the dual-delay deep deterministic strategy gradient algorithm.

[0097] In addition, to illustrate the effectiveness of the method of the present invention, the following experimental procedure is also provided.

[0098] This invention validates the effectiveness of its method on an IEEE 33-node standard distribution network system. The photovoltaic system is installed at nodes 10, 13, 15, 18, 20, 22, 23, 25, 27, 29, 30, and 32, totaling 12 photovoltaic nodes. All photovoltaic inverters are configured with a capacity of 1 MVA. The reference voltage level is 12.66 kV, the root node voltage standard is 1.00 pu, and the safe voltage limit is 0.95 pu to 1.05 pu. The deep reinforcement learning environment is configured for 3000 training epochs, with each epoch using photovoltaic and load data from a random day. The TD3 algorithm parameter settings are shown in Table 1.

[0099] Table 1 Algorithm Parameters

[0100] The probability of an attack occurring in each training round is 50%. Attacks occur between 10 and 15 time steps per day (corresponding to approximately a 1-hour window, assuming a time resolution of 10 minutes). The number of photovoltaic units in a single attack is randomly selected between 5 and 12. The attack intensity employs a progressive control strategy, with an initial intensity of 60% and a final intensity of 100%, linearly increasing with the number of attack steps and incorporating ±10% random perturbation. Attack value generation is achieved by learning the statistical distribution characteristics of normal photovoltaic power output data, ensuring that the attack value meets statistical concealment constraints. Spatial anomaly detection parameter settings: the threshold for spatial anomaly detection is set based on the 99th quantile of the chi-square distribution, i.e. Simultaneously, a rule of three standard deviations is introduced as a basis for dynamic adjustment. Temporal anomaly detection employs a multi-scale sliding window mechanism, with the window scale set set as follows: These correspond to time spans of 2 hours, 4 hours, and 8 hours, respectively. The number of isolated trees in the Isolation Forest algorithm is set to... The maximum height of each tree is set to The abnormal score threshold is set to the 99th percentile of the training sample scores, i.e. .

[0101] The weighting coefficients for the three anomaly detection methods were determined through cross-validation. The overall threshold is also taken as the 99th percentile of the overall score of normal samples. The low importance penalty coefficient of the empirical replay mechanism with sampling probability correction is set to... Importance recovery rate is set to The detailed comparison scheme is shown in Table 2 below.

[0102] Table 2 Comparison of Schemes

[0103] Figure 3The paper presents a comparison of the reward convergence curves of four schemes under covert attacks. It can be seen that Scheme 4 (the method of this invention) significantly outperforms the other schemes in convergence performance. Specifically, Scheme 4 has the fastest initial reward value increase and the shortest number of iterations required for convergence; the convergence trajectory of Scheme 4 is particularly smooth with minimal fluctuations, demonstrating superior algorithm stability; Scheme 4 achieves the highest final convergence reward value (close to -10), significantly better than Scheme 1 (approximately -60) and Scheme 2 (approximately -20); although Scheme 3 converges slightly faster than Scheme 4, its reward curve still exhibits some oscillations in the mid-to-late stages, making its stability weaker than Scheme 4. This result verifies the synergistic effect between the spatiotemporal feature detection correction module and the sampling probability correction experience replay mechanism proposed in this invention. The detection correction module provides the agent with accurate state input, while the sampling probability correction experience replay mechanism ensures that the agent prioritizes learning high-quality real experience. Together, they achieve efficient and stable training convergence.

[0104] Figure 4 , Figure 5 The spatiotemporal characteristics and data distribution characteristics of ADN voltage data under the control of Scheme 4 are shown. Table 3 shows the statistical results of voltage control under covert attacks for the four comparative schemes.

[0105] Table 3 Statistical Results

[0106] It can be seen that Scheme 4 performs best in maintaining voltage stability, with a voltage standard deviation of only 0.0085 pu, significantly lower than other schemes, indicating the smallest voltage fluctuation range. The mean voltage change between adjacent time steps is 0.0019 pu, and the standard deviation is 0.0022 pu, both of which are the smallest, indicating smooth control action and stable adjustment process. None of the schemes experienced voltage over-limit, but the voltage amplitude of Scheme 4 is closer to the rated value of 1.0 pu in most nodes and scenarios, with a significantly lower deviation than other schemes. Simulation results show that the error between the data and the true value after spatiotemporal correlation correction is significantly reduced, providing reliable state input for deep reinforcement learning agents. The method of this invention achieves robust voltage control under highly stealthy network attacks through the synergistic effect of spatiotemporal feature fusion detection, data correction, and empirical replay mechanism of sampling probability correction.

[0107] The simulation results show that the method of the present invention can effectively deal with covert attacks, significantly improve the convergence stability and voltage control accuracy of the DRL agent, and ensure the safe and robust operation of the active distribution network under continuous network attacks.

[0108] Example 2 This embodiment 2 describes an active distribution network voltage control system for resisting covert attacks by distributed photovoltaics. This system is based on the same concept as the active distribution network voltage control method for resisting covert attacks by distributed photovoltaics in embodiment 1 above.

[0109] The active distribution network voltage control system for resisting covert attacks by distributed photovoltaic systems in this embodiment includes the following modules: A Markov decision process model building module considering covert attacks is used to construct the Markov process used for training and controlling deep reinforcement learning agents. The stealth attack integrated detection and data correction module is used to perform spatial anomaly detection, temporal anomaly detection, and outlier anomaly detection on the distributed photovoltaic measurement data at each time step. The covariance matrix is ​​calculated based on historical normal data to measure the Mahalanobis distance of the active power vector of all current distributed photovoltaic nodes, and the normalized Mahalanobis distance is defined as the spatial anomaly score. For each distributed photovoltaic node, a multi-scale time series feature set is constructed, including mean, standard deviation, skewness, kurtosis, first-order difference autocorrelation coefficient, and reconstruction error, and a comprehensive time series anomaly score is calculated. Outlier score is obtained by performing outlier anomaly detection on all distributed photovoltaic node data measured at a certain time. Then, a weighted fusion of the three anomaly detection results is performed to obtain a comprehensive anomaly score; Set a preset threshold for determining whether a covert attack has occurred, and determine and locate the attack based on the comprehensive anomaly score obtained by weighted fusion calculation and the preset threshold. For distributed photovoltaic nodes identified as being attacked, data correction is performed using spatiotemporal correlation. An improved TD3 algorithm training module with sample probability correction experience replay is used to set up an experience replay mechanism with sample probability correction and to train a deep reinforcement learning agent using a dual-delay deep deterministic policy gradient algorithm with sample probability correction experience replay mechanism. During training, samples that have been corrected in step 2 and stored in the experience replay pool are given a low sampling probability during the initial sampling process to reduce their probability of use when updating the Critic and Actor networks. At the same time, an importance recovery rate is set so that the sampling probability of the corrected experience recovers as the number of samplings increases; And a voltage / reactive power control module, used to perform online voltage / reactive power control based on a trained agent.

[0110] It should be noted that any content not mentioned in the above-described functional modules of the system described in this embodiment can be referred to the step description of the corresponding method in Embodiment 1 above, and will not be repeated in detail here.

[0111] Example 3 This embodiment 3 describes a computer device including a memory and one or more processors. Executable code is stored in the memory. When the processor executes the executable code, it implements the steps of the active distribution network voltage control method for resisting distributed photovoltaic covert attacks described in embodiment 1 above.

[0112] In this embodiment, the computer device can be any device or apparatus with data processing capabilities, and will not be described in detail here.

[0113] Example 4 This embodiment 4 describes a computer-readable storage medium storing a program that, when executed by a processor, implements the steps of the active distribution network voltage control method for resisting distributed photovoltaic covert attacks in embodiment 1 above.

[0114] The computer-readable storage medium can be an internal storage unit of any device or apparatus with data processing capabilities, such as a hard disk or memory, or an external storage device of any device with data processing capabilities, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc.

[0115] Of course, the above description is only a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. It should be noted that any equivalent substitutions or obvious modifications made by those skilled in the art under the guidance of this specification fall within the scope of this specification and should be protected by the present invention.

Claims

1. An active distribution network voltage control method to resist covert attacks from distributed photovoltaic systems, characterized in that, Includes the following steps: Step 1. Construct a Markov process that considers covert attacks for training and controlling the deep reinforcement learning agent; Step 2. For the distributed photovoltaic measurement data at each time step, perform spatial anomaly detection, temporal anomaly detection, and outlier anomaly detection, respectively. The three anomaly detection methods are as follows: The covariance matrix is ​​calculated based on historical normal data to measure the Mahalanobis distance of the active power vector of all current distributed photovoltaic nodes, and the normalized Mahalanobis distance is defined as the spatial anomaly score. For each distributed photovoltaic node, a multi-scale time series feature set is constructed, including mean, standard deviation, skewness, kurtosis, first-order difference autocorrelation coefficient, and reconstruction error, and a comprehensive time series anomaly score is calculated. Outlier score is obtained by performing outlier anomaly detection on all distributed photovoltaic node data measured at a certain time. A comprehensive anomaly score is obtained by weighted fusion of the three anomaly detection results. Set a preset threshold for determining whether a covert attack has occurred, and determine and locate the attack based on the comprehensive anomaly score obtained by weighted fusion calculation and the preset threshold. For distributed photovoltaic nodes identified as being attacked, data correction is performed using spatiotemporal correlation. Step 3. Train the deep reinforcement learning agent using the double-delay deep deterministic policy gradient algorithm with an empirical replay mechanism that corrects sampling probability; For samples that have been corrected and stored in the experience replay pool, a low sampling probability is assigned during the initial sampling process to reduce their probability of use during updates of the Critic and Actor networks. Set an importance recovery rate so that the sampling probability of the corrected experience recovers as the number of samplings increases; Step 4. Perform voltage / reactive power control online based on the trained agent.

2. The active distribution network voltage control method for resisting covert attacks from distributed photovoltaic systems according to claim 1, characterized in that, Step 2 specifically involves: Step 2.

1. Perform spatial anomaly detection; Step 2.

2. Perform timing anomaly detection; Step 2.

3. Perform outlier detection; Step 2.

4. Attack detection and location; The spatial anomaly score, comprehensive temporal anomaly score, and outlier anomaly score obtained from steps 2.1 to 2.3 are combined and weighted to calculate the comprehensive anomaly score. A preset threshold is set to determine whether a covert attack has occurred. When the overall anomaly score exceeds the preset threshold, it is determined that a covert attack has occurred, and the specific distributed photovoltaic node being attacked is located using the 3σ principle. The preset threshold is set as the 99th percentile of the overall anomaly score of the unattacked samples; Step 2.

5. For distributed photovoltaic nodes identified as being attacked, perform data correction using spatiotemporal correlation; Specifically, based on the power output data of other distributed photovoltaic nodes spatially related to this node and the power output data of this node at the same historical moment, a corrected value is generated through weighted fusion to replace the original observation value.

3. The active distribution network voltage control method for resisting covert attacks from distributed photovoltaic systems according to claim 2, characterized in that, In step 2.1, the Mahalanobis distance The calculation formula is: ;in This represents the active power observation vector for all distributed photovoltaic nodes at the current time step. and These are the mean vector and covariance matrix estimated from historical normal data, respectively. Spatial anomaly score Defined as the normalized Mahalanobis distance, the formula is: ； in This represents the total number of distributed photovoltaic nodes. For degrees of freedom The 99th percentile of the chi-square distribution.

4. The active distribution network voltage control method for resisting covert attacks from distributed photovoltaic systems according to claim 2, characterized in that, In step 2.2, for the i-th distributed photovoltaic node and time scale... Multi-scale temporal feature vectors It includes mean characteristics, standard deviation characteristics, skewness characteristics, kurtosis characteristics, first-order difference autocorrelation coefficient, and reconstruction error characteristics; Mean characteristics The formula is as follows: ;in This represents the actual photovoltaic power at node i at time t-k+1; Standard deviation characteristics The formula is as follows: ； Skewness characteristics The formula is as follows: ； Kurtosis characteristics The formula is as follows: ； First-order difference autocorrelation coefficient The formula is as follows: ； definition , Represents the difference sequence The mean of is given by the formula: ; Reconstruction error characteristics The formula is as follows: ;in For the basis of the previous The predicted value at time 1, which uses the mean feature. replace; This represents the measured photovoltaic power at node i at time t; Integrate the time series anomaly score Defined as the weighted sum of the temporal feature vectors of each node at each scale, the formula is as follows: ； ;in Representing different time scales Represents all time scales The set, This is the scale weight vector. For normalization function, This indicates the number of distributed photovoltaic (PV) systems.

5. The active distribution network voltage control method for resisting covert attacks from distributed photovoltaic systems according to claim 2, characterized in that, In step 2.3, the outlier score Defined as: ； in Spatial outlier score Comprehensive time series anomaly score , based on the average Mahalanobis distance Multi-scale average power Multiscale power standard deviation The concatenated global feature vector is expressed by the following formula: ； ； ； ； in To adjust the sliding window size; This represents the actual photovoltaic power at time tk; This represents the expected path length of the sample in the isolated forest. definition The total number of training samples, This is a normalization factor for the average path length; ； in For the first A harmonic number, , It is the Euler-Marcheroni constant.

6. The active distribution network voltage control method for resisting covert attacks from distributed photovoltaic systems according to claim 2, characterized in that, In step 2.4, the overall abnormal score is calculated. The calculation formula is: ； in For spatial outlier scores, To synthesize the time series anomaly scores, The outlier score is the score indicating the degree of abnormality. It is a minimum-maximum normalization function. For the weighting coefficients, satisfying ; The attack determination criteria are: ； in This represents an attack detection flag; it is 1 if an attack is detected, and 0 if no attack is detected. The comprehensive threshold, i.e., the preset threshold, is the comprehensive anomaly score of the samples that have not been attacked. The 99th percentile; Set of indexes of attacked nodes for: ; in, and They are nodes exist Historical mean and standard deviation at any given time.

7. The active distribution network voltage control method for resisting covert attacks from distributed photovoltaic systems according to claim 1, characterized in that, In step 2.5, the formula for calculating the correction value based on spatiotemporal correlation is as follows: ； in For nodes exist The power correction at any given moment; For nodes Other spatially related distributed photovoltaic node sets, A collection of historical moments. These are weighting coefficients. ; This indicates taking the median; Corrected observation status for: .

8. The active distribution network voltage control method for resisting covert attacks from distributed photovoltaic systems according to claim 1, characterized in that, In step 3, the importance weight of the empirical replay mechanism for sampling probability correction Defined as: ;in The penalty coefficient is for low importance. ; For indicator functions; if the sample If the state contains the data corrected in step 2, then Otherwise =0; sample sampling probability for: ; in As a priority index, For the sample Number of times sampled, This is the recovery rate coefficient; Importance recovery rate is expressed as: ;in This represents the total number of samples in the experience replay pool.

9. A computer device comprising a memory and one or more processors; executable code is stored in the memory; characterized in that, When the processor executes the executable code, it performs the steps of the active distribution network voltage control method for resisting distributed photovoltaic covert attacks as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a program stored thereon, characterized in that, When executed by a processor, the program is used to implement the steps of the active distribution network voltage control method for resisting distributed photovoltaic covert attacks as described in any one of claims 1 to 8.

Images