SVG multi-modal adaptive control method and system based on deep reinforcement learning
Patent Information
- Application Number
- CN202611035272.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-13
- Publication Date
- 2026-10-09
AI Technical Summary
然而，这种架构在复杂电网环境下暴露出以下关键技术问题：第一，状态辨识存在快速性与准确性的根本矛盾
[0066]1、本发明，对高频暂态序列和低频趋势序列进行并行特征提取，高频采样获得的高频特征表示，能够在识别微秒级前向传播中的电网状态，无需显式建模和长时间数据累积；低频采样获得的低频特征表示，能够提供秒级的电网结构变化。通过融合门向量，可以动态分配两个通道的信息权重，实现了准确辨识电网状态与快速响应统一。
Smart Images
Figure CN122890448A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of reactive power compensation control technology, and in particular to an SVG multimodal adaptive control method and system based on deep reinforcement learning. Background Technology
[0002] Static var generators (SVG) are core equipment in flexible AC transmission systems, widely used in industrial power distribution networks to perform tasks such as reactive power compensation, voltage support, and power quality management. As power grid structures become increasingly complex, the operating conditions faced by SVG exhibit strong randomness, rapid time-varying characteristics, and multi-timescale coupling.
[0003] Traditional SVG control methods primarily employ a hierarchical architecture, decoupling sensing, decision-making, and execution into independent modules. At the sensing layer, grid conditions are identified using methods such as phase-locked loops (PLLs) and online impedance estimation. At the decision layer, switching between operating modes such as constant voltage, constant reactive power, and constant power factor is achieved based on preset thresholds and hysteresis comparators. At the execution layer, a proportional-integral (PI) controller tracks the commanded current. However, this architecture exposes the following key technical problems in complex grid environments: First, there is a fundamental contradiction between speed and accuracy in state identification. Traditional online impedance estimation methods require injecting characteristic harmonics or accumulating hundreds of milliseconds of disturbance data to obtain a reliable grid strength estimate, while protection functions such as fault ride-through require millisecond-level responses, making the two incompatible in terms of time scale. Simplifying the identification algorithm to pursue speed can easily lead to misjudging load fluctuations as changes in grid structure, resulting in malfunctions in the control strategy. Second, coordination across multiple time scales is difficult. Mode switching decisions occur on a second-level scale, current inner-loop control occurs on a millisecond-level scale, while adaptive parameter adjustment may occur on a hundred-millisecond scale. These control loops at different time scales are coupled together, which can easily induce mid-frequency oscillations (2~20Hz) in weak power grids or high-penetration renewable energy scenarios. Traditional decoupling design methods are unable to eliminate such oscillations from a mechanistic perspective. Third, fixed mode switching logic has inherent flaws. Threshold-based hard switching inevitably introduces hysteresis design, but the tuning of hysteresis width and switching delay is highly dependent on expert experience and cannot cover all operating condition combinations. Under boundary conditions, frequent mode switching itself becomes a source of disturbance, causing voltage fluctuations and command jumps. Fourth, the reliability and safety of deep learning models in power electronic control lack quantitative guarantees. Although data-driven methods show advantages in complex pattern recognition, their black-box nature makes the model's behavior unpredictable under unseen operating conditions, lacking analytical safety criteria such as gain margin and phase margin in traditional control theory.
[0004] Therefore, there is an urgent need for an SVG control method that can simultaneously achieve rapid and accurate identification, smooth modal transition, and reliable quantification capabilities. Summary of the Invention
[0005] The purpose of this invention is to provide an SVG multimodal adaptive control method based on deep reinforcement learning, overcoming the shortcomings of existing technologies.
[0006] The technical solution proposed in this invention is: an SVG multimodal adaptive control method based on deep reinforcement learning, the method comprising:
[0007] The three-phase voltage and three-phase current of the target access point are obtained, and a multi-source local observation vector sequence is generated using the obtained three-phase voltage and three-phase current. A multi-time-scale state sequence is constructed using the multi-source local observation vector sequence.
[0008] Temporal features are extracted from the multi-timescale state sequence to generate high-frequency feature representations and low-frequency feature representations. The high-frequency feature representations and low-frequency feature representations are then fused by weighted dimensionally to generate a fused feature vector.
[0009] The integrated mean of reactive current command is generated using the fused feature vector, and the cognitive uncertainty variance and random uncertainty variance of the integrated mean are quantified.
[0010] The trust factor is calculated based on the cognitive uncertainty variance, and the integrated mean and adaptive safety base value are weighted and mixed using the trust factor to obtain the corrected reactive current command.
[0011] The modified reactive current command is sequentially subjected to rate limiting and flexible smoothing to obtain the final reactive current command and output it.
[0012] Preferably, the step of acquiring the three-phase voltage and three-phase current of the target access point, and generating a multi-source local observation vector sequence using the acquired three-phase voltage and three-phase current, includes:
[0013] Real-time acquisition of three-phase voltage and three-phase current, and DC-side voltage at the SVG access point;
[0014] Performing Clark and Park transformations on the three-phase voltages and currents yields the voltage axis components and voltage values in a synchronous rotating coordinate system. Axis components, current Axis components, current Axial components;
[0015] Utilizing voltage Axis components, voltage Axis components, current Axis components, current The instantaneous reactive power at the connection point is obtained by calculating the axis components;
[0016] Extracting the phase angle of the positive sequence component of the grid voltage and grid frequency ;
[0017] Constructing local observation vectors ,in, This is the normalized connection point voltage amplitude. The normalized instantaneous reactive power at the access point. This is the normalized DC voltage deviation. The normalized power grid frequency, The modulation ratio, Normalized current Axial components;
[0018] The process of constructing a multi-timescale state sequence using the multi-source local observation vector sequence includes:
[0019] With the first sampling period For intervals, the nearest The local observation vectors are arranged in chronological order to construct a high-frequency transient sequence. ; express The local observation vector at time t;
[0020] Maintain a local storage capacity of A long-history circular buffer stores the most recent Local observation vectors;
[0021] With the second sampling period Extract at intervals from the long history circular buffer. A low-frequency trend sequence is constructed from local observation vectors at equal time intervals. .
[0022] Preferably, the step of extracting temporal features from the multi-timescale state sequence to generate high-frequency feature representations and low-frequency feature representations, and then performing a dimension-wise weighted fusion of the high-frequency feature representations and low-frequency feature representations to generate a fused feature vector, includes:
[0023] High-frequency transient sequences are extracted using the Long Short-Time Memory (LSTM) algorithm. The hidden state at the last time step is used as a high-frequency feature representation. , used to indicate the type, intensity, and duration of a transient event;
[0024] Extracting low-frequency trend sequences using the LSTM algorithm The hidden state at the last time step is used as a low-frequency feature representation. This is used to indicate the drift of the power grid operating point and changes in a weak power grid.
[0025] The high-frequency feature representation and the low-frequency feature representation are mapped to the same dimension to obtain the projection vector. and ;
[0026] Calculate the fusion gate vector ,in for Activation function Indicates to Perform vector concatenation; For fusion matrix; For fusion bias;
[0027] Through gate vector pairs and Perform fusion and output fused feature vector. ,in This represents element-wise multiplication. It is a vector consisting entirely of 1s;
[0028] Preferably, the step of generating an integrated mean of reactive current commands using the fused feature vectors and quantifying the cognitive uncertainty variance and random uncertainty variance of the integrated mean includes:
[0029] Construct an integrated decision network, the integrated decision network comprising A policy sub-model; the policy sub-model is a pre-trained multilayer perceptron;
[0030] The fused feature vectors are obtained through the strategy sub-model. Distributed parameters mapped to reactive current commands; the distributed parameters include the mean. and logarithmic standard deviation ;
[0031] The standard deviation is obtained through exponential transformation. ;
[0032] Calculate the arithmetic mean of the outputs of all policy sub-models as the ensemble mean. ;
[0033] Calculate the variance of the output mean of all policy sub-models with respect to the ensemble mean, as the variance of cognitive uncertainty. ;
[0034] Calculate the arithmetic mean of the variances of the outputs of all strategy sub-models, as the variance of random uncertainty. .
[0035] Preferably, the step of calculating the trust factor based on the cognitive uncertainty variance, and using the trust factor to weight and mix the integrated mean and the adaptive safety baseline to obtain the corrected reactive current command, includes:
[0036] Calculate the trust factor ,in To understand the standard deviation of uncertainty, The default positive penalty coefficient;
[0037] Generate adaptive safety baseline value ,in The voltage amplitude at the connection point. This is the voltage reference value. The safety droop factor is less than the rated control gain. This is a preset bias term;
[0038] Calculate the corrected reactive current command ;
[0039] Physical boundary constraints are applied to the modified reactive current command, including a reactive current limit based on the SVG rated reactive current and a maximum reactive current limit dynamically adjusted based on the DC voltage margin.
[0040] Preferably, the step of calculating the trust factor based on the cognitive uncertainty variance further includes:
[0041] Calculate the penalty factor for random uncertainty ,in The standard deviation of random uncertainty The preset normal noise threshold, Less than A positive penalty coefficient;
[0042] Update the trust factor to ,in The initial trust factor is determined solely by cognitive uncertainty.
[0043] Preferably, the step of sequentially performing rate limiting and flexible smoothing processing on the modified reactive current command to obtain the final reactive current command and outputting it includes:
[0044] Calculate the instruction changes in the current control cycle. The instruction changes Less than the maximum allowable change , The maximum permissible rate of change of reactive current;
[0045] Obtain current power grid status information; if the current power grid is a weak grid, reduce... ;
[0046] Adjusted maximum allowable change Among them, the derating factor for weak power grids ;
[0047] if If the reactive current command for the current control cycle remains unchanged, then the reactive current command for the current control cycle remains unchanged. ;otherwise, ;
[0048] right A first-order low-pass filter is used to obtain the final reactive current command. ;
[0049] The final reactive current command is sent to the inner current loop PI controller for execution, enabling the SVG output reactive current tracking. ;
[0050] Calculate the gain adjustment factor based on the power grid status information. The range of the gain adjustment factor is: Between 1 and 1; where the minimum gain is... ;
[0051] Adaptive adjustment of the proportional gain of the inner current loop and integral gain ;in The parameters of the proportional-integral PI controller are tuned for rated operating conditions.
[0052] Preferred options also include:
[0053] The decision reliability index is calculated from the cognitive uncertainty variance, the robustness margin is calculated from the output of the value assessment network in the integrated decision network, and a comprehensive health status assessment is performed by combining the current tracking residual and the modulation ratio margin, triggering a graded response, specifically as follows:
[0054] Calculate the decision reliability index ,in This is a preset scale factor;
[0055] Obtain the set of Q-values obtained by evaluating the ensemble mean in the current state of all value assessment networks in the ensemble decision network, and take the 10th percentile as the robustness margin. ;
[0056] Calculate current tracking residual ,in This is the actual feedback current;
[0057] Calculate the modulation ratio margin, and determine a saturation warning when the modulation ratio is close to 1.
[0058] A comprehensive state assessment is performed by combining the aforementioned decision reliability index, robustness margin, current tracking residual, and modulation ratio margin.
[0059] Adjust SVG based on evaluation results ,include:
[0060] if Above the first threshold and Above the second threshold and If the value is below the third threshold and the modulation ratio margin is below the fourth threshold, the evaluation result is considered healthy; otherwise... To impose restrictions;
[0061] If satisfied or If the value is within the preset middle range, the evaluation result is considered sub-healthy. In this case, the reactive current output of the SVG is limited to the rated value. The first proportion;
[0062] if or If the current exceeds the preset safety range, or the current tracking residual is abnormal, or the modulation ratio margin is close to saturation, the evaluation result is a warning level. In this case, the reactive current output by the SVG is limited to the rated value. The second ratio, and issued an alarm signal.
[0063] An SVG multimodal adaptive control system based on deep reinforcement learning is provided, wherein the system is used to execute the SVG multimodal adaptive control method based on deep reinforcement learning.
[0064] A computer-readable storage medium storing a computer program that is executed by a processor to implement the above-described deep reinforcement learning-based SVG multimodal adaptive control method.
[0065] The beneficial effects of this invention are:
[0066] 1. This invention performs parallel feature extraction on high-frequency transient sequences and low-frequency trend sequences. The high-frequency feature representation obtained from high-frequency sampling can identify the power grid state during microsecond-level forward propagation without explicit modeling or long-term data accumulation; the low-frequency feature representation obtained from low-frequency sampling can provide second-level power grid structure changes. By fusing the gate vector, the information weights of the two channels can be dynamically allocated, achieving a unified approach of accurate power grid state identification and rapid response.
[0067] 2. This invention replaces traditional hard threshold mode switching with a continuous trust factor, eliminating the switching boundary and fundamentally eliminating command jumps and voltage fluctuations caused by mode switching. That is, it ensures a smooth transition from normal mode (using reactive current commands given by the decision network) to a safe and conservative mode (using a safe baseline value). ), the smoothness. Attached Figure Description
[0068] Figure 1 This is a flowchart of the SVG multimodal adaptive control method based on deep reinforcement learning according to the present invention. Detailed Implementation
[0069] The following description is intended to disclose the present invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious modifications will occur to those skilled in the art. The basic principles of the invention defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the invention.
[0070] It is understood that the term "a" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element can be one, while in another embodiment, the number of the element can be multiple, and the term "a" should not be understood as a limitation on the number.
[0071] Example 1: Reference Figure 1 The technical solution provided by this invention is: an SVG multimodal adaptive control method based on deep reinforcement learning, comprising the following steps:
[0072] Step 1: Obtain the three-phase voltage and three-phase current of the target access point. Use the obtained three-phase voltage and three-phase current to generate a multi-source local observation vector sequence. Use the multi-source local observation vector sequence to construct a multi-timescale state sequence. Specifically:
[0073] Real-time acquisition of three-phase voltage at SVG access point and three-phase current DC side voltage ;
[0074] Perform Clark and Park transformations on the three-phase voltages and currents to obtain the voltages in a synchronous rotating coordinate system. Axial components ,Voltage Axial components Current Axial components Current Axial components ;
[0075] The phase angle of the positive sequence component of the grid voltage is extracted using a phase-locked loop. and grid frequency Calculate the voltage amplitude at the connection point. Instantaneous reactive power .
[0076] Obtain the DC voltage at the DC terminal of the SVG and modulation ratio ; Among them, modulation voltage ;in for Shaft-modulated voltage, used to control DC bus voltage, mainly consists of the PI output of active current in the current loop; for The shaft-modulated voltage is mainly composed of the PI output of reactive current (i.e., determined by the final reactive current command).
[0077] Calculate DC voltage deviation ,in, This is the rated voltage of the DC bus.
[0078] Local observation vector ,in, This is the normalized connection point voltage amplitude. The normalized instantaneous reactive power at the access point. This is the normalized DC voltage deviation. The normalized power grid frequency, The modulation ratio, Normalized current Axial components.
[0079] With the first sampling period For intervals, the nearest The local observation vectors are arranged in chronological order to construct a high-frequency transient sequence. ; express The local observation vector at time t;
[0080] Maintain a local storage capacity of A long-history circular buffer stores the most recent Local observation vectors;
[0081] With the second sampling period Extract at intervals from the long history circular buffer. A low-frequency trend sequence is constructed from local observation vectors at equal time intervals. .
[0082] Step 2: Extract time-series features from the multi-timescale state sequence to generate high-frequency and low-frequency feature representations. Then, perform a dimension-wise weighted fusion of the high-frequency and low-frequency feature representations to generate a fused feature vector. This includes the following steps:
[0083] Step 2.1: Extract high-frequency transient sequences using the Long Short-Term Memory (LSTM) algorithm. The hidden state at the last time step is used as a high-frequency feature representation. This is used to represent the type, intensity, and duration of a transient event. Specifically:
[0084] High-frequency transient sequences The hidden states are fed into the high-frequency long short-term memory (LSTM) encoder in forward chronological order. LSTM hidden state dimension. The initial hidden state and cell state are set to zero vectors, so that the model depends only on the sequence internals. Each time step establishes context, avoiding cross-sequence contamination by outdated information.
[0085] For each time step The LSTM unit executes four operations in sequence: forget gate, input gate, cell state update, and output gate.
[0086] The adaptive behavior of the forget gate: When a steep rising edge of a voltage drop is detected in the input, the output of the forget gate decreases briefly, allowing the cell state to be quickly reset to adapt to the new dynamics.
[0087] The adaptive behavior of the input gate: New information such as "fall depth" and "fall initiation slope" is written into the cell state. After the next gating iteration, retrieve the hidden state at the last time step. As a feature summary of the high-frequency channel.
[0088] The voltage drop depth is defined as the lowest steady-state value to which the voltage amplitude decreases from its rated value (usually the rated voltage). It is used to describe the severity of the fault. For example, the voltage amplitude at the connection point decreases from the rated value... Reduce to the lowest steady-state value This indicates a deep voltage drop, signifying a severe short-circuit fault. For example, the voltage amplitude at the connection point drops from the rated value... Reduce to the lowest steady-state value This indicates a slight drop in voltage, which means that a light-load start-up or a remote fault has occurred.
[0089] The initial slope of the voltage drop is: the rate at which the voltage begins to decrease from its rated value, i.e. The initial slope of the drop describes the suddenness of the failure; the steeper the slope, the more sudden the failure.
[0090] This vector selectively memorizes the entire vector through a gating mechanism. The most critical transient event characteristics within the window include: fault type (short circuit fault, remote fault) and severity (the magnitude of the drop initiation slope), implicit characterization of power grid strength (short circuit ratio), and event timing information (the time elapsed since the fault occurred).
[0091] Step 2.2: Extract low-frequency trend sequences using the LSTM algorithm. The hidden state at the last time step is used as a low-frequency feature representation. This is used to represent the drift of the power grid operating point and changes in a weak power grid. Specifically:
[0092] Low-frequency trend sequence The data is fed into the low-frequency LSTM encoder in forward chronological order. LSTM hidden state dimension. Unlike high-frequency channels, low-frequency channels inherit the initial hidden state and cell state from the final state of the previous control cycle, achieving memory continuation across inference cycles. This is because low-frequency sequences have a large sampling interval (100ms), and if each control cycle is initialized from zero, the continuity of trends that were not explicitly sampled will be lost.
[0093] The low-frequency LSTM cell structure is the same as the high-frequency channel but the parameters are independent. The forget gate bias converges to a value close to 1 during training, tending to retain long-term memory by default; the input gate opens when a change in trend direction is detected (such as a continuous rise in voltage across multiple consecutive sampling points), writing the recovery slope and current recovery progress into the cell state. The recovery slope is the average rate at which the access point voltage amplitude recovers from its lowest steady-state value to near its rated value after fault clearance, calculated using the following formula: Current recovery progress: at the current time The amount by which the voltage has recovered from its lowest steady-state value (the point of failure), and how far it is from returning to its rated value. Use a ratio. To describe the recovery progress, i.e. For example, if A reading of 20% indicates that the voltage has just risen and is still far from normal; the SVG should not prematurely withdraw from reactive power support. A value of 90% indicates that the voltage is close to the rated value. The SVG needs to reduce reactive power processing to avoid overcompensation leading to voltage overshoot.
[0094] Take the hidden state at the last time step As a feature summary of the low-frequency channel, its encoded trend features include: voltage recovery process (recovery slope and current recovery progress), slow drift of weak grid (ratio of reactive power to connection point voltage), and long-term trend of reactive power demand (minute-level reactive power change).
[0095] Step 2.3: Map the high-frequency feature representation and the low-frequency feature representation to the same dimension to obtain the projection vector. and Specifically:
[0096] Through a fully connected projection layer and Mapped to the same dimension :
[0097] ;
[0098] in , This is the preset projection matrix. This ensures that the output range matches the numerical range of the LSTM hidden state, guaranteeing the numerical stability of subsequent calculations. , For high-frequency projection bias and low-frequency projection bias.
[0099] Step 2.4: Calculate and obtain the fusion gate vector. ,in for Activation function for The concatenated vector; For fusion matrix; For fusion bias;
[0100] Step 2.5: Output the fused feature vector ,in This represents element-wise multiplication. It is a vector consisting entirely of 1s.
[0101] By setting a fusion gate vector for gating, during steady-state grid operation, the high-frequency channel shows no significant mode changes, and the gate value is biased towards 0.1~0.3, with low-frequency characteristics dominating. During sudden voltage drops, the high-frequency channel changes drastically, and the gate value of the affected dimension quickly rises to 0.8~0.95, with high-frequency characteristics dominating. During the recovery period after fault clearance, the gate value gradually returns to 0.2~0.5, allowing low-frequency trend information to regain dominance and preventing premature exit from support. This continuous adaptive weight adjustment achieves implicit "hands-free" multimodal control.
[0102] Step 3: Utilize the fused feature vector to generate the integrated mean of the reactive current command, and quantify the cognitive uncertainty variance and random uncertainty variance of the integrated mean; specifically:
[0103] Step 3.1: Construct an integrated decision network, which includes... A policy sub-model; the policy sub-model is a pre-trained multilayer perceptron; in this embodiment... Each multilayer perceptron's structure includes: an input layer with 128 dimensions (corresponding to...). The first hidden layer, connected to the input layer, has a dimension of 256 (using the ReLU activation function); the second hidden layer, connected to the first hidden layer, has a dimension of 256 (using the ReLU activation function) and the output layer has a dimension of 2 (linear output).
[0104] The fused feature vectors are obtained through the strategy sub-model. Distributed parameters mapped to reactive current commands; the distributed parameters include the mean. and logarithmic standard deviation Specifically:
[0105] The average value of the reactive current command output by the two neurons in the output layer. and logarithmic standard deviation .
[0106] Step 3.2, through The standard deviation obtained by transformation is always positive.
[0107] In areas with dense training data, It is usually around [-2, 0] (corresponding to) This indicates a moderate degree of random uncertainty; under conditions of intense noise or ambiguity. The independence of the policy sub-model parameters stems from the deep ensemble strategy during training: each policy sub-model is trained independently on the same dataset, converging to different local optima on the loss function surface only due to differences in random initialization seeds and training batch order. The predictive discrepancies between these functionally equivalent but parameter-different local optima provide a source for estimating cognitive uncertainty.
[0108] Step 3.3: Calculate the arithmetic mean of the outputs of all strategy sub-models as the ensemble mean. ;
[0109] Calculate the variance of the output mean of all policy sub-models with respect to the ensemble mean, as the variance of cognitive uncertainty. This value measures the degree of disagreement among the strategy sub-models regarding the mean predictions for the same input state. A small value indicates a high degree of consistency among the strategy sub-models, suggesting the current state is in a region of dense training data and the decision has high reliability; a large value indicates a serious disagreement among the strategy sub-models, suggesting the current state may be out of distribution or in a highly nonlinear fuzzy region and the decision has low reliability.
[0110] Calculate the arithmetic mean of the variances of the outputs of all strategy sub-models, as the variance of random uncertainty. This value measures the consistency of each strategy sub-model's assessment of the noise level inherent in the current state. A small value indicates a clean and stable power grid signal; a large value indicates the presence of irreducible random fluctuations (such as arc furnace flicker or measurement noise).
[0111] Each sub-model is equipped with a pair of evaluation networks (Critic networks). and (Dual-Q network structure, derived from the dual-delay deep deterministic policy gradient TD3 algorithm framework), input is fused feature vector and The output is a scalar Q value, representing the state. Execute The expected cumulative return. The Critic network has 128 hidden layer dimensions and outputs a linear scalar.
[0112] During the inference phase, ensemble mean is used. As a unified action input, the outputs of all Critic networks are calculated. A total of 2K=10 Q-value estimates were obtained, which were used for robustness margin calculation.
[0113] Step 4: Calculate the trust factor based on the cognitive uncertainty variance, and use the trust factor to weight and mix the integrated mean and the adaptive safety baseline to obtain the corrected reactive current command; specifically:
[0114] Calculate the trust factor ,in To understand the standard deviation of uncertainty, This is a preset positive penalty coefficient. In this embodiment, the mean divergence of each strategy sub-model is assumed to reach 0.5. If the trust level should drop to 0.1, then γ = -ln(0.1) / 0.5 ≈ 4.6, so we take γ = 5.0.
[0115] Generate adaptive safety baseline value ,in The voltage amplitude at the connection point. This is the voltage reference value. The safety droop factor is less than the rated control gain. This is a preset bias term;
[0116] Calculate the penalty factor for random uncertainty ,in The standard deviation of random uncertainty The preset normal noise threshold, Less than The positive penalty coefficient. In this embodiment... The value is 1.
[0117] Update the trust factor to ,in The initial trust factor is determined solely by cognitive uncertainty.
[0118] because The trust factor is further suppressed by high random uncertainty on top of cognitive uncertainty. When the signal noise is extremely high, even if the strategy sub-model is very familiar with the current state, the system tends to be conservative.
[0119] Calculate the corrected reactive current command ;
[0120] Physical boundary constraints are applied to the modified reactive current command, including a reactive current limit based on the SVG rated reactive current and a maximum reactive current limit dynamically adjusted based on the DC voltage margin; specifically:
[0121] Apply a safety baseline limit and calculate the safety baseline value. In other words, the safety mode uses only 30% of the SVG's rated current, leaving a margin for rapid recovery after the uncertainty is eliminated. This is the clipping function. This is the rated current for the SVG.
[0122] when At times (severe voltage drop), regardless of uncertainty, forced (Capacitive direction), adaptive droop output, ensuring the most basic fault support function under extreme low voltage conditions.
[0123] when When (complete trust), ;when (When you don't believe at all) The system then reverts to a safe baseline. The transition is continuous and has no threshold boundary, fundamentally eliminating hard-switching oscillations.
[0124] Maximum reactive current limiting, i.e. .
[0125] Step 5: Perform rate limiting and flexible smoothing processing on the modified reactive current command sequentially to obtain the final reactive current command and output it. Specifically:
[0126] Calculate the instruction changes in the current control cycle. The instruction changes Less than the maximum allowable change , The maximum permissible rate of change of reactive current;
[0127] Obtain current power grid status information; if the current power grid is a weak grid, reduce... ;
[0128] Adjusted maximum allowable change Among them, the derating factor for weak power grids ;
[0129] if If the reactive current command for the current control cycle remains unchanged, then the reactive current command for the current control cycle remains unchanged. ;otherwise, ;
[0130] Smoothing the current, that is, smoothing the current. A first-order low-pass filter is used to obtain the final reactive current command. ; Filtering time constant The typical value is 1ms, which is a trade-off between noise suppression and response speed. Similarly, it is adaptively adjusted according to the low-frequency trend characteristics: it is increased to 2~5ms under weak grid conditions to slow down the transition; and reduced to 0.5ms under strong grid conditions and emergency fault support to speed up the response.
[0131] The final reactive current command is sent to the inner current loop PI controller for execution, enabling the SVG output reactive current tracking. Specifically:
[0132] Smoothed reactive current command The current is fed into the inner loop of a traditional PI converter with feedforward decoupling, enabling the SVG output to track reactive current. .
[0133] The reason for retaining the traditional PI structure is that PI control is physically transparent and its stability is analytically guaranteed, which meets the requirements of power electronic equipment for high reliability and verifiability of the execution layer; the deep reinforcement learning model is only responsible for generating high-level instructions, and its black-box characteristics are limited to the level of monitorable instructions.
[0134] Calculate the gain adjustment factor based on the power grid status information. The range of the gain adjustment factor is: Between 1 and 1; where the minimum gain is... ;
[0135] In this embodiment, ;in σ(z) is the sigmoid function. The short-circuit ratio at the current moment. For reference short-circuit ratio, in this embodiment, the reference short-circuit ratio is 5; when A value greater than 5 indicates that the current power grid is a strong grid. When it is close to 1, When the value is less than 5, the current power grid is a weak power grid. Close to 0.
[0136] Adaptive adjustment of the proportional gain of the inner current loop and integral gain ;in The parameters of the proportional-integral PI controller are tuned for rated operating conditions.
[0137] This step solves the problem of weak grid oscillation in existing technologies: traditional fixed-parameter PI controllers oscillate under weak grid conditions due to the increase in equivalent open-loop gain. In weak grid conditions, adaptive adjustment reduces both the proportional gain and integral gain. (less than 1), fundamentally eliminating the oscillation source. Through a continuous function ( It replaces the parameter switching table based on SCR thresholds, eliminating parameter abrupt changes at switching boundaries.
[0138] Example 2: To convert the meaning of hidden state variables within a deep learning model into explicit reliability quantifications that are understandable to technicians and usable by existing protection systems, the following technical solution is also provided:
[0139] The decision reliability index is calculated from the cognitive uncertainty variance, the robustness margin is calculated from the output of the value assessment network in the integrated decision network, and a comprehensive health status assessment is performed by combining the current tracking residual and the modulation ratio margin, triggering a graded response, specifically as follows:
[0140] Calculate the decision reliability index ,in This is a preset scale factor;
[0141] Obtain the set of Q-values obtained by evaluating the ensemble mean in the current state of all value assessment networks in the ensemble decision network, and take the 10th percentile as the robustness margin. ;
[0142] Calculate current tracking residual ,in This is the actual feedback current;
[0143] Calculate the modulation ratio margin, and determine a saturation warning when the modulation ratio is close to 1.
[0144] A comprehensive state assessment is performed by combining the aforementioned decision reliability index, robustness margin, current tracking residual, and modulation ratio margin.
[0145] Adjust SVG based on evaluation results ,include:
[0146] if Above the first threshold of 0.85 and Above the second threshold of 0.5 and Below the third threshold of 0.05 If the modulation ratio margin is below the fourth threshold of 0.9, the evaluation result is considered healthy; otherwise... To impose restrictions;
[0147] If satisfied or Located in the preset middle range, i.e. or If the assessment result is sub-healthy, then the reactive current output of the SVG should be limited to the rated value. The first proportion, namely ;
[0148] if or Exceeding the preset safety range, i.e. or Or current tracking residual difference is normal Greater than 0.05 Or the modulation margin is close to saturation, that is If the assessment result is a warning level, then the reactive current output of the SVG is limited to the rated value. The second proportion, namely It will also issue an alarm signal to notify technical personnel.
[0149] The present invention also provides an SVG multimodal adaptive control system based on deep reinforcement learning, the system being used to execute the SVG multimodal adaptive control method based on deep reinforcement learning.
[0150] The present invention also provides a computer-readable storage medium storing a computer program that is executed by a processor to implement the above-described deep reinforcement learning-based SVG multimodal adaptive control method.
[0151] The processes described above with reference to the flowcharts in the embodiments disclosed in this invention can be implemented as computer software programs. The embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the functions defined in the methods of this application. It should be noted that the computer-readable medium described above in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wire segments, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless segments, wire segments, optical fibers, RF, etc., or any suitable combination thereof.
[0152] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0153] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are merely examples and do not limit the present invention. The purpose of the present invention has been fully and effectively achieved. The functions and structural principles of the present invention have been shown and explained in the embodiments. Without departing from the principles described, the implementation of the present invention may have any changes or modifications.
Claims
1. A deep reinforcement learning-based SVG multimodal adaptive control method, characterized in that, The method includes: The three-phase voltage and three-phase current of the target access point are obtained, and a multi-source local observation vector sequence is generated using the obtained three-phase voltage and three-phase current. A multi-time-scale state sequence is constructed using the multi-source local observation vector sequence. Temporal features are extracted from the multi-timescale state sequence to generate high-frequency feature representations and low-frequency feature representations. The high-frequency feature representations and low-frequency feature representations are then fused by weighted dimensionally to generate a fused feature vector. The integrated mean of reactive current command is generated using the fused feature vector, and the cognitive uncertainty variance and random uncertainty variance of the integrated mean are quantified. The trust factor is calculated based on the cognitive uncertainty variance, and the integrated mean and adaptive safety base value are weighted and mixed using the trust factor to obtain the corrected reactive current command. The modified reactive current command is sequentially subjected to rate limiting and flexible smoothing to obtain the final reactive current command and output it.
2. The SVG multimodal adaptive control method based on deep reinforcement learning according to claim 1, characterized in that, The process of acquiring the three-phase voltage and three-phase current of the target access point, and generating a multi-source local observation vector sequence using the acquired three-phase voltage and three-phase current, includes: Real-time acquisition of three-phase voltage and three-phase current, and DC-side voltage at the SVG access point; Performing Clark and Park transformations on the three-phase voltages and currents yields the voltage axis components and voltage values in a synchronous rotating coordinate system. Axis components, current Axis components, current Axial components; Utilizing voltage Axis components, voltage Axis components, current Axis components, current The instantaneous reactive power at the connection point is obtained by calculating the axis components; Extracting the phase angle of the positive sequence component of the grid voltage and power grid frequency ; Constructing local observation vectors ,in, This is the normalized connection point voltage amplitude. The normalized instantaneous reactive power at the access point. This is the normalized DC voltage deviation. The normalized power grid frequency, The modulation ratio, Normalized current Axial components; The process of constructing a multi-timescale state sequence using the multi-source local observation vector sequence includes: With the first sampling period For intervals, the nearest The local observation vectors are arranged in chronological order to construct a high-frequency transient sequence. ; express The local observation vector at time t; Maintain a local storage capacity of A long-history circular buffer stores the most recent Local observation vectors; With the second sampling period Extract at intervals from the long history circular buffer. A low-frequency trend sequence is constructed from local observation vectors at equal time intervals. .
3. The SVG multimodal adaptive control method based on deep reinforcement learning according to claim 2, characterized in that, The step of extracting temporal features from the multi-timescale state sequence to generate high-frequency and low-frequency feature representations, and then performing a dimension-wise weighted fusion of the high-frequency and low-frequency feature representations to generate a fused feature vector, includes: High-frequency transient sequences are extracted using the Long Short-Time Memory (LSTM) algorithm. The hidden state at the last time step is used as a high-frequency feature representation. , used to indicate the type, intensity, and duration of a transient event; Extracting low-frequency trend sequences using the LSTM algorithm The hidden state at the last time step is used as a low-frequency feature representation. This is used to indicate the drift of the power grid operating point and changes in a weak power grid. The high-frequency feature representation and the low-frequency feature representation are mapped to the same dimension to obtain the projection vector. and ; Calculate the fusion gate vector ,in for Activation function Indicates to Perform vector concatenation; For fusion matrix; For fusion bias; Through gate vector pairs and Perform fusion and output fused feature vector. ,in This represents element-wise multiplication. It is a vector consisting entirely of 1s.
4. The SVG multimodal adaptive control method based on deep reinforcement learning according to claim 3, characterized in that, The process of generating an integrated mean of reactive current commands using the fused feature vectors and quantifying the cognitive uncertainty variance and random uncertainty variance of the integrated mean includes: Construct an integrated decision network, the integrated decision network comprising A policy sub-model; the policy sub-model is a pre-trained multilayer perceptron; The fused feature vectors are obtained through the strategy sub-model. Distributed parameters mapped to reactive current commands; the distributed parameters include the mean. and logarithmic standard deviation ; The standard deviation is obtained through exponential transformation. ; Calculate the arithmetic mean of the outputs of all policy sub-models as the ensemble mean. ; Calculate the variance of the output mean of all policy sub-models with respect to the ensemble mean, as the variance of cognitive uncertainty. ; Calculate the arithmetic mean of the variances of the outputs of all strategy sub-models, as the variance of random uncertainty. .
5. The SVG multimodal adaptive control method based on deep reinforcement learning according to claim 4, characterized in that, The step of calculating a trust factor based on the cognitive uncertainty variance, and then using the trust factor to weight and mix the integrated mean and the adaptive safety baseline to obtain the corrected reactive current command includes: Calculate the trust factor ,in To understand the standard deviation of uncertainty, The default positive penalty coefficient; Generate adaptive safety baseline value ,in The voltage amplitude at the connection point. This is the voltage reference value. The safety droop factor is less than the rated control gain. This is a preset bias term; Calculate the corrected reactive current command ; Physical boundary constraints are applied to the modified reactive current command, including a reactive current limit based on the SVG rated reactive current and a maximum reactive current limit dynamically adjusted based on the DC voltage margin.
6. The SVG multimodal adaptive control method based on deep reinforcement learning according to claim 5, characterized in that, The calculation of the trust factor based on the cognitive uncertainty variance also includes: Calculate the penalty factor for random uncertainty ,in The standard deviation of random uncertainty The preset normal noise threshold, Less than A positive penalty coefficient; Update the trust factor to ,in The initial trust factor is determined solely by cognitive uncertainty.
7. The SVG multimodal adaptive control method based on deep reinforcement learning according to claim 6, characterized in that, The process of sequentially performing rate limiting and flexible smoothing on the modified reactive current command to obtain and output the final reactive current command includes: Calculate the instruction changes in the current control cycle. The instruction changes Less than the maximum allowable change , The maximum permissible rate of change of reactive current; Obtain current power grid status information; if the current power grid is a weak grid, reduce... ; Adjusted maximum allowable change Among them, the derating factor for weak power grids ; if If the reactive current command for the current control cycle remains unchanged, then the reactive current command for the current control cycle remains unchanged. ;otherwise, ; right A first-order low-pass filter is used to obtain the final reactive current command. ; The final reactive current command is sent to the inner current loop PI controller for execution, enabling the SVG output reactive current tracking. ; Calculate the gain adjustment factor based on the power grid status information. The range of the gain adjustment factor is: Between 1 and 1; where the minimum gain is... ; Adaptive adjustment of the proportional gain of the inner current loop and integral gain ;in The parameters of the proportional-integral PI controller are tuned for rated operating conditions.
8. The SVG multimodal adaptive control method based on deep reinforcement learning according to claim 7, characterized in that, Also includes: The decision reliability index is calculated from the cognitive uncertainty variance, the robustness margin is calculated from the output of the value assessment network in the integrated decision network, and a comprehensive health status assessment is performed by combining the current tracking residual and the modulation ratio margin, triggering a graded response, specifically as follows: Calculate the decision reliability index ,in This is a preset scale factor; Obtain the set of Q-values obtained by evaluating the ensemble mean in the current state of all value assessment networks in the ensemble decision network, and take the 10th percentile as the robustness margin. ; Calculate current tracking residual ,in This is the actual feedback current; Calculate the modulation ratio margin, and determine a saturation warning when the modulation ratio is close to 1. A comprehensive state assessment is performed by combining the aforementioned decision reliability index, robustness margin, current tracking residual, and modulation ratio margin. Adjust SVG based on evaluation results ,include: if Above the first threshold and Above the second threshold and If the value is below the third threshold and the modulation ratio margin is below the fourth threshold, the evaluation result is considered healthy; otherwise... To impose restrictions; If satisfied or If the value is within the preset middle range, the evaluation result is considered sub-healthy. In this case, the reactive current output of the SVG is limited to the rated value. The first proportion; if or If the current exceeds the preset safety range, or the current tracking residual is abnormal, or the modulation ratio margin is close to saturation, the evaluation result is a warning level. In this case, the reactive current output by the SVG is limited to the rated value. The second ratio, and issued an alarm signal.
9. An SVG multimodal adaptive control system based on deep reinforcement learning, characterized in that, The system is used to execute the SVG multimodal adaptive control method based on deep reinforcement learning as described in any one of claims 1-8.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is executed by a processor to implement the SVG multimodal adaptive control method based on deep reinforcement learning as described in any one of claims 1-8.