A network construction type equipment adaptive control method and system for high proportion of new energy access

Abstract

The application provides a network-oriented high-proportion new energy access network equipment adaptive control method and system, and relates to the technical field of power system automation control. The method comprises the following steps: determining the value interval of virtual inertia and damping coefficient according to the rotor motion equation and power angle relationship; obtaining a state space composed of frequency deviation, frequency change rate, active power deviation and change rate; constructing a deep reinforcement learning agent; defining the action space as virtual inertia increment and damping coefficient increment; updating the parameters after smoothing filtering; calculating the reward function according to the frequency deviation, active power deviation and recovery time; and optimizing the strategy network parameters through interaction. The application can realize online adaptive adjustment of virtual inertia and damping coefficient under weak power grid and multiple disturbance conditions, effectively suppress frequency and active power oscillation in the black start process, shorten the system recovery time, and improve the system dynamic response performance and operation stability.

Description

Technical Field

[0001] This application relates to the field of power system automation control technology, and in particular to an adaptive control method and system for grid-connected equipment with a high proportion of new energy access. Background Technology

[0002] With the rapid development of renewable energy and the increasing proportion of power electronic equipment in power systems, traditional synchronous generators are gradually being replaced by grid-connected converters, significantly reducing system inertia. Under the background of high-proportion photovoltaic and other new energy integration, power systems face problems such as insufficient support capacity, frequent operating condition switching, and strong disturbance uncertainty during black start and recovery processes, easily leading to frequency fluctuations and power oscillations, and in severe cases, even system instability. Therefore, how to improve the frequency stability and dynamic response performance of the system during black start has become an important research direction in the field of power system stability control.

[0003] Currently, Virtual Synchronous Generator (VSG) control can improve system frequency response characteristics to some extent by introducing virtual inertia and damping coefficients. However, traditional VSG control methods often use fixed parameters or empirical rules for adjustment. When facing complex operating conditions such as load switching, photovoltaic output fluctuations, and multi-source parallel operation, it is difficult to simultaneously meet the control requirements of oscillation suppression and rapid recovery, resulting in insufficient adaptability and limited robustness. Therefore, how to achieve online adaptive optimization of virtual inertia and damping coefficients has become the key to improving black start control performance. Summary of the Invention

[0004] This application provides an adaptive control method and system for grid-connected equipment with a high proportion of renewable energy integration. It aims to achieve black-start adaptive control of grid-connected photovoltaic-storage systems based on the Soft Actor Critic (SAC) algorithm, addressing the problems of insufficient weak grid support, frequent operating condition switching, and high disturbance uncertainty in existing technologies during the black-start recovery process of grid-connected photovoltaic-storage systems, thus overcoming multiple shortcomings of existing technologies. Specifically, it utilizes reinforcement learning technology to achieve continuous online adjustment of virtual inertia and damping coefficients through interactive learning between the agent and the environment. Without requiring a precise system model, the SAC algorithm can autonomously optimize the control strategy based on the system frequency and active power operating status, and improve the stability and dynamic response performance of the control process by introducing action smoothing constraints and recovery time optimization mechanisms. Compared with traditional control methods, this application has stronger adaptability and robustness under complex disturbances and multiple operating conditions, and is suitable for black-start and recovery control scenarios in power systems with a high proportion of renewable energy integration.

[0005] In a first aspect, this application provides an adaptive control method for grid-connected equipment oriented towards high-proportion renewable energy access, the method comprising: Based on the topology of the grid-type photovoltaic-storage system and the rotor motion equation and power angle relationship of the virtual synchronous generator, the range of values ​​for the virtual inertia and the range of values ​​for the damping coefficient are determined. The system acquires its frequency deviation, frequency change rate, active power deviation, and active power change rate in real time to form a state space. A deep reinforcement learning agent is constructed, defining the action space as virtual inertia increment and damping coefficient increment. The value range of the virtual inertia increment and damping coefficient increment is set according to their respective ranges, ensuring that the updated virtual inertia and damping coefficient do not exceed these ranges. The virtual inertia increment and damping coefficient increment are smoothed and filtered to obtain the actual execution increment. The virtual inertia and damping coefficient are then updated by adding the actual execution increment to the previous value. A reward function is calculated based on the frequency deviation, active power deviation, and recovery time. The agent aims to maximize the cumulative discount reward by interacting with the environment to optimize its policy network parameters until the policy converges, resulting in an optimized policy network parameter set. The optimized strategy network parameter set is loaded into the real-time controller; in each control cycle, the current system state is obtained and input into the strategy network, the strategy network outputs the corresponding virtual inertia increment and damping coefficient increment, and after smoothing and filtering, the virtual inertia and damping coefficient are updated and output to the VSG control loop.

[0006] As a preferred technical solution, determining the range of values ​​for the virtual inertia and the range of values ​​for the damping coefficient includes: Based on the rotor motion equation and power angle relationship of the virtual synchronous generator, the active power-frequency small-signal model, active power response transfer function, and frequency response transfer function are determined. Based on the active power response transfer function and frequency response transfer function, the system is equivalent to a standard second-order system, and the inequality relationship between virtual inertia and damping coefficient is determined using the damping ratio constraint condition. Based on the aforementioned inequality relationship, and in conjunction with the droop rate constraint and the real part constraint of the characteristic root, the range of values ​​for the virtual inertia and the range of values ​​for the damping coefficient are determined.

[0007] As a preferred technical solution, based on the rotor motion equation and power angle relationship of the virtual synchronous generator, the active power-frequency small-signal model, active power response transfer function, and frequency response transfer function are determined, including: The rotor motion equation of the virtual synchronous generator is determined as follows: ; in, P ref andP e These are the converter's reference active power and output active power, respectively. ω , ω 0 represents the rotor angular velocity and the rated angular velocity, respectively. J , D These are the virtual inertia coefficient and the damping coefficient, respectively. t For time; Define the deviation: ; in, , , and These represent the VSG angular frequency, reference active power, output active power, and the change in power angle, respectively. For the angle of attack, P ref0 This is the initial value of the converter's reference active power. P e0 This represents the initial value of the converter's output active power. δ 0 represents the initial value of the work angle; Substituting the deviation into the rotor motion equation and neglecting the second-order small-signal quantity, we obtain the standard linear form of the active-frequency outer loop: ; In the initial stage of black start, the virtual synchronous generator and the line are equivalently represented as a coupling channel dominated by reactance, and the approximate relationship of the output active power is determined as follows: ; in, E 0 represents the amplitude of the VSG's equivalent internal potential; U g This refers to the amplitude of the terminal voltage. X The equivalent reactance between the VSG and the load; The integral relationship between the power angle deviation and the angular velocity deviation is determined as follows: ; in, The rate of change of the work angle; Based on the standard linear form of the active-frequency outer loop, the approximate relationship of active power, and the integral relationship, the small-signal model of VSG is constructed as follows: ; Based on the small-signal model, the active power response transfer function and the frequency response transfer function are determined as follows: ; ; in, Gp ( s ) is the active power response transfer function, Δ Pe ( s ) represents the output active power deviation, Δ P ref ( s (This refers to the reference active power deviation.) s For the Laplace operator, K d For the synchronous power coefficient, Δ ω ( s ) represents the angular velocity deviation. G ω ( s ) is the frequency response transfer function.

[0008] As a preferred technical solution, based on the active power response transfer function and the frequency response transfer function, the system is equivalent to a standard second-order system, and the inequality relationship between the virtual inertia and the damping coefficient is determined using the damping ratio constraint condition, including: When the power angle is small and the power coupling term is ignored, the system is equivalent to a standard second-order system; The natural oscillation angular frequency is determined based on the active power response transfer function and the frequency response transfer function. Damping ratio The expression is: ; The relationship between overshoot and damping ratio is determined by the following formula: ; in, Let exp be the overshoot, and let exp be the natural exponential function. By constraining the damping ratio within a set range and substituting it into the damping ratio expression, the inequality relationship between the virtual inertia and the damping coefficient is obtained.

[0009] As a preferred technical solution, based on the aforementioned inequality relationship, and in conjunction with the droop rate constraint and the real part constraint of the eigenvalue, the range of values ​​for the virtual inertia and the damping coefficient are determined, including: Determine the damping coefficient D The relationship between the rate of change and the rate of change is: ; Where, Δ T Δ is the change in mechanical torque. P This represents the change in active power. The range of values ​​for the damping coefficient is determined by considering the maximum allowable frequency deviation during the black start phase. By introducing the real part constraint of the characteristic root and combining it with root locus analysis, the range of values ​​for the virtual inertia is determined.

[0010] As a preferred technical solution, the state space is represented as follows: ; in, , These respectively characterize the degree to which the frequency and active power deviate from their rated values; , These represent frequency and the rate of change of active power, respectively, and are used to reflect the strength of disturbances and the inertia support capability.

[0011] As a preferred technical solution, the action space is defined as the virtual inertia increment and the damping coefficient increment, and the value range of the virtual inertia increment and the damping coefficient increment is set according to the value range of the virtual inertia and the damping coefficient, so that the updated virtual inertia and the damping coefficient do not exceed the value range; the virtual inertia increment and the damping coefficient increment are smoothed and filtered to obtain the actual execution increment, and the virtual inertia and the damping coefficient are updated by adding the previous value to the actual execution increment, including: Define the action space as follows: ; in, a t For the action space, , These are the changes in virtual inertia and damping coefficient, respectively. After introducing dynamic smoothing constraints, the corresponding actual execution action increment is: ; in, , The agent at time t is at time t after introducing smoothness constraints. t The output action increment, and satisfies: ; in, The motion smoothing coefficient, , , These represent the actual execution increment from the previous moment; and the motion smoothing coefficient. Determined by the following formula: ; in, T s To enhance learning control of the sampling period, The motion smoothing time constant; After introducing motion smoothing constraints, the actual update expressions for virtual inertia and damping coefficients are: ; in, , This is the actual execution increment after smoothing. J 0 and D 0 represents the initial value of the virtual inertia and the initial value of the damping coefficient, respectively.

[0012] As a preferred technical solution, the reward function is calculated based on the frequency deviation, active power deviation, and recovery time, including: According to frequency deviation Δ f Determine the frequency deviation reward ,in α 1 represents the weighting coefficient of the frequency deviation reward function; Based on the active power deviation Δ P Determine the active power deviation reward ,in α 2 represents the weighting coefficient of the active power deviation reward function; Based on recovery time t Determine the recovery time reward r 3: ; in, c 1 represents the penalty amount for the recovery time item. t 0 represents the time window during which no time penalty is applied after the disturbance occurs. Furthermore, if the system has not yet met the steady-state criterion, a constant penalty is applied only to the timeout portion. The steady-state criterion is expressed as: The steady-state criterion is: ; Where, Δ f ( t ) represents the real-time frequency deviation, Δ f min Δ is the allowable threshold for frequency deviation. P ( t ) represents the real-time active power deviation, Δ P min This refers to the allowable threshold for active power deviation. Determine steady-state reward ,in c 2 represents the constant reward value given when the system reaches steady state; Determine the total reward function R for: .

[0013] As a preferred technical solution, when constructing a deep reinforcement learning agent, the adjustment of virtual inertia and damping coefficient is expressed as a Markov decision process, wherein the key elements of the Markov decision process are... M Represented as: in, S Let be the set of states, representing all possible system states; A The action set represents the actions that the agent can choose in each state; P Transition probability , indicating from state s Take action a After transitioning to state probability; R For reward function , indicating the state s Take action a Immediate benefits; Discount factor; The optimization objective is to find the optimal strategy. This maximizes the expected value of the cumulative discount reward: in, Value function; It is in state s t Execute action a t The immediate reward received; Representation Strategy The expectations below.

[0014] Secondly, this application provides an adaptive control system for grid-connected equipment oriented towards high-proportion renewable energy access, used to implement the method described above, the system comprising: The value range determination module is configured to determine the value range of virtual inertia and the value range of damping coefficient based on the topology of the grid-type photovoltaic energy storage system and the rotor motion equation and power angle relationship of the virtual synchronous generator. The optimal control parameter determination module is configured to acquire the system's frequency deviation, frequency change rate, active power deviation, and active power change rate in real time, forming a state space; construct a deep reinforcement learning agent, defining the action space as virtual inertia increment and damping coefficient increment, and setting the value range of the virtual inertia increment and damping coefficient increment according to the value range of the virtual inertia and damping coefficient, so that the updated virtual inertia and damping coefficient do not exceed the value range; perform smoothing filtering on the virtual inertia increment and damping coefficient increment to obtain the actual execution increment, and update the virtual inertia and damping coefficient by adding the actual execution increment to the value at the previous moment; calculate the reward function based on the frequency deviation, active power deviation, and recovery time; the agent aims to maximize the cumulative discount reward, and optimizes its policy network parameters by interacting with the environment until the policy converges, obtaining the optimized policy network parameter set; The real-time control module is configured to load the optimized strategy network parameter set into the real-time controller; in each control cycle, it acquires the current system state and inputs it into the strategy network, which outputs the corresponding virtual inertia increment and damping coefficient increment, updates the virtual inertia and damping coefficient after smoothing and filtering, and outputs it to the VSG control loop.

[0015] The adaptive control method and system for grid-connected equipment oriented towards high-proportion renewable energy access provided in this application have at least the following beneficial effects: This application transforms the problem of coordinating the adjustment of VSG virtual inertia and damping coefficient into an optimal control problem within a reinforcement learning framework, avoiding reliance on precise mathematical models and human experience. By constructing a state-action-reward closed-loop interaction mechanism, online adaptive adjustment of control parameters is achieved. Simultaneously, indices such as frequency deviation, active power deviation, and black-start recovery time are introduced into the reward function, combined with action smoothing constraints, to balance oscillation suppression and rapid recovery performance, thereby improving the system's dynamic response quality and control stability. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] Figure 1 A flowchart of an adaptive control method for grid-type equipment oriented towards high-proportion renewable energy access, provided in an embodiment of this application; Figure 2 A VSG control principle diagram for reinforcement learning of an adaptive control method for grid-type equipment with high proportion of new energy access provided in this application embodiment; Figure 3 This is a flowchart of song preprocessing provided in an embodiment of this application; Figure 4 The frequency response curves of VSG under different virtual inertia and damping coefficients are provided for an adaptive control method of grid-type equipment for high-proportion renewable energy access in an embodiment of this application; wherein, (a) VSG frequency response under different virtual inertia; (b) active power response under different virtual inertia; (c) VSG frequency response under different damping coefficients; (d) VSG active power response under different damping coefficients. Figure 5 The root locus diagram of global rotational inertia and damping coefficient for an adaptive control method for grid-type equipment with high proportion of new energy access provided in the embodiments of this application; Figure 6 The training result diagram shows an adaptive control method for grid-type equipment with a high proportion of new energy access, provided in an embodiment of this application. Figure 7 Simulation results of an adaptive control method for grid-connected equipment with high proportion of renewable energy access provided in this application embodiment; wherein, (a) frequency response under load input condition; (b) active power response under load input condition; Figure 8 This is a structural diagram of an adaptive control system for grid-type equipment designed for high-proportion renewable energy access, provided in an embodiment of this application.

[0018] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0019] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0020] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0021] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0022] The design of virtual inertia and damping coefficient in existing VSG control mainly relies on expert experience or fixed parameter tuning. Under typical operating conditions such as load switching, photovoltaic power output fluctuations, and multiple machines in parallel, problems such as frequency overshoot, slow oscillation decay, or excessively long recovery time are likely to occur, which may even lead to a decrease in system stability in severe cases.

[0023] To reduce reliance on expert experience, some studies employ heuristic optimization algorithms for VSG parameter tuning. Generally, heuristic algorithms can achieve multi-objective optimization based on the objective function and possess a certain degree of constraint handling capability. Compared to traditional empirical tuning methods, their applicability is improved. However, these methods still have significant limitations: on the one hand, their optimization process typically relies on a relatively accurate system mathematical model, while power systems are characterized by complex structures and high parameter uncertainty, making modeling difficult; on the other hand, heuristic algorithms have high computational complexity and resource consumption, making it difficult to meet the real-time requirements of dynamic processes such as black start.

[0024] Reinforcement learning, as a machine learning method that optimizes policies through interaction between an agent and its environment, can achieve adaptive optimization of complex control tasks even without an accurate model. Based on this, embodiments of this application provide an adaptive control method for grid-connected equipment with a high proportion of renewable energy access, such as... Figure 1 As shown, the method is performed in the following steps S10-S30 in sequence.

[0025] S10: Based on the topology of the grid-type photovoltaic-storage system and the rotor motion equation and power angle relationship of the virtual synchronous generator, determine the range of values ​​for the virtual inertia and the damping coefficient.

[0026] Step S10 aims to determine the range of values ​​for the virtual inertia and the damping coefficient through black-start modeling and dynamic characteristic analysis of the grid-type photovoltaic-storage system. In some embodiments, step S10 specifically includes the following steps S101-S103.

[0027] S101: Construct a black-start model for a grid-type photovoltaic-storage system.

[0028] Typical black-start topologies of grid-type photovoltaic-storage systems are as follows: Figure 2 As shown, the photovoltaic array and energy storage unit are DC-coupled to the grid-type converter side via a Boost converter and a bidirectional DC-DC converter, respectively. The photovoltaic array employs a perturbation and observation (P&O) maximum power point tracking (MPPT) control strategy, enabling optimal power output under different illumination conditions and ensuring maximum utilization of solar energy. The energy storage unit adopts a constant DC bus voltage control strategy, monitoring and dynamically adjusting the common DC bus voltage in real time to provide a stable DC voltage for the system. The grid-side converter uses grid-type VSG control, which can autonomously establish a stable voltage and frequency reference with the energy storage unit during the initial black start phase. Furthermore, when photovoltaic power fluctuates, the energy storage can also smooth out energy fluctuations in the photovoltaic array, stabilize the DC bus voltage, and improve the system's regulation capability and operational stability.

[0029] S102: Analysis of VSG dynamic response characteristics.

[0030] Figure 3 This is a schematic diagram of the VSG control principle based on reinforcement learning, where R f , L f and C f These represent the resistance, inductance, and capacitance of the filter circuit, respectively. R g , L g Indicates the resistance and inductance of the circuit; U dc This is the DC side voltage; U and I This refers to the converter's output voltage and current. This embodiment focuses on analyzing the dynamic characteristics of the VSG active power control loop. The VSG's rotor motion equation is shown below: ; in, P ref and P e These are the converter's reference active power and output active power, respectively. ω , ω 0 represents the rotor angular velocity and the rated angular velocity, respectively. J , D These are the virtual inertia coefficient and the damping coefficient, respectively.

[0031] To facilitate linearization analysis near the operating point, the deviation is defined as follows: ; in, , , and These represent the VSG angular frequency, reference active power, output active power, and the change in power angle, respectively. For the angle of attack, P ref0 This is the initial value of the converter's reference active power. P e0 This represents the initial value of the converter's output active power. δ 0 represents the initial value of the work angle.

[0032] Substituting the deviation into the VSG rotor motion equation and expanding it near the operating point, neglecting second-order small-signal quantities, we can obtain the standard linear form of the active-frequency outer loop: ; In the initial stage of black start, the VSG and the line can be considered as a coupling channel dominated by reactance, and the output active power can approximately satisfy the classical power angle relationship: ; in, E 0 represents the amplitude of the VSG's equivalent internal potential; U g This refers to the amplitude of the terminal voltage. For the angle of attack; X The equivalent reactance between the VSG and the load (including filter reactance, line reactance, etc.).

[0033] Gongjiao The deviation from the angular velocity satisfies an integral relationship, and is expressed as the deviation quantity: ; in, The rate of change of the work angle.

[0034] Therefore, a small-signal model for VSG is constructed: ; Therefore, the active power response transfer function and frequency response transfer function of the active power outer loop can be obtained as follows: ; ; in, G p ( s ) is the active power response transfer function, Δ Pe ( s ) represents the output active power deviation (Laplace domain), Δ P ref ( s () is the reference active power deviation (Laplace domain). s For the Laplace operator, K d For the synchronous power coefficient, Δ ω ( s ) represents the angular velocity deviation (Laplace domain). G ω ( s ) is the frequency response transfer function.

[0035] As can be seen from the transfer function, the dynamic response of the VSG is affected by the virtual inertia and the damping coefficient. Figure 4 Frequency response curves of the VSG under different virtual inertia and damping coefficients were plotted.

[0036] To facilitate the analysis of the effects of rotational inertia and damping coefficient on system stability, the following was made: Figure 5 The root locus diagram showing the global moment of inertia and damping coefficient is shown.

[0037] When the virtual inertia is small or the damping coefficient is insufficient, the system's characteristic roots gradually move closer to the imaginary axis, the system's damping ratio decreases, frequency oscillations become more pronounced, and system stability weakens. As the damping coefficient increases, the characteristic roots move towards the left half-plane, the system's oscillation decay rate accelerates, and stability improves. However, when the virtual inertia is too large, although the system's characteristic roots remain in the left half-plane, their real absolute value decreases, the system's dynamic response speed decreases, and this is detrimental to the rapid frequency establishment during black start.

[0038] S103: Parameter tuning of virtual inertia and damping coefficient.

[0039] Based on the VSG active-frequency small-signal model established in this embodiment, under the conditions of a small power angle and negligible power coupling terms, the system can be equivalent to a standard second-order system with a natural oscillation angular frequency. Damping ratio for: ; As can be seen from the above expression, virtual inertia J With damping coefficient D Together, they determine the inherent oscillation characteristics and decay rate of the system, and are key parameters affecting the system's stability and dynamic response performance.

[0040] To ensure system stability, the damping ratio of the second-order system is limited. Specifically, in the case of an underdamped second-order system, the overshoot is limited. With damping ratio The relationship between them is: ; As shown in the above equation, the damping ratio directly affects the dynamic response characteristics of the system. For a standard second-order system, when the damping ratio is too small, the system exhibits obvious underdamped characteristics, leading to significant power overshoot. Excessive transient power can cause the voltage and current of power electronic components to exceed their safe operating range, potentially causing irreversible faults such as thermal breakdown or insulation damage. When the damping ratio is too large, the system gradually enters an overdamped state. Overdamping prolongs the system's transient recovery time, making it difficult for key indicators such as voltage and frequency to quickly recover to a steady state during black start. This not only reduces start-up efficiency but also reduces the system's disturbance rejection capability due to prolonged instability, potentially leading to black start failure in severe cases. Therefore, the damping ratio range selected in this embodiment is [range to be specified in the original text]. Therefore, J and D The constraints are: ; because D The parameters are affected by the changes in mechanical torque and angular frequency, therefore we can obtain: ; Where, Δ T Δ is the change in mechanical torque. P This represents the change in active power.

[0041] According to GB / T34120, when an energy storage system participates in primary frequency regulation, its droop rate is generally taken as 0.5% to 3%, corresponding to an active power of the energy storage device adjusting within the range of 40% to 100% of its rated capacity when the system frequency changes by 1Hz. This standard mainly targets grid-connected operation scenarios. In this embodiment, the frequency regulation capability constraint is referenced in the black start parameter design of the grid-connected inverter to ensure that the parameter selection is engineeringly reasonable.

[0042] For example, given a grid-connected inverter with a rated capacity of 100kW, and considering the limited system frequency support during black start, this embodiment limits the maximum allowable frequency deviation during the black start phase to 0.5Hz according to GB / T43462 to improve operational safety. The damping coefficient can be obtained from the above formula. D The reasonable range of values ​​is: ; The selection of the rotational inertia parameter must balance system stability and dynamic response performance. Insufficient inertia will cause the system's eigenvalues ​​to approach the imaginary axis, leading to oscillations; excessive inertia will reduce the system's frequency regulation speed. To ensure the system's rapid decay characteristics under weak grid conditions, this embodiment introduces a constraint condition for the real part of the eigenvalues: ; Based on the root locus analysis results, and under the premise of satisfying the above constraints, the final range of values ​​for the moment of inertia is determined as follows: .

[0043] S20: Real-time acquisition of system frequency deviation, frequency change rate, active power deviation, and active power change rate constitutes the state space; construction of a deep reinforcement learning agent, defining the action space as virtual inertia increment and damping coefficient increment, and setting the value range of virtual inertia increment and damping coefficient increment according to the value range of virtual inertia and damping coefficient, so that the updated virtual inertia and damping coefficient do not exceed the value range; smoothing filtering is performed on the virtual inertia increment and damping coefficient increment to obtain the actual execution increment, and the virtual inertia and damping coefficient are updated by adding the actual execution increment to the value of the previous time step; the reward function is calculated based on frequency deviation, active power deviation, and recovery time; the agent aims to maximize the cumulative discount reward, and optimizes its policy network parameters by interacting with the environment until the policy converges, obtaining the optimized policy network parameter set.

[0044] In step S20, the virtual inertia ranges from a minimum to a maximum value, and the damping coefficient also ranges from a minimum to a maximum value. The action increment output by the agent consists of the virtual inertia increment and the damping coefficient increment. To ensure that the updated virtual inertia and damping coefficient do not exceed the range determined in step S10, the original action increment output by the agent needs to be pruned. Specifically, the current virtual inertia is added to the virtual inertia increment. If the calculated result is greater than the maximum value of the virtual inertia range, the virtual inertia increment is adjusted to the maximum value minus the current virtual inertia; if the calculated result is less than the minimum value of the virtual inertia range, the virtual inertia increment is adjusted to the minimum value minus the current virtual inertia. The damping coefficient increment is pruned in the same way. The pruned action increment is used to update the virtual inertia and damping coefficient, thereby ensuring that the updated parameters are always within the reasonable range determined in step S10, avoiding system instability caused by parameters exceeding the limits.

[0045] Step S20 is the core step of the method of this application, and its purpose is to realize VSG adaptive control based on the SAC algorithm. In some embodiments, step S20 includes the following steps S201-S204.

[0046] S201: Construct a Markov decision process.

[0047] The parameter adjustment problem of VSG during black-start recovery can be viewed as a sequential decision problem. Furthermore, it satisfies the Markov property, meaning the system's state at the next moment depends only on the current state and is independent of past states. Therefore, the adjustment of virtual inertia and damping coefficients can be formulated as a Markov decision process (MDP) and processed using a reinforcement learning algorithm. It consists of several key elements: ; in, S Let be the set of states, representing all possible system states; A This is the set of actions, representing the actions that the agent can choose in each state. P Transition probability , indicating from state s Take action a After transitioning to state probability; R For reward function In state s Take action a Immediate returns; and discount factors The decay weight of future rewards is used to balance short-term and long-term returns.

[0048] Its goal is to find the optimal strategy. This maximizes the expected value of the cumulative discount reward, starting from any initial state: ; in, Value function; It is in state s t Execute action a t The immediate reward received; Representation Strategy The expectations below.

[0049] S202: Design state space.

[0050] The state is obtained through environmental observation. The dynamic response of the VSG active power loop is mainly characterized by its frequency response and active power response. To comprehensively describe this response process, the state space is selected as follows: considering the lack of grid support and weak damping characteristics during the initial stage of black start, frequency deviation and active power deviation, along with their respective rates of change, are introduced as state components to construct the state space: ; in: , These respectively characterize the degree to which the frequency and active power deviate from their rated values; , These represent frequency and the rate of change of active power, respectively, and are used to reflect the strength of disturbances and the inertia support capability.

[0051] S203: Design motion space.

[0052] In this embodiment, the control variable is the virtual inertia of the VSG. J and damping coefficient D Therefore, the action space a t Defined as the change in virtual inertia and damping coefficient, it can be expressed as: ; in, , These are the changes in virtual inertia and damping coefficient, respectively. Therefore, the actual virtual inertia... J and damping coefficient D for: ; in, J 0 and D 0 represents the initial values ​​of the virtual inertia and damping coefficient, respectively.

[0053] As shown in the above equation, the action output by the agent is the increment of the virtual inertia and damping coefficient. However, it is worth noting that during the black-start recovery phase, the system's inertia support is insufficient, the equivalent damping is weak, and disturbances are frequent. If... and Large jumps between adjacent control steps can lead to overly drastic parameter updates, potentially causing active power fluctuations and amplifying frequency transient oscillations, thus reducing the engineering feasibility of the control strategy. Therefore, this paper introduces smoothing constraints at the action execution layer to filter action increments, making the update process of inertia and damping more continuous and controllable, thereby improving robustness and dynamic performance in black-start scenarios.

[0054] After introducing dynamic smoothing constraints, the corresponding actual execution action increment a t for: ; in, , The agent at time t is at time t after introducing smoothness constraints. t The output action increment, and satisfies: ; in, The motion smoothness coefficient ( ), , These represent the actual execution increments from the previous moment. The smaller the value, the stronger the smoothing effect, which can effectively suppress abrupt changes in actions between adjacent control steps, but the parameter response becomes relatively slower; The larger the value, the faster the original motion is followed, but the smoothing effect is relatively weakened. For ease of engineering selection, Comparable to the equivalent smoothing time constant Establish corresponding relationships: ; in T s To enhance learning control of the sampling period, This is the motion smoothing time constant.

[0055] Therefore, after introducing motion smoothing constraints, the actual update expressions for virtual inertia and damping coefficients can be written as: ; in, , This is the actual execution increment after smoothing. J 0 and D 0 represents the initial value of the virtual inertia and the initial value of the damping coefficient, respectively.

[0056] S204: Design the reward function.

[0057] This embodiment incorporates frequency deviation, active power deviation, and black-start recovery time into the reward function design to address potential problems in practical engineering applications. The specific design method is as follows: Frequency Deviation Reward Function r 1. In practical applications, the frequency of a power system needs to operate stably near its rated frequency, and the smaller the deviation, the better. Based on this, the frequency deviation reward function structure in this paper is as follows: ; in, The weighting coefficients represent the frequency deviation reward function.

[0058] Active power deviation reward function r 2. During the black-start load application phase, the system's active power needs to fluctuate around a reference value. The goal is to minimize the amplitude and duration of the oscillations. Therefore, the active power deviation reward function structure designed in this paper is as follows: ; in, The weighting coefficients represent the active bias reward function.

[0059] Recovery time reward function r 3. To prevent excessively long transient times, this paper incorporates the time term into the reward function. Specifically, the timing begins when a disturbance occurs, and the time term is calculated before the disturbance occurs. t No time penalty is applied within a 0-time window. Furthermore, if the system has not yet met the steady-state criterion, only the timeout portion is penalized with a constant value. As shown below: ; ; in, c 1 represents the penalty for the recovery time term, Δ f ( t ) represents the real-time frequency deviation, Δ f min Δ is the allowable threshold for frequency deviation. P ( t ) represents the real-time active power deviation, Δ P min The active power deviation is the allowable threshold; the following formula is the steady-state criterion of the system. Considering that the system is in a weak grid operation state during the black start recovery phase, disturbances will cause coupled oscillations of frequency and active power. It is difficult to fully characterize whether the recovery process has ended with a single indicator. Therefore, this embodiment selects both frequency deviation and active power deviation as steady-state criteria to guide the agent to learn the optimal control strategy more accurately.

[0060] Steady-state reward function r 4. When the system reaches steady state, adjustments to the virtual inertia and damping coefficient will not change the system's operating state. Therefore, this paper designs a steady-state reward function, which provides a positive constant reward value when the disturbed system state satisfies the steady-state criterion and is maintained for multiple sampling steps. ; in, c 2 represents the constant reward value given when the system reaches steady state. Integrating the above formulas, the final reward function can be expressed as: ; The reward function designed in this embodiment optimizes the training process through weight allocation and steady-state incentives, providing precise and comprehensive guidance for the agent to learn the optimal control strategy.

[0061] S30: Load the optimized strategy network parameter set into the real-time controller; in each control cycle, obtain the current system state and input it into the strategy network. The strategy network outputs the corresponding virtual inertia increment and damping coefficient increment. After smoothing and filtering, update the virtual inertia and damping coefficient and output it to the VSG control loop.

[0062] In the specific implementation of step S30, during the real-time control phase, each control cycle first collects the current frequency deviation, frequency change rate, active power deviation, and active power change rate, forming a state vector input to the trained policy network. The policy network calculates the virtual inertia increment and damping coefficient increment based on the current state, and then processes the increment using the same smoothing filtering method as in step S20 to obtain the actual execution increment. The virtual inertia from the previous moment is added to the actual execution increment to obtain the new virtual inertia, and the damping coefficient is updated similarly. If the new parameters exceed the value range determined in step S10, they are restricted to boundary values. Finally, the updated virtual inertia and damping coefficient are output to the VSG control loop to participate in the active power adjustment and frequency support of the next cycle. This cycle repeats to achieve real-time adaptive control of the system's dynamic characteristics during black start.

[0063] This embodiment further employs the SAC algorithm to train the agent and performs simulation verification. Specifically, based on the designed reward function, a reinforcement learning environment is constructed to train the SAC algorithm agent. Through multiple interactive learning processes, adaptive optimization of the virtual inertia and damping coefficient is achieved. The training results are as follows: Figure 6 As shown.

[0064] The trained agent was embedded into the Simulink simulation environment to verify the effectiveness of the proposed adaptive control strategy under a black-start scenario of a grid-connected photovoltaic-storage system. The simulation conditions were set as follows: In the initial stage, the energy storage system and the grid-connected converter autonomously established voltage and frequency references without external grid support. Subsequently, the photovoltaic system was connected to the system, and after the voltage and frequency stabilized, a 25kW load was put into operation.

[0065] The simulation results of traditional fixed parameter control methods and other DRL algorithms are compared with those of the method in this application. Figure 7 As shown in Table 1, the performance indicators such as frequency fluctuation, active power oscillation and system recovery time are analyzed.

[0066] Table 1 Performance Indicators under Load Input Conditions

[0067] This application also provides an adaptive control system for grid-type equipment oriented towards high-proportion renewable energy access, such as... Figure 8 As shown, the adaptive control system for grid-connected equipment oriented towards high-proportion renewable energy access includes: The value range determination module 801 is configured to determine the value range of virtual inertia and the value range of damping coefficient based on the topology of the grid-type photovoltaic energy storage system and the rotor motion equation and power angle relationship of the virtual synchronous generator. The optimal control parameter determination module 802 is configured to acquire the system's frequency deviation, frequency change rate, active power deviation, and active power change rate in real time, forming a state space; construct a deep reinforcement learning agent, defining the action space as virtual inertia increment and damping coefficient increment, and setting the value range of the virtual inertia increment and damping coefficient increment according to the value range of the virtual inertia and damping coefficient, so that the updated virtual inertia and damping coefficient do not exceed the value range; perform smoothing filtering on the virtual inertia increment and damping coefficient increment to obtain the actual execution increment, and update the virtual inertia and damping coefficient by adding the actual execution increment to the value at the previous moment; calculate the reward function based on the frequency deviation, active power deviation, and recovery time; the agent aims to maximize the cumulative discount reward, and optimizes its policy network parameters by interacting with the environment until the policy converges, obtaining the optimized policy network parameter set; The real-time control module 803 is configured to load the optimized strategy network parameter set into the real-time controller; in each control cycle, it acquires the current system state and inputs it into the strategy network, the strategy network outputs the corresponding virtual inertia increment and damping coefficient increment, and after smoothing and filtering, updates the virtual inertia and damping coefficient, and outputs it to the VSG control loop.

[0068] It should be noted that the aforementioned adaptive control system for grid-connected equipment oriented towards high proportion of new energy access belongs to the same technical concept as the prior method and can achieve the same technical effect, so it will not be elaborated here.

[0069] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An adaptive control method for grid-connected equipment oriented towards high-proportion renewable energy access, characterized in that, The method includes: Based on the topology of the grid-type photovoltaic-storage system and the rotor motion equation and power angle relationship of the virtual synchronous generator, the range of values ​​for the virtual inertia and the range of values ​​for the damping coefficient are determined. The system acquires its frequency deviation, frequency change rate, active power deviation, and active power change rate in real time to form a state space. A deep reinforcement learning agent is constructed, defining the action space as virtual inertia increment and damping coefficient increment. The value range of the virtual inertia increment and damping coefficient increment is set according to their respective ranges, ensuring that the updated virtual inertia and damping coefficient do not exceed these ranges. The virtual inertia increment and damping coefficient increment are smoothed and filtered to obtain the actual execution increment. The virtual inertia and damping coefficient are then updated by adding the actual execution increment to the previous value. A reward function is calculated based on the frequency deviation, active power deviation, and recovery time. The agent aims to maximize the cumulative discount reward and optimizes its policy network parameters through interaction with the environment until the policy converges, resulting in an optimized policy network parameter set. The optimized strategy network parameter set is loaded into the real-time controller; in each control cycle, the current system state is obtained and input into the strategy network, the strategy network outputs the corresponding virtual inertia increment and damping coefficient increment, and after smoothing and filtering, the virtual inertia and damping coefficient are updated and output to the VSG control loop.

2. The adaptive control method for grid-type equipment oriented towards high-proportion renewable energy access as described in claim 1, characterized in that, The determination of the range of values ​​for the virtual inertia and the range of values ​​for the damping coefficient includes: Based on the rotor motion equation and power angle relationship of the virtual synchronous generator, the active power-frequency small-signal model, active power response transfer function, and frequency response transfer function are determined. Based on the active power response transfer function and frequency response transfer function, the system is equivalent to a standard second-order system, and the inequality relationship between virtual inertia and damping coefficient is determined using the damping ratio constraint condition. Based on the aforementioned inequality relationship, and in conjunction with the droop rate constraint and the real part constraint of the characteristic root, the range of values ​​for the virtual inertia and the range of values ​​for the damping coefficient are determined.

3. The adaptive control method for grid-type equipment oriented towards high-proportion renewable energy access as described in claim 2, characterized in that, Based on the rotor motion equations and power angle relationships of the virtual synchronous generator, the active power-frequency small-signal model, active power response transfer function, and frequency response transfer function are determined, including: The rotor motion equation of the virtual synchronous generator is determined as follows: ； in, P ref and P e These are the converter's reference active power and output active power, respectively. ω , ω 0 represents the rotor angular velocity and the rated angular velocity, respectively. J , D These are the virtual inertia coefficient and the damping coefficient, respectively. t For time; Define the deviation: ； in, , , and These represent the VSG angular frequency, reference active power, output active power, and the change in power angle, respectively. For the angle of attack, P ref0 This is the initial value of the converter's reference active power. P e0 This represents the initial value of the converter's output active power. δ 0 represents the initial value of the work angle; Substituting the deviation into the rotor motion equation and neglecting the second-order small-signal quantity, we obtain the standard linear form of the active-frequency outer loop: ； In the initial stage of black start, the virtual synchronous generator and the line are equivalently represented as a coupling channel dominated by reactance, and the approximate relationship of the output active power is determined as follows: ； in, E 0 represents the amplitude of the VSG's equivalent internal potential; U g This refers to the amplitude of the terminal voltage. X This is the equivalent reactance between the VSG and the load; The integral relationship between the power angle deviation and the angular velocity deviation is determined as follows: ； in, The rate of change of the work angle; Based on the standard linear form of the active-frequency outer loop, the approximate relationship of active power, and the integral relationship, the small-signal model of VSG is constructed as follows: ； Based on the small-signal model, the active power response transfer function and the frequency response transfer function are determined as follows: ； ； in, G p ( s ) is the active power response transfer function, Δ Pe ( s ) represents the output active power deviation, Δ P ref ( s (This refers to the reference active power deviation.) s For the Laplace operator, K d For the synchronous power coefficient, Δ ω ( s ) represents the angular velocity deviation. G ω ( s ) is the frequency response transfer function.

4. The adaptive control method for grid-type equipment oriented towards high-proportion renewable energy access as described in claim 3, characterized in that, Based on the active power response transfer function and frequency response transfer function, the system is equivalent to a standard second-order system. Using the damping ratio constraint, the inequality relationship between the virtual inertia and the damping coefficient is determined, including: When the power angle is small and the power coupling term is ignored, the system is equivalent to a standard second-order system; The natural oscillation angular frequency is determined based on the active power response transfer function and the frequency response transfer function. Damping ratio The expression is: ； The relationship between overshoot and damping ratio is determined by the following formula: ； in, Let exp be the overshoot, and let exp be the natural exponential function. By constraining the damping ratio within a set range and substituting it into the damping ratio expression, the inequality relationship between the virtual inertia and the damping coefficient is obtained.

5. The adaptive control method for grid-type equipment oriented towards high-proportion renewable energy access as described in claim 4, characterized in that, Based on the aforementioned inequality, and in conjunction with the droop rate constraint and the real part constraint of the eigenvalue, the range of values ​​for the virtual inertia and the damping coefficient are determined, including: Determine the damping coefficient D The relationship between the rate of change and the rate of change is: ； Where, Δ T Δ is the change in mechanical torque. P This represents the change in active power. The range of values ​​for the damping coefficient is determined by considering the maximum allowable frequency deviation during the black start phase. By introducing the real part constraint of the characteristic root and combining it with root locus analysis, the range of values ​​for the virtual inertia is determined.

6. The adaptive control method for grid-type equipment oriented towards high-proportion renewable energy access according to claim 1, characterized in that, The state space is represented as follows: ； in, , These respectively characterize the degree to which the frequency and active power deviate from their rated values; , These represent frequency and the rate of change of active power, respectively, and are used to reflect the strength of disturbances and the inertia support capability.

7. The adaptive control method for grid-type equipment oriented towards high-proportion renewable energy access according to claim 1, characterized in that, The action space is defined as the virtual inertia increment and the damping coefficient increment. The value ranges of the virtual inertia increment and the damping coefficient increment are set according to their respective ranges, ensuring that the updated virtual inertia and damping coefficient do not exceed these ranges. The virtual inertia increment and the damping coefficient increment are then smoothed using a filtering process to obtain the actual execution increment. The virtual inertia and damping coefficient are updated by adding the previous value to the actual execution increment, including: Define the action space as follows: ； in, a t For the action space, , These are the changes in virtual inertia and damping coefficient, respectively. After introducing dynamic smoothing constraints, the corresponding actual execution action increment is: ； in, , The agent at time t is at time t after introducing smoothness constraints. t The output action increment, and satisfies: ； in, The motion smoothing coefficient, , , These represent the actual execution increment from the previous moment; and the motion smoothing coefficient. Determined by the following formula: ； in, T s To enhance learning control of the sampling period, The motion smoothing time constant; After introducing motion smoothing constraints, the actual update expressions for virtual inertia and damping coefficients are: ； in, , This is the actual execution increment after smoothing. J 0 and D 0 represents the initial value of the virtual inertia and the initial value of the damping coefficient, respectively.

8. The adaptive control method for grid-type equipment oriented towards high-proportion renewable energy access according to claim 1, characterized in that, The reward function is calculated based on the frequency deviation, active power deviation, and recovery time, including: According to frequency deviation Δ f Determine the frequency deviation reward ,in α 1 represents the weighting coefficient of the frequency deviation reward function; Based on the active power deviation Δ P Determine the active power deviation reward ,in α 2 represents the weighting coefficient of the active power deviation reward function; Based on recovery time t Determine the recovery time reward r 3: ； in, c 1 represents the penalty amount for the recovery time item. t 0 represents the time window during which no time penalty is applied after the disturbance occurs. Furthermore, if the system has not yet met the steady-state criterion, a constant penalty is applied only to the timeout portion. The steady-state criterion is expressed as: The steady-state criterion is: ； Where, Δ f ( t ) represents the real-time frequency deviation, Δ f min Δ is the allowable threshold for frequency deviation. P ( t ) represents the real-time active power deviation, Δ P min This refers to the allowable threshold for active power deviation. Determine steady-state reward ,in c 2 represents the constant reward value given when the system reaches steady state; Determine the total reward function R for: 。 9. The adaptive control method for grid-type equipment oriented towards high-proportion renewable energy access according to claim 1, characterized in that, When constructing a deep reinforcement learning agent, the adjustment of virtual inertia and damping coefficient is expressed as a Markov decision process, and the key elements of this Markov decision process are... M Represented as: in, S Let be the set of states, representing all possible system states; A The action set represents the actions that the agent can choose in each state; P Transition probability , indicating from state s Take action a After transitioning to state probability; R For reward function , indicating the state s Take action a Immediate benefits; Discount factor; The optimization objective is to find the optimal strategy. This maximizes the expected value of the cumulative discount reward: in, It is a value function; It is in state s t Execute action a t The immediate reward received; Representation Strategy The expectations below.

10. An adaptive control system for grid-connected equipment oriented towards high-proportion renewable energy access, used to implement the method as described in any one of claims 1 to 9, characterized in that, The system includes: The value range determination module is configured to determine the value range of virtual inertia and the value range of damping coefficient based on the topology of the grid-type photovoltaic energy storage system and the rotor motion equation and power angle relationship of the virtual synchronous generator. The optimal control parameter determination module is configured to acquire the system's frequency deviation, frequency change rate, active power deviation, and active power change rate in real time, forming a state space; construct a deep reinforcement learning agent, defining the action space as virtual inertia increment and damping coefficient increment, and setting the value range of the virtual inertia increment and damping coefficient increment according to the value range of the virtual inertia and damping coefficient, so that the updated virtual inertia and damping coefficient do not exceed the value range; perform smoothing filtering on the virtual inertia increment and damping coefficient increment to obtain the actual execution increment, and update the virtual inertia and damping coefficient by adding the actual execution increment to the value at the previous moment; calculate the reward function based on the frequency deviation, active power deviation, and recovery time; the agent aims to maximize the cumulative discount reward, and optimizes its policy network parameters by interacting with the environment until the policy converges, obtaining the optimized policy network parameter set; The real-time control module is configured to load the optimized strategy network parameter set into the real-time controller; in each control cycle, it acquires the current system state and inputs it into the strategy network, which outputs the corresponding virtual inertia increment and damping coefficient increment, updates the virtual inertia and damping coefficient after smoothing and filtering, and outputs it to the VSG control loop.

Images