A reinforcement learning-based method for modeling transient characteristics of wind turbines
By using the improved PILCO probabilistic dynamic reinforcement learning method, transient state transition samples of wind turbines are constructed and trained in stages. This solves the problem of insufficient adaptability of existing wind turbine modeling methods in describing the dynamic evolution after disturbances, and realizes the complete expression of multivariable transient response and risk analysis capabilities.
Patent Information
- Application Number
- CN202611345801.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-01
- Publication Date
- 2026-11-17
AI Technical Summary
Existing wind turbine modeling methods struggle to simultaneously express the state transition relationships between disturbance input, control actions, and multivariable transient responses when describing the dynamic evolution of wind turbines after disturbances. Furthermore, they lack comprehensive constraints on speed exceeding limits, bus voltage fluctuations, converter current surges, active power recovery deviations, and insufficient reactive power support, resulting in insufficient model adaptability.
A transient modeling method based on improved PILCO probabilistic dynamic reinforcement learning is adopted. By constructing transient state transition samples, performing staged training data processing, and combining transient state encoding, disturbance embedding, control action embedding, and Gaussian process dynamics calculation, a transient state transition distribution is generated. Combined with probabilistic rolling prediction and transient constraint reward feedback correction, a transient probabilistic dynamic model of wind turbine is established.
It achieves multi-dimensional state coverage for transient modeling of wind turbine units, which can express the transient state change patterns and uncertainty range under different disturbance stages, improves the stability and adaptability of the model under complex wind conditions and control switching conditions, and provides detailed basis for transient risk analysis.
Smart Images

Figure CN123115950A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of wind turbine operation modeling and intelligent control technology, specifically to a method for modeling the transient characteristics of wind turbines based on reinforcement learning. Background Technology
[0002] During grid-connected operation, wind turbines are affected by factors such as sudden wind speed changes, grid voltage drops, power command switching, pitch lag, converter current limiting, and inertial coupling. Their operating state may exhibit transient responses within a short period, including speed fluctuations, active power recovery, reactive power support changes, DC bus voltage oscillations, and converter current surges. Transient characteristic modeling provides a basis for wind turbine fault analysis, low-voltage ride-through assessment, control parameter optimization, and grid-connected stability assessment. Therefore, accurately describing the dynamic evolution of wind turbines after disturbances is a crucial issue in intelligent wind power operation analysis.
[0003] Existing wind turbine modeling methods mostly employ mechanistic models, equivalent models, or data-driven predictive models. Mechanistic models rely on precise parameters of aerodynamic, mechanical, electrical, and control components, making the modeling process complex and parameter identification difficult. They also lack adaptability to different turbine models, control strategies, and complex grid-connected environments. Traditional data-driven models typically fit power, speed, or voltage changes to historical sequences. While they can describe some operating trends, most methods focus on univariate prediction or steady-state power curve fitting, making it difficult to simultaneously express the state transition relationships between disturbance inputs, control actions, and multivariate transient responses.
[0004] In transient modeling scenarios, the response process of wind turbines is characterized by stages and uncertainties. The same type of disturbance may produce different recovery trajectories under different wind speeds, pitch angles, converter current limiting states, and grid support conditions. Existing methods often use fixed error loss to train models, lacking comprehensive constraints on speed overruns, bus voltage fluctuations, converter current surges, active power recovery deviations, insufficient reactive power support, and transient recovery time. As a result, although the model can fit local curves, it is difficult to form a transient dynamic model with probability distribution expression and risk constraint feedback.
[0005] Therefore, how to provide a reinforcement learning-based method for modeling the transient characteristics of wind turbines is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0006] One objective of this invention is to propose a method for modeling the transient characteristics of wind turbine generators based on reinforcement learning. This invention fully utilizes an improved PILCO probabilistic dynamic reinforcement learning transient modeling model, constructing transient state transition samples from wind turbine generator operation sampling data, and forming staged transient training data according to disturbance type and response stage. Transient state transition distributions are generated through transient state encoding, disturbance embedding, control action embedding, and Gaussian process dynamics calculation. Furthermore, by combining probabilistic rolling prediction and transient constraint reward feedback correction mechanisms, a transient probabilistic dynamic model of wind turbine generators is established. The invention details the modeling process from disturbance triggering, stage division, state transition distribution learning, transient trajectory generation to probabilistic characteristic curve output. It has the advantages of complete transient response expression, strong ability to characterize model uncertainty, high adaptability to complex disturbance conditions, and the ability to use the modeling results for transient risk analysis.
[0007] A method for modeling the transient characteristics of wind turbine generators based on reinforcement learning according to an embodiment of the present invention includes the following steps: S1. Obtain the operating sampling data of the wind turbine during the transient modeling period, perform time alignment, disturbance window truncation and state pairing on the operating sampling data, and generate transient state transition samples. S2. Divide the transient state transition samples into stages according to the type of disturbance, the time of occurrence of the control action and the time of state recovery to obtain staged transient training data. S3. Input the staged transient training data into the improved PILCO probabilistic dynamic reinforcement learning transient modeling model, and obtain the transient state transition distribution through transient state encoding, control action embedding, disturbance embedding and Gaussian process dynamics calculation. S4. Based on the transient state transition distribution, perform probabilistic rolling prediction by continuously inputting the state distribution and control action sequence of the previous prediction time to obtain candidate transient evolution trajectories. S5. Based on the transient constraint reward value, the candidate transient evolution trajectory is corrected by reward feedback, and the Gaussian process dynamic parameters, stage kernel parameters and strategy parameters are updated to obtain the transient probabilistic dynamic model of the wind turbine. S6. Input the transient data to be modeled into the transient probabilistic dynamics model of the wind turbine, and output the transient probabilistic characteristic curve cluster and transient characteristic record of the wind turbine.
[0008] Optionally, step S1 includes the following steps: S11. Obtain the wind speed sequence, generator speed sequence, pitch angle sequence, electromagnetic torque sequence, active power sequence, reactive power sequence, converter current sequence, DC bus voltage sequence and grid voltage sequence of the wind turbine, and unify the timestamps and unit numbers of each sequence to obtain the multi-source transient sampling sequence. S12. Perform sampling frequency unification, time alignment and missing segment marking on the multi-source transient sampling sequence, and write the wind speed, rotational speed, power, voltage, current, pitch angle and torque corresponding to the same sampling time into the same sampling record to obtain time-aligned transient data; S13. Based on the grid voltage drop point, wind speed step point, power command switching point, pitch action start point and converter current limiting trigger point in the time-aligned transient data, extract the continuous time window before and after the time window to obtain the disturbance window data containing the state before disturbance, the state of disturbance response and the state of recovery. S14. Extract the current state quantity, disturbance input quantity, control action quantity and next moment state quantity from the disturbance window data, and establish a corresponding relationship according to the unit number, disturbance number and sampling sequence number to generate transient state transition samples.
[0009] Optionally, step S2 includes the following steps: S21. Read the disturbance input and control action quantities in the transient state transition sample, and identify low voltage ride-through disturbance, wind speed step disturbance, power command switching disturbance and converter current limiting disturbance based on voltage drop amplitude, wind speed change rate, power command change rate and current limiting mark, and obtain disturbance type label. S22. Read the generator speed change rate, active power recovery slope, DC bus voltage fluctuation rate and converter current peak change in the transient state transition sample, divide the transient state transition sample into disturbance triggering stage, fast response stage, oscillation decay stage and recovery stabilization stage, and obtain stage labels; S23. Associate the disturbance type label, stage label and the state transition relationship in the transient state transition sample to form a labeled state transition sample; S24. Group the labeled state transition samples according to the perturbation type label and the stage label, and record the start time, end time, sampling length and variable missing label of each group of samples to obtain staged transient training data.
[0010] Optionally, step S3 includes the following steps: S31. Read the pre-disturbance state quantity, disturbance input quantity, control action quantity, post-disturbance state quantity, disturbance type label and stage label from the staged transient training data, and arrange the speed, power, voltage, current, pitch angle and torque according to the sampling time to generate a transient state vector. S32. Input the voltage drop amplitude, wind speed change rate and power command change rate into the disturbance embedding branch to generate the disturbance embedding vector. Input the pitch angle adjustment amount, torque control amount and converter reactive power regulation amount into the action embedding branch to generate the action embedding vector. S33. Input the transient state vector, perturbation embedding vector, action embedding vector, perturbation type label and stage label into the improved PILCO probabilistic dynamic reinforcement learning transient modeling model to obtain dynamic input features; S34. In the improved PILCO probabilistic dynamic reinforcement learning transient modeling model, the Gaussian process dynamics branch is called, and the kernel similarity matrix is calculated based on the dynamic input features to obtain the state transition kernel matrix corresponding to the perturbation stage. S35. Calculate the state transition mean based on the state transition kernel matrix and the state variables after disturbance, and calculate the state transition variance based on the sampling residual and the state transition kernel matrix to obtain the state mean and state variance. S36. Write the state mean, state variance, disturbance type label, stage label, stage kernel parameter and sample index into the same distribution record to obtain the transient state transition distribution.
[0011] Optionally, step S34 includes the following steps: S341. In the improved PILCO probabilistic dynamic reinforcement learning transient modeling model, a transient state encoding branch is established, which maps the speed, power, voltage, current, pitch angle and torque into continuous state features respectively, and concatenates the continuous state features with the stage label to obtain the stage state features. S342. Combine the stage state features, disturbance embedding vector and action embedding vector into dynamic input features, and select the low voltage ride-through kernel parameter group, wind speed step kernel parameter group, power command switching kernel parameter group or converter current limiting kernel parameter group according to the disturbance type label. S343. Input the dynamic input features into the corresponding kernel parameter group, calculate the kernel similarity between different sampling points, and write the kernel similarity into the kernel matrix position according to the stage label to obtain the stage kernel similarity matrix. S344. Based on the stage kernel similarity matrix, establish a probabilistic dynamic mapping from the current state to the next time step state, and bind the probabilistic dynamic mapping with the sample index of the training samples to obtain the initial dynamic parameters of the improved PILCO probabilistic dynamic reinforcement learning transient modeling model.
[0012] Optionally, step S4 includes the following steps: S41. Read the state transition mean, state transition variance and stage kernel parameters in the transient state transition distribution, and use the wind turbine transient state at the start time as the initial input for rolling prediction to obtain the initial state distribution. S42. Input the initial state distribution and the first action quantity in the control action sequence into the Gaussian process dynamics branch, calculate the state mean and state variance at the next prediction time, and obtain the state distribution at the next prediction time. S43. Write the state distribution of the next prediction time into the trajectory buffer, and use the latest state distribution in the trajectory buffer as the next round of rolling input. Repeat the state distribution prediction according to the control action sequence to obtain the state distribution sequence of continuous prediction times. S44. Extract the speed trajectory, active power trajectory, reactive power trajectory, DC bus voltage trajectory, converter current trajectory, and pitch angle trajectory from the state distribution sequence to generate a multivariable transient trajectory. S45. Configure uncertainty boundaries for multivariate transient trajectories based on state transition variance, and combine multivariate transient trajectories with corresponding uncertainty boundaries to obtain candidate transient evolution trajectories.
[0013] Optionally, step S5 includes the following steps: S51. Read the speed trajectory, active power trajectory, reactive power trajectory, DC bus voltage trajectory, converter current trajectory and pitch angle trajectory in the candidate transient evolution trajectory, and compare each trajectory with the corresponding operating constraint threshold time by time to obtain the trajectory constraint deviation. S52. Calculate the speed over-limit deviation, DC bus voltage fluctuation deviation, converter current impact deviation, active power recovery deviation, reactive power support deviation, pitch angle sudden change deviation, and transient recovery time deviation based on the trajectory constraint deviation to obtain the constraint deviation set. S53. Accumulate the constraint deviation set in segments according to the disturbance triggering stage, rapid response stage, oscillation decay stage and recovery stabilization stage to generate a stage constraint deviation sequence. S54. Calculate the transient constraint reward value based on the stage constraint deviation sequence, and establish a correlation between the transient constraint reward value and the candidate transient evolution trajectory, disturbance type label and stage label to obtain reward-correlation trajectory data; S55. Feedback the reward-related trajectory data to the improved PILCO probabilistic dynamic reinforcement learning transient model, perform reward feedback correction, and obtain the corrected model data; S56. Perform state boundary verification and phase consistency verification on the modified model data, and write the modified model data that meets the verification conditions into the model trajectory library to obtain the transient probabilistic dynamic model of the wind turbine.
[0014] Optionally, S55 includes the following steps: S551. Read the transient constraint reward value and candidate transient evolution trajectory from the reward-related trajectory data, calculate the strategy update direction based on the transient constraint reward value, and map the strategy update direction to the pitch angle adjustment, torque control and converter reactive power regulation to obtain the candidate control action correction amount. S552. Recombine the candidate control action correction, transient state vector, disturbance embedding vector and stage label to generate corrected dynamic input features, and input the corrected dynamic input features into the Gaussian process dynamics branch to recalculate the corrected state transition mean and corrected state transition variance. S553. Generate a modified state transition distribution based on the modified state transition mean and the modified state transition variance, and input the modified state transition distribution into the probability rolling prediction process to generate a modified transient evolution trajectory. S554. Compare the reward values of the corrected transient evolution trajectory and the candidate transient evolution trajectory, and write the trajectory with the higher reward value, the corresponding stage reward sequence, the corrected state transition distribution, and the candidate control action correction amount into the model trajectory library. S555. Update the Gaussian process dynamic parameters, stage kernel parameters, and strategy parameters according to the model trajectory library to obtain the transient probabilistic dynamic model of the wind turbine.
[0015] Optionally, step S6 includes the following steps: S61. Obtain the initial state quantity, disturbance input quantity, and control action sequence from the transient data to be modeled, and perform time alignment on the initial state quantity, disturbance input quantity, and control action sequence to obtain the input sample to be modeled. S62. Input the input sample to be modeled into the transient probabilistic dynamics model of the wind turbine, call the Gaussian process dynamics branch and probabilistic rolling prediction process in the improved PILCO probabilistic dynamics reinforcement learning transient modeling model, and generate the distribution sequence of the state to be modeled. S63. Extract the speed probability curve, active power recovery probability curve, reactive power support probability curve, DC bus voltage fluctuation probability curve, converter current impact probability curve and pitch angle response probability curve from the state distribution sequence to be modeled, and obtain the wind turbine transient probability characteristic curve cluster. S64. Calculate the transient recovery time and transient risk score based on the distribution sequence of the state to be modeled, bind the transient probability characteristic curve cluster of the wind turbine, transient recovery time, transient risk score, disturbance type label and stage label, and output the transient characteristic record.
[0016] The beneficial effects of this invention are as follows: By constructing transient state transition samples, this invention uniformly maps disturbance processes such as sudden wind speed changes, grid voltage drops, power command switching, pitch control actions, and converter current limiting into a state transition relationship of "current state quantity, disturbance input quantity, control action quantity, and next moment state quantity". This avoids the problem of traditional methods that only perform curve fitting on a single variable, and enables the transient modeling of wind turbine units to simultaneously cover multi-dimensional operating states such as speed, power, voltage, current, pitch angle, and torque.
[0017] This invention introduces an improved PILCO probabilistic dynamic reinforcement learning transient modeling model. In the Gaussian process dynamics branch, stage kernel parameters are established based on the disturbance type and response stage, and the mean and variance of state transitions are output. This enables the model to express the transient state change patterns and uncertainty range under different disturbance stages. Through a probabilistic rolling prediction mechanism, the model can continuously generate multi-time-stamp transient evolution trajectories, forming probability curves for speed, active power recovery, reactive power support, DC bus voltage fluctuation, converter current impact, and pitch angle response, thus providing more complete transient characteristic results.
[0018] This invention further incorporates speed exceedance, DC bus voltage fluctuations, converter current surges, active power recovery deviations, reactive power support deviations, sudden changes in pitch angle, and transient recovery time into the transient constraint reward value. Through reward feedback, the Gaussian process dynamic parameters, stage kernel parameters, and strategy parameters are corrected. This allows the model training process to not only rely on numerical errors but also to be dynamically corrected in conjunction with the transient safety constraints of the wind turbine. This improves the stability, adaptability, and engineering application value of transient modeling under complex wind conditions, weak grid disturbances, and control switching conditions. Attached Figure Description
[0019] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is an overall flowchart of a reinforcement learning-based method for modeling the transient characteristics of wind turbines. Figure 2 A schematic diagram of the structure of the improved PILCO probabilistic dynamics reinforcement learning transient modeling model; Figure 3 This is a schematic diagram of the process of transient constraint reward feedback correction and transient probability characteristic curve cluster generation. Detailed Implementation
[0020] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0021] refer to Figures 1-3 A method for modeling the transient characteristics of wind turbine generators based on reinforcement learning includes the following steps: S1. Obtain the operating sampling data of the wind turbine during the transient modeling period, perform time alignment, disturbance window truncation and state pairing on the operating sampling data, and generate transient state transition samples. S2. Divide the transient state transition samples into stages according to the type of disturbance, the time of occurrence of the control action and the time of state recovery to obtain staged transient training data. S3. Input the staged transient training data into the improved PILCO probabilistic dynamic reinforcement learning transient modeling model, and obtain the transient state transition distribution through transient state encoding, control action embedding, disturbance embedding and Gaussian process dynamics calculation. S4. Based on the transient state transition distribution, perform probabilistic rolling prediction by continuously inputting the state distribution and control action sequence of the previous prediction time to obtain candidate transient evolution trajectories. S5. Based on the transient constraint reward value, the candidate transient evolution trajectory is corrected by reward feedback, and the Gaussian process dynamic parameters, stage kernel parameters and strategy parameters are updated to obtain the transient probabilistic dynamic model of the wind turbine. S6. Input the transient data to be modeled into the transient probabilistic dynamics model of the wind turbine, and output the transient probabilistic characteristic curve cluster and transient characteristic record of the wind turbine.
[0022] refer to Figure 1 In this embodiment, S1 specifically includes: The wind speed sequence, generator speed sequence, pitch angle sequence, electromagnetic torque sequence, active power sequence, reactive power sequence, converter current sequence, DC bus voltage sequence, and grid voltage sequence of the wind turbine are obtained. The timestamps of each sequence are unified and the unit number is marked to obtain the multi-source transient sampling sequence. The sampling frequency of the multi-source transient sampling sequence is unified, the time is aligned, and missing segments are marked. The wind speed, rotational speed, power, voltage, current, pitch angle, and torque corresponding to the same sampling time are written into the same sampling record to obtain time-aligned transient data. Based on the grid voltage drop point, wind speed step point, power command switching point, pitch action start point and converter current limiting trigger point in the time-aligned transient data, continuous time windows before and after are extracted to obtain disturbance window data that includes the state before disturbance, the state of disturbance response and the state of recovery. Extract the current state quantity, disturbance input quantity, control action quantity, and next moment state quantity from the disturbance window data, and establish a correspondence according to the unit number, disturbance number, and sampling sequence number to generate transient state transition samples.
[0023] In this embodiment, the sources of the multi-source transient sampling sequence include the wind farm SCADA system, wind turbine main controller log, converter controller record, pitch control system record, grid-side fault waveform data, and wind turbine simulation platform output data.
[0024] The on-site sampling data are arranged in order of sampling time, with each sampling time corresponding to a set of unit state variables; the simulation platform output data covers disturbance scenarios such as low voltage ride-through, wind speed step, rapid power command switching, converter current limiting, and pitch hysteresis.
[0025] To ensure that field-sampled data and simulation output data enter the same training process, a unified sampling interval reconstruction is performed on data from different sources, and the unit number, data source, sampling time, disturbance number, and variable name are written into the sampling record.
[0026] Each disturbance window forms a sample table with the sampling time as the row and the speed, power, voltage, current, pitch angle, torque, wind speed, and disturbance input as the columns. The sample table is stored in association with the disturbance number, window number, and data source identifier.
[0027] In this embodiment, a missing marker is written for short-term missing segments and the missing marker is bound to the corresponding sampling record; a removal marker is established for disturbance windows with continuous missing lengths exceeding a preset length, and the removal marker is associated with the disturbance number, unit number and variable name to form trainable disturbance window data.
[0028] For grid voltage drop points, the grid voltage sequence is determined by falling below a preset voltage ratio and continuously reaching a preset number of sampling points; for wind speed step points, the wind speed change rate within adjacent sampling windows exceeds a preset change threshold; for power command switching points, the target power change is determined by the power command record; for pitch action start points, the pitch angle change rate exceeds the action trigger threshold; for converter current limiting trigger points, the converter current reaches the current limiting threshold and is accompanied by the controller current limiting flag.
[0029] Based on the aforementioned trigger points, continuous segments before the disturbance, continuous segments of the disturbance response, and continuous segments of the disturbance recovery are extracted to generate disturbance window data.
[0030] In this embodiment, the current state quantities include the generator speed, active power, reactive power, DC bus voltage, converter current, pitch angle, and electromagnetic torque at the current sampling time; the disturbance input quantities include voltage drop amplitude, wind speed change rate, power command change rate, and current limiting trigger flag; the control action quantities include pitch angle adjustment, torque control, and converter reactive power regulation; and the state quantities at the next sampling time include the generator speed, active power, reactive power, DC bus voltage, converter current, pitch angle, and electromagnetic torque at the next sampling time.
[0031] The current state quantity, disturbance input quantity, and control action quantity together constitute the state transition input, and the state quantity at the next moment constitutes the state transition output. The state transition input and state transition output are bound one by one according to the sampling sequence number to generate transient state transition samples.
[0032] refer to Figure 1 In this embodiment, S2 specifically includes: Read the disturbance input and control action quantities in the transient state transition sample, and identify low voltage ride-through disturbance, wind speed step disturbance, power command switching disturbance and converter current limiting disturbance based on voltage drop amplitude, wind speed change rate, power command change rate and current limiting mark, and obtain disturbance type label; The generator speed change rate, active power recovery slope, DC bus voltage fluctuation rate, and converter current peak change in the transient state transition samples are read. The transient state transition samples are divided into disturbance triggering stage, fast response stage, oscillation decay stage, and recovery stabilization stage, and stage labels are obtained. The disturbance type label, stage label, and state transition relationship in the transient state transition sample are associated to form labeled state transition samples; The labeled state transition samples are grouped according to the perturbation type label and the stage label, and the start time, end time, sampling length and variable missing label of each group are recorded to obtain staged transient training data.
[0033] In this embodiment, the disturbance type label is determined by the main controller event code, converter protection record, power grid fault waveform record, and variable threshold.
[0034] Low voltage ride-through disturbance corresponds to a sudden drop in grid voltage accompanied by reactive power support action; wind speed step disturbance corresponds to a step change in wind speed within a short period of time, causing speed and power response; power command switching disturbance corresponds to a rapid change in the active power target value, triggering torque control adjustment; converter current limiting disturbance corresponds to a current limiting range for converter current, causing power and bus voltage changes.
[0035] When multiple disturbance markers appear in the same disturbance window at the same time, the disturbance with the earliest start time and amplitude exceeding the main disturbance threshold is taken as the main disturbance type, and the other disturbances are written into the additional disturbance markers.
[0036] In this embodiment, the stage labels are generated according to the characteristics of state changes. The disturbance triggering stage is determined based on the grid voltage drop point, wind speed step point, power command switching point, or converter current limiting trigger point; the fast response stage is determined based on the pitch angle change rate, torque control change rate, converter current change rate, and reactive power regulation change rate; the oscillation decay stage is determined based on the generator speed fluctuation amplitude, active power fluctuation amplitude, DC bus voltage fluctuation amplitude, and converter current fluctuation amplitude; the recovery stabilization stage is determined based on the sampling length during which active power, grid voltage, DC bus voltage, converter current, and speed are continuously within the set operating range.
[0037] After the phase division is completed, each sample carries both a perturbation type label and a phase label, enabling the subsequent improved PILCO probabilistic dynamic reinforcement learning transient modeling model to select different phase kernel parameters according to the perturbation category and response phase.
[0038] refer to Figure 2 In this embodiment, S3 specifically includes: Read the pre-disturbance state variables, disturbance input variables, control action variables, post-disturbance state variables, disturbance type labels, and stage labels from the staged transient training data, and arrange the speed, power, voltage, current, pitch angle, and torque according to the sampling time to generate a transient state vector; Input the voltage drop amplitude, wind speed change rate, and power command change rate into the disturbance embedding branch to generate the disturbance embedding vector; input the pitch angle adjustment, torque control, and converter reactive power regulation into the action embedding branch to generate the action embedding vector. The transient state vector, perturbation embedding vector, action embedding vector, perturbation type label, and stage label are input into the improved PILCO probabilistic dynamic reinforcement learning transient modeling model to obtain dynamic input features; In the improved PILCO probabilistic dynamic reinforcement learning transient modeling model, the Gaussian process dynamics branch is invoked, and the kernel similarity matrix is calculated based on the dynamic input features to obtain the state transition kernel matrix corresponding to the perturbation stage; The state transition mean is calculated based on the state transition kernel matrix and the state variables after perturbation, and the state transition variance is calculated based on the sampling residual and the state transition kernel matrix to obtain the state mean and state variance. Write the state mean, state variance, disturbance type label, stage label, stage kernel parameter, and sample index into the same distribution record to obtain the transient state transition distribution.
[0039] refer to Figure 2 In this embodiment, the improved PILCO probabilistic dynamic reinforcement learning transient modeling model includes a transient state encoding branch, a perturbation embedding branch, an action embedding branch, a dynamic input feature construction layer, a stage kernel parameter selection layer, a Gaussian process dynamics branch, a probabilistic rolling prediction layer, a transient constraint reward layer, a reward feedback correction layer, and a model trajectory library.
[0040] The transient state encoding branch receives speed, power, voltage, current, pitch angle, and torque status, and normalizes the operating variables of different dimensions to map them into continuous state characteristics; the disturbance embedding branch receives voltage drop amplitude, wind speed change rate, power command change rate, and current limiting trigger flag, and maps disturbance amplitude, disturbance direction, and disturbance duration sampling length into disturbance embedding vectors; the action embedding branch receives pitch angle adjustment, torque control, and converter reactive power regulation, and maps control action amplitude, action direction, and action trigger time into action embedding vectors.
[0041] The outputs of the transient state encoding branch, the perturbation embedding branch, and the action embedding branch are all connected to the dynamic input feature construction layer. The perturbation type label and the stage label are synchronously written into the dynamic input feature construction layer to form dynamic input features.
[0042] In this embodiment, the improved PILCO probabilistic dynamic reinforcement learning transient modeling model, compared to the probabilistic dynamic model with a single kernel parameter, adds disturbance type label input, response stage label input, stage kernel parameter selection layer, and transient constraint reward feedback layer, so that the Gaussian process dynamics branch forms corresponding state transition distributions under different disturbance types and different response stages.
[0043] The stage kernel parameter selection layer selects the low voltage ride-through kernel parameter group, wind speed step kernel parameter group, power command switching kernel parameter group, or converter current limiting kernel parameter group according to the disturbance type label, and selects the disturbance trigger stage kernel parameter, fast response stage kernel parameter, oscillation decay stage kernel parameter, or recovery stabilization stage kernel parameter within the corresponding kernel parameter group according to the stage label.
[0044] Each stage kernel parameter includes a length scale parameter, a noise parameter, an output scale parameter, and a stage weight parameter. The stage kernel parameters are bound to the sample index and then input into the Gaussian process dynamics branch.
[0045] In this embodiment, a transient state encoding branch is established in the improved PILCO probabilistic dynamic reinforcement learning transient modeling model. The rotational speed, power, voltage, current, pitch angle and torque are mapped to continuous state features respectively, and the continuous state features are concatenated with stage labels to obtain stage state features. The stage state features, disturbance embedding vectors, and action embedding vectors are combined into dynamic input features, and low voltage ride-through kernel parameter groups, wind speed step kernel parameter groups, power command switching kernel parameter groups, or converter current limiting kernel parameter groups are selected according to the disturbance type label. Input the dynamic input features into the corresponding kernel parameter set, calculate the kernel similarity between different sampling points, and write the kernel similarity into the kernel matrix position according to the stage label to obtain the stage kernel similarity matrix; Based on the stage kernel similarity matrix, a probabilistic dynamic mapping from the current state to the next time step is established, and the probabilistic dynamic mapping is bound to the sample index of the training samples to obtain the initial dynamic parameters of the improved PILCO probabilistic dynamic reinforcement learning transient modeling model.
[0046] In this embodiment, the Gaussian process dynamics branch takes the dynamic input characteristics as input and the perturbation state variables as the monitoring targets, and outputs the mean and variance of the state transition from the current state to the state at the next time step.
[0047] The state transition mean corresponds to the predicted center values of speed, power, voltage, current, pitch angle, and torque at the next moment; the state transition variance corresponds to the uncertainty range of each state variable at the next moment.
[0048] For low-voltage ride-through samples, the Gaussian process dynamics branch focuses on establishing the state transition relationships between grid voltage, reactive power, DC bus voltage, and converter current; for wind speed step samples, the Gaussian process dynamics branch focuses on establishing the state transition relationships between wind speed, speed, pitch angle, active power, and electromagnetic torque; for power command switching samples, the Gaussian process dynamics branch focuses on establishing the state transition relationships between power command, torque control, active power, and speed; for converter current limiting samples, the Gaussian process dynamics branch focuses on establishing the state transition relationships between converter current, bus voltage, reactive power, and active power.
[0049] In this embodiment, the phased transient training data is divided into a training set, a validation set, and a test set. The training set includes field-collected samples and simulation-generated samples. The field-collected samples are obtained from the operating records of multiple wind turbines under different wind speed ranges, different grid connection states, and different disturbance conditions. The simulation-generated samples are generated by the wind turbine electrical control simulation platform.
[0050] The simulation samples provide low voltage ride-through, deep voltage drop, wind speed step, rapid power command switching, and current limiting impact scenarios, while the field-collected samples provide realistic pitch response, realistic torque control response, and realistic converter reactive power regulation response.
[0051] By aligning the simulation-generated samples with the field-collected samples using perturbation type labels and stage labels, and correcting the training weights of the simulation-generated samples based on the state transition errors of the field-collected samples, a virtual-real joint transient training sample is obtained.
[0052] In this embodiment, the loss function for model training consists of the state transition mean error, variance calibration error, stage consistency error, transient trajectory constraint error, and policy reward error.
[0053] The state transition mean error is determined based on the deviation between the predicted mean state at the next time step and the actual state at the next time step; the variance calibration error is determined based on the proportion of the actual state falling within the prediction uncertainty boundary; the stage consistency error is determined based on the deviation between the predicted trajectory stage changes and the stage labels; the transient trajectory constraint error is determined based on whether the speed, voltage, current, power, and pitch angle exceed the operating constraint range; and the strategy reward error is determined based on the change in the transient constraint reward value after the candidate control action is corrected. These losses are combined according to preset weights to drive the updating of the Gaussian process dynamics parameters, stage kernel parameters, and strategy parameters.
[0054] In this embodiment, the training parameters can be set as follows: each training batch contains 16 to 64 perturbation windows, the probability rolling prediction length is set to 20 to 100 sampling points, the initial learning rate is set to 0.0001 to 0.001, the initial length scale of the stage kernel parameter is determined according to the state distance between similar perturbation samples, and the noise parameter is determined according to the fluctuation amplitude of the same variable in the stable segment before the perturbation.
[0055] During training, when the mean error of the validation set state transition changes below a preset proportion for 5 to 10 consecutive times, the variance calibration error remains within a set range, the stage consistency error no longer decreases, and the transient constraint reward value tends to stabilize, training is stopped and the model parameters are saved. The saved content includes transient state encoding branch parameters, perturbation embedding branch parameters, action embedding branch parameters, Gaussian process dynamics parameters, stage kernel parameters, policy parameters, and the model trajectory library index.
[0056] refer to Figure 1 and Figure 3 In this embodiment, S4 specifically includes: Read the state transition mean, state transition variance and stage kernel parameters from the transient state transition distribution, and use the wind turbine transient state at the start time as the initial input for rolling prediction to obtain the initial state distribution; Input the initial state distribution and the first action quantity in the control action sequence into the Gaussian process dynamics branch, calculate the state mean and state variance at the next prediction time, and obtain the state distribution at the next prediction time. Write the state distribution of the next prediction time into the trajectory buffer, and use the latest state distribution in the trajectory buffer as the input for the next round of rolling. Repeat the state distribution prediction according to the control action sequence to obtain the state distribution sequence of continuous prediction times. Extract the speed trajectory, active power trajectory, reactive power trajectory, DC bus voltage trajectory, converter current trajectory, and pitch angle trajectory from the state distribution sequence to generate a multivariate transient trajectory; Based on the state transition variance, an uncertainty boundary is configured for the multivariate transient trajectory, and the multivariate transient trajectory is combined with the corresponding uncertainty boundary to obtain the candidate transient evolution trajectory.
[0057] In this embodiment, the probabilistic rolling prediction layer is connected to the Gaussian process dynamics branch. After receiving the initial state distribution and control action sequence, the probabilistic rolling prediction layer first calls the Gaussian process dynamics branch to obtain the state mean and state variance at the next prediction time, and then writes the state distribution at the next prediction time into the trajectory buffer.
[0058] The trajectory cache records the state mean, state variance, control action amount, disturbance type label, stage label, and stage kernel parameter in sequence according to the prediction time. When the state change corresponding to the prediction time enters the next response stage, the stage kernel parameter selection layer switches the stage kernel parameter according to the new stage label and writes the switched stage kernel parameter into the subsequent prediction record.
[0059] Through the above data transmission method, the state distribution is continuously transmitted between multiple prediction times, forming a multivariate transient trajectory.
[0060] In this embodiment, the candidate transient evolution trajectory includes the rotational speed trajectory, active power trajectory, reactive power trajectory, DC bus voltage trajectory, converter current trajectory, and pitch angle trajectory. Each trajectory contains a state mean sequence and an uncertainty boundary sequence.
[0061] For the speed trajectory, the uncertainty boundary is generated based on the generator speed state variance; for the active power trajectory and reactive power trajectory, the uncertainty boundary is generated based on the power state variance; for the DC bus voltage trajectory and converter current trajectory, the uncertainty boundary is generated based on the voltage and current state variances; for the pitch angle trajectory, the uncertainty boundary is generated based on the pitch angle state variance.
[0062] After the candidate transient evolution trajectory is bound with the disturbance number, control action sequence and stage label, it is input into the transient constraint reward layer.
[0063] refer to Figure 3 In this embodiment, S5 specifically includes: Read the speed trajectory, active power trajectory, reactive power trajectory, DC bus voltage trajectory, converter current trajectory and pitch angle trajectory from the candidate transient evolution trajectory, and compare each trajectory with the corresponding operating constraint threshold time by time to obtain the trajectory constraint deviation; Based on the trajectory constraint deviation, the following deviations are calculated: speed over-limit deviation, DC bus voltage fluctuation deviation, converter current impact deviation, active power recovery deviation, reactive power support deviation, pitch angle sudden change deviation, and transient recovery time deviation, thus obtaining the constraint deviation set. The constraint deviation set is segmented and accumulated according to the disturbance triggering stage, the rapid response stage, the oscillation decay stage, and the recovery and stabilization stage to generate a stage constraint deviation sequence. The transient constraint reward value is calculated based on the stage constraint deviation sequence, and the transient constraint reward value is associated with the candidate transient evolution trajectory, the disturbance type label and the stage label to obtain the reward-associated trajectory data; The reward-related trajectory data is fed back into the improved PILCO probabilistic dynamic reinforcement learning transient modeling model, and reward feedback correction is performed to obtain the corrected model data; The modified model data is subjected to state boundary verification and stage consistency verification. The modified model data that meets the verification conditions is written into the model trajectory library to obtain the transient probabilistic dynamic model of the wind turbine.
[0064] In this embodiment, after receiving the candidate transient evolution trajectory, the transient constraint reward layer calculates the stage constraint deviation according to the grid-connected operation constraint of the wind turbine.
[0065] The speed over-limit deviation is determined by the magnitude of the generator speed exceeding the upper limit or falling below the lower limit and the number of continuous sampling points; the DC bus voltage fluctuation deviation is determined by the magnitude of the bus voltage deviating from the rated range and the recovery time; the converter current impulse deviation is determined by the current peak value, peak duration, and current change rate; the active power recovery deviation is determined by the number of sampling points required for active power to recover to the target range and the deviation magnitude during the recovery process; the reactive power support deviation is determined by the number of sampling points where reactive power fails to meet the support requirements during the disturbance period and the deviation magnitude; the pitch angle action sudden change deviation is determined by the pitch angle change amplitude and action rate; and the transient recovery time deviation is determined by the difference between the initial sampling time when the main state variables are continuously in the stable range and the reference recovery time.
[0066] The constraint deviation set is accumulated segmented according to the stage label to generate the stage constraint deviation sequence.
[0067] In this embodiment, the transient constraint reward value is obtained by converting the stage constraint deviation sequence.
[0068] The key calculations for the disturbance triggering phase are: the initial deviation after voltage drop, wind speed jump, or current limiting triggering; the key calculations for the rapid response phase are: the state response deviation corresponding to pitch angle adjustment, torque control, and reactive power regulation; the key calculations for the oscillation decay phase are: the oscillation amplitude and decay rate of speed, power, voltage, and current; and the key calculations for the recovery and stabilization phase are: the recovery time and the length of the stabilization interval.
[0069] After the transient constraint reward value is bound to the candidate transient evolution trajectory, reward-related trajectory data is formed, and the reward-related trajectory data enters the reward feedback correction layer.
[0070] In this embodiment, the strategy parameters include pitch angle adjustment strategy parameters, torque control strategy parameters, and converter reactive power regulation strategy parameters.
[0071] The pitch angle adjustment strategy parameters correspond to the correction direction and correction magnitude of the pitch angle adjustment amount under different disturbance stages; the torque control strategy parameters correspond to the correction direction and correction magnitude of the electromagnetic torque control amount during power recovery and speed suppression; the converter reactive power regulation strategy parameters correspond to the correction direction and correction magnitude between reactive power support and bus voltage constraint.
[0072] A correspondence is established between the strategy parameters and the transient constraint reward values. The transient constraint reward values generate the strategy update direction, which is mapped to the candidate control action correction amount. The candidate control action correction amount re-enters the Gaussian process dynamics branch and generates the corrected state transition distribution.
[0073] In this embodiment, the transient constraint reward value and candidate transient evolution trajectory in the reward-related trajectory data are read, the strategy update direction is calculated based on the transient constraint reward value, and the strategy update direction is mapped to the pitch angle adjustment amount, torque control amount and converter reactive power regulation amount to obtain the candidate control action correction amount; The candidate control action correction, transient state vector, disturbance embedding vector and stage label are recombined to generate corrected dynamic input features. The corrected dynamic input features are then input into the Gaussian process dynamics branch to recalculate the corrected state transition mean and corrected state transition variance. A modified state transition distribution is generated based on the modified state transition mean and the modified state transition variance. The modified state transition distribution is then input into the probability rolling prediction process to generate the modified transient evolution trajectory. The corrected transient evolution trajectory is compared with the candidate transient evolution trajectory in terms of reward value. The trajectory with higher reward value, the corresponding stage reward sequence, the corrected state transition distribution, and the candidate control action correction amount are written into the model trajectory library. The transient probabilistic dynamic model of the wind turbine is obtained by updating the Gaussian process dynamic parameters, stage kernel parameters, and strategy parameters based on the model trajectory library.
[0074] In this embodiment, the model trajectory library stores the disturbance number, stage label, candidate transient evolution trajectory, corrected transient evolution trajectory, state transition mean, state transition variance, stage reward sequence, candidate control action correction amount, stage kernel parameter version, and model parameter version.
[0075] For each set of corrected transient evolution trajectories generated by the reward feedback correction layer, the model trajectory library compares the reward value of the corrected transient evolution trajectory with that of the original candidate transient evolution trajectory. When the reward value of the corrected transient evolution trajectory is higher than that of the original candidate transient evolution trajectory, and the corrected transient evolution trajectory passes the state boundary check and stage consistency check, the corrected transient evolution trajectory is written into the model trajectory library. When the corrected transient evolution trajectory fails the check, the original candidate transient evolution trajectory is retained and the variable name and stage label of the failed check are recorded.
[0076] State boundary verification checks whether the speed, bus voltage, converter current, and pitch angle are within the allowable range; phase consistency verification checks whether the time sequence of the disturbance triggering phase, rapid response phase, oscillation decay phase, and recovery stabilization phase is consistent.
[0077] refer to Figure 1 and Figure 3 In this embodiment, S6 specifically includes: Obtain the initial state quantity, disturbance input quantity, and control action sequence from the transient data to be modeled, and perform time alignment on the initial state quantity, disturbance input quantity, and control action sequence to obtain the input sample to be modeled; Input the input samples to be modeled into the transient probabilistic dynamics model of the wind turbine, call the Gaussian process dynamics branch and probabilistic rolling prediction process in the improved PILCO probabilistic dynamics reinforcement learning transient modeling model, and generate the distribution sequence of the state to be modeled. Extract the speed probability curve, active power recovery probability curve, reactive power support probability curve, DC bus voltage fluctuation probability curve, converter current impact probability curve and pitch angle response probability curve from the state distribution sequence to be modeled to obtain the transient probability characteristic curve cluster of wind turbine. The transient recovery time and transient risk score are calculated based on the distribution sequence of the state to be modeled. The transient probability characteristic curve cluster of the wind turbine, the transient recovery time, the transient risk score, the disturbance type label and the stage label are bound together, and the transient characteristic record is output.
[0078] In this embodiment, the input samples to be modeled include initial state quantities, disturbance input quantities, and control action sequences. Initial state quantities include speed, power, voltage, current, pitch angle, and torque states before or at the initial stage of the disturbance. Disturbance inputs include voltage sag, wind speed change rate, power command change rate, and current limiting trigger flag; The control action sequence includes the pitch angle adjustment sequence, the torque control sequence, and the converter reactive power regulation sequence.
[0079] After the input samples to be modeled are entered into the transient probabilistic dynamics model of the wind turbine, the model selects the disturbance kernel parameter group according to the disturbance type label, selects the stage kernel parameter according to the stage label, and calls the Gaussian process dynamics branch and the probabilistic rolling prediction layer to generate the state distribution sequence to be modeled.
[0080] In this embodiment, the transient probability characteristic curve cluster is output in units of disturbance events, with each disturbance event corresponding to a set of multivariate curves. The speed probability curve records the mean change and uncertainty boundary of the generator speed during the transient process; The active power recovery probability curve records the process of active power recovering from the disturbance deviation state to the target range; the reactive power support probability curve records the reactive power regulation change process during the disturbance; the DC bus voltage fluctuation probability curve records the fluctuation range of DC bus voltage during the disturbance impact and recovery process; the converter current impact probability curve records the converter current peak value and recovery trend; and the pitch angle response probability curve records the pitch action amplitude and recovery trend. Each probability curve is bound to the disturbance number, stage label, state distribution sequence, and model parameter version.
[0081] In this embodiment, the transient characteristic record includes the disturbance start time, disturbance type label, stage label, maximum speed deviation, maximum DC bus voltage deviation, maximum converter current peak value, active power recovery time, reactive power support duration, maximum pitch angle movement amplitude, transient recovery time, transient risk score, curve index, and model parameter version.
[0082] The transient risk score is generated based on a comprehensive analysis of the number of speed overruns, the number of DC bus voltage overruns, the peak value of converter current surges, the active power recovery deviation, and the reactive power support deviation. Transient characteristic records are stored in the model results library and associated with disturbance numbers, curve indices, and model parameter versions.
[0083] In this embodiment, the transient probabilistic dynamics model of the wind turbine can continuously receive new on-site transient samples during operation.
[0084] After a new sample enters the model, transient state transition samples are generated first according to S1, and then disturbance type labels and stage labels are generated according to S2. When a new sample belongs to the same disturbance type and the same stage as an existing sample in the model trajectory library, the actual state transition result of the new sample is compared with the predicted state transition result of the model to generate the on-site state transition error. The training weights of the corresponding simulation-generated samples are adjusted based on the on-site state transition error, and the stage kernel parameters are updated.
[0085] Through the above-described virtual-real joint update process, on-site sampled data and simulation-generated samples jointly participate in model parameter correction under the same disturbance type and at the same stage.
[0086] In this embodiment, the data transfer relationship between the components of the wind turbine transient probabilistic dynamics model is as follows: the transient state transition sample is obtained by time alignment and perturbation window interception of the multi-source transient sampling sequence; Transient state transition samples are subjected to perturbation type identification and stage division to obtain staged transient training data; The staged transient training data is fed into the transient state encoding branch, the perturbation embedding branch, and the action embedding branch to obtain dynamic input features; The dynamic input characteristics enter the stage kernel parameter selection layer and Gaussian process dynamics branch to obtain the transient state transition distribution; The transient state transition distribution enters the probabilistic rolling prediction layer to obtain candidate transient evolution trajectories; The candidate transient evolution trajectory enters the transient constraint reward layer and obtains the transient constraint reward value; The transient constraint reward value enters the reward feedback correction layer to obtain the corrected transient evolution trajectory and the correction amount of the candidate control action; The corrected transient evolution trajectory is entered into the model trajectory library, and the model trajectory library updates the Gaussian process dynamic parameters, stage kernel parameters, and strategy parameters to form a transient probabilistic dynamic model of the wind turbine. After the transient data to be modeled is input into the transient probabilistic dynamics model of the wind turbine, the transient probabilistic characteristic curve cluster and transient characteristic record are output.
[0087] Compared with the modeling method that only uses steady-state power curves for fitting, this implementation decomposes the transient process of wind turbine into state transition samples, disturbance type labels, stage labels, state transition distribution, candidate transient evolution trajectories, transient constraint reward values, and transient characteristic records, forming a complete data processing link; Compared with neural network modeling methods that only use a single prediction error for training, this implementation introduces state variance, uncertainty boundary, stage kernel parameter and transient constraint reward value in the model training and updating process; Compared with conventional reinforcement learning control methods, this implementation method corrects the probabilistic dynamics model through reward feedback, rather than directly replacing the original controller output control commands of the wind turbine. The candidate control action correction amount participates in the model parameter update as a strategy correction variable in the modeling process.
[0088] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited to the above embodiments. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept, should be covered within the scope of protection of the present invention.
[0089] Example 1: To verify the feasibility of this invention in practice, it was applied to a transient characteristic modeling scenario of a doubly-fed induction generator (DFIG) wind turbine in a wind farm. Thirty DFIG wind turbines with a rated power of 2.5MW were selected as the modeling objects. Operating data such as wind speed, generator speed, pitch angle, electromagnetic torque, active power, reactive power, converter current, DC bus voltage, and grid voltage were collected at a sampling interval of 0.1s. The modeling data covered five types of transient processes: low-voltage ride-through, wind speed step, power command switching, pitch delay, and converter current limiting. A total of 860 transient disturbance windows were extracted, of which 620 were used as training samples, 120 as parameter correction samples, and 120 as validation samples.
[0090] In this embodiment, the system first performs time alignment and disturbance window truncation on the multi-source transient sampling sequence. The grid voltage drop point, wind speed step point, power command switching point, pitch control start point, and converter current limiting trigger point are used as disturbance trigger records. Continuous time windows are truncated before and after each trigger record to form disturbance window data containing a pre-disturbance stable segment, a disturbance response segment, and a recovery segment. Subsequently, the current state quantity, disturbance input quantity, control action quantity, and next-moment state quantity are extracted from the disturbance window data, and a correspondence is established according to the unit number, disturbance number, and sampling sequence number to generate transient state transition samples.
[0091] During the phase division process, this invention determines the disturbance type label based on the voltage drop amplitude, wind speed change rate, power command change rate, and current limiting flag. Furthermore, based on the generator speed change rate, active power recovery slope, DC bus voltage fluctuation rate, and converter current peak change, the transient process is divided into a disturbance triggering phase, a rapid response phase, an oscillation decay phase, and a recovery and stabilization phase. After phased processing, there are 246 low-voltage ride-through samples, 214 wind speed step samples, 156 power command switching samples, 118 pitch action delay samples, and 126 converter current limiting samples. Each sample includes state variables, disturbance variables, control action variables, and the state variable for the next moment.
[0092] The improved PILCO probabilistic dynamics reinforcement learning transient modeling model uses staged transient training data as input. The transient state encoding branch maps speed, power, voltage, current, pitch angle, and torque into continuous state features; the disturbance embedding branch embeds voltage drop amplitude, wind speed change rate, and power command change rate; and the action embedding branch embeds pitch angle adjustment, torque control, and converter reactive power regulation. These features, along with disturbance type labels and stage labels, are input into the Gaussian process dynamics branch to form low-voltage ride-through kernel parameter sets, wind speed step kernel parameter sets, power command switching kernel parameter sets, and converter current limiting kernel parameter sets, respectively. The outputs are the state transition mean and state transition variance.
[0093] In the probabilistic rolling prediction stage, this invention inputs the initial transient state and control action sequence into the Gaussian process dynamics branch, generates the next state distribution time-by-time, and writes the latest state distribution into the trajectory cache as the input for the next round of prediction, ultimately obtaining a multivariate transient trajectory of speed, active power, reactive power, DC bus voltage, converter current, and pitch angle. Simultaneously, uncertainty boundaries are configured for each variable trajectory based on the state transition variance, ensuring that the modeling results include not only the central response curve but also the probabilistic fluctuation range of the transient response.
[0094] In the reward feedback correction stage, this invention compares the candidate transient evolution trajectory with the operating constraint thresholds time-by-time, calculating deviations such as speed exceeding limits, DC bus voltage fluctuations, converter current surges, active power recovery, reactive power support, pitch angle sudden change, and transient recovery time, and forming transient constraint reward values. The model updates the Gaussian process dynamics parameters, stage kernel parameters, and strategy parameters based on these transient constraint reward values, calculates the correction amount for candidate control actions, and then generates corrected transient evolution trajectories through probabilistic rolling prediction.
[0095] To compare and verify the effectiveness, a transient modeling method based on Long Short-Term Memory (LSTM) networks was selected as the comparison method. The comparison method only uses historical operating sequences to fit multivariate response curves, without setting kernel parameters for the disturbance phase or transient constraint reward feedback. The method of this invention outputs a cluster of transient probabilistic characteristic curves for wind turbines on 120 sets of verification samples, including probability curves for rotational speed, active power recovery, reactive power support, DC bus voltage fluctuation, converter current surge, and pitch angle response, while also outputting transient recovery time and transient risk scores. Verification results show that the modeling errors of this invention for transient recovery time, peak bus voltage fluctuation, and peak converter current surge are all lower than those of the comparison method, with higher uncertainty coverage and transient risk identification accuracy.
[0096] Table 1 Comparison of Transient Characteristic Modeling Methods for Wind Turbine Units
[0097] As shown in Table 1, under the same 120 sets of verification transient windows, the average error of the multivariate transient trajectory of the comparative method is 6.8%, while that of the method of this invention is reduced to 3.1%. This change mainly stems from the fact that this invention does not directly fit the transient response as an ordinary time series, but instead constructs a state transition relationship between the disturbance input, control action, and the state at the next moment. In the improved PILCO probabilistic dynamic reinforcement learning transient modeling model, the mean and variance of the state are output through the Gaussian process dynamics branch, enabling the model to simultaneously learn the joint changes between speed, power, voltage, current, pitch angle, and torque.
[0098] In terms of active power recovery time error, DC bus voltage peak error, and converter current surge identification accuracy, the method of this invention achieves 0.27s, 3.4%, and 94.1%, respectively, which are superior to the comparative methods' 0.64s, 7.5%, and 82.6%. This is because the present invention establishes stage kernel parameters according to the disturbance triggering stage, rapid response stage, oscillation decay stage, and recovery stabilization stage, and incorporates speed exceeding limits, bus voltage fluctuations, converter current surges, active power recovery deviations, and reactive power support deviations into the transient constraint reward value. The model can retain the influence of transient safety constraints on trajectory correction during the update process.
[0099] The coverage of probabilistic uncertainty increased from 78.3% to 91.7%, indicating that the transient probability characteristic curve cluster output by this invention can more fully cover the actual response range. The error reduction rate after model update increased from 12.4% to 31.6%, indicating that the reward feedback correction mechanism can continuously adjust the dynamic parameters, stage kernel parameters, and strategy parameters of the Gaussian process. The modeling time for a single disturbance increased from 2.8s to 3.4s, mainly because this invention adds probabilistic rolling prediction and reward feedback correction calculations, but the increase in time is small, resulting in lower modeling errors and a more complete expression of transient risks, making it suitable for transient analysis, disturbance response assessment, and operational status modeling of wind turbines.
[0100] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A method for modeling the transient characteristics of wind turbine generators based on reinforcement learning, characterized in that, Includes the following steps: S1. Obtain the operation sampling data of the wind turbine during the transient modeling period, perform time alignment, disturbance window truncation and state pairing on the operation sampling data, and generate transient state transition samples. S2. Divide the transient state transition samples into stages according to the type of disturbance, the time of occurrence of the control action and the time of state recovery to obtain staged transient training data. S3. Input the staged transient training data into the improved PILCO probabilistic dynamic reinforcement learning transient modeling model, and obtain the transient state transition distribution through transient state encoding, control action embedding, disturbance embedding and Gaussian process dynamics calculation. S4. Based on the transient state transition distribution, perform probabilistic rolling prediction by continuously inputting the state distribution and control action sequence of the previous prediction time to obtain candidate transient evolution trajectories. S5. Based on the transient constraint reward value, the candidate transient evolution trajectory is corrected by reward feedback, and the Gaussian process dynamic parameters, stage kernel parameters and strategy parameters are updated to obtain the transient probabilistic dynamic model of the wind turbine. S6. Input the transient data to be modeled into the transient probabilistic dynamics model of the wind turbine, and output the transient probabilistic characteristic curve cluster and transient characteristic record of the wind turbine.
2. The method for modeling the transient characteristics of wind turbine units based on reinforcement learning according to claim 1, characterized in that, S1 includes the following steps: S11. Obtain the wind speed sequence, generator speed sequence, pitch angle sequence, electromagnetic torque sequence, active power sequence, reactive power sequence, converter current sequence, DC bus voltage sequence and grid voltage sequence of the wind turbine, and unify the timestamps and unit numbers of each sequence to obtain the multi-source transient sampling sequence. S12. Perform sampling frequency unification, time alignment and missing segment marking on the multi-source transient sampling sequence, and write the wind speed, rotational speed, power, voltage, current, pitch angle and torque corresponding to the same sampling time into the same sampling record to obtain time-aligned transient data; S13. Based on the grid voltage drop point, wind speed step point, power command switching point, pitch action start point and converter current limiting trigger point in the time-aligned transient data, extract the continuous time window before and after the time window to obtain the disturbance window data containing the state before disturbance, the state of disturbance response and the state of recovery. S14. Extract the current state quantity, disturbance input quantity, control action quantity and next moment state quantity from the disturbance window data, and establish a corresponding relationship according to the unit number, disturbance number and sampling sequence number to generate transient state transition samples.
3. The method for modeling the transient characteristics of wind turbine units based on reinforcement learning according to claim 1, characterized in that, S2 includes the following steps: S21. Read the disturbance input and control action quantities in the transient state transition sample, and identify low voltage ride-through disturbance, wind speed step disturbance, power command switching disturbance and converter current limiting disturbance based on voltage drop amplitude, wind speed change rate, power command change rate and current limiting mark, and obtain disturbance type label. S22. Read the generator speed change rate, active power recovery slope, DC bus voltage fluctuation rate and converter current peak change in the transient state transition sample, divide the transient state transition sample into disturbance triggering stage, fast response stage, oscillation decay stage and recovery stabilization stage, and obtain stage labels; S23. Associate the disturbance type label, stage label and the state transition relationship in the transient state transition sample to form a labeled state transition sample; S24. Group the labeled state transition samples according to the perturbation type label and the stage label, and record the start time, end time, sampling length and variable missing label of each group of samples to obtain staged transient training data.
4. The method for modeling the transient characteristics of wind turbine units based on reinforcement learning according to claim 1, characterized in that, S3 includes the following steps: S31. Read the pre-disturbance state quantity, disturbance input quantity, control action quantity, post-disturbance state quantity, disturbance type label and stage label from the staged transient training data, and arrange the speed, power, voltage, current, pitch angle and torque according to the sampling time to generate a transient state vector. S32. Input the voltage drop amplitude, wind speed change rate and power command change rate into the disturbance embedding branch to generate the disturbance embedding vector. Input the pitch angle adjustment amount, torque control amount and converter reactive power regulation amount into the action embedding branch to generate the action embedding vector. S33. Input the transient state vector, perturbation embedding vector, action embedding vector, perturbation type label and stage label into the improved PILCO probabilistic dynamic reinforcement learning transient modeling model to obtain dynamic input features; S34. In the improved PILCO probabilistic dynamic reinforcement learning transient modeling model, the Gaussian process dynamics branch is called, and the kernel similarity matrix is calculated based on the dynamic input features to obtain the state transition kernel matrix corresponding to the perturbation stage. S35. Calculate the state transition mean based on the state transition kernel matrix and the state variables after disturbance, and calculate the state transition variance based on the sampling residual and the state transition kernel matrix to obtain the state mean and state variance. S36. Write the state mean, state variance, disturbance type label, stage label, stage kernel parameter and sample index into the same distribution record to obtain the transient state transition distribution.
5. The method for modeling the transient characteristics of wind turbine units based on reinforcement learning according to claim 4, characterized in that, S34 includes the following steps: S341. In the improved PILCO probabilistic dynamic reinforcement learning transient modeling model, a transient state encoding branch is established, which maps the speed, power, voltage, current, pitch angle and torque into continuous state features respectively, and concatenates the continuous state features with the stage label to obtain the stage state features. S342. Combine the stage state features, disturbance embedding vector and action embedding vector into dynamic input features, and select the low voltage ride-through kernel parameter group, wind speed step kernel parameter group, power command switching kernel parameter group or converter current limiting kernel parameter group according to the disturbance type label. S343. Input the dynamic input features into the corresponding kernel parameter group, calculate the kernel similarity between different sampling points, and write the kernel similarity into the kernel matrix position according to the stage label to obtain the stage kernel similarity matrix. S344. Based on the stage kernel similarity matrix, establish a probabilistic dynamic mapping from the current state to the next time step state, and bind the probabilistic dynamic mapping with the sample index of the training samples to obtain the initial dynamic parameters of the improved PILCO probabilistic dynamic reinforcement learning transient modeling model.
6. The method for modeling the transient characteristics of wind turbine units based on reinforcement learning according to claim 1, characterized in that, S4 includes the following steps: S41. Read the state transition mean, state transition variance and stage kernel parameters in the transient state transition distribution, and use the wind turbine transient state at the start time as the initial input for rolling prediction to obtain the initial state distribution. S42. Input the initial state distribution and the first action quantity in the control action sequence into the Gaussian process dynamics branch, calculate the state mean and state variance at the next prediction time, and obtain the state distribution at the next prediction time. S43. Write the state distribution of the next prediction time into the trajectory buffer, and use the latest state distribution in the trajectory buffer as the next round of rolling input. Repeat the state distribution prediction according to the control action sequence to obtain the state distribution sequence of continuous prediction times. S44. Extract the speed trajectory, active power trajectory, reactive power trajectory, DC bus voltage trajectory, converter current trajectory, and pitch angle trajectory from the state distribution sequence to generate a multivariable transient trajectory. S45. Configure uncertainty boundaries for multivariate transient trajectories based on state transition variance, and combine multivariate transient trajectories with corresponding uncertainty boundaries to obtain candidate transient evolution trajectories.
7. The method for modeling the transient characteristics of wind turbine units based on reinforcement learning according to claim 1, characterized in that, S5 includes the following steps: S51. Read the speed trajectory, active power trajectory, reactive power trajectory, DC bus voltage trajectory, converter current trajectory and pitch angle trajectory in the candidate transient evolution trajectory, and compare each trajectory with the corresponding operating constraint threshold time by time to obtain the trajectory constraint deviation. S52. Calculate the speed over-limit deviation, DC bus voltage fluctuation deviation, converter current impact deviation, active power recovery deviation, reactive power support deviation, pitch angle sudden change deviation, and transient recovery time deviation based on the trajectory constraint deviation to obtain the constraint deviation set. S53. Accumulate the constraint deviation set in segments according to the disturbance triggering stage, rapid response stage, oscillation decay stage and recovery stabilization stage to generate a stage constraint deviation sequence. S54. Calculate the transient constraint reward value based on the stage constraint deviation sequence, and establish a correlation between the transient constraint reward value and the candidate transient evolution trajectory, disturbance type label and stage label to obtain reward-correlation trajectory data; S55. Feedback the reward-related trajectory data to the improved PILCO probabilistic dynamic reinforcement learning transient model, perform reward feedback correction, and obtain the corrected model data; S56. Perform state boundary verification and phase consistency verification on the modified model data, and write the modified model data that meets the verification conditions into the model trajectory library to obtain the transient probabilistic dynamic model of the wind turbine.
8. The method for modeling the transient characteristics of wind turbine generators based on reinforcement learning according to claim 7, characterized in that, S55 includes the following steps: S551. Read the transient constraint reward value and candidate transient evolution trajectory from the reward-related trajectory data, calculate the strategy update direction based on the transient constraint reward value, and map the strategy update direction to the pitch angle adjustment, torque control and converter reactive power regulation to obtain the candidate control action correction amount. S552. Recombine the candidate control action correction, transient state vector, disturbance embedding vector and stage label to generate corrected dynamic input features, and input the corrected dynamic input features into the Gaussian process dynamics branch to recalculate the corrected state transition mean and corrected state transition variance. S553. Generate a modified state transition distribution based on the modified state transition mean and the modified state transition variance, and input the modified state transition distribution into the probability rolling prediction process to generate a modified transient evolution trajectory. S554. Compare the reward values of the corrected transient evolution trajectory and the candidate transient evolution trajectory, and write the trajectory with the higher reward value, the corresponding stage reward sequence, the corrected state transition distribution, and the candidate control action correction amount into the model trajectory library. S555. Update the Gaussian process dynamic parameters, stage kernel parameters, and strategy parameters according to the model trajectory library to obtain the transient probabilistic dynamic model of the wind turbine.
9. The method for modeling the transient characteristics of wind turbine generators based on reinforcement learning according to claim 1, characterized in that, S6 includes the following steps: S61. Obtain the initial state quantity, disturbance input quantity, and control action sequence from the transient data to be modeled, and perform time alignment on the initial state quantity, disturbance input quantity, and control action sequence to obtain the input sample to be modeled. S62. Input the input sample to be modeled into the transient probabilistic dynamics model of the wind turbine, call the Gaussian process dynamics branch and probabilistic rolling prediction process in the improved PILCO probabilistic dynamics reinforcement learning transient modeling model, and generate the distribution sequence of the state to be modeled. S63. Extract the speed probability curve, active power recovery probability curve, reactive power support probability curve, DC bus voltage fluctuation probability curve, converter current impact probability curve and pitch angle response probability curve from the state distribution sequence to be modeled, and obtain the wind turbine transient probability characteristic curve cluster. S64. Calculate the transient recovery time and transient risk score based on the distribution sequence of the state to be modeled, bind the transient probability characteristic curve cluster of the wind turbine, transient recovery time, transient risk score, disturbance type label and stage label, and output the transient characteristic record.
