A self-optimization method for yaw-pitch composite control parameters based on reinforcement learning

Abstract

The present invention discloses a self-optimization method for yaw-pitch composite control parameters based on reinforcement learning, which aims to optimize the control strategy of a wind turbine generator set; by constructing a causal graph and quantifying the causal strength in the causal relationship, physical explainability is provided for the control strategy; according to the designed causal intervention reward mechanism, control decisions are optimized through rewards based on reinforcement learning, and the decision rules are reversely traced in combination with the causal graph, thereby achieving self-optimization of control parameters; the method of the present invention can effectively improve the dynamic response performance of the wind turbine generator set, solve the local optimal problem existing in traditional control strategies, and enhance the adaptability and robustness of the system.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent control of wind turbine generator sets, and in particular to a self-optimization method for yaw-pitch composite control parameters based on reinforcement learning. Background Art

[0002] In recent years, the optimization of wind turbine control strategies has gradually evolved from traditional PID control and model predictive control to data-driven intelligent algorithms. With breakthroughs in machine learning technology, control parameter adaptation methods based on deep reinforcement learning (DRL) have shown potential in optimizing the dynamic response of wind turbines. Some current studies have shown that the physical interpretability of control strategies can be improved by introducing causal reasoning models, such as structural equation modeling (SEM), to analyze the influence paths between variables. However, existing technologies still have the following shortcomings: First, traditional reinforcement learning often relies on empirical settings or single-objective optimization in reward function design, lacking dynamic modeling of causal relationships between multiple variables, making policy updates prone to falling into local optimality. For example, during yaw angle adjustment, existing methods fail to effectively distinguish the differential impact paths of wind speed changes and mechanical inertia on the tip speed ratio, making it difficult to accurately assess the causal effects of action decisions. Secondly, the collaborative optimization of composite control parameters faces the problem of dimensionality curse. Although the existing DRL algorithm based on the black box model can handle high-dimensional inputs, its decision-making process lacks transparency and cannot reversely track the contribution of key variables to the control target, resulting in parameter adjustment lagging behind operating condition changes. In addition, since the physical constraints implicit in the historical operating data (the nonlinear coupling of the torque equation and the power coefficient) are not fully encoded into the learning framework, the algorithm's generalization ability under extreme working conditions is limited. For example, when the pitch angle and yaw angle are adjusted at the same time, the existing method finds it difficult to analyze the competitive influence mechanism of the two on the output power, resulting in control command conflicts. These problems seriously affect the operating efficiency of wind turbines in complex environments. Summary of the Invention

[0003] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of this application to avoid obscuring the purpose of this section, the abstract and the title of the invention, and such simplifications or omissions should not be used to limit the scope of the present invention.

[0004] In view of the above existing problems, the present invention is proposed. Therefore, the present invention provides a method for self-optimization of yaw-pitch composite control parameters based on reinforcement learning to solve the problems raised in the background art.

[0005] To solve the above technical problems, the present invention provides the following technical solution: a method for self-optimization of yaw-pitch composite control parameters based on reinforcement learning, comprising:

[0006] Acquiring historical operating data of the wind turbine generator set, constructing a causal relationship based on the historical operating data, and quantifying the causal strength in the causal relationship based on a constrained causal discovery algorithm to generate a causal graph with strong and weak labels;

[0007] Obtain real-time operating data of wind turbines, design a causal intervention reward mechanism based on reinforcement learning methods, and convert the causal intervention reward mechanism into an interpretable decision rule;

[0008] In combination with the cause-effect diagram, the decision rule is traced backwards, and the cause-effect diagram is updated with the traced result of the decision rule, thereby realizing automatic optimization of the yaw-pitch composite control parameters of the wind turbine generator set.

[0009] As a preferred solution of the method for self-optimization of yaw-pitch composite control parameters based on reinforcement learning described in the present invention, obtaining historical operating data of the wind turbine generator set includes:

[0010] Ambient wind speed, rotation speed of the wind turbine rotor, angle between the wind turbine nacelle and wind direction, angle between the blades and the rotation plane, and electrical power output by the wind turbine;

[0011] As well as, the torque equation and tip speed ratio.

[0012] As a preferred solution of the method for self-optimization of yaw-pitch composite control parameters based on reinforcement learning described in the present invention, the method includes: obtaining historical operating data of a wind turbine generator set, and establishing a causal relationship based on the historical operating data, including:

[0013] Based on the physical constraints of the wind turbine generator set itself, the causal relationships among wind speed → tip speed ratio, speed → tip speed ratio, speed → power, yaw angle → torque, pitch angle → power coefficient, power coefficient → torque, tip speed ratio → power coefficient, and torque → power are established.

[0014] Among them, the left side of the arrow is the influencing factor, and the right side of the arrow is the affected factor.

[0015] As a preferred solution of the method for self-optimization of yaw-pitch composite control parameters based on reinforcement learning described in the present invention, the method comprises: quantifying the causal strength in the causal relationship based on a constrained causal discovery algorithm, and generating a causal graph with strong and weak labels, including:

[0016] Based on the causal relationship, a fully connected directed graph is constructed. For each edge in the directed graph, its partial correlation coefficient is calculated, and the partial correlation coefficient is used as a measure of the causal strength in the causal relationship to generate a causal graph with strong and weak marks.

[0017] As a preferred solution of the method for self-optimization of yaw-pitch composite control parameters based on reinforcement learning described in the present invention, real-time operation data of the wind turbine generator set is obtained, and a causal intervention reward mechanism is designed based on the reinforcement learning method, including:

[0018] The real-time operating data of wind turbines are used as the environmental state input of reinforcement learning to construct a dynamic causal reward function and delayed causal effect compensation.

[0019] As a preferred solution of the method for self-optimization of yaw-pitch composite control parameters based on reinforcement learning described in the present invention, the construction of a dynamic causal reward function and delayed causal effect compensation includes:

[0020] Calculate the deviation between the real-time operation data and the historical operation data, extract the direction of change of the data action predicted by the causal graph with strong and weak marks, and calculate the reward value;

[0021] A sliding window is set. If the direction of change of the data action predicted by the causal graph shows an upward trend, the reward value is increased. If the direction of change of the data action predicted by the causal graph does not change or shows a downward trend, the reward value is not changed.

[0022] As a preferred solution of the reinforcement learning-based self-optimization method for yaw-pitch composite control parameters of the present invention, the causal intervention reward mechanism is converted into an interpretable decision rule, including:

[0023] According to the causal diagram with strong and weak marks, the importance of each variable in the causal diagram is calculated, and the variables are sorted according to their importance. The decision tree is constructed based on the importance of the sorted variables.

[0024] As a preferred solution of the method for self-optimization of yaw-pitch composite control parameters based on reinforcement learning according to the present invention, wherein: in combination with the causal graph, the decision rule is reversely traced, and the causal graph is updated based on the tracking result of the decision rule, including:

[0025] A decision path is formed along the root node to the leaf node of the decision tree. If the order of the root node and the leaf node in the decision path is different from that in the originally constructed decision tree, the importance of each variable is reordered according to the decision path. While reconstructing the decision tree, the causal diagram with strong and weak marks is updated in reverse.

[0026] Compared with the prior art, the invention has the following beneficial effects:

[0027] 1. This method quantitatively analyzes the causal relationship between wind turbine operating parameters and incorporates the physical constraints of the wind turbine itself to generate a causal diagram with strong and weak labels. This overcomes the limitations of fuzzy parameter relationships in traditional control methods and provides a scientific basis for control decision-making.

[0028] 2. The causal intervention reward mechanism is converted into an interpretable decision rule, and a decision tree is constructed based on the importance ranking of variables in the causal graph. This solves the "black box" problem of traditional reinforcement learning, makes the generation process of reinforcement learning control strategies have clear physical meaning and logical basis, and provides a transparent approach for engineering implementation and fault diagnosis.

[0029] 3. Through the reverse tracking mechanism, a decision path is formed according to the constructed decision tree, which realizes the dynamic update of the cause-and-effect diagram, effectively responds to the complex and changeable wind conditions, and makes the control parameter optimization process in the wind turbine generator set more intelligent in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort. Among them:

[0031] Figure 1 This is an overall flow chart of a method for self-optimization of yaw-pitch composite control parameters based on reinforcement learning according to an embodiment of the present invention;

[0032] Figure 2 This is a flowchart of constructing a decision tree for a method for self-optimization of yaw-pitch composite control parameters based on reinforcement learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the following detailed description of the specific embodiments of the present invention is given in conjunction with the accompanying drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in this field without creative work should fall within the scope of protection of the present invention.

[0034] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0035] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0036] The present invention is described in detail with reference to schematic diagrams. For ease of illustration, cross-sectional views of device structures may be partially enlarged and not to scale when describing embodiments of the present invention. Furthermore, the schematic diagrams are merely illustrative and should not limit the scope of the present invention. Furthermore, in actual production, the three-dimensional dimensions of length, width, and depth should be included.

[0037] In the description of the present invention, it should be noted that the terms "upper, lower, inner, and outer" and other references to orientations or positional relationships are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first, second, or third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0038] In this disclosure, unless otherwise specified or limited, the terms "mounted," "connected," and "connected" should be interpreted broadly. For example, they may refer to fixed, removable, or integral connections. They may also refer to mechanical, electrical, or direct connections, indirect connections through an intermediary, or internal communication between two components. Those skilled in the art will understand the specific meanings of these terms in this disclosure.

[0039] Example 1

[0040] Reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, provides a method for self-optimization of yaw-pitch composite control parameters based on reinforcement learning, comprising:

[0041] S1. Obtain historical operating data of the wind turbine generator set, construct a causal relationship based on the historical operating data, and quantify the causal strength in the causal relationship based on a constrained causal discovery algorithm to generate a causal graph with strong and weak labels;

[0042] Specifically, historical operation data and real-time operation data of the wind turbine are obtained from the wind turbine SCADA system;

[0043] Specifically, the historical operation data and real-time operation data include ambient wind speed, rotation speed of the wind turbine rotor, angle between the wind turbine nacelle and wind direction, angle between the blades and the rotation plane, electric power output by the wind turbine, torque equation, and tip speed ratio;

[0044] It should be noted that both the historical and real-time operation data need to be preprocessed, that is, to remove missing values ​​and outliers in the data;

[0045] It should be explained that the torque equation refers to the torque that drives the generator to rotate, and the tip speed ratio refers to the ratio of the rotor tip linear velocity to the wind speed;

[0046] Furthermore, the torque equation can be expressed as:

[0047]

[0048] Where ρ is the air density (kg / m 3 ), R represents the blade radius (m), C p (λ,β) is expressed as the power coefficient, which is composed of the tip speed ratio λ and the pitch angle β, C p It represents the efficiency of converting wind energy into mechanical energy; τ represents the torque. The torque in the scheme of the present invention refers to the torque of the wind turbine generator set; v represents the wind speed (m / s);

[0049] Furthermore, the tip speed ratio is expressed as:

[0050]

[0051] Where ω is the rotational speed (rad / s);

[0052] Furthermore, the relationship between power, torque and speed is:

[0053] P=τω

[0054] Where P represents power;

[0055] Specifically, taking the physical constraints of the wind turbine itself as a condition, the following causal relationship is obtained:

[0056] Establish the causal relationships between wind speed → tip speed ratio, speed → tip speed ratio, speed → power, yaw angle → torque, pitch angle → power coefficient, power coefficient → torque, tip speed ratio → power coefficient, and torque → power;

[0057] Among them, the left side of the arrow is the influencing factor, and the right side of the arrow is the affected factor;

[0058] It should be noted that for wind speed → tip speed ratio, changes in wind speed will lead to an inverse change in the tip speed ratio (i.e., when wind speed increases, if the speed remains unchanged, the tip speed ratio decreases); for speed → tip speed ratio, the speed directly determines the size of the tip speed ratio and works together with the wind speed (i.e., an increase in speed will positively increase the tip speed ratio); for speed → power, the speed directly affects the power output, and even if the torque remains unchanged, changes in speed will linearly change the power; for yaw angle → torque, the yaw angle affects the effectiveness of the wind turbine in capturing wind energy (i.e., when the yaw angle increases, the effective wind speed decreases, and the torque decreases accordingly). ); For pitch angle → power coefficient, the pitch angle directly affects the efficiency of converting wind energy into mechanical energy by changing the angle of attack of the wind turbine blades (i.e., increasing the pitch angle generally reduces the power coefficient); for power coefficient → torque, the power coefficient directly determines the magnitude of the torque, and changes in the power coefficient will affect the torque in a proportional manner; for tip speed ratio → power coefficient, the tip speed ratio reflects the degree of matching between the rotor speed and the wind speed, and can directly affect the aerodynamic performance and wind energy capture efficiency of the wind turbine blades; for torque → power, changes in torque will affect the power output in a proportional manner. This relationship is the basic physical law of mechanical energy conversion;

[0059] Furthermore, based on the constrained causal discovery algorithm, it is assumed that there is a relationship between all variables in the causal relationship, that is, there is an edge between all variables in the initial causal relationship. According to the causal relationship, a fully connected directed graph is constructed;

[0060] It should be noted that the advantage of the constrained causal discovery (PC) algorithm is that it can automatically discover causal structures from data, but its disadvantage is that it relies solely on data. In addition, in the actual application of wind turbines, relying solely on data may result in causal relationships that do not conform to physical laws. For example, if there is a relationship between all variables, then it will show that there is a direct correlation between wind speed and torque. However, according to the physical constraints of the wind turbine, wind speed cannot directly affect torque, but indirectly affects it through the tip speed ratio and power coefficient. Therefore, incorporating the physical constraints of the above-mentioned wind turbine into the PC algorithm can well solve this problem. In addition, since the physical constraints of the wind turbine itself have been considered in the causal relationship before constructing the directed graph through the PC algorithm, for the traditional PC algorithm, there is no need to delete edges that do not meet the conditions through conditional independence testing, such as the influence between wind speed and torque, which further optimizes the PC algorithm.

[0061] Furthermore, for each edge in the directed graph, the partial correlation coefficient is calculated, and the partial correlation coefficient is used as a measure of the causal strength in the causal relationship to generate a causal graph with strong and weak labels;

[0062] Furthermore, consider the influencing factor as X and the affected factor as Y, define other parent nodes of X (i.e., variables that directly affect X and may indirectly affect Y through X), and define other parent nodes of Y (i.e., variables that directly affect Y and may act together with X on Y), except X; for example, there are two edges: wind speed → blade tip speed ratio and rotational speed → blade tip speed ratio. Then, when calculating the partial correlation coefficient of wind speed → blade tip speed ratio, the influence of rotational speed needs to be controlled;

[0063] Specifically, the partial correlation coefficient ρ XY The calculation formula is expressed as:

[0064]

[0065] Among them, Cov and Var represent covariance and variance respectively, R X It is expressed as the residual value of X after controlling other variables, R Y Expressed as the residual value of Y after controlling other variables;

[0066] Specifically, the range of the partial correlation coefficient is [-1, 1]. The absolute value of the partial correlation coefficient is taken to represent the strength of the causal strength. The larger the absolute value, the stronger the direct impact of X on Y.

[0067] Specifically, the causal strength of absolute value division is as follows:

[0068] 0.0~0.2: weak causal relationship;

[0069] 0.2-0.5: moderate causal relationship;

[0070] 0.5-1.0: strong causal relationship;

[0071] Specifically, if the value of the partial correlation coefficient is negative, it indicates a reverse causal relationship (X increases, Y decreases), and if the value of the partial correlation coefficient is positive, it indicates a forward causal relationship (X increases, Y increases);

[0072] Specifically, the absolute value of the partial correlation coefficient (causal strength) is used to mark each directed edge of the directed graph, and finally a causal graph with strong and weak labels is generated;

[0073] S2. Obtain real-time operating data of wind turbines, design a causal intervention reward mechanism based on reinforcement learning methods, and convert the causal intervention reward mechanism into an interpretable decision rule;

[0074] Furthermore, the real-time operating data of wind turbines is used as the environmental state input for reinforcement learning to construct a dynamic causal reward function and delayed causal effect compensation;

[0075] Furthermore, the dynamic causal reward function is constructed by calculating the deviation between the real-time operation data and the historical operation data, extracting the data action change direction predicted by the causal graph with strong and weak marks, and calculating the reward value;

[0076] Specifically, the calculation of the deviation ΔD between the real-time operating data and the historical operating data can be expressed by the following formula:

[0077] ΔD=D rt_d -D h_d

[0078] Among them, D rt_d It is represented as real-time operation data, which can be any data contained in real-time operation data. h_d Represented as historical operation data. Similarly, it can be data contained in any historical operation data.

[0079] It should be noted that the deviation between the real-time operating data and the historical operating data needs to be calculated using the same data, for example, the deviation between the real-time wind speed and the historical wind speed;

[0080] Specifically, the direction of data action change refers to the control measures taken according to the deviation value. For example, if increasing the pitch angle → power reduction (strong causal relationship 0.6), then by reducing the pitch angle, the power reduction state is alleviated;

[0081] For example, assuming a wind turbine has a blade radius of 50 m, a rotation speed of 2 (rad / s), a historical wind speed of 10 m / s, and a real-time wind speed of 12 m / s, the effect of wind speed deviation on the tip speed ratio is as follows:

[0082] The historical tip speed ratio is calculated using the above formula:

[0083]

[0084] Calculate the real-time tip speed ratio:

[0085] (Keep two decimal places)

[0086] Analysis shows that when the wind speed increases from a historical value of 10 m / s to a real-time value of 12 m / s, the tip speed ratio decreases from 10 to 8.33. The power coefficient of a wind turbine is typically within a specific tip speed ratio range. Assuming the historical tip speed ratio of 10 is the optimal value in the example above, the real-time tip speed ratio of 8.33 deviates from this optimal value, which will result in a decrease in the wind turbine's power coefficient and, consequently, lower actual power generation efficiency.

[0087] It should be noted that in order to prevent the competitive influence mechanism generated by the simultaneous adjustment of two variables, which leads to control instruction conflicts, it is necessary to balance them through reward values;

[0088] Specifically, the reward value calculation includes the base reward, which is the negative value of the specific deviation (e.g., -0.1 represents a 10% deviation); the causal compliance reward, which is a reward if the actual data change is consistent with the relationship in the causal diagram, otherwise the reward is deducted; and the total reward, which is the weighted sum of the base reward and the causal compliance reward.

[0089] Furthermore, the delayed causal effect compensation adopts a sliding window method. If the direction of change of the data action predicted by the causal graph shows an upward trend, the reward value is increased; if the direction of change of the data action predicted by the causal graph does not change or shows a downward trend, the reward value is not changed.

[0090] It should be noted that when the direction of change in the data action predicted by the causal diagram shows an upward trend, it indicates that the control measures taken for the deviation value are positive (i.e., efforts are being made to balance the parameters to the ideal state). Increasing the reward value at this time is equivalent to positive feedback in reinforcement learning, which helps the wind turbine maintain stability and power generation efficiency under complex conditions. In addition, by using a sliding window method (such as 5 minutes), the delayed effect after the predicted data action can be tracked and its continued impact on the system state can be evaluated. For example, wind speed may fluctuate violently within a few seconds, but the changing trend of the current predicted data action can be more accurately reflected within a 5-minute sliding window.

[0091] Furthermore, based on the causal diagram with strong and weak marks, the importance of each variable in the causal diagram is calculated, and the variables are sorted according to their importance. The decision tree is constructed based on the importance of the sorted variables.

[0092] Specifically, find the path from a certain variable to the target variable. For each path, calculate the product of all edge weights on the path, use the absolute value of the partial correlation coefficient (causal strength) as the edge weight, and add up the products of all paths to obtain the total causal strength from a certain variable to the target variable.

[0093] For example, suppose there are two paths in the causal graph with strong and weak labels: X1→X2→Y and X1→Y;

[0094] Then for path 1 (X1→X2→Y), which contains two edges, assuming that the edge weights are 0.8 for X1→X2 and 0.5 for X2→Y, the strength of path 1 is 0.8×0.5=0.4; for path 2 (X1→Y), which contains one edge, assuming that the edge weight is 0.6, the strength of path 2 is 0.6; adding the products of all paths 0.4+0.6=1, the total causal strength of X1→Y is 1, and the causal strength of a certain variable X1 on the target variable Y is obtained as its importance;

[0095] Specifically, each variable is ranked according to its importance. The higher the strength of the variable, the greater its influence on the target variable.

[0096] It should be noted that in standard decision tree algorithms (such as CART), when selecting split variables at each step, the information gain (or Gini index reduction) of all candidate variables is calculated, and the variable with the largest gain is selected for splitting. In order to give priority to variables with high importance, causal strength can be incorporated into the construction of the decision tree. Figure 2 , the steps to construct a decision tree are as follows:

[0097] S201, determining the target variable;

[0098] S202. In each node of the causal graph with strong and weak labels, calculate the information gain of each variable from the candidate variables;

[0099] S203, multiplying the information gain by the causal strength of the variable to obtain a weighted information gain;

[0100] S204, selecting the variable with the highest weighted information gain for segmentation;

[0101] S205, recursively repeat this process for the child nodes under each node until the decision tree reaches the maximum depth, that is, all nodes in the causal graph are assigned;

[0102] For example, assume that the causal graph contains three variables X1, X2, X3 and the target variable Y, and the causal strengths are: X1 is 1; X2 is 0.5; X3 is 0.7; at the root node of the decision tree, assume that the information gains are: X1 is 0.3; X2 is 0.4; X3 is 0.35;

[0103] The calculated weighted information gain is:

[0104] X1: 1×0.3=0.3;

[0105] X2: 0.5×0.4=0.2;

[0106] X3: 0.7 × 0.35 = 0.245;

[0107] Then select X1 as the root node segmentation variable because it has the highest weighted information gain;

[0108] S3. Reversely trace the decision rule in combination with the cause-effect diagram, and update the cause-effect diagram with the tracking result of the decision rule, thereby achieving automatic optimization of the yaw and pitch composite control parameters of the wind turbine generator set;

[0109] Furthermore, a decision path is formed along the root node to the leaf node of the decision tree. For example, in the yaw pitch control of a wind turbine, the decision path may be wind speed > 10 m / s → speed > 15 rad / s → power ≈ 800 kW.

[0110] It should be explained that a leaf node refers to a node without child nodes. In other words, a child node can be a leaf node;

[0111] Furthermore, if the order of the root node and the leaf node in the decision path is different from that in the originally constructed decision tree, the importance of each variable is reordered according to the decision path, and while reconstructing the decision tree, the causal diagram with the strong and weak marks is updated in reverse.

[0112] It should be noted that if the order of the root node and leaf nodes in the decision path is inconsistent with that in the originally constructed decision tree, it indicates that the operating environment of the wind turbine has changed and the original decision tree is no longer fully adapted to the current operating conditions. Therefore, it is necessary to reconsider the importance of each variable and reconstruct the original decision tree.

[0113] Specifically, reverse updating means that if the decision path reveals a new causal relationship or negates the original relationship, then the edge structure of the original causal graph needs to be adjusted (deleting edges, adjusting edges, adding edges).

[0114] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Furthermore, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present application may be implemented in various computer languages, for example, object-oriented programming language Java and interpreted scripting language JavaScript, etc.

[0115] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0116] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0117] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0118] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0119] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A self-optimization method for yaw-pitch composite control parameters based on reinforcement learning, characterized in that: include: Acquiring historical operating data of the wind turbine generator set, constructing a causal relationship based on the historical operating data, and quantifying the causal strength in the causal relationship based on a constrained causal discovery algorithm to generate a causal graph with strong and weak labels; Obtain real-time operating data of wind turbines, design a causal intervention reward mechanism based on reinforcement learning methods, and convert the causal intervention reward mechanism into an interpretable decision rule; The real-time operating data of the wind turbine generator set is obtained, and a causal intervention reward mechanism is designed based on the reinforcement learning method, including: The real-time operating data of wind turbines is used as the environment state input for reinforcement learning to construct a dynamic causal reward function and delayed causal effect compensation. The construction of a dynamic causal reward function and delayed causal effect compensation includes: Calculate the deviation between the real-time operation data and the historical operation data, extract the direction of change of the data action predicted by the causal graph with strong and weak marks, and calculate the reward value; Setting a sliding window, if the direction of change of the data action predicted by the causal graph shows an upward trend, then increasing the reward value, if the direction of change of the data action predicted by the causal graph does not change or shows a downward trend, then not changing the reward value; Convert the causal intervention reward mechanism into an interpretable decision rule, including: According to the causal diagram with strong and weak marks, calculate the importance of each variable in the causal diagram, and sort the variables according to their importance, and build a decision tree based on the importance of the sorted variables; In combination with the cause-effect diagram, the decision rule is traced backwards, and the cause-effect diagram is updated with the traced result of the decision rule, thereby realizing automatic optimization of the yaw-pitch composite control parameters of the wind turbine generator set.

2. The method for self-optimization of yaw-pitch composite control parameters based on reinforcement learning according to claim 1, characterized in that: Obtain historical operating data of wind turbines, including: Ambient wind speed, rotation speed of the wind turbine rotor, angle between the wind turbine nacelle and wind direction, angle between the blades and the rotation plane, and electrical power output by the wind turbine; As well as, the torque equation and tip speed ratio.

3. The method for self-optimization of yaw-pitch composite control parameters based on reinforcement learning according to claim 2, characterized in that: Acquiring historical operating data of the wind turbine generator set and establishing a causal relationship based on the historical operating data include: Based on the physical constraints of the wind turbine generator set itself, the causal relationships among wind speed → tip speed ratio, speed → tip speed ratio, speed → power, yaw angle → torque, pitch angle → power coefficient, power coefficient → torque, tip speed ratio → power coefficient, and torque → power are established. Among them, the left side of the arrow is the influencing factor, and the right side of the arrow is the affected factor.

4. The method for self-optimization of yaw-pitch composite control parameters based on reinforcement learning according to claim 3, characterized in that: Based on the constrained causal discovery algorithm, the causal strength in the causal relationship is quantified, and a causal graph with strength and weakness labels is generated, including: Based on the causal relationship, a fully connected directed graph is constructed. For each edge in the directed graph, its partial correlation coefficient is calculated, and the partial correlation coefficient is used as a measure of the causal strength in the causal relationship to generate a causal graph with strong and weak marks.

5. The method for self-optimization of yaw-pitch composite control parameters based on reinforcement learning according to claim 1, characterized in that: In combination with the causal diagram, the decision rules are reversely traced, and the causal diagram is updated based on the tracing results of the decision rules, including: forming a decision path along the root node to the leaf node of the constructed decision tree; if the order of the root node and the leaf node in the decision path is different from that in the originally constructed decision tree, the importance of each variable is reordered based on the decision path, and while reconstructing the decision tree, the causal diagram with strong and weak marks is reversely updated.

Images