Active load shedding optimization control method, system and device for wind power plant and storage medium

Abstract

The invention provides an active load shedding optimization control method, system and device of a wind power plant and a storage medium, and belongs to the technical field of active power control of the wind power plant, and the method comprises the steps: constructing and training an SAC intelligent agent through an SAC algorithm; acquiring a wind power plant state quantity; inputting the state quantity of the wind power plant into the SAC intelligent agent to obtain a vector # imgabs0 # composed of active power initial reference values of all wind turbine generators; inputting the # imgabs 1 # into a constraint controller for secondary calculation to obtain a vector # imgabs 2 # composed of active power optimization reference values of the wind turbine generators; the wind power plant executes # imgabs3 #; and repeating the steps, and adjusting the active power of each wind turbine generator in real time. The model is high in generalization, stability, reliability and learning efficiency, real-time online calculation of the equivalent fatigue load can be achieved, reasonable distribution of the active power of the wind power plant can be achieved under the condition that the active power constraint of the wind power plant is met, the equivalent fatigue load of the whole wind power plant can be reduced, and the maintenance cost can be reduced.

Description

Background Art

[0002] Wind power generation is an environmentally friendly renewable energy generation method. With the large-scale installation and use of wind turbines, and the increase in the operating capacity and scale of wind farms, the safety and reliability of wind farm operations have received widespread attention. Wind turbines are affected by time-varying wind fields. While generating energy, they are accompanied by structural fatigue. After long-term operation, they will cause fatigue failure of wind turbines, which in turn threatens the reliability of wind farms. In order to ensure the reliability of wind farms, it is necessary to frequently carry out maintenance operations on wind turbines in wind farms. Therefore, reducing fatigue damage to wind turbines and reducing fatigue loads on wind farms are crucial to reducing the operation and maintenance costs of wind farms.

[0003] In the research on reducing the fatigue load of wind farms, active power control of wind farms is the key research direction. At present, the control strategies that consider the fatigue load of wind farms include wind farm fatigue control strategies based on physical mechanisms. They construct fatigue models for each wind turbine in the wind farm through physical mechanisms, aiming to minimize the total fatigue load of the wind farm or balance the fatigue load distribution, and combine commercial solvers to solve the optimization strategy to achieve fatigue suppression. This method has good effects in specific application scenarios, but the mechanism model it constructs is difficult to cover all nonlinear dynamics of wind turbines, which will affect the optimization effect of model predictive control.

[0004] In order to improve the optimization effect of fatigue load of wind farm, the existing technology also proposes active optimization control strategy based on data-driven method, which has low requirements for wind turbine modeling, and even some can adopt model-free control algorithm. Among them, some studies build wind turbine fatigue damage prediction model based on RBF (Radial Basis Function) neural network or residual neural network, and combine genetic algorithm or particle swarm algorithm to solve control strategy. The model has good prediction performance, but it is difficult to solve due to the non-convex characteristics of the model, and it is often easy to fall into local optimum; some studies directly use reinforcement learning method for fatigue load suppression control, such as DDPG (Deep Deterministic Policy Gradient) and its derivative improved algorithm, which solves the problem of solving control strategy of strong nonlinear model of wind farm to a certain extent, but the convergence of intelligent agent is greatly affected by hyperparameters and the generalization is relatively weak. Summary of the invention

[0005] In view of the technical problems that in the existing data-driven wind farm fatigue load control strategy, some models exhibit non-convex characteristics, which makes it difficult to solve and easy to fall into local optimal situations; the convergence of the intelligent agent in the reinforcement learning method is greatly affected by hyperparameters and the generalization is weak, the invention provides an active load reduction optimization control method, system, equipment and storage medium for wind farms. The model of the invention has strong generalization, high stability, reliability and learning efficiency, can realize real-time online calculation of equivalent fatigue load, can realize reasonable allocation of active power of wind farms while satisfying the active power constraints of wind farms, can reduce the equivalent fatigue load of the entire wind farm, and reduce maintenance costs.

[0006] In a first aspect, the present invention provides a method for optimizing active load shedding control of a wind farm, the steps comprising: S1. Build and train a SAC agent. Build and train a SAC agent using the SAC algorithm. S2. Obtaining wind farm status; S3. Generate initial action, input the wind farm state quantity into the SAC agent, and obtain the vector composed of the initial reference value of the active power of each wind turbine in the wind farm, recorded as ; S4. Input constraint controller, constraint controller After a secondary calculation, we can obtain the vector composed of the active power optimization reference values ​​of each wind turbine in the wind farm, which is recorded as ; S5. Adjust the wind farm power, and the wind farm executes , adjust the active power of each wind turbine in the wind farm to same; S6. Repeat the cycle, repeat steps S2-S5, and adjust the active power of each wind turbine in the wind farm in real time.

[0007] It should be further explained that the wind farm state quantity includes the power command of the power grid, the current input wind speed of the wind turbine, the current output power of the wind turbine, the instantaneous change of the wind turbine main shaft torque and the instantaneous change of the tower thrust.

[0008] It should be further explained that in step S1, the steps of constructing and training the SAC agent include: S101: Initialize the neural network, build the policy network, state value network, target state value network, action-state value network and target action-state value network; The function representing the policy network is: ; In the formula, is the state quantity of the wind farm, which is the input value of the strategy network; It is a vector composed of the active power values ​​of each wind turbine in the wind farm, and is the output value of the strategy network; The function representing the state value network is: ; In the formula, is the state quantity of the wind farm, which is the input value of the state value network; The output value of the state value network is V value; The function representing the target state value network is: ; In the formula, is the state quantity of the wind farm, which is the input value of the target state value network; The output value of the target state value network is the target V value; The function representing the action-state value network is: ; Where s is the state quantity of the wind farm, is a vector composed of the active power values ​​of each wind turbine in the wind farm, and They are all input values ​​of the action-state value network; The output value of the action-state value network is the Q value; The function representing the target action-state value network is: ; In the formula, is the state quantity of the wind farm, is a vector composed of the active power values ​​of each wind turbine in the wind farm, and Both are input values ​​of the target action-state value network; The output value of the target action-state value network is the target Q value; The state-value network of the initial state is the same as the target state-value network, and the action-state-value network is the same as the target action-state-value network; S102: Experience replay buffer setting and hyperparameter setting; S103: Collect experience and obtain the wind farm state quantity at time t0, recorded as ,Will Input the strategy network to obtain the vector composed of the initial reference values ​​of active power training of each wind turbine in the wind farm, which is recorded as ; Will Input the constraint controller, and after the secondary calculation of the constraint controller, the active power training optimization reference value of each wind turbine in the wind farm is obtained, which is recorded as ; Wind Farm Execution , adjust the active power of each wind turbine in the wind farm to same, The execution ends at time t1; The wind farm state quantity, load data and load data within the specified time window before time t1 are obtained. The wind farm state quantity of the wind farm at time t1 is recorded as , then input the load data at time t1 and the load data within the specified time window before time t1 into the reward function to obtain The reward value of Will , , the load data and reward value at time t1 form an experience tuple, and the experience tuple is stored in the experience playback buffer; S104: training the network, randomly extracting a batch of experience samples from the experience playback buffer; Using the extracted experience samples, the target Q value is calculated using the target action-state value network, and the target V value is calculated using the target state value network; The state value network is updated according to the calculated target Q value, action entropy and the output of the current state value network. The loss function formula of the state value network is:

[0009] In the formula, is the weight of the state value network; is the weight of the target action-value network; is the weight of the policy network, is the state quantity of the wind farm at time t0; Represents the experience replay buffer; is the function representing the state-value network; is the function representing the target action-state value network; is the function representing the policy network; According to the calculated target V value, action entropy and the output of the current action-state value network, the action-state value network is updated. The loss function formula of the action-state value network is:

[0010]

[0011] In the formula, For In this state, take The reward value obtained by the action; is the weight of the target state value network; is the weight of the action-value network; is the discount factor, ranging from 0 to 1; is the function representing the target state value network; is the function representing the action-state value network; According to the extracted experience samples and the reinforcement learning algorithm, the update gradient of the policy network is calculated, and the policy network is updated. The loss function formula of the policy network is:

[0012] In the formula, is the function of adding noise; is the noise vector, which conforms to the Gaussian spherical distribution; Use soft update to update the target action-state value network and target state value network; S105: Iterative training, repeating S103-104 to perform iterative training of the SAC agent, and after the iterative training is completed, the required SAC agent is obtained.

[0013] It should be further explained that the load data includes the main shaft torque and tower thrust.

[0014] It should be further explained that, in step S103, the reward function is constructed by the sliding window equivalent fatigue load substitution model based on ICNN, and the function expression in the sliding window equivalent fatigue load substitution model based on ICNN is:

[0015] In the formula, is the input vector, which is the load data of the wind farm at time t1 and the load data within the specified time window before time t1; For the The output vector of the layer; For the The output vector of the layer; As the activation function, the convex non-decreasing function Smoothed ReLU is used as the activation function ; and is the weight vector associated with the layer; For the The bias term of the layer; About The convex function of After training, it is constant; is the total number of layers; The reward value calculated using the reward function is the equivalent fatigue load of the wind farm at time t1 and within the specified time window before time t1.

[0016] It should be further explained that, in step S105, the total number of iterations of iterative training is ≥ 80,000 times.

[0017] It should be further explained that the parameters of the constraint controller include the active output constraint of the wind turbine, the dispatch instruction constraint of the transmission system operator and the active power ramp rate constraint of the wind turbine. The solution condition of the constraint controller is:

[0018] In the formula, To optimize the goal, Minimize is the theoretical output lower limit of the j-th wind turbine unit, is the theoretical output upper limit of the j-th wind turbine; m is the total number of wind turbines; is the active power dispatch instruction of the j-th wind turbine; is the initial value of the active power dispatch instruction of the j-th wind turbine; is the active power dispatch instruction of the j-th wind turbine at the previous moment; dispatching instructions for transmission system operators; is the absolute value of the power change of the j-th wind turbine per unit time; is the maximum change in power per unit time of the j-th wind turbine group.

[0019] In a second aspect, the present invention provides a method for realizing the above-mentioned wind farm active load shedding optimization control system, comprising: Data acquisition module, used to obtain wind farm status and load data; SAC agent construction and training module, used to build and train SAC agents; The constraint controller module is used to perform secondary calculations on the initial actions generated by the SAC agent; The execution module is used to execute the actions optimized by the constraint controller module and adjust the active power of each wind turbine in the wind farm.

[0020] In a third aspect, the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to implement the steps of the above-mentioned active load reduction optimization control method for a wind farm when executing the computer program.

[0021] In a fourth aspect, the present invention provides a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned active load reduction optimization control method for a wind farm are implemented.

[0022] The beneficial effects of the present invention are: 1. The active load reduction optimization control method, system, device and storage medium of a wind farm provided by the present invention builds and trains a SAC intelligent agent through a SAC algorithm, utilizes deep learning to process nonlinear factors in a wind farm, and realizes a reasonable distribution of active power in the wind farm, which can reduce the equivalent fatigue load of the entire wind farm, reduce fatigue damage of wind turbines, and thus reduce maintenance costs; the SAC algorithm has low hyperparameter sensitivity, strong model generalization, fast calculation speed, can adapt to different environments, has high stability and reliability, and high learning efficiency.

[0023] 2. The present invention constructs the reward function of the SAC agent based on the sliding window equivalent fatigue load substitution model of ICNN, which can transform the complex fatigue load calculation in the wind farm into a convex function form, improve the calculation speed while maintaining high precision, realize the real-time online calculation of the equivalent fatigue load, improve the construction and training efficiency of the SAC agent, and further improve the convergence of reinforcement learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the description will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 It is a flow chart of a method for optimizing active load shedding control of a wind farm in one embodiment of the present invention.

[0026] Figure 2 It is a flowchart of the steps of constructing and training a SAC agent in one embodiment of the present invention.

[0027] Figure 3It is a schematic block diagram of an active load shedding optimization control system of a wind farm in one embodiment of the present invention.

[0028] Figure 4 It is a schematic diagram of the hardware structure of an electronic device in one embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0030] The active load reduction optimization control method of the wind farm involved in this application is mainly aimed at the active power control of the wind farm. During the operation of the wind farm, by constructing and training a SAC agent, deep learning is used to process the nonlinear factors in the wind farm, and the SAC agent and the constraint controller are used to process the wind farm state quantity in sequence to obtain a vector composed of the active power optimization reference values ​​of each wind turbine in the wind farm, and then the active power of each wind turbine in the wind farm is adjusted in real time to achieve a reasonable distribution of the active power of the wind farm, reduce the equivalent fatigue load of the entire wind farm, reduce the fatigue damage of the wind turbine, and thus reduce the maintenance cost. At the same time, this application constructs the reward function of the SAC agent based on the sliding window equivalent fatigue load substitution model of ICNN, which can convert the complex fatigue load calculation in the wind farm into a convex function form, improve the calculation speed while maintaining high precision, and realize the real-time online calculation of the equivalent fatigue load, which can improve the construction and training efficiency of the SAC agent and further improve the convergence of reinforcement learning.

[0031] The active load reduction optimization control method for wind farms involved in this application is mainly aimed at the technical problems in the existing data-driven wind farm fatigue load control strategy, in which some models exhibit non-convex characteristics, resulting in difficulty in solving and easy falling into local optimal situations; and in the reinforcement learning method, the convergence of the intelligent agent is greatly affected by hyperparameters and the generalization is weak.

[0032] The following is a detailed description of the active load reduction optimization control method for a wind farm involved in the present application. For the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are provided to facilitate a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details.

[0033] In the active load reduction optimization control method of the wind farm involved in the present application, the term "including" used indicates the existence of the described features, wholes, steps, operations, elements and / or components, but does not exclude the existence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0034] In order to clearly describe the technical solution of the present application, the words "first", "second" and the like are used to distinguish the same or similar items with substantially the same functions and effects. Those skilled in the art can understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not necessarily limit the difference.

[0035] The phrases such as "one embodiment" or "some embodiments" described in the present application mean that the specific features, structures or characteristics described in the embodiment are included in one or more embodiments of the present application. Therefore, the phrases such as "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments" etc. that appear in different places in the present application do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] The active load shedding optimization control method of a wind farm provided by the embodiment of the present invention is executed by a computer device. Accordingly, the active load shedding optimization control system of a wind farm runs in the computer device.

[0038] Figure 1 1 is a flow chart of a method for optimizing active load shedding control of a wind farm according to an embodiment of the present invention. Figure 1 The execution subject may be an active load shedding optimization control system for a wind farm. According to different requirements, the order of the steps in the flow chart may be changed, and some may be omitted.

[0039] like Figure 1 As shown, the active load reduction optimization control method of the wind farm includes: S1. Build and train a SAC agent using the SAC algorithm.

[0040] The purpose of constructing a SAC agent through the SAC algorithm is to use the reinforcement learning method to enable the SAC agent to automatically learn how to adjust the active power of each wind turbine under different wind farm conditions to achieve optimal control of active load shedding in wind farms. This intelligent optimization control can replace traditional control methods based on fixed rules or experience, and improve the accuracy and adaptability of control.

[0041] In some embodiments, Figure 2 As shown, the steps of constructing and training the SAC agent in step S1 include: S101: Construct strategy network, state value network, target state value network, action-state value network and target action-state value network; The function representing the policy network is: ; In the formula, is the state quantity of the wind farm, which is the input value of the strategy network; It is a vector composed of the active power values ​​of each wind turbine in the wind farm, and is the output value of the strategy network; The function representing the state value network is: ; In the formula, is the state quantity of the wind farm, which is the input value of the state value network; The output value of the state value network is V value; The function representing the target state value network is: ; In the formula, is the state quantity of the wind farm, which is the input value of the target state value network; The output value of the target state value network is the target V value; The function representing the action-state value network is: ; Where s is the state quantity of the wind farm, is a vector composed of the active power values ​​of each wind turbine in the wind farm, and They are all input values ​​of the action-state value network; The output value of the action-state value network is the Q value; The function representing the target action-state value network is: ; In the formula, is the state quantity of the wind farm, is a vector composed of the active power values ​​of each wind turbine in the wind farm, and Both are input values ​​of the target action-state value network; The output value of the target action-state value network is the target Q value; The state-value network of the initial state is the same as the target state-value network, and the action-state-value network is the same as the target action-state-value network; The strategy network includes an input layer, a first hidden layer, a second hidden layer, and an output layer. The first hidden layer has 256 neurons and the second hidden layer has 128 neurons. The state value network includes an input layer, a first hidden layer, a second hidden layer, and an output layer. The first hidden layer has 256 neurons and the second hidden layer has 128 neurons. The target state value network includes an input layer, a first hidden layer, a second hidden layer, and an output layer. The first hidden layer has 256 neurons and the second hidden layer has 128 neurons. The action-state value network consists of an input layer, a first hidden layer, a second hidden layer, and an output layer. The first hidden layer has 256 neurons and the second hidden layer has 128 neurons. The target action-state value network includes an input layer, a first hidden layer, a second hidden layer and an output layer. The first hidden layer has 256 neurons and the second hidden layer has 128 neurons.

[0042] The construction of strategy network, action-state value network and target action-state value network, state value network and target state value network provides a learning and decision-making framework for SAC agents. These networks can output reasonable reference values ​​of wind turbine active power according to different input parameters, thereby realizing intelligent adjustment of wind farm active power, laying the foundation for subsequent training and optimization processes, so that SAC agents can gradually learn the optimal decision-making strategy.

[0043] S102: Experience replay buffer setting and hyperparameter setting, set the SAC algorithm control step size to 1s, the number of training steps to 162000 steps, BatchSize 256, Gamma 0.9, and the learning rate of all networks to 0.001.

[0044] The experience replay buffer can store the experience of the SAC agent in the process of interacting with the environment, avoiding starting from a completely new state every time training, and improving training efficiency. By randomly extracting experience samples for training, the SAC agent can better explore different states and action spaces, increasing the stability and generalization ability of training; the hyperparameter settings can be adjusted according to the specific wind farm conditions and optimization goals to obtain better training results.

[0045] S103: Obtain the wind farm state quantity at time t0, recorded as ,Will Input the strategy network to obtain the vector composed of the initial reference values ​​of active power training of each wind turbine in the wind farm, which is recorded as ; Will Input the constraint controller, and after the secondary calculation of the constraint controller, the active power training optimization reference value of each wind turbine in the wind farm is obtained, which is recorded as ; Wind Farm Execution , adjust the active power of each wind turbine in the wind farm to same, The execution ends at time t1; The wind farm state quantity, load data and load data within the specified time window before time t1 are obtained. The wind farm state quantity of the wind farm at time t1 is recorded as , then input the load data at time t1 and the load data within the specified time window before time t1 into the reward function to obtain The reward value of Will , , the load data and reward value at time t1 form an experience tuple, and the experience tuple is stored in the experience replay buffer.

[0046] Among them, the state quantity of the wind farm reflects the real-time operating status and demand of the wind farm, and the load data is crucial for evaluating the operating status and fatigue life of the wind turbine. Obtaining this data can provide a comprehensive decision-making basis for the SAC agent; the wind farm state quantity is input into the strategy network to obtain the initial reference value of active power training, and then the optimized reference value is obtained through the constraint controller. This process enables the SAC agent to learn how to adjust the active power of the wind turbine to meet various constraints and maximize the reward value under different wind farm conditions; the wind farm executes the optimized active power reference value, and inputs the state quantity and load data after execution into the reward function to obtain the reward value, and then forms an experience tuple for storage. This step provides a rich source of data for subsequent training, enabling the SAC agent to continuously learn and improve from its own experience.

[0047] S104: randomly extracting a batch of experience samples from the experience playback buffer; Using the extracted experience samples, the target Q value is calculated using the target action-state value network, and the target V value is calculated using the target state value network; The state value network is updated according to the calculated target Q value, action entropy and the output of the current state value network. The loss function formula of the state value network is:

[0048] In the formula, is the weight of the state value network; is the weight of the target action-value network; is the weight of the policy network, is the state quantity of the wind farm at time t0; Represents the experience replay buffer; is the function representing the state-value network; is the function representing the target action-state value network; is the function representing the policy network; According to the calculated target V value, action entropy and the output of the current action-state value network, the action-state value network is updated. The loss function formula of the action-state value network is:

[0049]

[0050] In the formula, For In this state, take The reward value obtained by the action; is the weight of the target state value network; is the weight of the action-value network; is the discount factor, ranging from 0 to 1; is the function representing the target state value network; is the function representing the action-state value network; According to the extracted experience samples and the reinforcement learning algorithm, the update gradient of the policy network is calculated, and the policy network is updated. The loss function formula of the policy network is:

[0051] In the formula, is the function of adding noise; is the noise vector, which conforms to the Gaussian spherical distribution; Use soft update to update the target action-state value network and target state value network.

[0052] The state value network is used to evaluate the value of different states, and the action-state value network is used to evaluate the value of different states and actions. By continuously updating the state value network and the action-state value network, the SAC agent can more accurately determine which actions are beneficial and which actions are disadvantageous; the policy network is used to determine the actions taken by the SAC agent in different states. By continuously updating the policy network, the SAC agent can learn better decision-making strategies to maximize long-term rewards; the target state value network and the target action-state value network are updated using soft updates, which avoids drastic changes in the target network and improves the stability of training. The soft update method can make the target network gradually approach the current network, thereby reducing fluctuations during training.

[0053] S105: Repeat S103-104 to perform iterative training of the SAC agent. After the iterative training is completed, the required SAC agent is obtained.

[0054] Through iterative training, the SAC agent can continuously optimize its decision-making strategy. As the number of iterations increases, the SAC agent can gradually learn a better active power adjustment plan for the wind farm, thereby improving the operating efficiency and reliability of the wind farm.

[0055] In some embodiments, the load data includes main shaft torque and tower thrust.

[0056] In some embodiments, in step S103, a reward function is constructed based on the sliding window equivalent fatigue load substitution model of ICNN, and the function expression in the sliding window equivalent fatigue load substitution model based on ICNN is:

[0057] In the formula, is the input vector, which is the load data of the wind farm at time t1 and the load data within the specified time window before time t1; For the The output vector of the layer; For the The output vector of the layer; As the activation function, the convex non-decreasing function Smoothed ReLU is used as the activation function ; and is the weight vector associated with the layer; For the The bias term of the layer; About The convex function of After training, it is constant; is the total number of layers; The reward value calculated using the reward function is the equivalent fatigue load of the wind farm at time t1 and within the specified time window before time t1.

[0058] By constructing a reward function based on the ICNN-based sliding window equivalent fatigue load substitution model, the key fatigue load factors in the operation of wind turbines, such as main shaft torque and tower thrust, are taken into consideration. The sliding window is used to dynamically evaluate the load conditions over a period of time. With the help of ICNN, the data features are accurately processed to guide the SAC agent to learn a strategy that can effectively adjust the active power and reduce fatigue damage of wind turbines.

[0059] In some embodiments, in step S105, the total number of iterations of iterative training is ≥ 80,000.

[0060] Limiting the total number of iterations of iterative training to ≥ 80,000 times can ensure that the SAC agent is fully trained, enabling the SAC agent to fully explore the state and action space and converge to a better decision-making strategy.

[0061] Step S2, obtaining the state quantity of the wind farm.

[0062] In some embodiments, the wind farm state quantity includes a power grid command, a current wind turbine input wind speed, a current wind turbine output power, an instantaneous change in wind turbine main shaft torque, and an instantaneous change in tower thrust.

[0063] Obtaining the above wind farm status information can provide comprehensive wind farm operation status data for subsequent decision-making, enabling the SAC intelligent agent to make reasonable active power adjustments according to actual conditions.

[0064] Step S3, input the wind farm state quantity into the SAC agent to obtain a vector composed of the initial reference value of the active power of each wind turbine in the wind farm, recorded as .

[0065] This step is to generate the initial action, which can preliminarily generate the initial reference value of the active power of each wind turbine, providing a basis for the constraint adjustment in the subsequent steps.

[0066] Step S4: Input constraint controller, constraint controller After a secondary calculation, we can obtain the vector composed of the active power optimization reference values ​​of each wind turbine in the wind farm, which is recorded as .

[0067] By performing secondary calculation on the initial action, a vector consisting of active power optimization reference values ​​is obtained, thereby improving the stability and reliability of wind farm operation.

[0068] In some embodiments, the parameters of the constraint controller include wind turbine active output constraint, transmission system operator dispatch instruction constraint and wind turbine active power ramp rate constraint. The solution condition of the constraint controller is:

[0069] In the formula, To optimize the goal, Minimize is the theoretical output lower limit of the j-th wind turbine unit, is the theoretical output upper limit of the j-th wind turbine; m is the total number of wind turbines; is the active power dispatch instruction of the j-th wind turbine; is the initial value of the active power dispatch instruction of the j-th wind turbine; is the active power dispatch instruction of the j-th wind turbine at the previous moment; dispatching instructions for transmission system operators; is the absolute value of the power change of the j-th wind turbine per unit time; is the maximum change in power per unit time of the j-th wind turbine group.

[0070] The constraint controller takes into account parameters such as the active output constraint of the wind turbine, the dispatch instruction constraint of the transmission system operator and the active power ramp rate constraint of the wind turbine, ensuring that the active power of the wind turbine is adjusted within a reasonable range and meets the requirements of actual operation.

[0071] Step S5: The wind farm executes , adjust the active power of each wind turbine in the wind farm to same.

[0072] The wind farm executes the optimized active power reference value, which enables the wind turbines to adjust according to the decisions of the SAC intelligent body, realizes the active load reduction optimization control, and improves the operation efficiency and reliability of the wind farm.

[0073] Step S6, repeating steps S2-S5 to adjust the active power of each wind turbine in the wind farm in real time.

[0074] By repeating steps S2 to S5 in real time, active power can be adjusted continuously according to the actual state of the wind farm, adapting to the dynamic changes in the operation of the wind farm, and ensuring that the wind farm is always in the optimal operating state.

[0075] In a specific embodiment, the active load reduction optimization control method of the wind farm includes: Step S1, build and train the SAC agent. The SAC agent is built and trained through the SAC algorithm. A 50MW small wind farm with 10 wind turbines is built based on the NERL5MW wind turbine model as a simulation environment to verify the algorithm performance. The hardware environment used is: 12th Gen Intel Core i7-12700H, NVIDIA GeForce RTX 3060 Laptop, 16Gb memory. The steps include: S101: Initialize the neural network, build the policy network, state value network, target state value network, action-state value network and target action-state value network; The strategy network includes an input layer, a first hidden layer, a second hidden layer, and an output layer. The first hidden layer has 256 neurons and the second hidden layer has 128 neurons. The state value network includes an input layer, a first hidden layer, a second hidden layer, and an output layer. The first hidden layer has 256 neurons and the second hidden layer has 128 neurons. The action-state value network consists of an input layer, a first hidden layer, a second hidden layer, and an output layer. The first hidden layer has 256 neurons and the second hidden layer has 128 neurons. The action-state value network consists of an input layer, a first hidden layer, a second hidden layer, and an output layer. The first hidden layer has 256 neurons and the second hidden layer has 128 neurons. The target action-state value network includes an input layer, a first hidden layer, a second hidden layer, and an output layer. The first hidden layer has 256 neurons and the second hidden layer has 128 neurons. The function representing the policy network is: ; In the formula, is the state quantity of the wind farm, which is the input value of the strategy network; It is a vector composed of the active power values ​​of each wind turbine in the wind farm, and is the output value of the strategy network; The function representing the state value network is: ; In the formula, is the state quantity of the wind farm, which is the input value of the state value network; The output value of the state value network is V value; The function representing the target state value network is: ; In the formula, is the state quantity of the wind farm, which is the input value of the target state value network; The output value of the target state value network is the target V value; The function representing the action-state value network is: ; Where s is the state quantity of the wind farm, is a vector composed of the active power values ​​of each wind turbine in the wind farm, and They are all input values ​​of the action-state value network; The output value of the action-state value network is the Q value; The function representing the target action-state value network is: ; In the formula, is the state quantity of the wind farm, is a vector composed of the active power values ​​of each wind turbine in the wind farm, and Both are input values ​​of the target action-state value network; The output value of the target action-state value network is the target Q value; The state-value network of the initial state is the same as the target state-value network, and the action-state-value network is the same as the target action-state-value network; S102: Experience replay buffer settings and hyperparameter settings, set the SAC algorithm control step size to 1s, the number of training steps to 162000 steps, BatchSize 256, Gamma 0.9, and the learning rate of all networks to 0.001; S103: Obtain the wind farm state quantity at time t0, recorded as ,Will Input the strategy network to obtain the vector composed of the initial reference values ​​of active power training of each wind turbine in the wind farm, which is recorded as ; Will Input the constraint controller, and after the secondary calculation of the constraint controller, the active power training optimization reference value of each wind turbine in the wind farm is obtained, which is recorded as ; Wind Farm Execution , adjust the active power of each wind turbine in the wind farm to same, The execution ends at time t1; The wind farm state quantity, load data and load data within the specified time window before time t1 are obtained. The wind farm state quantity of the wind farm at time t1 is recorded as , then input the load data at time t1 and the load data within the specified time window before time t1 into the reward function to obtain The reward value of The reward function is constructed based on the sliding window equivalent fatigue load substitution model of ICNN. ICNN uses 4 hidden layers, the dimensions are all 128 dimensions, the time window size is 10s, the sliding step is 1s, the number of training times is 500000 times, the BatchSize is 128, and the learning rate is 0.0001. The function expression in the sliding window equivalent fatigue load substitution model based on ICNN is:

[0076] In the formula, is the input vector, which is the load data of the wind farm at time t1 and the load data within the specified time window before time t1; For the The output vector of the layer; For the The output vector of the layer; As the activation function, the convex non-decreasing function Smoothed ReLU is used as the activation function ; and is the weight vector associated with the layer; For the The bias term of the layer; About The convex function of After training, it is constant; is the total number of layers; The reward value calculated using the reward function is the equivalent fatigue load of the wind farm at time t1 and in the specified time window before time t1; Will , , the load data and reward value at time t1 form an experience tuple, and the experience tuple is stored in the experience playback buffer; S104: randomly extracting a batch of experience samples from the experience playback buffer; Using the extracted experience samples, the target Q value is calculated using the target action-state value network, and the target V value is calculated using the target state value network; The state value network is updated according to the calculated target Q value, action entropy and the output of the current state value network. The loss function formula of the state value network is:

[0077] In the formula, is the weight of the state value network; is the weight of the target action-value network; is the weight of the policy network, is the state quantity of the wind farm at time t0; Represents the experience replay buffer; is the function representing the state-value network; is the function representing the target action-state value network; is the function representing the policy network; According to the calculated target V value, action entropy and the output of the current action-state value network, the action-state value network is updated. The loss function formula of the action-state value network is:

[0078]

[0079] In the formula, For In this state, take The reward value obtained by the action; is the weight of the target state value network; is the weight of the action-value network; is the discount factor, ranging from 0 to 1; is the function representing the target state value network; is the function representing the action-state value network; According to the extracted experience samples and the reinforcement learning algorithm, the update gradient of the policy network is calculated, and the policy network is updated. The loss function formula of the policy network is:

[0080] In the formula, is the function of adding noise; is the noise vector, which conforms to the Gaussian spherical distribution; Use soft update to update the target action-state value network and target state value network; S105: Repeat S103-104 to perform iterative training of the SAC agent. The number of training steps is 162,000. After the iterative training is completed, the desired SAC agent is obtained. Step S2, obtaining the wind farm state quantity, the wind farm state quantity including the power command of the power grid, the current input wind speed of the wind turbine, the current output power of the wind turbine, the instantaneous change of the wind turbine main shaft torque and the instantaneous change of the tower thrust; Step S3, input the wind farm state quantity into the SAC agent to obtain a vector composed of the initial reference value of the active power of each wind turbine in the wind farm, recorded as ; Step S4: Input constraint controller. The parameters of constraint controller include wind turbine active output constraint, transmission system operator dispatch instruction constraint and wind turbine active power ramp rate constraint. The solution condition of constraint controller is:

[0081] In the formula, To optimize the goal, Minimize is the theoretical output lower limit of the j-th wind turbine unit, is the theoretical output upper limit of the j-th wind turbine; m is the total number of wind turbines; is the active power dispatch instruction of the j-th wind turbine; is the initial value of the active power dispatch instruction of the j-th wind turbine; is the active power dispatch instruction of the j-th wind turbine at the previous moment; dispatching instructions for transmission system operators; is the absolute value of the power change of the j-th wind turbine per unit time; is the maximum change value of the power of the j-th wind turbine per unit time; Constraint Controller After a secondary calculation, we can obtain the vector composed of the active power optimization reference values ​​of each wind turbine in the wind farm, which is recorded as ; Step S5: The wind farm executes , adjust the active power of each wind turbine in the wind farm to same; Step S6, repeating steps S2-S5 to adjust the active power of each wind turbine in the wind farm in real time.

[0082] The following is an embodiment of an active load shedding optimization control system for a wind farm provided by an embodiment of the present disclosure. The active load shedding optimization system and the active load shedding optimization control method for a wind farm in the above-mentioned embodiments belong to the same inventive concept and are used to implement the active load shedding optimization control method for a wind farm in the above-mentioned embodiments. For details not fully described in the embodiment of the active load shedding optimization control system for a wind farm, reference can be made to the embodiment of the active load shedding optimization control method for a wind farm mentioned above.

[0083] A mobile terminal implementing various embodiments of the present invention will now be described with reference to the accompanying drawings. In the subsequent description, suffixes such as "module", "component" or "unit" used to represent elements are used only to facilitate the description of the embodiments of the present invention and have no specific meaning in themselves. Therefore, "module" and "component" can be used interchangeably.

[0084] like Figure 3 As shown in the figure, the active load reduction optimization control system includes: Data acquisition module, used to obtain wind farm status and load data; Agent construction and training module, used to build and train SAC agents; The constraint controller module is used to perform secondary calculations on the initial actions generated by the agent; The execution module is used to execute the actions optimized by the constraint controller module and adjust the active power of each wind turbine in the wind farm.

[0085] The present application also provides an electronic device for implementing various embodiments of the present invention. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor.

[0086] Those skilled in the art will appreciate that the electronic device structure involved in the embodiments of the present invention does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0087] Figure 4 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0088] The electronic device 500 includes, but is not limited to, components such as a processor 501, a network module 502, an audio output unit 503, an input unit 504, a display unit 506, a user input unit 507, an interface unit 508, and a memory 509. Those skilled in the art will appreciate that the electronic device structure involved in the embodiments of the present invention does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0089] In the embodiments of the present invention, electronic devices include but are not limited to laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the embodiments of the present application described and / or required herein.

[0090] In the embodiment of the present application, the processor 501 can be implemented by using at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a processor, a controller, a microcontroller, a microprocessor, and an electronic unit designed to perform the functions described herein. In some cases, such an implementation can be implemented in a controller. For software implementation, implementations such as processes or functions can be implemented with separate software modules that allow execution of at least one function or operation. The software code can be implemented by a software application (or program) written in any appropriate programming language, and the software code can be stored in a memory and executed by a controller.

[0091] The display unit 506 is used to display information input by the user or information provided to the user. The display unit 506 may include a display panel, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc.

[0092] The user input unit 507 may include, but is not limited to, a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, and a joystick, which will not be described in detail here.

[0093] The interface unit 508 is an interface for connecting external devices to the electronic device 500. For example, the external devices may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, an earphone port, and the like.

[0094] In addition, the electronic device 500 includes some functional modules not shown, which will not be described in detail here.

[0095] Those skilled in the art will appreciate that the various aspects of the electronic device provided by the present application may be implemented as a system, method or program product. Therefore, the various aspects of the present disclosure may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software, which may be collectively referred to as a "circuit", "module" or "system" herein.

[0096] The present application also provides a storage medium, in which a program product capable of implementing the active load reduction optimization control method of a wind farm is stored. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary implementations of the present disclosure described in the above “Exemplary Method” section of this specification.

[0097] The storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0098] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for optimizing active load shedding control of a wind farm, characterized in that the steps include: S1. Build and train a SAC agent using the SAC algorithm; S2. Obtaining wind farm status; S3. Input the wind farm state quantity into the SAC agent to obtain the vector composed of the initial reference value of the active power of each wind turbine in the wind farm, denoted as ; S4. Input constraint controller, constraint controller After a secondary calculation, we can obtain the vector composed of the active power optimization reference values ​​of each wind turbine in the wind farm, which is recorded as ; S5. Wind Farm Implementation , adjust the active power of each wind turbine in the wind farm to same; S6. Repeat steps S2-S5 to adjust the active power of each wind turbine in the wind farm in real time.

2. The active load shedding optimization control method according to claim 1, characterized in that: The wind farm status variables include the power command of the power grid, the current input wind speed of the wind turbine, the current output power of the wind turbine, the instantaneous change of the wind turbine main shaft torque and the instantaneous change of the tower thrust.

3. The active load shedding optimization control method according to claim 2, characterized in that: In step S1, the steps of building and training the SAC agent include: S101: Construct strategy network, state value network, target state value network, action-state value network and target action-state value network; The function representing the policy network is: ; In the formula, is the state quantity of the wind farm, which is the input value of the strategy network; It is a vector composed of the active power values ​​of each wind turbine in the wind farm, and is the output value of the strategy network; The function representing the state value network is: ; In the formula, is the state quantity of the wind farm, which is the input value of the state value network; The output value of the state value network is V value; The function representing the target state value network is: ; In the formula, is the state quantity of the wind farm, which is the input value of the target state value network; The output value of the target state value network is the target V value; The function representing the action-state value network is: ; Where s is the state quantity of the wind farm, is a vector composed of the active power values ​​of each wind turbine in the wind farm, and They are all input values ​​of the action-state value network; The output value of the action-state value network is the Q value; The function representing the target action-state value network is: ; In the formula, is the state quantity of the wind farm, is a vector composed of the active power values ​​of each wind turbine in the wind farm, and Both are input values ​​of the target action-state value network; The output value of the target action-state value network is the target Q value; The state-value network of the initial state is the same as the target state-value network, and the action-state-value network is the same as the target action-state-value network; S102: Experience replay buffer setting and hyperparameter setting; S103: Obtain the wind farm state quantity at time t0, recorded as ,Will Input the strategy network to obtain the vector composed of the initial reference values ​​of active power training of each wind turbine in the wind farm, which is recorded as ; Will Input the constraint controller, and after the secondary calculation of the constraint controller, the active power training optimization reference value of each wind turbine in the wind farm is obtained, which is recorded as ; Wind Farm Execution , adjust the active power of each wind turbine in the wind farm to same, The execution ends at time t1; The wind farm state quantity, load data and load data within the specified time window before t1 are obtained. The wind farm state quantity of the wind farm at t1 is recorded as , then input the load data at time t1 and the load data within the specified time window before time t1 into the reward function to obtain The reward value of Will , , the load data and reward value at time t1 form an experience tuple, and the experience tuple is stored in the experience playback buffer; S104: randomly extracting a batch of experience samples from the experience playback buffer; Using the extracted experience samples, the target Q value is calculated using the target action-state value network, and the target V value is calculated using the target state value network; The state value network is updated according to the calculated target Q value, action entropy and the output of the current state value network. The loss function formula of the state value network is: In the formula, is the weight of the state value network; is the weight of the target action-value network; is the weight of the policy network, is the state quantity of the wind farm at time t0; Represents the experience replay buffer; is the function representing the state-value network; is the function representing the target action-state value network; is the function representing the policy network; According to the calculated target V value, action entropy and the output of the current action-state value network, the action-state value network is updated. The loss function formula of the action-state value network is: In the formula, For In this state, take The reward value obtained by the action; is the weight of the target state value network; is the weight of the action-value network; is the discount factor, ranging from 0 to 1; is the function representing the target state value network; is the function representing the action-state value network; According to the extracted experience samples and the reinforcement learning algorithm, the update gradient of the policy network is calculated, and the policy network is updated. The loss function formula of the policy network is: In the formula, is the function of adding noise; is the noise vector, which conforms to the Gaussian spherical distribution; Use soft update to update the target action-state value network and target state value network; S105: Repeat S103-104 to perform iterative training of the SAC agent, and obtain the desired SAC agent after the iterative training is completed.

4. The active load shedding optimization control method according to claim 3, characterized in that: The load data includes main shaft torque and tower thrust.

5. The active load shedding optimization control method according to claim 4, characterized in that: In step S103, the reward function is constructed by the sliding window equivalent fatigue load substitution model based on ICNN. The function expression in the sliding window equivalent fatigue load substitution model based on ICNN is: In the formula, is the input vector, which is the load data of the wind farm at time t1 and the load data within the specified time window before time t1; For the The output vector of the layer; For the The output vector of the layer; As the activation function, the convex non-decreasing function Smoothed ReLU is used as the activation function ; and is the weight vector associated with the layer; For the The bias term of the layer; About The convex function of After training, it is constant; is the total number of layers; The reward value calculated using the reward function is the equivalent fatigue load of the wind farm at time t1 and within the specified time window before time t1.

6. The active load shedding optimization control method according to claim 3, characterized in that: In step S105, the total number of iterations of iterative training is ≥ 80,000 times.

7. The active load shedding optimization control method according to claim 1, characterized in that: The parameters of the constraint controller include the active output constraint of the wind turbine, the dispatch instruction constraint of the transmission system operator and the active power ramp rate constraint of the wind turbine. The solution condition of the constraint controller is: In the formula, For the optimization goal, Minimize is the theoretical output lower limit of the j-th wind turbine unit, is the theoretical output upper limit of the j-th wind turbine; m is the total number of wind turbines; is the active power dispatch instruction of the j-th wind turbine; is the initial value of the active power dispatch instruction of the j-th wind turbine; is the active power dispatch instruction of the j-th wind turbine at the previous moment; dispatching instructions for transmission system operators; is the absolute value of the power change of the j-th wind turbine per unit time; is the maximum change in power per unit time of the j-th wind turbine group.

8. A wind farm active load shedding optimization control system for implementing the wind farm active load shedding optimization control method as claimed in any one of claims 1 to 7, characterized in that: include: Data acquisition module, used to obtain wind farm status and load data; SAC agent construction and training module, used to build and train SAC agents; The constraint controller module is used to perform secondary calculations on the initial actions generated by the SAC agent; The execution module is used to execute the actions optimized by the constraint controller module and adjust the active power of each wind turbine in the wind farm.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor is used to implement the steps of the active load shedding optimization control method for a wind farm as claimed in any one of claims 1 to 7 when executing the computer program.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the active load reduction optimization control method for a wind farm as described in any one of claims 1 to 7 are implemented.

Images