Photovoltaic power generation flexible power point tracking control method, device and system

Abstract

The application provides a photovoltaic power generation flexible power point tracking control method, which comprises the following steps: S1, taking a power supply decision module of a photovoltaic model as an intelligent agent of reinforcement learning, and intelligently tracking a photovoltaic power generation flexible power point in the photovoltaic model; S2, based on the interaction between the feedback signal of the intelligent agent and the environment, the reinforcement learning algorithm adjusts and improves the intelligent decision behavior, and the optimal tracking strategy of the flexible power point is decided; S3, the intelligent agent decides the optimal scheduling strategy through the interaction with the environment, so as to track the photovoltaic power generation flexible power point in the changing environment. The application can quickly and accurately track the photovoltaic power generation flexible power point in the case of sudden change of environmental conditions, and is not prone to misjudgment, and is also suitable for the case that the environmental conditions are fixed and there is no prior knowledge.

Description

Technical Field

[0001] This invention relates to the technical field of flexible power point tracking in photovoltaic power generation, and more particularly to a control device and method for flexible power output of photovoltaic power generation based on reinforcement learning. Background Technology

[0002] In recent years, the photovoltaic industry has been exploring and applying various methods to find the maximum power point of photovoltaic power generation. However, the power generation of photovoltaic power plants has strong randomness and volatility, and is greatly affected by weather and geographical location. Simply pursuing maximum power can easily lead to unstable power output. Furthermore, the large-scale centralized grid connection of photovoltaic systems will inevitably affect the system's thermal stability, transient stability, voltage stability, frequency stability, and power quality. To ensure the safe and stable operation of the power system, the safety and stability of photovoltaic power generation has become one of the primary considerations, leading to the emergence and development of research on flexible power output control for photovoltaic power generation. Traditional control methods are simple, but they are difficult to cope with rapidly changing environmental conditions, and their control accuracy is poor when environmental conditions are complex and changing. Summary of the Invention

[0003] This invention proposes a flexible power point tracking control method, device, and system for photovoltaic power generation, which can overcome the problems of traditional methods being unable to cope with rapidly changing complex environmental conditions and being prone to misjudgment.

[0004] To achieve the above objectives, this invention proposes a flexible power point tracking control method for photovoltaic power generation, comprising the following steps:

[0005] S1. The power decision module of the photovoltaic model is used as an intelligent agent for reinforcement learning, and intelligent tracking is performed on the flexible power point of photovoltaic power generation in the photovoltaic model.

[0006] S2. Based on the interaction between the agent's feedback signals and the environment, the reinforcement learning algorithm adjusts and improves the intelligent decision-making behavior, and determines the optimal tracking strategy for the flexible power point.

[0007] S3, the intelligent agent interacts with the environment to determine the optimal scheduling strategy in order to track the flexible power point of photovoltaic power generation in a constantly changing environment.

[0008] Furthermore, the agent is based on an Actor-Critic architecture with an action value function, comprising four networks: an actor network, an actor-target network, a critic network, and a critic-target network; wherein the actor network learns the policy function π, and the critic network learns the action value function Q; the action value function and the action function are respectively derived from the deep neural network Q(s). t ,a t |θ Q ) and μ(s t |θ μ ) is obtained through training, where θQ With θ μ These correspond to the parameters of weights and biases for each node in each layer of the critic network and the actor network; the parameter θ in the actor network... μ Updates can be made using gradient methods:

[0009]

[0010] in Let ρ be the partial derivative, J be the distribution of the actor network parameters with respect to expected returns, and ρ be the partial derivative. β For state s t The distribution it follows.

[0011] The Critic network minimizes the loss function using stochastic gradient descent:

[0012] L(θ Q ) = E (s,a) [(Q(s t ,a t |θ Q )-y t ) 2 ],

[0013] y t =r t (s t ,a t )+γQ(s t+1 ,μ(s t |θ μ )|θ Q ).

[0014] Where L(θ) Q ) is the loss function, y t For the expected return, r t The reward is the action in the current state, and γ is the decay rate, which is generally taken as 0.9 to 1.

[0015] Furthermore, the method for intelligent tracking of flexible power points in a photovoltaic model includes the following steps: modeling the intelligent tracking process of flexible power points in a photovoltaic model as a Markov decision process, and establishing an environment model, state space model, action space model, reward function model, and neural network model for tracking flexible power points in a photovoltaic model based on the Markov decision process and the DDPG reinforcement learning algorithm.

[0016] Furthermore, step 2 includes the following steps:

[0017] S2.1 Set the corresponding task objectives;

[0018] S2.2 The intelligent agent interacts with the environment through actions;

[0019] S2.3 The reinforcement learning algorithm is trained using data from the interaction between the agent and the environment to modify its own action strategy;

[0020] S2.4 After several iterations, the optimal tracking strategy for the corresponding task is finally obtained.

[0021] Furthermore, the reinforcement learning algorithm includes a training model, the training model comprising:

[0022] A tuple consisting of (state, behavior, reward, next state) t a t r t s t-1 s t-2 Training is performed using samples such that the initial distribution J = E(R) t Maximize the expected return of ), where s t Given the current state, a t For the action to be performed in the current state, r t s is the immediate reward obtained after performing an action. t-1 The state at the previous moment, s t-2 This refers to the state at the previous moment;

[0023] Use the state action value function Q π (s,a) represents the state s. t Follow the strategy π to take action a t Reward R t Expectations:

[0024] Introducing reward and discount factors, the Bellman equation for reinforcement learning is constructed as follows:

[0025] The objective function is the reward obtained from the current action plus the maximum expected value obtained from the next action, where r(s) t ,a t ) represents the immediate reward obtained after performing the action, and γ is a discount factor with a value range of [0,1], which is set according to the importance of future rewards in learning.

[0026] Furthermore, the method for training a reinforcement learning algorithm includes the following steps:

[0027] S2.3.1: At each time step, the agent's observed environmental quantities include the state s t s t-1 s t-2 Action a t and reward function r t ;

[0028] S2.3.2: The agent in the current state s t Take action a t And transition to the next state s through action function A. t+1 :s t+1 =A(s) t a t );

[0029] S2.3.3: The environment model is based on the current state s t Execute action a t and the next state s t+1 Through the reward function R:r t =R(s) t a t s t+1 )return;

[0030] S2.3.4: Q π (s, a) is called the state-action value function Q. π (s,a)=E π [G t |S t =s,A t =a], which represents the value of the agent's corresponding strategy under a certain state s and a certain action a;

[0031] S2.3.5: When an agent takes an action in a certain state s, a cumulative reward is defined. To measure the value of an action taken in state s, V(s) = E[G t |S t =s];

[0032] Wherein, the value function V is the total expectation obtained from the Markov decision process starting from this state.

[0033] Furthermore, the state-space model includes: temperature, illumination, voltage, and power, wherein voltage and power contain the values ​​of the three most recent moments, and there are also the rate of change of power in the PV curve and the rate of change of current in the IV curve, the two rates of change containing the values ​​of the two most recent moments;

[0034] The state-space model is represented as follows:

[0035]

[0036] Where G is the light intensity at the current moment, T is the temperature at the current moment, V(t) is the voltage at the current moment, V(t-1) is the voltage at the previous moment, V(t-2) is the voltage at the two moments before that, P(t) is the power at the current moment, P(t-1) is the power at the previous moment, P(t-2) is the power at the two moments before that, I is the current value, I(t) = P(t) / V(t), and ΔP is the power change value, ΔP(t) = P(t) - P(t-1). The rate of change of current, The current change rate at the previous moment is denoted as . The change in conductivity at the current moment. This represents the change in conductance at the previous moment;

[0037] The action space model includes: action value a t The voltage change is set as the variable, and the agent determines the next increase or decrease in voltage by acquiring the current state in order to complete the continuous tracking of the flexible power point.

[0038] The action space model is represented as follows:

[0039] A=(ΔV min , ΔV max );

[0040] Where A represents the action, and ΔV min ΔV is the threshold value for voltage reduction. max The threshold for the voltage increase;

[0041] The reward function model includes a reward function. Based on the actual characteristics of photovoltaic power point tracking control, a piecewise reward function is adopted, which decomposes the difficulty of the reinforcement learning task and allows partial rewards to be obtained even if the task is not completed.

[0042] The reward function is represented as follows:

[0043]

[0044]

[0045] Where, r t As a reward, Δe is the rate of change of current at the current moment compared to the previous moment.

[0046] Based on the same concept, this invention also proposes a flexible power point tracking control device for photovoltaic power generation, comprising: a photovoltaic model module and a reinforcement learning model;

[0047] The photovoltaic model module includes a photovoltaic power source, a DC / DC buck converter, a resistive load, and a power decision module, which acts as an intelligent agent for reinforcement learning.

[0048] The reinforcement learning module also includes an environment model module, a state space model module, an action space model module, a reward function model module, and a neural network model module, which are used to configure the environment model, state space model, action space model, reward function model, and neural network model, respectively, to achieve tracking control of flexible power output of photovoltaic power generation.

[0049] The present invention also proposes a control system for flexible photovoltaic power output, comprising: the aforementioned flexible photovoltaic power point tracking control device, a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the aforementioned flexible photovoltaic power point tracking control method.

[0050] This invention has the following advantages:

[0051] This invention uses the power decision module of a photovoltaic model as a reinforcement learning agent. Based on the interaction between the agent and the environment, reinforcement learning adjusts and improves the intelligent decision-making behavior to obtain the optimal tracking strategy for the flexible power point of photovoltaic power generation. Then, through the agent's interaction with the environment, the optimal energy storage scheduling strategy is determined, tracking the flexible power point in a constantly changing environment. This invention can quickly and accurately track the flexible power point of photovoltaic power generation even under abrupt changes in environmental conditions, with less likelihood of misjudgment. It is also applicable to situations with fixed environmental conditions and no prior knowledge. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating the flexible power point tracking control method for photovoltaic power generation according to the present invention.

[0053] Figure 2 This is a schematic diagram of the circuit structure of a single diode model of a photovoltaic power source in a photovoltaic model.

[0054] Figure 3 This is a schematic diagram of the PV and IV curves of a traditional photovoltaic power source.

[0055] Figure 4 This is a schematic diagram of the reinforcement learning algorithm of this invention.

[0056] Figure 5 A schematic diagram of the control device for photovoltaic flexible power output in this invention. Detailed Implementation

[0057] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The advantages and features of the present invention will become clearer from the following description and claims. It should be noted that the drawings are all in a very simplified form and use non-precise ratios, and are only used to facilitate and clarify the illustration of the embodiments of the present invention.

[0058] This invention addresses the problems in existing flexible power output control for photovoltaic power generation by proposing a flexible power point tracking control method for photovoltaic power generation based on the Deep Deterministic Policy Gradient (DDPG) reinforcement learning algorithm. DDPG reinforcement learning is a model-free, self-learning control method. Based on the self-learning characteristics of reinforcement learning, this invention can overcome the problems of traditional methods being unable to cope with rapidly changing complex environmental conditions and being prone to misjudgment.

[0059] like Figure 1 As shown, this invention proposes a flexible power point tracking control method for photovoltaic power generation, comprising the following steps:

[0060] S1. The power decision module of the photovoltaic model is used as a reinforcement learning agent, and intelligent tracking is performed on the flexible power point of photovoltaic power generation in the photovoltaic model.

[0061] The photovoltaic model includes a photovoltaic power source, a DC / DC buck converter, and a resistive load. The photovoltaic power source is connected to the front end of the converter, and the converter outputs the converted voltage to the load connected to the back end. Figure 2 The image shows a single-diode model of a photovoltaic power source, where I ph It is the electric current generated by sunlight, I DS It is the current flowing through the diode, V DS It is the voltage across the diode, I sh To shunt the parallel resistor R sh The current generated by sunlight, I, when the ambient temperature and light intensity remain constant. ph If kept constant, the photovoltaic (PV) power source can be considered a constant current source. The power decision module of the PV model is an intelligent agent structure based on a reinforcement learning algorithm, used for tracking the flexible power point of PV power generation. The flexible power point refers to the point where the output power at the original operating point changes when the temperature and light intensity in the PV environment change. By adjusting the change in output voltage, the operating point position is adjusted, ultimately adjusting the output power of the PV system. The position of the operating point when the output power of the PV system reaches the required power is called the flexible power point. The flexible power point of PV power generation is not the peak value on the PV characteristic curve of the PV array, but rather the point where the system requires power. When the operating point is at this point, the output power of the PV array will meet the power requirements of the grid-connected system. The PV curve (power-voltage curve) of the PV array under constant environmental conditions is formed by the output power PPV and voltage VPV of the PV array at any given time. The output curve of a traditional PV power source is as follows: Figure 3As shown, Figures a and b are the PV and IV curves at the same temperature but different irradiances, respectively, while Figures c and d are the PV and IV curves at different temperatures but the same irradiance, respectively.

[0062] The intelligent agent is based on an Actor-Critic architecture with an action value function, comprising an actor network (Actor), an actor-target network, and a critic network (Critic and Critic-target networks). The actor network learns the policy function π, and the critic network learns the action value function Q. The action value function and the action function are respectively derived from the deep neural network Q(s). t ,a t |θ Q ) and μ(s t |θ μ ) is obtained through training, where θ Q With θ μ These parameters correspond to the weights and biases of each node in each layer of the critic network and the actor network.

[0063] The parameters θ of the actor network μ Updates can be made using gradient methods:

[0064]

[0065] in Let ρ be the partial derivative, J be the distribution of the actor network parameters with respect to expected returns, and ρ be the partial derivative. β For state s t The distribution it follows;

[0066] The Critic network minimizes the loss function using stochastic gradient descent:

[0067] L(θ Q ) = E (s,a) [(Q(s t ,a t |θ Q )-y t ) 2 ],

[0068] y t =r t (s t ,a t )+γQ(s t+1 ,μ(s t |θ μ )|θ Q )

[0069] Where, L(θ) Q ) is the loss function, y t For the expected return, r tThe reward is the action in the current state, and γ is the decay rate, which is generally taken as 0.9 to 1.

[0070] The process of flexible power point tracking in photovoltaic power generation is modeled as a Markov decision process. Based on the Markov decision process, an environment model, state space model, action space model, reward function model, and neural network model for flexible power point tracking in photovoltaic power generation based on the DDPG reinforcement learning algorithm are established.

[0071] The environmental model refers to the background of the photovoltaic power generation flexible power point tracking process, that is, the world in which the intelligent agent operates. Specifically, the environmental model consists of a photovoltaic power generation flexible power point, whose state value is the position of the operating point, and whose behavior is the increase and decrease of the duty cycle.

[0072] The state-space model includes: temperature, illumination, voltage, and power. Voltage and power contain the values ​​of the three most recent moments. In addition, there are the rate of change of power in the PV curve and the rate of change of current in the IV curve. This can solve the problems of excessive training time and high convergence difficulty in the ordinary DDPG algorithm. The two rates of change contain the values ​​of the two most recent moments.

[0073] The state-space model is represented as follows:

[0074]

[0075] Where G is the light intensity at the current moment, T is the temperature at the current moment, V(t) is the voltage at the current moment, V(t-1) is the voltage at the previous moment, V(t-2) is the voltage at the two moments before, P(t) is the power at the current moment, P(t-1) is the power at the previous moment, P(t-2) is the power at the two moments before, I is the current value, I(t) = P(t) / V(t), and ΔP is the power change value, ΔP(t) = P(t) - P(t-1). The rate of change of current, The current change rate at the previous moment is denoted as . The change in conductivity at the current moment. This represents the change in conductivity at the previous moment.

[0076] The action space model includes: the action value *at* is set as the change in voltage, and the agent determines the next increase or decrease in voltage by acquiring the current state to complete the continuous tracking of the flexible power point. The action space model is represented as follows:

[0077] A=(ΔV min , ΔV max )

[0078] Where A represents the action, and ΔV min ΔV is the threshold value for voltage reduction.max This is the threshold for the voltage increase.

[0079] The voltage change has different action values ​​for different states, but they are all specified within the minimum and maximum action range to prevent slow training speed due to too small an action and power oscillation due to too large an action.

[0080] The reward function model includes a reward function. Based on the actual characteristics of photovoltaic power point tracking control, a piecewise reward function is adopted, which decomposes the difficulty of the reinforcement learning task and allows for partial rewards even when the task is not completed. Compared with sparse rewards, this approach increases the likelihood of model convergence. The reward function is represented as follows:

[0081]

[0082]

[0083] Where, r t As a reward, Δe is the rate of change of current at the current moment compared to the previous moment.

[0084] S2. Based on the interaction between the agent's feedback signals and the environment, the reinforcement learning algorithm adjusts and improves the intelligent decision-making behavior, and determines the optimal tracking strategy for the flexible power point.

[0085] The agent determines the optimal tracking strategy, which is the process by which the agent can quickly track the flexible photovoltaic power point in a constantly changing environment. During each training session, the agent interacts with the environment model to obtain different strategies. Through repeated training and rewards, the reinforcement learning process ensures that at each moment, the agent is in a specific state. The agent acts based on the value of the current state and its historical strategies. Subsequently, the agent obtains new environmental observations and rewards, learns from these new observations, and makes new actions. This cycle continues until the optimal strategy is obtained.

[0086] Specifically, step 2 includes the following steps:

[0087] S2.1 Set the corresponding task objectives;

[0088] S2.2 The intelligent agent interacts with the environment through actions;

[0089] S2.3 The reinforcement learning algorithm is trained using data from the interaction between the agent and the environment to modify its own action strategy;

[0090] S2.4 After several iterations, the optimal tracking strategy for the corresponding task is finally obtained.

[0091] This application uses the DDPG reinforcement learning algorithm to train an optimal tracking policy. Reinforcement learning is a goal-oriented intelligent method where learners learn through the consequences of their actions without being told what to do. DDPG is a classic policy gradient algorithm in reinforcement learning, its core being its unique Actor-Critic architecture, which includes four networks: the Actor network, the Actor-Target network, the Critic network, and the Critic-Target network. The Actor network (policy network) is responsible for outputting actions, while the Critic network (Q-network) evaluates the reward of the Actor's output action at each step, estimating the Q-value of that action. Thus, the Actor adjusts its policy based on the Critic's evaluation, updating its neural network parameters, while the Critic adjusts its evaluation policy based on the environmental reward feedback, updating its neural network parameters. Both aim to maximize future total rewards. In this application, reinforcement learning mainly consists of an agent and an environment. By setting corresponding task objectives, the agent will interact with the environment through actions. The reinforcement learning algorithm uses the data of the interaction between the agent and the environment to modify its own action strategy. After several iterations of learning, the optimal tracking strategy for the corresponding task is finally obtained.

[0092] like Figure 4 The diagram shown illustrates the principle of reinforcement learning. The training model of the reinforcement learning algorithm consists of tuples (s) of (state, action, reward, next state). t a t r t s t-1 s t-2 ) are used as samples for training, where s t Given the current state, a t For the action to be performed in the current state, r t s is the immediate reward obtained after performing an action. t-1 The state at the previous moment, s t-2 This represents the state at the previous time step. The goal of reinforcement learning is to learn a policy such that the initial distribution J = E(R0). t Maximizing the expected return of an action, which is a value function describing the state s. t The expected return after taking the next action, and thereafter following strategy π: Introducing reward and discount factors to construct the Bellman equation for reinforcement learning The objective function is the reward obtained from the current action plus the maximum expected value to be obtained in the next step, where r(s) t a t) represents the immediate reward obtained after performing the action, and γ is the discount factor; the maximum expected value obtained in the next step is evaluated by multiplying by the discount factor γ to assess the impact of future rewards on the current state; γ is set according to the importance of future rewards in learning, and γ∈[0,1].

[0093] Specifically, the method for training the reinforcement learning algorithm in this embodiment includes the following steps:

[0094] S2.3.1: At each time step, the agent's observed environmental quantities include the state s t s t-1 s t-2 Action a t and reward function r t ;

[0095] S2.3.2: The agent in the current state s t Take action a t And transition to the next state s through action function A. t+1 :s t+1 =A(s) t a t );

[0096] S2.3.3: The environment model is based on the current state s t Execute action a t and the next state s t+1 Through the reward function R:r t =R(s) t a t s t+1 )return;

[0097] S2.3.4: Q π (s, a) is called the state-action value function Q. π (s,a)=E π [G t |S t =s,A t =a], which represents the value of the agent's corresponding strategy under a certain state s and a certain action a;

[0098] S2.3.5: When an agent takes an action in a certain state s, a cumulative reward is defined. To measure the value of an action taken in state s, V(s) = E[G t |S t =s];

[0099] Wherein, the value function V is the total expectation obtained from the Markov decision process starting from this state.

[0100] S3: The agent interacts with the environment to determine the optimal scheduling strategy in order to track the flexible power point of photovoltaic power generation in a constantly changing environment.

[0101] The optimal scheduling strategy can be understood as follows: the system provides a set of current and voltage to the load, but under these conditions, the system's output power does not reach the required flexible power. Therefore, we need to track the most suitable current and voltage to reach the flexible power point. This tracking process is called the scheduling strategy. Specifically, the scheduling strategy works as follows: when the temperature and light intensity in the photovoltaic environment change, the original operating point output power will change. By adjusting the change in output voltage, the operating point position is adjusted, ultimately adjusting the photovoltaic system's output power. When the photovoltaic system's output power reaches the required power, the operating point position at this time is called the flexible power point. This process is called the scheduling strategy.

[0102] When the temperature and light intensity in the photovoltaic environment change, the power decision module of the photovoltaic power supply, based on the scheduling strategy, will change the output power of the original operating point. The power decision module adjusts the voltage change and thus the operating point position, thereby adjusting the output power of the photovoltaic system. Through the optimal scheduling strategy, the flexible power point can be tracked in the shortest time. In this embodiment, the state variables are light intensity, temperature, power, and voltage, the control variable is the voltage change, and the reward function is set to a limit value for the voltage change. The control variable, the voltage change, affects the power output by controlling the voltage change, thus ultimately tracking the flexible power point.

[0103] Based on the same design concept, this invention also proposes a control device for flexible photovoltaic power output, such as... Figure 5 As shown, the system includes a photovoltaic (PV) model module and a reinforcement learning module. The PV model module includes a PV power supply, a DC / DC step-down converter, a resistive load, and a decision-making module for the PV power supply. This power supply decision-making module acts as the agent in the reinforcement learning module. The reinforcement learning module includes an environment model module, a state-space model module, an action-space model module, a reward function model module, and a neural network model module. These modules are used to configure the environment model, state-space model, action-space model, reward function model, and neural network model, respectively, to achieve tracking control of the flexible power output of PV power generation.

[0104] Furthermore, this invention also proposes a control system for flexible photovoltaic power output, comprising: a flexible photovoltaic power point tracking control device, a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a flexible photovoltaic power point tracking control method.

[0105] It should be recognized that embodiments of the present invention can be implemented or carried out by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable storage medium. The method can be implemented using standard programming techniques—including implementation in a computer program on a non-transitory computer-readable storage medium configured to allow the computer to operate in a specific and predefined manner—according to the methods and drawings described in the specific embodiments. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if desired, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. Furthermore, for this purpose, the program can run on a programmed application-specific integrated circuit (ASIC).

[0106] Example

[0107] To evaluate the effectiveness and accuracy of the scheduling strategy proposed in this invention, this embodiment conducts simulations under both fixed and varying environmental conditions (light intensity, temperature) to verify the effectiveness and accuracy of the proposed scheduling strategy. This embodiment uses Python based on Gym to design and develop a flexible power point tracking control method based on the reinforcement learning DDPG algorithm, aiming to rapidly output flexible power. In this set of simulations, this embodiment uses an open-circuit voltage Voc of 17.1V and a short-circuit current Isc of 3.8A, conducting simulations under varying temperature and light intensity conditions to verify the effectiveness of the proposed scheduling strategy when temperature and light intensity are constant.

[0108] In the reinforcement learning algorithm, T = 0.001s, λ = 0.00001, and γ = 0.8 are set. Since the environment is known initially, the agent will first explore until the algorithm converges. After training, the network parameters are saved. At this point, the control algorithm can accurately track the flexible power point under changing environmental conditions, thus achieving flexible power output control.

[0109] This invention demonstrates that the agent successfully tracked the flexible power point by maintaining the power at the level required by the system, thus proving the effectiveness of the proposed method in changing environments.

[0110] Although the present invention has been described in detail through the preferred embodiments above, it should be understood that the above description should not be considered as a limitation of the present invention. Various modifications and substitutions to the present invention will be apparent to those skilled in the art after reading the above description. Therefore, the scope of protection of the present invention should be defined by the appended claims.

Claims

1. A flexible power point tracking control method for photovoltaic power generation, characterized in that, Includes the following steps: S1. The power decision module of the photovoltaic model is used as an intelligent agent for reinforcement learning, and intelligent tracking is performed on the flexible power point of photovoltaic power generation in the photovoltaic model. S2. Based on the interaction between the agent's feedback signals and the environment, the reinforcement learning algorithm adjusts and improves the intelligent decision-making behavior, and determines the optimal tracking strategy for the flexible power point. S3. The intelligent agent interacts with the environment to determine the optimal scheduling strategy in order to track the flexible power point of photovoltaic power generation in a constantly changing environment. The method for intelligent tracking of flexible power points in a photovoltaic model includes the following steps: modeling the intelligent tracking process of flexible power points in a photovoltaic model as a Markov decision process, and establishing an environment model, state space model, action space model, reward function model, and neural network model for tracking flexible power points in a photovoltaic model based on the Markov decision process and the DDPG reinforcement learning algorithm. The state-space model includes: temperature, light intensity, voltage, and power, where voltage and power contain the values ​​of the three most recent moments. In addition, there are the rate of change of power in the PV curve and the rate of change of current in the IV curve, with the two rates of change containing the values ​​of the two most recent moments. The state-space model is represented as follows: ； in, G The light intensity at the current moment, T The temperature at the current moment. V ( t () represents the voltage at the current moment. V ( t- 1) is the voltage at the previous moment. V ( t- 2) represents the voltage at the previous two moments. P ( t () represents the power at the current moment. P ( t -1) represents the power at the previous moment. P ( t -2) represents the power at the previous two moments. I This is the current value. , This represents the power change value. , The rate of change of current, Let be the rate of change of current at the previous moment. The change in conductivity at the current moment. This represents the change in conductance at the previous moment; The action space model includes: action value a t The voltage change is set as the variable, and the agent determines the next increase or decrease in voltage by acquiring the current state in order to complete the continuous tracking of the flexible power point. The action space model is represented as follows: ； Where A represents the action. For the voltage reduction threshold, The threshold for the voltage increase; The reward function model includes a reward function. Based on the actual characteristics of photovoltaic power point tracking control, a piecewise reward function is adopted, which decomposes the difficulty of the reinforcement learning task and allows partial rewards to be obtained even if the task is not completed. The reward function is represented as follows: ； in, r t As a reward, Δ e This represents the rate of change of current from the previous moment to the current at the current moment.

2. The photovoltaic power generation flexible power point tracking control method as described in claim 1, characterized in that, Step S2 includes the following steps: S2.1 Set the corresponding task objectives; S2.2 The agent interacts with the environment model through actions; S2.3 The reinforcement learning algorithm is trained using data from the interaction between the agent and the environment model to modify its own action strategy; S2.4 After several iterations, the optimal tracking strategy for the corresponding task is finally obtained.

3. A control device applied to the flexible power point tracking control method for photovoltaic power generation as described in any one of claims 1-2, characterized in that, include: Photovoltaic model module and reinforcement learning module; The photovoltaic model module includes a photovoltaic power source, a DC / DC buck converter, a resistive load, and a power decision module, which acts as an intelligent agent for reinforcement learning. The reinforcement learning module also includes an environment model module, a state space model module, an action space model module, a reward function model module, and a neural network model module, which are used to configure the environment model, state space model, action space model, reward function model, and neural network model, respectively, to achieve tracking control of flexible power output of photovoltaic power generation.

4. A control system for flexible photovoltaic power output, characterized in that, include: The control device, processor, memory, and computer program stored in the memory and executable on the processor are described in claim 3. The processor executes the computer program to implement the photovoltaic power generation flexible power point tracking control method as described in any one of claims 1-2.

Images