Multi-scene photovoltaic power generation situation awareness method based on RL and edge algorithm

Abstract

The invention discloses a multi-scene photovoltaic power generation situation sensing method based on an RL and an edge algorithm, and belongs to the field of photovoltaic power generation, and the method comprises the following steps: S1, constructing a photovoltaic power generation edge network; s2, constructing a deep reinforcement learning neural network and initializing the deep reinforcement learning neural network; s3, constructing tetrad data (st, at, rt, st + 1); s4, updating the deep reinforcement learning neural network, judging whether the updated deep reinforcement learning neural network converges or not, if yes, outputting the deep reinforcement learning neural network, and if not, returning to the step S3 until convergence; and S5, performing multi-scene photovoltaic power generation situation awareness by using the convergent deep reinforcement learning neural network. According to the multi-scene photovoltaic power generation situation sensing method based on the RL and the edge algorithm, reinforcement learning is combined with the neural network, so that the photovoltaic power generation assembly can adapt to power generation situation sensing of any illumination intensity.

Description

Technical Field

[0001] The present invention relates to the field of photovoltaic power generation technology, and in particular to a multi-scenario photovoltaic power generation situation awareness method based on RL and edge algorithm. Background Art

[0002] With the continuous development of energy transformation (energy transformation refers to the process of shifting from an energy system that relies on traditional fossil fuels (such as coal, oil and natural gas) to more sustainable and environmentally friendly energy sources), the application of environmentally friendly energy is becoming more and more extensive. Among them, solar energy, as one of the most abundant clean energy resources available to mankind, has become a target of vigorous research in the field of power generation at this stage.

[0003] At the same time, due to the rapid development of science and technology, the photovoltaic industry has developed rapidly. According to statistics, by the end of 2021, the cumulative installed capacity of photovoltaic power generation reached 306 million kilowatts, ranking first in the world in both new and cumulative installed capacity. In the same year, the national photovoltaic power generation continued to increase, reaching 325.9 billion kilowatts, an increase of about 25% year-on-year, accounting for 4% of the total power generation, an increase of 0.6 percentage points year-on-year. The photovoltaic industry will usher in a broad prospect, especially distributed photovoltaic power generation has become a new trend.

[0004] Affected by factors such as temperature, light, and geographical location, distributed photovoltaic power generation has the characteristics of intermittent, random, and volatile. In order to ensure the stable operation of photovoltaic power generation and the consumption of excess electricity, situational awareness technology for photovoltaic power generation has emerged.

[0005] Photovoltaic power generation situation awareness is based on the environmental status of photovoltaic power generation components. It uses machine learning and other methods to fully analyze and explore the current status of photovoltaic power generation components, and reasonably predict the future power generation of photovoltaic power generation components, so as to formulate corresponding power consumption or grid dispatching plans in advance, realize real-time, dynamic and overall control of distributed photovoltaic power generation components, and form an efficient new "source-load interaction" model.

[0006] Situational awareness is divided into situational awareness, situational understanding and situational prediction. Among them, situational awareness mainly involves factor extraction and state perception. Specifically, in the photovoltaic power generation process, it refers to the acquisition of environmental factors such as temperature and light, and the use of MPPT (maximum power point tracking) technology enables photovoltaic power generation components to output maximum power stably for a long time.

[0007] Considering the existing MPPT control methods, they are mainly divided into constant voltage tracking method (CVT) and perturbation and observation method (P&O). Among them, the constant voltage tracking method is a curve fitting working mode, and its working principle is: when the outdoor temperature is constant, the distribution of the maximum power point of the photovoltaic cell almost exists on both sides of a vertical line; it has the advantages of simple control and high reliability, and can improve the efficiency of photovoltaic cells by 20%; at the same time, it ignores the influence of ambient temperature on the output voltage of photovoltaic cells, so the constant voltage tracking method is not suitable for places with large temperature changes.

[0008] The working principle of the perturbation observation method is: first give a disturbance output voltage signal (Uw+△U), then measure the change in the output power of the photovoltaic cell and compare it with the power before the disturbance. If it is greater than the power value before the disturbance, it indicates that the disturbance direction is correct and the disturbance can continue in the same (+△U) direction; if it is less than the previous power value, then disturb in the opposite (-△U) direction. It has the advantages of simple structure and few measurement parameters. At the same time, due to the need to select the initial value Uw and the tracking step △U value, the tracking accuracy and speed are affected, and the disturbance will continue when the photovoltaic reaches the maximum power point, making it possible to oscillate near the maximum power point, resulting in power loss and reducing the efficiency of the photovoltaic cell.

[0009] At the same time, most of the current distributed photovoltaic components are far away from the power cloud data center, which leads to the problem of high pressure on data transmission. Summary of the invention

[0010] To solve the above problems, the present invention provides a multi-scenario photovoltaic power generation situation awareness method based on RL and edge algorithm, which utilizes reinforcement learning combined with neural networks to enable photovoltaic power generation components to adapt to power generation situation awareness of any light intensity.

[0011] To achieve the above objectives, the present invention provides a multi-scenario photovoltaic power generation situation awareness method based on RL and edge algorithm, comprising the following steps:

[0012] S1. Build a photovoltaic power generation edge network;

[0013] S2. Build and initialize a deep reinforcement learning neural network;

[0014] S3, embed the deep reinforcement learning neural network into the MPPT controller, and use the MPPT controller to construct the four-tuple data (s) for the photovoltaic power generation process based on the photovoltaic power generation edge network. t ,a t ,r t ,s t+1 ), and store the four-tuple data into the experience pool;

[0015] S4, randomly extracting four-tuple data samples in the experience pool to update the deep reinforcement learning neural network, and judging whether the updated deep reinforcement learning neural network converges, if it converges, outputting the deep reinforcement learning neural network, if not, returning to step S3 until convergence;

[0016] S5. Use convergent deep reinforcement learning neural network to perform multi-scenario photovoltaic power generation situation awareness.

[0017] Preferably, the photovoltaic power generation edge network described in step S1 includes base stations, local network connection points and regional network connection points that communicate with each other, so as to enable photovoltaic power generation components to access the edge network.

[0018] Preferably, step S2 specifically includes the following steps:

[0019] S21, introducing reinforcement learning into the deep neural network to obtain the deep reinforcement learning neural network, and taking the environmental state of the photovoltaic system in the current period as the input of the deep reinforcement learning neural network, and taking the MPPT adjustment action taken by the environmental state as the output of the deep reinforcement learning neural network, wherein the environmental state includes the temperature and light conditions of the photovoltaic cell assembly;

[0020] S22. Initialization: Initialize the learning network Q(s,a;w) and the target network Q*(s,a;w') of the deep reinforcement learning neural network so that the initial target network Q*(s,a;w') is consistent with the initial learning network Q(s,a;w).

[0021] Preferably, step S3 specifically includes the following steps:

[0022] S31, read the status S through the MPPT controller t :

[0023] S t =[T(t),Lux(t),U(t),I(t)]

[0024] Where, T(t) is the temperature of the photovoltaic cell module during period t; Lux(t) is the light intensity received by the photovoltaic cell module during period t; U(t) is the voltage output by the photovoltaic cell module during period t; I(t) is the current output by the photovoltaic cell module during period t;

[0025] S32, select action a through ε-greedy strategy according to the current learning network Q(s,a;w) t :

[0026]

[0027] In the formula, a represents action;

[0028] S33. Calculate the reward r obtained by the MPPT controller t :

[0029] r t =Δp

[0030] In the formula, Δp is the increment of the output power of the photovoltaic power generation module;

[0031] S34. Calculate the new state s for the next period according to step S31. t+1 ;

[0032] The calculated reward r t , the new state s for the next period t+1 , the state s in period t t 、Action a t As a four-tuple (s t ,a t ,r t ,s t+1 ) is stored in the pre-set experience pool.

[0033] Preferably, in step S32, the MPPT regulation is achieved by adjusting the duty cycle D(t) of the PWM drive signal of the DC / DC converter. Therefore, the action space of the MPPT controller is:

[0034] A=[D(t)]

[0035] Then, we select a set step size to discretize the action space, and then use the random gradient ascent method combined with the variable step size method to find the action a t The corresponding global optimal Q value.

[0036] Preferably, step S4 specifically includes the following steps:

[0037] S41, the new state s of the next period t+1 Considered as the new current state s t , repeat step S3 until the data in the experience pool exceeds the set amount, and then execute step S42;

[0038] S42, randomly extract a four-tuple data sample from the experience pool, and give the reward r obtained by sampling t and the new state s t+1 Send to the target network to calculate the Q* value y t :

[0039]

[0040] Where, γ is the attenuation factor of the correlation between the next state and the current state of the MPPT controller; a t+1 is the action of the next period, w' is the parameter of the target network Q*;

[0041] S43, y t Input into the learning network Q(s,a;w), and use the loss function to calculate the loss value;

[0042] S44. Update the parameters of the learning network Q(s,a;w) based on stochastic gradient descent until the loss value is less than the set value, determine that the learning network Q(s,a;w) converges, and copy the learning network Q(s,a;w) value to the target network Q*(s,a;w'), and the deep reinforcement learning neural network completes convergence.

[0043] Preferably, in step S43, the loss function L of the T period t (w t )The formula is as follows:

[0044]

[0045] In the formula, is the mathematical expectation, y is the optimization target value;

[0046] And the optimization target value y calculation formula is as follows:

[0047]

[0048] Where r is the reward, Q*(s t+1 ,a t+1 ; w') is the new state s evaluated by the target network t+1 The maximum Q value of all actions under the target network is evaluated; a t+1 It is the action of the next period.

[0049] Preferably, in step S44, the parameters of the stochastic gradient descent learning network Q(s,a;w) are derived by partial derivative of the parameter w:

[0050]

[0051] Preferably, step S5 specifically includes the following steps:

[0052] S51. Deploy the converged deep reinforcement learning neural network to the edge computing node: deploy the deep reinforcement learning neural network on the edge computing device located in the photovoltaic power station;

[0053] S52, real-time data collection and preprocessing: collect real-time data of photovoltaic power generation components through the edge network and perform preprocessing;

[0054] S53, Situational awareness and decision generation: Using the deep reinforcement learning neural network deployed on the edge computing node, the current operating status and environmental factors of photovoltaic power generation are continuously evaluated based on the real-time collected data, and the optimal MPPT adjustment action is output;

[0055] S54, action execution and feedback loop: implement the optimal MPPT adjustment action recommended by the deep reinforcement learning neural network, monitor the results after the action is executed, and feed back the execution results to the deep reinforcement learning neural network;

[0056] S55. Continuous optimization and iteration: Based on the collected feedback, the deep reinforcement learning neural network is regularly updated.

[0057] The present invention has the following beneficial effects:

[0058] 1. Considering that the existing photovoltaic power generation maximum power tracking methods have different defects in different scenarios, the use of reinforcement learning combined with neural networks can make photovoltaic power generation components adapt to the power generation situation perception of any light intensity, that is, using reinforcement learning (RL) to interact with the environment and adopt multiple "trial and error" methods to achieve the learning goal; the DQN algorithm (Deep Q-network) that combines deep neural networks with reinforcement learning Q-leaning algorithms can better face photovoltaic power generation situation perception;

[0059] 2. Through edge computing, photovoltaic power generation components are sunk to the edge of the network, that is, edge computing provides the nearest service close to the data source, which can not only process the calculations in photovoltaic power generation in a timely and rapid manner, but also has lower energy consumption, and can achieve real-time and rapid response to changes in photovoltaic power generation situation awareness, support the development of massive distributed photovoltaic operation situation awareness, and then study the technical solutions for distributed photovoltaics to achieve local autonomy, and promote the consumption of distributed photovoltaics.

[0060] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a flow chart of a multi-scenario photovoltaic power generation situation awareness method based on RL and edge algorithm of the present invention;

[0062] Figure 2 This is a model diagram of an edge computing method of a multi-scenario photovoltaic power generation situation awareness method based on RL and edge algorithm of the present invention;

[0063] Figure 3 It is a basic framework diagram of an edge computing method of a multi-scenario photovoltaic power generation situation awareness method based on RL and edge algorithm of the present invention;

[0064] Figure 4 It is the result diagram of the simulation test of the present invention. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical scheme and advantages disclosed in the embodiments of the present invention clearer, the embodiments of the present invention are further described in detail in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the embodiments of the present invention and are not intended to limit the embodiments of the present invention. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions.

[0066] It should be noted that the terms "include", "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or devices.

[0067] Like reference numerals and letters denote similar items in the following drawings, and thus, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0068] At present, most distributed photovoltaic components are far away from the power cloud data center. In order to reduce the pressure of system data transmission and reduce the computing load of the cloud data center, edge computing is introduced into photovoltaic power generation situation awareness. The functions of the power cloud data center such as storage computing, data compression, and feature extraction are sunk to the photovoltaic power generation components at the edge of the network through edge computing. Providing the nearest segment service close to the data source through edge computing can not only process the calculations in photovoltaic power generation in a timely and rapid manner, but also have lower energy consumption.

[0069] Based on the above analysis, the present invention is designed as follows: Figure 1-Figure 3 As shown, a multi-scenario photovoltaic power generation situation awareness method based on RL and edge algorithm includes the following steps:

[0070] S1. Network is the basis of communication. The premise of any communication is that the network is unimpeded. In the process of photovoltaic power generation situation awareness, there is also a communication part. However, unlike the communication of general photovoltaic power generation components, the network communication here specifically refers to the communication of photovoltaic power generation components in the edge network; therefore, the photovoltaic power generation edge network is constructed;

[0071] The photovoltaic power generation edge network described in step S1 includes base stations, local network connection points and regional network connection points that communicate with each other to enable photovoltaic power generation components to access the edge network.

[0072] S2. Build and initialize a deep reinforcement learning neural network;

[0073] As a value-based reinforcement learning algorithm, the core of the Q-leaning algorithm is the state of the agent in different time periods and the action taken in the state, and finally the value Q is obtained. It constructs the state and corresponding action of all time periods into a Q(s,a) table to store the Q value, and then selects the action that can obtain the maximum benefit according to the size of the Q value. However, in the situational awareness for photovoltaic power generation, the state of the MPPT controller is high-dimensional and continuous, resulting in a large amount of data in the traditional Q(s,a) table, which is not conducive to the search of data. Therefore, a deep neural network is introduced into the Q-leaning algorithm, and a deep learning model is used to fit the Q(s,a) table.

[0074] Step S2 specifically includes the following steps:

[0075] S21, introducing reinforcement learning into the deep neural network to obtain the deep reinforcement learning neural network, and taking the environmental state of the photovoltaic system in the current period as the input of the deep reinforcement learning neural network, and taking the MPPT adjustment action taken by the environmental state as the output of the deep reinforcement learning neural network, wherein the environmental state includes the temperature and light conditions of the photovoltaic cell assembly;

[0076] S22. Initialization: Initialize the learning network Q(s,a;w) and the target network Q*(s,a;w') of the deep reinforcement learning neural network so that the initial target network Q*(s,a;w') is consistent with the initial learning network Q(s,a;w).

[0077] S3, embed the deep reinforcement learning neural network into the MPPT controller, and use the MPPT controller to construct the four-tuple data (s) for the photovoltaic power generation process based on the photovoltaic power generation edge network. t ,a t ,r t ,s t+1 ), and store the four-tuple data into the experience pool;

[0078] Step S3 specifically includes the following steps:

[0079] S31, read the status S through the MPPT controller t :

[0080] S t =[T(t),Lux(t),U(t),I(t)]

[0081] Where, T(t) is the temperature of the photovoltaic cell module during period t; Lux(t) is the light intensity received by the photovoltaic cell module during period t; U(t) is the voltage output by the photovoltaic cell module during period t; I(t) is the current output by the photovoltaic cell module during period t;

[0082] S32, select action a through ε-greedy strategy according to the current learning network Q(s,a;w) t :

[0083]

[0084] In the formula, a represents action;

[0085] In step S32, the duty cycle D(t) of the PWM drive signal of the DC / DC converter is adjusted to achieve MPPT regulation. Therefore, the action space of the MPPT controller is:

[0086] A=[D(t)]

[0087] Then, we select a set step size to discretize the action space, and then use the random gradient ascent method combined with the variable step size method to find the action a t The corresponding global optimal Q value.

[0088] S33. Calculate the reward r obtained by the MPPT controller t :

[0089] r t =Δp

[0090] In the formula, Δp is the increment of the output power of the photovoltaic power generation module;

[0091] S34. Calculate the new state s for the next period according to step S31. t+1 ;

[0092] The calculated reward r t , the new state s for the next period t+1 , the state s in period t t 、Action a t As a four-tuple (s t ,a t ,r t ,s t+1 ) is stored in the pre-set experience pool.

[0093] S4, randomly extracting four-tuple data samples in the experience pool to update the deep reinforcement learning neural network, and judging whether the updated deep reinforcement learning neural network converges, if it converges, outputting the deep reinforcement learning neural network, if not, returning to step S3 until convergence;

[0094] Step S4 specifically includes the following steps:

[0095] S41, the new state s of the next period t+1 Considered as the new current state s t , repeat step S3 until the data in the experience pool exceeds the set amount, and then execute step S42;

[0096] S42, randomly extract a four-tuple data sample from the experience pool, and give the reward r obtained by sampling t and the new state s t+1 Send to the target network to calculate the Q* value y t :

[0097]

[0098] Where, γ is the attenuation factor of the correlation between the next state and the current state of the MPPT controller; a t+1 is the action of the next period, w' is the parameter of the target network Q*;

[0099] S43, y t Input into the learning network Q(s,a;w), and use the loss function to calculate the loss value;

[0100] In step S43, the loss function L of the T period t (w t )The formula is as follows:

[0101]

[0102] In the formula, is the mathematical expectation, y is the optimization target value;

[0103] And the optimization target value y calculation formula is as follows:

[0104]

[0105] Where r is the reward, Q*(s t+1 ,a t+1 ; w') is the new state s evaluated by the target network t+1 The maximum Q value of all actions under the target network is evaluated; a t+1 It is the action of the next period.

[0106] S44. Update the parameters of the learning network Q(s,a;w) based on stochastic gradient descent until the loss value is less than the set value, determine that the learning network Q(s,a;w) converges, and copy the learning network Q(s,a;w) value to the target network Q*(s,a;w'), and the deep reinforcement learning neural network completes convergence.

[0107] In step S44, the parameters of the stochastic gradient descent learning network Q(s,a;w) are derived by taking the partial derivative of the parameter w:

[0108]

[0109] S5. Use convergent deep reinforcement learning neural network to perform multi-scenario photovoltaic power generation situation awareness.

[0110] Step S5 specifically includes the following steps:

[0111] S51. Deploy the converged deep reinforcement learning neural network to the edge computing node: Deploy the deep reinforcement learning neural network on the edge computing device located in the photovoltaic power station to achieve fast response and data processing, reduce network transmission delay and the burden on the central server;

[0112] S52, real-time data collection and preprocessing: collect real-time data (such as temperature, light intensity, voltage, current, etc.) of photovoltaic power generation components through the edge network and perform preprocessing;

[0113] S53, Situational awareness and decision generation: Using the deep reinforcement learning neural network deployed on the edge computing node, the current operating status and environmental factors of photovoltaic power generation are continuously evaluated based on the real-time collected data, and the optimal MPPT adjustment action is output to adapt to the changing environmental conditions and improve power generation efficiency;

[0114] S54, action execution and feedback loop: implement the optimal MPPT adjustment action recommended by the deep reinforcement learning neural network, monitor the results after the action is executed (such as changes in power output), and feed back the execution results to the deep reinforcement learning neural network;

[0115] S55. Continuous optimization and iteration: Based on the collected feedback, the deep reinforcement learning neural network is regularly updated to cope with new scenarios and challenges that arise in the photovoltaic power generation process, ensuring that the system maintains efficient and stable operation in the long term.

[0116] Simulation test

[0117] like Figure 4 As shown in the figure, in this simulation experiment, 68,000 distributed photovoltaic real output data were selected for verification, the ratio of training set to test set was set to 4:1, and the average prediction error was about 0.03%.

[0118] Table 1 Partial results

[0119]

[0120]

[0121]

[0122] Therefore, the present invention adopts the above-mentioned multi-scenario photovoltaic power generation situation awareness method based on RL and edge algorithm, and uses reinforcement learning combined with neural network to enable photovoltaic power generation components to adapt to power generation situation awareness of any light intensity.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A multi-scenario photovoltaic power generation situation awareness method based on RL and edge algorithm, characterized by: The following steps are involved: S1. Build a photovoltaic power generation edge network; S2. Build and initialize a deep reinforcement learning neural network; S3, embed the deep reinforcement learning neural network into the MPPT controller, and use the MPPT controller to construct the four-tuple data (s) for the photovoltaic power generation process based on the photovoltaic power generation edge network. t , a t , r t ,s t+1 ), and store the four-tuple data into the experience pool; S4, randomly extracting four-tuple data samples in the experience pool to update the deep reinforcement learning neural network, and judging whether the updated deep reinforcement learning neural network converges, if it converges, outputting the deep reinforcement learning neural network, if not, returning to step S3 until convergence; S5. Use convergent deep reinforcement learning neural network to perform multi-scenario photovoltaic power generation situation awareness.

2. According to claim 1, a multi-scenario photovoltaic power generation situation awareness method based on RL and edge algorithm is characterized in that: The photovoltaic power generation edge network described in step S1 includes base stations, local network connection points and regional network connection points that communicate with each other to enable photovoltaic power generation components to access the edge network.

3. According to claim 1, a multi-scenario photovoltaic power generation situation awareness method based on RL and edge algorithm is characterized in that: Step S2 specifically includes the following steps: S21, introducing reinforcement learning into the deep neural network to obtain the deep reinforcement learning neural network, and taking the environmental state of the photovoltaic system in the current period as the input of the deep reinforcement learning neural network, and taking the MPPT adjustment action taken by the environmental state as the output of the deep reinforcement learning neural network, wherein the environmental state includes the temperature and light conditions of the photovoltaic cell assembly; S22. Initialization: Initialize the learning network Q(s, a; w) and the target network Q*(s, a; w') of the deep reinforcement learning neural network so that the initial target network Q*(s, a; w') is consistent with the initial learning network Q(s, a; w).

4. According to claim 3, a multi-scenario photovoltaic power generation situation awareness method based on RL and edge algorithm is characterized in that: Step S3 specifically includes the following steps: S31, read the status S through the MPPT controller t : S t =[T(t),Lux(t),U(t),I(t)] Where, T(t) is the temperature of the photovoltaic cell module during the period t; Lux(t) is the light intensity received by the photovoltaic cell module during the period t; I(t) is the voltage output by the photovoltaic cell module during the period t; I(t) is the current output by the photovoltaic cell module during the period t; S32, select action a through ε-greedy strategy according to the current learning network Q(s, a; w) t : In the formula, a represents action; S33. Calculate the reward rt obtained by the MPPT controller: r t =Δp In the formula, Δp is the increment of the output power of the photovoltaic power generation module; S34. Calculate the new state s for the next period according to step S31. t+1 ; The calculated reward r t , the new state s for the next period t+1 , the state s in period t t 、Action a t As a four-tuple (s t , a t , r t ,s t+1 ) is stored in the pre-set experience pool.

5. The multi-scenario photovoltaic power generation situation awareness method based on RL and edge algorithm according to claim 4 is characterized in that: In step S32, the duty cycle D(t) of the PWM drive signal of the DC / DC converter is adjusted to achieve MPPT regulation. Therefore, the action space of the MPPT controller is: A=[D(t)] Then, we select a set step size to discretize the action space, and then use the random gradient ascent method combined with the variable step size method to find the action a t The corresponding global optimal Q value.

6. The multi-scenario photovoltaic power generation situation awareness method based on RL and edge algorithm according to claim 5 is characterized by: Step S4 The specific steps include: S41, the new state s of the next period t+1 Considered as the new current state s t , repeat step S3 until the data in the experience pool exceeds the set amount, and then execute step S42; S42, randomly extract a four-tuple data sample from the experience pool, and give the reward r obtained by sampling t and the new state s t+1 Send to the target network to calculate the Q* value y t : Where, γ is the attenuation factor of the correlation between the next state and the current state of the MPPT controller; a t+1 is the action of the next period, w' is the parameter of the target network Q*; S43, y t Input into the learning network Q(s, a; w), and use the loss function to calculate the loss value; S44. Update the parameters of the learning network Q(s, a; w) based on stochastic gradient descent until the loss value is less than the set value, determine that the learning network Q(s, a; w) converges, and copy the learning network Q(s, a; w) value to the target network Q*(s, a; w'), and the deep reinforcement learning neural network completes convergence.

7. The multi-scenario photovoltaic power generation situation awareness method based on RL and edge algorithm according to claim 6 is characterized by: In step S43, the loss function L of the T period t (w t )The formula is as follows: In the formula, is the mathematical expectation, y is the optimization target value; And the optimization target value y calculation formula is as follows: In the formula, r is the reward, Q*(s t+1 , a t+1 ; w') is the new state s evaluated by the target network t+1 The maximum Q value of all actions; a t+1 It is the action of the next period.

8. The multi-scenario photovoltaic power generation situation awareness method based on RL and edge algorithm according to claim 6 is characterized by: In step S44, the parameters of the stochastic gradient descent learning network Q(s, a; w) are derived by taking the partial derivative of the parameter w:

9. The multi-scenario photovoltaic power generation situation awareness method based on RL and edge algorithm according to claim 8 is characterized by: Step S5 specifically includes the following steps: S51. Deploy the converged deep reinforcement learning neural network to the edge computing node: deploy the deep reinforcement learning neural network on the edge computing device located in the photovoltaic power station; S52, real-time data collection and preprocessing: collect real-time data of photovoltaic power generation components through the edge network and perform preprocessing; S53, Situational awareness and decision generation: Using the deep reinforcement learning neural network deployed on the edge computing node, the current operating status and environmental factors of photovoltaic power generation are continuously evaluated based on the real-time collected data, and the optimal MPPT adjustment action is output; S54, action execution and feedback loop: implement the optimal MPPT adjustment action recommended by the deep reinforcement learning neural network, monitor the results after the action is executed, and feed back the execution results to the deep reinforcement learning neural network; S55. Continuous optimization and iteration: Based on the collected feedback, the deep reinforcement learning neural network is regularly updated.

Images