Method for determining maximum power point of port photovoltaic system related to marine environment disturbance

Abstract

The invention relates to a method for determining the maximum power point of a port photovoltaic system related to marine environment disturbance, and belongs to the technical field of port district new energy application, and the method comprises the steps: employing an improved Q-learning reinforcement learning algorithm to track the maximum power point of the photovoltaic system, extracting the corresponding feature data, and updating and building a Q table knowledge base; training the optimal Q table matching model of the photovoltaic system by taking the Q table knowledge base as a training set to obtain a completely trained optimal Q table matching model of the photovoltaic system; acquiring feature data of the photovoltaic system during actual operation, and inputting the feature data into the completely trained optimal Q table matching model of the photovoltaic system to obtain an optimal Q table during actual operation; and determining the maximum power point during actual operation based on the optimal Q table. According to the method, the technical problems that the maximum power point tracking method of the port area photovoltaic system in the prior art is easy to fall into a local optimal value, the algorithm convergence time is long and the reaction speed is low are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of new energy application in port areas, and in particular to a method for determining a maximum power point of a port photovoltaic system involving marine environmental disturbance. Background Art

[0002] When building green ports, solar energy as a renewable energy source has always been the focus of research. For coastal port photovoltaic systems, the environment they are in is more complex than that of inland photovoltaic power stations that have been strictly sited. This is mainly manifested in the following aspects: on the one hand, buildings, warehouses, piled goods, and towering port machinery equipment in the port area will cause local and dynamic shading to the photovoltaic system; on the other hand, the high salinity of the air in coastal ports will cause asymmetric aging damage to photovoltaic modules. These problems will cause significant changes in the output characteristics of the port photovoltaic system, thereby increasing the difficulty of Maximum Power Point Tracking (MPPT).

[0003] At present, the maximum power point tracking method of the port photovoltaic system has the following main technical difficulties: first, it is easy to fall into the local optimal value. At present, there are few studies on the MPPT of the port photovoltaic system under the disturbance of the marine environment. If the traditional MPPT control method is used, it will be easy to fall into the local optimal value. It is difficult to adapt to the complex and changeable natural environment of the port area and cannot meet the control requirements of the port application scenario. Second, the convergence time is long and the response speed is slow. Generally, the Q-learning algorithm is used for MPPT, but this algorithm takes a lot of time in the exploration process, which leads to the inability of the port photovoltaic system to quickly find the optimal solution during actual operation, which makes the MPPT control time long and the rapidity poor. In addition, in order to suppress the problem of local shading, it is necessary to expand the action space on the basis of the traditional RL-MPPT, which will further increase the complexity of the algorithm search. Therefore, when using RL-MPPT for the port photovoltaic array, there will be problems such as long convergence time and slow response speed. Summary of the invention

[0004] In view of this, it is necessary to provide a method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbances, so as to solve the technical problems in the existing technology that the maximum power point tracking method of the port photovoltaic system is prone to fall into the local optimal value and the algorithm has a long convergence time and slow response speed.

[0005] In order to solve the above problems, the present invention provides a method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbance, comprising: The preset Q-learning reinforcement learning algorithm is used to track the maximum power point of the photovoltaic system and extract the corresponding characteristic data, and a Q table knowledge base is established based on the maximum power point and the corresponding characteristic data; The optimal Q-table matching model for the photovoltaic system is trained using the Q-table knowledge base as the training set, and a fully trained optimal Q-table matching model for the photovoltaic system is obtained; Acquire characteristic data of the photovoltaic system during actual operation, and input the characteristic data into a well-trained optimal Q-table matching model for the photovoltaic system to obtain the optimal Q-table during actual operation; Based on the optimal Q table, the maximum power point during actual operation is determined.

[0006] In a possible implementation, before establishing the Q table knowledge base, the following steps are included: All photovoltaic panels of the photovoltaic panel system are grouped according to the area where the photovoltaic panels are located, wherein multiple photovoltaic panels are grouped into full cross-connections; Based on the grouping of multiple photovoltaic panels, the maximum power point tracking method control cycle and the photovoltaic system optimal Q table matching model training cycle are initialized.

[0007] In a possible implementation, the characteristic data includes at least average light intensity, average temperature, maximum light intensity, maximum temperature, minimum light intensity and minimum temperature.

[0008] In a possible implementation, the use of a preset Q-learning reinforcement learning algorithm to track the maximum power point of the photovoltaic system and extract corresponding characteristic data includes: Define the state space, action space, and reward function of the preset Q-learning reinforcement learning algorithm; The position of the working point is characterized by the discrete working current output by the photovoltaic panel group and the state space is expressed; the action space is described by the change step of the working current, and the Q-learning reinforcement learning algorithm is constructed; The constructed Q-learning reinforcement learning algorithm is used to track the maximum power point of the photovoltaic system and extract the corresponding feature data In a possible implementation, the feature data of the photovoltaic system during actual operation is obtained and input into a well-trained optimal Q table matching model for the photovoltaic system to obtain the optimal Q table during actual operation, including: Obtain the actual characteristic data of the photovoltaic panel group under the current lighting conditions and construct an actual characteristic data set; Combining the actual feature data set with each feature data sample set in the Q table knowledge base to obtain multiple combined input parameter sets; Input multiple combined input parameter sets one by one into the well-trained optimal Q-table matching model of the photovoltaic system to obtain the corresponding multiple maximum power point absolute values; The sample Q corresponding to the minimum absolute value of multiple maximum power points is used as the optimal Q table during the actual operation of the photovoltaic panel.

[0009] In a possible implementation, determining the maximum power point during actual operation based on the optimal Q table includes: Based on the optimal Q-table, the constants are randomly generated according to the constructed Q-learning reinforcement learning algorithm, and the probability that the photovoltaic panel group can randomly select an action in the current state is defined; According to the relationship between the random constant and the probability, the action selection strategy of the photovoltaic panel based on the action space is determined; The action space of the photovoltaic panel is determined according to the action selection strategy, and the corresponding maximum power point is determined according to the action space.

[0010] In a possible implementation, after obtaining the optimal Q table during actual operation, the following steps are further included: Update the Q table according to the characteristic data of the actual operation of the photovoltaic system and the corresponding optimal Q table; The updated Q table is stored in the Q table knowledge base according to the preset time window mechanism; Based on the preset time window mechanism, determine the real-time nature of Q table updates; If the PV system reaches the MPPT control period but does not reach the Q-table matching model training period, a Q-table knowledge base is constructed; If the Q-table matching model training cycle is reached, the characteristic data of the photovoltaic system during actual operation is obtained and input into the fully trained optimal Q-table matching model of the photovoltaic system to obtain the optimal Q-table during actual operation.

[0011] In a second aspect, the present invention further provides a device for determining a maximum power point of a port photovoltaic system involving marine environmental disturbances, comprising: A knowledge base building module is used to track the maximum power point of the photovoltaic system and extract the corresponding characteristic data using a preset Q-learning reinforcement learning algorithm, and to establish a Q table knowledge base based on the maximum power point and the corresponding characteristic data; The model training module is used to train the optimal Q-table matching model of the photovoltaic system using the Q-table knowledge base as a training set to obtain a fully trained optimal Q-table matching model of the photovoltaic system; The optimal Q table determination module is used to obtain characteristic data of the photovoltaic system during actual operation, and input the characteristic data into a well-trained optimal Q table matching model for the photovoltaic system to obtain the optimal Q table during actual operation; The maximum power point determination module is used to determine the maximum power point during actual operation based on the optimal Q table.

[0012] In a third aspect, the present invention further provides an electronic device, comprising: a processor and a memory; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, the steps in the method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbances as described above are implemented.

[0013] In a fourth aspect, the present invention further provides a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement the steps in the method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbances as described above.

[0014] The beneficial effects of the present invention are as follows: first, the improved Q-learning reinforcement learning algorithm is used to track the maximum power point of the photovoltaic system and extract the corresponding characteristic data, a Q table is constructed and updated, and a Q table knowledge base is established; and multiple Q tables in the Q table knowledge base are used as training sets to train the BP neural network model to obtain the optimal Q table matching model of the photovoltaic system; finally, the characteristic data of the photovoltaic system during actual operation is obtained, and input into the optimal Q table matching model of the photovoltaic system to obtain the optimal Q table during actual operation; and based on the optimal Q table, the improved Q-learning reinforcement learning algorithm is used to determine the maximum power point during actual operation. And by adding a time window mechanism of appropriate length to correct the error caused by the timeliness of the data, the maximum power point tracking accuracy of the port photovoltaic system under the disturbance of the marine environment is improved. The present invention has the advantages of fast response, short convergence time and high accuracy, which can effectively ensure the efficient operation of the port photovoltaic system. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 In the method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbance provided by the present invention, PVgroup1 is based on the control result of RL-MPPT; Figure 2 A method flow chart of an embodiment of a method for determining a maximum power point of a port photovoltaic system involving marine environmental disturbances provided by the present invention; Figure 3 A schematic diagram of the grouping layout of the photovoltaic system in the port area in the method for determining the maximum power point of the photovoltaic system in the port involving the marine environment disturbance provided by the present invention; Figure 4 A schematic diagram of the full cross connection of PVgroup1 in the method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbance provided by the present invention; Figure 5 A schematic diagram of the installation layout of the light intensity sensor and temperature sensor of PVgroup 1 in the method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbances provided by the present invention; Figure 6A flow chart of an embodiment of step S204 in the method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbance provided by the present invention; Figure 7 In the method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbance provided by the present invention, the shading condition of PVgroup1; Figure 8 It is a schematic diagram of an embodiment of a device for determining the maximum power point of a port photovoltaic system involving marine environmental disturbance provided by the present invention; Fig. 9 It is a schematic diagram of the operating environment of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0016] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0017] The traditional RL-MPPT algorithm is divided into an exploration process and a utilization process during operation. The exploration process is to update a Q table, while the utilization process is to select actions based on the Q table to quickly achieve the goal. The exploration process takes a long time to update the Q table, so there will be problems such as long convergence time and slow response speed. Even the improved RL-MPPT still has this problem. In a specific implementation method, when the photovoltaic system uses the improved RL-MPPT algorithm, its control results are as follows: Figure 1 As shown in (a), the first 45 seconds are the exploration process. The solution proposed in this application is to remove the exploration process. Since the function of the Q-table is to allow the photovoltaic system to quickly reach the maximum power point current, if the maximum power point current of two photovoltaic systems under different lighting conditions is the same, the Q-table can be universal, so the photovoltaic system can directly match the corresponding Q-table for it during operation to eliminate the exploration process. Therefore, the solution to this problem is to first update the Q-table based on the improved RL-MPPT algorithm through the photovoltaic system, record the relevant data, and store the Q-table in the knowledge base. When the knowledge base reaches a certain number, it is combined in pairs to generate a new data group and sent to the BPNN model for training. The photovoltaic system can match the Q-table with a maximum power point current close to the knowledge base according to the collected data during the next actual operation, thus saving the time of the exploration process.

[0018] A specific embodiment of the present invention discloses a method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbances. Figure 2 ,include: S201, using a preset Q-learning reinforcement learning algorithm to track the maximum power point of the photovoltaic system and extract corresponding characteristic data, and establishing a Q table knowledge base according to the maximum power point and the corresponding characteristic data; S202, using the Q table knowledge base as a training set, training the optimal Q table matching model for the photovoltaic system to obtain a fully trained optimal Q table matching model for the photovoltaic system; S203, acquiring characteristic data of the photovoltaic system during actual operation, and inputting the characteristic data into a well-trained optimal Q table matching model for the photovoltaic system to obtain an optimal Q table during actual operation; S204: Determine the maximum power point during actual operation based on the optimal Q table.

[0019] In this embodiment, the improved Q-learning reinforcement learning algorithm is first used to track the maximum power point of the photovoltaic system and extract the corresponding feature data, construct and update the Q table, and establish a Q table knowledge base; and multiple Q tables in the Q table knowledge base are used as training sets to train the BP neural network model to obtain the optimal Q table matching model of the photovoltaic system; finally, the feature data of the photovoltaic system during actual operation is obtained, and input into the optimal Q table matching model of the photovoltaic system to obtain the optimal Q table during actual operation; and based on the optimal Q table, the improved Q-learning reinforcement learning algorithm is used to determine the maximum power point during actual operation. And by adding a time window mechanism of appropriate length to correct the error caused by the timeliness of the data, the maximum power point tracking accuracy of the port photovoltaic system under the disturbance of the marine environment is improved. The present invention has the advantages of fast response, short convergence time and high accuracy, which can effectively ensure the efficient operation of the port photovoltaic system.

[0020] It should be noted that the Q-table knowledge base is a database that stores various Q-tables and their corresponding sample labels for subsequent learning and optimization. Specifically, the Q-table is a data structure in reinforcement learning that is used to store the "Q value" of each state-action pair. These Q values ​​represent the expected reward of taking an action in a specific state. They include: State: Definition: The current environmental conditions or configuration of the system. For example, light intensity, temperature, current, voltage, etc.

[0021] Purpose: Describe the current state of the system to facilitate the selection of appropriate actions for optimization.

[0022] Action: Definition: Actions that can be taken in a specific state. For example, adjusting the angle of a photovoltaic panel, changing the current output, etc.

[0023] Function: Used to change the system state, thereby affecting the overall performance of the system.

[0024] Q-value: Definition: The expected cumulative reward of taking an action in a certain state.

[0025] Purpose: Helps the algorithm decide which action to choose in a particular state to maximize long-term rewards.

[0026] Each entry in the Q-table records the expected reward value of taking an action in a specific state. As the learning process proceeds, these Q-values ​​will be continuously updated to reflect more accurate expected rewards.

[0027] Sample labels are data related to the Q-table update process, which are used to record the specific performance of the state-action pair. Specifically, they include: The actual return of a state-action pair: Definition: The actual return or reward obtained by taking an action in a specific state.

[0028] Function: Used to update the Q value in the Q table to make it more accurately reflect the actual situation.

[0029] Status information: Definition: Current system status information, such as light intensity, temperature, etc.

[0030] Function: Helps understand the effect of actions in a specific state.

[0031] Action information: Definition: A specific action selected in a specific state.

[0032] Purpose: To record the measures taken so as to analyze their effects.

[0033] Reward: Definition: The actual reward value received, usually determined by the performance of the system (such as power output).

[0034] Function: Used to update the Q value to improve the quality of future decisions.

[0035] Furthermore, the optimal Q-table matching model for the photovoltaic system is a machine learning model, which may be a neural network model, a deep learning model or a support vector machine model, etc. In this embodiment, the optimal Q-table matching model for the photovoltaic system is a BP neural network model. The Q table in the Q-table knowledge base and the corresponding sample labels are used as training sets to train the BP neural network to obtain a fully trained BP neural network model. The optimal Q-table matching model for the photovoltaic system is predicted by the fully trained BP neural network model.

[0036] In some embodiments, when a photovoltaic system is established in a port area, all photovoltaic panels of the system are grouped. Specifically, the photovoltaic panels are grouped according to the areas where the photovoltaic panels are located, such as Figure 3 As shown; initialize the MPPT control cycle and the Q table matching model training cycle; arrange the photovoltaic panels according to the size of the physical space in the area where the photovoltaic panel group is located, and fully cross-connect them, as shown Figure 4 As shown; a DC / DC control circuit is connected to the road end of the photovoltaic panel group after full cross connection, and a small number of light intensity sensors and temperature sensors are placed in the installation area of ​​the photovoltaic panel group, such as Figure 5 shown.

[0037] In a specific embodiment, the port photovoltaic system is divided into g groups, and the grouped photovoltaic panels are recorded in order as PVgroup i ( i =1, 2, …, g), the number of photovoltaic panels in each group is recorded as M i ( i =1, 2, …, g); Place the photovoltaic panels in the form of n×m; First, connect the photovoltaic panels in series and parallel, and then connect the corresponding photovoltaic panels in different series branches horizontally, then n×m=M i Initialize the MPPT control period, denoted as TMPPT; Q table matching model training period TQM.

[0038] In some embodiments, the improved Q-learning reinforcement learning algorithm is used to track the maximum power point of the photovoltaic system and extract the corresponding characteristic data, and to update and establish a Q table knowledge base, including: Define the state space, action space and reward function of the improved Q-learning reinforcement learning algorithm; The position of the working point is characterized by the discrete working current output by the photovoltaic panel group and the state space is expressed; the action space is described by the change step of the working current.

[0039] In this embodiment, when the photovoltaic panel group performs MPPT based on Q-learning, the state space, action space and reward function are first defined.

[0040] The working current output by the photovoltaic panel group is discretized to describe the position of the working point, so the state space is expressed as:

[0041] in, I pv It is the working current output by the photovoltaic panel group.

[0042] The action is the change step of the working current, i.e. ΔI, and the action space is defined as:

[0043] in, a is the value of ΔI. a 1. a 2 and a 3 represents a negative change in the working current, and the change value decreases successively; a 5. a 6 and a 7 represents the positive change of the working current, and the change value increases successively; finally a 4 means no change is made and the original working current is maintained.

[0044] The reward function is defined as:

[0045]

[0046] in, P t+1 Indicates the power at the current moment; P t Indicates the power at the previous moment; set δ Eliminate the impact of measurement noise on the reward function, and its value is a very small positive number; ω p and ω n is the reward weight.

[0047] In some embodiments, the characteristic data includes at least average light intensity, average temperature, maximum light intensity, maximum temperature, minimum light intensity and minimum temperature.

[0048] Furthermore, the number of light intensity sensors and temperature sensors placed is: N; the light data collected by the i-th light intensity sensor is recorded as S i ( i =1, 2, …, N), the i The temperature data collected by the temperature sensor is recorded as T i ( i =1, 2, …,N).

[0049] In this embodiment, feature extraction is performed on the data collected by the photovoltaic panel group through the light intensity sensor and the temperature sensor, specifically:

[0050] in, S mean is the average light intensity; T mean is the average temperature; Smax is the maximum light intensity; T max is the maximum temperature; S min is the minimum light intensity; T min is the minimum temperature; Furthermore, each photovoltaic panel group finally operates at the maximum power point current, which is recorded as: I m ; The Q table and sample labels are stored in the Q table knowledge base, specifically: Set the Q table and sample label Sample-C i ={ S 1, S 2, ..., S N , T 1, T 2, ..., T N ，S mean , T mean , S max , T max , S min , T min | I m}( i =1, 2, …, K) are stored in the Q table knowledge base; The required number of Q tables in the knowledge base is denoted as: K; Construct a Q table matching sample library, specifically: Match each sample in the Q table knowledge base in pairs to get K 2 Q table difference samples, specifically: Sample- Q i ={ S 1A ，S 2A ，…，S NA ，T 1A ，T 2A ，…，T NA ，S mean-A ，T mean-A ，Smax-A ，T max-A ，S min-A ，T min-A ，S 1B ， S 2B ，…，S NB ，T 1B ，T 2B ，…，T NB ，S mean-B ，T mean-B ，S max-B ，T max-B ，S min-B ，T min-B | }, i=1, 2, …, K; ΔI Q =|I A - I B | in,{ S 1A ，S 2A ，…，S NA ，T 1A ，T 2A ，…，T NA ，S mean-A ，T mean-A ，S max-A ，T max-A ，S min-A ，T min-A} represents the light intensity and temperature data collected by the photovoltaic panel group under the A type of light condition and its characteristic parameters; { S 1B， S 2B ，…，S NB， T 1B ，T 2B ，…，T NB ，S mean-B ，T mean-B ，S max-B ，T max-B ，S min-B ，T min-B} represents the light intensity and temperature data and its characteristic parameters collected by the photovoltaic panel group under the B light condition; I A is the maximum power point current of the photovoltaic panel group under type A illumination condition; I B is the maximum power point current of the photovoltaic panel group under type B lighting conditions; Will K 2 The Q-table difference samples are stored in the Q-table matching sample library; Construct a Q-table matching model for the photovoltaic panel group and train it using the Q-table matching sample library. Specifically: The Q table matching model is constructed based on the BP neural network. The model input layer node is the input parameter part of the Q table difference sample, that is, S 1A ，S 2A ，…，S NA ，T 1A ，T 2A ，…，T NA ，S mean-A ，T mean-A ，S max-A ，T max-A ，S min-A ，T min-A ，S 1B ，S 2B ，…， S NB ，T 1B ，T 2B ，…，TNB ，S mean-B ，T mean-B ，S max-B ，T max-B ，S min-B ，T min-B , the model output layer node is the output parameter part of the Q table difference sample, that is Then, the Q table is used to match the K 2 The BP neural network model is trained with Q table difference samples; The PV panel group matching Q table under the current lighting conditions is as follows: The photovoltaic panel group collects light intensity and temperature data through sensors under the current light conditions, and extracts features to form a Q table matching sample Sample-Qcurrent under the current light conditions. Specifically: Sample-Qcurrent={ S 1C ，S 2C ，…，S NC ，T 1C ，T 2C ，…，T NC ，S mean-C ，T mean-C ，S max-C ，T max-C ，S min-C ，T min-C}; The Q table matching sample Sample-Qcurrent under the current lighting conditions is compared with each sample Sample-C in the knowledge base one by one. i ( i =1, 2, …, K) and send them one by one to the well-trained Q-table matching model for Q-table matching prediction to obtain the matching output with each sample Sample-Ci ( i =1, 2, …, K), will match the output The smallest sample Q table is used as the optimal Q table Qbest under the current lighting conditions.

[0051] It should be noted that the traditional RL-MPPT algorithm directly selects the action with the maximum Q value in the state during use, and the output curve of the photovoltaic panel group is multi-peak. If the photovoltaic panel group uses the traditional RL-MPPT, it is easy to fall into the local optimal value; the improved RL-MPPT algorithm allows the photovoltaic panel group to randomly select actions with a certain probability during use, thereby jumping out of the local optimal value. In some embodiments, the maximum power point is determined based on the optimal Q table during actual operation, please refer to Figure 6 ,include: S601, based on the optimal Q table, randomly generate constants according to the improved Q-learning reinforcement learning algorithm, and define the probability that the photovoltaic panel group can randomly select an action in the current state; S602: Determine an action selection strategy of the photovoltaic panel based on the action space according to the relationship between the random constant and the probability.

[0052] In this embodiment, during the use of the Q-learning algorithm, when the photovoltaic panel group selects an action in the current state, a constant c is first randomly generated, and then the probability that the photovoltaic panel group can randomly select an action is defined, that is, z (z∈[0,1]), specifically: If c ≤ z, the photovoltaic panel group can randomly select an action from the action space; if c>z, the photovoltaic panel group selects the action with the largest Q value in the current state according to the Q table.

[0053] In some embodiments, after obtaining the optimal Q table during actual operation, the method further includes: Update the Q table according to the characteristic data of the actual operation of the photovoltaic system and the corresponding optimal Q table; The updated Q table is stored in the Q table knowledge base according to the preset time window mechanism; Based on the preset time window mechanism, determine the real-time nature of Q table updates; If the PV system reaches the MPPT control period but does not reach the Q-table matching model training period, a Q-table knowledge base is constructed; If the Q-table matching model training cycle is reached, the characteristic data of the photovoltaic system during actual operation is obtained and input into the fully trained optimal Q-table matching model of the photovoltaic system to obtain the optimal Q-table during actual operation.

[0054] In this embodiment, in order to allow the system to find a balance between real-time optimization and long-term learning, the maximum power point tracking accuracy under marine environmental disturbances is improved. The system will periodically check whether the strategy needs to be adjusted to respond to environmental changes, and perform strategy optimization and data processing at the appropriate time. The error caused by the timeliness of the data is corrected by adding a time window mechanism of appropriate length. Specifically, when processing data, the freshness of the data is considered by establishing a time window. Since the photovoltaic system is affected by changes in instantaneous environmental factors (such as sunlight intensity, wind speed, temperature, etc.), old data may no longer be the current optimal reference. Therefore, the time window mechanism can help the system better process historical data and eliminate errors caused by data timeliness. The age window mechanism ensures that the latest and most relevant Q value is used to determine the next action.

[0055] In a specific embodiment, the illumination data collected by the sensor of the photovoltaic panel group under a certain illumination condition is 330W / m2, 310W / m2, 760W / m2, 890W / m2, 25℃, 24℃, 25℃ and 26℃, and the photovoltaic panel group updates the Q table based on the improved reinforcement learning MPPT under this illumination condition, and finally the photovoltaic panel group works at 24.5A. Finally, the Q table is combined with its sample label Sample-C1={330, 310, 760, 890, 25, 24, 25, 26, 572.5, 25, 890, 26, 310, 24 | 24.5} are stored in the Q-table knowledge base together; the illumination data collected by the sensor under a certain illumination condition by the photovoltaic panel group is 580W / m2, 340W / m2, 830W / m2, 570W / m2, 25℃, 24℃, 26℃ and 25℃, and the photovoltaic panel group updates the Q-table based on the improved reinforcement learning MPPT under this illumination condition, and finally the photovoltaic panel group works at 24.5A. Finally, this Q-table is stored in the Q-table knowledge base together with its sample label Sample-C2={580, 340, 830, 570, 25, 24, 26, 25, 580, 25, 580, 26, 340, 24 | 14}; As a preferred method, the required number of Q tables in the knowledge base is recorded as: 1000; As a preferred method, a Q table matching sample library is constructed, specifically: Pair each sample in the Q table knowledge base and get K 2 Q table difference samples, specifically: Sample-Q i ={ S 1A ，S 2A ，…，S NA ，T 1A ，T2A ，…，T NA ，S mean-A ，T mean-A ，S max-A ，T max-A ，S min-A ，T min-A ，S 1B ， S 2B ，…，S NB ，T 1B ，T 2B ，…，T NB ，S mean-B ，T mean-B ，S max-B ，T max-B ，S min-B ，T min-B |}, i=1, 2, …, K ; ΔI Q =|I A - I B | in,{ S 1A ，S 2A ，…，S NA ，T 1A ，T 2A ，…，T NA ，S mean-A ，T mean-A ，S max-A ，T max-A ，S min-A ，T min-A} indicates A type of lighting conditions PVgroup 1 The collected light intensity and temperature data and their characteristic parameters; S 1B ，S 2B ，…，S NB ， T 1B ，T 2B ，…，T NB ，S mean-B ，T mean-B ，S max-B ，T max-B ，S min-B ，T min-B} indicates that under B type lighting conditions PVgroup 1. The collected light intensity and temperature data and their characteristic parameters; I A Under A light condition PVgroup Maximum power point current of 1; I B Under B light conditions PVgroup Maximum power point current of 1; Store 1,000,000 Q-table difference samples into the Q-table matching sample library; For example, Sample-C 1 and Sample-C 1 is matched to obtain the difference sample, that is, Sample-Q1={330, 310, 760, 890, 25, 24, 25, 26, 572.5, 25, 890, 26, 310, 24, 580, 340, 830, 570, 25, 24, 26, 25, 580, 25, 580, 26, 340, 24 | 10.5}, and it is stored in the Q table difference sample library; The Q-table matching model of PVgroup1 was constructed based on BP neural network, and trained using the Q-table matching sample library; As a preferred method, a Q-table matching model of PVgroup1 is constructed and trained using a Q-table matching sample library, specifically: The Q table matching model is constructed based on the BP neural network. The model input layer node is the input parameter part of the Q table difference sample, that is, S 1A ，S 2A ，…，S NA，T 1A ，T 2A ，…，T NA ，S mean-A ，T mean-A ，S max-A ，T max-A ，S min-A ，T min-A ，S 1B ，S 2B ，…， S NB ，T 1B ，T 2B ，…，T NB ，S mean-B ，T mean-B ，S max-B ，T max-B ，S min-B ，T min-B , the model output layer node is the output parameter part of the Q table difference sample, that is. For example, Sample-Q1's 330, 310, 760, 890, 25, 24, 25, 26, 572.5, 25, 890, 26, 310, 24, 580, 340, 830, 570, 25, 24, 26, 25, 580, 25, 580, 26, 340, 24 is used as the model input parameter, and 10.5 is the model output parameter; then, the BP neural network model is trained using 1,000,000 Q table difference samples in the Q table matching sample library; PVgroup1 collects light intensity and temperature data through sensors under current lighting conditions, and extracts features to form a Q-table matching sample Sample-Qcurrent under current lighting conditions. Specifically, Sample-Qcurrent={300, 300, 800, 1000, 24, 24, 25, 26, 600, 24.75, 600, 26, 300, 24}. The current lighting conditions are as follows: Figure 7 As shown; The Q table matching sample Sample-Qcurrent under the current lighting condition is compared with each sample Sample-C in the knowledge base one by one. i (i=1, 2, …, K) are matched and sent to the well-trained Q-table matching model one by one for Q-table matching prediction to obtain the i The matching output of (i=1, 2, …, K) is used, and the sample Q table with the smallest matching output is taken as the optimal Q table Qbest under the current lighting conditions; For example, Sample-Q current They are matched with Sample-C1 one by one, specifically = {300, 300, 800, 1000, 24, 24, 25, 26, 600, 24.75, 600, 26, 300, 24, 330, 310, 760, 890, 25, 24, 25, 26, 572.5, 25, 890, 26, 310, 24}, and then sent to the well-trained Q-table matching model for Q-table matching prediction; PVgroup1 starts the process of using the reinforcement learning MPPT based on the matched optimal Q table (Qbest replaces the Q table generated by the exploration process of the reinforcement learning MPPT), and continues to update the Q value based on this table; after a certain period of time, the process of using PVgroup1 is suspended, and the current with the maximum output power during operation is transmitted to the DC / DC circuit controller at the end of PVgroup1, and the working point of PVgroup1 is directly driven to this point in one step; As a preferred method, the use of PVgroup1 is suspended after a certain period of time, and the time is 2s; As a preference, the current at which the output power of PVgroup1 is the maximum during operation is 26.5A; The updated Q table is stored in the Q table knowledge base according to the time window mechanism; The updated Q table is stored in the Q table knowledge base according to the time window mechanism, specifically: The current environmental condition parameters {300, 300, 800, 1000, 24, 24, 25, 26, 600, 24.75, 600, 26, 300, 24 | 26.5} and the updated Q table are stored in the Q table knowledge base; if the time window length is exceeded, the earliest Q table is deleted according to the time sequence in which the Q table is stored in the knowledge base; if the MPPT control period is reached and the Q table matching model training period is not reached, jump to continue training.

[0056] Based on the above-mentioned method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbance, an embodiment of the present invention further provides a device for determining the maximum power point of a port photovoltaic system involving marine environmental disturbance, see Figure 8 ,include: The knowledge base building module 810 is used to track the maximum power point of the photovoltaic system and extract the corresponding characteristic data by using a preset Q-learning reinforcement learning algorithm, and to establish a Q table knowledge base according to the maximum power point and the corresponding characteristic data; The model training module 820 is used to train the optimal Q-table matching model of the photovoltaic system using the Q-table knowledge base as a training set to obtain a fully trained optimal Q-table matching model of the photovoltaic system; The optimal Q table determination module 830 is used to obtain characteristic data of the photovoltaic system during actual operation, and input the characteristic data into a well-trained photovoltaic system optimal Q table matching model to obtain the optimal Q table during actual operation; The maximum power point determination module 840 is used to determine the maximum power point in actual operation based on the optimal Q table. like Fig. 9 As shown, based on the above-mentioned method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbance, the present invention also provides an electronic device, which can be a computing electronic device such as a mobile terminal, a desktop computer, a notebook, a palm computer, and a server. The electronic device includes a processor 910, a memory 920, and a display 930. Fig. 9 Only some components of the electronic device are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0057] In some embodiments, the memory 920 may be an internal storage unit of the electronic device, such as a hard disk or memory of the electronic device. In other embodiments, the memory 920 may also be an external storage electronic device of the electronic device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory 920 may also include both an internal storage unit of the electronic device and an external storage electronic device. The memory 920 is used to store application software and various types of data installed in the electronic device, such as program codes installed in the electronic device. The memory 920 may also be used to temporarily store data that has been output or is to be output. In one embodiment, the memory 920 stores a maximum power point determination program 940 for a port photovoltaic system involving marine environmental disturbances, and the maximum power point determination program 940 for a port photovoltaic system involving marine environmental disturbances can be executed by the processor 910, thereby realizing the maximum power point determination method for a port photovoltaic system involving marine environmental disturbances in each embodiment of the present application.

[0058] In some embodiments, the processor 910 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 920, such as executing a method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbances.

[0059] In some embodiments, the display 930 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 930 is used to display information of the electronic device for determining the maximum power point of the port photovoltaic system involving marine environmental disturbances and to display a visual user interface. The components 910-930 of the electronic device communicate with each other via a system bus.

[0060] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0061] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbances, characterized in that: include: The preset Q-learning reinforcement learning algorithm is used to track the maximum power point of the photovoltaic system and extract the corresponding characteristic data, and a Q table knowledge base is established based on the maximum power point and the corresponding characteristic data; The optimal Q-table matching model for the photovoltaic system is trained using the Q-table knowledge base as the training set, and a fully trained optimal Q-table matching model for the photovoltaic system is obtained; Acquire characteristic data of the photovoltaic system during actual operation, and input the characteristic data into a well-trained optimal Q-table matching model for the photovoltaic system to obtain the optimal Q-table during actual operation; Based on the optimal Q table, the maximum power point during actual operation is determined.

2. The method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbance according to claim 1, characterized in that: Before building the Q table knowledge base, include: All photovoltaic panels of the photovoltaic panel system are grouped according to the area where the photovoltaic panels are located, wherein multiple photovoltaic panels are grouped into full cross-connections; Based on the grouping of multiple photovoltaic panels, the maximum power point tracking method control cycle and the photovoltaic system optimal Q table matching model training cycle are initialized.

3. The method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbance according to claim 1, characterized in that: The characteristic data at least include average light intensity, average temperature, maximum light intensity, maximum temperature, minimum light intensity and minimum temperature.

4. The method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbance according to claim 3, characterized in that: The method of using a preset Q-learning reinforcement learning algorithm to track the maximum power point of the photovoltaic system and extract corresponding characteristic data includes: Define the state space, action space, and reward function of the preset Q-learning reinforcement learning algorithm; The position of the working point is characterized by the discrete working current output by the photovoltaic panel group and the state space is expressed; the action space is described by the change step of the working current, and the Q-learning reinforcement learning algorithm is constructed; The constructed Q-learning reinforcement learning algorithm is used to track the maximum power point of the photovoltaic system and extract the corresponding feature data.

5. The method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbance according to claim 1, characterized in that: The method of obtaining characteristic data of the photovoltaic system during actual operation and inputting the characteristic data into a well-trained optimal Q table matching model of the photovoltaic system to obtain the optimal Q table during actual operation includes: Obtain the actual characteristic data of the photovoltaic panel group under the current lighting conditions and construct an actual characteristic data set; Combining the actual feature data set with each feature data sample set in the Q table knowledge base to obtain multiple combined input parameter sets; Input multiple combined input parameter sets one by one into the well-trained optimal Q-table matching model of the photovoltaic system to obtain the corresponding multiple maximum power point absolute values; The sample Q corresponding to the minimum absolute value of multiple maximum power points is used as the optimal Q table during the actual operation of the photovoltaic panel.

6. The method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbance according to claim 3, characterized in that: Determining the maximum power point during actual operation based on the optimal Q table includes: Based on the optimal Q-table, the constants are randomly generated according to the constructed Q-learning reinforcement learning algorithm, and the probability that the photovoltaic panel group can randomly select an action in the current state is defined; According to the relationship between the random constant and the probability, the action selection strategy of the photovoltaic panel based on the action space is determined; The action space of the photovoltaic panel is determined according to the action selection strategy, and the corresponding maximum power point is determined according to the action space.

7. The method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbance according to claim 3, characterized in that: After obtaining the optimal Q table for actual runtime, it also includes: Update the Q table according to the characteristic data of the actual operation of the photovoltaic system and the corresponding optimal Q table; The updated Q table is stored in the Q table knowledge base according to the preset time window mechanism; Based on the preset time window mechanism, determine the real-time nature of Q table updates; If the PV system reaches the MPPT control period but does not reach the Q-table matching model training period, a Q-table knowledge base is constructed; If the Q-table matching model training cycle is reached, the characteristic data of the photovoltaic system during actual operation is obtained and input into the fully trained optimal Q-table matching model of the photovoltaic system to obtain the optimal Q-table during actual operation.

8. A device for determining the maximum power point of a port photovoltaic system involving marine environmental disturbances, characterized in that: include: A knowledge base building module is used to track the maximum power point of the photovoltaic system and extract the corresponding characteristic data using a preset Q-learning reinforcement learning algorithm, and to establish a Q table knowledge base based on the maximum power point and the corresponding characteristic data; The model training module is used to train the optimal Q-table matching model of the photovoltaic system using the Q-table knowledge base as a training set to obtain a fully trained optimal Q-table matching model of the photovoltaic system; The optimal Q table determination module is used to obtain characteristic data of the photovoltaic system during actual operation, and input the characteristic data into a well-trained optimal Q table matching model for the photovoltaic system to obtain the optimal Q table during actual operation; The maximum power point determination module is used to determine the maximum power point during actual operation based on the optimal Q table.

9. An electronic device, characterized in that: include: Processor and memory; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, the steps in the method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbances as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the method for determining the maximum power point of a port photovoltaic system involving marine environmental disturbances as described in any one of claims 1-7.

Images