Photovoltaic energy storage control method and system based on intelligent learning and storage medium

Abstract

The invention relates to the technical field of photovoltaic energy storage, in particular to a photovoltaic energy storage control method and system based on intelligent learning and a storage medium, and the method comprises the steps: obtaining environment information, and generating an exploration stage containing an exploration rate and a utilization stage containing a utilization rate based on the environment information; state information of the distributed photovoltaic clusters is acquired, and an initial return value is configured for each photovoltaic cluster based on the state information; obtaining energy storage information, and generating a demand value on a unit time length based on the energy storage information; associating the demand value with the initial return value to judge and optimize balance logic and generate optimized return values corresponding to different behaviors; obtaining an energy storage strategy type, and selecting a reward and control scheme based on the energy storage strategy type; and generating a selection task containing the number of attempts and actions, and selecting a control scheme with the highest average reward in the selection task through an exploration stage and a utilization stage. The method has the effect of selecting the optimal photovoltaic energy storage control scheme in complex and uncertain weather.

Description

Technical Field

[0001] The present application relates to the technical field of photovoltaic energy storage, and in particular to a photovoltaic energy storage control method, system and storage medium based on intelligent learning. Background Art

[0002] Photovoltaic energy storage refers to a technology that combines photovoltaic power generation systems with energy storage devices to achieve energy storage and regulation. With the rapid development of renewable energy, photovoltaic power generation has been widely used around the world as a clean and renewable energy form. Photovoltaic energy storage systems are mainly composed of photovoltaic power generation modules, energy storage devices (such as batteries) and control systems. Photovoltaic power generation modules convert sunlight into electrical energy, energy storage devices are responsible for storing excess electrical energy and releasing it when needed, and control systems are responsible for regulating the energy flow between photovoltaic power generation and energy storage devices to ensure the stable operation of the system.

[0003] At present, in order to further optimize the energy planning of photovoltaic energy storage systems and reduce energy waste while providing sufficient and timely energy supply, each photovoltaic energy storage system will be equipped with a certain charging and discharging decision-making plan to achieve the minimum energy consumption. However, general decision-making plans are ideal static plans. Photovoltaics are affected by their own characteristics, and their power generation will be affected by weather, temperature, dust, etc. Its power generation itself is a highly dynamic and highly uncertain object. The use of more idealized decision-making plans cannot be accurately and ideally carried out on the basis of highly random photovoltaic power generation, which sometimes leads to a situation where the charging and discharging capacity is in short supply or in excess of demand, which still leads to energy waste and untimely supply. Summary of the invention

[0004] In order to select the optimal photovoltaic energy storage control solution in complex and uncertain weather, the present application provides a photovoltaic energy storage control method, system and storage medium based on intelligent learning.

[0005] In a first aspect, the present application provides a photovoltaic energy storage control method based on intelligent learning, which adopts the following technical solutions: A photovoltaic energy storage control method based on intelligent learning includes the following steps: Acquire environmental information, and generate an exploration phase including a corresponding exploration rate and a utilization phase including a corresponding utilization rate based on the environmental information; Acquire status information of distributed photovoltaic clusters, and configure an initial reward value for each photovoltaic cluster based on the status information; Acquire energy storage information, and generate a demand value per unit time based on the energy storage information; Associating the demand value with the initial reward value to determine an optimization balance logic, and generating optimization reward values ​​corresponding to different behaviors based on the optimization balance logic; Acquire the energy storage strategy type, and based on the energy storage strategy type, select the initial reward value or the optimized reward value as a reward and generate a corresponding control scheme; A selection task including the number of attempts and actions is generated, and in the selection task, the control scheme with the highest average reward is selected according to the number of times the action is triggered in the exploration phase and the utilization phase and the reward obtained by the action.

[0006] In some embodiments, the environmental information includes weather, dust volume, temperature, and humidity. Acquiring the environmental information and generating an exploration phase including a corresponding exploration rate and a utilization phase including a corresponding utilization rate based on the environmental information comprises the following steps: Calculating a power generation efficiency score based on the environmental information; An exploration rate ε inversely proportional to the power generation efficiency score is generated, and a utilization rate 1-ε is generated based on the exploration rate ε.

[0007] In some embodiments, obtaining status information of distributed photovoltaic clusters and configuring an initial reward value for each photovoltaic cluster based on the status information includes the following steps: Acquire the installation status of each photovoltaic device and record it as first information, wherein the installation status includes an installation horizontal angle, an installation pitch angle, and an installation height; Obtaining the basic status of each photovoltaic device and recording it as second information, wherein the basic status includes capacity, number of photovoltaic panels, type of photovoltaic panels, and efficiency of photovoltaic panels; integrating a plurality of the photovoltaic devices having the same first information and the same second information into a photovoltaic cluster; Acquire the current unit power generation of each of the photovoltaic clusters, determine the contribution value of the current unit power generation and the average adjustment step length of each photovoltaic cluster corresponding to other photovoltaic clusters to generate third information; An initial reward value is configured for each of the photovoltaic clusters according to the third information.

[0008] In some embodiments, obtaining energy storage information and generating a demand value per unit time based on the energy storage information includes the following steps: Acquire the current storage capacity of the energy storage device and the current power consumption of the power consumption device to jointly constitute the energy storage information; The unit coefficient of the preset time length is generated, the required electric energy per unit time length is calculated by the unit coefficient, and a positively correlated demand value is generated based on the required electric energy.

[0009] In some embodiments, the demand value and the initial reward value are associated to determine the optimization balance logic, and the optimization reward values ​​corresponding to different behaviors are generated based on the optimization balance logic, including the following steps: generating a plurality of simulation tasks including random numbers, and adding the random numbers to the demand values ​​in each simulation task to generate demand value variables; Traversing the basic reward value and determining whether it matches the demand value variable; If so, the classification cluster corresponding to the basic reward value increases the positive matching count by one; If not, the classification cluster corresponding to the basic reward value increases the negative matching count by one; When all the simulation tasks are completed, the counts in each classification cluster are counted and the optimization reward value is calculated based on the following formula: S'= , Among them, S represents the basic return value, S' represents the optimized return value, n1 represents the positive matching count, and n2 represents the negative matching count. It is characterized by a preset ratio, and Step is characterized by the step length required for the basic reward value to be adjusted from not matching the demand value variable to matching the demand value variable.

[0010] In some embodiments, obtaining the energy storage strategy type, and selecting the initial reward value or the optimized reward value as a reward based on the energy storage strategy type and generating a corresponding control scheme include the following steps: The energy storage strategy types include unilateral strategy and bilateral strategy. In the unilateral strategy, the demand value is used as determined demand data. In the bilateral strategy, the demand value variable with random tendency is used as uncertain demand data. The random tendency is characterized by the desire to increase or reduce the storage energy as a tendency to change the size of the demand value. In the one-sided strategy, the initial reward value is used as the reward and a control scheme for charging the photovoltaic cluster is generated. In the two-sided strategy, the optimized reward value is used as the reward and a control scheme for charging the photovoltaic cluster and discharging the energy storage device is generated.

[0011] In some of the embodiments, in the one-sided strategy, generating a selection task including the number of attempts and actions, and selecting the control scheme with the highest average reward in the selection task by the number of times the action is triggered in the exploration phase and the utilization phase and the reward obtained by the action, comprises the following steps: Generate a random value between 0 and 1; When the random amount corresponds to the exploration rate, it corresponds to the exploration phase, in which one of the photovoltaic clusters is randomly selected based on the determined demand data and a reward is obtained; When the random amount corresponds to the utilization rate, it corresponds to the utilization stage, and in the utilization stage, the photovoltaic cluster with the largest reward is selected based on the determined demand data and the reward is obtained; When all the attempts are completed, the accumulated rewards and trigger times in each classification cluster are obtained to calculate the average reward; The classification cluster with the highest average reward is taken as the optimal state, and is issued as the control scheme to control the other classification clusters to perform charging adjustments.

[0012] In some of the embodiments, in the two-sided strategy, a selection task including the number of attempts and actions is generated, and in the selection task, the control scheme with the highest average reward is selected by the number of times the action is triggered in the exploration phase and the utilization phase and the reward obtained by the action, and the following steps are also included: Generate a random value between 0 and 1; When the random amount corresponds to the exploration rate, it corresponds to the exploration stage, in which one of the photovoltaic clusters and one of the uncertain demand data are randomly selected to obtain a corresponding reward; When the random amount corresponds to the utilization rate, it corresponds to the utilization stage, and in the utilization stage, the photovoltaic cluster and the uncertain demand data with the largest reward are selected and corresponding rewards are obtained; When all the attempts are completed, the rewards and trigger times accumulated in each of the classification clusters and the uncertain demand data are obtained to calculate an average reward; The classification cluster with the highest average reward and the uncertain demand data are taken as the optimal state and issued as the control scheme to control the other classification clusters to perform charging adjustment and control the energy storage device to perform discharging adjustment.

[0013] In the second aspect, the present application provides a photovoltaic energy storage control system based on intelligent learning, which adopts the following technical solutions: A photovoltaic energy storage control system based on intelligent learning is used to implement the above method.

[0014] In a third aspect, the present application provides a storage medium, which adopts the following technical solution: A storage medium stores at least one instruction, at least one program, a code set or an instruction set, wherein the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the above method.

[0015] The technical solution provided by the embodiments of the present application has the following technical effects: First, based on the specific content of environmental information, the exploration rate and utilization rate are dynamically generated, and the return value that each photovoltaic cluster can provide is generated in combination with the static characteristics of each photovoltaic cluster. At the same time, the degree of electricity demand is calculated according to the static characteristics of the energy storage side, and the electricity consumption information on the energy storage side and the charging information on the photovoltaic side are associated to dynamically generate the balance logic of the two. At the same time, the dynamic optimization return value is generated through the balance logic. Finally, the corresponding control scheme is selected according to the required energy storage strategy, and the objects of each control scheme are selected based on the exploration rate and utilization rate. Finally, the object with the highest average reward is output as the basis of the control scheme for distribution. By converting unstable weather content into exploration rates of different sizes, high randomness is converted into highly stable task parameters. Combined with different energy storage control strategies, comprehensive selection tasks are carried out to achieve a balance between charging and discharging on both sides. Reinforcement learning is used to formalize strategies under uncertainty to improve the wisdom and accuracy of intelligent agent decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the steps of a photovoltaic energy storage control method based on intelligent learning provided in this embodiment. DETAILED DESCRIPTION

[0017] To more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. However, it should be understood by those of ordinary skill in the art that the present application can be implemented without these details. In some cases, in order to avoid unnecessary descriptions that make various aspects of the present application obscure, well-known methods, processes, systems, components and / or circuits that have been described at a higher level will not be described in detail. For those of ordinary skill in the art, it is obvious that various changes can be made to the embodiments disclosed in the present application, and without departing from the principles and scope of the present application, the general principles defined in the present application can be applied to other embodiments and application scenarios. Therefore, the present application is not limited to the embodiments shown, but conforms to the broadest scope consistent with the scope claimed for protection of the present application.

[0018] It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention. In addition, the technical features involved in each embodiment of the present invention described below can be combined with each other as long as there is no conflict between them.

[0019] In the description of this application, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used to distinguish the technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0020] In the description of the present application, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples.

[0021] like Figure 1 As shown, the embodiment of the present application discloses a photovoltaic energy storage control method based on intelligent learning, comprising the following steps: S100, acquiring environmental information, and generating an exploration phase including a corresponding exploration rate and a utilization phase including a corresponding utilization rate based on the environmental information.

[0022] Environmental information is represented by the corresponding environmental data of the photovoltaic panels in the week, such as weather, temperature and humidity, visibility, etc.

[0023] The same photovoltaic panels in different locations have different power generation and efficiency. In addition to their own parameters, the environment is the most important factor affecting the performance of photovoltaic equipment, and it is also an uncontrollable factor in the prediction, adjustment and optimization of photovoltaic panels.

[0024] Exploration and utilization are actions required in the strategy selection of machine learning. Exploration is characterized by randomly selecting an action or object, and utilization is characterized by using existing knowledge or experience to select the best action or best object currently known.

[0025] The exploration rate and utilization rate represent the probability of the agent choosing to explore or exploit in a certain number of choices. Different numerical exploration rates and utilization rates will make the agent weigh between trying new actions and choosing the best known action. When taking corresponding actions, the agent is in the exploration stage or the utilization stage.

[0026] Among them, the size of the exploration rate determines the behavioral decision-making of the intelligent agent. If the exploration rate is set too small, the intelligent agent will rely too much on the current knowledge and miss the opportunity to explore new strategies; if the exploration rate is set too large, the intelligent agent will frequently perform random exploration, resulting in performance degradation.

[0027] Therefore, different weather conditions will cause different photovoltaic panels to have a decline in performance. By dynamically adjusting the exploration rate using changes in weather information, the agent can use the optimal behavioral strategy to perform subsequent steps in both complex and good weather.

[0028] S200, obtaining status information of distributed photovoltaic clusters, and configuring an initial reward value for each photovoltaic cluster based on the status information.

[0029] A photovoltaic cluster is defined as a collection of photovoltaic panels with the same status and performance. The photovoltaic panels in a region are often of relatively dispersed distributed design, so the corresponding photovoltaic cluster is also of distributed design.

[0030] The initial reward value is configured for each photovoltaic cluster through the current state information of each photovoltaic cluster. The initial reward value is represented by the initial reward size configured for each photovoltaic cluster's static performance, which is used to give rewards for corresponding actions when the subsequent intelligent agent selects actions.

[0031] S300: Acquire energy storage information, and generate a demand value per unit time based on the energy storage information.

[0032] The energy storage information is represented by the overall electric energy utilization information on the other side of photovoltaic charging. The demand value per unit time length is generated by the electricity consumption information matched by photovoltaic charging. The demand value is matched with the return value, which is used to evaluate whether the relationship between charging and electricity consumption in the photovoltaic energy storage system is balanced and corresponding.

[0033] S400, associating the demand value with the initial reward value to determine the optimization balance logic, and generating optimization reward values ​​corresponding to different behaviors based on the optimization balance logic.

[0034] First, the demand value and the initial reward value are associated to determine the numerical relationship between the two values, and the optimization balance logic is determined by the numerical relationship. The optimization balance logic represents the action logic and action range required to balance the reward value and the demand value.

[0035] The corresponding optimized return value is generated by considering the optimized balancing logic. The optimized return value is different from the initial return value. The initial return value is the return value corresponding to when only the photovoltaic side is considered. It is used to make the photovoltaic charging approach the required energy storage power consumption side, while the optimized return value is used when both the photovoltaic charging side and the energy storage power consumption side change dynamically and require interaction between the two to make the two sides approach the required balance.

[0036] S500, obtaining an energy storage strategy type, and selecting an initial reward value or an optimized reward value as a reward based on the energy storage strategy type and generating a corresponding control scheme.

[0037] Obtain the energy storage method required or needed by the current management party, and different energy storage methods correspond to different energy storage strategy types.

[0038] The energy storage strategy type is used to select whether to use the initial return value as the reward or the optimized return value as the reward. At the same time, the corresponding rewards are combined according to different energy storage strategy types to generate corresponding control schemes.

[0039] S600, generating a selection task including the number of attempts and actions, and selecting a control scheme with the highest average reward in the selection task by using the number of times the actions are triggered in the exploration phase and the utilization phase and the rewards obtained by the actions.

[0040] Generate a selection task, which includes a certain number of attempts, and each attempt corresponds to an action.

[0041] The action is triggered in the exploration phase or the utilization phase to obtain the corresponding reward for the object corresponding to the action, and the average reward of each object is calculated by selecting the number of times each object is triggered by the action after the task is completed and the accumulated reward size, and the object with the highest average reward is output as a control solution.

[0042] Through the above scheme, the exploration rate and utilization rate are first dynamically generated based on the specific content of the environmental information, and the return value that each photovoltaic cluster can provide is generated in combination with the static characteristics of each photovoltaic cluster. At the same time, the power demand degree is calculated according to the static characteristics of the energy storage side, and the power consumption information on the energy storage side and the charging information on the photovoltaic side are associated to dynamically generate the balance logic of the two. At the same time, the dynamic optimization return value is generated through the balance logic. Finally, the corresponding control scheme is selected according to the required energy storage strategy, and the objects of each control scheme are selected based on the exploration rate and utilization rate. Finally, the object with the highest average reward is output as the basis of the control scheme for distribution. By converting unstable weather content into exploration rates of different sizes, high randomness is converted into highly stable task parameters. Different energy storage control strategies are combined to comprehensively select tasks so that both sides can achieve a balance between charging and discharging. Reinforcement learning is used to formalize strategies under uncertainty to improve the wisdom and accuracy of intelligent agent decision-making.

[0043] In some other embodiments, obtaining environmental information and generating an exploration phase including a corresponding exploration rate and a utilization phase including a corresponding utilization rate based on the environmental information includes the following steps: Environmental information includes weather, dust level, temperature, humidity, and may also include season, air quality, geographic location, etc.

[0044] S110, calculating a power generation efficiency score based on the environmental information.

[0045] Due to the characteristics of photovoltaics, the degree to which it obtains solar radiation determines its overall power generation effect.

[0046] When it is cloudy, dusty, hazy, or the air quality is poor, the solar radiation intensity is low, and its power generation efficiency is low; when the temperature is high, the temperature of the photovoltaic cells rises, causing the cell efficiency to decrease and thus reducing the power generation efficiency; when the humidity is high, water droplets will condense on the surface of the photovoltaic panels to affect the light absorption efficiency of the photovoltaic panels; and different latitudes and terrains determine the difficulty and distance for photovoltaic panels to receive sunlight; and different seasons determine the amount of sunlight.

[0047] Therefore, various types of environmental information are obtained and the power generation efficiency is scored according to the difficulty of receiving light.

[0048] S120 , generating an exploration rate ε that is inversely proportional to the power generation efficiency score, and generating a utilization rate 1-ε based on the exploration rate ε.

[0049] The lower the power generation efficiency score, the higher the exploration rate and the lower the utilization rate. Conversely, when the power generation efficiency score is higher, the exploration rate is lower and the utilization rate is higher.

[0050] This can increase the probability of exploration when environmental factors become complex and most photovoltaic devices cannot effectively obtain solar radiation, so that subsequent selection tasks can be carried out more through exploration to find the optimal solution to avoid local optimality. Conversely, when the environment is good and most photovoltaic devices can effectively obtain solar radiation, more actions can be used to repeatedly find the optimal action.

[0051] In some other embodiments, obtaining status information of distributed photovoltaic clusters and configuring an initial reward value for each photovoltaic cluster based on the status information includes the following steps: S210, obtaining the installation status of each photovoltaic device and recording it as first information, where the installation status includes an installation horizontal angle, an installation pitch angle, and an installation height.

[0052] The installation status information of each photovoltaic panel is generated by using the installation angle, height and other information of the photovoltaic device, and the status is defined as the first information.

[0053] S220, obtaining the basic status of each photovoltaic device and recording it as the second information, where the basic status includes capacity, number of photovoltaic panels, type of photovoltaic panels, and efficiency of photovoltaic panels.

[0054] The second information is generated by using basic equipment information of photovoltaic panels such as capacity, type, current and voltage of each photovoltaic device.

[0055] S230: Integrate a plurality of photovoltaic devices having the same first information and second information into a photovoltaic cluster.

[0056] Photovoltaic panels in the same static state are selected through the first information and the second information. Since these photovoltaic panels have the same installation state and the same equipment foundation state, the power generation that can be provided in the same sunlight environment is the same or extremely similar. Then these photovoltaic devices are integrated into the same photovoltaic cluster, so that several photovoltaic devices can be classified into several photovoltaic clusters.

[0057] S240, obtaining the current unit power generation of each photovoltaic cluster, determining the contribution value of the current unit power generation and the average adjustment step length of each photovoltaic cluster corresponding to other photovoltaic clusters to generate third information.

[0058] First, the current unit power generation of each photovoltaic cluster is obtained. The unit power generation is characterized by the average power generation that can be generated by each photovoltaic device in the photovoltaic cluster.

[0059] Different average power generation corresponds to different contribution values. The contribution value represents the amount of electricity contributed by each photovoltaic panel to the current energy storage side. Generally speaking, the greater the unit power generation of the photovoltaic equipment, the higher the corresponding contribution value.

[0060] Secondly, it is also necessary to determine the average adjustment step required for each photovoltaic cluster to change to the unit power generation corresponding to other photovoltaic clusters. The adjustment step is characterized by the adjustment amplitude required to adjust the installation state and basic state of the photovoltaic equipment to another state. A step corresponds to a different adjustment amplitude in different adjustment objects. For example, in angle adjustment, a step can be 5 degrees to 10 degrees; in current adjustment, a step can be 1A or 2A; in power adjustment, a step can be 5% to 10%.

[0061] Then, the total adjustment step required for adjusting the photovoltaic equipment in one photovoltaic cluster to the photovoltaic equipment in all other photovoltaic clusters is determined, and then the average adjustment step corresponding to each photovoltaic cluster can be calculated based on the number of photovoltaic equipment in the photovoltaic cluster.

[0062] The average adjustment step can characterize the adjustment difficulty of a photovoltaic device when an unknown target is arbitrarily selected when adjustment is required. The larger the average adjustment step, the higher the adjustment difficulty, and the smaller the average adjustment step, the lower the adjustment difficulty.

[0063] S250: configuring an initial reward value for each photovoltaic cluster according to the third information.

[0064] The initial value is configured for each photovoltaic cluster by the contribution degree and adjustment difficulty of each photovoltaic cluster. The initial return value of all photovoltaic devices in a photovoltaic cluster is the same.

[0065] In other embodiments, obtaining energy storage information and generating a demand value per unit time based on the energy storage information includes the following steps: S310, obtaining the current storage capacity of the energy storage device and the current power consumption of the power consumption device to jointly constitute energy storage information.

[0066] Energy storage information includes two parts: the amount of stored electrical energy in the energy storage device itself and the total current power consumption of all electrical devices connected to the energy storage device. Photovoltaic energy storage mainly considers the charging and discharging of electrical energy, and the charging action is performed on the photovoltaic equipment. Both energy storage and energy consumption correspond to the use of the charged electrical energy, and both belong to discharge.

[0067] S320, generating a unit coefficient of a preset time length, calculating the required electric energy per unit time length through the unit coefficient, and generating a positively correlated demand value based on the required electric energy.

[0068] Generate a corresponding unit coefficient for one day, one week, one month, etc., use the unit coefficient to calculate the electric energy required for the preset unit time, and generate a positively correlated demand value through the demanded electric energy, that is, the more electric energy is required, the greater the corresponding demand value.

[0069] The demand value and the return value are two variables with the same numerical range and numerical unit. When the return value and the demand value are balanced, it is considered that the photovoltaic energy storage system as a whole is in a balance between supply and demand.

[0070] In some other embodiments, the demand value and the initial reward value are associated to determine the optimization balance logic, and the optimization reward values ​​corresponding to different behaviors are generated based on the optimization balance logic, including the following steps: S410, generating a plurality of simulation tasks including random numbers, and adding the random number to the demand value in each simulation task to generate a demand value variable.

[0071] Generate a simulation task, in which random numbers are added to the demand value multiple times to obtain several demand value variables. The random numbers can be positive or negative.

[0072] When the increased random number is a positive number, it indicates that the power demand on the simulated energy storage side increases; when the increased random number is a negative number, it indicates that the power demand on the simulated energy storage side decreases. It should be noted that when the demand value increases by a negative random number, it must be greater than 0.

[0073] S420, traverse the basic return value and determine whether it matches the demand value variable.

[0074] Iterate through all the basic return values ​​to calculate the difference between them and the adjusted demand value variable, and define a match when the basic return value is equal to the demand value variable.

[0075] When the demand value changes randomly, the classification cluster corresponding to the basic return value that matches it is characterized as the classification cluster that best matches the energy storage side after the energy storage demand changes. The matching classification cluster does not need to change due to the change in energy storage demand.

[0076] S430: If yes, the classification cluster corresponding to the basic reward value increases the positive matching count by one.

[0077] Each time a match occurs with the demand value variable, the classification cluster corresponding to the matched basic reward value increases the positive matching count by one.

[0078] S440: If not, the classification cluster corresponding to the basic reward value increases the negative matching count by one.

[0079] Each time there is no match with the demand value variable, the classification cluster corresponding to the unmatched basic reward value increases a negative match count.

[0080] S450, when all simulation tasks are completed, count the counts in each classification cluster and calculate the optimization reward value based on the following formula: S'= , Among them, S represents the basic return value, S' represents the optimized return value, n1 represents the positive matching count, and n2 represents the negative matching count. The representation is a preset ratio, which is 0.7 in the embodiment of the present application, and Step is the step length required for the basic reward value to be adjusted from the mismatched demand value variable to the matched demand value variable.

[0081] Through the above method, the corresponding bonus value is accumulated for each positive matching count, and the corresponding minus value is accumulated according to each negative matching count. The optimization return value is comprehensively calculated by considering the number of occurrences of various bases, the formulation of bonus and minus points, and the adjustment step required to transform to a matching state each time the classification clusters do not match.

[0082] The optimization return value can be used to represent the reward of the corresponding control scheme when the energy storage side needs to adjust the energy consumption and the photovoltaic side needs to further adjust and control its own parameters so that the charging and discharging data match each other.

[0083] The larger the optimization return value, the greater the corresponding reward when the photovoltaic side and the energy storage side change together to match.

[0084] In some other embodiments, obtaining the energy storage strategy type, selecting an initial reward value or an optimized reward value as a reward based on the energy storage strategy type, and generating a corresponding control scheme include the following steps: S510, energy storage strategy types include unilateral strategy and bilateral strategy. In the unilateral strategy, the demand value is used as the determined demand data, and in the bilateral strategy, the demand value variable with random tendency is used as the uncertain demand data. The random tendency is characterized by the desire to increase the storage capacity or reduce the storage capacity as a tendency to change the size of the demand value.

[0085] There are two main types of energy storage strategies, and different energy storage strategies correspond to different controlled objects.

[0086] In the unilateral strategy, the controlled object is the photovoltaic cluster, that is, through a series of actions to select the photovoltaic energy storage to adjust the control, only the corresponding parameters of the photovoltaic cluster are adjusted so that most photovoltaic clusters adjust their own states according to the optimal state to approach the balance with the target energy storage demand, and the energy demand on the energy storage side does not need to take action. At this time, the goal of photovoltaic optimization is based on a certain and known demand value, so the demand value is used as the determined demand target.

[0087] In the two-sided strategy, the controlled objects are the photovoltaic cluster and the energy storage side, that is, it is necessary to adjust the state of the photovoltaic side and the state of the energy storage side at the same time to select the optimal state that meets the management requirements and achieves a balance between photovoltaic charging and energy storage discharge to adjust the state of other photovoltaic clusters and the state of energy storage equipment and corresponding power-using equipment. Therefore, at this time, the return corresponding to the photovoltaic cluster and the power demand on the energy storage side need to be adjusted and controlled. Therefore, the goal of photovoltaic optimization at this time is based on uncertain demand value variables that need to be changed (changing the state of the photovoltaic cluster to make the power generation approach the uncertain discharge amount), and the demand value variable is used as the demand target.

[0088] S520, in the one-sided strategy, the initial reward value is used as a reward and a control scheme for charging the photovoltaic cluster is generated. In the two-sided strategy, the optimized reward value is used as a reward and a control scheme for charging the photovoltaic cluster and discharging the energy storage device is generated.

[0089] In the unilateral strategy, there is only one controlled object, the photovoltaic cluster. Therefore, the initial return value is used as a reward to select the optimal photovoltaic cluster and learn its state through reinforcement learning. The corresponding control scheme also only corresponds to the state adjustment control of the photovoltaic cluster. In the two-sided strategy, there are two controlled objects, the photovoltaic cluster and the energy storage side. Therefore, the optimized return value is used as a reward to select the state of optimal balance after adjusting the storage capacity and power generation in accordance with management requirements through reinforcement learning. The corresponding control scheme includes multiple state adjustment controls for the photovoltaic cluster and the energy storage side.

[0090] In some other embodiments, in a one-sided strategy, a selection task including the number of attempts and actions is generated, and in the selection task, the number of times the actions are triggered in the exploration phase and the utilization phase and the rewards obtained by the actions are used to select a control scheme with the highest average reward, including the following steps: S610, generate a random value between 0 and 1.

[0091] S611, when the random amount corresponds to the exploration rate, it corresponds to the exploration stage, in which a photovoltaic cluster is randomly selected based on the determined demand data and a reward is obtained.

[0092] S612, when the random amount corresponds to the utilization rate, it corresponds to the utilization stage. In the utilization stage, the photovoltaic cluster with the largest reward is selected based on the determined demand data and the reward is obtained.

[0093] For example, if the exploration rate is set to 0.30 and the utilization rate is set to 0.70, a random amount between zero and one is randomly generated. When the random amount is less than 0.30, it corresponds to the exploration stage, and when the random amount is greater than 0.30, it corresponds to the utilization stage.

[0094] At any stage, as long as the agent selects a target, the reward corresponding to the target will be accumulated. In the exploration stage, the agent randomly selects a photovoltaic cluster from several photovoltaic clusters, obtains the reward (initial return value) corresponding to the photovoltaic cluster and accumulates it; in the utilization stage, the agent will select a photovoltaic cluster with the largest reward, where the maximum reward can be the maximum initial return value of each photovoltaic cluster, or the maximum current accumulated reward. The specific strategy can be adjusted according to the actual situation.

[0095] S613, when all attempts are completed, the accumulated rewards and trigger times in each classification cluster are obtained to calculate the average reward.

[0096] S614, taking the classification cluster with the highest average reward as the optimal state, and issuing it as a control scheme to control other classification clusters to make charging adjustments.

[0097] After all attempts have been made, the average reward of each cluster is calculated based on the accumulated rewards of each cluster and the number of times each cluster is selected, and the cluster with the highest average reward is selected. The state information corresponding to this cluster is taken as the optimal state. Based on the optimal state, control plans for other photovoltaic clusters are generated and issued, so that other photovoltaic clusters can adjust their own state information to approach the optimal state.

[0098] In some other embodiments, in the two-sided strategy, a selection task including the number of attempts and actions is generated, and in the selection task, the number of times the actions are triggered in the exploration phase and the utilization phase and the rewards obtained by the actions are used to select the control scheme with the highest average reward, and the following steps are also included: S620, generate a random value between 0 and 1.

[0099] S621, when the random amount corresponds to the exploration rate, it corresponds to the exploration stage, in which a photovoltaic cluster and an uncertain demand data are randomly selected to obtain a corresponding reward.

[0100] S622, when the random quantity corresponds to the utilization rate, it corresponds to the utilization stage, in which the photovoltaic cluster with the largest reward and the uncertain demand data are selected and the corresponding rewards are obtained.

[0101] For example, if the exploration rate is set to 0.30 and the utilization rate is set to 0.70, a random amount between zero and one is randomly generated. When the random amount is less than 0.30, it corresponds to the exploration stage, and when the random amount is greater than 0.30, it corresponds to the utilization stage.

[0102] At any stage, as long as the agent selects a target, the reward corresponding to the target will be accumulated. In the exploration stage, the agent randomly selects a photovoltaic cluster and a random uncertain demand data from several photovoltaic clusters, and obtains the corresponding reward (optimized return value) and accumulates it; in the utilization stage, the agent selects a combination of photovoltaic clusters and uncertain demand data with the largest reward, where the maximum reward can be the maximum of each independent optimization return value, or the maximum current accumulated reward. The specific strategy can be adjusted according to the actual situation.

[0103] S623, when all attempts are completed, the accumulated rewards and trigger times in each classification cluster and uncertain demand data are obtained to calculate the average reward.

[0104] S624, taking the classification cluster with the highest average reward and the uncertain demand data as the optimal state, and issuing it as a control scheme to control other classification clusters to make charging adjustments and control the energy storage device to make discharging adjustments.

[0105] After all attempts have been made, the average reward corresponding to each combination of classification clusters and uncertain demand data is calculated based on the accumulated rewards of each classification cluster and uncertain demand data and the number of times each classification cluster and uncertain demand data is selected, and the classification cluster and uncertain demand data with the highest average reward are selected. The state information corresponding to the classification cluster is used as the optimal state that other photovoltaic groups need to adjust their control to achieve, and the demand value corresponding to the uncertain demand data is used as the optimal state that the energy storage device needs to adjust its discharge to achieve. In this optimal state, the charging amount of the photovoltaic and the discharging amount of the energy storage device are balanced, and a control plan for other photovoltaic clusters and energy storage systems is generated and issued based on the optimal state.

[0106] The present application also discloses a photovoltaic energy storage control system based on intelligent learning, which is used to implement the above method.

[0107] The present application also discloses a storage medium, characterized in that the storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the above method.

[0108] The implementation principle is: First, based on the specific content of environmental information, the exploration rate and utilization rate are dynamically generated, and the return value that each photovoltaic cluster can provide is generated in combination with the static characteristics of each photovoltaic cluster. At the same time, the degree of electricity demand is calculated according to the static characteristics of the energy storage side, and the electricity consumption information on the energy storage side and the charging information on the photovoltaic side are associated to dynamically generate the balance logic of the two. At the same time, the dynamic optimization return value is generated through the balance logic. Finally, the corresponding control scheme is selected according to the required energy storage strategy, and the objects of each control scheme are selected based on the exploration rate and utilization rate. Finally, the object with the highest average reward is output as the basis of the control scheme for distribution. By converting unstable weather content into exploration rates of different sizes, high randomness is converted into highly stable task parameters. Combined with different energy storage control strategies, comprehensive selection tasks are carried out to achieve a balance between charging and discharging on both sides. Reinforcement learning is used to formalize strategies under uncertainty to improve the wisdom and accuracy of intelligent agent decision-making.

[0109] It should be understood that although the steps in the flowchart of the accompanying drawings are shown in sequence according to the instructions of the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise clearly stated in this document, the execution of these steps is not strictly limited in order and can be performed in other orders.

[0110] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A photovoltaic energy storage control method based on intelligent learning, characterized in that: The following steps are involved: Acquire environmental information, and generate an exploration phase including a corresponding exploration rate and a utilization phase including a corresponding utilization rate based on the environmental information; Acquire status information of distributed photovoltaic clusters, and configure an initial reward value for each photovoltaic cluster based on the status information; Acquire energy storage information, and generate a demand value per unit time based on the energy storage information; Associating the demand value with the initial reward value to determine an optimization balance logic, and generating optimization reward values ​​corresponding to different behaviors based on the optimization balance logic; Acquire the energy storage strategy type, and based on the energy storage strategy type, select the initial reward value or the optimized reward value as a reward and generate a corresponding control scheme; A selection task including the number of attempts and actions is generated, and in the selection task, the control scheme with the highest average reward is selected according to the number of times the action is triggered in the exploration phase and the utilization phase and the reward obtained by the action.

2. The photovoltaic energy storage control method based on intelligent learning according to claim 1 is characterized in that: The environmental information includes weather, dust volume, temperature, and humidity. Acquiring the environmental information and generating an exploration phase including a corresponding exploration rate and a utilization phase including a corresponding utilization rate based on the environmental information includes the following steps: Calculating a power generation efficiency score based on the environmental information; An exploration rate ε inversely proportional to the power generation efficiency score is generated, and a utilization rate 1-ε is generated based on the exploration rate ε.

3. The photovoltaic energy storage control method based on intelligent learning according to claim 1 is characterized in that: Acquiring status information of distributed photovoltaic clusters and configuring an initial reward value for each photovoltaic cluster based on the status information includes the following steps: Acquire the installation status of each photovoltaic device and record it as first information, wherein the installation status includes an installation horizontal angle, an installation pitch angle, and an installation height; Obtaining the basic status of each photovoltaic device and recording it as second information, wherein the basic status includes capacity, number of photovoltaic panels, type of photovoltaic panels, and efficiency of photovoltaic panels; integrating a plurality of the photovoltaic devices having the same first information and the same second information into a photovoltaic cluster; Acquire the current unit power generation of each of the photovoltaic clusters, determine the contribution value of the current unit power generation and the average adjustment step length of each photovoltaic cluster corresponding to other photovoltaic clusters to generate third information; An initial reward value is configured for each of the photovoltaic clusters according to the third information.

4. The photovoltaic energy storage control method based on intelligent learning according to claim 1 is characterized in that: Acquiring energy storage information and generating a demand value per unit time based on the energy storage information includes the following steps: Acquire the current storage capacity of the energy storage device and the current power consumption of the power consumption device to jointly constitute the energy storage information; The unit coefficient of the preset time length is generated, the required electric energy per unit time length is calculated by the unit coefficient, and a positively correlated demand value is generated based on the required electric energy.

5. The photovoltaic energy storage control method based on intelligent learning according to claim 1 is characterized in that: Associating the demand value with the initial reward value to determine the optimization balance logic, and generating optimization reward values ​​corresponding to different behaviors based on the optimization balance logic, including the following steps: generating a plurality of simulation tasks including random numbers, and adding the random numbers to the demand values ​​in each simulation task to generate demand value variables; Traversing the basic reward value and determining whether it matches the demand value variable; If so, the classification cluster corresponding to the basic reward value increases the positive matching count by one; If not, the classification cluster corresponding to the basic reward value increases the negative matching count by one; When all the simulation tasks are completed, the counts in each classification cluster are counted and the optimization reward value is calculated based on the following formula: S’= ， Among them, S represents the basic return value, S' represents the optimized return value, n1 represents the positive matching count, and n2 represents the negative matching count. It is represented by a preset ratio, and Step is represented by the step length required for the basic reward value to be adjusted from not matching the demand value variable to matching the demand value variable.

6. The photovoltaic energy storage control method based on intelligent learning according to claim 5 is characterized in that: Acquiring the energy storage strategy type, and selecting the initial reward value or the optimized reward value as a reward based on the energy storage strategy type and generating a corresponding control scheme, including the following steps: The energy storage strategy types include unilateral strategy and bilateral strategy. In the unilateral strategy, the demand value is used as determined demand data. In the bilateral strategy, the demand value variable with random tendency is used as uncertain demand data. The random tendency is characterized by the desire to increase or reduce the storage energy as a tendency to change the size of the demand value. In the one-sided strategy, the initial reward value is used as the reward and a control scheme for charging the photovoltaic cluster is generated. In the two-sided strategy, the optimized reward value is used as the reward and a control scheme for charging the photovoltaic cluster and discharging the energy storage device is generated.

7. The photovoltaic energy storage control method based on intelligent learning according to claim 6 is characterized in that: In the one-sided strategy, a selection task including the number of attempts and actions is generated, and in the selection task, the control scheme with the highest average reward is selected according to the number of times the action is triggered in the exploration phase and the utilization phase and the reward obtained by the action, including the following steps: Generate a random value between 0 and 1; When the random amount corresponds to the exploration rate, it corresponds to the exploration phase, in which one of the photovoltaic clusters is randomly selected based on the determined demand data and a reward is obtained; When the random amount corresponds to the utilization rate, it corresponds to the utilization stage, and in the utilization stage, the photovoltaic cluster with the largest reward is selected based on the determined demand data and the reward is obtained; When all the attempts are completed, the accumulated rewards and trigger times in each classification cluster are obtained to calculate the average reward; The classification cluster with the highest average reward is taken as the optimal state, and is issued as the control scheme to control the other classification clusters to perform charging adjustments.

8. The photovoltaic energy storage control method based on intelligent learning according to claim 6 is characterized in that: In the two-sided strategy, a selection task including the number of attempts and actions is generated, and in the selection task, the control scheme with the highest average reward is selected by the number of times the action is triggered in the exploration phase and the utilization phase and the reward obtained by the action, and the following steps are also included: Generate a random value between 0 and 1; When the random amount corresponds to the exploration rate, it corresponds to the exploration stage, in which one of the photovoltaic clusters and one of the uncertain demand data are randomly selected to obtain a corresponding reward; When the random amount corresponds to the utilization rate, it corresponds to the utilization stage, and in the utilization stage, the photovoltaic cluster and the uncertain demand data with the largest reward are selected and corresponding rewards are obtained; When all the attempts are completed, the rewards and trigger times accumulated in each of the classification clusters and the uncertain demand data are obtained to calculate an average reward; The classification cluster with the highest average reward and the uncertain demand data are taken as the optimal state and issued as the control scheme to control the other classification clusters to perform charging adjustment and control the energy storage device to perform discharging adjustment.

9. A photovoltaic energy storage control system based on intelligent learning, characterized in that: Used to implement the method according to any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method as described in any one of claims 1-8.

Images