Strategy generation methods, apparatus and electronic devices for integrated energy systems

Abstract

This invention provides a strategy generation method, apparatus, and electronic device for integrated energy systems, applicable to the field of energy dispatching technology. The method includes: processing historical load information from the user end using a dynamic trading adjustment model to generate initial resource trading information for the distribution network; processing the initial resource trading information, initial electricity load transfer information, and a predetermined proportion of load that can be shifted using a dynamic load transfer model to generate a power consumption strategy for the user end; processing initial environmental information and photovoltaic power generation equipment parameters using an objective function to generate the expected output power of the photovoltaic power generation equipment; determining the distribution strategy of the distribution network based on the power consumption strategy, expected output power, and energy storage status of the energy storage system; determining the energy storage strategy of the energy storage system based on the initial resource trading information; and generating a target strategy by iteratively updating the initial resource trading information, power consumption strategy, and energy storage strategy using a reinforcement learning algorithm.

Description

Technical Field

[0001] This invention relates to the field of energy dispatching technology, and more specifically, to a strategy generation method, apparatus, and electronic device for integrated energy systems. Background Technology

[0002] With the integration of renewable energy into distribution networks and the development of electricity markets, time-of-use resource trading information struggles to reflect the fluctuating characteristics of distribution network load and renewable energy, thus hindering incentives for users to adjust their loads rationally according to distribution network demand. In regional integrated energy systems, the interaction and optimization of multiple energy sources become even more complex.

[0003] In related technologies, the resource scheduling of power systems cannot fully consider the resource utilization status of distribution networks and other energy systems, as well as the game relationship between multiple entities, resulting in a low resource utilization rate of integrated energy systems. Summary of the Invention

[0004] In view of this, the present invention provides a strategy generation method, apparatus and electronic device for integrated energy systems.

[0005] One aspect of the present invention provides a strategy generation method for an integrated energy system, comprising: acquiring historical load information, initial electricity load transfer information, initial environmental information, and energy storage status of an energy storage system at the user end; processing the historical load information at the user end by calling a dynamic transaction adjustment model to generate initial resource transaction information for the distribution network; processing the initial resource transaction information, the initial electricity load transfer information, and a predetermined proportion of load that can be transferred by calling a dynamic load transfer model to generate an electricity consumption strategy at the user end; the electricity consumption strategy indicates the expected time period for the load that can be transferred from the user end to the energy storage system, so that the energy storage system can perform energy storage operations during the expected time period; processing the initial environmental information and photovoltaic power generation equipment parameters by calling an objective function to generate the expected output power of the photovoltaic power generation equipment; determining the distribution strategy of the distribution network based on the electricity consumption strategy, the expected output power, and the energy storage status of the energy storage system; and determining the energy storage strategy of the energy storage system based on the initial resource transaction information. A state space is constructed using the historical load information of the aforementioned user terminals, initial electricity load transfer information, initial environmental information, and energy storage status of the energy storage system. An action space is constructed using the electricity consumption strategy of the aforementioned user terminals, the power distribution strategy of the aforementioned power distribution network, and the aforementioned energy storage strategy. Based on the objective reward function, the aforementioned initial resource transaction information, the aforementioned electricity consumption strategy, and the aforementioned energy storage strategy are iteratively updated using a reinforcement learning algorithm to generate the objective strategy.

[0006] According to an embodiment of the present invention, the aforementioned historical load information includes: dynamic electricity consumption strategies for historical periods and resource transaction information for historical periods. The process of processing the historical load information at the user end by calling a dynamic transaction adjustment model to generate initial resource transaction information for the distribution network includes: averaging the dynamic electricity consumption strategies for the aforementioned historical periods to obtain an average value for the historical periods; performing difference processing on the dynamic electricity consumption strategies and the average value for the historical periods based on a predetermined period to obtain the variance of the historical periods' electricity consumption strategies; and processing the variance of the historical periods' electricity consumption strategies, the dynamic electricity consumption strategies for the aforementioned historical periods, the average value for the aforementioned historical periods' electricity consumption strategies, and the resource transaction information for the aforementioned historical periods based on a predetermined adjustment coefficient to obtain the initial resource transaction information for the historical periods.

[0007] According to an embodiment of the present invention, the energy storage system includes a water-cooled storage device and a battery device. The determination of the energy storage strategy of the energy storage system based on the initial resource transaction information and a predetermined transaction threshold includes: in response to determining that the initial resource transaction information is less than the predetermined transaction threshold, determining that the water-cooled storage device initiates a cold storage operation and the battery device initiates a charging operation. In response to determining that the initial resource transaction information is greater than or equal to the predetermined transaction threshold, determining that the water-cooled storage device initiates a cold release operation and the battery device initiates a discharge operation.

[0008] According to an embodiment of the present invention, the strategy generation method further includes: the user terminal determining cooling demand based on the initial environmental information and the initial resource transaction information; and determining the cooling release rate during the start-up cooling operation of the water-based cooling storage device based on the cooling demand.

[0009] According to an embodiment of the present invention, determining the power distribution strategy of the distribution network based on the user demand at the user end, the expected output power of the photovoltaic power generation equipment, and the energy storage status of the energy storage system includes: determining the priority of the photovoltaic power generation equipment, the energy storage system, and the distribution network in supplying energy to the user end. Based on the user demand at the user end, the expected output power of the photovoltaic power generation equipment, and the energy storage status of the energy storage system, the power distribution strategy of the distribution network is determined according to the priority.

[0010] According to an embodiment of the present invention, the target reward function includes: a reward function for the user end, a reward function for the energy storage system, and a reward function for the distribution network; the strategy generation method further includes: calling the reward function of the distribution network to process the initial resource transaction information and the load demand of the distribution network, and generating a reward value for the distribution network; calling the reward function of the user end to process the initial resource transaction information, the user end's electricity consumption strategy, and predetermined electricity consumption preference parameters, and generating a reward value for the user end; and calling the reward function of the energy storage system to process the initial resource transaction information and the energy storage status of the energy storage system, and generating a reward value for the energy storage end. The target strategy includes the electricity consumption strategy of the user end, the energy storage strategy of the energy storage end, and the target resource transaction information of the distribution network when the reward values ​​of the user end, the energy storage end, and the distribution network respectively satisfy the convergence condition.

[0011] According to an embodiment of the present invention, the above-described method of generating a user-side electricity consumption strategy by invoking a dynamic load transfer model to process the initial resource transaction information, the initial electricity load transfer information, and the predetermined transferable load ratio includes: determining the user's expected electricity consumption behavior based on the initial resource transaction information; and generating the user-side electricity consumption strategy by invoking the dynamic load transfer model to process the initial electricity load transfer information, the predetermined transferable load ratio, and the user's expected electricity consumption behavior.

[0012] Another aspect of the present invention provides a strategy generation device for an integrated energy system, comprising: an acquisition module for acquiring historical load information, initial electricity load transfer information, initial environmental information, and energy storage status of an energy storage system; a first generation module for processing the historical load information of the user terminal by calling a dynamic transaction adjustment model to generate initial resource transaction information of the distribution network; a second generation module for processing the initial resource transaction information, the initial electricity load transfer information, and a predetermined transferable load ratio by calling a dynamic load transfer model to generate an electricity consumption strategy for the user terminal; the electricity consumption strategy indicates the expected time period information for the transferable load from the user terminal to the energy storage system, so that the energy storage system performs energy storage operations during the expected time period information; a third generation module for processing the initial environmental information and photovoltaic power generation equipment parameters by calling an objective function to generate the expected output power of the photovoltaic power generation equipment; a first determination module for determining the distribution strategy of the distribution network based on the electricity consumption strategy, the expected output power, and the energy storage status of the energy storage system; and a second determination module for determining the energy storage strategy of the energy storage system based on the initial resource transaction information. The fourth generation module is used to construct a state space based on the historical load information of the user terminal, the initial power load transfer information, the initial environmental information, and the energy storage status of the energy storage system; to construct an action space based on the power consumption strategy of the user terminal, the power distribution strategy of the distribution network, and the energy storage strategy; and to generate a target strategy based on the target reward function by iteratively updating the initial resource transaction information, the power consumption strategy, and the energy storage strategy using a reinforcement learning algorithm.

[0013] Another aspect of the present invention provides an electronic device comprising:

[0014] One or more processors;

[0015] Memory, used to store one or more programs.

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the strategy generation method as described above.

[0017] Another aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the strategy generation method described above.

[0018] According to embodiments of the present invention, historical load information, initial load transfer information, initial environmental information, and energy storage status of the energy storage system at the user end are acquired to provide a data foundation for the generation of subsequent target strategies. By calling a dynamic transaction adjustment model to process the historical load information at the user end, initial resource transaction information of the distribution network is generated, thereby guiding the adjustment strategies of the user end, photovoltaic power generation system, and energy storage system. By calling a dynamic load transfer model to process the initial resource transaction information, initial load transfer information, and predetermined transferable load ratio, a power consumption strategy at the user end is generated, thereby generating a power consumption strategy that meets the user end's power demand under the guidance of the initial resource transaction information. By calling a target function to process the initial environmental information and photovoltaic power generation equipment parameters, the expected output power of the photovoltaic power generation equipment is generated, thereby providing the distribution network with energy supply information for the photovoltaic power generation equipment. Based on the power consumption strategy, expected output power, and energy storage status of the energy storage system, the distribution strategy of the distribution network is determined, thereby achieving demand matching between the energy supply end and the energy consumption end. Based on the initial resource transaction information, the energy storage strategy of the energy storage system is determined, enabling the energy storage system to adjust its energy storage strategy to achieve optimal resource utilization of the energy storage system. A state space is constructed using historical load information from the user end, initial electricity load transfer information, initial environmental information, and the energy storage status of the energy storage system. An action space is constructed using the user end's electricity consumption strategy, the distribution network's distribution strategy, and the energy storage strategy. Based on a target reward function, a reinforcement learning algorithm is used to iteratively update the initial resource transaction information, electricity consumption strategy, and energy storage strategy to generate the target strategy. This achieves optimal resource utilization of the integrated energy system under conditions of balanced electricity supply and demand. Attached Figure Description

[0019] The above and other objects, features and advantages of the present invention will become more apparent from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0020] Figure 1 An application scenario diagram of a strategy generation method for an integrated energy system according to an embodiment of the present invention is shown;

[0021] Figure 2 A flowchart of a strategy generation method for an integrated energy system according to an embodiment of the present invention is shown;

[0022] Figure 3 A schematic diagram of a game theory model for an integrated energy system according to an embodiment of the present invention is shown;

[0023] Figure 4 A schematic diagram of the structure of the deep deterministic policy gradient algorithm according to an embodiment of the present invention is shown;

[0024] Figure 5 A schematic diagram showing the annual electricity load and photovoltaic power generation of a certain park according to an embodiment of the present invention is provided.

[0025] Figure 6 The following are four typical daily electrical loads and photovoltaic power generation scenarios according to embodiments of the present invention;

[0026] Figure 7 A schematic diagram illustrating the target strategy iterative calculation process for a workday during the cooling season, according to an embodiment of the present invention, is shown.

[0027] Figure 8 A schematic diagram illustrating the target strategy iterative calculation process for rest days during the cooling season according to an embodiment of the present invention is shown.

[0028] Figure 9 A schematic diagram illustrating the target strategy iterative calculation process for the workday during the transition season according to an embodiment of the present invention is shown.

[0029] Figure 10 A schematic diagram illustrating the target strategy iterative calculation process for transitional season rest days according to an embodiment of the present invention is shown.

[0030] Figure 11 The following diagrams show a comparison of four typical intraday time-sharing resource transaction information and dynamic resource transaction information according to embodiments of the present invention;

[0031] Figure 12 The electrical loads of four typical daytime users are shown according to embodiments of the present invention;

[0032] Figure 13 The following illustrations show the electrical loads obtained from the distribution network by four typical daily users according to embodiments of the present invention;

[0033] Figure 14 The power generation and grid connection of a photovoltaic power generation device according to an embodiment of the present invention are shown;

[0034] Figure 15A A schematic diagram comparing the energy storage status of a battery device and a water-cooled storage device on a working day during the cooling season is shown in an energy storage system according to an embodiment of the present invention.

[0035] Figure 15B A schematic diagram comparing the energy storage status of a battery device and the energy storage status of a water-cooled storage device on a rest day during the cooling season is shown in an energy storage system according to an embodiment of the present invention.

[0036] Figure 15C A schematic diagram comparing the energy storage status of a battery device and a water-cooled storage device during a working day in a transitional season is shown in an energy storage system according to an embodiment of the present invention.

[0037] Figure 15D A schematic diagram comparing the energy storage status of a battery device and a water-cooled storage device on a rest day during the transition season is shown in an energy storage system according to an embodiment of the present invention.

[0038] Figure 16 A block diagram of a strategy generation apparatus for an integrated energy system according to an embodiment of the present invention is shown;

[0039] Figure 17 A block diagram of an electronic device suitable for implementing a strategy generation method for an integrated energy system, according to an embodiment of the present invention, is shown. Detailed Implementation

[0040] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0041] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0042] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0043] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0044] In related technologies, distribution networks guide users to shift loads by formulating time-of-use resource trading information to utilize renewable energy and simultaneously balance the net load distribution of the distribution network. However, formulating time-of-use resource trading information based on the total electricity demand of users in each time period of the electricity consumption cycle makes it difficult to match renewable energy output, energy storage devices, and user electricity demand, thus failing to incentivize users to shift their electricity loads, resulting in suboptimal resource utilization in the energy system.

[0045] In view of this, embodiments of the present invention provide a strategy generation method, apparatus, and electronic device for integrated energy systems.

[0046] Figure 1 An application scenario diagram of a strategy generation method for an integrated energy system according to an embodiment of the present invention is shown.

[0047] like Figure 1 As shown, application scenario 100 according to an embodiment of this application may include a distribution network 110, a user terminal 120, and an energy storage system 130. The distribution network can formulate resource transaction information, and use the resource transaction information as a link to link the user terminal and the energy storage system to build multi-source data interaction and strategy response, and quantify the resource utilization status of each entity by constructing a cooperative game model.

[0048] By solving the game equilibrium point in the continuous action space through reinforcement learning algorithms, the integrated energy system can be driven to adaptively optimize in complex multi-energy coupling scenarios through a two-way feedback mechanism of resource transaction information and target strategy. Ultimately, this achieves the comprehensive goal of efficient energy allocation and flexible load adjustment, thereby rationally adjusting the resource utilization status.

[0049] Figure 2 A flowchart of a strategy generation method for an integrated energy system according to an embodiment of the present invention is shown.

[0050] like Figure 2 As shown, the method may include operations S210 to S270.

[0051] During operation S210, historical load information, initial power load transfer information, initial environmental information, and energy storage status of the energy storage system are obtained from the user terminal.

[0052] Historical load information at the user end can be a time-series record of energy consumption such as electricity and cooling over a past period. One implementation method is to use 24 hours as a power consumption cycle, dividing the 24 hours into 96 time periods with a 15-minute step. That is, a time period can be 15 minutes, and 24 hours includes 96 time periods.

[0053] Initial load transfer information can be transferable loads that users can flexibly adjust based on historical resource transaction changes or incentive measures. The electricity consumption time of transferable loads can be intermittent and not subject to continuity constraints, and can be transferred from one time period to another.

[0054] Initial environmental information can be environmental data that affects photovoltaic power generation equipment, such as light intensity, solar radiation intensity, or temperature.

[0055] The energy storage state of an energy storage system can refer to the energy storage state or operating state of the energy storage system (e.g., battery devices and water-cooled storage devices). For example, the energy storage state can indicate that a battery device is in a charging state or a discharging state, and a water-cooled storage device is in a cooling storage state or a cooling release state.

[0056] In operation S220, the historical load information of the user end is processed by calling the dynamic transaction adjustment model to generate the initial resource transaction information of the distribution network.

[0057] The dynamic trading adjustment model can be constructed based on historical resource trading information, predetermined adjustment coefficients, user-side electricity consumption strategies for historical periods, the average value of user-side electricity consumption strategies for historical periods, and the variance of electricity consumption strategies for historical periods. As one implementation method, the initial resource trading information can be information on resource trading between the user and the distribution network.

[0058] In operation S230, the initial resource transaction information, initial electricity load transfer information, and predetermined transferable load ratio are processed by calling the dynamic load transfer model to generate the user's electricity consumption strategy.

[0059] The electricity consumption strategy indicates the user's electricity load information, which may include the total electricity load, electricity consumption period information, and the period information of transferable electricity load. The energy storage system can obtain the expected period information of transferable load based on the electricity consumption strategy, so that the energy storage system can perform energy storage or energy release operations during the expected period information.

[0060] The dynamic load transfer model can be constructed based on the proportion of untransferred and transferable electricity loads at the user end. The electricity consumption strategy generated by the dynamic load transfer model at the user end needs to meet the electricity consumption strategy constraints constituted by the proportion of untransferred and transferable electricity loads. The user end's electricity consumption strategy can be the electricity consumption behavior adjusted based on the initial resource transaction information, and the expected time period information of transferable load can be the electricity consumption time period information of transferable load obtained by the user end based on the initial resource transaction information.

[0061] The predetermined proportion of transferable load can be a pre-set value, which is a fixed proportion of the transferable load at the user end to the total electricity load. It can obtain the maximum boundary for load adjustment at the user end, thereby avoiding excessive load transfer from affecting the user's electricity demand. It is a constraint parameter of the dynamic load transfer model.

[0062] In operation S240, the target function is called to process the initial environmental information and photovoltaic power generation equipment parameters, and to generate the expected output power of the photovoltaic power generation equipment.

[0063] The objective function can represent the relationship between initial environmental information and the expected output power of photovoltaic power generation equipment.

[0064] Photovoltaic power generation equipment parameters can be parameters that affect the power generation of photovoltaic power generation equipment, such as the operating temperature, power generation efficiency, or solar panel area of ​​the photovoltaic power generation equipment.

[0065] The expected output power of photovoltaic power generation equipment will fluctuate under different seasons, weather and other initial environmental information changes. The functional relationship between the expected output power and the initial environmental information can be expressed by formula (1).

[0066] (1)

[0067] (2)

[0068] (3)

[0069] in, It is the expected output power of the photovoltaic power generation equipment at time t. It is the actual light intensity at time t, and the unit is W / m². 2 . The power generation efficiency of the photovoltaic power generation equipment at time t; It refers to the area of ​​the solar panels in a photovoltaic power generation system, measured in square meters (m²). 2 . It is the actual operating temperature of the photovoltaic power generation equipment at time t; It is the ambient temperature at time t. It is the operating temperature of photovoltaic power generation equipment under standard testing conditions; It is the ambient temperature under standard test conditions, and the unit of temperature is °C; It is the temperature power coefficient, which can be 0.0047 / ℃. This is the solar radiation intensity under standard test conditions, measured in W / m². 2 . This refers to the power generation efficiency of photovoltaic power generation equipment under standard conditions.

[0070] When operating S250, the power distribution strategy of the distribution network is determined based on the power consumption strategy, expected output power, and energy storage status of the energy storage system.

[0071] The power distribution strategy of a power distribution network can be determined based on the user's power consumption strategy, the expected output power of photovoltaic power generation equipment, and the energy storage status of the energy storage system, thereby achieving a balance between power supply and demand in the power distribution network.

[0072] In operation S260, the energy storage strategy of the energy storage system is determined based on the initial resource transaction information.

[0073] The energy storage strategy of an energy storage system can be based on the energy storage state of the system determined by initial resource transaction information. The energy storage state can be either energy storage or energy release by the energy storage system.

[0074] In operation S270, the state space is constructed using the user's historical load information, initial power load transfer information, initial environmental information, and energy storage status of the energy storage system. The action space is constructed using the user's power consumption strategy, the distribution network's power distribution strategy, and the energy storage strategy. Based on the objective reward function, the initial resource transaction information, power consumption strategy, and energy storage strategy are iteratively updated using a reinforcement learning algorithm to generate the objective strategy.

[0075] The state space can be a set describing all possible states of the integrated energy system, the action space can be a set of all possible actions of the integrated energy system, and the objective reward function can be a function that quantifies the action results of the integrated energy system, with the output being the reward value.

[0076] The target strategy can be the optimal solution obtained by iteratively updating the initial resource transaction information, electricity consumption strategy and energy storage strategy using reinforcement learning algorithms. It can balance the electricity consumption strategy of the user end, the energy storage strategy of the energy storage system and the power distribution strategy of the distribution network, thereby achieving the optimal resource utilization state.

[0077] The relationships between the distribution network, user terminals, and energy storage systems in an integrated energy system can be constructed using a game theory model.

[0078] Figure 3 A schematic diagram of a game theory model for an integrated energy system according to an embodiment of the present invention is shown.

[0079] like Figure 3 As shown, the participants in the game model can include the distribution network, the user end, the energy storage system, and the power generation equipment. The distribution network acts as the leader in the game model, while the user end, the energy storage system, and the photovoltaic power generation equipment act as followers, thus constructing a leader-follower game model. The user end, the energy storage system, and the photovoltaic power generation equipment can be in a cooperative game relationship.

[0080] As the dominant force in the game theory model, the distribution network can formulate resource trading information to guide users to adjust their electricity consumption strategies, energy storage systems to adjust their energy storage strategies, and photovoltaic power generation equipment to determine its energy supply. Users' electricity consumption strategies can involve adjusting their own electricity load based on resource trading information. Energy storage systems can determine whether to initiate charging or discharging operations for battery equipment and whether to initiate cooling or releasing operations for water-based cooling systems based on user electricity consumption strategies and the distribution network's resource trading information.

[0081] According to embodiments of the present invention, historical load information, initial load transfer information, initial environmental information, and energy storage status of the energy storage system at the user end are acquired to provide a data foundation for the generation of subsequent target strategies. By calling a dynamic transaction adjustment model to process the historical load information at the user end, initial resource transaction information of the distribution network is generated, thereby guiding the adjustment strategies of the user end, photovoltaic power generation system, and energy storage system. By calling a dynamic load transfer model to process the initial resource transaction information, initial load transfer information, and predetermined transferable load ratio, a power consumption strategy at the user end is generated, thereby generating a power consumption strategy that meets the user end's power demand under the guidance of the initial resource transaction information. By calling a target function to process the initial environmental information and photovoltaic power generation equipment parameters, the expected output power of the photovoltaic power generation equipment is generated, thereby providing the distribution network with energy supply information for the photovoltaic power generation equipment. Based on the power consumption strategy, expected output power, and energy storage status of the energy storage system, the distribution strategy of the distribution network is determined, thereby achieving demand matching between the energy supply end and the energy consumption end. Based on the initial resource transaction information, the energy storage strategy of the energy storage system is determined, enabling the energy storage system to adjust its energy storage strategy to achieve optimal resource utilization of the energy storage system. A state space is constructed using historical load information from the user end, initial electricity load transfer information, initial environmental information, and the energy storage status of the energy storage system. An action space is constructed using the user end's electricity consumption strategy, the distribution network's distribution strategy, and the energy storage strategy. Based on a target reward function, a reinforcement learning algorithm is used to iteratively update the initial resource transaction information, electricity consumption strategy, and energy storage strategy to generate the target strategy. This achieves optimal resource utilization of the integrated energy system under conditions of balanced electricity supply and demand.

[0082] The following explains how the distribution network uses a dynamic transaction adjustment model to generate initial resource transaction information for the distribution network.

[0083] Historical load information at the user end includes: dynamic electricity consumption strategies for historical periods and resource transaction information for historical periods.

[0084] By calling the dynamic transaction adjustment model to process the historical load information of the user end, the initial resource transaction information of the distribution network can be generated, which may include operations A1 to A3.

[0085] In operation A1, the dynamic electricity consumption strategy for historical periods is averaged to obtain the average value of the electricity consumption strategy for historical periods.

[0086] The dynamic electricity consumption strategies for historical periods are averaged to eliminate the interference of fluctuations in dynamic electricity consumption strategies within a single period. The average value of electricity consumption strategies for historical periods can reflect the average value of the user's electricity consumption strategy patterns, providing a reference for subsequent initial resource transaction information and strategy optimization for historical periods.

[0087] In operation A2, based on a predetermined period, the difference between the dynamic electricity consumption strategy and the average value of the electricity consumption strategy for historical periods is processed to obtain the variance of the electricity consumption strategy for historical periods.

[0088] The variance of electricity consumption strategies over historical periods can reflect the degree of fluctuation of the user's dynamic electricity consumption strategy over historical periods relative to the average value of the electricity consumption strategy over historical periods, and can characterize the stability of the dynamic electricity consumption strategy over historical periods.

[0089] As one implementation method, the predetermined period can be 24 hours, with 15 minutes as a time period, and the predetermined period can be divided into 96 time periods.

[0090] The variance of electricity consumption strategies during historical periods can be represented by formula (4).

[0091] (4)

[0092] in, It is the variance of the electricity consumption strategy. It is the dynamic electricity consumption strategy for user-side time period t. It is the average power consumption strategy of the user terminal during time period t, and N is 96, which represents 96 time periods within a predetermined power consumption cycle.

[0093] In operation A3, based on a predetermined adjustment coefficient, the variance of electricity consumption strategies, the dynamic electricity consumption strategies of historical periods, the average value of electricity consumption strategies of historical periods, and the resource transaction information of historical periods are processed to obtain the initial resource transaction information of historical periods.

[0094] The initial resource transaction information obtained in the historical period can be represented by formula (5).

[0095] (5)

[0096] in, This is the initial resource transaction information for time period t. It is resource transaction information from historical periods. It is the dynamic electricity consumption strategy for user-side time period t. It is the average value of the user's electricity consumption strategy during time period t. It is the predetermined adjustment factor, which is always greater than zero.

[0097] The power distribution network can influence market electricity demand and supply by adjusting initial resource trading information. The initial resource trading information must meet the following constraints:

[0098] (6)

[0099] in, It is the minimum value of the initial resource transaction information in time period t. It represents the maximum value of the initial resource transaction information during time period t. The initial resource transaction information constraint can be used to limit the adjustment behavior of the distribution network to the initial resource transaction information. The minimum value can be used to ensure the normal operation of the distribution network, and the maximum value can meet the electricity demand of the user end.

[0100] According to embodiments of the present invention, the fluctuation level of the user's dynamic electricity consumption strategy during historical periods is obtained by calculating the average value and variance of the electricity consumption strategy during historical periods. Based on a predetermined adjustment coefficient, the variance of the electricity consumption strategy during historical periods, the dynamic electricity consumption strategy during historical periods, the average value of the electricity consumption strategy during historical periods, and the resource transaction information during historical periods are processed to obtain initial resource transaction information that conforms to the user's dynamic electricity consumption strategy, thereby providing a basis for adjusting the resource utilization status of the integrated energy system.

[0101] The following explains how to generate user-side electricity consumption strategies by calling the dynamic load transfer model.

[0102] By calling the dynamic load transfer model to process initial resource transaction information, initial electricity load transfer information, and the predetermined proportion of load that can be transferred, the electricity consumption strategy generated at the user end can include operations B1 to B2.

[0103] In operation B1, the user's expected electricity consumption behavior is determined based on the initial resource transaction information.

[0104] Users can adjust the timing of some of their electricity consumption based on initial resource transaction information, thereby shifting some of their electricity load. For example, if the initial resource transaction information is higher than expected, users tend to reduce their electricity load; if the initial resource transaction information is lower than expected, users tend to increase their electricity load.

[0105] In some embodiments, based on statistical analysis of the historical electricity consumption behavior of users in the target area and the dynamic resource transaction information of the distribution network, the electricity load that users in the target area may transfer in the future based on the initial resource transaction information can be estimated according to the statistical analysis results.

[0106] The electricity load after the transfer needs to be consistent with that before the transfer in order to maintain the electricity demand of the user end, which can be expressed by formula (7).

[0107] (7)

[0108] (8)

[0109] in, It is the transferred electrical load. The current user's electricity load is the untransferred electricity load, t is the electricity consumption period, and N is 96 time periods within a predetermined cycle. It is the proportion of loads that can be shifted.

[0110] In operation B2, the initial power load transfer information, the predetermined proportion of load that can be transferred, and the user's expected power consumption behavior are processed by calling the dynamic load transfer model to generate the user's power consumption strategy.

[0111] Dynamic load transfer models can transform users' expected electricity consumption behavior into executable electricity consumption plans. By integrating constraints and users' electricity consumption behavior tendencies through dynamic load transfer models, electricity consumption strategies that match users' electricity consumption tendencies can be generated.

[0112] The user's electricity consumption strategy can be represented by formula (9).

[0113] (9)

[0114] in, It is the user-side electricity consumption strategy. These are the user electricity loads that have not been transferred. It is the proportion of movable electrical load, which can be a numerical value, for example, 0.10.

[0115] According to embodiments of the present invention, based on initial resource transaction information, the expected electricity consumption behavior of users is determined, thereby obtaining the users' electricity load tendencies under different conditions. A dynamic load transfer model is used to transform the users' electricity load tendencies into electricity consumption strategies, so that the distribution network can adjust its distribution strategy according to the users' electricity consumption strategies.

[0116] The following explains how to determine the distribution strategy for the distribution network.

[0117] Based on user demand at the user end, the expected output power of photovoltaic power generation equipment, and the energy storage status of the energy storage system, the power distribution strategy of the distribution network can include operations C1 to C2.

[0118] In operation C1, the priority of photovoltaic power generation equipment, energy storage system and distribution network to supply energy to users is determined.

[0119] The power supply priorities from photovoltaic power generation equipment, energy storage systems, and distribution networks to users can be flexibly configured according to the users' electricity consumption strategies. As one implementation method, the power supply priority to users can be: first, power from photovoltaic power generation equipment; second, power from energy storage systems; and finally, power from the distribution network.

[0120] In operation C2, the power distribution strategy of the distribution network is determined according to priority, based on the user's power consumption strategy, the expected output power of photovoltaic power generation equipment, and the energy storage status of the energy storage system.

[0121] The distribution network can be the main energy supplier of the integrated energy system, and the distribution strategy of the distribution network needs to be subject to the constraints of power supply and demand balance. The power output of the distribution network can be expressed by formula (10).

[0122] (10)

[0123] (11)

[0124] (12)

[0125] (13)

[0126] in, It is the net power interaction between the user terminal and the distribution network at time t. It is the power consumed by the user from the distribution network at time t. and These are the minimum and maximum power consumption by the user from the distribution network, respectively. It is the internet power at time t. and These are the minimum and maximum power consumption for internet access, respectively. and These represent the grid connection status of electricity purchase and sale at time t, with values ​​of 0 or 1.

[0127] The power distribution strategy of the power distribution network is subject to the power load supply and demand balance constraint, which can be expressed by formula (14).

[0128] (14)

[0129] in, It refers to the discharge power of the battery devices in the energy storage system. It is the charging power of battery devices in an energy storage system.

[0130] According to embodiments of the present invention, the power distribution strategy of the distribution network is dynamically determined according to priority, and the resource utilization status of photovoltaic power generation equipment, energy storage system and distribution network is reasonably allocated, so as to achieve optimal resource utilization status while meeting the user's power consumption strategy.

[0131] Energy storage systems can include water-cooled storage devices and battery devices.

[0132] The operation mode of a water-based cold storage device can be either cold storage mode or cold release mode. The cold storage capacity of a water-based cold storage device can be expressed by formula (15).

[0133] (15)

[0134] in, This refers to the remaining cooling capacity of the water-based cooling equipment at time t. This refers to the remaining cooling capacity of the water-based cooling system at time t-1, expressed in kWh or RT·h. It is the cold storage efficiency (dimensionless). It is the cold storage rate, and the unit is kW; It is the cooling efficiency (dimensionless). It is the cooling rate, and the unit is kW.

[0135] The cold storage capacity constraint of water-cooled equipment can be expressed by formula (16).

[0136] (16)

[0137] in, This is the minimum cooling capacity of water-based cooling equipment. This is the upper limit of the cooling capacity of water-based cooling equipment, measured in kW.

[0138] The energy storage state of a battery device can be represented by formula (17).

[0139] (17)

[0140] in, and These represent the state of charge (SOC) of the battery device at time t+1 and time t, respectively. It is the charging power of the battery device at time t; This refers to charging efficiency, for example, 0.9. It is the discharge power of the battery device at time t; This refers to the discharge efficiency, for example, 0.95. It is the maximum energy stored in a battery device; It represents the difference between time t and time t+1, for example, 15 minutes; M is a 0-1 variable, where M=1 when the battery device is charging and M=0 when the battery device is discharging. The variable M is used to ensure that the charging and discharging of the battery device do not occur simultaneously.

[0141] The state of charge constraint of the battery device at time t can be expressed by formula (18).

[0142] (18)

[0143] in, It is the minimum amount of energy that a battery device can store, for example, Set to 20% of the energy stored in the battery device. It is the maximum energy storage capacity of the battery device, for example, Set to 90% of the energy stored in the battery device.

[0144] The charging and discharging power constraints of battery devices can be expressed by formulas (19) and (20).

[0145] (19)

[0146] (20)

[0147] in, This indicates the rated charge and discharge power of the battery device.

[0148] In order to ensure that the battery device can start smoothly in the next power cycle, it is necessary to set the SOC to equal the initial value at the end of the power cycle. This process can be represented by formula (21).

[0149] (twenty one)

[0150] in, It is the starting time of the first period of the electricity consumption cycle. It is the start time of the last period of the electricity cycle, that is, the start time of the 96th period.

[0151] The following explains the energy storage strategy for determining an energy storage system.

[0152] Based on the initial resource transaction information and the predetermined transaction threshold, the energy storage strategy of the energy storage system can be determined, which may include operation D1 to operation D2.

[0153] In operation D1, in response to determining that the initial resource transaction information is less than the predetermined transaction threshold, the water storage equipment starts the cold storage operation and the battery equipment starts the charging operation.

[0154] The predetermined transaction threshold can be used to measure whether the initial resource transaction information meets the expected value for the energy storage system to start its energy storage strategy. If the initial resource transaction information is less than the predetermined transaction threshold, the energy storage system will start its energy storage strategy, thereby improving the resource utilization of the distribution network.

[0155] Water-based cooling equipment can initiate cooling when the initial resource transaction information is less than a predetermined transaction threshold, and battery equipment can initiate charging when the initial resource transaction information is less than a predetermined transaction threshold, thereby adjusting the resource utilization status of the distribution network to the optimal level.

[0156] In operation D2, in response to determining that the initial resource transaction information is greater than or equal to the predetermined transaction threshold, the water storage cooling equipment is determined to start the cooling release operation and the battery equipment is determined to start the discharge operation.

[0157] When the initial resource transaction information is determined to be greater than or equal to the predetermined transaction threshold, the energy storage system activates the energy release strategy, that is, the water-cooled storage equipment starts the cooling operation and the battery equipment starts the discharge operation, thereby reducing the resource utilization of the distribution network and instead utilizing the energy stored in the energy storage system, thereby adjusting the resource utilization status.

[0158] Within a power cycle, the total charge and discharge of the battery device should remain consistent, and this process can be expressed by formula (22).

[0159] (twenty two)

[0160] in, It refers to the charging efficiency of battery devices. It refers to the discharge efficiency of battery devices.

[0161] The remaining energy of the battery device at time t can be represented by the sum of the remaining energy at time t-1 and the charging / discharging power at time t.

[0162] (twenty three)

[0163] in, It represents the remaining energy of the battery device at time t. It represents the remaining energy of the battery device at time t-1.

[0164] The discharge capacity constraint of the battery device can be expressed by formula (24).

[0165] (twenty four)

[0166] This means that the amount of electricity supplied to the user by the battery device is less than the user's electricity consumption minus the power supply from the photovoltaic power generation device.

[0167] The charging power constraint of the battery device can be expressed by formula (25).

[0168] (25)

[0169] Among them, E ESS This refers to the rated capacity of the battery device, which can be 8000kWh.

[0170] According to an embodiment of the present invention, the water-cooled storage device is activated to perform either cold storage or cold release operations, and the battery device is activated to perform either charging or discharging operations, based on the initial resource transaction information and the predetermined transaction threshold, thereby realizing flexible adjustment of the energy storage strategy of the energy storage system.

[0171] The following explains how users determine their cooling needs.

[0172] The user determines the cooling demand based on the initial environmental information and initial resource transaction information.

[0173] Determine the cooling rate during the start-up cooling operation of the water-based cooling storage equipment based on the cooling demand.

[0174] Cooling demand is the total cooling capacity required by users within a certain period. Dynamically matching the cooling release rate according to the cooling demand can optimize the resource utilization efficiency of water-based cooling systems.

[0175] The constraints on the storage and release rates of water-cooled equipment can be expressed by formula (26).

[0176] (26)

[0177] in, It is the maximum cold storage rate. That is the maximum cooling rate.

[0178] According to embodiments of the present invention, the cooling release rate is determined based on the cooling demand so that the cooling release amount can both meet the cooling demand of the user end and reduce resource waste, thereby optimizing the resource utilization status of the integrated energy system.

[0179] The following explains how to generate a target strategy by iteratively updating the initial resource transaction information, electricity consumption strategy, and energy storage strategy using a reinforcement learning algorithm.

[0180] The target reward function can include: the reward function at the user end, the reward function of the energy storage system, and the reward function of the distribution network.

[0181] By using reinforcement learning algorithms to iteratively update the initial resource transaction information, electricity consumption strategy, and energy storage strategy, the generated target strategy may also include operations E1 to E3.

[0182] In operation E1, the distribution network's reward function is called to process the initial resource transaction information and the distribution network's load demand, generating the distribution network's reward value.

[0183] The reward value of the distribution network can reflect the resource utilization status of the distribution network during operation. It can be used to quantify the resource utilization status of the distribution network, thereby guiding the reinforcement learning algorithm to generate the optimal distribution strategy for the distribution network.

[0184] The reward function of the distribution network can be expressed by formula (27).

[0185] (27)

[0186] (28)

[0187] (29)

[0188] in, It is the reward value of the distribution network. It is the total load information of the integrated energy system. It is information on the utilization of power generation resources in the power distribution network. It is a coefficient.

[0189] In operation E2, the user-side reward function is called to process the initial resource transaction information, the user-side electricity consumption strategy, and the predetermined electricity consumption preference parameters, and to generate the user-side reward value.

[0190] The reward value on the user side can reflect the reward value of the user's dynamic electricity consumption strategy, and can quantify the resource transaction information on the user side, thereby guiding the reinforcement learning algorithm to generate the user's electricity consumption strategy.

[0191] The reward function on the user side can be represented by formula (30).

[0192] (30)

[0193] in, It is the reward value on the user's end. This represents the user's utility function during time period t. It can be expressed by formula (31).

[0194] (31)

[0195] The utility function can be represented by a quadratic function, where It is the user-side electricity consumption strategy. It is a parameter that represents the user's level of preference.

[0196] It satisfies the constraints of increasing utility function and decreasing marginal utility function.

[0197] (32)

[0198] (33)

[0199] In operation E3, the reward function of the energy storage system is called to process the initial resource transaction information and the energy storage status of the energy storage system, and to generate the reward value of the energy storage end.

[0200] The reward value at the energy storage end can reflect the reward value of the energy storage strategy of the energy storage system, and can quantify the resource utilization status of the energy storage strategy, thereby guiding the reinforcement learning algorithm to generate the energy storage strategy of the energy storage system.

[0201] The reward function of the energy storage system can be expressed by formula (34).

[0202] (34)

[0203] in, It is the reward value of the energy storage system. It is the reward value of the battery device. It is the bonus value for water-cooled storage equipment.

[0204] The reward function of the battery device can be expressed by formula (35).

[0205] (35)

[0206] (36)

[0207] (37)

[0208] in, It represents the amount of discharge from the battery device at time t. It represents the amount of charge the battery device receives at time t.

[0209] The reward function of a water-cooled storage device can be expressed by formula (38).

[0210] (38)

[0211] Where T is the number of time periods within a predetermined period, for example, it could be 96; This refers to the resource transaction information provided to users by the water-cooled storage equipment, which can be 0.55; This refers to the cooling capacity provided to users by water-based cooling equipment. It is resource trading information for water-cooled storage equipment to obtain electricity from the power distribution network.

[0212] The target strategy includes the electricity consumption strategy at the user end, the energy storage strategy at the energy storage end, and the target resource transaction information of the distribution network when the respective reward values ​​at the user end, energy storage end, and distribution network meet the convergence conditions.

[0213] The convergence condition can be a state in which the resource utilization of the user end, energy storage end, and distribution network in the integrated energy system reaches equilibrium. Under the condition of convergence, the target strategy is obtained to stabilize the resource utilization of the integrated energy system.

[0214] The process of generating a target policy using reinforcement learning algorithms can include operations F1 to F3.

[0215] In operation F1, the game iteration begins. The power grid, as the game leader, generates the initial resource transaction information of the distribution network based on the historical load information of the user end.

[0216] In F2 operation, the user terminal, energy storage system, and photovoltaic power generation equipment act as followers, adjusting their electricity consumption strategies, battery charging and discharging strategies, and cold storage and release strategies respectively based on the initial resource transaction information. The resource utilization status of each entity is calculated, and the system status is updated as environmental feedback.

[0217] In operation F3, the distribution network updates resource trading information based on the follower response results and the expected output power fluctuations of photovoltaic power generation equipment. Through multiple iterations, the power grid adjusts the resource trading information, users adjust their electricity consumption strategies, and energy storage devices adjust their energy storage strategies, ultimately forming a stable resource utilization equilibrium point and generating the target strategy.

[0218] According to embodiments of the present invention, the resource utilization status of each entity is obtained by generating reward values ​​for the distribution network, user terminals, and energy storage system. When the resource utilization status of the distribution network, user terminals, and energy storage system meets the convergence condition, a target strategy is generated to enable the integrated energy system to achieve efficient and stable resource utilization.

[0219] As one implementation method, reinforcement learning algorithms can be Deep Deterministic Policy Gradient (DDPG) algorithms, which can solve policy optimization problems in a continuous action space.

[0220] Figure 4 A schematic diagram of the structure of the deep deterministic policy gradient algorithm according to an embodiment of the present invention is shown.

[0221] like Figure 4 As shown, the DDPG algorithm's network structure includes an Actor policy network and an Actor value evaluation network. The Actor network consists of a 3-layer fully connected neural network with 256 hidden nodes, using the ReLU activation function. The output layer normalizes the action values ​​using the Tanh function, and the current state S is the input. t And output the deterministic action a t The Critic network uses the same structure, with input being a joint vector of states and deterministic actions, and outputting the Q-value that evaluates the state-action relationship with resource utilization. .

[0222] The processing flow of the DDPG algorithm can be as follows: initialize the parameters of the Actor and Critic networks, and create an experience replay pool to store the state s. t Action a t Rewards r t and the next state s t+1 .

[0223] Environment output current state s tActor networks are based on s t Output action a t =μ(st), where μ is the policy function of the Actor network. In the environment, action a is received. t In this case, return reward r t and the next state s t+1 .

[0224] Actor networks can be used to learn deterministic policies and output continuous actions. In an actor network, the first policy network outputs action 'a' based on the current state. t The policy gradient is updated using the first optimizer. The second policy network outputs the next state s based on the update parameters of the first policy network. t+1 .

[0225] Critic networks can be used to evaluate the action value of Actor networks, based on the output action value function. Output the Q-value. In the Critic network, the first Q-network takes action 'a' as input and outputs the Q-value, which is then updated via the second optimizer using Q-gradients. The second Q-network combines the next state 's' output from the second policy network. t+1 Calculate the target Q value.

[0226] During the training of the DDPG algorithm, batch data N*(S) can be sampled from the experience replay pool. t、 a t、 r t、 s t+1 ), used to update the Actor network and Critic network.

[0227] In training reinforcement learning algorithms, small batches of data can be randomly sampled from the replay pool. The training process may include: updating the Critic network, updating the Actor network, updating the target network, and determining whether the reward value meets the convergence condition.

[0228] The Critic network can be updated by minimizing the temporal-difference error (TD-error) through experience replay and training. The loss function of the training process can be expressed by formula (39).

[0229] (39)

[0230] in, It is a loss function. The expectation operator represents averaging over the state distribution (or samples in the experience replay buffer). As one implementation, it can be approximated by sampling a batch of experiences (states, actions, or rewards, etc.). It is a Critic network for state-action pairs Q-value estimation. It is a discount factor used to balance the importance of current and future reward values; It is the Q-value of the next state-action pair evaluated by the target network.

[0231] The Actor network can be updated using policy gradient ascent to maximize the Q value, which can be represented by formula (40).

[0232] (40)

[0233] in, It is the policy gradient, which can represent the gradient of the performance objective function J with respect to the Actor network parameters θ. It is the policy function of the Actor network. It is the gradient of the policy function of the Actor network with respect to the Actor network parameters θ. It is the gradient of the Q-value with respect to the action, which can be used to guide the direction of policy updates.

[0234] After updating the Critic and Actor networks, update the target network. As one implementation, the main network parameters can be synchronized to the target network every 100 updates.

[0235] When the rewards for each agent approach zero, the rewards converge, indicating an equilibrium state in the game. At this point, any further changes in strategy by the agents will not optimize resource utilization. The objective strategy under convergence conditions considers both the overall resource utilization of the integrated energy system and the optimal resource utilization of each individual agent. Applying the objective strategy optimizes both the overall resource utilization and stability of the system.

[0236] This invention uses a certain industrial park as an example, with a 24-hour electricity consumption cycle and a step size of 15 minutes, to record the resource utilization status within the park, in order to verify the impact of the strategy generation method for integrated energy systems provided in this invention on the various subjects in the game-theoretic optimization scheduling of regional integrated energy systems. This invention selects four typical days: cooling season workdays, cooling season rest days, transition season workdays, and transition season rest days, and sets up three scenarios under each typical day, as follows.

[0237] Scenario 1: The distribution network responds through dynamic resource trading information, and the integrated energy system implements optimized dispatching strategies.

[0238] Scenario 2: The distribution network maintains time-sharing resource trading information, and the integrated energy system implements optimized scheduling strategies.

[0239] Scenario 3: The distribution network maintains time-sharing resource trading information, and the integrated energy system implements traditional dispatching strategies.

[0240] Table 1 shows the time-of-use resource trading information of the distribution network. The time-of-use resource trading information is formulated according to the peak and valley characteristics of the power system and different resource trading information is formulated according to the time period. It guides users to transfer some of their electricity load to the off-peak period, reduce the electricity load during the peak period, thereby achieving the effect of peak shaving and valley filling and optimizing the resource utilization status.

[0241] Table 1. Time-of-use resource transaction information for distribution networks

[0242]

[0243] Table 2 shows the parameters of each device in the integrated energy system.

[0244] By setting parameters for each device in the integrated energy system, the operating parameters of each device can be made consistent under different scenarios.

[0245] Table 2 Equipment Parameters

[0246]

[0247] Figure 5 A schematic diagram showing the annual electricity load and photovoltaic power generation of a certain park according to an embodiment of the present invention is provided.

[0248] like Figure 5 As shown, Figure 5 (a) shows how the electrical load varies with the number of days of electricity consumption throughout the year. The electrical load peaks between day 200 and day 250. Figure 5 (b) shows the variation of photovoltaic (PV) power generation with the number of days of electricity consumption throughout the year. PV power generation peaks between day 150 and day 200. The peak electricity load and peak PV power generation within the park do not perfectly correspond throughout the year. Therefore, a strategy generation method for integrated energy systems can be used to dynamically schedule resource utilization while achieving a balance between electricity supply and demand, thereby adjusting resource utilization to its optimal state.

[0249] Figure 6 The following illustrations show four typical daily electrical loads and photovoltaic power generation scenarios according to embodiments of the present invention.

[0250] K-means clustering method was used to... Figure 5 The data shown is processed to obtain Figure 6 The following are four typical daily electrical loads and photovoltaic power generation scenarios.

[0251] like Figure 6As shown in the figure, squares represent electrical load, circles represent photovoltaic power generation, the horizontal axis is time, and the vertical axis is load. Figure 6 Figure (a) shows the changes in daily load and photovoltaic (PV) power generation during the cooling season. During the cooling season, the electrical load initially rises, then falls, then rises again, and finally falls again, exhibiting two peaks within a day. PV power generation initially rises and then falls, with the peak occurring around the 48th time point. The peak in PV power generation corresponds to the trough in the electrical load, indicating that renewable energy is difficult to fully utilize.

[0252] Figure 6 (b) shows the changes in daily load and photovoltaic power generation during the cooling season rest days. During the cooling season rest days, the overall electrical load fluctuates less, while photovoltaic power generation shows a trend of first rising and then falling, with the peak occurring around the 48th time point.

[0253] Figure 6 (c) shows the changes in daily load and photovoltaic power generation during the transition season. During the transition season, the changes in electricity load are similar to those during the cooling season, with an overall trend showing two load peaks. Photovoltaic power generation shows a trend of first rising and then falling, but the overall power generation load is low and difficult to meet the electricity demand.

[0254] Figure 6 Figure (d) shows the changes in electricity load and photovoltaic power generation during the transitional season rest days. The electricity load shows a trend of first decreasing and then increasing, with an overall low load. Photovoltaic power generation shows a trend of first increasing and then decreasing, with the peak occurring around the 48th time point.

[0255] In summary, across the four typical days, photovoltaic power generation showed an initial upward trend followed by a downward trend, with the peak value consistently around the 48th time point. This inconsistency between photovoltaic power generation trends and electricity load trends hinders the full utilization of renewable energy.

[0256] Figure 7 A schematic diagram illustrating the target strategy iterative calculation process for working days during the cooling season, according to an embodiment of the present invention, is shown.

[0257] like Figure 7 As shown, the resource utilization status of the distribution network, energy storage system and user end on weekdays during the cooling season changes with the number of iterations. The resource utilization status of the distribution network, energy storage system and user end all show a trend of first fluctuating and then stabilizing. Finally, the target strategy iterative calculation reaches a stable state, i.e. convergence state, in the 64th iteration, indicating that the target strategy has an equilibrium solution.

[0258] Figure 8 A schematic diagram illustrating the target strategy iterative calculation process for rest days during the cooling season according to an embodiment of the present invention is shown.

[0259] like Figure 8 As shown, the resource utilization status of the distribution network, energy storage system, and user terminals on rest days during the cooling season changes with the number of iterations. The resource utilization status of the distribution network, energy storage system, and user terminals all show a trend of first fluctuating and then stabilizing, with the resource utilization status of the distribution network showing a decreasing trend followed by stabilization. The iterative calculation of the objective strategy reaches a stable state, i.e., a convergent state, at the 84th iteration, indicating that the objective strategy has an equilibrium solution.

[0260] Figure 9 A schematic diagram illustrating the target strategy iterative calculation process for transitional season workdays according to an embodiment of the present invention is shown.

[0261] like Figure 9 As shown, the resource utilization status of the distribution network, energy storage system, and user terminals during the transition season's working days changes with the number of iterations. The resource utilization status of all three systems exhibits a trend of initial fluctuation followed by stabilization. Furthermore, the resource utilization status of the energy storage system initially shows significant fluctuations before stabilizing. The target strategy iterative calculation reaches a steady state, i.e., a convergent state, in the 74th iteration, indicating that the target strategy has an equilibrium solution.

[0262] Figure 10 A schematic diagram illustrating the target strategy iterative calculation process for transitional rest days according to an embodiment of the present invention is shown.

[0263] like Figure 10 As shown, the resource utilization status of the distribution network, energy storage system, and user end during the transition season rest days changes with the number of iterations. The resource utilization status of all three entities during the transition season rest days shows a slow decreasing trend, eventually stabilizing. The iterative calculation of the target strategy during the transition season rest days reaches a steady state, i.e., a convergent state, at the 80th iteration, indicating that the target strategy has an equilibrium solution.

[0264] Table 3 shows the resource utilization status of each game player in three scenarios under four typical day types: working days during the cooling season, rest days during the cooling season, working days during the transition season, and rest days during the transition season.

[0265] Table 3 Resource Utilization Status of Each Entity

[0266]

[0267] In the comparison of resource utilization status during cooling season rest days, the strategy generation method provided in this embodiment of the invention demonstrates advantages: in Scenario 1, the resource utilization status of the energy storage operator reaches 28,898, representing increases of 23.1% and 148.2% compared to Scenario 2 and Scenario 3, respectively. This improvement reflects that the dynamic resource trading information mechanism can incentivize the energy storage system to adjust its energy storage strategy, thereby enhancing its resource utilization efficiency. Simultaneously, the resource utilization status of the distribution network also improves by 42.0% compared to Scenario 3, indicating that by flexibly adjusting resource trading information strategies, the distribution network not only enhances the overall flexibility of the system but also improves its own resource utilization, verifying the effectiveness of the strategy generation method provided in this embodiment of the invention in coordinating the interests of multiple parties and achieving system collaborative optimization.

[0268] Figure 11 A comparison chart of four typical intraday time-sharing resource transaction information and dynamic resource transaction information according to embodiments of the present invention is shown.

[0269] like Figure 11 As shown in the figure, the changes in time-sharing resource trading information and dynamic resource trading information over time are illustrated in four typical intraday scenarios. The light-colored curve represents time-sharing resource trading information, while the dark-colored curve represents dynamic resource trading information derived from the target strategy. Figure 11 (a) shows the changes in hourly resource transaction information and dynamic resource transaction information over time during the working days of the cooling season. Figure 11 (b) shows the changes in time-sharing resource transaction information and dynamic resource transaction information over a day during the cooling season rest days. Figure 11 (c) shows the changes in time-sharing resource transaction information and dynamic resource transaction information during the weekdays of the transition season. Figure 11 (d) shows the changes in time-sharing resource transaction information and dynamic resource transaction information during the transitional season rest days over a day.

[0270] As shown in the graph, the time-of-use resource trading information exhibits a consistent trend across the four typical days, making it difficult to guide users in adjusting their transferable electricity load. Dynamic resource trading information, on the other hand, can dynamically adjust resource trading information based on the electricity load on different days. Guided by dynamic resource trading information, users prioritize using electricity generated by photovoltaic equipment, followed by electricity generated by batteries in the energy storage system, and finally electricity from the grid.

[0271] Figure 12 The electrical loads of four typical daytime users are shown according to embodiments of the present invention.

[0272] like Figure 12 As shown in the figure, the electrical load of four typical daytime users varies over time. Figure 12(a) shows the changes in users' electrical load over time in three scenarios during the cooling season workday. The users' electrical load in the three scenarios shows a trend of first rising and then falling, and the rising and falling speed is relatively fast. The electrical load is at a high level from the 30th to the 60th time point.

[0273] Figure 12 (b) shows the changes in user electricity load over time under three scenarios during the cooling season rest days. The user electricity load in the three scenarios shows a trend of first rising and then falling. The peak electricity load occurs near the 48th time point, and the electricity load fluctuates near the peak.

[0274] Figure 12 (c) shows the changes in user electricity load over time under three scenarios during the transitional season workday. In all three scenarios, user electricity load rises rapidly around time point 30, then fluctuates, and subsequently decreases after time point 72.

[0275] Figure 12 Figure (d) shows the changes in user electricity load over time under three scenarios during the transitional season rest days. The user electricity load in the three scenarios shows a trend of first rising and then fluctuating downward.

[0276] Figure 12 In the three scenarios shown, the trend of user electricity load changes is consistent on the same typical day, indicating that the trend of user electricity load does not change with the change of electricity consumption scenario.

[0277] Figure 13 The following illustrations show four typical daily users' electrical loads obtained from the distribution network according to embodiments of the present invention.

[0278] like Figure 13 As shown in the figure, the electrical load obtained by four typical daily users from the distribution network varies over time. Among them, Figure 13 (a) shows the changes in electrical load obtained by users from the distribution network under three scenarios on weekdays during the cooling season. Figure 13 (b) shows the changes in electrical load obtained by users from the distribution network under three scenarios during the cooling season rest days. Figure 13 (c) shows the changes in electrical load obtained by users from the distribution network under three scenarios on weekdays during the transition season. Figure 13 (d) shows the changes in electrical load obtained by users from the distribution network under three scenarios during the transitional season rest days.

[0279] On the same typical day, the electrical load obtained by users from the distribution network shows a consistent trend, but after optimization of the target strategy, the electrical load obtained by users from the distribution network decreases. Combined with... Figure 12 and Figure 13As can be seen from the figure, after optimization, the electricity load obtained by users in scenarios 1 and 2 from the distribution network is reduced by 3.9%-11.5% compared with scenario 3. Moreover, the valley value of the electricity load has increased, the peak value has decreased, and the load peak-valley difference has decreased by 6.9%-8.7%.

[0280] Figure 14 The power generation and grid connection of a photovoltaic power generation device according to an embodiment of the present invention are shown.

[0281] like Figure 14 As shown in the figure, the changes in grid-connected photovoltaic capacity and photovoltaic power generation over time are illustrated in three scenarios under four typical daytime conditions. The vertical axis represents the load, and the horizontal axis represents time. Figure 14 (a) shows the changes in grid-connected photovoltaic capacity and photovoltaic power generation over time for three scenarios during the cooling season workday. Figure 14 (b) shows the changes in grid-connected photovoltaic power and photovoltaic power generation over time for three scenarios during the cooling season rest days. Figure 14 (c) shows the changes in photovoltaic grid connection and photovoltaic power generation over time for three scenarios during the transition season workdays. Figure 14 (d) shows the changes in grid-connected photovoltaic power and photovoltaic power generation over time for three scenarios during the transitional season rest days.

[0282] In the three scenarios of high electricity load on weekdays, the photovoltaic grid connection is zero, while on rest days there is a large peak-valley load difference and a low renewable energy absorption rate. With the optimization and scheduling of the target strategy, the photovoltaic absorption rate on rest days can be increased by 54%, verifying the role of the collaborative optimization strategy in the efficient utilization of clean energy.

[0283] Figure 15A A schematic diagram comparing the energy storage status of a battery device and a water-cooled storage device on a working day during the cooling season is shown in an energy storage system according to an embodiment of the present invention.

[0284] like Figure 15A As shown in the figure, the energy storage status of the battery devices and water-based cooling systems in Scenario 1 and Scenario 2 respectively during a weekday in the cooling season is illustrated. Data is recorded every 15 minutes (min), totaling 96 data points over 24 hours. In Scenario 1, the energy storage system charges during off-peak hours, resulting in a step-wise increase in energy storage status; during peak hours, it discharges, resulting in a step-wise decrease in energy storage status. In Scenario 2, the battery devices and water-based cooling systems adjust their energy storage status according to a predetermined energy storage strategy. Both the battery devices and water-based cooling systems in Scenario 1 exhibit dynamically adjusted energy storage status compared to Scenario 2.

[0285] Figure 15BA schematic diagram comparing the energy storage status of a battery device and the energy storage status of a water-cooled storage device on a rest day during the cooling season is shown in an energy storage system according to an embodiment of the present invention.

[0286] like Figure 15B As shown in the figure, the energy storage status of the battery equipment and water-based cooling equipment in Scenario 1 and Scenario 2 respectively during the cooling season's rest days. During off-peak electricity demand, the energy storage system in Scenario 1 can achieve a more sufficient energy storage status compared to Scenario 2.

[0287] Figure 15C A schematic diagram comparing the energy storage status of a battery device and a water-cooled storage device during a working day in a transitional season is shown in an energy storage system according to an embodiment of the present invention.

[0288] like Figure 15C As shown in the figure, the energy storage status of battery devices and water-cooled storage devices in Scenario 1 and Scenario 2 respectively is illustrated on weekdays during the transition season. During weekdays during the transition season, the battery devices in the energy storage systems of Scenario 1 and Scenario 2 show little change during peak electricity consumption periods, with energy storage and release operations mainly performed by the water-cooled storage devices.

[0289] Figure 15D A schematic diagram comparing the energy storage status of a battery device and a water-cooled storage device during a rest day in a transitional season is shown in an energy storage system according to an embodiment of the present invention.

[0290] like Figure 15D As shown in the figure, the energy storage status of the battery device and the water-cooled storage device in Scenario 1 and Scenario 2 respectively during the transitional season rest day. The battery device in Scenario 1 starts charging earlier than the battery device in Scenario 2. The water-cooled storage device in Scenario 1 exhibits a dynamically changing energy storage strategy compared to the water-cooled storage device in Scenario 2.

[0291] In four typical days, the energy storage system prioritizes power supply from photovoltaic devices, and selects the distribution network for power supply when the initial resource trading information is less than the predetermined trading threshold. In scenario three of the traditional dispatch strategy, the corresponding State of Charge (SOC) data could not be obtained because water-cooled storage and battery equipment were not deployed at the time. This difference in infrastructure configuration reflects how the optimized dispatch strategy, by introducing an energy storage system, achieves flexible regulation capabilities for the integrated energy system.

[0292] In summary, the strategy generation method for integrated energy systems can optimize the resource utilization status of various entities, and the reinforcement learning algorithm can determine the equilibrium solution of the game when solving multiple games, and the solution results have good convergence.

[0293] Figure 16 A block diagram of a strategy generation apparatus for an integrated energy system according to an embodiment of the present invention is shown.

[0294] like Figure 16 As shown, the strategy generation device 1600 includes an acquisition module 1610, a first generation module 1620, a second generation module 1630, a third generation module 1640, a first determination module 1650, a second determination module 1660, and a fourth generation module 1670.

[0295] The acquisition module 1610 is used to acquire historical load information, initial power load transfer information, initial environmental information, and energy storage status of the energy storage system from the user end.

[0296] The first generation module 1620 is used to process the historical load information of the user end by calling the dynamic transaction adjustment model to generate the initial resource transaction information of the distribution network.

[0297] The second generation module 1630 is used to process initial resource transaction information, initial electricity load transfer information, and the predetermined proportion of load that can be transferred by calling the dynamic load transfer model, and generate the user-side electricity consumption strategy. The electricity consumption strategy indicates the expected time period information of the load that can be transferred from the user-side to the energy storage system, so that the energy storage system can perform energy storage operations during the expected time period information.

[0298] The third generation module 1640 is used to call the target function to process the initial environmental information and photovoltaic power generation equipment parameters, and generate the expected output power of the photovoltaic power generation equipment.

[0299] The first determining module 1650 is used to determine the distribution strategy of the distribution network based on the power consumption strategy, expected output power and energy storage status of the energy storage system.

[0300] The second determining module 1660 is used to determine the energy storage strategy of the energy storage system based on the initial resource transaction information.

[0301] The fourth generation module 1670 is used to construct a state space based on the user's historical load information, initial power load transfer information, initial environmental information, and energy storage status of the energy storage system, and to construct an action space based on the user's power consumption strategy, the distribution strategy of the distribution network, and the energy storage strategy. Based on the target reward function, the target strategy is generated by iteratively updating the initial resource transaction information, power consumption strategy, and energy storage strategy using a reinforcement learning algorithm.

[0302] Any one or more of the modules, submodules, units, and subunits according to embodiments of the present invention, or at least part of the functions of any one or more of them, can be implemented in a single module. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present invention can be implemented by being divided into multiple modules. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present invention can be at least partially implemented as hardware circuits, such as field-programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems-on-a-chip, systems-on-a-substrate, systems-on-package, application-specific integrated circuits (ASICs), or implemented in hardware or firmware by any other reasonable means of integrating or packaging circuits, or implemented in software, hardware, and firmware, or in any suitable combination of any of these three implementation methods. Alternatively, one or more of the modules, submodules, units, and subunits according to embodiments of the present invention can be at least partially implemented as computer program modules, which, when run, can perform corresponding functions.

[0303] For example, any and more of the following modules can be combined into one module / unit / subunit: the acquisition module 1610, the first generation module 1620, the second generation module 1630, the third generation module 1640, the first determination module 1650, the second determination module 1660, and the fourth generation module 1670. Alternatively, any one of these modules / units / subunits can be split into multiple modules / units / subunits. Or, at least some of the functionality of one or more of these modules / units / subunits can be combined with at least some of the functionality of other modules / units / subunits and implemented in one module / unit / subunit. According to embodiments of the present invention, at least one of the acquisition module 1610, the first generation module 1620, the second generation module 1630, the third generation module 1640, the first determination module 1650, the second determination module 1660, and the fourth generation module 1670 can be at least partially implemented as hardware circuits, such as field-programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems-on-a-chip, systems-on-a-substrate, systems-on-package, application-specific integrated circuits (ASICs), or any other reasonable means of integrating or packaging circuits, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the acquisition module 1610, the first generation module 1620, the second generation module 1630, the third generation module 1640, the first determination module 1650, the second determination module 1660, and the fourth generation module 1670 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0304] It should be noted that the strategy generation device part for integrated energy systems in the embodiments of the present invention corresponds to the strategy generation part for integrated energy systems in the embodiments of the present invention. For a detailed description of the strategy generation device part for integrated energy systems, please refer to the strategy generation method part for integrated energy systems, which will not be repeated here.

[0305] Figure 17 A block diagram of an electronic device suitable for implementing the strategy generation method for integrated energy systems described above, according to an embodiment of the present invention, is shown. Figure 17 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0306] like Figure 17 As shown, an electronic device 1700 according to an embodiment of the present invention includes a processor 1701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1702 or a program loaded from a storage portion 1708 into a random access memory (RAM) 1703. The processor 1701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1701 may also include onboard memory for caching purposes. The processor 1701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0307] RAM 1703 stores various programs and data required for the operation of electronic device 1700. Processor 1701, ROM 1702, and RAM 1703 are interconnected via bus 1704. Processor 1701 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 1702 and / or RAM 1703. It should be noted that the programs may also be stored in one or more memories other than ROM 1702 and RAM 1703. Processor 1701 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in said one or more memories.

[0308] According to an embodiment of the present invention, the electronic device 1700 may further include an input / output (I / O) interface 1705, which is also connected to a bus 1704. The electronic device 1700 may also include one or more of the following components connected to the input / output (I / O) interface 1705: an input section 1706 including a keyboard, mouse, etc.; an output section 1707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1708 including a hard disk, etc.; and a communication section 1709 including a network interface card such as a LAN card, modem, etc. The communication section 1709 performs communication processing via a network such as the Internet. A drive 1710 is also connected to the input / output (I / O) interface 1705 as needed. A removable medium 1711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1710 as needed so that computer programs read from it can be installed into the storage section 1708 as needed.

[0309] According to embodiments of the present invention, the method flow according to embodiments of the present invention can be implemented as a computer software program. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1709, and / or installed from removable medium 1711. When the computer program is executed by processor 1701, it performs the functions defined in the system of the embodiments of the present invention. According to embodiments of the present invention, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0310] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0311] According to embodiments of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0312] For example, according to embodiments of the present invention, a computer-readable storage medium may include one or more memories other than the ROM 1702 and / or RAM 1703 described above and / or ROM 1702 and RAM 1703.

[0313] Embodiments of the present invention also include a computer program product comprising a computer program containing program code for performing the methods provided in the embodiments of the present invention. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the strategy generation method for integrated energy systems provided in the embodiments of the present invention.

[0314] When the computer program is executed by the processor 1701, it performs the functions defined in the system / apparatus of this embodiment of the invention. According to embodiments of the invention, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0315] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 1709, and / or installed from a removable medium 1711. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0316] According to embodiments of the present invention, program code for executing the computer programs provided in the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0317] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention can be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or pairings fall within the scope of this invention.

[0318] The embodiments of the present invention have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

Claims

1. A strategy generation method for integrated energy systems, characterized in that, include: The system acquires historical load information, initial power load transfer information, initial environmental information, and energy storage status of the energy storage system, which includes water-cooled storage equipment and battery equipment. By calling the dynamic transaction adjustment model to process the historical load information of the user end, the initial resource transaction information of the distribution network is generated. Based on the initial resource transaction information, the user's expected electricity consumption behavior is determined; by calling the dynamic load transfer model to process the initial electricity load transfer information, the predetermined proportion of load that can be transferred, and the user's expected electricity consumption behavior, an electricity consumption strategy for the user is generated; the electricity consumption strategy indicates the expected time period information of the load that can be transferred from the user to the energy storage system, so that the energy storage system can perform energy storage operations during the expected time period information. The target function is called to process the initial environmental information and photovoltaic power generation equipment parameters to generate the expected output power of the photovoltaic power generation equipment. Determine the priority of the photovoltaic power generation equipment, the energy storage system, and the distribution network in supplying energy to the user end; and determine the power distribution strategy of the distribution network according to the priority, based on the user demand at the user end, the expected output power of the photovoltaic power generation equipment, and the energy storage status of the energy storage system. The energy storage strategy of the energy storage system is determined based on the initial resource transaction information. as well as A state space is constructed using the user's historical load information, initial electricity load transfer information, initial environmental information, and energy storage status of the energy storage system. An action space is constructed using the user's electricity consumption strategy, the distribution network's distribution strategy, and the energy storage strategy. Based on the target reward function, the target strategy is generated by iteratively updating the initial resource transaction information, the electricity consumption strategy, and the energy storage strategy using a reinforcement learning algorithm. The step of determining the energy storage strategy of the energy storage system based on the initial resource transaction information includes: In response to determining that the initial resource transaction information is less than a predetermined transaction threshold, the water-cooled storage device initiates a cooling operation, and the battery device initiates a charging operation; and In response to determining that the initial resource transaction information is greater than or equal to the predetermined transaction threshold, the water-cooled storage device is determined to start a cooling release operation and the battery device is determined to start a discharge operation.

2. The strategy generation method according to claim 1, characterized in that: The historical load information includes: dynamic electricity consumption strategies for historical periods and resource transaction information for historical periods; The process of generating initial resource transaction information for the distribution network by calling the dynamic transaction adjustment model to process historical load information from the user end includes: The dynamic electricity consumption strategies for the historical period are averaged to obtain the average value of the electricity consumption strategies for the historical period. Based on a predetermined period, the dynamic electricity consumption strategy for the historical time period and the average value of the electricity consumption strategy for the historical time period are processed to obtain the variance of the electricity consumption strategy for the historical time period; and Based on a predetermined adjustment coefficient, the variance of the electricity consumption strategy, the dynamic electricity consumption strategy of the historical period, the average value of the electricity consumption strategy of the historical period, and the resource transaction information of the historical period are processed to obtain the initial resource transaction information of the historical period.

3. The strategy generation method according to claim 1, characterized in that, The strategy generation method further includes: The user terminal determines its cooling demand based on the initial environmental information and the initial resource transaction information; and Based on the cooling demand, determine the cooling release rate during the start-up cooling operation of the water-based cooling storage equipment.

4. The strategy generation method according to claim 1, characterized in that, The target reward function includes: the reward function for the user end, the reward function for the energy storage system, and the reward function for the distribution network; the strategy generation method further includes: The initial resource transaction information and the load demand of the distribution network are processed by calling the reward function of the distribution network to generate the reward value of the distribution network; The user terminal's reward function is invoked to process the initial resource transaction information, the user terminal's electricity consumption strategy, and the predetermined electricity consumption preference parameters, thereby generating the user terminal's reward value. The reward function of the energy storage system is called to process the initial resource transaction information and the energy storage status of the energy storage system, and to generate the reward value of the energy storage end; The target strategy includes the electricity consumption strategy of the user terminal, the energy storage strategy of the energy storage terminal, and the target resource transaction information of the distribution network when the reward values ​​of the user terminal, the energy storage terminal, and the distribution network respectively meet the convergence conditions.

5. A strategy generation device for an integrated energy system, characterized in that, include: The acquisition module is used to acquire historical load information, initial power load transfer information, initial environmental information, and energy storage status of the energy storage system from the user terminal; the energy storage system includes water-cooled storage equipment and battery equipment. The first generation module is used to process the historical load information of the user end by calling the dynamic transaction adjustment model to generate the initial resource transaction information of the distribution network. The second generation module is used to determine the user's expected electricity consumption behavior based on the initial resource transaction information; and to generate the user's electricity consumption strategy by calling the dynamic load transfer model to process the initial electricity load transfer information, the predetermined transferable load ratio, and the user's expected electricity consumption behavior. The electricity consumption strategy indicates the expected time period information of the load that can be transferred from the user to the energy storage system, so that the energy storage system can perform energy storage operations during the expected time period information. The third generation module is used to call the target function to process the initial environmental information and photovoltaic power generation equipment parameters, and generate the expected output power of the photovoltaic power generation equipment. The first determining module is used to determine the priority of the photovoltaic power generation equipment, the energy storage system, and the distribution network in supplying energy to the user end; and, according to the priority, based on the user demand at the user end, the expected output power of the photovoltaic power generation equipment, and the energy storage status of the energy storage system, determine the power distribution strategy of the distribution network. The second determining module is used to determine the energy storage strategy of the energy storage system based on the initial resource transaction information. The fourth generation module is used to construct a state space based on the user's historical load information, initial power load transfer information, initial environmental information, and energy storage status of the energy storage system; construct an action space based on the user's power consumption strategy, the distribution strategy of the distribution network, and the energy storage strategy; and generate a target strategy based on the target reward function by iteratively updating the initial resource transaction information, the power consumption strategy, and the energy storage strategy using a reinforcement learning algorithm. The second determining module is used for: In response to determining that the initial resource transaction information is less than a predetermined transaction threshold, the water-cooled storage device is determined to start a cooling operation and the battery device is determined to start a charging operation. as well as In response to determining that the initial resource transaction information is greater than or equal to the predetermined transaction threshold, the water-cooled storage device is determined to start a cooling release operation and the battery device is determined to start a discharge operation.

6. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the strategy generation method according to any one of claims 1 to 4.

7. A computer-readable storage medium having executable instructions stored thereon, characterized in that, When executed by the processor, this instruction causes the processor to implement the strategy generation method as described in any one of claims 1 to 4.

Images